[Leaderboard] DataTrail v1— Muse Spark 1.3 — 82.01% Pass@1

[abdlhannan]

DataTrail v1— Muse Spark 1.3 — 82.01% Pass@1

• Submitter: Abdul Hannan (@abdlhannan)
• Backbone: meta/muse-spark-1.3-contributor
• Reasoning: medium
• Hints: yes
• Coverage: all 54 official queries across 12 datasets, five fresh trials each, 270 retained outcomes
• Original local dataset-macro Pass@1: 82.0128% (0.8201282051282051), subject to official revalidation
• Raw successes: 215/270

Results

Each query has five trials. Scores below are from the preserved local evaluation, subject to official revalidation.

Dataset	Pass@1
DEPS_DEV_V1	80.00%
GITHUB_REPOS	55.00%
PANCANCER_ATLAS	66.67%
PATENTS	100.00%
agnews	55.00%
bookreview	100.00%
crmarenapro	66.15%
googlelocal	80.00%
music_brainz_20k	100.00%
stockindex	93.33%
stockmarket	88.00%
yelp	100.00%
Overall (dataset-macro)	82.01%

Leaderboard metric: 82.0128% dataset-macro Pass@1, calculated by averaging the 12 dataset scores equally using unrounded values, rather than pooling all trials.

Architecture

One root foundation-model loop, one execute shell tool, and a persistent per-trial workspace. The agent writes and runs Python/SQL, inspects sanctioned data, and can use installed analytical packages and reusable skills. Subagents are disabled and a separate challenger is off. Workspaces for each run are separate to one another.

The original run used a macOS sandbox with network disabled, read-only source-data mounts and an explicit file-read allowlist. SQL dumps/BSON can be parsed locally. Grading, ground truths, previous results and packaging operate outside the solver context. Effective prompt events and run settings are included for inspection. Standalone harness implementation source and bundled skill scripts are not distributed; any helper or skill code the agent read, and any code it generated, remain visible in the recorded traces.

Answers and trace integrity

The runner selects the persisted final_answer.txt, rather than necessarily the last conversational message. The package includes that exact file, its original renderer receipt and persisted result artifact. All 270 packaged answer strings match the saved answers byte-for-byte; none was reformatted for submission.

For privacy, the local username token is replaced with REDACTED_USER in traces and receipt metadata. All original artifacts remain retained locally. Original and public trace hashes plus redaction counts are supplied. Final answers and persisted calculations are unchanged.

Files and execution traces

• leaderboard_submissions/datatrail_one_tool_v2_muse_medium.json: 270 answers.
• Trace release: review archive, recorded prompts, hashes, run configuration, per-dataset results and audit notes.

Operational notes

Known provider cost was $3.58, with incomplete cost coverage; this is not presented as the full bill. The reported score comes from the original local grading; we welcome official validation and any requested clarification.