Muse Spark Open

variant: xhigh effort, with tools; Mean answer accuracy over all 388 prompts, averaged over five runs; tools enabled. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

Собственный прогон Epoch AI; mean_score × 100 (scale 1.0). Общая группа задана по опубликованной методике и дате; настройки отдельного запуска не подтверждены.

Собственный прогон Epoch AI; mean_score × 100 (scale 1.0). Общая группа задана по опубликованной методике и дате; настройки отдельного запуска не подтверждены.

Собственный прогон Epoch AI; mean_score × 100 (scale 1.0). Общая группа задана по опубликованной методике и дате; настройки отдельного запуска не подтверждены.

variant: xhigh effort, no tools; Overall answer accuracy on the 1,000-question HLE-Diamond set, without tools; Scale AI leaderboard. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

variant: xhigh effort, with tools; HLE answer accuracy; Meta internal runs with bash/browser and a website blocklist. Exact question subset not specified. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

variant: xhigh effort; Accuracy under the official violation-of-expectation protocol on the main split of 1,012 videos. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

variant: xhigh effort; Mean task weighted rubric score ×100 over 65 tasks; not a task pass rate. JobBench scoring methodology. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

variant: xhigh effort; Task pass rate: mean claim coverage >=0.75; Scale AI evaluation harness and scoring pipeline. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

variant: xhigh effort; Mean per-task reward ×100, averaged over four attempts; 361-task split, 200-step limit, 1920×1080 GUI. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

Собственный прогон Epoch AI; mean_score × 100 (scale 1.0). Сопоставимость с другими прогонами не подтверждена.

Собственный прогон Epoch AI; mean_score × 100 (scale 1.0). Общая группа задана по опубликованной методике и дате; настройки отдельного запуска не подтверждены.

variant: xhigh effort, with tools; Raw answer accuracy over 2,024 English/Chinese questions, averaged over five runs; tools enabled. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

variant: xhigh effort; Pass@1 on Toolathlon-Verified; official Toolathlon evaluation service with the model API. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

version: 2; variant: xhigh effort; Video-MME v2 grouped accuracy, 0-shot on 800 videos/3,200 questions; subtitles in prompt, no audio. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

variant: xhigh effort, with tools; Mean question-level accuracy on the 3,000-question headline subset, excluding Notable People; tools enabled. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology

variant: xhigh effort, with tools; Questions with at least one correct response among five samples, 100 questions; tools enabled. Verified against https://research.meta.ai/static/muse-spark-open-evaluation-methodology