Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

About the model report

Earlier-2026 baseline comparison from the official Qwen model card. Original model labels and score formatting are retained. Language and vision tables use different Qwen reference models. Tools, image inputs, agent scaffolds and task subsets can differ; follow the model card’s evaluation details. Video-MME subtitle settings and HLE tool settings stay in separate rows. Original MCP-Mark results predate MCPMark Verified. Reviewed 2026-10-26.

Read the official model report ↗

26 benchmarks in the report

Search this collection →

CTX / BENCHMARK34 reported results

AA-LCR v1.1

Long-document questions test extraction, synthesis, and reasoning across substantial inputs.

Answer score↗

RSN / BENCHMARK131 reported results

Humanity’s Last Exam

Expert-authored questions stretch academic reasoning and knowledge across many disciplines.

Answer accuracy↗

RSN / BENCHMARK146 reported results

MMMU-Pro

Academic questions require visual understanding as well as language reasoning, with measures against text-only shortcuts.

Multimodal accuracy↗

ALN / BENCHMARK33 reported results

IFBench

Verifiable output constraints probe whether instruction following transfers beyond familiar formats.

Constraint satisfaction↗

KNW / BENCHMARK171 reported results

MMLU-Pro

A more demanding successor to MMLU emphasizes reasoning across a broad academic subject mix.

Multiple-choice accuracy↗

COD / BENCHMARK180 reported results

LiveCodeBench

Date-bounded contest problems test coding and related code-repair or execution skills.

Pass@1↗

COD / BENCHMARK133 reported results

SWE-bench Verified

Human-reviewed repository issues test whether an agent can implement a repair in an existing Python codebase.

Resolved issues↗

AGT / BENCHMARK6 reported results

BFCL V4

Berkeley’s function-calling evaluation covers tool invocation and broader agentic interactions.

Function-call accuracy↗

CTX / BENCHMARK6 reported results

LongBench v2

Long-context understanding and reasoning are evaluated through demanding questions over substantial inputs.

Question accuracy↗

AGT / BENCHMARK29 reported results

OSWorld-Verified

Language-model agents act in real computer environments across applications, using reproducible setup and evaluation scripts.

Verified task success↗

ALN / BENCHMARK6 reported results

IFEval

Programmatically checkable instructions test whether model outputs satisfy explicit constraints.

Instruction satisfaction↗

RSN / BENCHMARK26 reported results

ZeroBench

Challenging visual questions probe failures that remain after ordinary image understanding becomes strong.

Visual task accuracy↗

AGT / BENCHMARK7 reported results

ScreenSpot-Pro

A model must locate the interface element referred to by a user instruction on a professional screenshot.

Visual grounding accuracy↗

RSN / BENCHMARK10 reported results

LVBench

A language model with visual input must understand and reason over long video material.

Long-video understanding accuracy↗

AGT / BENCHMARK8 reported results

WideSearch

Broad information gathering asks an agent to identify many entities and their attributes across web sources.

Item-level F1↗

AGT / BENCHMARK11 reported results

MCPMark

Agents work through MCP-connected application and filesystem tasks with programmatic verification.

Verified tool task success↗

COD / BENCHMARK280 reported results

Terminal-Bench 2.0

An agent completes technical work in a terminal under executable task checks.

Verified terminal task success↗

COD / BENCHMARK10 reported results

SWE-bench Multilingual

Repository-level software repair broadens evaluation across programming languages.

Resolved repository issues↗

MTH / BENCHMARK11 reported results

AIME 2026

A new annual contest set tests mathematical reasoning through short integer answers.

Contest answer accuracy↗

MTH / BENCHMARK11 reported results

IMO-AnswerBench

Olympiad-related mathematics is evaluated through answer-oriented scoring in frontier model reports.

Answer accuracy↗

RSN / BENCHMARK26 reported results

CharXiv

Scientific charts require models to read visual information and reason over what the figure actually shows.

Chart reasoning accuracy↗

MTH / BENCHMARK16 reported results

MathVision

Mathematical problems with visual context require reasoning over both the statement and the supplied figures.

Visual mathematics accuracy↗

MTH / BENCHMARK6 reported results

MathVista

Mathematical reasoning is tested through diagrams, charts, scientific figures and other visual contexts.

Visual mathematics accuracy↗

RSN / BENCHMARK62 reported results

Video-MME

Video understanding is evaluated across short, medium and long clips through human-annotated questions.

Video question-answering accuracy↗

RSN / BENCHMARK6 reported results

OCRBench

Text-rich visual tasks test recognition and understanding of writing inside images.

OCR-related task score↗

RSN / BENCHMARK6 reported results

MMMU

College-level questions combine language, visual material and subject knowledge across disciplines.

Multidiscipline multimodal accuracy↗

COMPARISON CHECKLIST

Match benchmark version, task subset, model effort, harness, tool access, attempt budget, scoring rule, and fallback behavior. A shared benchmark name is the start of a comparison, not proof that two reported scores use the same contract.

Qwen original report ↗