RLTasksby ulam.ai
Models / OpenAI

OpenAI / MODEL EVIDENCE

GPT-6 Astra

145 benchmarks · 338 observations · 117 sources · released 2026-09-03

Scores at a glance

137 benchmarks · large dot: GPT-6 Astra; small dots: closest models where reported

AA-Briefcase v1.1 ↗1568.14 Elo
833.061811.82

GPT-6 Astra (Max) · AA-Briefcase v1.1 · overall Elo ↗

1 more reported result

1569 Elo · GPT-6 Astra (max, with fallback) · AA-Briefcase v1.1 · AA-Briefcase v1.1 (Elo-score) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

GDP.pdf ↗31.00%
7.432.6

GPT-6 Astra (Max) · GDP.pdf · all-pass ↗

4 more reported results

34.2 · GPT-6 Astra (max) · GDP.pdf · Muse Wonder official evaluation report ↗

34.2% · GPT-6 Astra (max) · GDP.pdf · GDP.pdf score · % · Epoch AI · GDP.pdf ↗

85.5% · GPT-6 Astra (max, with fallback) · GDP.pdf · GDP.pdf w/ python · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

34.2% · GPT-6 Astra (max, with fallback) · GDP.pdf · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

Terminal-Bench 4.0 ↗59.09%
067.17

GPT-6 Astra (Max) · Terminal-Bench 4.0 · AA harness ↗

6 more reported results

59.6 % · GPT-6 Astra (Mini-SWE-agent) · Terminal-Bench 4.0 · Score · Vals AI · Terminal-Bench 4.0 ↗

57.9% · GPT-6 Astra · Terminal-Bench 4.0 · Claude Opus 5.5 launch report ↗

58.2% · GPT-6 Astra · Terminal-Bench 4.0 · Gemini 4 Argon model card ↗

57.9% · GPT-6 Astra · Terminal-Bench 4.0 · GPT-6 Astra launch report ↗

59.1 · GPT-6 Astra (max) · Terminal-Bench 4.0-AA · Muse Wonder official evaluation report ↗

58.2% · GPT-6 Astra (max, with fallback) · Terminal-Bench 4.0 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

Terminal-Bench-Science 0.1 ↗63.33%
063.33

GPT-6 Astra (Max) · Terminal-Bench-Science 0.1 · AA harness ↗

5 more reported results

62.86 % · GPT-6 Astra (Mini-SWE-agent) · Terminal-Bench-Science · Score · Vals AI · Terminal-Bench-Science ↗

64.6% · GPT-6 Astra · Terminal-Bench-Science 0.1 · Claude Opus 5.5 launch report ↗

68.1% · GPT-6 Astra · Terminal-Bench Science 0.1 · Gemini 4 Argon model card ↗

64.6% · GPT-6 Astra · Terminal-Bench Science 0.1 · GPT-6 Astra launch report ↗

68.1% · GPT-6 Astra (max, with fallback) · Terminal-Bench-Science 0.1 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

AA-3D ↗1164.13 Elo
572.551198.83

GPT-6 Astra (Max) · AA-3D · Elo ↗

SciCode ↗56.48%
42.9466.9

GPT-6 Astra (Max) · SciCode · AA implementation ↗

5 more reported results

56.48% · GPT-6 Astra (max) · SciCode · Score · % · Epoch AI · SciCode ↗

55.67% · GPT-6 Astra (xhigh) · SciCode · Score · % · Epoch AI · SciCode ↗

55.44% · GPT-6 Astra (high) · SciCode · Score · % · Epoch AI · SciCode ↗

54.17% · GPT-6 Astra (medium) · SciCode · Score · % · Epoch AI · SciCode ↗

54.05% · GPT-6 Astra (low) · SciCode · Score · % · Epoch AI · SciCode ↗

Humanity’s Last Exam ↗54.68%
30.7261.35

GPT-6 Astra (Max) · Humanity’s Last Exam · AA implementation ↗

5 more reported results

57.2% · tools · GPT-6 Astra · Humanity’s Last Exam · tools · Claude Opus 5.5 launch report ↗

57.2% · GPT-6 Astra · Humanity’s Last Exam · tools · GPT-6 Astra launch report ↗

54.8% · GPT-6 Astra (unknown thinking) · HLE · Accuracy · % · Epoch AI · HLE ↗

57.2% · GPT-6 Astra (max, with fallback) · Humanity’s Last Exam · HLE-Full w/ tools · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

53.6% · GPT-6 Astra (max, with fallback) · Humanity’s Last Exam · HLE-Full · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

ITBench-AA ↗48.59%
35.2256.21

GPT-6 Astra (Max) · ITBench-AA · root-cause score ↗

MMMU-Pro ↗86.88%
73.4787.69

GPT-6 Astra (Max) · MMMU-Pro · AA implementation ↗

2 more reported results

87.8% · GPT-6 Astra (max, with fallback) · MMMU-Pro · MMMU-Pro w/ python · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

86.9% · GPT-6 Astra (max, with fallback) · MMMU-Pro · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

AA-PersonalityBench ↗54.31 percentile
28.92100

GPT-6 Astra (Max) · AA-PersonalityBench · expressiveness ↗

7 more reported results

67.00 percentile · GPT-6 Astra (Max) · AA-PersonalityBench · organization · Artificial Analysis ↗

21.31 percentile · GPT-6 Astra (Max) · AA-PersonalityBench · informativeness · Artificial Analysis ↗

94.92 percentile · GPT-6 Astra (Max) · AA-PersonalityBench · transparency · Artificial Analysis ↗

72.08 percentile · GPT-6 Astra (Max) · AA-PersonalityBench · warmth · Artificial Analysis ↗

41.62 percentile · GPT-6 Astra (Max) · AA-PersonalityBench · conviction · Artificial Analysis ↗

72.08 percentile · GPT-6 Astra (Max) · AA-PersonalityBench · imagination · Artificial Analysis ↗

13.69 percentile · GPT-6 Astra (Max) · AA-PersonalityBench · slop · Artificial Analysis ↗

DeepSWE v1.1 ↗74.1%
67.477.9

GPT-6 Astra · DeepSWE v1.1 ↗

8 more reported results

74.1% · GPT-6 Astra · DeepSWE v1.1 · GPT-6 Astra launch report ↗

74.1 · GPT-6 Astra (max) · DeepSWE 1.1 · Muse Wonder official evaluation report ↗

74.12% · GPT-6 Astra (xhigh) · DeepSWE · Pass@1 · % · Epoch AI · DeepSWE ↗

73.23% · GPT-6 Astra (high) · DeepSWE · Pass@1 · % · Epoch AI · DeepSWE ↗

73.23% · GPT-6 Astra (max) · DeepSWE · Pass@1 · % · Epoch AI · DeepSWE ↗

72.79% · GPT-6 Astra (medium) · DeepSWE · Pass@1 · % · Epoch AI · DeepSWE ↗

67.04% · GPT-6 Astra (low) · DeepSWE · Pass@1 · % · Epoch AI · DeepSWE ↗

74.1% · GPT-6 Astra (max, with fallback) · DeepSWE v1.1 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

PostTrainBench ↗44.3%
40.249.3

GPT-6 Astra · PostTrainBench ↗

1 more reported result

41.9% · GPT-6 Astra (max, with fallback) · PostTrainBench · PostTrain Bench v1.2 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

Agents’ Last Exam ↗34.2%
34.239.5

GPT-6 Astra · Agents’ Last Exam · pass rate ↗

3 more reported results

59.3% · GPT-6 Astra · Agents’ Last Exam · source scoring · GPT-6 Astra launch report ↗

41.8% · GPT-6 Astra (max, with fallback) · Agents’ Last Exam · Agents' Last Exam (ALL) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

44.2% · GPT-6 Astra (max, with fallback) · Agents’ Last Exam · Agents' Last Exam (CLI) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

Chartography ↗71.0%
46.271.6

GPT-6 Astra · Chartography · no tools ↗

1 more reported result

91% · GPT-6 Astra (max, with fallback) · Chartography · ChartoGraphy w/ python · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

BenchCAD ↗95.9%
67.595.9

GPT-6 Astra · BenchCAD · tools ↗

1 more reported result

95.9% · GPT-6 Astra (max, with fallback) · BenchCAD · BenchCAD w/ tools · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

OSWorld 2.1 ↗75.8% partial / 34.3% binary
72.681.8

GPT-6 Astra (max) · OSWorld 2.1 · partial and binary ↗

1 more reported result

75.8% · GPT-6 Astra (max, with fallback) · OSWorld 2.1 · OSWorld V2.1 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

SWE-Marathon ↗42.5
42.557.5

GPT-6 Astra (max) · SWE-Marathon ↗

1 more reported result

42.5% · GPT-6 Astra (max, with fallback) · SWE-Marathon · SWE Marathon v1.1 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

STEMBench ↗2.809 / 4
2.762.81

GPT-6 Astra xhigh · STEMBench120 · mean progress / 4 · tools ↗

ErdősBench ↗3.230 / 4
2.683.23

GPT-6 Astra xhigh · ErdősBench · mean progress / 4 · Codex ↗

FrontierMath Tier 4 (v2) ↗97.6%
97.697.6

GPT-6 Astra (high) · FrontierMath-Tier-4-v2-Private · Best score (across scorers) · % ↗

4 more reported results

97.6% · GPT-6 Astra (xhigh) · FrontierMath-Tier-4-v2-Private · Best score (across scorers) · % · Epoch AI · FrontierMath-Tier-4-v2-Private ↗

97.6% · GPT-6 Astra (max) · FrontierMath-Tier-4-v2-Private · Best score (across scorers) · % · Epoch AI · FrontierMath-Tier-4-v2-Private ↗

97.56% · GPT-6 Astra (medium) · FrontierMath-Tier-4-v2-Private · Best score (across scorers) · % · Epoch AI · FrontierMath-Tier-4-v2-Private ↗

87.8% · GPT-6 Astra (low) · FrontierMath-Tier-4-v2-Private · Best score (across scorers) · % · Epoch AI · FrontierMath-Tier-4-v2-Private ↗

MirrorCode ↗46.7%
8.8977.4

GPT-6 Astra (high) · MirrorCode · Best score (across scorers) · % ↗

BALROG ↗68.3%
3.768.3

GPT-6 Astra (max) · Balrog · Average progress · % ↗

GSO ↗79.41%
9.888.24

GPT-6 Astra (unknown thinking) · GSO-Bench · Score OPT@1 · % ↗

SpireClimb ↗25.67%
025.67

GPT-6 Astra (max) · SpireClimb · Best score (across scorers) · % ↗

Vals RSI Index ↗0.43 reference score
00.46

GPT-6 Astra (max) · Compression · reference-anchored score ↗

3 more reported results

0.48 reference score · GPT-6 Astra (max) · LM training · reference-anchored score · Vals RSI Index · LM training ↗

0.10 reference score · GPT-6 Astra (max) · Harness engineering · reference-anchored score · Vals RSI Index · Harness engineering ↗

0.08 reference score · GPT-6 Astra (max) · Post-training · reference-anchored score · Vals RSI Index · Post-training ↗

BoxBench ↗77%
7277

GPT-6 Astra · BoxBench · weighted rubric accuracy · OpenAI · ChatGPT financial services launch ↗

FocusBench ↗47.7%
36.654.2

GPT-6 Astra (max) · FocusBench · score · Meta · Muse Wonder launch · maximum effort ↗

Kimi Code Bench 3.0 ↗66.5%
45.566.5

GPT-6 Astra (max, with fallback) · Kimi Code Bench 3.0 · Kimi Code Bench 3.0 Long-horizon (Internal) · Kimi K3.1 Pro · complete release comparison ↗

CharXiv ↗92.8%
85.693.8

GPT-6 Astra (max, with fallback) · CharXiv · CharXiv (RQ) w/ python · Kimi K3.1 Pro · complete release comparison ↗

1 more reported result

90% · GPT-6 Astra (max, with fallback) · CharXiv · CharXiv (RQ) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

BabyVision ↗96.6%
92.597.7

GPT-6 Astra (max, with fallback) · BabyVision · BabyVision w/ python · Kimi K3.1 Pro · complete release comparison ↗

ZeroBench ↗58%
5163

GPT-6 Astra (max, with fallback) · ZeroBench · ZeroBench_main w/ python (pass@5) · Kimi K3.1 Pro · complete release comparison ↗

2 more reported results

43.2% · GPT-6 Astra (max, with fallback) · ZeroBench · ZeroBench_main · avg@5 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

52% · GPT-6 Astra (max, with fallback) · ZeroBench · ZeroBench_main · pass@5 · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

DECK-Arena ↗1423 Elo
11191456

GPT-6 Astra (max, with fallback) · DECK-Arena · DECK-Arena (Internal, Elo-score) · Kimi K3.1 Pro · complete release comparison ↗

Design Bench ↗1076 Elo
7181076

GPT-6 Astra (max, with fallback) · Design Bench · Design Bench (Internal, Elo-score) · Kimi K3.1 Pro · complete release comparison ↗

Kimi Hallucination Benchmark ↗5.5%
5.523.3

GPT-6 Astra (max, with fallback) · Kimi Hallucination Benchmark · Kimi Hallu Bench Hard w/ tools (Internal, lower is better) · Kimi K3.1 Pro · complete release comparison ↗

1 more reported result

4.2% · GPT-6 Astra (max, with fallback) · Kimi Hallucination Benchmark · Kimi Hallu Bench w/o tools (Internal, lower is better) · Kimi K3.1 Pro · complete release comparison · Introducing Kimi K3.1 Pro ↗

FrontierMath: Open Problems ↗1 credited problems
12

GPT-6 Astra · FrontierMath: Open Problems · FM Open Problems · first credited AI solution records · Epoch public solution registry · October 27, 2026 ↗

BTF-3 ↗0.135 Brier scale
0.120.15

GPT-6 Astra (low) · Pooled forecasting score ↗

GDPx v1.0 ↗9.6%
1.6712.03

GPT-6 Astra · GDPx v1.0 · Overall · intended grading ↗

53 more reported results

11.8% · GPT-6 Astra · GDPx v1.0 · Eight-domain subset · intended grading · GDPx v1.0 — SemiAnalysis ↗

24.9% · GPT-6 Astra · GDPx v1.0 · Recursive Self-Improvement · intended grading · GDPx v1.0 — SemiAnalysis ↗

4.4% · GPT-6 Astra · GDPx v1.0 · Chip Design · intended grading · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Cybersecurity · intended grading · GDPx v1.0 — SemiAnalysis ↗

5.1% · GPT-6 Astra · GDPx v1.0 · Biology · intended grading · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Finance · intended grading · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Legal · intended grading · GDPx v1.0 — SemiAnalysis ↗

20.2% · GPT-6 Astra · GDPx v1.0 · Quant Trading · intended grading · GDPx v1.0 — SemiAnalysis ↗

16.5% · GPT-6 Astra · GDPx v1.0 · Hardware Design · intended grading · GDPx v1.0 — SemiAnalysis ↗

21.6% · GPT-6 Astra · GDPx v1.0 · Materials Science · intended grading · GDPx v1.0 — SemiAnalysis ↗

6.2% · GPT-6 Astra · GDPx v1.0 · Alignment · intended grading · GDPx v1.0 — SemiAnalysis ↗

6.7% · GPT-6 Astra · GDPx v1.0 · Software Engineering · intended grading · GDPx v1.0 — SemiAnalysis ↗

19.2% · GPT-6 Astra · GDPx v1.0 · Overall · partial credit · GDPx v1.0 — SemiAnalysis ↗

24.3% · GPT-6 Astra · GDPx v1.0 · Eight-domain subset · partial credit · GDPx v1.0 — SemiAnalysis ↗

24.9% · GPT-6 Astra · GDPx v1.0 · Recursive Self-Improvement · partial credit · GDPx v1.0 — SemiAnalysis ↗

9.7% · GPT-6 Astra · GDPx v1.0 · Chip Design · partial credit · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Cybersecurity · partial credit · GDPx v1.0 — SemiAnalysis ↗

10.3% · GPT-6 Astra · GDPx v1.0 · Biology · partial credit · GDPx v1.0 — SemiAnalysis ↗

52.9% · GPT-6 Astra · GDPx v1.0 · Finance · partial credit · GDPx v1.0 — SemiAnalysis ↗

31.0% · GPT-6 Astra · GDPx v1.0 · Legal · partial credit · GDPx v1.0 — SemiAnalysis ↗

20.2% · GPT-6 Astra · GDPx v1.0 · Quant Trading · partial credit · GDPx v1.0 — SemiAnalysis ↗

16.5% · GPT-6 Astra · GDPx v1.0 · Hardware Design · partial credit · GDPx v1.0 — SemiAnalysis ↗

32.6% · GPT-6 Astra · GDPx v1.0 · Materials Science · partial credit · GDPx v1.0 — SemiAnalysis ↗

6.2% · GPT-6 Astra · GDPx v1.0 · Alignment · partial credit · GDPx v1.0 — SemiAnalysis ↗

6.7% · GPT-6 Astra · GDPx v1.0 · Software Engineering · partial credit · GDPx v1.0 — SemiAnalysis ↗

6.2% · GPT-6 Astra · GDPx v1.0 · Aria · Chip Design · intended grading · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Arcverex · Chip Design · intended grading · GDPx v1.0 — SemiAnalysis ↗

7.1% · GPT-6 Astra · GDPx v1.0 · Ergodex · Chip Design · intended grading · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Arithmetic · Cybersecurity · intended grading · GDPx v1.0 — SemiAnalysis ↗

5.1% · GPT-6 Astra · GDPx v1.0 · LatchBio · Biology · intended grading · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Living Context · Finance · intended grading · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Chakra · Legal · intended grading · GDPx v1.0 — SemiAnalysis ↗

20.2% · GPT-6 Astra · GDPx v1.0 · Third Moment · Quant Trading · intended grading · GDPx v1.0 — SemiAnalysis ↗

24.9% · GPT-6 Astra · GDPx v1.0 · Extrapolate · Recursive Self-Improvement · intended grading · GDPx v1.0 — SemiAnalysis ↗

16.5% · GPT-6 Astra · GDPx v1.0 · Beautiful Machines · Hardware Design · intended grading · GDPx v1.0 — SemiAnalysis ↗

26.6% · GPT-6 Astra · GDPx v1.0 · ValenceForge · Materials Science · intended grading · GDPx v1.0 — SemiAnalysis ↗

16.5% · GPT-6 Astra · GDPx v1.0 · Emulated · Materials Science · intended grading · GDPx v1.0 — SemiAnalysis ↗

6.2% · GPT-6 Astra · GDPx v1.0 · Fleet · Alignment · intended grading · GDPx v1.0 — SemiAnalysis ↗

6.7% · GPT-6 Astra · GDPx v1.0 · Datacurve · Software Engineering · intended grading · GDPx v1.0 — SemiAnalysis ↗

22.1% · GPT-6 Astra · GDPx v1.0 · Aria · Chip Design · partial credit · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Arcverex · Chip Design · partial credit · GDPx v1.0 — SemiAnalysis ↗

7.1% · GPT-6 Astra · GDPx v1.0 · Ergodex · Chip Design · partial credit · GDPx v1.0 — SemiAnalysis ↗

0.0% · GPT-6 Astra · GDPx v1.0 · Arithmetic · Cybersecurity · partial credit · GDPx v1.0 — SemiAnalysis ↗

10.3% · GPT-6 Astra · GDPx v1.0 · LatchBio · Biology · partial credit · GDPx v1.0 — SemiAnalysis ↗

52.9% · GPT-6 Astra · GDPx v1.0 · Living Context · Finance · partial credit · GDPx v1.0 — SemiAnalysis ↗

31.0% · GPT-6 Astra · GDPx v1.0 · Chakra · Legal · partial credit · GDPx v1.0 — SemiAnalysis ↗

20.2% · GPT-6 Astra · GDPx v1.0 · Third Moment · Quant Trading · partial credit · GDPx v1.0 — SemiAnalysis ↗

24.9% · GPT-6 Astra · GDPx v1.0 · Extrapolate · Recursive Self-Improvement · partial credit · GDPx v1.0 — SemiAnalysis ↗

16.5% · GPT-6 Astra · GDPx v1.0 · Beautiful Machines · Hardware Design · partial credit · GDPx v1.0 — SemiAnalysis ↗

44.0% · GPT-6 Astra · GDPx v1.0 · ValenceForge · Materials Science · partial credit · GDPx v1.0 — SemiAnalysis ↗

21.3% · GPT-6 Astra · GDPx v1.0 · Emulated · Materials Science · partial credit · GDPx v1.0 — SemiAnalysis ↗

6.2% · GPT-6 Astra · GDPx v1.0 · Fleet · Alignment · partial credit · GDPx v1.0 — SemiAnalysis ↗

6.7% · GPT-6 Astra · GDPx v1.0 · Datacurve · Software Engineering · partial credit · GDPx v1.0 — SemiAnalysis ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

6.9 percentile-point gap

61 of 76 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

POST-TRAINING / SOURCE DISCLOSURE

What the GPT-6 Astra report says

The system card discusses reinforcement-learning grading, alignment improvements and safety behavior integrated into post-training. It reports how these changes are evaluated rather than providing the full production training recipe.

  • Reinforcement learning with alignment-focused grading
  • Safety behavior integrated into post-training
Preference learning and RLHF · background →Reward design · background →

A specific optimizer, training-data mixture and complete compute ledger are not given in the reviewed system card. Evaluation scores do not identify those missing details.

  1. GPT-6 Astra System Card ↗

    OpenAI · 2026

Filter comparisons to this model →

Reports featuring this model

Kimi K3.1 Pro · Kimi →

Claude Opus 5.5 · Anthropic →

GPT-6 Astra · OpenAI →

Muse Wonder and Muse Wonder Full · Meta →

Gemini 4 Argon · Google →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. Blueprint-Bench 2 | Andon Labs ↗

    Andon Labs

  2. ARC Prize - Leaderboard ↗

    ARC Prize Foundation

  3. Arena leaderboard — WebDev evaluation ↗

    Arena

  4. AA-3D: Interactive 3D Experience Benchmark | Artificial Analysis ↗

    Artificial Analysis

  5. AA-AnalystAgent Benchmark | Artificial Analysis ↗

    Artificial Analysis

  6. AA-Briefcase v1.1: Agentic Knowledge Work Benchmark | Artificial Analysis ↗

    Artificial Analysis

  7. AA-PersonalityBench: AI Personality & Communication Style Benchmark | Artificial Analysis ↗

    Artificial Analysis

  8. Artificial Analysis Intelligence Index v5.0 — Artificial Analysis ↗

    Artificial Analysis

  9. AA-LCR v1.1: Long Context Reasoning Benchmark | Artificial Analysis ↗

    Artificial Analysis

  10. AutomationBench-AA: Agentic SaaS Workflow Benchmark | Artificial Analysis ↗

    Artificial Analysis

  11. CritPt Benchmark | Artificial Analysis ↗

    Artificial Analysis

  12. CritPt v2 Benchmark | Artificial Analysis ↗

    Artificial Analysis

  13. CritPt v2 — Hard Subset — Artificial Analysis ↗

    Artificial Analysis

  14. FrontierCode 1.1 Benchmark | Artificial Analysis ↗

    Artificial Analysis

  15. FrontierCode 2.0 Benchmark — Artificial Analysis ↗

    Artificial Analysis

  16. GDP.pdf Benchmark | Artificial Analysis ↗

    Artificial Analysis

  17. GDPval-AA v2.1 Benchmark | Artificial Analysis ↗

    Artificial Analysis

  18. GPQA Diamond Benchmark | Artificial Analysis ↗

    Artificial Analysis

  19. Harvey LAB-AA v1.1 Benchmark | Artificial Analysis ↗

    Artificial Analysis

  20. HealthBench Professional Benchmark | Artificial Analysis ↗

    Artificial Analysis

  21. HLE-Diamond Benchmark — Artificial Analysis ↗

    Artificial Analysis

  22. Humanity's Last Exam Benchmark | Artificial Analysis ↗

    Artificial Analysis

  23. ITBench-AA Benchmark | Artificial Analysis ↗

    Artificial Analysis

  24. MLCR-AA: Medical Long Context Reasoning Benchmark | Artificial Analysis ↗

    Artificial Analysis

  25. MMMU-Pro Benchmark | Artificial Analysis ↗

    Artificial Analysis

  26. AA-Omniscience v1.1: Knowledge and Hallucination Benchmark — Artificial Analysis ↗

    Artificial Analysis

  27. SciCode Benchmark | Artificial Analysis ↗

    Artificial Analysis

  28. 𝜏³-Banking Benchmark | Artificial Analysis ↗

    Artificial Analysis

  29. Terminal-Bench-Science 0.1 Benchmark | Artificial Analysis ↗

    Artificial Analysis

  30. Terminal-Bench 2.1 Benchmark | Artificial Analysis ↗

    Artificial Analysis

  31. Terminal-Bench 4.0 Benchmark | Artificial Analysis ↗

    Artificial Analysis

  32. TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts — published result table ↗

    Jaffe, Oliver, Sherburn, Dane · 2026

  33. BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games ↗

    Paglieri, Davide, Cupiał, Bartłomiej, Coward, Samuel, Piterbarg, Ulyana, Wolczyk, Maciej, Khan, Akbir et al. · 2024

    All 13 authors

    Paglieri, Davide, Cupiał, Bartłomiej, Coward, Samuel, Piterbarg, Ulyana, Wolczyk, Maciej, Khan, Akbir, Pignatelli, Eduardo, Kuciński, Łukasz, Pinto, Lerrel, Fergus, Rob, Foerster, Jakob Nicolaus, Parker-Holder, Jack, Rocktäschel, Tim

  34. FrontierCode Leaderboard ↗

    Cognition

  35. DTBench — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

  36. LMCA — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

  37. Gemini 4 Argon Model Card ↗

    Google DeepMind · 2026

  38. DeepSWE ↗

    Datacurve

  39. Muse Wonder | Meta ↗

    Meta AI

  40. APEX-Agents ↗

    Epoch AI

  41. ARC-AGI-2 ↗

    Epoch AI

  42. Chess Puzzles ↗

    Epoch AI

  43. CritPt v1 ↗

    Epoch AI

  44. Earthborne Rangers (EBR-bench) ↗

    Epoch AI

  45. FrontierMath Erdős ↗

    Epoch AI

  46. FrontierMath Tier 4 (v2) ↗

    Epoch AI

  47. FrontierMath Tiers 1-3 (v2) ↗

    Epoch AI

  48. Furniture Assembly Benchmark ↗

    Epoch AI

  49. GPQA Diamond ↗

    Epoch AI

  50. Humanity's Last Exam ↗

    Epoch AI

  51. MirrorCode ↗

    Epoch AI

  52. Mystery Game Puzzles ↗

    Epoch AI

  53. OTIS Mock AIME 2024-2025 ↗

    Epoch AI

  54. Remote Labor Index ↗

    Epoch AI

  55. SimpleQA Verified ↗

    Epoch AI

  56. SpireClimb ↗

    Epoch AI

  57. Vending Bench 2 — methodology and published results ↗

    Epoch AI · 2026

  58. AI benchmarks & capabilities ↗

    Epoch AI

  59. FrontierMath: Open Problems — public problem and solution registry ↗

    Epoch AI · 2026

  60. FutureSearch Evals ↗

    FutureSearch

  61. GDPx v1.0: The General Agentic Benchmark — SemiAnalysis ↗

    SemiAnalysis · 2026

  62. GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents ↗

    Shetty, Manish, Jain, Naman, Liu, Jinjian, Kethanaboyina, Vijay, Sen, Koushik, Stoica, Ion · 2025

  63. WeirdML ↗

    Håvard Tveit Ihle

  64. DistressBench — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  65. DrugDiscoveryBench — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  66. HiL-Bench (Human-in-Loop Benchmark) — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  67. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  68. Humanity's Sixth Sense — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  69. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  70. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  71. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  72. SWE-Bench Pro V2 — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  73. SWE-Bench Pro V2 — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  74. SWE Atlas - Refactoring — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  75. SWE Atlas - Test Writing — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  76. GPT-6 Astra: A new generation of intelligence ↗

    OpenAI

  77. Introducing ChatGPT for financial services ↗

    OpenAI · 2026

  78. Introducing Muse Wonder ↗

    Meta AI · 2026

  79. GDP.pdf Benchmark | Surge AI ↗

    Surge AI

  80. Introducing Claude Opus 5.5 ↗

    Anthropic

  81. FrontierSWE — software engineering benchmark ↗

    FrontierSWE

  82. Introducing Kimi K3.1 Pro ↗

    Moonshot AI · 2026

  83. ErdősBench: research mathematics, scoped progress and proof audits ↗

    Ulam AI · 2026

  84. STEMBench — Six-Domain Model Evaluation | Ulam ↗

    Ulam

  85. BioMysteryBench — evaluation methodology and results ↗

    Vals AI · 2026

  86. Code Migration v1 — evaluation methodology and results ↗

    Vals AI · 2026

  87. Code Migration — evaluation methodology and results ↗

    Vals AI · 2026

  88. CUA-bench — evaluation methodology and results ↗

    Vals AI · 2026

  89. CyberBench — evaluation methodology and results ↗

    Vals AI · 2026

  90. Excel Modeling Benchmark v1 — evaluation methodology and results ↗

    Vals AI · 2026

  91. Excel Modeling Benchmark — evaluation methodology and results ↗

    Vals AI · 2026

  92. Finance Agent v2 — evaluation methodology and results ↗

    Vals AI · 2026

  93. Finance Agent — evaluation methodology and results ↗

    Vals AI · 2026

  94. Vals Finance Index — evaluation methodology and results ↗

    Vals AI · 2026

  95. HLAB-Vals-1.1 — evaluation methodology and results ↗

    Vals AI · 2026

  96. Harvey's Legal Agent Benchmark v1 — evaluation methodology and results ↗

    Vals AI · 2026

  97. IOI — evaluation methodology and results ↗

    Vals AI · 2026

  98. Vals Legal Index — evaluation methodology and results ↗

    Vals AI · 2026

  99. Legal Research Bench — evaluation methodology and results ↗

    Vals AI · 2026

  100. MedCode — evaluation methodology and results ↗

    Vals AI · 2026

  101. MedScribe — evaluation methodology and results ↗

    Vals AI · 2026

  102. MysteryMechanism — evaluation methodology and results ↗

    Vals AI · 2026

  103. ProgramBench — evaluation methodology and results ↗

    Vals AI · 2026

  104. ProofBench — evaluation methodology and results ↗

    Vals AI · 2026

  105. Robo-bench — evaluation methodology and results ↗

    Vals AI · 2026

  106. Vals RSI Index — evaluation methodology and results ↗

    Vals AI · 2026

  107. SAGE — evaluation methodology and results ↗

    Vals AI · 2026

  108. Vals Science Index — evaluation methodology and results ↗

    Vals AI · 2026

  109. SRE Bench — evaluation methodology and results ↗

    Vals AI · 2026

  110. SWE-bench Multimodal — evaluation methodology and results ↗

    Vals AI · 2026

  111. Tax Agent Bench — evaluation methodology and results ↗

    Vals AI · 2026

  112. Terminal-Bench 4.0 — evaluation methodology and results ↗

    Vals AI · 2026

  113. Terminal-Bench-Science — evaluation methodology and results ↗

    Vals AI · 2026

  114. Time Horizon Index: KSP — evaluation methodology and results ↗

    Vals AI · 2026

  115. Vals Index — evaluation methodology and results ↗

    Vals AI · 2026

  116. Vibe Code Bench 1-100 — evaluation methodology and results ↗

    Vals AI · 2026

  117. Vibe Code Bench — evaluation methodology and results ↗

    Vals AI · 2026

Names used in source tables

GPT 6 Astra · GPT 6 Astra (Codex) max · GPT 6 Astra (Codex) xHigh · GPT 6 Astra (mini-SWE-agent) max · GPT-6 Astra · GPT-6 Astra (Codex) high · GPT-6 Astra (Max) · GPT-6 Astra (Mini-SWE-agent) · GPT-6 Astra (OpenHands) · GPT-6 Astra (high) · GPT-6 Astra (low) · GPT-6 Astra (max) · GPT-6 Astra (max, with fallback) · GPT-6 Astra (medium) · GPT-6 Astra (pro, max) · GPT-6 Astra (unknown thinking) · GPT-6 Astra (xhigh) · GPT-6 Astra xhigh · GPT-6-Astra (Codex) high · GPT-6-Astra (max) · gpt-6-astra