IOI v1 & IOI v2

Official International Olympiad in Informatics problems evaluated under full subtask grading and submission limits.26 Oct 2026

One row per model and compatible score measure. Mean = equal-weight arithmetic average of imported reports. Distinct configurations reported for the same model in a source table remain separate. Reports can overlap; the range is not a confidence interval. Unbundle to inspect each observation.
		Evidence	What the result shows
Gemini 4 Argon →Google	100% 1 report	IOI v1 & IOI v2[1]Cost / Test: $5.31; Input / Output Cost: $4 / $20; Duration: 24m19s	Leads this displayed comparison (tied). $5.31 / test24m 19s
GPT-6 Astra →OpenAI	100% 1 report	IOI v1 & IOI v2[1]Cost / Test: $6.50; Input / Output Cost: $10 / $50; Duration: 31m40s	Leads this displayed comparison (tied). $6.50 / test31m 40s
GPT-6.1 Sol →OpenAI	96.89% 1 report	IOI v1 & IOI v2[1]Cost / Test: $1.26; Input / Output Cost: $2 / $10; Duration: 31m31s	#3 of 45 in this displayed comparison; 3.11 pp below the leader. $1.26 / test31m 31s
Claude Opus 5.5 →Anthropic	95.06% 1 report	IOI v1 & IOI v2[1]Cost / Test: $5.26; Input / Output Cost: $4 / $20; Duration: 17m15s	#4 of 45 in this displayed comparison; 4.94 pp below the leader. $5.26 / test17m 15s
GPT-5.6 Sol →OpenAI	91.17% 1 report	IOI v1 & IOI v2[1]Cost / Test: $7.98; Input / Output Cost: $4 / $20; Duration: 1h02m	#5 of 45 in this displayed comparison; 8.83 pp below the leader. $7.98 / test1h 02m
Claude Fable 5.1 →Anthropic	90.78% 1 report	IOI v1 & IOI v2[1]Cost / Test: $11.20; Input / Output Cost: $10 / $50; Duration: 30m06s	#6 of 45 in this displayed comparison; 9.22 pp below the leader. $11.20 / test30m 06s
GPT-5.6 Terra →OpenAI	87.61% 1 report	IOI v1 & IOI v2[1]Cost / Test: $8.66; Input / Output Cost: $2 / $12; Duration: 1h23m	#7 of 45 in this displayed comparison; 12.39 pp below the leader. $8.66 / test1h 23m
Claude Opus 5 →Anthropic	84.33% 1 report	IOI v1 & IOI v2[1]Cost / Test: $16.48; Input / Output Cost: $5 / $25; Duration: 56m42s	#8 of 45 in this displayed comparison; 15.67 pp below the leader. $16.48 / test56m 42s
Claude Sonnet 5.5 →Anthropic	83.06% 1 report	IOI v1 & IOI v2[1]Cost / Test: $7.52; Input / Output Cost: $2 / $10; Duration: 45m51s	#9 of 45 in this displayed comparison; 16.94 pp below the leader. $7.52 / test45m 51s
GPT-6 Sol →OpenAI	82.61% 1 report	IOI v1 & IOI v2[1]Cost / Test: $2.70; Input / Output Cost: $2 / $10; Duration: 30m38s	#10 of 45 in this displayed comparison; 17.39 pp below the leader. $2.70 / test30m 38s
Wonder Full →	79.83% 1 report	IOI v1 & IOI v2[1]Cost / Test: $7.08; Input / Output Cost: $4 / $20; Duration: 1h31m	#11 of 45 in this displayed comparison; 20.17 pp below the leader. $7.08 / test1h 31m
Qwen 3.8 Max →Qwen	68.89% 1 report	IOI v1 & IOI v2[1]Cost / Test: $9.47; Input / Output Cost: $2 / $6; Duration: 1h47m	#12 of 45 in this displayed comparison; 31.11 pp below the leader. $9.47 / test1h 47m
GLM 5.3 →Z.ai	68.44% 1 report	IOI v1 & IOI v2[1]Cost / Test: $7.67; Input / Output Cost: $1.4 / $4.4; Duration: 1h46m	#13 of 45 in this displayed comparison; 31.56 pp below the leader. $7.67 / test1h 46m
Gemini 3.7 Flash →Google	67.83% 1 report	IOI v1 & IOI v2[1]Cost / Test: $3.40; Input / Output Cost: $1.5 / $7.5; Duration: 21m09s	#14 of 45 in this displayed comparison; 32.17 pp below the leader. $3.40 / test21m 09s
Wonder →	66.94% 1 report	IOI v1 & IOI v2[1]Cost / Test: $4.54; Input / Output Cost: $4 / $20; Duration: 25m02s	#15 of 45 in this displayed comparison; 33.06 pp below the leader. $4.54 / test25m 02s
GPT-5.6 Luna →OpenAI	61.78% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.58; Input / Output Cost: $0.2 / $1.2; Duration: 1h10m	#16 of 45 in this displayed comparison; 38.22 pp below the leader. $0.58 / test1h 10m
Hy4 Preview →	59.33% 1 report	IOI v1 & IOI v2[1]Cost / Test: $1.35; Input / Output Cost: $0.834 / $2.501; Duration: 1h01m	#17 of 45 in this displayed comparison; 40.67 pp below the leader. $1.35 / test1h 01m
Grok 4.7 →xAI	57.72% 1 report	IOI v1 & IOI v2[1]Cost / Test: $12.71; Input / Output Cost: $2 / $6; Duration: 46m48s	#18 of 45 in this displayed comparison; 42.28 pp below the leader. $12.71 / test46m 48s
Gemini 3.8 Flash →Google	56.94% 1 report	IOI v1 & IOI v2[1]Cost / Test: $3.98; Input / Output Cost: $1.5 / $7.5; Duration: 14m20s	#19 of 45 in this displayed comparison; 43.06 pp below the leader. $3.98 / test14m 20s
Muse Spark 1.3 →MetaCost / Test: $1.73; Input / Output Cost: $1.25 / $4.25; Duration: 28m58s	56.56% 1 report	IOI v1 & IOI v2[1]Cost / Test: $1.73; Input / Output Cost: $1.25 / $4.25; Duration: 28m58s	#20 of 45 in this displayed comparison; 43.44 pp below the leader. $1.73 / test28m 58s
GPT-6 Luna →OpenAI	55.56% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.20; Input / Output Cost: $0.1 / $0.5; Duration: 35m47s	#21 of 45 in this displayed comparison; 44.44 pp below the leader. $0.20 / test35m 47s
GPT-5.3 Codex →OpenAI	53.83% 1 report	IOI v1 & IOI v2[1]Cost / Test: $2.68; Input / Output Cost: $1.75 / $14; Duration: 29m52s	#22 of 45 in this displayed comparison; 46.17 pp below the leader. $2.68 / test29m 52s
GLM 5.3 Flash →Z.ai	52.5% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.43; Input / Output Cost: $0.075 / $0.25; Duration: 1h32m	#23 of 45 in this displayed comparison; 47.5 pp below the leader. $0.43 / test1h 32m
Gemini 3.1 Pro Preview (02/26) →Google	51.83% 1 report	IOI v1 & IOI v2[1]Cost / Test: $1.25; Input / Output Cost: $2 / $12; Duration: 12m03s	#24 of 45 in this displayed comparison; 48.17 pp below the leader. $1.25 / test12m 03s
DeepSeek V4 Pro 0813 →DeepSeek	51.61% 1 report	IOI v1 & IOI v2[1]Cost / Test: $2.15; Input / Output Cost: $1.32 / $3.96; Duration: 1h07m	#25 of 45 in this displayed comparison; 48.39 pp below the leader. $2.15 / test1h 07m
Kimi K3 →Kimi	48.94% 1 report	IOI v1 & IOI v2[1]Cost / Test: $14.17; Input / Output Cost: $3 / $15; Duration: 4h18m	#26 of 45 in this displayed comparison; 51.06 pp below the leader. $14.17 / test4h 18m
MiMo V2.6 Flash →Xiaomi	47.67% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.22; Input / Output Cost: $0.14 / $0.28; Duration: 1h49m	#27 of 45 in this displayed comparison; 52.33 pp below the leader. $0.22 / test1h 49m
Grok 4.6 →xAI	47.61% 1 report	IOI v1 & IOI v2[1]Cost / Test: $8.18; Input / Output Cost: $2 / $6; Duration: 3h57m	#28 of 45 in this displayed comparison; 52.39 pp below the leader. $8.18 / test3h 57m
Claude Haiku 5.5 →Anthropic	47.28% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.83; Input / Output Cost: $0.1 / $0.5; Duration: 12m34s	#29 of 45 in this displayed comparison; 52.72 pp below the leader. $0.83 / test12m 34s
Ember-1 →	46.33% 1 report	IOI v1 & IOI v2[1]Cost / Test: $24.46; Input / Output Cost: $3 / $15; Duration: 1h47m	#30 of 45 in this displayed comparison; 53.67 pp below the leader. $24.46 / test1h 47m
Mistral Large 4 →Mistral	45.28% 1 report	IOI v1 & IOI v2[1]Cost / Test: $5.39; Input / Output Cost: $1.36 / $4.18; Duration: 1h24m	#31 of 45 in this displayed comparison; 54.72 pp below the leader. $5.39 / test1h 24m
Claude Sonnet 5 →Anthropic	45% 1 report	IOI v1 & IOI v2[1]Cost / Test: $12.89; Input / Output Cost: $2 / $10; Duration: 59m44s	#32 of 45 in this displayed comparison; 55 pp below the leader. $12.89 / test59m 44s
Muse Spark 1.3 →MetaCost / Test: $2.36; Input / Output Cost: $1.25 / $4.25; Duration: 29m51s	43.94% 1 report	IOI v1 & IOI v2[1]Cost / Test: $2.36; Input / Output Cost: $1.25 / $4.25; Duration: 29m51s	#33 of 45 in this displayed comparison; 56.06 pp below the leader. $2.36 / test29m 51s
Grok 4.5 →xAI	40.56% 1 report	IOI v1 & IOI v2[1]Cost / Test: $5.64; Input / Output Cost: $2 / $6; Duration: 1h27m	#34 of 45 in this displayed comparison; 59.44 pp below the leader. $5.64 / test1h 27m
DeepSeek V4.1 Flash →DeepSeek	40.28% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.24; Input / Output Cost: $0.3 / $1.2; Duration: 18m30s	#35 of 45 in this displayed comparison; 59.72 pp below the leader. $0.24 / test18m 30s
MiMo V2.6 Pro →Xiaomi	39.33% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.84; Input / Output Cost: $0.435 / $0.87; Duration: 2h26m	#36 of 45 in this displayed comparison; 60.67 pp below the leader. $0.84 / test2h 26m
Qwen 3.8 27B →Qwen	39.06% 1 report	IOI v1 & IOI v2[1]Cost / Test: $3.55; Input / Output Cost: $0.5 / $3; Duration: 1h39m	#37 of 45 in this displayed comparison; 60.94 pp below the leader. $3.55 / test1h 39m
Step 5 Preview →StepFun	38.06% 1 report	IOI v1 & IOI v2[1]Cost / Test: $2.03; Input / Output Cost: $1 / $2.7; Duration: 1h20m	#38 of 45 in this displayed comparison; 61.94 pp below the leader. $2.03 / test1h 20m
Gemini 3.6 Flash →Google	35.06% 1 report	IOI v1 & IOI v2[1]Cost / Test: $8.67; Input / Output Cost: $1.5 / $7.5; Duration: 48m23s	#39 of 45 in this displayed comparison; 64.94 pp below the leader. $8.67 / test48m 23s
DeepSeek V4 Flash 0731 →DeepSeek	32.72% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.37; Input / Output Cost: $0.44 / $1.32; Duration: 27m43s	#40 of 45 in this displayed comparison; 67.28 pp below the leader. $0.37 / test27m 43s
Beam 1.0 →	22.83% 1 report	IOI v1 & IOI v2[1]Cost / Test: N/A; Input / Output Cost: N/A; Duration: 7m13s	#41 of 45 in this displayed comparison; 77.17 pp below the leader. 7m 13s
Muse Spark 1.2 →Meta	21.78% 1 report	IOI v1 & IOI v2[1]Cost / Test: $2.64; Input / Output Cost: $1.25 / $4.25; Duration: 25m35s	#42 of 45 in this displayed comparison; 78.22 pp below the leader. $2.64 / test25m 35s
Inkling →	14.94% 1 report	IOI v1 & IOI v2[1]Cost / Test: $1.71; Input / Output Cost: $1 / $4.05; Duration: 30m12s	#43 of 45 in this displayed comparison; 85.06 pp below the leader. $1.71 / test30m 12s
Inkling Small →	9.33% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.34; Input / Output Cost: $0.3 / $1.2; Duration: 11m18s	#44 of 45 in this displayed comparison; 90.67 pp below the leader. $0.34 / test11m 18s
Mercury 2.5 →	2.44% 1 report	IOI v1 & IOI v2[1]Cost / Test: $0.16; Input / Output Cost: $0.2 / $0.75; Duration: 4m03s	#45 of 45 in this displayed comparison; 97.56 pp below the leader. $0.16 / test4m 03s

Sources

IOI — evaluation methodology and results ↗
Vals AI · 2026
Dates & evaluation details
IOI · Accuracy · 2026-10-26
Public source leaderboard captured 26 October 2026. Model names, configurations, native score units and source-reported uncertainty are retained. Version and task split follow this specific Vals methodology page. These are published observations, not evaluations run by RLTasks.

TRACE THE SCORE · Per-model notes use published breakdowns where available.

Score vs. cost per test

Vals AI · 44 observations

IOI · Accuracy · higher is better

Costs are Vals AI’s published Cost / Test in USD, paired with the score from the same result. They reflect that evaluation’s workload and configuration; they are not token prices or cost per successful task. Missing costs are omitted. Explore all reported results ↓

The dashed line traces the best observed score as reported cost increases. It describes this selection of reports, with their original configurations. It does not estimate unmeasured models or costs.

View plotted values and sources (44)

IOI v1 & IOI v2 · IOI · Accuracy. Published measurements in their original score units.
Model	Score	Cost per test (USD)	Configuration & source
Gemini 4 Argon →	100 %	$5.31	Cost / Test: $5.31; Input / Output Cost: $4 / $20; Duration: 24m19s Vals AI · IOI · Accuracy ↗
GPT-6 Astra →	100 %	$6.50	Cost / Test: $6.50; Input / Output Cost: $10 / $50; Duration: 31m40s Vals AI · IOI · Accuracy ↗
GPT-6.1 Sol →	96.89 %	$1.26	Cost / Test: $1.26; Input / Output Cost: $2 / $10; Duration: 31m31s Vals AI · IOI · Accuracy ↗
Claude Opus 5.5 →	95.06 %	$5.26	Cost / Test: $5.26; Input / Output Cost: $4 / $20; Duration: 17m15s Vals AI · IOI · Accuracy ↗
GPT-5.6 Sol →	91.17 %	$7.98	Cost / Test: $7.98; Input / Output Cost: $4 / $20; Duration: 1h02m Vals AI · IOI · Accuracy ↗
Claude Fable 5.1 →	90.78 %	$11.20	Cost / Test: $11.20; Input / Output Cost: $10 / $50; Duration: 30m06s Vals AI · IOI · Accuracy ↗
GPT-5.6 Terra →	87.61 %	$8.66	Cost / Test: $8.66; Input / Output Cost: $2 / $12; Duration: 1h23m Vals AI · IOI · Accuracy ↗
Claude Opus 5 →	84.33 %	$16.48	Cost / Test: $16.48; Input / Output Cost: $5 / $25; Duration: 56m42s Vals AI · IOI · Accuracy ↗
Claude Sonnet 5.5 →	83.06 %	$7.52	Cost / Test: $7.52; Input / Output Cost: $2 / $10; Duration: 45m51s Vals AI · IOI · Accuracy ↗
GPT-6 Sol →	82.61 %	$2.70	Cost / Test: $2.70; Input / Output Cost: $2 / $10; Duration: 30m38s Vals AI · IOI · Accuracy ↗
Wonder Full →	79.83 %	$7.08	Cost / Test: $7.08; Input / Output Cost: $4 / $20; Duration: 1h31m Vals AI · IOI · Accuracy ↗
Qwen 3.8 Max →	68.89 %	$9.47	Cost / Test: $9.47; Input / Output Cost: $2 / $6; Duration: 1h47m Vals AI · IOI · Accuracy ↗
GLM 5.3 →	68.44 %	$7.67	Cost / Test: $7.67; Input / Output Cost: $1.4 / $4.4; Duration: 1h46m Vals AI · IOI · Accuracy ↗
Gemini 3.7 Flash →	67.83 %	$3.40	Cost / Test: $3.40; Input / Output Cost: $1.5 / $7.5; Duration: 21m09s Vals AI · IOI · Accuracy ↗
Wonder →	66.94 %	$4.54	Cost / Test: $4.54; Input / Output Cost: $4 / $20; Duration: 25m02s Vals AI · IOI · Accuracy ↗
GPT-5.6 Luna →	61.78 %	$0.58	Cost / Test: $0.58; Input / Output Cost: $0.2 / $1.2; Duration: 1h10m Vals AI · IOI · Accuracy ↗
Hy4 Preview →	59.33 %	$1.35	Cost / Test: $1.35; Input / Output Cost: $0.834 / $2.501; Duration: 1h01m Vals AI · IOI · Accuracy ↗
Grok 4.7 →	57.72 %	$12.71	Cost / Test: $12.71; Input / Output Cost: $2 / $6; Duration: 46m48s Vals AI · IOI · Accuracy ↗
Gemini 3.8 Flash →	56.94 %	$3.98	Cost / Test: $3.98; Input / Output Cost: $1.5 / $7.5; Duration: 14m20s Vals AI · IOI · Accuracy ↗
Muse Spark 1.3 Max →	56.56 %	$1.73	Cost / Test: $1.73; Input / Output Cost: $1.25 / $4.25; Duration: 28m58s Vals AI · IOI · Accuracy ↗
GPT-6 Luna →	55.56 %	$0.20	Cost / Test: $0.20; Input / Output Cost: $0.1 / $0.5; Duration: 35m47s Vals AI · IOI · Accuracy ↗
GPT 5.3 Codex →	53.83 %	$2.68	Cost / Test: $2.68; Input / Output Cost: $1.75 / $14; Duration: 29m52s Vals AI · IOI · Accuracy ↗
GLM 5.3 Flash →	52.5 %	$0.43	Cost / Test: $0.43; Input / Output Cost: $0.075 / $0.25; Duration: 1h32m Vals AI · IOI · Accuracy ↗
Gemini 3.1 Pro Preview (02/26) →	51.83 %	$1.25	Cost / Test: $1.25; Input / Output Cost: $2 / $12; Duration: 12m03s Vals AI · IOI · Accuracy ↗
DeepSeek V4 Pro 0813 →	51.61 %	$2.15	Cost / Test: $2.15; Input / Output Cost: $1.32 / $3.96; Duration: 1h07m Vals AI · IOI · Accuracy ↗
Kimi K3 →	48.94 %	$14.17	Cost / Test: $14.17; Input / Output Cost: $3 / $15; Duration: 4h18m Vals AI · IOI · Accuracy ↗
MiMo V2.6 Flash →	47.67 %	$0.22	Cost / Test: $0.22; Input / Output Cost: $0.14 / $0.28; Duration: 1h49m Vals AI · IOI · Accuracy ↗
Grok 4.6 →	47.61 %	$8.18	Cost / Test: $8.18; Input / Output Cost: $2 / $6; Duration: 3h57m Vals AI · IOI · Accuracy ↗
Claude Haiku 5.5 →	47.28 %	$0.83	Cost / Test: $0.83; Input / Output Cost: $0.1 / $0.5; Duration: 12m34s Vals AI · IOI · Accuracy ↗
Ember-1 →	46.33 %	$24.46	Cost / Test: $24.46; Input / Output Cost: $3 / $15; Duration: 1h47m Vals AI · IOI · Accuracy ↗
Mistral Large 4 →	45.28 %	$5.39	Cost / Test: $5.39; Input / Output Cost: $1.36 / $4.18; Duration: 1h24m Vals AI · IOI · Accuracy ↗
Claude Sonnet 5 →	45 %	$12.89	Cost / Test: $12.89; Input / Output Cost: $2 / $10; Duration: 59m44s Vals AI · IOI · Accuracy ↗
Muse Spark 1.3 →	43.94 %	$2.36	Cost / Test: $2.36; Input / Output Cost: $1.25 / $4.25; Duration: 29m51s Vals AI · IOI · Accuracy ↗
Grok 4.5 →	40.56 %	$5.64	Cost / Test: $5.64; Input / Output Cost: $2 / $6; Duration: 1h27m Vals AI · IOI · Accuracy ↗
DeepSeek V4.1 Flash →	40.28 %	$0.24	Cost / Test: $0.24; Input / Output Cost: $0.3 / $1.2; Duration: 18m30s Vals AI · IOI · Accuracy ↗
MiMo V2.6 Pro →	39.33 %	$0.84	Cost / Test: $0.84; Input / Output Cost: $0.435 / $0.87; Duration: 2h26m Vals AI · IOI · Accuracy ↗
Qwen 3.8 27B →	39.06 %	$3.55	Cost / Test: $3.55; Input / Output Cost: $0.5 / $3; Duration: 1h39m Vals AI · IOI · Accuracy ↗
Step 5 Preview →	38.06 %	$2.03	Cost / Test: $2.03; Input / Output Cost: $1 / $2.7; Duration: 1h20m Vals AI · IOI · Accuracy ↗
Gemini 3.6 Flash →	35.06 %	$8.67	Cost / Test: $8.67; Input / Output Cost: $1.5 / $7.5; Duration: 48m23s Vals AI · IOI · Accuracy ↗
DeepSeek V4 Flash 0731 →	32.72 %	$0.37	Cost / Test: $0.37; Input / Output Cost: $0.44 / $1.32; Duration: 27m43s Vals AI · IOI · Accuracy ↗
Muse Spark 1.2 →	21.78 %	$2.64	Cost / Test: $2.64; Input / Output Cost: $1.25 / $4.25; Duration: 25m35s Vals AI · IOI · Accuracy ↗
Inkling →	14.94 %	$1.71	Cost / Test: $1.71; Input / Output Cost: $1 / $4.05; Duration: 30m12s Vals AI · IOI · Accuracy ↗
Inkling Small →	9.33 %	$0.34	Cost / Test: $0.34; Input / Output Cost: $0.3 / $1.2; Duration: 11m18s Vals AI · IOI · Accuracy ↗
Mercury 2.5 →	2.44 %	$0.16	Cost / Test: $0.16; Input / Output Cost: $0.2 / $0.75; Duration: 4m03s Vals AI · IOI · Accuracy ↗

What it measures

Official International Olympiad in Informatics problems evaluated under full subtask grading and submission limits.

Reading the measurement

Look beyond a plausible code sample to the executable artifact. Tests, repository context, language coverage and time limits shape the result. Separate first-attempt correctness from success after debugging, retries or agent assistance.

Published evaluation

The linked publication documents the task and scoring procedure. Model results retain the source’s evaluation setting, native score units and available uncertainty.

Where this fits in an evaluation suite

The agent wrapper is part of the system. Pin tools, dependencies, timeouts, tests, and the repository revision. Compare contest coding separately from repository maintenance.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Record the source publication, benchmark version, scoring units, denominator, tool access and attempt budget. Preserve uncertainty and distinguish developer-reported results from independent evaluations.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.

PRIMARY EVIDENCE & FURTHER READING

IOI — evaluation methodology and results ↗
Vals AI · 2026

Reviewed 2026-10-26. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.

Continue exploring

All code & engineering benchmarks →

Sources

Score vs. cost per test

What it measures

Reading the measurement

Published evaluation

Where this fits in an evaluation suite

A reproducible comparison

Continue exploring

AA-3D

Terminal-Bench 4.0

FrontierCode 1.1

Terminal-Bench 2.1