Public Benefits Bench v1.1

Evaluates whether AI models can accurately determine household SNAP eligibility and benefit amounts across complex state/federal rules.26 Oct 2026

One row per model and compatible score measure. Mean = equal-weight arithmetic average of imported reports. Distinct configurations reported for the same model in a source table remain separate. Reports can overlap; the range is not a confidence interval. Unbundle to inspect each observation.
		Evidence	What the result shows
Claude Opus 5 →Anthropic	76.93% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $3.95; Input / Output Cost: $5 / $25; Target-agent time: 30m09s	Leads this displayed comparison. $3.95 / test
Claude Fable 5.1 →Anthropic	74.9% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $7.48; Input / Output Cost: $10 / $50; Target-agent time: 48m02s	#2 of 50 in this displayed comparison; 2.03 pp below the leader. $7.48 / test
Wonder Full →	73.75% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $4.64; Input / Output Cost: $4 / $20; Target-agent time: 1h13m	#3 of 50 in this displayed comparison; 3.18 pp below the leader. $4.64 / test
Claude Fable 5 →Anthropic	71.04% 2 reports · range 70.43–71.65%	Public Benefits Bench v1.1[2][1] View 2 reports 71.65 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $1.05; Input / Output Cost: $10 / $50; Duration: 17m18s [2] 70.43 % · Public Benefits Bench · Accuracy · Cost / Test: $4.59; Input / Output Cost: $10 / $50; Target-agent time: 21m17s [1]	Published observations span 1.22 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #4 of 50 in this displayed comparison; 5.89 pp below the leader.
Claude Opus 5.5 →Anthropic	70.64% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $5.95; Input / Output Cost: $4 / $20; Target-agent time: 1h20m	#5 of 50 in this displayed comparison; 6.29 pp below the leader. $5.95 / test
Wonder →	70.23% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $2.46; Input / Output Cost: $4 / $20; Target-agent time: 19m44s	#6 of 50 in this displayed comparison; 6.7 pp below the leader. $2.46 / test
Gemini 4 Argon →Google	69.76% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $2.79; Input / Output Cost: $4 / $20; Target-agent time: 26m07s	#7 of 50 in this displayed comparison; 7.17 pp below the leader. $2.79 / test
MiMo V2.6 Pro →Xiaomi	68.94% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.08; Input / Output Cost: $0.435 / $0.87; Target-agent time: 37m14s	#8 of 50 in this displayed comparison; 7.99 pp below the leader. $0.08 / test
Hy4 Preview →	68.61% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.25; Input / Output Cost: $0.834 / $2.501; Target-agent time: 1h03m	#9 of 50 in this displayed comparison; 8.32 pp below the leader. $0.25 / test
GLM 5.3 →Z.ai	68.54% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.95; Input / Output Cost: $1.4 / $4.4; Target-agent time: 35m51s	#10 of 50 in this displayed comparison; 8.39 pp below the leader. $0.95 / test
Muse Spark 1.2 →Meta	68.47% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.59; Input / Output Cost: $1.25 / $4.25; Target-agent time: 8m55s	#11 of 50 in this displayed comparison; 8.46 pp below the leader. $0.59 / test
Kimi K3 →Kimi	68.2% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.91; Input / Output Cost: $3 / $15; Target-agent time: 7m19s	#12 of 50 in this displayed comparison; 8.73 pp below the leader. $0.91 / test
MiMo V2.6 Flash →Xiaomi	67.59% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.03; Input / Output Cost: $0.14 / $0.28; Target-agent time: 20m56s	#13 of 50 in this displayed comparison; 9.34 pp below the leader. $0.03 / test
Claude Sonnet 5.5 →Anthropic	67.19% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $4.25; Input / Output Cost: $2 / $10; Target-agent time: 1h15m	#14 of 50 in this displayed comparison; 9.74 pp below the leader. $4.25 / test
Qwen 3.8 Max →Qwen	67.12% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.99; Input / Output Cost: $2 / $6; Target-agent time: 40m44s	#15 of 50 in this displayed comparison; 9.81 pp below the leader. $0.99 / test
Grok 4.6 →xAI	66.85% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.92; Input / Output Cost: $2 / $6; Target-agent time: 25m53s	#16 of 50 in this displayed comparison; 10.08 pp below the leader. $0.92 / test
GPT-5.6 Sol →OpenAI	66.51% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $8.85; Input / Output Cost: $4 / $20; Target-agent time: 1h01m	#17 of 50 in this displayed comparison; 10.42 pp below the leader. $8.85 / test
Ember-1 →	66.17% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $1.02; Input / Output Cost: $3 / $15; Target-agent time: 13m21s	#18 of 50 in this displayed comparison; 10.76 pp below the leader. $1.02 / test
Claude Sonnet 5 →Anthropic	66.03% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $1.29; Input / Output Cost: $2 / $10; Target-agent time: 21m09s	#19 of 50 in this displayed comparison; 10.9 pp below the leader. $1.29 / test
Grok 4.7 →xAI	65.63% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $2.25; Input / Output Cost: $2 / $6; Target-agent time: 16m57s	#20 of 50 in this displayed comparison; 11.3 pp below the leader. $2.25 / test
Gemini 3.8 Flash →Google	65.29% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $1.01; Input / Output Cost: $1.5 / $7.5; Target-agent time: 3m21s	#21 of 50 in this displayed comparison; 11.64 pp below the leader. $1.01 / test
Claude Opus 4.8 →Anthropic	65.12% 2 reports · range 62.11–68.13%	Public Benefits Bench v1.1[1][2] View 2 reports 68.13 % · Public Benefits Bench · Accuracy · Cost / Test: $1.89; Input / Output Cost: $5 / $25; Target-agent time: 12m48s [1] 62.11 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $1.32; Input / Output Cost: $5 / $25; Duration: 9m58s [2]	Published observations span 6.02 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #22 of 50 in this displayed comparison; 11.81 pp below the leader.
DeepSeek V4.1 Flash →DeepSeek	64.28% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.07; Input / Output Cost: $0.3 / $1.2; Target-agent time: 9m59s	#23 of 50 in this displayed comparison; 12.65 pp below the leader. $0.07 / test
Claude Haiku 5.5 →Anthropic	63.19% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.53; Input / Output Cost: $0.1 / $0.5; Target-agent time: 36m32s	#24 of 50 in this displayed comparison; 13.74 pp below the leader. $0.53 / test
MiniMax M3 →MiniMax	62.41% 2 reports · range 60.69–64.14%	Public Benefits Bench v1.1[1][2] View 2 reports 64.14 % · Public Benefits Bench · Accuracy · Cost / Test: $0.25; Input / Output Cost: $0.6 / $2.4; Target-agent time: 11m31s [1] 60.69 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.20; Input / Output Cost: $0.6 / $2.4; Duration: 11m46s [2]	Published observations span 3.45 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #25 of 50 in this displayed comparison; 14.52 pp below the leader.
GPT-5.6 Terra →OpenAI	62.38% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $1.20; Input / Output Cost: $2 / $12; Target-agent time: 19m31s	#26 of 50 in this displayed comparison; 14.55 pp below the leader. $1.20 / test
Muse Spark 1.1 →Meta	61.3% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.28; Input / Output Cost: $1.25 / $4.25; Target-agent time: 6m35s	#27 of 50 in this displayed comparison; 15.63 pp below the leader. $0.28 / test
GPT-5.6 Luna →OpenAI	61.16% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.27; Input / Output Cost: $0.2 / $1.2; Target-agent time: 38m40s	#28 of 50 in this displayed comparison; 15.77 pp below the leader. $0.27 / test
Claude Sonnet 4.6 →Anthropic	60.48% 2 reports · range 58.52–62.45%	Public Benefits Bench v1.1[1][2] View 2 reports 62.45 % · Public Benefits Bench · Accuracy · Cost / Test: $1.11; Input / Output Cost: $3 / $15; Target-agent time: 14m43s [1] 58.52 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.82; Input / Output Cost: $3 / $15; Duration: 13m08s [2]	Published observations span 3.93 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #29 of 50 in this displayed comparison; 16.45 pp below the leader.
DeepSeek V4 →DeepSeek	60.25% 2 reports · range 57.58–62.92%	Public Benefits Bench v1.1[1][2] View 2 reports 62.92 % · Public Benefits Bench · Accuracy · Cost / Test: $0.46; Input / Output Cost: $1.32 / $3.96; Target-agent time: 15m22s [1] 57.58 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.36; Input / Output Cost: $1.32 / $3.96; Duration: 16m18s [2]	Published observations span 5.34 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #30 of 50 in this displayed comparison; 16.68 pp below the leader.
GLM 5.1 →Z.ai	59.88% 2 reports · range 57.92–61.84%	Public Benefits Bench v1.1[1][2] View 2 reports 61.84 % · Public Benefits Bench · Accuracy · Cost / Test: $0.34; Input / Output Cost: $1 / $3.2; Target-agent time: 9m40s [1] 57.92 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.34; Input / Output Cost: $1 / $3.2; Duration: 7m18s [2]	Published observations span 3.92 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #31 of 50 in this displayed comparison; 17.05 pp below the leader.
Inkling Small →	59.41% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.10; Input / Output Cost: $0.3 / $1.2; Target-agent time: 8m41s	#32 of 50 in this displayed comparison; 17.52 pp below the leader. $0.10 / test
GPT-6.1 Sol →OpenAI	59.34% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.69; Input / Output Cost: $2 / $10; Target-agent time: 50m24s	#33 of 50 in this displayed comparison; 17.59 pp below the leader. $0.69 / test
GPT-5.5 →OpenAI	59.06% 2 reports · range 57.24–60.89%	Public Benefits Bench v1.1[1][2] View 2 reports 60.89 % · Public Benefits Bench · Accuracy · Cost / Test: $3.97; Input / Output Cost: $5 / $30; Target-agent time: 39m37s [1] 57.24 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $2.02; Input / Output Cost: $5 / $30; Duration: 26m21s [2]	Published observations span 3.65 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #34 of 50 in this displayed comparison; 17.87 pp below the leader.
Gemini 3.5 Flash →Google	58.72% 2 reports · range 57.98–59.47%	Public Benefits Bench v1.1[1][2] View 2 reports 59.47 % · Public Benefits Bench · Accuracy · Cost / Test: $1.11; Input / Output Cost: $1.5 / $9; Target-agent time: 5m39s [1] 57.98 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.78; Input / Output Cost: $1.5 / $9; Duration: 4m34s [2]	Published observations span 1.49 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #35 of 50 in this displayed comparison; 18.21 pp below the leader.
Inkling →	58.46% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.37; Input / Output Cost: $1 / $4.05; Target-agent time: 13m31s	#36 of 50 in this displayed comparison; 18.47 pp below the leader. $0.37 / test
Step 5 Preview →StepFun	58.12% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.50; Input / Output Cost: $1 / $2.7; Target-agent time: 27m19s	#37 of 50 in this displayed comparison; 18.81 pp below the leader. $0.50 / test
GPT-6 Luna →OpenAI	57.65% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.22; Input / Output Cost: $0.1 / $0.5; Target-agent time: 52m59s	#38 of 50 in this displayed comparison; 19.28 pp below the leader. $0.22 / test
GPT-6 Sol →OpenAI	56.63% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $2.49; Input / Output Cost: $2 / $10; Target-agent time: 31m02s	#39 of 50 in this displayed comparison; 20.3 pp below the leader. $2.49 / test
Gemini 3.6 Flash →Google	56.56% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.39; Input / Output Cost: $1.5 / $7.5; Target-agent time: 2m02s	#40 of 50 in this displayed comparison; 20.37 pp below the leader. $0.39 / test
Kimi K2.6 →Kimi	55.01% 2 reports · range 53.38–56.63%	Public Benefits Bench v1.1[1][2] View 2 reports 56.63 % · Public Benefits Bench · Accuracy · Cost / Test: $0.91; Input / Output Cost: $0.95 / $4; Target-agent time: 9m55s [1] 53.38 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.61; Input / Output Cost: $0.95 / $4; Duration: 9m39s [2]	Published observations span 3.25 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #41 of 50 in this displayed comparison; 21.93 pp below the leader.
Gemini 3.1 Pro Preview (02/26) →Google	52.34% 2 reports · range 50.88–53.79%	Public Benefits Bench v1.1[1][2] View 2 reports 53.79 % · Public Benefits Bench · Accuracy · Cost / Test: $0.38; Input / Output Cost: $2 / $12; Target-agent time: 3m06s [1] 50.88 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.30; Input / Output Cost: $2 / $12; Duration: 2m58s [2]	Published observations span 2.91 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #42 of 50 in this displayed comparison; 24.6 pp below the leader.
Claude Haiku 4.5 →Anthropic	51.89% 2 reports · range 49.53–54.26%	Public Benefits Bench v1.1[1][2] View 2 reports 54.26 % · Public Benefits Bench · Accuracy · Cost / Test: $0.22; Input / Output Cost: $1 / $5; Target-agent time: 2m58s [1] 49.53 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.19; Input / Output Cost: $1 / $5; Duration: 3m10s [2]	Published observations span 4.73 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #43 of 50 in this displayed comparison; 25.04 pp below the leader.
Gemini 3.5 Flash Lite →Google	51.83% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.10; Input / Output Cost: $0.3 / $2.5; Target-agent time: 69.26s	#44 of 50 in this displayed comparison; 25.1 pp below the leader. $0.10 / test
Grok 4.3 →xAI	50.88% 2 reports · range 50.07–51.69%	Public Benefits Bench v1.1[1][2] View 2 reports 51.69 % · Public Benefits Bench · Accuracy · Cost / Test: $0.26; Input / Output Cost: $1.25 / $2.5; Target-agent time: 3m25s [1] 50.07 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $1.21; Input / Output Cost: $1.25 / $2.5; Duration: 6m58s [2]	Published observations span 1.62 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #45 of 50 in this displayed comparison; 26.05 pp below the leader.
Ling 3.0 Flash →	50.74% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.05; Input / Output Cost: $0.075 / $0.22; Target-agent time: 4m35s	#46 of 50 in this displayed comparison; 26.19 pp below the leader. $0.05 / test
Mercury 2.5 →	45.26% 1 report	Public Benefits Bench v1.1[1]Cost / Test: $0.07; Input / Output Cost: $0.2 / $0.75; Target-agent time: 68.33s	#47 of 50 in this displayed comparison; 31.67 pp below the leader. $0.07 / test
Grok 4.1 Fast (Reasoning) →xAI	44.55% 2 reports · range 44.32–44.79%	Public Benefits Bench v1.1[1][2] View 2 reports 44.79 % · Public Benefits Bench · Accuracy · Cost / Test: $0.03; Input / Output Cost: $0.2 / $0.5; Target-agent time: 116.82s [1] 44.32 % · Public Benefits Bench v1 · Accuracy · Cost / Test: $0.06; Input / Output Cost: $0.2 / $0.5; Duration: 3m05s [2]	Published observations span 0.47 percentage points. Configuration and source differences contribute to this spread; it is not measured run-to-run uncertainty. #48 of 50 in this displayed comparison; 32.38 pp below the leader.
Laguna M.1 →	43.84% 1 report	Public Benefits Bench v1.1[1]Cost / Test: N/A; Input / Output Cost: N/A; Target-agent time: 5m35s	#49 of 50 in this displayed comparison; 33.09 pp below the leader.
Laguna XS.2 →	41.41% 1 report	Public Benefits Bench v1.1[1]Cost / Test: N/A; Input / Output Cost: N/A; Target-agent time: 5m05s	#50 of 50 in this displayed comparison; 35.52 pp below the leader.

Sources

Public Benefits Bench — evaluation methodology and results ↗
Vals AI · 2026
Dates & evaluation details
Public Benefits Bench · Accuracy · 2026-10-26
Public source leaderboard captured 26 October 2026. Model names, configurations, native score units and source-reported uncertainty are retained. Version and task split follow this specific Vals methodology page. These are published observations, not evaluations run by RLTasks.
Public Benefits Bench v1 — evaluation methodology and results ↗
Vals AI · 2026
Dates & evaluation details
Public Benefits Bench v1 · Accuracy · 2026-10-26
Public source leaderboard captured 26 October 2026. Model names, configurations, native score units and source-reported uncertainty are retained. Version and task split follow this specific Vals methodology page. These are published observations, not evaluations run by RLTasks.

TRACE THE SCORE · Per-model notes use published breakdowns where available.

Score vs. cost per test

Vals AI · 48 observations

Public Benefits Bench · Accuracy · higher is better

Costs are Vals AI’s published Cost / Test in USD, paired with the score from the same result. They reflect that evaluation’s workload and configuration; they are not token prices or cost per successful task. Missing costs are omitted. Explore all reported results ↓

The dashed line traces the best observed score as reported cost increases. It describes this selection of reports, with their original configurations. It does not estimate unmeasured models or costs.

View plotted values and sources (48)

Public Benefits Bench v1.1 · Public Benefits Bench · Accuracy. Published measurements in their original score units.
Model	Score	Cost per test (USD)	Configuration & source
Claude Opus 5 →	76.93 %	$3.95	Cost / Test: $3.95; Input / Output Cost: $5 / $25; Target-agent time: 30m09s Vals AI · Public Benefits Bench · Accuracy ↗
Claude Fable 5.1 →	74.9 %	$7.48	Cost / Test: $7.48; Input / Output Cost: $10 / $50; Target-agent time: 48m02s Vals AI · Public Benefits Bench · Accuracy ↗
Wonder Full →	73.75 %	$4.64	Cost / Test: $4.64; Input / Output Cost: $4 / $20; Target-agent time: 1h13m Vals AI · Public Benefits Bench · Accuracy ↗
Claude Opus 5.5 →	70.64 %	$5.95	Cost / Test: $5.95; Input / Output Cost: $4 / $20; Target-agent time: 1h20m Vals AI · Public Benefits Bench · Accuracy ↗
Claude Fable 5 →	70.43 %	$4.59	Cost / Test: $4.59; Input / Output Cost: $10 / $50; Target-agent time: 21m17s Vals AI · Public Benefits Bench · Accuracy ↗
Wonder →	70.23 %	$2.46	Cost / Test: $2.46; Input / Output Cost: $4 / $20; Target-agent time: 19m44s Vals AI · Public Benefits Bench · Accuracy ↗
Gemini 4 Argon →	69.76 %	$2.79	Cost / Test: $2.79; Input / Output Cost: $4 / $20; Target-agent time: 26m07s Vals AI · Public Benefits Bench · Accuracy ↗
MiMo V2.6 Pro →	68.94 %	$0.08	Cost / Test: $0.08; Input / Output Cost: $0.435 / $0.87; Target-agent time: 37m14s Vals AI · Public Benefits Bench · Accuracy ↗
Hy4 Preview →	68.61 %	$0.25	Cost / Test: $0.25; Input / Output Cost: $0.834 / $2.501; Target-agent time: 1h03m Vals AI · Public Benefits Bench · Accuracy ↗
GLM 5.3 →	68.54 %	$0.95	Cost / Test: $0.95; Input / Output Cost: $1.4 / $4.4; Target-agent time: 35m51s Vals AI · Public Benefits Bench · Accuracy ↗
Muse Spark 1.2 →	68.47 %	$0.59	Cost / Test: $0.59; Input / Output Cost: $1.25 / $4.25; Target-agent time: 8m55s Vals AI · Public Benefits Bench · Accuracy ↗
Kimi K3 →	68.2 %	$0.91	Cost / Test: $0.91; Input / Output Cost: $3 / $15; Target-agent time: 7m19s Vals AI · Public Benefits Bench · Accuracy ↗
Claude Opus 4.8 →	68.13 %	$1.89	Cost / Test: $1.89; Input / Output Cost: $5 / $25; Target-agent time: 12m48s Vals AI · Public Benefits Bench · Accuracy ↗
MiMo V2.6 Flash →	67.59 %	$0.03	Cost / Test: $0.03; Input / Output Cost: $0.14 / $0.28; Target-agent time: 20m56s Vals AI · Public Benefits Bench · Accuracy ↗
Claude Sonnet 5.5 →	67.19 %	$4.25	Cost / Test: $4.25; Input / Output Cost: $2 / $10; Target-agent time: 1h15m Vals AI · Public Benefits Bench · Accuracy ↗
Qwen 3.8 Max →	67.12 %	$0.99	Cost / Test: $0.99; Input / Output Cost: $2 / $6; Target-agent time: 40m44s Vals AI · Public Benefits Bench · Accuracy ↗
Grok 4.6 →	66.85 %	$0.92	Cost / Test: $0.92; Input / Output Cost: $2 / $6; Target-agent time: 25m53s Vals AI · Public Benefits Bench · Accuracy ↗
GPT-5.6 Sol →	66.51 %	$8.85	Cost / Test: $8.85; Input / Output Cost: $4 / $20; Target-agent time: 1h01m Vals AI · Public Benefits Bench · Accuracy ↗
Ember-1 →	66.17 %	$1.02	Cost / Test: $1.02; Input / Output Cost: $3 / $15; Target-agent time: 13m21s Vals AI · Public Benefits Bench · Accuracy ↗
Claude Sonnet 5 →	66.03 %	$1.29	Cost / Test: $1.29; Input / Output Cost: $2 / $10; Target-agent time: 21m09s Vals AI · Public Benefits Bench · Accuracy ↗
Grok 4.7 →	65.63 %	$2.25	Cost / Test: $2.25; Input / Output Cost: $2 / $6; Target-agent time: 16m57s Vals AI · Public Benefits Bench · Accuracy ↗
Gemini 3.8 Flash →	65.29 %	$1.01	Cost / Test: $1.01; Input / Output Cost: $1.5 / $7.5; Target-agent time: 3m21s Vals AI · Public Benefits Bench · Accuracy ↗
DeepSeek V4.1 Flash →	64.28 %	$0.07	Cost / Test: $0.07; Input / Output Cost: $0.3 / $1.2; Target-agent time: 9m59s Vals AI · Public Benefits Bench · Accuracy ↗
MiniMax-M3 →	64.14 %	$0.25	Cost / Test: $0.25; Input / Output Cost: $0.6 / $2.4; Target-agent time: 11m31s Vals AI · Public Benefits Bench · Accuracy ↗
Claude Haiku 5.5 →	63.19 %	$0.53	Cost / Test: $0.53; Input / Output Cost: $0.1 / $0.5; Target-agent time: 36m32s Vals AI · Public Benefits Bench · Accuracy ↗
DeepSeek V4 →	62.92 %	$0.46	Cost / Test: $0.46; Input / Output Cost: $1.32 / $3.96; Target-agent time: 15m22s Vals AI · Public Benefits Bench · Accuracy ↗
Claude Sonnet 4.6 →	62.45 %	$1.11	Cost / Test: $1.11; Input / Output Cost: $3 / $15; Target-agent time: 14m43s Vals AI · Public Benefits Bench · Accuracy ↗
GPT-5.6 Terra →	62.38 %	$1.20	Cost / Test: $1.20; Input / Output Cost: $2 / $12; Target-agent time: 19m31s Vals AI · Public Benefits Bench · Accuracy ↗
GLM 5.1 →	61.84 %	$0.34	Cost / Test: $0.34; Input / Output Cost: $1 / $3.2; Target-agent time: 9m40s Vals AI · Public Benefits Bench · Accuracy ↗
Muse Spark 1.1 →	61.3 %	$0.28	Cost / Test: $0.28; Input / Output Cost: $1.25 / $4.25; Target-agent time: 6m35s Vals AI · Public Benefits Bench · Accuracy ↗
GPT-5.6 Luna →	61.16 %	$0.27	Cost / Test: $0.27; Input / Output Cost: $0.2 / $1.2; Target-agent time: 38m40s Vals AI · Public Benefits Bench · Accuracy ↗
GPT 5.5 →	60.89 %	$3.97	Cost / Test: $3.97; Input / Output Cost: $5 / $30; Target-agent time: 39m37s Vals AI · Public Benefits Bench · Accuracy ↗
Gemini 3.5 Flash →	59.47 %	$1.11	Cost / Test: $1.11; Input / Output Cost: $1.5 / $9; Target-agent time: 5m39s Vals AI · Public Benefits Bench · Accuracy ↗
Inkling Small →	59.41 %	$0.10	Cost / Test: $0.10; Input / Output Cost: $0.3 / $1.2; Target-agent time: 8m41s Vals AI · Public Benefits Bench · Accuracy ↗
GPT-6.1 Sol →	59.34 %	$0.69	Cost / Test: $0.69; Input / Output Cost: $2 / $10; Target-agent time: 50m24s Vals AI · Public Benefits Bench · Accuracy ↗
Inkling →	58.46 %	$0.37	Cost / Test: $0.37; Input / Output Cost: $1 / $4.05; Target-agent time: 13m31s Vals AI · Public Benefits Bench · Accuracy ↗
Step 5 Preview →	58.12 %	$0.50	Cost / Test: $0.50; Input / Output Cost: $1 / $2.7; Target-agent time: 27m19s Vals AI · Public Benefits Bench · Accuracy ↗
GPT-6 Luna →	57.65 %	$0.22	Cost / Test: $0.22; Input / Output Cost: $0.1 / $0.5; Target-agent time: 52m59s Vals AI · Public Benefits Bench · Accuracy ↗
Kimi K2.6 →	56.63 %	$0.91	Cost / Test: $0.91; Input / Output Cost: $0.95 / $4; Target-agent time: 9m55s Vals AI · Public Benefits Bench · Accuracy ↗
GPT-6 Sol →	56.63 %	$2.49	Cost / Test: $2.49; Input / Output Cost: $2 / $10; Target-agent time: 31m02s Vals AI · Public Benefits Bench · Accuracy ↗
Gemini 3.6 Flash →	56.56 %	$0.39	Cost / Test: $0.39; Input / Output Cost: $1.5 / $7.5; Target-agent time: 2m02s Vals AI · Public Benefits Bench · Accuracy ↗
Claude Haiku 4.5 (Nonthinking) →	54.26 %	$0.22	Cost / Test: $0.22; Input / Output Cost: $1 / $5; Target-agent time: 2m58s Vals AI · Public Benefits Bench · Accuracy ↗
Gemini 3.1 Pro Preview (02/26) →	53.79 %	$0.38	Cost / Test: $0.38; Input / Output Cost: $2 / $12; Target-agent time: 3m06s Vals AI · Public Benefits Bench · Accuracy ↗
Gemini 3.5 Flash Lite →	51.83 %	$0.10	Cost / Test: $0.10; Input / Output Cost: $0.3 / $2.5; Target-agent time: 69.26s Vals AI · Public Benefits Bench · Accuracy ↗
Grok 4.3 →	51.69 %	$0.26	Cost / Test: $0.26; Input / Output Cost: $1.25 / $2.5; Target-agent time: 3m25s Vals AI · Public Benefits Bench · Accuracy ↗
Ling 3.0 Flash →	50.74 %	$0.05	Cost / Test: $0.05; Input / Output Cost: $0.075 / $0.22; Target-agent time: 4m35s Vals AI · Public Benefits Bench · Accuracy ↗
Mercury 2.5 →	45.26 %	$0.07	Cost / Test: $0.07; Input / Output Cost: $0.2 / $0.75; Target-agent time: 68.33s Vals AI · Public Benefits Bench · Accuracy ↗
Grok 4.1 Fast (Reasoning) →	44.79 %	$0.03	Cost / Test: $0.03; Input / Output Cost: $0.2 / $0.5; Target-agent time: 116.82s Vals AI · Public Benefits Bench · Accuracy ↗

What it measures

Evaluates whether AI models can accurately determine household SNAP eligibility and benefit amounts across complex state/federal rules.

Reading the measurement

Accuracy is only part of the picture. Inspect unsupported answers, abstentions, language coverage and the evidence available to the model. A confident response can still be wrong, and a broad average can hide a specialist gap.

Published evaluation

The linked publication documents the task and scoring procedure. Model results retain the source’s evaluation setting, native score units and available uncertainty.

Where this fits in an evaluation suite

Read accuracy together with error and abstention rates. Domain averages can hide language gaps, specialist failures, and poorly calibrated confidence.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Record the source publication, benchmark version, scoring units, denominator, tool access and attempt budget. Preserve uncertainty and distinguish developer-reported results from independent evaluations.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.

PRIMARY EVIDENCE & FURTHER READING

Public Benefits Bench — evaluation methodology and results ↗
Vals AI · 2026
Public Benefits Bench v1 — evaluation methodology and results ↗
Vals AI · 2026

Reviewed 2026-10-26. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.

Continue exploring

All knowledge & truth benchmarks →

Sources

Score vs. cost per test

What it measures

Reading the measurement

Published evaluation

Where this fits in an evaluation suite

A reproducible comparison

Continue exploring

Harvey LAB-AA v1.1

AA-Omniscience v1.1

HealthBench

MMLU-Pro