MODEL EVIDENCE / SOURCE-REPORTED RESULTS
Choose models for the work.
Choose the capabilities and benchmarks that matter to you. Select several, or leave both on All to explore every result.
Values retain the model card’s original formatting and score units. HLE is without tools; the final three visual evaluations use tools. The dagger marks source qualifications: inspect the original report before treating those results as matched.
11 measurements · 6 models · source order retained
| Benchmark / conditions | DS-V4.1-Flash-1017 | DS-V4.1-Flash | K3 | GLM-5.3 | GPT-5.6 Sol | Opus 5 |
|---|---|---|---|---|---|---|
| HLE w/o tools ↗ | 42.6 (44.0†) | 36.8 (39.1†) | 43.5 | 42.0† | 44.5 | 56.3 |
| Terminal-Bench 4.0 ↗ | 42.3 | 31.2 | 12.6 | 37.9 | 39.9 | 51.8 |
| DeepSWE v1.1 ↗ | 77.0 | 74.2 | 67.5 | 66.9 | 73.0 | 74.0 |
| ProgramBench ↗ | 29.1 | 20.3 | 17.5 | 19.0 | 23.0 | 37.0 |
| NL2Repo-Bench ↗ | 68.2 | 65.4 | 58.0 | 58.0 | 56.8 | 75.3 |
| Automation-Bench v1.0.6 ↗ | 60.5 | 54.8 | 46.7 | 48.8 | 45.8 | 50.3 |
| Agents’ Last Exam ↗ | 35.4 | 31.8 | 27.6 | 28.5 | 26.7 | 28.6 |
| Apex Bench v1.1 ↗ | 72.1 | 64.6 | 47.8 | 56.6 | 54.3 | 65.8 |
| Chartography (w/tools) ↗ | 83.0 | 78.9 | 68.1 | — | 79.9 | 84.0 |
| BabyVision (w/tools) ↗ | 92.5 | 89.6 | 85.7 | — | 88.9 | 94.1 |
| ZeroBench-main (w/tools) ↗ | 51.0 | 49.0 | 41.0 | — | 53.0 | 52.0 |
PRIMARY REPORT / REVIEWED 2026-10-25
DeepSeek V4.1 Flash 1017 model card and evaluation footnotes ↗These are source-reported results, transcribed by RLTasks. A report can combine its own experiments with outside measurements. Different tools, effort settings and grading can produce different values for the same model and benchmark.
What else belongs in a model comparison?
Find relevant benchmarks →Quality & reliability
Use task success, proof validity, instruction compliance, factual error, calibration, and failure severity. Keep their original units.
Cost per successful task
Include input, output, reasoning, cache, and tool costs plus retries. Token price alone does not capture total work.
Latency & throughput
Separate time to first token, output rate, end-to-end task time, and tail latency. Pin provider, region, concurrency, and workload.
Context & modalities
Test effective retrieval and synthesis at realistic lengths. Label text, image, document, and tool inputs separately.
Deployment & access
Record model revision, provider, reasoning effort, context limits, rate limits, weight availability, and license constraints.
Post-training & robustness
Match the exact base checkpoint and inference budget. Check transfer, regressions, contamination, seed variation, and held-out tasks.
Results are dated source snapshots. Benchmark leaderboards collect the available reports without treating different evaluation settings as one matched experiment. Broad-market prices and provider speed are not yet ingested here. Explore the Artificial Analysis measurements and use this checklist to keep quality, economics, and deployment constraints distinct.