RLTasksby ulam.ai
Models / OpenAI

OpenAI / MODEL EVIDENCE

o3

59 benchmarks · 101 observations · 63 sources · released 2025-04-16

Scores at a glance

59 benchmarks · large dot: o3; small dots: closest models where reported

MATH Level 5 ↗97.77%
3.2998.13

o3 (high) · MATH level 5 · Best score (across scorers) · % ↗

Aider coding benchmarks ↗76.9%
76.976.9

o3 · Aider coding benchmarks · Polyglot · percent correct ↗

4 more reported results

76.9% · o3 (unknown thinking) · Aider coding benchmarks · Polyglot · percent correct · Epoch AI · Aider polyglot ↗

81.3% · o3 (high) · Aider coding benchmarks · Polyglot · percent correct · Epoch AI · Aider polyglot ↗

76.9% · o3 (medium) · Aider coding benchmarks · Polyglot · percent correct · Epoch AI · Aider polyglot ↗

81.3% · o3 (high) · Aider coding benchmarks · Polyglot · percent correct · Aider code editing benchmark ↗

GSO ↗8.8%
026.5

o3 (high) · GSO-Bench · Score OPT@1 · % ↗

GDPval ↗30.8%
9.949.7

o3 (medium) · GDPval · Win Rate (%) · % ↗

METR task performance & time horizons ↗65.44%
10.185.2

o3 (medium) · METR task performance & time horizons · Average task accuracy ↗

3 more reported results

63.62% · o3 (unknown thinking) · METR task performance & time horizons · Average task accuracy · Epoch AI · METR Time Horizons ↗

119.73263 minutes · o3 (unknown thinking) · METR task performance & time horizons · 50% success task horizon · METR-Horizon-v1.1 · Epoch AI · METR Autonomy / Task-Horizon Suite · public result data ↗

91.27135 minutes · o3 (medium) · METR task performance & time horizons · 50% success task horizon · Epoch AI · METR Autonomy / Task-Horizon Suite · public result data ↗

LMCA ↗39.7%
2.7868.26

o3 (high) · LMCA · Score · % ↗

SimpleQA ↗49.4%
7.662.5

o3 · SimpleQA · Correct answers · OpenAI Simple Evals · archived July 2025 ↗

HumanEval ↗87.4%
71.599.3

o3 · HumanEval · Pass@1 · OpenAI Simple Evals · archived July 2025 ↗

EconBench ↗0.4224 error points
0.420.42

o3 · EconBench · EconBench prototype · risk-consistency error · published prototype observations ↗

CyberSecEval ↗29.44%
22.9933.51

o3 · CyberSecEval · CyberSOCEval · malware analysis · Overall accuracy · CyberSecEval 4 · CyberSOCEval defensive-analysis comparison ↗

8 more reported results

31% · o3 · CyberSecEval · CyberSOCEval · malware analysis · Easy accuracy · CyberSecEval 4 · CyberSOCEval defensive-analysis comparison · CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning ↗

26% · o3 · CyberSecEval · CyberSOCEval · malware analysis · Medium accuracy · CyberSecEval 4 · CyberSOCEval defensive-analysis comparison · CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning ↗

14% · o3 · CyberSecEval · CyberSOCEval · malware analysis · Hard accuracy · CyberSecEval 4 · CyberSOCEval defensive-analysis comparison · CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning ↗

52.87% · o3 · CyberSecEval · CyberSOCEval · threat intelligence · image input · Overall accuracy · CyberSecEval 4 · CyberSOCEval image-input comparison · CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning ↗

69% · o3 · CyberSecEval · CyberSOCEval · threat intelligence · image input · NSA reports · CyberSecEval 4 · CyberSOCEval image-input comparison · CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning ↗

55% · o3 · CyberSecEval · CyberSOCEval · threat intelligence · image input · CrowdStrike reports · CyberSecEval 4 · CyberSOCEval image-input comparison · CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning ↗

46% · o3 · CyberSecEval · CyberSOCEval · threat intelligence · image input · IC3 reports · CyberSecEval 4 · CyberSOCEval image-input comparison · CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning ↗

50% · o3 · CyberSecEval · CyberSOCEval · threat intelligence · image input · CISA reports · CyberSecEval 4 · CyberSOCEval image-input comparison · CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

GPT-5 →OpenAI
16.1 percentile-point gap

20 of 21 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. Aider code editing benchmark ↗

    Paul Gauthier, Aider contributors

  2. Aider LLM Leaderboards — Polyglot coding benchmark ↗

    Aider

  3. ARC Prize - Leaderboard ↗

    ARC Prize Foundation

  4. LiveCodeBench Benchmark | Artificial Analysis ↗

    Artificial Analysis

  5. MATH-500 Benchmark | Artificial Analysis ↗

    Artificial Analysis

  6. MMLU-Pro Benchmark | Artificial Analysis ↗

    Artificial Analysis

  7. HealthBench: Evaluating Large Language Models Towards Improved Human Health — published result table ↗

    Arora, Rahul K., Wei, Jason, Hicks, Rebecca Soskin, Bowman, Preston, Quiñonero-Candela, Joaquin, Tsimpourlas, Foivos et al. · 2025

    All 12 authors

    Arora, Rahul K., Wei, Jason, Hicks, Rebecca Soskin, Bowman, Preston, Quiñonero-Candela, Joaquin, Tsimpourlas, Foivos, Sharman, Michael, Shah, Meghan, Vallone, Andrea, Beutel, Alex, Heidecke, Johannes, Singhal, Karan

  8. SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models — published result table ↗

    Pham, Thinh, Nguyen, Nguyen, Zunjare, Pratibha, Chen, Weiyuan, Tseng, Yu-Min, Vu, Tu · 2025

  9. TextQuests: How Good are LLMs at Text-Based Video Games? — published result table ↗

    Phan, Long, Mazeika, Mantas, Zou, Andy, Hendrycks, Dan · 2025

  10. CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning — published result table ↗

    Deason, Lauren, Bali, Adam, Bejean, Ciprian, Bolocan, Diana, Crnkovich, James, Croitoru, Ioana et al. · 2025

    All 23 authors

    Deason, Lauren, Bali, Adam, Bejean, Ciprian, Bolocan, Diana, Crnkovich, James, Croitoru, Ioana, Durai, Krishna, Midler, Chase, Miron, Calin, Molnar, David, Moon, Brad, Ostarcevic, Bruno, Peltea, Alberto, Rosenberg, Matt, Sandu, Catalin, Saputkin, Arthur, Shah, Sagar, Stan, Daniel, Szocs, Ernest, Wan, Shengye, Whitman, Spencer, Krasser, Sven, Saxe, Joshua

  11. TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts — published result table ↗

    Jaffe, Oliver, Sherburn, Dane · 2026

  12. DTBench — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

  13. LMCA — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

  14. FutureSearch Evals ↗

    FutureSearch

  15. EconBench: Economic choice by language models — o3 prototype metrics ↗

    EconBench project · 2026

  16. ARC-AGI-2 ↗

    Epoch AI

  17. Chess Puzzles ↗

    Epoch AI

  18. CL-bench ↗

    Epoch AI

  19. CritPt v1 ↗

    Epoch AI

  20. FrontierMath Tier 4 (v1, old) ↗

    Epoch AI

  21. FrontierMath Tiers 1-3 (v1, old) ↗

    Epoch AI

  22. FrontierMath Tiers 1-3 (v2) ↗

    Epoch AI

  23. GDPval ↗

    Epoch AI

  24. GPQA Diamond ↗

    Epoch AI

  25. Humanity's Last Exam ↗

    Epoch AI

  26. MATH Level 5 ↗

    Epoch AI

  27. METR Time Horizons ↗

    Epoch AI

  28. Mystery Game Puzzles ↗

    Epoch AI

  29. OTIS Mock AIME 2024-2025 ↗

    Epoch AI

  30. SimpleQA Verified ↗

    Epoch AI

  31. SimpleBench ↗

    Epoch AI

  32. SWE-bench Verified ↗

    Epoch AI

  33. Capabilities & benchmarking — retained external evaluation records ↗

    Epoch AI · 2026

  34. AI benchmarks & capabilities ↗

    Epoch AI

  35. Fiction.liveBench — March 14, 2025 benchmark release ↗

    Fiction.live · 2025

  36. GeoBench — GeoGuessr model benchmark ↗

    GeoBench

  37. Writing benchmark — incorporating mandatory story elements ↗

    lechmazur

  38. GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents ↗

    Shetty, Manish, Jain, Naman, Liu, Jinjian, Kethanaboyina, Vijay, Sen, Koushik, Stoica, Ion · 2025

  39. WeirdML ↗

    Håvard Tveit Ihle

  40. EnigmaEval ↗

    Scale Labs / SEAL

  41. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  42. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  43. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

  44. Measuring AI Ability to Complete Long Software Tasks ↗

    METR

  45. Metr Autonomy Task Horizon Suite — methodology and published results ↗

    METR · 2026

  46. OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments ↗

    Xie, Tianbao, Zhang, Danyang, Chen, Jixuan, Li, Xiaochuan, Zhao, Siheng, Cao, Ruisheng et al. · 2024

    All 17 authors

    Xie, Tianbao, Zhang, Danyang, Chen, Jixuan, Li, Xiaochuan, Zhao, Siheng, Cao, Ruisheng, Hua, Toh Jing, Cheng, Zhoujun, Shin, Dongchan, Lei, Fangyu, Liu, Yitao, Xu, Yiheng, Zhou, Shuyan, Savarese, Silvio, Xiong, Caiming, Zhong, Victor, Yu, Tao

  47. Simple Evals — model benchmark results ↗

    OpenAI · 2025

  48. CadEval Dashboard — run 20250422_095709 ↗

    willpatrick · 2025

  49. AIME — evaluation methodology and results ↗

    Vals AI · 2026

  50. CorpFin — evaluation methodology and results ↗

    Vals AI · 2026

  51. GPQA Diamond — evaluation methodology and results ↗

    Vals AI · 2026

  52. LiveCodeBench — evaluation methodology and results ↗

    Vals AI · 2026

  53. LegalBench — evaluation methodology and results ↗

    Vals AI · 2026

  54. MATH 500 — evaluation methodology and results ↗

    Vals AI · 2026

  55. MedCode — evaluation methodology and results ↗

    Vals AI · 2026

  56. MedQA — evaluation methodology and results ↗

    Vals AI · 2026

  57. MedScribe — evaluation methodology and results ↗

    Vals AI · 2026

  58. MGSM — evaluation methodology and results ↗

    Vals AI · 2026

  59. MMLU Pro — evaluation methodology and results ↗

    Vals AI · 2026

  60. MMMU Pro — evaluation methodology and results ↗

    Vals AI · 2026

  61. MortgageTax — evaluation methodology and results ↗

    Vals AI · 2026

  62. SAGE — evaluation methodology and results ↗

    Vals AI · 2026

  63. TaxEval — evaluation methodology and results ↗

    Vals AI · 2026

Names used in source tables

o3 · o3 (high) · o3 (low) · o3 (medium) · o3 (unknown thinking)