Claude 3 Sonnet

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

1. WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild — published result table ↗

   Lin, Bill Yuchen, Deng, Yuntian, Chandu, Khyathi, Brahman, Faeze, Ravichander, Abhilasha, Pyatkin, Valentina et al. · 2024

   All 9 authors

   Lin, Bill Yuchen, Deng, Yuntian, Chandu, Khyathi, Brahman, Faeze, Ravichander, Abhilasha, Pyatkin, Valentina, Dziri, Nouha, Bras, Ronan Le, Choi, Yejin

2. MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures — published result table ↗

   Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain et al. · 2024

   All 8 authors

   Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, Yang You

3. $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains — published result table ↗

   Yao, Shunyu, Shinn, Noah, Razavi, Pedram, Narasimhan, Karthik · 2024

4. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents — published result table ↗

   Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, Florian Tramèr · 2024

5. BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions — published result table ↗

   Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari et al. · 2024

   All 33 authors

   Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, Simon Brunner, Chen Gong, Thong Hoang, Armel Randy Zebaze, Xiaoheng Hong, Wen-Ding Li, Jean Kaddour, Ming Xu, Zhihan Zhang, Prateek Yadav, Naman Jain, Alex Gu, Zhoujun Cheng, Jiawei Liu, Qian Liu, Zijian Wang, Binyuan Hui, Niklas Muennighoff, David Lo, Daniel Fried, Xiaoning Du, Harm de Vries, Leandro Von Werra

6. The Art of Saying No: Contextual Noncompliance in Language Models — published result table ↗

   Brahman, Faeze, Kumar, Sachin, Balachandran, Vidhisha, Dasigi, Pradeep, Pyatkin, Valentina, Ravichander, Abhilasha et al. · 2024

   All 14 authors

   Brahman, Faeze, Kumar, Sachin, Balachandran, Vidhisha, Dasigi, Pradeep, Pyatkin, Valentina, Ravichander, Abhilasha, Wiegreffe, Sarah, Dziri, Nouha, Chandu, Khyathi, Hessel, Jack, Tsvetkov, Yulia, Smith, Noah A., Choi, Yejin, Hajishirzi, Hannaneh

7. Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries — published result table ↗

   Vodrahalli, Kiran, Ontanon, Santiago, Tripuraneni, Nilesh, Xu, Kelvin, Jain, Sanil, Shivanna, Rakesh et al. · 2024

   All 24 authors

   Vodrahalli, Kiran, Ontanon, Santiago, Tripuraneni, Nilesh, Xu, Kelvin, Jain, Sanil, Shivanna, Rakesh, Hui, Jeffrey, Dikkala, Nishanth, Kazemi, Mehran, Fatemi, Bahare, Anil, Rohan, Dyer, Ethan, Shakeri, Siamak, Vij, Roopali, Mehta, Harsh, Ramasesh, Vinay, Le, Quoc, Chi, Ed, Lu, Yifeng, Firat, Orhan, Lazaridou, Angeliki, Lespiau, Jean-Baptiste, Attaluri, Nithya, Olszewska, Kate

8. HalluLens: LLM Hallucination Benchmark — published result table ↗

   Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang et al. · 2025

   All 8 authors

   Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung

9. The Claude 3 Model Family: Opus, Sonnet, Haiku ↗

   Anthropic · 2024

10. DTBench — Conceptual Reasoning Index ↗

    Conceptual Reasoning Index

11. Holistic Evaluation of Language Models (HELM) ↗

    Stanford Center for Research on Foundation Models

12. DarkBench: Benchmarking Dark Patterns in Large Language Models — publisher leaderboard data — published result table ↗

    Esben Kran, Hieu Minh Nguyen, Akash Kundu, Sami Jawhar, Jinsuk Park, Mateusz Maria Jurewicz · 2025

13. GPQA Diamond ↗

    Epoch AI

14. MATH Level 5 ↗

    Epoch AI

15. OTIS Mock AIME 2024-2025 ↗

    Epoch AI

16. AI benchmarks & capabilities ↗

    Epoch AI

17. WeirdML ↗

    Håvard Tveit Ihle

18. Grok-1.5 Vision Preview ↗

    xAI · 2024