RLTasksby ulam.ai
Models / Meta

Meta / MODEL EVIDENCE

Llama 3.1 405B Instruct

7 benchmarks · 33 observations · 7 sources

Scores at a glance

7 benchmarks · large dot: Llama 3.1 405B Instruct; small dots: closest models where reported

HalluLens ↗31.62%
8.7352.59

Llama-3.1-405B-Instruct · HalluLens · PreciseWikiQA correct answers · three-trial mean ↗

1 more reported result

11.48% · Llama-3.1-405B-Instruct · HalluLens · MixedEntities false acceptance · fabricated-entity acceptance · HalluLens: LLM Hallucination Benchmark ↗

JudgeBench ↗56.86%
33.1480.86

Llama-3.1-405B-Instruct · JudgeBench · Overall pairwise accuracy · Arena-Hard judge scaffold ↗

LiveCodeBench Pro ↗574 Elo
1999

Meta Llama 3.1 405B Instruct · LiveCodeBench Pro · Elo rating · human contest calibration ↗

3 more reported results

0% · Meta Llama 3.1 405B Instruct · LiveCodeBench Pro · Hard · pass@1 · difficulty breakdown · LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? ↗

0% · Meta Llama 3.1 405B Instruct · LiveCodeBench Pro · Medium · pass@1 · difficulty breakdown · LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? ↗

9.9% · Meta Llama 3.1 405B Instruct · LiveCodeBench Pro · Easy · pass@1 · difficulty breakdown · LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? ↗

Apollo Scheming & Sandbagging Evals ↗3%
05

Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Disabled oversight · Goal nudging · 300 rollouts · Apollo scheming evaluations ↗

19 more reported results

3% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Deleted successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

4% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Attempted to delete successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Copied itself onto other server · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

3% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Attempted to copy to other server · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Replaced successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

5% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Removed successor prompt · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Replaced successor prompt · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Covertly produced false ranking · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Manipulated input data · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Overwrote output file · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Disabled oversight · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Deleted successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Attempted to delete successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Copied itself onto other server · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Attempted to copy to other server · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Replaced successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Removed successor prompt · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Replaced successor prompt · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Llama 3.1 405B Instruct · Apollo Scheming & Sandbagging Evals · Covertly produced false ranking · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

There are not enough shared scored settings to identify close comparisons reliably.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. JudgeBench: A Benchmark for Evaluating LLM-based Judges — published result table ↗

    Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y. Tang, Alejandro Cuadron, Chenguang Wang et al. · 2024

    All 8 authors

    Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y. Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, Ion Stoica

  2. Commit0: Library Generation from Scratch — published result table ↗

    Zhao, Wenting, Jiang, Nan, Lee, Celine, Chiu, Justin T, Cardie, Claire, Gallé, Matthias et al. · 2024

    All 7 authors

    Zhao, Wenting, Jiang, Nan, Lee, Celine, Chiu, Justin T, Cardie, Claire, Gallé, Matthias, Rush, Alexander M

  3. Frontier Models are Capable of In-context Scheming — published result table ↗

    Meinke, Alexander, Schoen, Bronson, Scheurer, Jérémy, Balesni, Mikita, Shah, Rusheb, Hobbhahn, Marius · 2024

  4. HalluLens: LLM Hallucination Benchmark — published result table ↗

    Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang et al. · 2025

    All 8 authors

    Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung

  5. LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? — published result table ↗

    Zihan Zheng, Zerui Cheng, Zeyu Shen, Shang Zhou, Kaiyuan Liu, Hansen He et al. · 2025

    All 19 authors

    Zihan Zheng, Zerui Cheng, Zeyu Shen, Shang Zhou, Kaiyuan Liu, Hansen He, Dongruixuan Li, Stanley Wei, Hangyi Hao, Jianzhu Yao, Peiyao Sheng, Zixuan Wang, Wenhao Chai, Aleksandra Korolova, Peter Henderson, Sanjeev Arora, Pramod Viswanath, Jingbo Shang, Saining Xie

  6. Are large language models superhuman chemists? — published result table ↗

    Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Martiño Ríos-García, Benedict Emoekabu, Aswanth Krishnan et al. · 2024

    All 35 authors

    Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Martiño Ríos-García, Benedict Emoekabu, Aswanth Krishnan, Tanya Gupta, Mara Schilling-Wilhelmi, Macjonathan Okereke, Anagha Aneesh, Amir Mohammad Elahi, Mehrdad Asgari, Juliane Eberhardt, Hani M. Elbeheiry, María Victoria Gil, Maximilian Greiner, Caroline T. Holick, Christina Glaubitz, Tim Hoffmann, Abdelrahman Ibrahim, Lea C. Klepsch, Yannik Köster, Fabian Alexander Kreth, Jakob Meyer, Santiago Miret, Jan Matthias Peschel, Michael Ringleb, Nicole Roesner, Johanna Schreiber, Ulrich S. Schubert, Leanne M. Stafast, Dinga Wonanke, Michael Pieler, Philippe Schwaller, Kevin Maik Jablonka

  7. Scale Labs — evaluation methodology and results ↗

    Scale Labs / SEAL · 2026

Names used in source tables

Llama 3.1 405B Instruct · Llama-3.1-405B-Instruct · Meta Llama 3.1 405B Instruct