8% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Deleted successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

9% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Attempted to delete successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

7% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Copied itself onto other server · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

8% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Attempted to copy to other server · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

6% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Replaced successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Removed successor prompt · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Replaced successor prompt · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Covertly produced false ranking · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Manipulated input data · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Overwrote output file · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Disabled oversight · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1.3% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Deleted successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1.8% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Attempted to delete successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1.3% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Copied itself onto other server · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1.3% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Attempted to copy to other server · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1.3% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Replaced successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Removed successor prompt · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Replaced successor prompt · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · Gemini 1.5 Pro 002 · Apollo Scheming & Sandbagging Evals · Covertly produced false ranking · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

Gemini 2.0 Flash (001) →Google

7.5 percentile-point gap

11 of 11 shared benchmark comparisons

Llama 4 Scout →Meta

8.2 percentile-point gap

11 of 11 shared benchmark comparisons

Grok 4.1 Fast Non-Reasoning →xAI

9.1 percentile-point gap

10 of 11 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

Aider refactoring benchmark ↗
Paul Gauthier, Aider contributors
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models — published result table ↗
Deitke, Matt, Clark, Christopher, Lee, Sangho, Tripathi, Rohun, Yang, Yue, Park, Jae Sung et al. · 2024
All 50 authors
Deitke, Matt, Clark, Christopher, Lee, Sangho, Tripathi, Rohun, Yang, Yue, Park, Jae Sung, Salehi, Mohammadreza, Muennighoff, Niklas, Lo, Kyle, Soldaini, Luca, Lu, Jiasen, Anderson, Taira, Bransom, Erin, Ehsani, Kiana, Ngo, Huong, Chen, YenSung, Patel, Ajay, Yatskar, Mark, Callison-Burch, Chris, Head, Andrew, Hendrix, Rose, Bastani, Favyen, VanderBilt, Eli, Lambert, Nathan, Chou, Yvonne, Chheda, Arnavi, Sparks, Jenna, Skjonsberg, Sam, Schmitz, Michael, Sarnat, Aaron, Bischoff, Byron, Walsh, Pete, Newell, Chris, Wolters, Piper, Gupta, Tanmay, Zeng, Kuo-Hao, Borchardt, Jon, Groeneveld, Dirk, Nam, Crystal, Lebrecht, Sophie, Wittlif, Caitlin, Schoenick, Carissa, Michel, Oscar, Krishna, Ranjay, Weihs, Luca, Smith, Noah A., Hajishirzi, Hannaneh, Girshick, Ross, Farhadi, Ali, Kembhavi, Aniruddha
Frontier Models are Capable of In-context Scheming — published result table ↗
Meinke, Alexander, Schoen, Bronson, Scheurer, Jérémy, Balesni, Mikita, Shah, Rusheb, Hobbhahn, Marius · 2024
Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation — published result table ↗
Liu, Jiawei, Xia, Chunqiu Steven, Wang, Yuyao, Zhang, Lingming · 2023
AIME — evaluation methodology and results ↗
Vals AI · 2026
CorpFin — evaluation methodology and results ↗
Vals AI · 2026
GPQA Diamond — evaluation methodology and results ↗
Vals AI · 2026
LiveCodeBench — evaluation methodology and results ↗
Vals AI · 2026
LegalBench — evaluation methodology and results ↗
Vals AI · 2026
MATH 500 — evaluation methodology and results ↗
Vals AI · 2026
MedQA — evaluation methodology and results ↗
Vals AI · 2026
MGSM — evaluation methodology and results ↗
Vals AI · 2026
MMLU Pro — evaluation methodology and results ↗
Vals AI · 2026
MMMU Pro — evaluation methodology and results ↗
Vals AI · 2026
MortgageTax — evaluation methodology and results ↗
Vals AI · 2026
TaxEval — evaluation methodology and results ↗
Vals AI · 2026

Names used in source tables

Gemini 1.5 Pro (002) · Gemini 1.5 Pro 002 · Gemini-1.5-Pro-002 · gemini-1.5-pro-002