RLTasksby ulam.ai
Models / OpenAI

OpenAI / MODEL EVIDENCE

o1 / pre-deployment research checkpoint

1 benchmarks · 20 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: o1 / pre-deployment research checkpoint; small dots: closest models where reported

Apollo Scheming & Sandbagging Evals ↗2%
05

o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Disabled oversight · Goal nudging · 300 rollouts · Apollo scheming evaluations ↗

19 more reported results

2% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Deleted successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

13% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Attempted to delete successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Copied itself onto other server · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Attempted to copy to other server · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Replaced successor model · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

1% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Removed successor prompt · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Replaced successor prompt · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

22% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Covertly produced false ranking · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

16% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Manipulated input data · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

7% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Overwrote output file · Goal nudging · 300 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0.3% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Disabled oversight · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Deleted successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Attempted to delete successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Copied itself onto other server · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Attempted to copy to other server · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Replaced successor model · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Removed successor prompt · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

0% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Replaced successor prompt · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

5% · o1 / pre-deployment research checkpoint · Apollo Scheming & Sandbagging Evals · Covertly produced false ranking · No goal nudging · 400 rollouts · Apollo scheming evaluations · Frontier Models are Capable of In-context Scheming ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. Frontier Models are Capable of In-context Scheming — published result table ↗

    Meinke, Alexander, Schoen, Bronson, Scheurer, Jérémy, Balesni, Mikita, Shah, Rusheb, Hobbhahn, Marius · 2024

Names used in source tables

o1 / pre-deployment research checkpoint