RLTasksby ulam.ai
Models / OpenAI

OpenAI / MODEL EVIDENCE

GPT-4Trb

1 benchmarks · 16 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: GPT-4Trb; small dots: closest models where reported

AppWorld ↗26.8%
7.148.8

GPT4Trb · AppWorld · Test-N · Task goal completion ↗

15 more reported results

32.7% · GPT4Trb · AppWorld · Test-N · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

25.6% · GPT4Trb · AppWorld · Test-N · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

30.4% · GPT4Trb · AppWorld · Test-N · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

12.5% · GPT4Trb · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

16.1% · GPT4Trb · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

19.6% · GPT4Trb · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

21.4% · GPT4Trb · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

17.5% · GPT4Trb · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

11% · GPT4Trb · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

12.5% · GPT4Trb · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

14.6% · GPT4Trb · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

5.8% · GPT4Trb · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

3.6% · GPT4Trb · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

7.2% · GPT4Trb · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

9.3% · GPT4Trb · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

33.3 percentile-point gap

1 of 1 shared benchmark comparisons

33.3 percentile-point gap

1 of 1 shared benchmark comparisons

DeepSeek →DeepSeek
66.7 percentile-point gap

1 of 1 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents — published result table ↗

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li et al. · 2024

    All 9 authors

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, Niranjan Balasubramanian

Names used in source tables

GPT4Trb