RLTasksby ulam.ai
Models / DeepSeek

DeepSeek / MODEL EVIDENCE

DeepSeek

1 benchmarks · 12 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: DeepSeek; small dots: closest models where reported

AppWorld ↗7.1%
7.148.8

DeepSeek · AppWorld · Test-N · Task goal completion ↗

11 more reported results

1.8% · DeepSeek · AppWorld · Test-N · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

13.1% · DeepSeek · AppWorld · Test-N · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

1.8% · DeepSeek · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

0% · DeepSeek · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

8.9% · DeepSeek · AppWorld · Test-N · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

2.9% · DeepSeek · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

0.7% · DeepSeek · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

5.8% · DeepSeek · AppWorld · Test-C · Task goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

0.7% · DeepSeek · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

0% · DeepSeek · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

2.9% · DeepSeek · AppWorld · Test-C · Scenario goal completion · AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

33.3 percentile-point gap

1 of 1 shared benchmark comparisons

66.7 percentile-point gap

1 of 1 shared benchmark comparisons

100.0 percentile-point gap

1 of 1 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents — published result table ↗

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li et al. · 2024

    All 9 authors

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, Niranjan Balasubramanian

Names used in source tables

DeepSeek