RLTasksby ulam.ai
Models / Google

Google / MODEL EVIDENCE

Gemini Pro 1.5

5 benchmarks · 5 observations · 4 sources

Scores at a glance

5 benchmarks · large dot: Gemini Pro 1.5; small dots: closest models where reported

HumanEval+ ↗61%
1.889

Gemini Pro 1.5 · HumanEval+ · Pass@1 · EvalPlus · official result data ↗

HumanEval ↗68.3%
2.496.3

Gemini Pro 1.5 · HumanEval · Pass@1 · EvalPlus · official result data ↗

VisualWebArena ↗11.98%
6.5919.78

Gemini-Pro-1.5 · VisualWebArena · Overall task success · updated model comparison ↗

RealWorldQA ↗67.5%
49.868.7

Gemini Pro 1.5 · RealWorldQA · accuracy · xAI · Grok-1.5 Vision release · April 12, 2024 ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

There are not enough shared scored settings to identify close comparisons reliably.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks — published result table ↗

    Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang et al. · 2024

    All 10 authors

    Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, Daniel Fried

  2. One Thousand and One Pairs: A "novel" challenge for long-context language models — published result table ↗

    Marzena Karpinska, Katherine Thai, Kyle Lo, Tanya Goyal, Mohit Iyyer · 2024

  3. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation — published result table ↗

    Liu, Jiawei, Xia, Chunqiu Steven, Wang, Yuyao, Zhang, Lingming · 2023

  4. Grok-1.5 Vision Preview ↗

    xAI · 2024

Names used in source tables

Gemini Pro 1.5 · Gemini-Pro-1.5