RLTasksby ulam.ai
Models / Reported model

Reported model / MODEL EVIDENCE

PaLM-2-XXS

1 benchmarks · 5 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: PaLM-2-XXS; small dots: closest models where reported

BhasaBench / IndicGenBench ↗7.2 points
041.1

PaLM-2-XXS · BhasaBench / IndicGenBench · CrossSum-In · one-shot evaluation ↗

4 more reported results

24 points · PaLM-2-XXS · BhasaBench / IndicGenBench · Flores-In · one-shot evaluation · IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages ↗

34.6 points · PaLM-2-XXS · BhasaBench / IndicGenBench · XQuAD-In · one-shot evaluation · IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages ↗

13.5 points · PaLM-2-XXS · BhasaBench / IndicGenBench · XorQA-In-Xx · one-shot evaluation · IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages ↗

36.8 points · PaLM-2-XXS · BhasaBench / IndicGenBench · XorQA-In-En · one-shot evaluation · IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

8.3 percentile-point gap

1 of 1 shared benchmark comparisons

8.3 percentile-point gap

1 of 1 shared benchmark comparisons

16.7 percentile-point gap

1 of 1 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages — published result table ↗

    Harman Singh, Nitish Gupta, Shikhar Bharadwaj, Dinesh Tewari, Partha Talukdar · 2024

Names used in source tables

PaLM-2-XXS