Claude2

2 benchmarks · 5 observations · 2 sources

Scores at a glance

2 benchmarks · large dot: Claude2; small dots: closest models where reported

FinanceBench ↗76%

Claude2 · FinanceBench · Correct-answer rate · human evaluation · 150-question evaluated sample ↗

SciBench ↗14.94%

Claude2 · SciBench · Textbook overall accuracy · prompting and tool conditions ↗

3 more reported results

13.89% · Claude2 · SciBench · Textbook overall accuracy · prompting and tool conditions · SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models ↗

15.26% · Claude2 · SciBench · Textbook overall accuracy · prompting and tool conditions · SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models ↗

14.92% · Claude2 · SciBench · Textbook overall accuracy · prompting and tool conditions · SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

GPT-4-Turbo →OpenAI

41.7 percentile-point gap

2 of 2 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models — published result table ↗
Wang, Xiaoxuan, Hu, Ziniu, Lu, Pan, Zhu, Yanqiao, Zhang, Jieyu, Subramaniam, Satyen et al. · 2023
All 10 authors
Wang, Xiaoxuan, Hu, Ziniu, Lu, Pan, Zhu, Yanqiao, Zhang, Jieyu, Subramaniam, Satyen, Loomba, Arjun R., Zhang, Shichang, Sun, Yizhou, Wang, Wei
FinanceBench: A New Benchmark for Financial Question Answering — published result table ↗
Islam, Pranab, Kannappan, Anand, Kiela, Douwe, Qian, Rebecca, Scherrer, Nino, Vidgen, Bertie · 2023

Names used in source tables

Claude2