OpenAI / MODEL EVIDENCE

GPT-J-6B

11 benchmarks · 15 observations · 6 sources · released 2021-08-05

Scores at a glance

11 benchmarks · large dot: GPT-J-6B; small dots: closest models where reported

65.879.4

GPT-J-6B · PIQA · Score · % ↗

1.889

GPT-J-6B · HumanEval+ · Pass@1 · EvalPlus · official result data ↗

2.496.3

GPT-J-6B · HumanEval · Pass@1 · EvalPlus · official result data ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

There are not enough shared scored settings to identify close comparisons reliably.

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. XGen-7B Technical Report ↗

    Nijkamp, Erik, Xie, Tian, Hayashi, Hiroaki, Pang, Bo, Xia, Congying, Xing, Chen et al. · 2023

    All 25 authors

    Nijkamp, Erik, Xie, Tian, Hayashi, Hiroaki, Pang, Bo, Xia, Congying, Xing, Chen, Vig, Jesse, Yavuz, Semih, Laban, Philippe, Krause, Ben, Purushwalkam, Senthil, Niu, Tong, Kryściński, Wojciech, Murakhovs'ka, Lidiya, Choubey, Prafulla Kumar, Fabbri, Alex, Liu, Ye, Meng, Rui, Tu, Lifu, Bhat, Meghana, Wu, Chien-Sheng, Savarese, Silvio, Zhou, Yingbo, Joty, Shafiq, Xiong, Caiming

  2. Holistic Evaluation of Language Models (HELM) ↗

    Stanford Center for Research on Foundation Models

  3. Holistic Evaluation of Language Models (HELM) ↗

    Stanford Center for Research on Foundation Models

  4. Holistic Evaluation of Language Models (HELM) ↗

    Stanford Center for Research on Foundation Models

  5. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation — published result table ↗

    Liu, Jiawei, Xia, Chunqiu Steven, Wang, Yuyao, Zhang, Lingming · 2023

  6. TruthfulQA: Measuring How Models Mimic Human Falsehoods — published result table ↗

    Lin, Stephanie, Hilton, Jacob, Evans, Owain · 2021

Names used in source tables

GPT-J (6B) · GPT-J 6B · GPT-J-6B