Models / Reported model

Reported model / MODEL EVIDENCE

roberta-base-race

1 benchmarks · 62 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: roberta-base-race; small dots: closest models where reported

BBQ: Bias Benchmark for QA ↗61.4526%

roberta-base-race · BBQ · Overall answer accuracy ↗

71 more reported results

67.4807% · roberta-base-race · BBQ · All released categories · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

55.4244% · roberta-base-race · BBQ · All released categories · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

45.7609% · roberta-base-race · BBQ · Age · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

50.964% · roberta-base-race · BBQ · Disability status · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

56.5549% · roberta-base-race · BBQ · Gender identity · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

61.5909% · roberta-base-race · BBQ · Nationality · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

40.4822% · roberta-base-race · BBQ · Physical appearance · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

61.4362% · roberta-base-race · BBQ · Race ethnicity · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

59.5% · roberta-base-race · BBQ · Religion · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

67.6136% · roberta-base-race · BBQ · SES · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

52.1991% · roberta-base-race · BBQ · Sexual orientation · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

28.4783% · roberta-base-race · BBQ · Age · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

6.3043 bias points · roberta-base-race · BBQ · Age · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

63.0435% · roberta-base-race · BBQ · Age · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-3.0048 bias points · roberta-base-race · BBQ · Age · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

50.5141% · roberta-base-race · BBQ · Disability status · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

9.8972 bias points · roberta-base-race · BBQ · Disability status · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

51.4139% · roberta-base-race · BBQ · Disability status · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

5.3604 bias points · roberta-base-race · BBQ · Disability status · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

67.0732% · roberta-base-race · BBQ · Gender identity · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.6098 bias points · roberta-base-race · BBQ · Gender identity · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

46.0366% · roberta-base-race · BBQ · Gender identity · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

20.3883 bias points · roberta-base-race · BBQ · Gender identity · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

70.12% · roberta-base-race · BBQ · Gender identity (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

4.36 bias points · roberta-base-race · BBQ · Gender identity (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

59.16% · roberta-base-race · BBQ · Gender identity (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

5.2573 bias points · roberta-base-race · BBQ · Gender identity (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

73.1169% · roberta-base-race · BBQ · Nationality · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

2.2078 bias points · roberta-base-race · BBQ · Nationality · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

50.0649% · roberta-base-race · BBQ · Nationality · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-0.0913 bias points · roberta-base-race · BBQ · Nationality · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

36.2944% · roberta-base-race · BBQ · Physical appearance · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

17.0051 bias points · roberta-base-race · BBQ · Physical appearance · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

44.6701% · roberta-base-race · BBQ · Physical appearance · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

18.5053 bias points · roberta-base-race · BBQ · Physical appearance · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

69.1489% · roberta-base-race · BBQ · Race ethnicity · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.9149 bias points · roberta-base-race · BBQ · Race ethnicity · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

53.7234% · roberta-base-race · BBQ · Race ethnicity · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.2809 bias points · roberta-base-race · BBQ · Race ethnicity · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

73.24% · roberta-base-race · BBQ · Race ethnicity (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-0.04 bias points · roberta-base-race · BBQ · Race ethnicity (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

59.12% · roberta-base-race · BBQ · Race ethnicity (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.4233 bias points · roberta-base-race · BBQ · Race ethnicity (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

70.4167% · roberta-base-race · BBQ · Race x SES · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-0.0595 bias points · roberta-base-race · BBQ · Race x SES · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

39.7024% · roberta-base-race · BBQ · Race x SES · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-4.2945 bias points · roberta-base-race · BBQ · Race x SES · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

54.0256% · roberta-base-race · BBQ · Race x SES (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-5.6154 bias points · roberta-base-race · BBQ · Race x SES (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

48.4359% · roberta-base-race · BBQ · Race x SES (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-7.8485 bias points · roberta-base-race · BBQ · Race x SES (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

70.614% · roberta-base-race · BBQ · Race x gender · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-1.3158 bias points · roberta-base-race · BBQ · Race x gender · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

61.9298% · roberta-base-race · BBQ · Race x gender · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-3.8034 bias points · roberta-base-race · BBQ · Race x gender · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

83.2281% · roberta-base-race · BBQ · Race x gender (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.6491 bias points · roberta-base-race · BBQ · Race x gender (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

60.3158% · roberta-base-race · BBQ · Race x gender (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-2.4857 bias points · roberta-base-race · BBQ · Race x gender (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

73.6667% · roberta-base-race · BBQ · Religion · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-1 bias points · roberta-base-race · BBQ · Religion · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

45.3333% · roberta-base-race · BBQ · Religion · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

5.2326 bias points · roberta-base-race · BBQ · Religion · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

76.2238% · roberta-base-race · BBQ · SES · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

4.3706 bias points · roberta-base-race · BBQ · SES · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

59.0035% · roberta-base-race · BBQ · SES · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

7.0074 bias points · roberta-base-race · BBQ · SES · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

56.713% · roberta-base-race · BBQ · Sexual orientation · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.2315 bias points · roberta-base-race · BBQ · Sexual orientation · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

47.6852% · roberta-base-race · BBQ · Sexual orientation · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

6.5217 bias points · roberta-base-race · BBQ · Sexual orientation · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

deberta-v3-large-race →

33.3 percentile-point gap

1 of 1 shared benchmark comparisons

deberta-v3-base-race →

66.7 percentile-point gap

1 of 1 shared benchmark comparisons

roberta-large-race →

100.0 percentile-point gap

1 of 1 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

BBQ: A Hand-Built Bias Benchmark for Question Answering — released RoBERTa and DeBERTa predictions ↗
Parrish, Alicia, Chen, Angelica, Nangia, Nikita, Padmakumar, Vishakh, Phang, Jason, Thompson, Jana et al. · 2021
All 8 authors
Parrish, Alicia, Chen, Angelica, Nangia, Nikita, Padmakumar, Vishakh, Phang, Jason, Thompson, Jana, Htut, Phu Mon, Bowman, Samuel R.

Names used in source tables

roberta-base-race