RLTasks by ulam.ai

Models / Reported model

Reported model / MODEL EVIDENCE

deberta-v3-large-race

1 benchmarks · 62 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: deberta-v3-large-race; small dots: closest models where reported

BBQ: Bias Benchmark for QA ↗62.7522%

deberta-v3-large-race · BBQ · Overall answer accuracy ↗

71 more reported results

29.6121% · deberta-v3-large-race · BBQ · All released categories · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

95.8923% · deberta-v3-large-race · BBQ · All released categories · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

64.2391% · deberta-v3-large-race · BBQ · Age · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

62.1465% · deberta-v3-large-race · BBQ · Disability status · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

66.0061% · deberta-v3-large-race · BBQ · Gender identity · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

64.1234% · deberta-v3-large-race · BBQ · Nationality · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

57.2335% · deberta-v3-large-race · BBQ · Physical appearance · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

62.5% · deberta-v3-large-race · BBQ · Race ethnicity · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

66.0833% · deberta-v3-large-race · BBQ · Religion · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

66.6521% · deberta-v3-large-race · BBQ · SES · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

60.1852% · deberta-v3-large-race · BBQ · Sexual orientation · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

31.413% · deberta-v3-large-race · BBQ · Age · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

30.6522 bias points · deberta-v3-large-race · BBQ · Age · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

97.0652% · deberta-v3-large-race · BBQ · Age · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

2.3849 bias points · deberta-v3-large-race · BBQ · Age · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

25.4499% · deberta-v3-large-race · BBQ · Disability status · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

38.3033 bias points · deberta-v3-large-race · BBQ · Disability status · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

98.8432% · deberta-v3-large-race · BBQ · Disability status · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.6905 bias points · deberta-v3-large-race · BBQ · Disability status · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

44.2073% · deberta-v3-large-race · BBQ · Gender identity · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

32.622 bias points · deberta-v3-large-race · BBQ · Gender identity · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

87.8049% · deberta-v3-large-race · BBQ · Gender identity · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0 bias points · deberta-v3-large-race · BBQ · Gender identity · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

26.48% · deberta-v3-large-race · BBQ · Gender identity (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

33.04 bias points · deberta-v3-large-race · BBQ · Gender identity (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

98.6% · deberta-v3-large-race · BBQ · Gender identity (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.4462 bias points · deberta-v3-large-race · BBQ · Gender identity (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

31.6883% · deberta-v3-large-race · BBQ · Nationality · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

20.3896 bias points · deberta-v3-large-race · BBQ · Nationality · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

96.5584% · deberta-v3-large-race · BBQ · Nationality · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.8767 bias points · deberta-v3-large-race · BBQ · Nationality · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

25.7614% · deberta-v3-large-race · BBQ · Physical appearance · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

39.9746 bias points · deberta-v3-large-race · BBQ · Physical appearance · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

88.7056% · deberta-v3-large-race · BBQ · Physical appearance · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-3.2342 bias points · deberta-v3-large-race · BBQ · Physical appearance · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

25.4255% · deberta-v3-large-race · BBQ · Race ethnicity · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

24.1489 bias points · deberta-v3-large-race · BBQ · Race ethnicity · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

99.5745% · deberta-v3-large-race · BBQ · Race ethnicity · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.4274 bias points · deberta-v3-large-race · BBQ · Race ethnicity · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

19.4% · deberta-v3-large-race · BBQ · Race ethnicity (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

4.76 bias points · deberta-v3-large-race · BBQ · Race ethnicity (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

97.8% · deberta-v3-large-race · BBQ · Race ethnicity (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0 bias points · deberta-v3-large-race · BBQ · Race ethnicity (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

36.25% · deberta-v3-large-race · BBQ · Race x SES · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

26.7262 bias points · deberta-v3-large-race · BBQ · Race x SES · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

96.7262% · deberta-v3-large-race · BBQ · Race x SES · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.1692 bias points · deberta-v3-large-race · BBQ · Race x SES · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

32.6154% · deberta-v3-large-race · BBQ · Race x SES (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

10.359 bias points · deberta-v3-large-race · BBQ · Race x SES (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

98.0769% · deberta-v3-large-race · BBQ · Race x SES (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-0.1045 bias points · deberta-v3-large-race · BBQ · Race x SES (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

37.6316% · deberta-v3-large-race · BBQ · Race x gender · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

12.4561 bias points · deberta-v3-large-race · BBQ · Race x gender · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

94.8246% · deberta-v3-large-race · BBQ · Race x gender · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.3636 bias points · deberta-v3-large-race · BBQ · Race x gender · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

22.8947% · deberta-v3-large-race · BBQ · Race x gender (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.9649 bias points · deberta-v3-large-race · BBQ · Race x gender (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

92.5965% · deberta-v3-large-race · BBQ · Race x gender (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-0.2739 bias points · deberta-v3-large-race · BBQ · Race x gender (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

42.1667% · deberta-v3-large-race · BBQ · Religion · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

20.1667 bias points · deberta-v3-large-race · BBQ · Religion · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

90% · deberta-v3-large-race · BBQ · Religion · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.7123 bias points · deberta-v3-large-race · BBQ · Religion · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

36.9755% · deberta-v3-large-race · BBQ · SES · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

29.7494 bias points · deberta-v3-large-race · BBQ · SES · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

96.3287% · deberta-v3-large-race · BBQ · SES · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

2.9003 bias points · deberta-v3-large-race · BBQ · SES · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

21.7593% · deberta-v3-large-race · BBQ · Sexual orientation · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

6.4815 bias points · deberta-v3-large-race · BBQ · Sexual orientation · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

98.6111% · deberta-v3-large-race · BBQ · Sexual orientation · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-0.232 bias points · deberta-v3-large-race · BBQ · Sexual orientation · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

deberta-v3-base-race →

33.3 percentile-point gap

1 of 1 shared benchmark comparisons

roberta-base-race →

33.3 percentile-point gap

1 of 1 shared benchmark comparisons

roberta-large-race →

66.7 percentile-point gap

1 of 1 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

BBQ: A Hand-Built Bias Benchmark for Question Answering — released RoBERTa and DeBERTa predictions ↗
Parrish, Alicia, Chen, Angelica, Nangia, Nikita, Padmakumar, Vishakh, Phang, Jason, Thompson, Jana et al. · 2021
All 8 authors
Parrish, Alicia, Chen, Angelica, Nangia, Nikita, Padmakumar, Vishakh, Phang, Jason, Thompson, Jana, Htut, Phu Mon, Bowman, Samuel R.

Names used in source tables

deberta-v3-large-race