RLTasks by ulam.ai

Models / Reported model

Reported model / MODEL EVIDENCE

deberta-v3-base-race

1 benchmarks · 62 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: deberta-v3-base-race; small dots: closest models where reported

BBQ: Bias Benchmark for QA ↗64.818%

deberta-v3-base-race · BBQ · Overall answer accuracy ↗

71 more reported results

48.4472% · deberta-v3-base-race · BBQ · All released categories · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

81.1889% · deberta-v3-base-race · BBQ · All released categories · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

59.538% · deberta-v3-base-race · BBQ · Age · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

58.2262% · deberta-v3-base-race · BBQ · Disability status · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

66.7683% · deberta-v3-base-race · BBQ · Gender identity · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

64.2532% · deberta-v3-base-race · BBQ · Nationality · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

51.967% · deberta-v3-base-race · BBQ · Physical appearance · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

62.0213% · deberta-v3-base-race · BBQ · Race ethnicity · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

68.0833% · deberta-v3-base-race · BBQ · Religion · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

77.8118% · deberta-v3-base-race · BBQ · SES · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

61.3426% · deberta-v3-base-race · BBQ · Sexual orientation · answer accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

41.1957% · deberta-v3-base-race · BBQ · Age · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

24.6739 bias points · deberta-v3-base-race · BBQ · Age · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

77.8804% · deberta-v3-base-race · BBQ · Age · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

4.4118 bias points · deberta-v3-base-race · BBQ · Age · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

39.4602% · deberta-v3-base-race · BBQ · Disability status · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

10.6684 bias points · deberta-v3-base-race · BBQ · Disability status · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

76.9923% · deberta-v3-base-race · BBQ · Disability status · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

8.134 bias points · deberta-v3-base-race · BBQ · Disability status · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

50.6098% · deberta-v3-base-race · BBQ · Gender identity · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

17.6829 bias points · deberta-v3-base-race · BBQ · Gender identity · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

82.9268% · deberta-v3-base-race · BBQ · Gender identity · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.4388 bias points · deberta-v3-base-race · BBQ · Gender identity · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

57.56% · deberta-v3-base-race · BBQ · Gender identity (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

12.68 bias points · deberta-v3-base-race · BBQ · Gender identity (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

85.36% · deberta-v3-base-race · BBQ · Gender identity (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

4.7491 bias points · deberta-v3-base-race · BBQ · Gender identity (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

46.8182% · deberta-v3-base-race · BBQ · Nationality · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

18.3766 bias points · deberta-v3-base-race · BBQ · Nationality · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

81.6883% · deberta-v3-base-race · BBQ · Nationality · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

5.6957 bias points · deberta-v3-base-race · BBQ · Nationality · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

28.2995% · deberta-v3-base-race · BBQ · Physical appearance · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

41.7513 bias points · deberta-v3-base-race · BBQ · Physical appearance · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

75.6345% · deberta-v3-base-race · BBQ · Physical appearance · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

5.6769 bias points · deberta-v3-base-race · BBQ · Physical appearance · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

48.9362% · deberta-v3-base-race · BBQ · Race ethnicity · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

4.6809 bias points · deberta-v3-base-race · BBQ · Race ethnicity · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

75.1064% · deberta-v3-base-race · BBQ · Race ethnicity · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.6386 bias points · deberta-v3-base-race · BBQ · Race ethnicity · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

44.24% · deberta-v3-base-race · BBQ · Race ethnicity (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.16 bias points · deberta-v3-base-race · BBQ · Race ethnicity (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

80.16% · deberta-v3-base-race · BBQ · Race ethnicity (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-0.2868 bias points · deberta-v3-base-race · BBQ · Race ethnicity (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

52.4405% · deberta-v3-base-race · BBQ · Race x SES · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

2.9167 bias points · deberta-v3-base-race · BBQ · Race x SES · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

75.9524% · deberta-v3-base-race · BBQ · Race x SES · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.0463 bias points · deberta-v3-base-race · BBQ · Race x SES · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

26.7179% · deberta-v3-base-race · BBQ · Race x SES (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-0.9744 bias points · deberta-v3-base-race · BBQ · Race x SES (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

75.7179% · deberta-v3-base-race · BBQ · Race x SES (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-2.439 bias points · deberta-v3-base-race · BBQ · Race x SES (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

59.1228% · deberta-v3-base-race · BBQ · Race x gender · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-3.9474 bias points · deberta-v3-base-race · BBQ · Race x gender · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

84.1667% · deberta-v3-base-race · BBQ · Race x gender · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

0.8313 bias points · deberta-v3-base-race · BBQ · Race x gender · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

51.1228% · deberta-v3-base-race · BBQ · Race x gender (names) · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-3.614 bias points · deberta-v3-base-race · BBQ · Race x gender (names) · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

82.4561% · deberta-v3-base-race · BBQ · Race x gender (names) · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

2.1507 bias points · deberta-v3-base-race · BBQ · Race x gender (names) · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

57.6667% · deberta-v3-base-race · BBQ · Religion · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

13 bias points · deberta-v3-base-race · BBQ · Religion · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

78.5% · deberta-v3-base-race · BBQ · Religion · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

1.7682 bias points · deberta-v3-base-race · BBQ · Religion · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

66.2879% · deberta-v3-base-race · BBQ · SES · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

9.8193 bias points · deberta-v3-base-race · BBQ · SES · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

89.3357% · deberta-v3-base-race · BBQ · SES · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

3.8227 bias points · deberta-v3-base-race · BBQ · SES · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

41.4352% · deberta-v3-base-race · BBQ · Sexual orientation · ambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-4.3981 bias points · deberta-v3-base-race · BBQ · Sexual orientation · ambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

81.25% · deberta-v3-base-race · BBQ · Sexual orientation · disambig · Accuracy · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

-4.7619 bias points · deberta-v3-base-race · BBQ · Sexual orientation · disambig · Signed bias score · BBQ: A Hand-Built Bias Benchmark for Question Answering — released prediction aggregation ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

deberta-v3-large-race →

33.3 percentile-point gap

1 of 1 shared benchmark comparisons

roberta-large-race →

33.3 percentile-point gap

1 of 1 shared benchmark comparisons

roberta-base-race →

66.7 percentile-point gap

1 of 1 shared benchmark comparisons

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

BBQ: A Hand-Built Bias Benchmark for Question Answering — released RoBERTa and DeBERTa predictions ↗
Parrish, Alicia, Chen, Angelica, Nangia, Nikita, Padmakumar, Vishakh, Phang, Jason, Thompson, Jana et al. · 2021
All 8 authors
Parrish, Alicia, Chen, Angelica, Nangia, Nikita, Padmakumar, Vishakh, Phang, Jason, Thompson, Jana, Htut, Phu Mon, Bowman, Samuel R.

Names used in source tables

deberta-v3-base-race