RLTasksby ulam.ai
Models / Mistral

Mistral / MODEL EVIDENCE

Mistral-7b-inst

1 benchmarks · 18 observations · 1 sources

Scores at a glance

1 benchmarks · large dot: Mistral-7b-inst; small dots: closest models where reported

WikiContradict ↗50.6%
1.250.6

Mistral-7b-inst · WikiContradict · All instances · Fully correct · Llama-3-70B judge · template 5 ↗

17 more reported results

39.9% · Mistral-7b-inst · WikiContradict · All instances · Partially correct · Llama-3-70B judge · template 5 · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

5.9% · Mistral-7b-inst · WikiContradict · All instances · Incorrect · Llama-3-70B judge · template 5 · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

57.8% · Mistral-7b-inst · WikiContradict · Explicit contradictions · Fully correct · Llama-3-70B judge · template 5 · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

36% · Mistral-7b-inst · WikiContradict · Explicit contradictions · Partially correct · Llama-3-70B judge · template 5 · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

3.1% · Mistral-7b-inst · WikiContradict · Explicit contradictions · Incorrect · Llama-3-70B judge · template 5 · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

38% · Mistral-7b-inst · WikiContradict · Implicit contradictions · Fully correct · Llama-3-70B judge · template 5 · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

46.7% · Mistral-7b-inst · WikiContradict · Implicit contradictions · Partially correct · Llama-3-70B judge · template 5 · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

10.9% · Mistral-7b-inst · WikiContradict · Implicit contradictions · Incorrect · Llama-3-70B judge · template 5 · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

4.2% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 1 · All · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

6.7% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 1 · Explicit · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

0% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 1 · Implicit · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

2.1% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 4 · All · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

3.3% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 4 · Explicit · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

0% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 4 · Implicit · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

20.8% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 5 · All · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

26.7% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 5 · Explicit · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

11.8% · Mistral-7b-inst · WikiContradict · WikiContradict HumanEval · Template 5 · Implicit · fully correct · human annotation · WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia ↗

Closest models by score

Similar positions on shared benchmark leaderboards. Smaller gaps are closer.

How similarity is measured

Mean absolute gap in percentile positions on shared benchmark settings; lower is closer. At least 60% overlap is required, with a minimum of three when available. Independent evaluations are used when there are at least three; otherwise published report comparisons are used. Repeated model observations within one setting are averaged before ranking. Writing-style traits and composite indices are excluded. This describes similarity, not an overall quality ranking.

Filter comparisons to this model →

Sources

Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.

  1. WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia — published result table ↗

    Hou, Yufang, Pascale, Alessandra, Carnerero-Cano, Javier, Tchrakian, Tigran, Marinescu, Radu, Daly, Elizabeth et al. · 2024

    All 8 authors

    Hou, Yufang, Pascale, Alessandra, Carnerero-Cano, Javier, Tchrakian, Tigran, Marinescu, Radu, Daly, Elizabeth, Padhi, Inkit, Sattigeri, Prasanna

Names used in source tables

Mistral-7b-inst