WikiContradict

Sources

1. WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia — published result table ↗

   Hou, Yufang, Pascale, Alessandra, Carnerero-Cano, Javier, Tchrakian, Tigran, Marinescu, Radu, Daly, Elizabeth et al. · 2024

   All 8 authors

   Hou, Yufang, Pascale, Alessandra, Carnerero-Cano, Javier, Tchrakian, Tigran, Marinescu, Radu, Daly, Elizabeth, Padhi, Inkit, Sattigeri, Prasanna

   Dates & evaluation details

   WikiContradict · All instances · Fully correct · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · All instances · Partially correct · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · All instances · Incorrect · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · Explicit contradictions · Fully correct · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · Explicit contradictions · Partially correct · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · Explicit contradictions · Incorrect · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · Implicit contradictions · Fully correct · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · Implicit contradictions · Partially correct · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · Implicit contradictions · Incorrect · Llama-3-70B judge · template 5 · 2026-10-27

   Published model comparison, 2024. WikiContradict · Llama-3-70B judge · template 5. Share of responses assigned the named correctness label by the source Llama-3-70B judge. Fully correct means preserving the conflicting evidence, rather than choosing one side. Explicit and implicit contradictions are separate subsets. This is not the separate judge-validation F1 table and is not a ConflictBank score. Table 5; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 1 · All · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 1. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 1 · Explicit · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 1. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 1 · Implicit · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 1. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 4 · All · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 4. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 4 · Explicit · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 4. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 4 · Implicit · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 4. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 5 · All · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 5. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 5 · Explicit · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 5. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

   WikiContradict · WikiContradict HumanEval · Template 5 · Implicit · fully correct · human annotation · 2026-10-27

   Published model comparison, 2024. WikiContradict · human annotation · Template 5. Human-evaluated fully correct response rate on the 48-item WikiContradict HumanEval set or its named subset. Prompt templates change how the model is asked to handle conflicting evidence; human and automated judging remain separate measures. Table 2; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

TRACE THE SCORE · Per-model notes use published breakdowns where available.