DTBench ↗53.3%
Mistral Medium · DTBench · Accuracy · % ↗
Mistral / MODEL EVIDENCE
Mistral
3 benchmarks · 30 observations · 3 sources · released 2023-12-11
Scores at a glance
3 benchmarks · large dot: Mistral; small dots: closest models where reported
Mistral · CoCoNot (Contextual Non-Compliance) · No system prompt · Incomplete · compliance rate · reported model comparison ↗
11 more reported results
23.6% · Mistral · CoCoNot (Contextual Non-Compliance) · No system prompt · Unsupported · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
2.1% · Mistral · CoCoNot (Contextual Non-Compliance) · No system prompt · Indeterminate · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
28.1% · Mistral · CoCoNot (Contextual Non-Compliance) · No system prompt · Safety · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
23.2% · Mistral · CoCoNot (Contextual Non-Compliance) · No system prompt · Humanizing · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
88.4% · Mistral · CoCoNot (Contextual Non-Compliance) · No system prompt · Contrast set · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
13.8% · Mistral · CoCoNot (Contextual Non-Compliance) · With system prompt · Incomplete · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
19.1% · Mistral · CoCoNot (Contextual Non-Compliance) · With system prompt · Unsupported · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
1.4% · Mistral · CoCoNot (Contextual Non-Compliance) · With system prompt · Indeterminate · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
10.1% · Mistral · CoCoNot (Contextual Non-Compliance) · With system prompt · Safety · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
3.7% · Mistral · CoCoNot (Contextual Non-Compliance) · With system prompt · Humanizing · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
89.5% · Mistral · CoCoNot (Contextual Non-Compliance) · With system prompt · Contrast set · compliance rate · reported model comparison · The Art of Saying No: Contextual Noncompliance in Language Models ↗
ClinicBench ↗35.7%
Mistral · ClinicBench · MedQA · Accuracy · zero-shot main comparison ↗
16 more reported results
37.8% · Mistral · ClinicBench · MedMCQA · Accuracy · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
46.3% · Mistral · ClinicBench · MMLU · Accuracy · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
69.4% · Mistral · ClinicBench · PubMedQA · Accuracy · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
77.7% · Mistral · ClinicBench · Referral QA · Accuracy · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
5 F1 points · Mistral · ClinicBench · Treat Recom. · F1 · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
13.2 ROUGE-L points · Mistral · ClinicBench · MIMIC · ROUGE-L · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
7.9 ROUGE-L points · Mistral · ClinicBench · IU-Xray · ROUGE-L · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
6.1 ROUGE-L points · Mistral · ClinicBench · Hospitaliz. Summari. · ROUGE-L · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
5.3 ROUGE-L points · Mistral · ClinicBench · Patient Education · ROUGE-L · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
46.8 F1 entity-level points · Mistral · ClinicBench · BC5 · F1 entity-level · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
39.9 F1 entity-level points · Mistral · ClinicBench · NCBI · F1 entity-level · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
43.5 Micro F1 points · Mistral · ClinicBench · DDI · Micro F1 · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
44.3 Micro F1 points · Mistral · ClinicBench · GAD · Micro F1 · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
59.6 Micro F1 points · Mistral · ClinicBench · HoC · Micro F1 · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
51.2% · Mistral · ClinicBench · Pharma. QA · Accuracy · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
53% · Mistral · ClinicBench · Drug Inter. · Accuracy · zero-shot main comparison · Large Language Models in the Clinic: A Comprehensive Benchmark ↗
Closest models by score
Similar positions on shared benchmark leaderboards. Smaller gaps are closer.
There are not enough shared scored settings to identify close comparisons reliably.
Sources
Every plotted observation links to its evaluation. Reported configurations, benchmark versions and task subsets can differ; similarity describes shared scores, not training lift.
- Large Language Models in the Clinic: A Comprehensive Benchmark — published result table ↗
All 19 authors
Liu, Fenglin, Li, Zheng, Zhou, Hongjian, Yin, Qingyu, Yang, Jingfeng, Tang, Xianfeng, Luo, Chen, Zeng, Ming, Jiang, Haoming, Gao, Yifan, Nigam, Priyanka, Nag, Sreyashi, Yin, Bing, Hua, Yining, Zhou, Xuan, Rohanian, Omid, Thakur, Anshul, Clifton, Lei, Clifton, David A.
- The Art of Saying No: Contextual Noncompliance in Language Models — published result table ↗
All 14 authors
Brahman, Faeze, Kumar, Sachin, Balachandran, Vidhisha, Dasigi, Pradeep, Pyatkin, Valentina, Ravichander, Abhilasha, Wiegreffe, Sarah, Dziri, Nouha, Chandu, Khyathi, Hessel, Jack, Tsvetkov, Yulia, Smith, Noah A., Choi, Yejin, Hajishirzi, Hannaneh
- DTBench — Conceptual Reasoning Index ↗
Names used in source tables
Mistral · Mistral Medium