MMMLU (OpenAI Multilingual MMLU)

Professional human translation of the entire MMLU test set into 14 languages (Arabic, Bengali, Chinese, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Spanish, Swahili, Yoruba).26 Oct 2026

One row per model and compatible score measure. Mean = equal-weight arithmetic average of imported reports. Distinct configurations reported for the same model in a source table remain separate. Reports can overlap; the range is not a confidence interval. Unbundle to inspect each observation.
		Evidence	What the result shows
Qwen3-235B-A22B Base →Qwen	86.7% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	Leads this displayed comparison. Source notes Source cell: 86.70
DeepSeek V3 Base →DeepSeek	85.88% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#2 of 21 in this displayed comparison; 0.82 pp below the leader. Source notes Source cell: 85.88
Qwen2.5-72B Base →Qwen	84.4% 2 reports	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1] View 2 reports 84.4% · MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · Qwen3 report · base-model comparison [1] 84.4% · MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · Qwen3 report · base-model comparison [1]	The imported reports agree at the published precision. Repeated citations may describe the same underlying evaluation. #3 of 21 in this displayed comparison; 2.3 pp below the leader.
Qwen3-32B Base →Qwen	83.83% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#4 of 21 in this displayed comparison; 2.87 pp below the leader. Source notes Source cell: 83.83
Qwen2.5-Plus Base →Qwen	83.49% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#5 of 21 in this displayed comparison; 3.21 pp below the leader. Source notes Source cell: 83.49
Llama-4-Maverick Base →Meta	83.09% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#6 of 21 in this displayed comparison; 3.61 pp below the leader. Source notes Source cell: 83.09
Qwen2.5-32B Base →Qwen	82.4% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#7 of 21 in this displayed comparison; 4.3 pp below the leader. Source notes Source cell: 82.40
Qwen2.5-14B Base →Qwen	78.34% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#8 of 21 in this displayed comparison; 8.36 pp below the leader. Source notes Source cell: 78.34
Gemma-3-27B Base →Google	77.62% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#9 of 21 in this displayed comparison; 9.08 pp below the leader. Source notes Source cell: 77.62
Qwen3-8B Base →Qwen	75.72% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#10 of 21 in this displayed comparison; 10.98 pp below the leader. Source notes Source cell: 75.72
Llama-4-Scout Base →Meta	74.83% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#11 of 21 in this displayed comparison; 11.87 pp below the leader. Source notes Source cell: 74.83
Qwen3-4B Base →Qwen	71.42% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#12 of 21 in this displayed comparison; 15.28 pp below the leader. Source notes Source cell: 71.42
Qwen2.5-7B Base →Qwen	71.34% 2 reports	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1] View 2 reports 71.34% · MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · Qwen3 report · base-model comparison [1] 71.34% · MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · Qwen3 report · base-model comparison [1]	The imported reports agree at the published precision. Repeated citations may describe the same underlying evaluation. #13 of 21 in this displayed comparison; 15.36 pp below the leader.
Qwen2.5-3B Base →Qwen	65.55% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#14 of 21 in this displayed comparison; 21.15 pp below the leader. Source notes Source cell: 65.55
Qwen3-1.7B Base →Qwen	63.27% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#15 of 21 in this displayed comparison; 23.43 pp below the leader. Source notes Source cell: 63.27
Qwen2.5-1.5B Base →Qwen	60.27% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#16 of 21 in this displayed comparison; 26.43 pp below the leader. Source notes Source cell: 60.27
Llama-3-8B Base →Meta	59.65% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#17 of 21 in this displayed comparison; 27.05 pp below the leader. Source notes Source cell: 59.65
Gemma-3-4B Base →Google	59.62% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#18 of 21 in this displayed comparison; 27.08 pp below the leader. Source notes Source cell: 59.62
Qwen3-0.6B Base →Qwen	50.16% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#19 of 21 in this displayed comparison; 36.54 pp below the leader. Source notes Source cell: 50.16
Qwen2.5-0.5B Base →Qwen	31.53% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#20 of 21 in this displayed comparison; 55.17 pp below the leader. Source notes Source cell: 31.53
Gemma-3-1B Base →Google	26.57% 1 report	MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy[1]Qwen3 report · base-model comparison	#21 of 21 in this displayed comparison; 60.13 pp below the leader. Source notes Source cell: 26.57

Sources

Qwen3 Technical Report — published result table ↗
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng et al. · 2025
All 60 authors
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, Zihan Qiu
Dates & evaluation details
MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · 2026-10-26
Published model comparison, 2025. Qwen3 report · base-model comparison. Base-model results from the Qwen3 technical report. MultiPL-E and McEval are distinct suites; this observation covers only the explicitly named measure. Table 3; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.
MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · 2026-10-26
Published model comparison, 2025. Qwen3 report · base-model comparison. Base-model results from the Qwen3 technical report. MultiPL-E and McEval are distinct suites; this observation covers only the explicitly named measure. Table 4; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.
MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · 2026-10-26
Published model comparison, 2025. Qwen3 report · base-model comparison. Base-model results from the Qwen3 technical report. MultiPL-E and McEval are distinct suites; this observation covers only the explicitly named measure. Table 6; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.
MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · 2026-10-26
Published model comparison, 2025. Qwen3 report · base-model comparison. Base-model results from the Qwen3 technical report. MultiPL-E and McEval are distinct suites; this observation covers only the explicitly named measure. Table 7; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.
MMMLU (OpenAI Multilingual MMLU) · MMMLU accuracy · Qwen3 report · base-model comparison · 2026-10-26
Published model comparison, 2025. Qwen3 report · base-model comparison. Base-model results from the Qwen3 technical report. MultiPL-E and McEval are distinct suites; this observation covers only the explicitly named measure. Table 8; source precision and any reported uncertainty retained. These are source-reported observations, not evaluations run by RLTasks.

TRACE THE SCORE · Per-model notes use published breakdowns where available.

What it measures

Reading the measurement

Accuracy is only part of the picture. Inspect unsupported answers, abstentions, language coverage and the evidence available to the model. A confident response can still be wrong, and a broad average can hide a specialist gap.

Published evaluation

Base-model results from the Qwen3 technical report. MultiPL-E and McEval are distinct suites; this observation covers only the explicitly named measure.

Where this fits in an evaluation suite

Read accuracy together with error and abstention rates. Domain averages can hide language gaps, specialist failures, and poorly calibrated confidence.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Base-model results from the Qwen3 technical report. MultiPL-E and McEval are distinct suites; this observation covers only the explicitly named measure.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.

PRIMARY EVIDENCE & FURTHER READING

Qwen3 Technical Report — published result table ↗
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng et al. · 2025
All 60 authors
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, Zihan Qiu

Reviewed 2026-10-26. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.

Continue exploring

All knowledge & truth benchmarks →

Sources

What it measures

Reading the measurement

Published evaluation

Where this fits in an evaluation suite

A reproducible comparison

Continue exploring

Harvey LAB-AA v1.1

AA-Omniscience v1.1

HealthBench

MMLU-Pro