Benchmark
MMLU-Pro
language
reasoning
math
general
text
A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
语言EN
满分1
参评模型69
模型排名
| 名次 | 模型 | 机构 | 分数 | 来源 |
|---|---|---|---|---|
| 1 | Nemotron 3 Ultra 550B A55B | NVIDIA | 86.8 | 来源 ↗ |
| 2 | DeepSeek-V3.2-Exp | DeepSeek | 85.0 | 来源 ↗ |
| 3 | DeepSeek-R1-0528 | DeepSeek | 85.0 | 来源 ↗ |
| 4 | GLM-4.5 | Zhipu AI | 84.6 | 来源 ↗ |
| 5 | Qwen3-235B-A22B-Thinking-2507 | Alibaba Cloud / Qwen Team | 84.4 | 来源 ↗ |
| 6 | DeepSeek-V3.1 | DeepSeek | 83.7 | 来源 ↗ |
| 7 | Qwen3-235B-A22B-Instruct-2507 | Alibaba Cloud / Qwen Team | 83.0 | 来源 ↗ |
| 8 | Qwen3-Next-80B-A3B-Thinking | Alibaba Cloud / Qwen Team | 82.7 | 来源 ↗ |
| 9 | Kimi K2 0905 | Moonshot AI | 82.5 | 来源 ↗ |
| 10 | GLM-4.5-Air | Zhipu AI | 81.4 | 来源 ↗ |
| 11 | DeepSeek-V3 0324 | DeepSeek | 81.2 | 来源 ↗ |
| 12 | Kimi K2-Instruct-0905 | Moonshot AI | 81.1 | 来源 ↗ |
| 13 | Kimi K2 Instruct | Moonshot AI | 81.1 | 来源 ↗ |
| 14 | Qwen3-Next-80B-A3B-Instruct | Alibaba Cloud / Qwen Team | 80.6 | 来源 ↗ |
| 15 | Llama 4 Maverick | Meta | 80.5 | 来源 ↗ |
| 16 | Claude 3.5 Sonnet | Anthropic | 77.6 | 来源 ↗ |
| 17 | Gemini 2.0 Flash | 76.4 | 来源 ↗ | |
| 18 | Claude 3.5 Sonnet | Anthropic | 76.1 | 来源 ↗ |
| 19 | Phi 4 Reasoning Plus | Microsoft | 76.0 | 来源 ↗ |
| 20 | DeepSeek-V3 | DeepSeek | 75.9 | 来源 ↗ |
| 21 | Gemini 1.5 Pro | 75.8 | 来源 ↗ | |
| 22 | Grok-2 | xAI | 75.5 | 来源 ↗ |
| 23 | GPT-4o | OpenAI | 74.7 | 来源 ↗ |
| 24 | Phi 4 Reasoning | Microsoft | 74.3 | 来源 ↗ |
| 25 | Llama 4 Scout | Meta | 74.3 | 来源 ↗ |
| 26 | Llama 3.1 405B Instruct | Meta | 73.3 | 来源 ↗ |
| 27 | GPT-4o | OpenAI | 72.6 | 来源 ↗ |
| 28 | Grok-2 mini | xAI | 72.0 | 来源 ↗ |
| 29 | Gemini 2.0 Flash-Lite | 71.6 | 来源 ↗ | |
| 30 | Qwen2.5 72B Instruct | Alibaba Cloud / Qwen Team | 71.1 | 来源 ↗ |
| 31 | Phi 4 | Microsoft | 70.4 | 来源 ↗ |
| 32 | Kimi K2 Base | Moonshot AI | 69.2 | 来源 ↗ |
| 33 | Mistral Small 3.2 24B Instruct | Mistral AI | 69.1 | 来源 ↗ |
| 34 | Qwen2.5 32B Instruct | Alibaba Cloud / Qwen Team | 69.0 | 来源 ↗ |
| 35 | Llama 3.3 70B Instruct | Meta | 68.9 | 来源 ↗ |
| 36 | Qwen2.5 VL 32B Instruct | Alibaba Cloud / Qwen Team | 68.8 | 来源 ↗ |
| 37 | Claude 3 Opus | Anthropic | 68.5 | 来源 ↗ |
| 38 | Qwen3 235B A22B | Alibaba Cloud / Qwen Team | 68.2 | 来源 ↗ |
| 39 | Gemma 3 27B | 67.5 | 来源 ↗ | |
| 40 | Gemini 1.5 Flash | 67.3 | 来源 ↗ | |
| 41 | Mistral Small 3.1 24B Instruct | Mistral AI | 66.8 | 来源 ↗ |
| 42 | Llama 3.1 70B Instruct | Meta | 66.4 | 来源 ↗ |
| 43 | Mistral Small 3 24B Instruct | Mistral AI | 66.3 | 来源 ↗ |
| 44 | Claude 3.5 Haiku | Anthropic | 65.0 | 来源 ↗ |
| 45 | Qwen2 72B Instruct | Alibaba Cloud / Qwen Team | 64.4 | 来源 ↗ |
| 46 | Qwen2.5 14B Instruct | Alibaba Cloud / Qwen Team | 63.7 | 来源 ↗ |
| 47 | Gemma 3 12B | 60.6 | 来源 ↗ | |
| 48 | Gemini 1.5 Flash 8B | 58.7 | 来源 ↗ | |
| 49 | Claude 3 Sonnet | Anthropic | 56.8 | 来源 ↗ |
| 50 | Qwen2.5 7B Instruct | Alibaba Cloud / Qwen Team | 56.3 | 来源 ↗ |
| 51 | Mistral Small 3.1 24B Base | Mistral AI | 56.0 | 来源 ↗ |
| 52 | Mistral Small 3 24B Base | Mistral AI | 54.4 | 来源 ↗ |
| 53 | Jamba 1.5 Large | AI21 Labs | 53.5 | 来源 ↗ |
| 54 | Phi 4 Mini | Microsoft | 52.8 | 来源 ↗ |
| 55 | Grok-1.5 | xAI | 51.0 | 来源 ↗ |
| 56 | Gemma 3n E4B Instructed LiteRT Preview | 50.6 | 来源 ↗ | |
| 57 | Gemma 3n E4B Instructed | 50.6 | 来源 ↗ | |
| 58 | Qwen2.5-Coder 32B Instruct | Alibaba Cloud / Qwen Team | 50.4 | 来源 ↗ |
| 59 | Llama 3.1 8B Instruct | Meta | 48.3 | 来源 ↗ |
| 60 | Phi-3.5-mini-instruct | Microsoft | 47.4 | 来源 ↗ |
| 61 | Qwen2.5-Omni-7B | Alibaba Cloud / Qwen Team | 47.0 | 来源 ↗ |
| 62 | Phi-3.5-MoE-instruct | Microsoft | 45.3 | 来源 ↗ |
| 63 | Qwen2 7B Instruct | Alibaba Cloud / Qwen Team | 44.1 | 来源 ↗ |
| 64 | Gemma 3 4B | 43.6 | 来源 ↗ | |
| 65 | Jamba 1.5 Mini | AI21 Labs | 42.5 | 来源 ↗ |
| 66 | Gemma 3n E2B Instructed LiteRT (Preview) | 40.5 | 来源 ↗ | |
| 67 | Gemma 3n E2B Instructed | 40.5 | 来源 ↗ | |
| 68 | Qwen2.5-Coder 7B Instruct | Alibaba Cloud / Qwen Team | 40.1 | 来源 ↗ |
| 69 | Gemma 3 1B | 14.7 | 来源 ↗ |