Benchmark
HumanEval
reasoning
code
text
A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
语言EN
满分1
参评模型63
模型排名
| 名次 | 模型 | 机构 | 分数 | 来源 |
|---|---|---|---|---|
| 1 | Kimi K2 0905 | Moonshot AI | 94.5 | 来源 ↗ |
| 2 | Claude 3.5 Sonnet | Anthropic | 93.7 | 来源 ↗ |
| 3 | GPT-5 | OpenAI | 93.4 | 来源 ↗ |
| 4 | Kimi K2 Instruct | Moonshot AI | 93.3 | 来源 ↗ |
| 5 | Qwen2.5-Coder 32B Instruct | Alibaba Cloud / Qwen Team | 92.7 | 来源 ↗ |
| 6 | o1-mini | OpenAI | 92.4 | 来源 ↗ |
| 7 | Mistral Large 2 | Mistral AI | 92.0 | 来源 ↗ |
| 8 | Claude 3.5 Sonnet | Anthropic | 92.0 | 来源 ↗ |
| 9 | Qwen2.5 VL 32B Instruct | Alibaba Cloud / Qwen Team | 91.5 | 来源 ↗ |
| 10 | GPT-4o | OpenAI | 90.2 | 来源 ↗ |
| 11 | Granite 3.3 8B Instruct | IBM | 89.7 | 来源 ↗ |
| 12 | Granite 3.3 8B Base | IBM | 89.7 | 来源 ↗ |
| 13 | Gemini Diffusion | 89.6 | 来源 ↗ | |
| 14 | Nova Pro | Amazon | 89.0 | 来源 ↗ |
| 15 | DeepSeek-V2.5 | DeepSeek | 89.0 | 来源 ↗ |
| 16 | Llama 3.1 405B Instruct | Meta | 89.0 | 来源 ↗ |
| 17 | Mistral Small 3.1 24B Instruct | Mistral AI | 88.4 | 来源 ↗ |
| 18 | Grok-2 | xAI | 88.4 | 来源 ↗ |
| 19 | Qwen2.5-Coder 7B Instruct | Alibaba Cloud / Qwen Team | 88.4 | 来源 ↗ |
| 20 | Qwen2.5 32B Instruct | Alibaba Cloud / Qwen Team | 88.4 | 来源 ↗ |
| 21 | Llama 3.3 70B Instruct | Meta | 88.4 | 来源 ↗ |
| 22 | o1 | OpenAI | 88.1 | 来源 ↗ |
| 23 | Claude 3.5 Haiku | Anthropic | 88.1 | 来源 ↗ |
| 24 | GPT-4.5 | OpenAI | 88.0 | 来源 ↗ |
| 25 | Gemma 3 27B | 87.8 | 来源 ↗ | |
| 26 | GPT-4o mini | OpenAI | 87.2 | 来源 ↗ |
| 27 | GPT-4 Turbo | OpenAI | 87.1 | 来源 ↗ |
| 28 | Qwen2.5 72B Instruct | Alibaba Cloud / Qwen Team | 86.6 | 来源 ↗ |
| 29 | Qwen2 72B Instruct | Alibaba Cloud / Qwen Team | 86.0 | 来源 ↗ |
| 30 | Grok-2 mini | xAI | 85.7 | 来源 ↗ |
| 31 | Gemma 3 12B | 85.4 | 来源 ↗ | |
| 32 | Nova Lite | Amazon | 85.4 | 来源 ↗ |
| 33 | Claude 3 Opus | Anthropic | 84.9 | 来源 ↗ |
| 34 | Qwen2.5 7B Instruct | Alibaba Cloud / Qwen Team | 84.8 | 来源 ↗ |
| 35 | Mistral Small 3 24B Instruct | Mistral AI | 84.8 | 来源 ↗ |
| 36 | Gemini 1.5 Pro | 84.1 | 来源 ↗ | |
| 37 | Qwen2.5 14B Instruct | Alibaba Cloud / Qwen Team | 83.5 | 来源 ↗ |
| 38 | Phi 4 | Microsoft | 82.6 | 来源 ↗ |
| 39 | IBM Granite 4.0 Tiny Preview | IBM | 82.4 | 来源 ↗ |
| 40 | Codestral-22B | Mistral AI | 81.1 | 来源 ↗ |
| 41 | Nova Micro | Amazon | 81.1 | 来源 ↗ |
| 42 | Llama 3.1 70B Instruct | Meta | 80.5 | 来源 ↗ |
| 43 | Qwen2 7B Instruct | Alibaba Cloud / Qwen Team | 79.9 | 来源 ↗ |
| 44 | Qwen2.5-Omni-7B | Alibaba Cloud / Qwen Team | 78.7 | 来源 ↗ |
| 45 | Claude 3 Haiku | Anthropic | 75.9 | 来源 ↗ |
| 46 | Gemma 3n E4B Instructed | 75.0 | 来源 ↗ | |
| 47 | Gemma 3n E4B Instructed LiteRT Preview | 75.0 | 来源 ↗ | |
| 48 | Gemini 1.5 Flash | 74.3 | 来源 ↗ | |
| 49 | Grok-1.5 | xAI | 74.1 | 来源 ↗ |
| 50 | Claude 3 Sonnet | Anthropic | 73.0 | 来源 ↗ |
| 51 | Llama 3.1 8B Instruct | Meta | 72.6 | 来源 ↗ |
| 52 | Pixtral-12B | Mistral AI | 72.0 | 来源 ↗ |
| 53 | Gemma 3 4B | 71.3 | 来源 ↗ | |
| 54 | Phi-3.5-MoE-instruct | Microsoft | 70.7 | 来源 ↗ |
| 55 | GPT-3.5 Turbo | OpenAI | 68.0 | 来源 ↗ |
| 56 | GPT-4 | OpenAI | 67.0 | 来源 ↗ |
| 57 | Gemma 3n E2B Instructed LiteRT (Preview) | 66.5 | 来源 ↗ | |
| 58 | Gemma 3n E2B Instructed | 66.5 | 来源 ↗ | |
| 59 | Phi-3.5-mini-instruct | Microsoft | 62.8 | 来源 ↗ |
| 60 | Gemma 2 27B | 51.8 | 来源 ↗ | |
| 61 | Gemma 3 1B | 41.5 | 来源 ↗ | |
| 62 | Gemma 2 9B | 40.2 | 来源 ↗ | |
| 63 | Ministral 8B Instruct | Mistral AI | 34.8 | 来源 ↗ |