Benchmark

MBPP

reasoning general text

MBPP (Mostly Basic Python Problems) is a benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers. Each problem consists of a task description, code solution, and 3 automated test cases covering programming fundamentals and standard library functionality.

语言EN
满分100
参评模型31
论文 ↗

模型排名

名次 模型 机构 分数 来源
1 Llama-3.3 Nemotron Super 49B v1 NVIDIA 91.3 来源 ↗
2 Qwen2.5-Coder 32B Instruct Alibaba Cloud / Qwen Team 90.2 来源 ↗
3 Qwen2.5 72B Instruct Alibaba Cloud / Qwen Team 88.2 来源 ↗
4 Llama 3.1 Nemotron Nano 8B V1 NVIDIA 84.6 来源 ↗
5 Qwen2.5 VL 32B Instruct Alibaba Cloud / Qwen Team 84.0 来源 ↗
6 Qwen2.5 32B Instruct Alibaba Cloud / Qwen Team 84.0 来源 ↗
7 Qwen2.5-Coder 7B Instruct Alibaba Cloud / Qwen Team 83.5 来源 ↗
8 Qwen2.5 14B Instruct Alibaba Cloud / Qwen Team 82.0 来源 ↗
9 Qwen3 235B A22B Alibaba Cloud / Qwen Team 81.4 来源 ↗
10 Phi-3.5-MoE-instruct Microsoft 80.8 来源 ↗
11 Qwen2 72B Instruct Alibaba Cloud / Qwen Team 80.2 来源 ↗
12 Qwen2.5 7B Instruct Alibaba Cloud / Qwen Team 79.2 来源 ↗
13 Codestral-22B Mistral AI 78.2 来源 ↗
14 Llama 4 Maverick Meta 77.6 来源 ↗
15 Gemini Diffusion Google 76.0 来源 ↗
16 Mistral Small 3.1 24B Instruct Mistral AI 74.7 来源 ↗
17 Gemma 3 27B Google 74.4 来源 ↗
18 Qwen2.5-Omni-7B Alibaba Cloud / Qwen Team 73.2 来源 ↗
19 Gemma 3 12B Google 73.0 来源 ↗
20 Mistral Small 3 24B Base Mistral AI 69.6 来源 ↗
21 Phi-3.5-mini-instruct Microsoft 69.6 来源 ↗
22 Llama 4 Scout Meta 67.8 来源 ↗
23 Qwen2 7B Instruct Alibaba Cloud / Qwen Team 67.2 来源 ↗
24 Gemma 3n E4B Instructed LiteRT Preview Google 63.6 来源 ↗
25 Gemma 3n E4B Instructed Google 63.6 来源 ↗
26 Gemma 3 4B Google 63.2 来源 ↗
27 Gemma 2 27B Google 62.6 来源 ↗
28 Gemma 3n E2B Instructed LiteRT (Preview) Google 56.6 来源 ↗
29 Gemma 3n E2B Instructed Google 56.6 来源 ↗
30 Gemma 2 9B Google 52.4 来源 ↗
31 Gemma 3 1B Google 35.2 来源 ↗