Benchmark
Natural2Code
reasoning
general
text
NaturalCodeBench (NCB) is a challenging code benchmark designed to mirror the complexity and variety of real-world coding tasks. It comprises 402 high-quality problems in Python and Java, selected from natural user queries from online coding services, covering 6 different domains.
语言EN
满分1
参评模型8
模型排名
| 名次 | 模型 | 机构 | 分数 | 来源 |
|---|---|---|---|---|
| 1 | Gemini 2.0 Flash | 92.9 | 来源 ↗ | |
| 2 | Gemini 1.5 Pro | 85.4 | 来源 ↗ | |
| 3 | Gemma 3 27B | 84.5 | 来源 ↗ | |
| 4 | Gemma 3 12B | 80.7 | 来源 ↗ | |
| 5 | Gemini 1.5 Flash | 79.8 | 来源 ↗ | |
| 6 | Gemini 1.5 Flash 8B | 75.5 | 来源 ↗ | |
| 7 | Gemma 3 4B | 70.3 | 来源 ↗ | |
| 8 | Gemma 3 1B | 56.0 | 来源 ↗ |