Benchmark
SQuALITY
summarization
long_context
language
text
SQuALITY (Summarization-format QUestion Answering with Long Input Texts, Yes!) is a long-document summarization dataset built by hiring highly-qualified contractors to read public-domain short stories (3000-6000 words) and write original summaries from scratch. Each document has five summaries: one overview and four question-focused summaries. Designed to address limitations in existing summarization datasets by providing high-quality, faithful summaries.
语言EN
满分1
参评模型5
模型排名
| 名次 | 模型 | 机构 | 分数 | 来源 |
|---|---|---|---|---|
| 1 | Phi-3.5-mini-instruct | Microsoft | 24.3 | 来源 ↗ |
| 2 | Phi-3.5-MoE-instruct | Microsoft | 24.1 | 来源 ↗ |
| 3 | Nova Pro | Amazon | 19.8 | 来源 ↗ |
| 4 | Nova Lite | Amazon | 19.2 | 来源 ↗ |
| 5 | Nova Micro | Amazon | 18.8 | 来源 ↗ |