Benchmark
AndroidWorld_SR
general
multimodal
reasoning
multimodal
AndroidWorld Success Rate (SR) benchmark - A dynamic benchmarking environment for autonomous agents operating on Android devices. Evaluates agents on 116 programmatic tasks across 20 real-world Android apps using multimodal inputs (screen screenshots, accessibility trees, and natural language instructions). Measures success rate of agents completing tasks like sending messages, creating calendar events, and navigating mobile interfaces. Published at ICLR 2025. Best current performance: 30.6% success rate (M3A agent) vs 80.0% human performance.
Top 10
榜单概览
数据说明
分数为 0-100 百分制综合成绩（Elo 榜为相对评分，仅供同榜内比较）。评测分数属事实数据，整理自公开评测体系与厂商公开资料，部分分数为厂商自报口径，不同榜单的评测方法与数据集可能存在差异，请结合多个榜单交叉参考。
Ranking
模型排名
暂无评测数据。