综合能力
GDPval-AA v2.1
真实职场任务:能否交付可用的文档、表格、幻灯片和图示
它测什么
GDPval-AA v2.1 是 Artificial Analysis 基于 OpenAI GDPval 数据集进行的独立评测。它使用公开 gold 集中的 220 个任务,覆盖 9 个行业、44 种职业,要求模型提交实际工作成果,而不只是回答问题。
比如,官方示例会让模型给一支巡演乐队制作一页舞台布置 PDF:把歌手、鼓手、音箱和话筒放对位置,再列清楚输入、输出线路。能说出“舞台图应该怎么画”还不够,模型需要真正做出文件,内容和排版都要满足需求。
所有模型通过 Stirrup Agent 框架工作,可以使用命令行、文件处理、网页搜索与浏览等工具,每个任务最多执行 250 轮。评估时隐藏模型身份,把同一任务的两份成果交给从三模型评审团中抽取的 LLM 裁判比较,再将这些对比结果汇总成 Elo 评分。
怎么看分数
Elo 越高,表示交付成果在这套任务的两两比较中更受评审偏好;DeepSeek V4.1 Flash (Max) 固定为 1,600 分作为锚点,其他模型都相对它计分。比如 A 得 1,500 分、B 得 1,200 分,只能说明 A 在这一评测中的相对表现更强,不表示 A 的正确率是 150%,也不表示效率高 25%。Elo 可以低于 0,表中的 ± 数值是官方公布的 95% 置信区间,分数接近时不要把名次差当成确定差距。本页保留 2026 年 11 月 1 日核对到的全部 308 条成绩和官方模型配置;标有 Fallback 的条目包含备用模型机制,不能当成单一模型独立完成的结果。v2.1 沿用 v2 的任务、工具和评审流程,但更换了锚点并改用 Crowd-BT 模型拟合评分,分数整体平移,因此不能与 v2 快照混用。榜单会随新增比较而变化,也不能直接与 v1、模型发布时的旧快照或 OpenAI 自己公布的 GDPval 胜率混用;自 Intelligence Index v5.0 起,它也不再计入该指数,是一项单独的评测。
模型表现
Claude Opus 5.5 (Max, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1866± 26 |
|---|---|---|
Kimi K3.1 ProKimiAgent · Stirrup | Stirrup | 1858± 28 |
Muse Wonder Full (Max)MetaAgent · Stirrup | Stirrup | 1857± 25 |
Claude Sonnet 5.5 (Max, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1838± 23 |
Claude Opus 5.5 (Xhigh, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1835± 23 |
Muse Wonder (Max)MetaAgent · Stirrup | Stirrup | 1770± 23 |
Claude Fable 5.1 (Max, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1763± 19 |
DeepSeek V4.1 Flash 1017 (Max)DeepSeekAgent · Stirrup | Stirrup | 1746± 21 |
Claude Fable 5.1 (Xhigh, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1740± 19 |
Claude Sonnet 5.5 (Xhigh, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1735± 23 |
Gemini 4 Carbon (High)GoogleAgent · Stirrup | Stirrup | 1733± 24 |
Claude Opus 5 (Max)AnthropicAgent · Stirrup | Stirrup | 1730± 19 |
Grok 4.7 (Xhigh)SpaceXAIAgent · Stirrup | Stirrup | 1718± 21 |
Grok 4.7 (High)SpaceXAIAgent · Stirrup | Stirrup | 1716± 19 |
Claude Opus 5.5 (High, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1714± 21 |
Claude Opus 5 (Xhigh)AnthropicAgent · Stirrup | Stirrup | 1696± 18 |
MiMo-V2.6-ProXiaomiAgent · Stirrup | Stirrup | 1689± 21 |
Muse Spark 1.3 (Max)MetaAgent · Stirrup | Stirrup | 1681± 19 |
Qwen3.8 Max (0902)AlibabaAgent · Stirrup | Stirrup | 1674± 20 |
GLM-5.3 (Max)Z AIAgent · Stirrup | Stirrup | 1657± 19 |
Grok 4.6 (Xhigh)SpaceXAIAgent · Stirrup | Stirrup | 1654± 19 |
Ling 3.1 Flash 1019InclusionAIAgent · Stirrup | Stirrup | 1653± 25 |
GLM-5.3-FlashZ AIAgent · Stirrup | Stirrup | 1649± 17 |
Claude Fable 5.1 (High, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1644± 18 |
Qwen3.8-Flash-NextAlibabaAgent · Stirrup | Stirrup | 1636± 18 |
Muse Spark 1.3 (Xhigh)MetaAgent · Stirrup | Stirrup | 1632± 18 |
Claude Haiku 5.5 (Max)AnthropicAgent · Stirrup | Stirrup | 1631± 22 |
Gemini 4 Argon (High)GoogleAgent · Stirrup | Stirrup | 1631± 20 |
Ling 3.1 FlashInclusionAIAgent · Stirrup | Stirrup | 1626± 21 |
Grok 4.6 (High)SpaceXAIAgent · Stirrup | Stirrup | 1626± 16 |
Grok 4.6 (Medium)SpaceXAIAgent · Stirrup | Stirrup | 1620± 18 |
Claude Fable 5 (Max, Opus 4.8 Fallback)AnthropicAgent · Stirrup | Stirrup | 1618± 16 |
GPT-5.6 Sol (Max)OpenAIAgent · Stirrup | Stirrup | 1618± 16 |
Qwen3.8 2.4T A95BAlibabaAgent · Stirrup | Stirrup | 1616± 19 |
MiMo-V2.6-FlashXiaomiAgent · Stirrup | Stirrup | 1614± 21 |
Qwen3.8 MaxAlibabaAgent · Stirrup | Stirrup | 1611± 17 |
GPT-6.1 Sol (Max)OpenAIAgent · Stirrup | Stirrup | 1609± 22 |
Claude Opus 5 (High)AnthropicAgent · Stirrup | Stirrup | 1604± 18 |
DeepSeek V4.1 Flash (Max)DeepSeekAgent · Stirrup | Stirrup | 16001 |
Step 5 PreviewStepFunAgent · Stirrup | Stirrup | 1594± 23 |
Claude Opus 5.5 (Medium, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1593± 20 |
Grok 4.7 (Low)SpaceXAIAgent · Stirrup | Stirrup | 1591± 22 |
GPT-6 Astra (Max)OpenAIAgent · Stirrup | Stirrup | 1584± 19 |
GPT-5.6 Sol (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1578± 16 |
GPT-6 Astra (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1569± 19 |
Claude Sonnet 5.5 (High, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1557± 22 |
GPT-6.1 Sol (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1555± 21 |
Claude Fable 5.1 (Medium, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1554± 18 |
DeepSeek V4 Flash Vision (Max)DeepSeekAgent · Stirrup | Stirrup | 1554± 17 |
Kimi K3 (Max)KimiAgent · Stirrup | Stirrup | 1543± 18 |
GPT-6 Astra (High)OpenAIAgent · Stirrup | Stirrup | 1536± 19 |
Claude Haiku 5.5 (Xhigh)AnthropicAgent · Stirrup | Stirrup | 1522± 22 |
GPT-6.1 Sol (High)OpenAIAgent · Stirrup | Stirrup | 1520± 21 |
GPT-6 Sol (Max)OpenAIAgent · Stirrup | Stirrup | 1516± 22 |
GPT-5.6 Sol (High)OpenAIAgent · Stirrup | Stirrup | 1512± 16 |
Claude Opus 5 (Medium)AnthropicAgent · Stirrup | Stirrup | 1499± 18 |
GPT-6 Astra (Medium)OpenAIAgent · Stirrup | Stirrup | 1498± 18 |
Muse Spark 1.2 (Xhigh)MetaAgent · Stirrup | Stirrup | 1480± 19 |
Claude Fable 5.1 (Low, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1477± 18 |
GPT-6.1 Terra (Max)OpenAIAgent · Stirrup | Stirrup | 1476± 23 |
Claude Sonnet 5 (Max)AnthropicAgent · Stirrup | Stirrup | 1470± 16 |
GPT-5.6 Luna (Max)OpenAIAgent · Stirrup | Stirrup | 1468± 16 |
GPT-6 Sol (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1466± 22 |
MiniMax-M3.1-FlashMiniMaxAgent · Stirrup | Stirrup | 1465± 23 |
Claude Opus 4.8 (Max)AnthropicAgent · Stirrup | Stirrup | 1462± 15 |
GPT-5.6 Terra (Max)OpenAIAgent · Stirrup | Stirrup | 1461± 16 |
GPT-6.1 Sol (Medium)OpenAIAgent · Stirrup | Stirrup | 1459± 21 |
DeepSeek V4 Pro 0813 (Max)DeepSeekAgent · Stirrup | Stirrup | 1457± 19 |
GPT-5.6 Terra (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1455± 16 |
Gemini 3.8 Flash (High)GoogleAgent · Stirrup | Stirrup | 1445± 19 |
GPT-6 Luna (Max)OpenAIAgent · Stirrup | Stirrup | 1444± 20 |
DeepSeek V4 Flash 0731 (Max)DeepSeekAgent · Stirrup | Stirrup | 1441± 19 |
Mistral Large 4 PreviewMistralAgent · Stirrup | Stirrup | 1433± 23 |
GPT-5.6 Sol (Medium)OpenAIAgent · Stirrup | Stirrup | 1432± 16 |
Gemini 3.8 Flash (Medium)GoogleAgent · Stirrup | Stirrup | 1431± 20 |
Qwen3.8 27B (Xhigh)AlibabaAgent · Stirrup | Stirrup | 1427± 20 |
Claude Haiku 5.5 (High)AnthropicAgent · Stirrup | Stirrup | 1424± 19 |
Grok 4.6 (Low)SpaceXAIAgent · Stirrup | Stirrup | 1418± 19 |
GPT-6.1 Terra (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1415± 23 |
GPT-5.6 Luna (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1408± 17 |
GPT-6 Sol (High)OpenAIAgent · Stirrup | Stirrup | 1404± 21 |
GPT-6 Astra (Low)OpenAIAgent · Stirrup | Stirrup | 1403± 20 |
Gemini 3.7 Flash (High)GoogleAgent · Stirrup | Stirrup | 1403± 19 |
Muse Spark Open (Xhigh)MetaAgent · Stirrup | Stirrup | 1401± 19 |
Grok 4.5 (High)SpaceXAIAgent · Stirrup | Stirrup | 1389± 19 |
Qwen3.8 27B (Medium)AlibabaAgent · Stirrup | Stirrup | 1389± 18 |
GPT-5.6 Terra (High)OpenAIAgent · Stirrup | Stirrup | 1387± 17 |
GLM-5.2 (Max)Z AIAgent · Stirrup | Stirrup | 1381± 16 |
GPT-6 Luna (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1373± 21 |
Qwen3.8 27B (Low)AlibabaAgent · Stirrup | Stirrup | 1370± 19 |
Gemini 3.7 Flash (Medium)GoogleAgent · Stirrup | Stirrup | 1367± 20 |
Claude Sonnet 5 (Xhigh)AnthropicAgent · Stirrup | Stirrup | 1366± 17 |
K2 Horizon 375B A23BMBZUAI Institute of Foundation ModelsAgent · Stirrup | Stirrup | 1365± 19 |
Claude Opus 4.7 (Max)AnthropicAgent · Stirrup | Stirrup | 1362± 16 |
GPT-5.5 (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1359± 16 |
GPT-6.1 Terra (High)OpenAIAgent · Stirrup | Stirrup | 1353± 23 |
GPT-6 Luna (High)OpenAIAgent · Stirrup | Stirrup | 1352± 21 |
GPT-6 Sol (Medium)OpenAIAgent · Stirrup | Stirrup | 1352± 22 |
DeepSeek V4.1 Flash (Non-reasoning)DeepSeekAgent · Stirrup | Stirrup | 1349± 20 |
GPT-5.6 Luna (High)OpenAIAgent · Stirrup | Stirrup | 1339± 17 |
GPT-5.5 (High)OpenAIAgent · Stirrup | Stirrup | 1336± 16 |
Agnes 2.5 Pro BetaSapiens AIAgent · Stirrup | Stirrup | 1335± 20 |
JT-4.1 Flash 236B A21B (Reasoning)China MobileAgent · Stirrup | Stirrup | 1335± 23 |
GLM-5.3 (Low)Z AIAgent · Stirrup | Stirrup | 1333± 20 |
Claude Sonnet 5.5 (Medium, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1329± 21 |
Gemini 3.7 Flash (Low)GoogleAgent · Stirrup | Stirrup | 1327± 20 |
GPT-6.1 Terra (Medium)OpenAIAgent · Stirrup | Stirrup | 1325± 22 |
GPT-6.1 Sol (Low)OpenAIAgent · Stirrup | Stirrup | 1322± 21 |
Gemini 3.8 Flash (Low)GoogleAgent · Stirrup | Stirrup | 1316± 17 |
GPT-5.6 Sol (Low)OpenAIAgent · Stirrup | Stirrup | 1313± 17 |
Claude Opus 5 (Low)AnthropicAgent · Stirrup | Stirrup | 1311± 20 |
Gemini 3.6 Flash (High)GoogleAgent · Stirrup | Stirrup | 1295± 19 |
GPT-6.1 Terra (Non-reasoning)OpenAIAgent · Stirrup | Stirrup | 1286± 23 |
Claude Haiku 5.5 (Medium)AnthropicAgent · Stirrup | Stirrup | 1283± 20 |
GPT-5.6 Terra (Medium)OpenAIAgent · Stirrup | Stirrup | 1274± 17 |
GPT-6 Luna (Medium)OpenAIAgent · Stirrup | Stirrup | 1270± 21 |
Claude Sonnet 5 (High)AnthropicAgent · Stirrup | Stirrup | 1263± 17 |
GPT-5.4 (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1256± 16 |
GPT-6 Sol (Non-reasoning)OpenAIAgent · Stirrup | Stirrup | 1254± 21 |
GLM-5.2 (Non-reasoning)Z AIAgent · Stirrup | Stirrup | 1252± 21 |
MiniMax-M3MiniMaxAgent · Stirrup | Stirrup | 1250± 16 |
GPT-5.6 Sol (Non-reasoning)OpenAIAgent · Stirrup | Stirrup | 1250± 18 |
GPT-5.5 (Medium)OpenAIAgent · Stirrup | Stirrup | 1243± 18 |
Claude Sonnet 4.6 (Max)AnthropicAgent · Stirrup | Stirrup | 1241± 16 |
Claude Opus 5.5 (Low, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1240± 21 |
Claude Sonnet 5 (Non-reasoning, High)AnthropicAgent · Stirrup | Stirrup | 1221± 19 |
Muse Spark 1.1 (Xhigh)MetaAgent · Stirrup | Stirrup | 1217± 20 |
Gemini 3.5 Flash (High)GoogleAgent · Stirrup | Stirrup | 1212± 17 |
GPT-6.1 Terra (Low)OpenAIAgent · Stirrup | Stirrup | 1204± 23 |
GPT-6 Sol (Low)OpenAIAgent · Stirrup | Stirrup | 1201± 22 |
K2 Horizon MoVA 36B A4BMBZUAI Institute of Foundation ModelsAgent · Stirrup | Stirrup | 1197± 21 |
Apodex 1.1ApodexAgent · Stirrup | Stirrup | 1196± 20 |
Quasar 438B (Max, Based on GLM-5.2)Multiverse ComputingAgent · Stirrup | Stirrup | 1190± 20 |
Claude Sonnet 5.5 (Low, Default Fallback)AnthropicAgent · Stirrup | Stirrup | 1182± 22 |
Beam 1.0ReflectionAgent · Stirrup | Stirrup | 1177± 19 |
Ling-3.0-flash-VLInclusionAIAgent · Stirrup | Stirrup | 1169± 21 |
Claude Sonnet 5 (Medium)AnthropicAgent · Stirrup | Stirrup | 1167± 18 |
DeepSeek V4 Pro 0424 (Max)DeepSeekAgent · Stirrup | Stirrup | 1166± 17 |
Nemotron 3.5 Super VLNVIDIAAgent · Stirrup | Stirrup | 1162± 19 |
DeepSeek V4 Pro 0424 (High)DeepSeekAgent · Stirrup | Stirrup | 1160± 22 |
GPT-5.6 Luna (Medium)OpenAIAgent · Stirrup | Stirrup | 1138± 19 |
Kimi K3 (Low)KimiAgent · Stirrup | Stirrup | 1136± 22 |
Qwen3.7 MaxAlibabaAgent · Stirrup | Stirrup | 1135± 17 |
Inkling SmallThinking MachinesAgent · Stirrup | Stirrup | 1133± 22 |
Claude Haiku 5.5 (Low)AnthropicAgent · Stirrup | Stirrup | 1130± 22 |
MiMo-V2.5-Pro (Reasoning)XiaomiAgent · Stirrup | Stirrup | 1129± 17 |
JT-4.1 Flash 236B A21B (Non-reasoning)China MobileAgent · Stirrup | Stirrup | 1128± 21 |
Motif 3Motif TechnologiesAgent · Stirrup | Stirrup | 1127± 22 |
GLM-5.1 (Reasoning)Z AIAgent · Stirrup | Stirrup | 1125± 17 |
GPT-5.6 Terra (Low)OpenAIAgent · Stirrup | Stirrup | 1117± 18 |
Nex-N2-Pro (Based on Qwen3.5-397B-A17B)Nex AGIAgent · Stirrup | Stirrup | 1109± 19 |
G9v3-39A5BAI9StarsAgent · Stirrup | Stirrup | 1109± 23 |
Motif 3 (Beta)Motif TechnologiesAgent · Stirrup | Stirrup | 1109± 22 |
Ling-3.0-flash-FinInclusionAIAgent · Stirrup | Stirrup | 1108± 22 |
Solar Pro 4UpstageAgent · Stirrup | Stirrup | 1106± 20 |
GPT-5.6 Terra (Non-reasoning)OpenAIAgent · Stirrup | Stirrup | 1105± 19 |
Solar Mini 4UpstageAgent · Stirrup | Stirrup | 1092± 23 |
Qwen3.8 27B (Non-reasoning)AlibabaAgent · Stirrup | Stirrup | 1082± 20 |
Inkling (Xhigh)Thinking MachinesAgent · Stirrup | Stirrup | 1080± 22 |
Claude Sonnet 5 (Low)AnthropicAgent · Stirrup | Stirrup | 1076± 18 |
Grok Build 0.1 0616SpaceXAIAgent · Stirrup | Stirrup | 1073± 18 |
Hy3TencentAgent · Stirrup | Stirrup | 1071± 22 |
DeepSeek V4 Pro 0813 (Non-reasoning)DeepSeekAgent · Stirrup | Stirrup | 1066± 22 |
GPT-5.5 (Low)OpenAIAgent · Stirrup | Stirrup | 1053± 19 |
GPT-6 Luna (Non-reasoning)OpenAIAgent · Stirrup | Stirrup | 1052± 22 |
Kimi K2.6 (Reasoning)KimiAgent · Stirrup | Stirrup | 1047± 17 |
Kimi K2.7 CodeKimiAgent · Stirrup | Stirrup | 1046± 17 |
DeepSeek V4 Flash 0420 (Max)DeepSeekAgent · Stirrup | Stirrup | 1046± 18 |
GPT-6 Luna (Low)OpenAIAgent · Stirrup | Stirrup | 1040± 22 |
K2 Horizon 7BMBZUAI Institute of Foundation ModelsAgent · Stirrup | Stirrup | 1024± 22 |
GPT-5.4 mini (Xhigh)OpenAIAgent · Stirrup | Stirrup | 1022± 17 |
GLM-4.7 (Reasoning)Z AIAgent · Stirrup | Stirrup | 1021± 20 |
Nemotron 3 Ultra 550B A55B (Reasoning)NVIDIAAgent · Stirrup | Stirrup | 1021± 17 |
MiniMax-M2.7MiniMaxAgent · Stirrup | Stirrup | 1016± 17 |
DeepSeek V4 Flash 0420 (High)DeepSeekAgent · Stirrup | Stirrup | 1009± 23 |
Muse SparkMetaAgent · Stirrup | Stirrup | 1006± 17 |
MiMo-V2.5XiaomiAgent · Stirrup | Stirrup | 1005± 22 |
GPT-5.6 Luna (Low)OpenAIAgent · Stirrup | Stirrup | 1002± 19 |
Qwen3.6 PlusAlibabaAgent · Stirrup | Stirrup | 997± 17 |
Gemini 3.5 Flash-LiteGoogleAgent · Stirrup | Stirrup | 995± 22 |
Qwen3.6 27B (Reasoning)AlibabaAgent · Stirrup | Stirrup | 993± 17 |
Nemotron 3.5 Bolt 74B-A9BNVIDIAAgent · Stirrup | Stirrup | 979± 19 |
Solar Open2 250BUpstageAgent · Stirrup | Stirrup | 978± 24 |
GPT-5.5 (Non-reasoning)OpenAIAgent · Stirrup | Stirrup | 978± 17 |
Qwen3.6 27B (Non-reasoning)AlibabaAgent · Stirrup | Stirrup | 969± 21 |
A.X-K2SK TelecomAgent · Stirrup | Stirrup | 968± 23 |
Ling 3.0 FlashInclusionAIAgent · Stirrup | Stirrup | 957± 23 |
GPT-5.4 nano (Xhigh)OpenAIAgent · Stirrup | Stirrup | 955± 17 |
Grok 4.3 (Non-reasoning)SpaceXAIAgent · Stirrup | Stirrup | 949± 18 |
Grok 4.3 (High)SpaceXAIAgent · Stirrup | Stirrup | 946± 18 |
GPT-5.6 Luna (Non-reasoning)OpenAIAgent · Stirrup | Stirrup | 928± 20 |
GPT-5 (High)OpenAIAgent · Stirrup | Stirrup | 925± 21 |
Claude 4.5 Sonnet (Reasoning)AnthropicAgent · Stirrup | Stirrup | 915± 21 |
Qwen3.6 35B A3B (Reasoning)AlibabaAgent · Stirrup | Stirrup | 899± 18 |
K2 Horizon 3.7BMBZUAI Institute of Foundation ModelsAgent · Stirrup | Stirrup | 889± 24 |
LongCat 2.0LongCatAgent · Stirrup | Stirrup | 877± 21 |
Step 3.7 FlashStepFunAgent · Stirrup | Stirrup | 864± 18 |
Qwen3.6 35B A3B (Non-reasoning)AlibabaAgent · Stirrup | Stirrup | 854± 24 |
Kimi K2.5 (Reasoning)KimiAgent · Stirrup | Stirrup | 847± 23 |
GPT-5.1 (High)OpenAIAgent · Stirrup | Stirrup | 834± 22 |
Qwen3.5 122B A10B (Reasoning)AlibabaAgent · Stirrup | Stirrup | 822± 18 |
K-EXAONE 2.0 0803LG AI ResearchAgent · Stirrup | Stirrup | 816± 25 |
Qwen3.5 397B A17B (Reasoning)AlibabaAgent · Stirrup | Stirrup | 801± 19 |
Gemini 3.1 Pro PreviewGoogleAgent · Stirrup | Stirrup | 799± 19 |
Muse Glimmer (High)MetaAgent · Stirrup | Stirrup | 792± 23 |
Qwen3.7 PlusAlibabaAgent · Stirrup | Stirrup | 775± 19 |
GPT-5 mini (High)OpenAIAgent · Stirrup | Stirrup | 773± 22 |
Mistral Medium 3.5MistralAgent · Stirrup | Stirrup | 769± 19 |
GLM-4.6 (Reasoning)Z AIAgent · Stirrup | Stirrup | 764± 22 |
Ring-2.6-1TInclusionAIAgent · Stirrup | Stirrup | 753± 19 |
KAT Coder Pro V2KwaiKATAgent · Stirrup | Stirrup | 742± 23 |
Claude 4.5 Haiku (Reasoning)AnthropicAgent · Stirrup | Stirrup | 740± 19 |
KAT-Coder-Pro V1KwaiKATAgent · Stirrup | Stirrup | 738± 21 |
DeepSeek V3.1 Terminus (Reasoning)DeepSeekAgent · Stirrup | Stirrup | 718± 24 |
Qwen3.5 122B A10B (Non-reasoning)AlibabaAgent · Stirrup | Stirrup | 716± 23 |
MiniCPM5-2BOpenBMBAgent · Stirrup | Stirrup | 715± 25 |
DeepSeek V3.2 (Reasoning)DeepSeekAgent · Stirrup | Stirrup | 699± 25 |
Claude 4 Sonnet (Reasoning)AnthropicAgent · Stirrup | Stirrup | 698± 22 |
Nemotron 3.5 LightningNVIDIAAgent · Stirrup | Stirrup | 646± 24 |
MiMo-V2-Flash (Non-reasoning)XiaomiAgent · Stirrup | Stirrup | 636± 24 |
Gemma 4 31B (Reasoning)GoogleAgent · Stirrup | Stirrup | 627± 20 |
Qwen3.5 35B A3B (Non-reasoning)AlibabaAgent · Stirrup | Stirrup | 616± 25 |
gpt-oss-120b (High)OpenAIAgent · Stirrup | Stirrup | 616± 19 |
GPT-5.4 mini (Non-reasoning)OpenAIAgent · Stirrup | Stirrup | 597± 20 |
Granite 4.2 30BIBMAgent · Stirrup | Stirrup | 583± 23 |
Gemma 4 26B A4B (Reasoning)GoogleAgent · Stirrup | Stirrup | 573± 19 |
Ling 3.0 TinyInclusionAIAgent · Stirrup | Stirrup | 568± 26 |
Gemma 4 31B (Non-reasoning)GoogleAgent · Stirrup | Stirrup | 555± 23 |
Devstral 2MistralAgent · Stirrup | Stirrup | 552± 21 |
Devstral Small 2MistralAgent · Stirrup | Stirrup | 535± 20 |
Command A+CohereAgent · Stirrup | Stirrup | 522± 22 |
Qwen3 Coder NextAlibabaAgent · Stirrup | Stirrup | 521± 23 |
GPT-5.5 Instant (June 2026)OpenAIAgent · Stirrup | Stirrup | 516± 21 |
Mercury 2InceptionAgent · Stirrup | Stirrup | 502± 23 |
Granite 4.2 8BIBMAgent · Stirrup | Stirrup | 499± 24 |
Nemotron 3 Super 120B A12B (Reasoning)NVIDIAAgent · Stirrup | Stirrup | 497± 19 |
Mercury 2.5InceptionAgent · Stirrup | Stirrup | 482± 25 |
Nova 2.0 Pro Preview (Medium)AmazonAgent · Stirrup | Stirrup | 477± 19 |
EXAONE 4.5 33B (Reasoning)LG AI ResearchAgent · Stirrup | Stirrup | 474± 24 |
Gemini 2.5 ProGoogleAgent · Stirrup | Stirrup | 463± 21 |
HyperNova 60B 2605 (High, Based on gpt-oss-120b)Multiverse ComputingAgent · Stirrup | Stirrup | 449± 23 |
Nova 2.0 Pro Preview (Low)AmazonAgent · Stirrup | Stirrup | 444± 20 |
Gemini 3.1 Flash-LiteGoogleAgent · Stirrup | Stirrup | 438± 19 |
Gemma 4 12B (Reasoning)GoogleAgent · Stirrup | Stirrup | 435± 24 |
Mistral Large 3MistralAgent · Stirrup | Stirrup | 428± 20 |
Qwen3.5 9B (Reasoning)AlibabaAgent · Stirrup | Stirrup | 427± 22 |
Mistral Medium 3.1MistralAgent · Stirrup | Stirrup | 393± 21 |
Mistral Small 3.1MistralAgent · Stirrup | Stirrup | 384± 21 |
K-EXAONE (Reasoning)LG AI ResearchAgent · Stirrup | Stirrup | 375± 24 |
Nova 2.0 Lite (High)AmazonAgent · Stirrup | Stirrup | 374± 20 |
Mistral Small 4 (Reasoning)MistralAgent · Stirrup | Stirrup | 370± 22 |
gpt-oss-20b (High)OpenAIAgent · Stirrup | Stirrup | 347± 19 |
Trinity Large ThinkingArcee AIAgent · Stirrup | Stirrup | 344± 24 |
Nova 2.0 Pro Preview (Non-reasoning)AmazonAgent · Stirrup | Stirrup | 339± 20 |
DiffusionGemma 26B A4BGoogleAgent · Stirrup | Stirrup | 329± 21 |
Ling 2.6 FlashInclusionAIAgent · Stirrup | Stirrup | 326± 23 |
Qwen3 235B A22B 2507 (Reasoning)AlibabaAgent · Stirrup | Stirrup | 321± 22 |
North Mini CodeCohereAgent · Stirrup | Stirrup | 319± 22 |
Celeris-1CelerisAgent · Stirrup | Stirrup | 309± 24 |
DeepSeek R1 (Jan '25)DeepSeekAgent · Stirrup | Stirrup | 302± 22 |
GPT-4.1 miniOpenAIAgent · Stirrup | Stirrup | 277± 22 |
Nemotron Cascade 2 30B A3BNVIDIAAgent · Stirrup | Stirrup | 272± 24 |
Solar Pro 3UpstageAgent · Stirrup | Stirrup | 272± 19 |
NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)NVIDIAAgent · Stirrup | Stirrup | 262± 20 |
Ministral 3 14BMistralAgent · Stirrup | Stirrup | 255± 20 |
o3-mini (High)OpenAIAgent · Stirrup | Stirrup | 238± 23 |
Nemotron 3 Nano Omni 30B A3B ReasoningNVIDIAAgent · Stirrup | Stirrup | 237± 24 |
Ministral 3 8BMistralAgent · Stirrup | Stirrup | 221± 20 |
Claude 3.5 HaikuAnthropicAgent · Stirrup | Stirrup | 218± 21 |
Granite 4.2 3BIBMAgent · Stirrup | Stirrup | 209± 25 |
Granite 4.1 30BIBMAgent · Stirrup | Stirrup | 195± 20 |
gpt-oss-120b (Low)OpenAIAgent · Stirrup | Stirrup | 179± 25 |
Magistral Medium 1.2MistralAgent · Stirrup | Stirrup | 176± 21 |
K2 Think V2MBZUAI Institute of Foundation ModelsAgent · Stirrup | Stirrup | 137± 22 |
Qwen3 Next 80B A3B (Reasoning)AlibabaAgent · Stirrup | Stirrup | 136± 23 |
DeepSeek V3 0324DeepSeekAgent · Stirrup | Stirrup | 77± 22 |
Qwen3 30B A3B 2507 (Reasoning)AlibabaAgent · Stirrup | Stirrup | 75± 23 |
Qwen3 32B (Reasoning)AlibabaAgent · Stirrup | Stirrup | 41± 21 |
Ministral 3 3BMistralAgent · Stirrup | Stirrup | 37± 21 |
Magistral Small 1.2MistralAgent · Stirrup | Stirrup | 12± 22 |
LFM2.5-2.6BLiquid AIAgent · Stirrup | Stirrup | -1± 26 |
GPT-4o (Nov '24)OpenAIAgent · Stirrup | Stirrup | -9± 29 |
GPT-4OpenAIAgent · Stirrup | Stirrup | -13± 22 |
GPT-4o miniOpenAIAgent · Stirrup | Stirrup | -15± 23 |
Qwen3 14B (Reasoning)AlibabaAgent · Stirrup | Stirrup | -17± 22 |
DeepSeek V3 (Dec '24)DeepSeekAgent · Stirrup | Stirrup | -21± 23 |
Gemma 4 E4B (Reasoning)GoogleAgent · Stirrup | Stirrup | -29± 25 |
Qwen3.5 2B (Reasoning)AlibabaAgent · Stirrup | Stirrup | -39± 24 |
Qwen3 8B (Reasoning)AlibabaAgent · Stirrup | Stirrup | -41± 23 |
NVIDIA Nemotron 3 Nano 4BNVIDIAAgent · Stirrup | Stirrup | -51± 24 |
Granite 4.1 3BIBMAgent · Stirrup | Stirrup | -138± 24 |
Llama 4 ScoutMetaAgent · Stirrup | Stirrup | -156± 21 |
Llama 3.3 Instruct 70BMetaAgent · Stirrup | Stirrup | -166± 22 |
Gemma 4 E2B (Reasoning)GoogleAgent · Stirrup | Stirrup | -181± 24 |
Mistral Small 3.2MistralAgent · Stirrup | Stirrup | -188± 23 |
GPT-4.1 nanoOpenAIAgent · Stirrup | Stirrup | -206± 22 |
Llama 4 MaverickMetaAgent · Stirrup | Stirrup | -268± 20 |
NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)NVIDIAAgent · Stirrup | Stirrup | -349± 20 |
Qwen3.5 2B (Non-reasoning)AlibabaAgent · Stirrup | Stirrup | -359± 21 |
MiniCPM-V 4.6 1.3BOpenBMBAgent · Stirrup | Stirrup | -362± 20 |
Qwen3.5 0.8B (Non-reasoning)AlibabaAgent · Stirrup | Stirrup | -363± 21 |
Llama 3.1 Instruct 8BMetaAgent · Stirrup | Stirrup | -380± 20 |
Qwen3.5 0.8B (Reasoning)AlibabaAgent · Stirrup | Stirrup | -381± 21 |
Nanbeige4.1-3BNanbeigeAgent · Stirrup | Stirrup | -397± 20 |
Gemma 3 27B InstructGoogleAgent · Stirrup | Stirrup | -401± 20 |
Gemma 3 12B InstructGoogleAgent · Stirrup | Stirrup | -401± 20 |
Phi-4 Mini InstructMicrosoftAgent · Stirrup | Stirrup | -401± 20 |
K2 Horizon 0.9BMBZUAI Institute of Foundation ModelsAgent · Stirrup | Stirrup | -402± 20 |