Bubble's BrainSearch⌘K
中文
Sign in
Benchmarks

General intelligence

GDPval-AA v2.1

Real-world knowledge work: producing usable documents, spreadsheets, slides and diagrams

What it measures

GDPval-AA v2.1 is an independent Artificial Analysis evaluation built on OpenAI’s GDPval dataset. It uses the 220-task public gold set across 44 occupations and nine industries, requiring actual work products rather than answers alone.

For example, an official sample asks for a one-page stage-plot PDF for a touring band: place the singers, drummer, amplifiers and microphones correctly, then list the input and output connections. Explaining how to draw the diagram is not enough; the model must produce a file whose content and layout meet the brief.

All models work through the Stirrup agent harness with shell access, file processing, web search and browsing, for up to 250 turns per task. Model identities are hidden and an LLM judge sampled from a three-model panel compares two submissions for the same task; these comparisons are aggregated into Elo ratings.

Reading the scores

Higher Elo means the deliverables are preferred more often in pairwise comparisons on this task set; DeepSeek V4.1 Flash (Max) is pinned at 1,600 as the anchor and every other rating is fitted relative to it. For example, ratings of 1,500 for A and 1,200 for B indicate stronger relative performance for A, not 150% accuracy or 25% greater efficiency. Elo can be negative, and the ± values are the published 95% confidence intervals; small ranking gaps should not be treated as certain differences. This snapshot preserves all 308 results and official model configurations checked on November 1, 2026. Entries labeled Fallback include a backup-model mechanism and are not results from a single model working entirely alone. V2.1 keeps v2’s tasks, tools and judging but changes the anchor and fits ratings with a Crowd-BT model, shifting all values, so it must not be mixed with the v2 snapshot. Ratings can change as new comparisons are added, and should not be mixed with v1, launch-day snapshots or OpenAI’s own GDPval win rates; since Intelligence Index v5.0, they are also a standalone evaluation that no longer counts toward that index.

SourceArtificial Analysis

Read the official GDPval-AA v2.1 methodology

Model performance

Checked 308 results

GDPval-AA v2.1 — Model performance. Select a column heading to sort.
Claude Opus 5.5 (Max, Default Fallback)AnthropicAgent · Stirrup
Stirrup1866± 26
Kimi K3.1 ProKimiAgent · Stirrup
Stirrup1858± 28
Muse Wonder Full (Max)MetaAgent · Stirrup
Stirrup1857± 25
Claude Sonnet 5.5 (Max, Default Fallback)AnthropicAgent · Stirrup
Stirrup1838± 23
Claude Opus 5.5 (Xhigh, Default Fallback)AnthropicAgent · Stirrup
Stirrup1835± 23
Muse Wonder (Max)MetaAgent · Stirrup
Stirrup1770± 23
Claude Fable 5.1 (Max, Default Fallback)AnthropicAgent · Stirrup
Stirrup1763± 19
DeepSeek V4.1 Flash 1017 (Max)DeepSeekAgent · Stirrup
Stirrup1746± 21
Claude Fable 5.1 (Xhigh, Default Fallback)AnthropicAgent · Stirrup
Stirrup1740± 19
Claude Sonnet 5.5 (Xhigh, Default Fallback)AnthropicAgent · Stirrup
Stirrup1735± 23
Gemini 4 Carbon (High)GoogleAgent · Stirrup
Stirrup1733± 24
Claude Opus 5 (Max)AnthropicAgent · Stirrup
Stirrup1730± 19
Grok 4.7 (Xhigh)SpaceXAIAgent · Stirrup
Stirrup1718± 21
Grok 4.7 (High)SpaceXAIAgent · Stirrup
Stirrup1716± 19
Claude Opus 5.5 (High, Default Fallback)AnthropicAgent · Stirrup
Stirrup1714± 21
Claude Opus 5 (Xhigh)AnthropicAgent · Stirrup
Stirrup1696± 18
MiMo-V2.6-ProXiaomiAgent · Stirrup
Stirrup1689± 21
Muse Spark 1.3 (Max)MetaAgent · Stirrup
Stirrup1681± 19
Qwen3.8 Max (0902)AlibabaAgent · Stirrup
Stirrup1674± 20
GLM-5.3 (Max)Z AIAgent · Stirrup
Stirrup1657± 19
Grok 4.6 (Xhigh)SpaceXAIAgent · Stirrup
Stirrup1654± 19
Ling 3.1 Flash 1019InclusionAIAgent · Stirrup
Stirrup1653± 25
GLM-5.3-FlashZ AIAgent · Stirrup
Stirrup1649± 17
Claude Fable 5.1 (High, Default Fallback)AnthropicAgent · Stirrup
Stirrup1644± 18
Qwen3.8-Flash-NextAlibabaAgent · Stirrup
Stirrup1636± 18
Muse Spark 1.3 (Xhigh)MetaAgent · Stirrup
Stirrup1632± 18
Claude Haiku 5.5 (Max)AnthropicAgent · Stirrup
Stirrup1631± 22
Gemini 4 Argon (High)GoogleAgent · Stirrup
Stirrup1631± 20
Ling 3.1 FlashInclusionAIAgent · Stirrup
Stirrup1626± 21
Grok 4.6 (High)SpaceXAIAgent · Stirrup
Stirrup1626± 16
Grok 4.6 (Medium)SpaceXAIAgent · Stirrup
Stirrup1620± 18
Claude Fable 5 (Max, Opus 4.8 Fallback)AnthropicAgent · Stirrup
Stirrup1618± 16
GPT-5.6 Sol (Max)OpenAIAgent · Stirrup
Stirrup1618± 16
Qwen3.8 2.4T A95BAlibabaAgent · Stirrup
Stirrup1616± 19
MiMo-V2.6-FlashXiaomiAgent · Stirrup
Stirrup1614± 21
Qwen3.8 MaxAlibabaAgent · Stirrup
Stirrup1611± 17
GPT-6.1 Sol (Max)OpenAIAgent · Stirrup
Stirrup1609± 22
Claude Opus 5 (High)AnthropicAgent · Stirrup
Stirrup1604± 18
DeepSeek V4.1 Flash (Max)DeepSeekAgent · Stirrup
Stirrup16001
Step 5 PreviewStepFunAgent · Stirrup
Stirrup1594± 23
Claude Opus 5.5 (Medium, Default Fallback)AnthropicAgent · Stirrup
Stirrup1593± 20
Grok 4.7 (Low)SpaceXAIAgent · Stirrup
Stirrup1591± 22
GPT-6 Astra (Max)OpenAIAgent · Stirrup
Stirrup1584± 19
GPT-5.6 Sol (Xhigh)OpenAIAgent · Stirrup
Stirrup1578± 16
GPT-6 Astra (Xhigh)OpenAIAgent · Stirrup
Stirrup1569± 19
Claude Sonnet 5.5 (High, Default Fallback)AnthropicAgent · Stirrup
Stirrup1557± 22
GPT-6.1 Sol (Xhigh)OpenAIAgent · Stirrup
Stirrup1555± 21
Claude Fable 5.1 (Medium, Default Fallback)AnthropicAgent · Stirrup
Stirrup1554± 18
DeepSeek V4 Flash Vision (Max)DeepSeekAgent · Stirrup
Stirrup1554± 17
Kimi K3 (Max)KimiAgent · Stirrup
Stirrup1543± 18
GPT-6 Astra (High)OpenAIAgent · Stirrup
Stirrup1536± 19
Claude Haiku 5.5 (Xhigh)AnthropicAgent · Stirrup
Stirrup1522± 22
GPT-6.1 Sol (High)OpenAIAgent · Stirrup
Stirrup1520± 21
GPT-6 Sol (Max)OpenAIAgent · Stirrup
Stirrup1516± 22
GPT-5.6 Sol (High)OpenAIAgent · Stirrup
Stirrup1512± 16
Claude Opus 5 (Medium)AnthropicAgent · Stirrup
Stirrup1499± 18
GPT-6 Astra (Medium)OpenAIAgent · Stirrup
Stirrup1498± 18
Muse Spark 1.2 (Xhigh)MetaAgent · Stirrup
Stirrup1480± 19
Claude Fable 5.1 (Low, Default Fallback)AnthropicAgent · Stirrup
Stirrup1477± 18
GPT-6.1 Terra (Max)OpenAIAgent · Stirrup
Stirrup1476± 23
Claude Sonnet 5 (Max)AnthropicAgent · Stirrup
Stirrup1470± 16
GPT-5.6 Luna (Max)OpenAIAgent · Stirrup
Stirrup1468± 16
GPT-6 Sol (Xhigh)OpenAIAgent · Stirrup
Stirrup1466± 22
MiniMax-M3.1-FlashMiniMaxAgent · Stirrup
Stirrup1465± 23
Claude Opus 4.8 (Max)AnthropicAgent · Stirrup
Stirrup1462± 15
GPT-5.6 Terra (Max)OpenAIAgent · Stirrup
Stirrup1461± 16
GPT-6.1 Sol (Medium)OpenAIAgent · Stirrup
Stirrup1459± 21
DeepSeek V4 Pro 0813 (Max)DeepSeekAgent · Stirrup
Stirrup1457± 19
GPT-5.6 Terra (Xhigh)OpenAIAgent · Stirrup
Stirrup1455± 16
Gemini 3.8 Flash (High)GoogleAgent · Stirrup
Stirrup1445± 19
GPT-6 Luna (Max)OpenAIAgent · Stirrup
Stirrup1444± 20
DeepSeek V4 Flash 0731 (Max)DeepSeekAgent · Stirrup
Stirrup1441± 19
Mistral Large 4 PreviewMistralAgent · Stirrup
Stirrup1433± 23
GPT-5.6 Sol (Medium)OpenAIAgent · Stirrup
Stirrup1432± 16
Gemini 3.8 Flash (Medium)GoogleAgent · Stirrup
Stirrup1431± 20
Qwen3.8 27B (Xhigh)AlibabaAgent · Stirrup
Stirrup1427± 20
Claude Haiku 5.5 (High)AnthropicAgent · Stirrup
Stirrup1424± 19
Grok 4.6 (Low)SpaceXAIAgent · Stirrup
Stirrup1418± 19
GPT-6.1 Terra (Xhigh)OpenAIAgent · Stirrup
Stirrup1415± 23
GPT-5.6 Luna (Xhigh)OpenAIAgent · Stirrup
Stirrup1408± 17
GPT-6 Sol (High)OpenAIAgent · Stirrup
Stirrup1404± 21
GPT-6 Astra (Low)OpenAIAgent · Stirrup
Stirrup1403± 20
Gemini 3.7 Flash (High)GoogleAgent · Stirrup
Stirrup1403± 19
Muse Spark Open (Xhigh)MetaAgent · Stirrup
Stirrup1401± 19
Grok 4.5 (High)SpaceXAIAgent · Stirrup
Stirrup1389± 19
Qwen3.8 27B (Medium)AlibabaAgent · Stirrup
Stirrup1389± 18
GPT-5.6 Terra (High)OpenAIAgent · Stirrup
Stirrup1387± 17
GLM-5.2 (Max)Z AIAgent · Stirrup
Stirrup1381± 16
GPT-6 Luna (Xhigh)OpenAIAgent · Stirrup
Stirrup1373± 21
Qwen3.8 27B (Low)AlibabaAgent · Stirrup
Stirrup1370± 19
Gemini 3.7 Flash (Medium)GoogleAgent · Stirrup
Stirrup1367± 20
Claude Sonnet 5 (Xhigh)AnthropicAgent · Stirrup
Stirrup1366± 17
K2 Horizon 375B A23BMBZUAI Institute of Foundation ModelsAgent · Stirrup
Stirrup1365± 19
Claude Opus 4.7 (Max)AnthropicAgent · Stirrup
Stirrup1362± 16
GPT-5.5 (Xhigh)OpenAIAgent · Stirrup
Stirrup1359± 16
GPT-6.1 Terra (High)OpenAIAgent · Stirrup
Stirrup1353± 23
GPT-6 Luna (High)OpenAIAgent · Stirrup
Stirrup1352± 21
GPT-6 Sol (Medium)OpenAIAgent · Stirrup
Stirrup1352± 22
DeepSeek V4.1 Flash (Non-reasoning)DeepSeekAgent · Stirrup
Stirrup1349± 20
GPT-5.6 Luna (High)OpenAIAgent · Stirrup
Stirrup1339± 17
GPT-5.5 (High)OpenAIAgent · Stirrup
Stirrup1336± 16
Agnes 2.5 Pro BetaSapiens AIAgent · Stirrup
Stirrup1335± 20
JT-4.1 Flash 236B A21B (Reasoning)China MobileAgent · Stirrup
Stirrup1335± 23
GLM-5.3 (Low)Z AIAgent · Stirrup
Stirrup1333± 20
Claude Sonnet 5.5 (Medium, Default Fallback)AnthropicAgent · Stirrup
Stirrup1329± 21
Gemini 3.7 Flash (Low)GoogleAgent · Stirrup
Stirrup1327± 20
GPT-6.1 Terra (Medium)OpenAIAgent · Stirrup
Stirrup1325± 22
GPT-6.1 Sol (Low)OpenAIAgent · Stirrup
Stirrup1322± 21
Gemini 3.8 Flash (Low)GoogleAgent · Stirrup
Stirrup1316± 17
GPT-5.6 Sol (Low)OpenAIAgent · Stirrup
Stirrup1313± 17
Claude Opus 5 (Low)AnthropicAgent · Stirrup
Stirrup1311± 20
Gemini 3.6 Flash (High)GoogleAgent · Stirrup
Stirrup1295± 19
GPT-6.1 Terra (Non-reasoning)OpenAIAgent · Stirrup
Stirrup1286± 23
Claude Haiku 5.5 (Medium)AnthropicAgent · Stirrup
Stirrup1283± 20
GPT-5.6 Terra (Medium)OpenAIAgent · Stirrup
Stirrup1274± 17
GPT-6 Luna (Medium)OpenAIAgent · Stirrup
Stirrup1270± 21
Claude Sonnet 5 (High)AnthropicAgent · Stirrup
Stirrup1263± 17
GPT-5.4 (Xhigh)OpenAIAgent · Stirrup
Stirrup1256± 16
GPT-6 Sol (Non-reasoning)OpenAIAgent · Stirrup
Stirrup1254± 21
GLM-5.2 (Non-reasoning)Z AIAgent · Stirrup
Stirrup1252± 21
MiniMax-M3MiniMaxAgent · Stirrup
Stirrup1250± 16
GPT-5.6 Sol (Non-reasoning)OpenAIAgent · Stirrup
Stirrup1250± 18
GPT-5.5 (Medium)OpenAIAgent · Stirrup
Stirrup1243± 18
Claude Sonnet 4.6 (Max)AnthropicAgent · Stirrup
Stirrup1241± 16
Claude Opus 5.5 (Low, Default Fallback)AnthropicAgent · Stirrup
Stirrup1240± 21
Claude Sonnet 5 (Non-reasoning, High)AnthropicAgent · Stirrup
Stirrup1221± 19
Muse Spark 1.1 (Xhigh)MetaAgent · Stirrup
Stirrup1217± 20
Gemini 3.5 Flash (High)GoogleAgent · Stirrup
Stirrup1212± 17
GPT-6.1 Terra (Low)OpenAIAgent · Stirrup
Stirrup1204± 23
GPT-6 Sol (Low)OpenAIAgent · Stirrup
Stirrup1201± 22
K2 Horizon MoVA 36B A4BMBZUAI Institute of Foundation ModelsAgent · Stirrup
Stirrup1197± 21
Apodex 1.1ApodexAgent · Stirrup
Stirrup1196± 20
Quasar 438B (Max, Based on GLM-5.2)Multiverse ComputingAgent · Stirrup
Stirrup1190± 20
Claude Sonnet 5.5 (Low, Default Fallback)AnthropicAgent · Stirrup
Stirrup1182± 22
Beam 1.0ReflectionAgent · Stirrup
Stirrup1177± 19
Ling-3.0-flash-VLInclusionAIAgent · Stirrup
Stirrup1169± 21
Claude Sonnet 5 (Medium)AnthropicAgent · Stirrup
Stirrup1167± 18
DeepSeek V4 Pro 0424 (Max)DeepSeekAgent · Stirrup
Stirrup1166± 17
Nemotron 3.5 Super VLNVIDIAAgent · Stirrup
Stirrup1162± 19
DeepSeek V4 Pro 0424 (High)DeepSeekAgent · Stirrup
Stirrup1160± 22
GPT-5.6 Luna (Medium)OpenAIAgent · Stirrup
Stirrup1138± 19
Kimi K3 (Low)KimiAgent · Stirrup
Stirrup1136± 22
Qwen3.7 MaxAlibabaAgent · Stirrup
Stirrup1135± 17
Inkling SmallThinking MachinesAgent · Stirrup
Stirrup1133± 22
Claude Haiku 5.5 (Low)AnthropicAgent · Stirrup
Stirrup1130± 22
MiMo-V2.5-Pro (Reasoning)XiaomiAgent · Stirrup
Stirrup1129± 17
JT-4.1 Flash 236B A21B (Non-reasoning)China MobileAgent · Stirrup
Stirrup1128± 21
Motif 3Motif TechnologiesAgent · Stirrup
Stirrup1127± 22
GLM-5.1 (Reasoning)Z AIAgent · Stirrup
Stirrup1125± 17
GPT-5.6 Terra (Low)OpenAIAgent · Stirrup
Stirrup1117± 18
Nex-N2-Pro (Based on Qwen3.5-397B-A17B)Nex AGIAgent · Stirrup
Stirrup1109± 19
G9v3-39A5BAI9StarsAgent · Stirrup
Stirrup1109± 23
Motif 3 (Beta)Motif TechnologiesAgent · Stirrup
Stirrup1109± 22
Ling-3.0-flash-FinInclusionAIAgent · Stirrup
Stirrup1108± 22
Solar Pro 4UpstageAgent · Stirrup
Stirrup1106± 20
GPT-5.6 Terra (Non-reasoning)OpenAIAgent · Stirrup
Stirrup1105± 19
Solar Mini 4UpstageAgent · Stirrup
Stirrup1092± 23
Qwen3.8 27B (Non-reasoning)AlibabaAgent · Stirrup
Stirrup1082± 20
Inkling (Xhigh)Thinking MachinesAgent · Stirrup
Stirrup1080± 22
Claude Sonnet 5 (Low)AnthropicAgent · Stirrup
Stirrup1076± 18
Grok Build 0.1 0616SpaceXAIAgent · Stirrup
Stirrup1073± 18
Hy3TencentAgent · Stirrup
Stirrup1071± 22
DeepSeek V4 Pro 0813 (Non-reasoning)DeepSeekAgent · Stirrup
Stirrup1066± 22
GPT-5.5 (Low)OpenAIAgent · Stirrup
Stirrup1053± 19
GPT-6 Luna (Non-reasoning)OpenAIAgent · Stirrup
Stirrup1052± 22
Kimi K2.6 (Reasoning)KimiAgent · Stirrup
Stirrup1047± 17
Kimi K2.7 CodeKimiAgent · Stirrup
Stirrup1046± 17
DeepSeek V4 Flash 0420 (Max)DeepSeekAgent · Stirrup
Stirrup1046± 18
GPT-6 Luna (Low)OpenAIAgent · Stirrup
Stirrup1040± 22
K2 Horizon 7BMBZUAI Institute of Foundation ModelsAgent · Stirrup
Stirrup1024± 22
GPT-5.4 mini (Xhigh)OpenAIAgent · Stirrup
Stirrup1022± 17
GLM-4.7 (Reasoning)Z AIAgent · Stirrup
Stirrup1021± 20
Nemotron 3 Ultra 550B A55B (Reasoning)NVIDIAAgent · Stirrup
Stirrup1021± 17
MiniMax-M2.7MiniMaxAgent · Stirrup
Stirrup1016± 17
DeepSeek V4 Flash 0420 (High)DeepSeekAgent · Stirrup
Stirrup1009± 23
Muse SparkMetaAgent · Stirrup
Stirrup1006± 17
MiMo-V2.5XiaomiAgent · Stirrup
Stirrup1005± 22
GPT-5.6 Luna (Low)OpenAIAgent · Stirrup
Stirrup1002± 19
Qwen3.6 PlusAlibabaAgent · Stirrup
Stirrup997± 17
Gemini 3.5 Flash-LiteGoogleAgent · Stirrup
Stirrup995± 22
Qwen3.6 27B (Reasoning)AlibabaAgent · Stirrup
Stirrup993± 17
Nemotron 3.5 Bolt 74B-A9BNVIDIAAgent · Stirrup
Stirrup979± 19
Solar Open2 250BUpstageAgent · Stirrup
Stirrup978± 24
GPT-5.5 (Non-reasoning)OpenAIAgent · Stirrup
Stirrup978± 17
Qwen3.6 27B (Non-reasoning)AlibabaAgent · Stirrup
Stirrup969± 21
A.X-K2SK TelecomAgent · Stirrup
Stirrup968± 23
Ling 3.0 FlashInclusionAIAgent · Stirrup
Stirrup957± 23
GPT-5.4 nano (Xhigh)OpenAIAgent · Stirrup
Stirrup955± 17
Grok 4.3 (Non-reasoning)SpaceXAIAgent · Stirrup
Stirrup949± 18
Grok 4.3 (High)SpaceXAIAgent · Stirrup
Stirrup946± 18
GPT-5.6 Luna (Non-reasoning)OpenAIAgent · Stirrup
Stirrup928± 20
GPT-5 (High)OpenAIAgent · Stirrup
Stirrup925± 21
Claude 4.5 Sonnet (Reasoning)AnthropicAgent · Stirrup
Stirrup915± 21
Qwen3.6 35B A3B (Reasoning)AlibabaAgent · Stirrup
Stirrup899± 18
K2 Horizon 3.7BMBZUAI Institute of Foundation ModelsAgent · Stirrup
Stirrup889± 24
LongCat 2.0LongCatAgent · Stirrup
Stirrup877± 21
Step 3.7 FlashStepFunAgent · Stirrup
Stirrup864± 18
Qwen3.6 35B A3B (Non-reasoning)AlibabaAgent · Stirrup
Stirrup854± 24
Kimi K2.5 (Reasoning)KimiAgent · Stirrup
Stirrup847± 23
GPT-5.1 (High)OpenAIAgent · Stirrup
Stirrup834± 22
Qwen3.5 122B A10B (Reasoning)AlibabaAgent · Stirrup
Stirrup822± 18
K-EXAONE 2.0 0803LG AI ResearchAgent · Stirrup
Stirrup816± 25
Qwen3.5 397B A17B (Reasoning)AlibabaAgent · Stirrup
Stirrup801± 19
Gemini 3.1 Pro PreviewGoogleAgent · Stirrup
Stirrup799± 19
Muse Glimmer (High)MetaAgent · Stirrup
Stirrup792± 23
Qwen3.7 PlusAlibabaAgent · Stirrup
Stirrup775± 19
GPT-5 mini (High)OpenAIAgent · Stirrup
Stirrup773± 22
Mistral Medium 3.5MistralAgent · Stirrup
Stirrup769± 19
GLM-4.6 (Reasoning)Z AIAgent · Stirrup
Stirrup764± 22
Ring-2.6-1TInclusionAIAgent · Stirrup
Stirrup753± 19
KAT Coder Pro V2KwaiKATAgent · Stirrup
Stirrup742± 23
Claude 4.5 Haiku (Reasoning)AnthropicAgent · Stirrup
Stirrup740± 19
KAT-Coder-Pro V1KwaiKATAgent · Stirrup
Stirrup738± 21
DeepSeek V3.1 Terminus (Reasoning)DeepSeekAgent · Stirrup
Stirrup718± 24
Qwen3.5 122B A10B (Non-reasoning)AlibabaAgent · Stirrup
Stirrup716± 23
MiniCPM5-2BOpenBMBAgent · Stirrup
Stirrup715± 25
DeepSeek V3.2 (Reasoning)DeepSeekAgent · Stirrup
Stirrup699± 25
Claude 4 Sonnet (Reasoning)AnthropicAgent · Stirrup
Stirrup698± 22
Nemotron 3.5 LightningNVIDIAAgent · Stirrup
Stirrup646± 24
MiMo-V2-Flash (Non-reasoning)XiaomiAgent · Stirrup
Stirrup636± 24
Gemma 4 31B (Reasoning)GoogleAgent · Stirrup
Stirrup627± 20
Qwen3.5 35B A3B (Non-reasoning)AlibabaAgent · Stirrup
Stirrup616± 25
gpt-oss-120b (High)OpenAIAgent · Stirrup
Stirrup616± 19
GPT-5.4 mini (Non-reasoning)OpenAIAgent · Stirrup
Stirrup597± 20
Granite 4.2 30BIBMAgent · Stirrup
Stirrup583± 23
Gemma 4 26B A4B (Reasoning)GoogleAgent · Stirrup
Stirrup573± 19
Ling 3.0 TinyInclusionAIAgent · Stirrup
Stirrup568± 26
Gemma 4 31B (Non-reasoning)GoogleAgent · Stirrup
Stirrup555± 23
Devstral 2MistralAgent · Stirrup
Stirrup552± 21
Devstral Small 2MistralAgent · Stirrup
Stirrup535± 20
Command A+CohereAgent · Stirrup
Stirrup522± 22
Qwen3 Coder NextAlibabaAgent · Stirrup
Stirrup521± 23
GPT-5.5 Instant (June 2026)OpenAIAgent · Stirrup
Stirrup516± 21
Mercury 2InceptionAgent · Stirrup
Stirrup502± 23
Granite 4.2 8BIBMAgent · Stirrup
Stirrup499± 24
Nemotron 3 Super 120B A12B (Reasoning)NVIDIAAgent · Stirrup
Stirrup497± 19
Mercury 2.5InceptionAgent · Stirrup
Stirrup482± 25
Nova 2.0 Pro Preview (Medium)AmazonAgent · Stirrup
Stirrup477± 19
EXAONE 4.5 33B (Reasoning)LG AI ResearchAgent · Stirrup
Stirrup474± 24
Gemini 2.5 ProGoogleAgent · Stirrup
Stirrup463± 21
HyperNova 60B 2605 (High, Based on gpt-oss-120b)Multiverse ComputingAgent · Stirrup
Stirrup449± 23
Nova 2.0 Pro Preview (Low)AmazonAgent · Stirrup
Stirrup444± 20
Gemini 3.1 Flash-LiteGoogleAgent · Stirrup
Stirrup438± 19
Gemma 4 12B (Reasoning)GoogleAgent · Stirrup
Stirrup435± 24
Mistral Large 3MistralAgent · Stirrup
Stirrup428± 20
Qwen3.5 9B (Reasoning)AlibabaAgent · Stirrup
Stirrup427± 22
Mistral Medium 3.1MistralAgent · Stirrup
Stirrup393± 21
Mistral Small 3.1MistralAgent · Stirrup
Stirrup384± 21
K-EXAONE (Reasoning)LG AI ResearchAgent · Stirrup
Stirrup375± 24
Nova 2.0 Lite (High)AmazonAgent · Stirrup
Stirrup374± 20
Mistral Small 4 (Reasoning)MistralAgent · Stirrup
Stirrup370± 22
gpt-oss-20b (High)OpenAIAgent · Stirrup
Stirrup347± 19
Trinity Large ThinkingArcee AIAgent · Stirrup
Stirrup344± 24
Nova 2.0 Pro Preview (Non-reasoning)AmazonAgent · Stirrup
Stirrup339± 20
DiffusionGemma 26B A4BGoogleAgent · Stirrup
Stirrup329± 21
Ling 2.6 FlashInclusionAIAgent · Stirrup
Stirrup326± 23
Qwen3 235B A22B 2507 (Reasoning)AlibabaAgent · Stirrup
Stirrup321± 22
North Mini CodeCohereAgent · Stirrup
Stirrup319± 22
Celeris-1CelerisAgent · Stirrup
Stirrup309± 24
DeepSeek R1 (Jan '25)DeepSeekAgent · Stirrup
Stirrup302± 22
GPT-4.1 miniOpenAIAgent · Stirrup
Stirrup277± 22
Nemotron Cascade 2 30B A3BNVIDIAAgent · Stirrup
Stirrup272± 24
Solar Pro 3UpstageAgent · Stirrup
Stirrup272± 19
NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)NVIDIAAgent · Stirrup
Stirrup262± 20
Ministral 3 14BMistralAgent · Stirrup
Stirrup255± 20
o3-mini (High)OpenAIAgent · Stirrup
Stirrup238± 23
Nemotron 3 Nano Omni 30B A3B ReasoningNVIDIAAgent · Stirrup
Stirrup237± 24
Ministral 3 8BMistralAgent · Stirrup
Stirrup221± 20
Claude 3.5 HaikuAnthropicAgent · Stirrup
Stirrup218± 21
Granite 4.2 3BIBMAgent · Stirrup
Stirrup209± 25
Granite 4.1 30BIBMAgent · Stirrup
Stirrup195± 20
gpt-oss-120b (Low)OpenAIAgent · Stirrup
Stirrup179± 25
Magistral Medium 1.2MistralAgent · Stirrup
Stirrup176± 21
K2 Think V2MBZUAI Institute of Foundation ModelsAgent · Stirrup
Stirrup137± 22
Qwen3 Next 80B A3B (Reasoning)AlibabaAgent · Stirrup
Stirrup136± 23
DeepSeek V3 0324DeepSeekAgent · Stirrup
Stirrup77± 22
Qwen3 30B A3B 2507 (Reasoning)AlibabaAgent · Stirrup
Stirrup75± 23
Qwen3 32B (Reasoning)AlibabaAgent · Stirrup
Stirrup41± 21
Ministral 3 3BMistralAgent · Stirrup
Stirrup37± 21
Magistral Small 1.2MistralAgent · Stirrup
Stirrup12± 22
LFM2.5-2.6BLiquid AIAgent · Stirrup
Stirrup-1± 26
GPT-4o (Nov '24)OpenAIAgent · Stirrup
Stirrup-9± 29
GPT-4OpenAIAgent · Stirrup
Stirrup-13± 22
GPT-4o miniOpenAIAgent · Stirrup
Stirrup-15± 23
Qwen3 14B (Reasoning)AlibabaAgent · Stirrup
Stirrup-17± 22
DeepSeek V3 (Dec '24)DeepSeekAgent · Stirrup
Stirrup-21± 23
Gemma 4 E4B (Reasoning)GoogleAgent · Stirrup
Stirrup-29± 25
Qwen3.5 2B (Reasoning)AlibabaAgent · Stirrup
Stirrup-39± 24
Qwen3 8B (Reasoning)AlibabaAgent · Stirrup
Stirrup-41± 23
NVIDIA Nemotron 3 Nano 4BNVIDIAAgent · Stirrup
Stirrup-51± 24
Granite 4.1 3BIBMAgent · Stirrup
Stirrup-138± 24
Llama 4 ScoutMetaAgent · Stirrup
Stirrup-156± 21
Llama 3.3 Instruct 70BMetaAgent · Stirrup
Stirrup-166± 22
Gemma 4 E2B (Reasoning)GoogleAgent · Stirrup
Stirrup-181± 24
Mistral Small 3.2MistralAgent · Stirrup
Stirrup-188± 23
GPT-4.1 nanoOpenAIAgent · Stirrup
Stirrup-206± 22
Llama 4 MaverickMetaAgent · Stirrup
Stirrup-268± 20
NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)NVIDIAAgent · Stirrup
Stirrup-349± 20
Qwen3.5 2B (Non-reasoning)AlibabaAgent · Stirrup
Stirrup-359± 21
MiniCPM-V 4.6 1.3BOpenBMBAgent · Stirrup
Stirrup-362± 20
Qwen3.5 0.8B (Non-reasoning)AlibabaAgent · Stirrup
Stirrup-363± 21
Llama 3.1 Instruct 8BMetaAgent · Stirrup
Stirrup-380± 20
Qwen3.5 0.8B (Reasoning)AlibabaAgent · Stirrup
Stirrup-381± 21
Nanbeige4.1-3BNanbeigeAgent · Stirrup
Stirrup-397± 20
Gemma 3 27B InstructGoogleAgent · Stirrup
Stirrup-401± 20
Gemma 3 12B InstructGoogleAgent · Stirrup
Stirrup-401± 20
Phi-4 Mini InstructMicrosoftAgent · Stirrup
Stirrup-401± 20
K2 Horizon 0.9BMBZUAI Institute of Foundation ModelsAgent · Stirrup
Stirrup-402± 20
  1. The GDPval-AA v2.1 Elo anchor: DeepSeek V4.1 Flash (Max) is fixed at 1600, so it has no confidence interval.source↩

⌘KSearch the knowledge base

RECENTLY ADDED

Loading the knowledge index…