Knowledge / Benchmark profile
An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.
Observed results
20 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| Qwen3.7-Max | #1 | qwen-3-7-max | Alibaba Cloud | 89.6 | percent |
| Claude Opus 4.5 | #2 | claude-opus-4-5 | Anthropic | 89.5 | percent |
| Qwen3.6 Plus | #3 | qwen3-6-plus | Alibaba Cloud | 88.5 | percent |
| Qwen3.7-Plus | #4 | qwen-3-7-plus | Alibaba Cloud | 88.5 | percent |
| Qwen3.5 397B A17B | #5 | qwen3-5-397b | Alibaba Cloud | 87.8 | percent |
| DeepSeek V4 Pro | #6 | deepseek-v4-pro | DeepSeek | 87.5 | percent |
| Kimi K2.5 | #7 | kimi-k2-5 | Moonshot AI | 87.1 | percent |
| Nemotron 3 Ultra | #8 | nemotron-3-ultra | NVIDIA | 86.8 | percent |
| Qwen3.5-122B-A10B | #9 | qwen3-5-122b-a10b | Alibaba Cloud | 86.7 | percent |
| DeepSeek V4 Flash | #10 | deepseek-v4-flash | DeepSeek | 86.4 | percent |
| Qwen3.6 27B | #11 | qwen3-6-27b | Alibaba Cloud | 86.2 | percent |
| Qwen3.5 27B | #12 | qwen3-5-27b | Alibaba Cloud | 86.1 | percent |
| GLM-5 | #13 | glm-5 | Z.AI | 85.7 | percent |
| Qwen3.5-35B-A3B | #14 | qwen3-5-35b-a3b | Alibaba Cloud | 85.3 | percent |
| Gemma 4 31B | #15 | gemma-4-31b | 85.2 | percent | |
| Qwen3.6-35B-A3B | #16 | qwen3-6-35b-a3b | Alibaba Cloud | 85.2 | percent |
| MAI-Thinking-1 | #17 | mai-thinking-1 | Microsoft | 85 | percent |
| GLM-4.7 | #18 | glm-4-7 | Z.AI | 84.3 | percent |
| Qwen3 235B 2507 | #19 | qwen3-235b-2507 | Alibaba Cloud | 83 | percent |
| Gemma 4 26B A4B | #20 | gemma-4-26b-a4b | 82.6 | percent | |
| Claude Opus 4.6 | #21 | claude-opus-4-6 | Anthropic | 82 | percent |
| Exaone 4.0 32B | #22 | exaone-4-0-32b | LG AI Research | 81.8 | percent |
| Claude Sonnet 4.6 | #23 | claude-sonnet-4-6 | Anthropic | 79.2 | percent |
| Nemotron 3 Nano Omni 30B A3B | #24 | nemotron-3-nano-omni-30b-a3b | NVIDIA | 77.3 | percent |
| Gemma 4 12B Unified | #25 | gemma-4-12b | 77.2 | percent | |
| Celeris-1 | #26 | celeris-1 | Celeris | 75.9 | percent |
| DeepSeek V3 | #27 | deepseek-v3 | DeepSeek | 75.9 | percent |
| ZAYA1-8B | #28 | zaya1-8b | Zyphra | 74.2 | percent |
| DeepSeek V4 Pro Base | #29 | deepseek-v4-pro-base | DeepSeek | 73.5 | percent |
| Gemma 4 E4B | #30 | gemma-4-e4b | 69.4 | percent | |
| DeepSeek V4 Flash Base | #31 | deepseek-v4-flash-base | DeepSeek | 68.3 | percent |
| ZAYA1-74B-Preview | #32 | zaya1-74b-preview | Zyphra | 68.1 | percent |
| Gemma 4 E2B | #33 | gemma-4-e2b | 60 | percent | |
| Soofi S 30B-A3B | #34 | soofi-s-30b-a3b | Soofi Project | 51.4 | percent |
| MiniCPM5-1B | #35 | minicpm5-1b | OpenBMB | 48.85 | percent |
| LFM2.5-230M | #36 | lfm2-5-230m | LiquidAI | 20.25 | percent |
| LFM2.5-VL-450M | #37 | lfm2-5-vl-450m | LiquidAI | 19.32 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | Qwen3.7-MaxAlibaba Cloud | Closed weights | Source carrier | 89.6% |
| 02 | Claude Opus 4.5Anthropic | Closed weights | Source carrier | 89.5% |
| 03 | Qwen3.6 PlusAlibaba Cloud | Closed weights | Source carrier | 88.5% |
| 04 | Qwen3.7-PlusAlibaba Cloud | Closed weights | Source carrier | 88.5% |
| 05 | Qwen3.5 397B A17BAlibaba Cloud | Open weights | Source carrier | 87.8% |
| 06 | DeepSeek V4 ProDeepSeek | Open weights | Source carrier | 87.5% |
| 07 | Kimi K2.5Moonshot AI | Open weights | Source carrier | 87.1% |
| 08 | Nemotron 3 UltraNVIDIA | Open weights | Source carrier | 86.8% |
| 09 | Qwen3.5-122B-A10BAlibaba Cloud | Open weights | Source carrier | 86.7% |
| 10 | DeepSeek V4 FlashDeepSeek | Open weights | Source carrier | 86.4% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Exact BenchLM registry variant Qwen3.7 Max; bulk export does not retain a complete upstream harness configuration.Qwen3.7-MaxExact identitySource label without a registered configuration ID Canonical product: qwen-3-7-max | 89.6% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Opus 4.5; bulk export does not retain a complete upstream harness configuration.Claude Opus 4.5Exact identitySource label without a registered configuration ID Canonical product: claude-opus-4-5 | 89.5% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Qwen3.6 Plus; bulk export does not retain a complete upstream harness configuration.Qwen3.6 PlusExact identitySource label without a registered configuration ID Canonical product: qwen3-6-plus | 88.5% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Qwen3.7 Plus; bulk export does not retain a complete upstream harness configuration.Qwen3.7-PlusExact identitySource label without a registered configuration ID Canonical product: qwen-3-7-plus | 88.5% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Qwen3.5 397B; bulk export does not retain a complete upstream harness configuration.Qwen3.5 397B A17BExact identitySource label without a registered configuration ID Canonical product: qwen3-5-397b | 87.8% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant DeepSeek V4 Pro (Max); bulk export does not retain a complete upstream harness configuration.DeepSeek V4 ProExact identitydeepseek-v4-pro-max Canonical product: deepseek-v4-pro | 87.5% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant DeepSeek V4 Pro (High); bulk export does not retain a complete upstream harness configuration.DeepSeek V4 ProExact identitydeepseek-v4-pro-high Canonical product: deepseek-v4-pro | 87.1% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Kimi K2.5; bulk export does not retain a complete upstream harness configuration.Kimi K2.5Exact identitySource label without a registered configuration ID Canonical product: kimi-k2-5 | 87.1% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Kimi K2.5 (Reasoning); bulk export does not retain a complete upstream harness configuration.Kimi K2.5Exact identitykimi-k2-5-thinking Canonical product: kimi-k2-5 | 87.1% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Nemotron 3 Ultra; bulk export does not retain a complete upstream harness configuration.Nemotron 3 UltraExact identitySource label without a registered configuration ID Canonical product: nemotron-3-ultra | 86.8% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
From result to context
An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.
