Mathematics / Benchmark profile
Epoch AI's corrected v2 Tier 4 expansion, a separate set of exceptionally difficult research-level mathematics problems evaluated with Python-enabled iterative reasoning.
Observed results
20 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| GPT-5.6 Sol | #1 | gpt-5-6-sol | OpenAI | 83 | percent |
| GPT-5.6 Terra | #2 | gpt-5-6-terra | OpenAI | 68.3 | percent |
| GPT-5.6 Luna | #3 | gpt-5-6-luna | OpenAI | 58.5 | percent |
| GPT-5.5 | #4 | gpt-5-5 | OpenAI | 39.6 | percent |
| GPT-5.4 | #5 | gpt-5-4 | OpenAI | 37.5 | percent |
| Claude Opus 4.8 | #6 | claude-opus-4-8 | Anthropic | 31.25 | percent |
| GLM-5.3 | #7 | glm-5-3 | Z.AI | 29.268293 | percent |
| Claude Opus 4.7 | #8 | claude-opus-4-7 | Anthropic | 22.917 | percent |
| Claude Opus 4.6 | #9 | claude-opus-4-6 | Anthropic | 22.9 | percent |
| GPT-5.2 | #10 | gpt-5-2 | OpenAI | 18.8 | percent |
| Muse Spark | #11 | muse-spark | Meta | 14.6 | percent |
| Gemini 3.5 Flash | #12 | gemini-3-5-flash | 14.583 | percent | |
| Kimi K2.6 | #13 | kimi-k2-6 | Moonshot AI | 14.58 | percent |
| GLM-5.1 | #14 | glm-5-1 | Z.AI | 12.5 | percent |
| GPT-5.1 | #15 | gpt-5-1 | OpenAI | 12.5 | percent |
| Qwen3.6 Plus | #16 | qwen3-6-plus | Alibaba Cloud | 8.333 | percent |
| Claude Sonnet 4.6 | #17 | claude-sonnet-4-6 | Anthropic | 8.3 | percent |
| GPT-5.4 nano | #18 | gpt-5-4-nano | OpenAI | 6.25 | percent |
| o4-mini | #19 | o4-mini | OpenAI | 6.25 | percent |
| Kimi K2.5 | #20 | kimi-k2-5 | Moonshot AI | 4.2 | percent |
| Claude Opus 4.5 | #21 | claude-opus-4-5 | Anthropic | 4.167 | percent |
| Claude Sonnet 4.5 | #22 | claude-sonnet-4-5 | Anthropic | 4.167 | percent |
| Gemini 2.5 Flash | #23 | gemini-2-5-flash | 4.167 | percent | |
| Gemini 2.5 Pro | #24 | gemini-2-5-pro | 4.167 | percent | |
| Gemini 3 Flash | #25 | gemini-3-flash | 4.167 | percent | |
| Qwen 3.6 Max (preview) | #26 | qwen3-6-max-preview | Alibaba Cloud | 4.167 | percent |
| GLM-4.6 | #27 | glm-4-6 | Z.AI | 2.128 | percent |
| DeepSeek V3.2 | #28 | deepseek-v3-2 | DeepSeek | 2.1 | percent |
| GLM-5 | #29 | glm-5 | Z.AI | 2.1 | percent |
| Claude Haiku 4.5 | #30 | claude-haiku-4-5 | Anthropic | 2.083 | percent |
| Grok 4 | #31 | grok-4 | xAI | 2.083 | percent |
| o3 | #32 | o3 | OpenAI | 2.083 | percent |
| Qwen3.5 Plus | #33 | qwen3-5-plus | Alibaba Cloud | 2.083 | percent |
| GPT-5.4 mini | #34 | gpt-5-4-mini | OpenAI | 2.08 | percent |
| Claude 3.5 Sonnet | #35 | claude-3-5-sonnet | Anthropic | 0 | percent |
| GLM-4.7 | #36 | glm-4-7 | Z.AI | 0 | percent |
| GPT-4.1 | #37 | gpt-4-1 | OpenAI | 0 | percent |
| Grok 3 [Beta] | #38 | grok-3-beta | xAI | 0 | percent |
| Kimi K2 | #39 | kimi-k2 | Moonshot AI | 0 | percent |
| Qwen3 235B 2507 (Reasoning) | #40 | qwen3-235b-2507-reasoning | Alibaba Cloud | 0 | percent |
| Qwen3.5 Flash | #41 | qwen3-5-flash | Alibaba Cloud | 0 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | GPT-5.6 SolOpenAI | Closed weights | Source carrier | 83% |
| 02 | GPT-5.6 TerraOpenAI | Closed weights | Source carrier | 68.3% |
| 03 | GPT-5.6 LunaOpenAI | Closed weights | Source carrier | 58.5% |
| 04 | GPT-5.5OpenAI | Closed weights | Source carrier | 39.6% |
| 05 | GPT-5.4OpenAI | Closed weights | Source carrier | 37.5% |
| 06 | Claude Opus 4.8Anthropic | Closed weights | Source carrier | 31.3% |
| 07 | GLM-5.3Z.AI | Open announced | Public reference | 29.3% |
| 08 | Claude Opus 4.7Anthropic | Closed weights | Source carrier | 22.9% |
| 09 | Claude Opus 4.6Anthropic | Closed weights | Source carrier | 22.9% |
| 10 | GPT-5.2OpenAI | Closed weights | Source carrier | 18.8% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
GLM-5.3 (max)GLM-5.3Exact identityglm-5-3-max Canonical product: glm-5-3 | 29.3% | Public referenceindependently-verifiedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
Gemini 3.7 Flash (high)Gemini 3.7 FlashExact identitygemini-3-7-flash-high Canonical product: gemini-3-7-flash | 36.6% | Public referencesource-checkedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
Grok 4.6 (xhigh)Grok 4.6Exact identitygrok-4-6-xhigh Canonical product: grok-4-6 | 31.7% | Public referencesource-checkedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
Inkling Small (xhigh)Inkling-SmallExact identityinkling-small-epoch-inkling-small-xhigh Canonical product: inkling-small | 17.1% | Public referencesource-checkedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
Inkling (xhigh)InklingExact identityinkling-xhigh Canonical product: inkling | 4.9% | Public referencesource-checkedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
Qwen3.8 Max (xhigh)Qwen3.8 MaxExact identityqwen-3-8-max-xhigh Canonical product: qwen-3-8-max | 46.3% | Public referencesource-checkedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
DeepSeek V4 Flash 0731 (max)DeepSeek V4 Flash 0731Exact identitydeepseek-v4-flash-0731-epoch-deepseek-v4-flash-0731-max Canonical product: deepseek-v4-flash-0731 | 24.4% | Public referencesource-checkedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
Gemini 3.6 Flash (high)Gemini 3.6 FlashExact identitygemini-3-6-flash-high Canonical product: gemini-3-6-flash | 22.0% | Public referencesource-checkedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
Gemini 3.5 Flash-Lite (high)Gemini 3.5 Flash-LiteExact identitygemini-3-5-flash-lite-livebench-2026-06-25-high Canonical product: gemini-3-5-flash-lite | 0% | Public referencesource-checkedVersion & systemv2 / task implementation 2.0.0 Epoch AI benchmark runner | Epoch AI permanent refresh sourceObserved Checked |
Exact BenchLM registry variant GPT-5.6 Sol; bulk export does not retain a complete upstream harness configuration.GPT-5.6 SolExact identitySource label without a registered configuration ID Canonical product: gpt-5-6-sol | 83% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
From result to context
Epoch AI's corrected v2 Tier 4 expansion, a separate set of exceptionally difficult research-level mathematics problems evaluated with Python-enabled iterative reasoning.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.
