A task with its own rules.
A display-only Artificial Analysis knowledge metric for the proportion of correctly answered questions.
- Organisation
- Artificial Analysis
- Version
- 2026
Loading benchmark header
Loading benchmark registry
Knowledge / Benchmark profile
A display-only Artificial Analysis knowledge metric for the proportion of correctly answered questions.
Observed results
20 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| Claude Fable 5 | #1 | claude-fable-5 | Anthropic | 61.4 | percent |
| GPT-5.6 Sol | #2 | gpt-5-6-sol | OpenAI | 58.5 | percent |
| GPT-5.5 | #3 | gpt-5-5 | OpenAI | 56.9 | percent |
| Claude Opus 5 | #4 | claude-opus-5 | Anthropic | 54.2 | percent |
| Grok 4.5 | #5 | grok-4-5 | xAI | 52.1 | percent |
| Gemini 3.5 Flash | #6 | gemini-3-5-flash | 51.9 | percent | |
| GPT-5.3-Codex | #7 | gpt-5-3-codex | OpenAI | 51.8 | percent |
| Gemini 3.6 Flash | #8 | gemini-3-6-flash | 50.2 | percent | |
| GPT-5.4 | #9 | gpt-5-4 | OpenAI | 50 | percent |
| Claude Opus 4.8 | #10 | claude-opus-4-8 | Anthropic | 46.6 | percent |
| Claude Opus 4.6 | #11 | claude-opus-4-6 | Anthropic | 46.4 | percent |
| Kimi K3 | #12 | kimi-k3 | Moonshot AI | 46 | percent |
| GPT-5.6 Terra | #13 | gpt-5-6-terra | OpenAI | 45.9 | percent |
| Claude Opus 4.7 | #14 | claude-opus-4-7 | Anthropic | 45.8 | percent |
| Claude Opus 4.5 Thinking | #15 | claude-opus-4-5-thinking | Anthropic | 45.7 | percent |
| Gemini 3 Flash | #16 | gemini-3-flash | 45.5 | percent | |
| Muse Spark | #17 | muse-spark | Meta | 44.6 | percent |
| GPT-5.2 | #18 | gpt-5-2 | OpenAI | 43.8 | percent |
| DeepSeek V4 Pro | #19 | deepseek-v4-pro | DeepSeek | 43.3 | percent |
| GPT-5.6 Luna | #20 | gpt-5-6-luna | OpenAI | 41.5 | percent |
| Grok 4 | #21 | grok-4 | xAI | 41.4 | percent |
| Claude Opus 4.5 | #22 | claude-opus-4-5 | Anthropic | 40.7 | percent |
| GPT-5 | #23 | gpt-5 | OpenAI | 40.7 | percent |
| GPT-5.2 Codex | #24 | gpt-5-2-codex | OpenAI | 40.7 | percent |
| Muse Spark 1.1 | #25 | muse-spark-1-1 | Meta | 40.6 | percent |
| Inkling | #26 | inkling | Thinking Machines Lab | 40 | percent |
| GPT-5.1-Codex | #27 | gpt-5-1-codex | OpenAI | 39.2 | percent |
| GPT-5.1-Codex-Max | #28 | gpt-5-1-codex-max | OpenAI | 39.2 | percent |
| Gemini 2.5 Pro | #29 | gemini-2-5-pro | 39 | percent | |
| Kimi K2.7 Code | #30 | kimi-k2-7-code | Moonshot AI | 38.6 | percent |
| o3 | #31 | o3 | OpenAI | 38.4 | percent |
| Claude Sonnet 5 | #32 | claude-sonnet-5 | Anthropic | 38.3 | percent |
| Claude Sonnet 4.6 | #33 | claude-sonnet-4-6 | Anthropic | 38 | percent |
| Qwen 3.6 Max (preview) | #34 | qwen3-6-max-preview | Alibaba Cloud | 37.7 | percent |
| GPT-5.1 | #35 | gpt-5-1 | OpenAI | 37.6 | percent |
| GPT-5.4 mini | #36 | gpt-5-4-mini | OpenAI | 37.5 | percent |
| DeepSeek V4 Flash | #37 | deepseek-v4-flash | DeepSeek | 37.2 | percent |
| Gemini 3.1 Flash-Lite | #38 | gemini-3-1-flash-lite | 36.4 | percent | |
| o1 | #39 | o1 | OpenAI | 34.7 | percent |
| Grok 4.3 | #40 | grok-4-3 | xAI | 34.6 | percent |
| Kimi K2.5 | #41 | kimi-k2-5 | Moonshot AI | 34.3 | percent |
| Kimi K2.6 | #42 | kimi-k2-6 | Moonshot AI | 32.8 | percent |
| Hy3 | #43 | hy3 | Tencent | 31.5 | percent |
| Hy3 Preview | #44 | hy3-preview | Tencent | 31.5 | percent |
| Qwen3.5 397B A17B | #45 | qwen3-5-397b | Alibaba Cloud | 31.4 | percent |
| DeepSeek-R1 | #46 | deepseek-r1 | DeepSeek | 31 | percent |
| Gemini 3.5 Flash-Lite | #47 | gemini-3-5-flash-lite | 30.3 | percent | |
| Qwen3.7-Max | #48 | qwen-3-7-max | Alibaba Cloud | 30.1 | percent |
| GLM-4.7 | #49 | glm-4-7 | Z.AI | 29.3 | percent |
| GLM-5V-Turbo | #50 | glm-5v-turbo | Z.AI | 29.1 | percent |
| GLM-5-Turbo | #51 | glm-5-turbo | Z.AI | 29 | percent |
| DeepSeek V3.1 | #52 | deepseek-v3-1 | DeepSeek | 28.8 | percent |
| GLM-5 | #53 | glm-5 | Z.AI | 26.9 | percent |
| Kimi K2 | #54 | kimi-k2 | Moonshot AI | 26.8 | percent |
| MiMo-V2-Pro | #55 | mimo-v2-pro | Xiaomi | 26.8 | percent |
| Gemini 2.5 Flash | #56 | gemini-2-5-flash | 26.5 | percent | |
| Qwen3.6 Plus | #57 | qwen3-6-plus | Alibaba Cloud | 26.2 | percent |
| MiniMax M2.7 | #58 | minimax-m2-7 | MiniMax | 26.1 | percent |
| DeepSeek V3 | #59 | deepseek-v3 | DeepSeek | 25.4 | percent |
| GPT-5.4 nano | #60 | gpt-5-4-nano | OpenAI | 25.4 | percent |
| Step 3.7 Flash | #61 | step-3-7-flash | StepFun | 25.4 | percent |
| Grok 4.1 Fast (Reasoning) | #62 | grok-4-1-fast-reasoning | xAI | 25.3 | percent |
| GLM-5.2 | #63 | glm-5-2 | Z.AI | 25.1 | percent |
| Mistral Medium 3.5 128B | #64 | mistral-medium-3-5-128b | Mistral AI | 25.1 | percent |
| Qwen3.5-122B-A10B | #65 | qwen3-5-122b-a10b | Alibaba Cloud | 24.7 | percent |
| Qwen3 Max | #66 | qwen3-max | Alibaba Cloud | 24.4 | percent |
| Llama 4 Maverick | #67 | llama-4-maverick | Meta | 24.3 | percent |
| DeepSeek V3.2 | #68 | deepseek-v3-2 | DeepSeek | 24.2 | percent |
| GLM-5.1 | #69 | glm-5-1 | Z.AI | 24.2 | percent |
| GPT-4.1 | #70 | gpt-4-1 | OpenAI | 24.2 | percent |
| Mistral Large 3 | #71 | mistral-large-3 | Mistral AI | 24.1 | percent |
| Grok Code Fast 1 | #72 | grok-code-fast-1 | xAI | 23.8 | percent |
| Trinity-Large-Preview | #73 | trinity-large-preview | Arcee AI | 22.8 | percent |
| Trinity-Large-Thinking | #74 | trinity-large-thinking | Arcee AI | 22.8 | percent |
| Grok 4 Fast (Reasoning) | #75 | grok-4-fast-reasoning | xAI | 22.6 | percent |
| MiMo-V2.5-Pro | #76 | mimo-v2-5-pro | Xiaomi | 22.6 | percent |
| Claude 4 Sonnet | #77 | claude-4-sonnet | Anthropic | 22.4 | percent |
| Llama 3.1 405B | #78 | llama-3-1-405b | Meta | 22.3 | percent |
| Qwen3.7-Plus | #79 | qwen-3-7-plus | Alibaba Cloud | 22.2 | percent |
| Mistral Small 4 | #80 | mistral-small-4 | Mistral AI | 22.1 | percent |
| Nemotron 3 Ultra | #81 | nemotron-3-ultra | NVIDIA | 21.6 | percent |
| GPT-OSS 120B | #82 | gpt-oss-120b | OpenAI | 21.5 | percent |
| Qwen3.5 27B | #83 | qwen3-5-27b | Alibaba Cloud | 21 | percent |
| GLM-4.6 | #84 | glm-4-6 | Z.AI | 20.8 | percent |
| Qwen3.5-35B-A3B | #85 | qwen3-5-35b-a3b | Alibaba Cloud | 20.5 | percent |
| Mistral Large 2 | #86 | mistral-large-2 | Mistral AI | 20.1 | percent |
| Gemma 4 31B | #87 | gemma-4-31b | 19.9 | percent | |
| Nemotron Ultra 253B | #88 | nemotron-ultra-253b | NVIDIA | 19.9 | percent |
| GPT-4o | #89 | gpt-4o | OpenAI | 19.7 | percent |
| Qwen3.6 27B | #90 | qwen3-6-27b | Alibaba Cloud | 19.2 | percent |
| Qwen3.6-35B-A3B | #91 | qwen3-6-35b-a3b | Alibaba Cloud | 18.9 | percent |
| MiMo-V2-Omni | #92 | mimo-v2-omni | Xiaomi | 18.7 | percent |
| Mistral Medium 3 | #93 | mistral-medium-3 | Mistral AI | 18.3 | percent |
| Gemma 4 26B A4B | #94 | gemma-4-26b-a4b | 18.2 | percent | |
| Sarvam 105B | #95 | sarvam-105b | Sarvam | 17.6 | percent |
| GPT-4.1 mini | #96 | gpt-4-1-mini | OpenAI | 17.5 | percent |
| Claude 3 Haiku | #97 | claude-3-haiku | Anthropic | 17.2 | percent |
| Nemotron 3 Nano 30B | #98 | nemotron-3-nano-30b | NVIDIA | 17.1 | percent |
| Grok 4.1 Fast | #99 | grok-4-1-fast | xAI | 17 | percent |
| Nova Pro | #100 | nova-pro | Amazon | 17 | percent |
| K-Exaone | #101 | k-exaone | LG AI Research | 16.5 | percent |
| Gemma 4 12B Unified | #102 | gemma-4-12b | 16 | percent | |
| Solar Pro 2 | #103 | solar-pro-2 | Upstage | 15.6 | percent |
| GLM-4.5-Air | #104 | glm-4-5-air | Z.AI | 15.5 | percent |
| GPT-OSS 20B | #105 | gpt-oss-20b | OpenAI | 15.5 | percent |
| Ling 2.6 Flash | #106 | ling-2-6-flash | InclusionAI | 15.4 | percent |
| MiniMax M3 | #107 | minimax-m3 | MiniMax | 15 | percent |
| Nemotron 3 Nano Omni 30B A3B | #108 | nemotron-3-nano-omni-30b-a3b | NVIDIA | 14.8 | percent |
| Llama 4 Scout | #109 | llama-4-scout | Meta | 14.6 | percent |
| GPT-4.1 nano | #110 | gpt-4-1-nano | OpenAI | 13.3 | percent |
| Phi-4 | #111 | phi-4 | Microsoft | 13.2 | percent |
| Sarvam 30B | #112 | sarvam-30b | Sarvam | 12.7 | percent |
| Gemma 3 27B | #113 | gemma-3-27b | 12.5 | percent | |
| Exaone 4.0 32B | #114 | exaone-4-0-32b | LG AI Research | 10.4 | percent |
| LFM2.5-8B-A1B | #115 | lfm2-5-8b-a1b | LiquidAI | 9.4 | percent |
| Command A+ | #116 | command-a-plus | Cohere | 8.9 | percent |
| Gemma 4 E4B | #117 | gemma-4-e4b | 8.6 | percent | |
| Gemma 4 E2B | #118 | gemma-4-e2b | 6.7 | percent | |
| Granite-4.0-1B | #119 | granite-4-0-1b | IBM | 6.1 | percent |
| Granite-4.0-H-1B | #120 | granite-4-0-h-1b | IBM | 5.3 | percent |
| LFM2.5-VL-1.6B-Extract | #121 | lfm2-5-vl-1-6b-extract | LiquidAI | 5.2 | percent |
| Exaone 4.0 1.2B | #122 | exaone-4-0-1-2b | LG AI Research | 4.7 | percent |
| Granite-4.0-H-350M | #123 | granite-4-0-h-350m | IBM | 3.7 | percent |
| Granite-4.0-350M | #124 | granite-4-0-350m | IBM | 3.2 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | Claude Fable 5Anthropic | Closed weights | Source carrier | 61.4% |
| 02 | GPT-5.6 SolOpenAI | Closed weights | Source carrier | 58.5% |
| 03 | GPT-5.5OpenAI | Closed weights | Source carrier | 56.9% |
| 04 | Claude Opus 5Anthropic | Closed weights | Source carrier | 54.2% |
| 05 | Grok 4.5xAI | Closed weights | Source carrier | 52.1% |
| 06 | Gemini 3.5 FlashGoogle | Closed weights | Source carrier | 51.9% |
| 07 | GPT-5.3-CodexOpenAI | Closed weights | Source carrier | 51.8% |
| 08 | Gemini 3.6 FlashGoogle | Closed weights | Source carrier | 50.2% |
| 09 | GPT-5.4OpenAI | Closed weights | Source carrier | 50% |
| 10 | Claude Opus 4.8Anthropic | Closed weights | Source carrier | 46.6% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Exact BenchLM registry variant Claude Fable 5; bulk export does not retain a complete upstream harness configuration.Claude Fable 5Exact identitySource label without a registered configuration ID Canonical product: claude-fable-5 | 61.4% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.6 Sol; bulk export does not retain a complete upstream harness configuration.GPT-5.6 SolExact identitySource label without a registered configuration ID Canonical product: gpt-5-6-sol | 58.5% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.5; bulk export does not retain a complete upstream harness configuration.GPT-5.5Exact identitySource label without a registered configuration ID Canonical product: gpt-5-5 | 56.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Opus 5; bulk export does not retain a complete upstream harness configuration.Claude Opus 5Exact identitySource label without a registered configuration ID Canonical product: claude-opus-5 | 54.2% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Grok 4.5; bulk export does not retain a complete upstream harness configuration.Grok 4.5Exact identitySource label without a registered configuration ID Canonical product: grok-4-5 | 52.1% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Gemini 3.5 Flash; bulk export does not retain a complete upstream harness configuration.Gemini 3.5 FlashExact identitySource label without a registered configuration ID Canonical product: gemini-3-5-flash | 51.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.3 Codex; bulk export does not retain a complete upstream harness configuration.GPT-5.3-CodexExact identitySource label without a registered configuration ID Canonical product: gpt-5-3-codex | 51.8% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Gemini 3.6 Flash; bulk export does not retain a complete upstream harness configuration.Gemini 3.6 FlashExact identitySource label without a registered configuration ID Canonical product: gemini-3-6-flash | 50.2% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.4; bulk export does not retain a complete upstream harness configuration.GPT-5.4Exact identitySource label without a registered configuration ID Canonical product: gpt-5-4 | 50% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Opus 4.8; bulk export does not retain a complete upstream harness configuration.Claude Opus 4.8Exact identitySource label without a registered configuration ID Canonical product: claude-opus-4-8 | 46.6% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
From result to context
A display-only Artificial Analysis knowledge metric for the proportion of correctly answered questions.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.