A task with its own rules.
A display-only Artificial Analysis factuality metric for the rate of incorrect answers among non-correct responses.
- Organisation
- Artificial Analysis
- Version
- 2026
Loading benchmark header
Loading benchmark registry
Knowledge / Benchmark profile
A display-only Artificial Analysis factuality metric for the rate of incorrect answers among non-correct responses.
Observed results
20 ranked models · lower is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| Command A+ | #1 | command-a-plus | Cohere | 14.1 | percent |
| MiniMax M3 | #2 | minimax-m3 | MiniMax | 16.1 | percent |
| Qwen3.7-Max | #3 | qwen-3-7-max | Alibaba Cloud | 22.9 | percent |
| MiMo-V2.5-Pro | #4 | mimo-v2-5-pro | Xiaomi | 24.5 | percent |
| Grok 4.3 | #5 | grok-4-3 | xAI | 25 | percent |
| Qwen3.7-Plus | #6 | qwen-3-7-plus | Alibaba Cloud | 25.5 | percent |
| GLM-5.2 | #7 | glm-5-2 | Z.AI | 28.1 | percent |
| Nemotron 3 Ultra | #8 | nemotron-3-ultra | NVIDIA | 28.5 | percent |
| GLM-5.1 | #9 | glm-5-1 | Z.AI | 29.4 | percent |
| MiMo-V2-Pro | #10 | mimo-v2-pro | Xiaomi | 29.9 | percent |
| Gemma 4 E4B | #11 | gemma-4-e4b | 31.3 | percent | |
| Qwen3.6 Plus | #12 | qwen3-6-plus | Alibaba Cloud | 32 | percent |
| Gemma 4 E2B | #13 | gemma-4-e2b | 32.9 | percent | |
| Gemini 3.5 Flash-Lite | #14 | gemini-3-5-flash-lite | 33.5 | percent | |
| GLM-5 | #15 | glm-5 | Z.AI | 34 | percent |
| MiniMax M2.7 | #16 | minimax-m2-7 | MiniMax | 34.4 | percent |
| Claude Opus 4.8 | #17 | claude-opus-4-8 | Anthropic | 35.9 | percent |
| Claude Opus 4.7 | #18 | claude-opus-4-7 | Anthropic | 36.2 | percent |
| Claude Sonnet 5 | #19 | claude-sonnet-5 | Anthropic | 37.3 | percent |
| GPT-4o | #20 | gpt-4o | OpenAI | 37.9 | percent |
| Muse Spark 1.1 | #21 | muse-spark-1-1 | Meta | 38.1 | percent |
| Kimi K2.6 | #22 | kimi-k2-6 | Moonshot AI | 39.3 | percent |
| Claude 4 Sonnet | #23 | claude-4-sonnet | Anthropic | 40.8 | percent |
| Qwen 3.6 Max (preview) | #24 | qwen3-6-max-preview | Alibaba Cloud | 44.2 | percent |
| MiMo-V2-Omni | #25 | mimo-v2-omni | Xiaomi | 44.4 | percent |
| LFM2.5-8B-A1B | #26 | lfm2-5-8b-a1b | LiquidAI | 47 | percent |
| Qwen3.6 27B | #27 | qwen3-6-27b | Alibaba Cloud | 48.3 | percent |
| Qwen3.6-35B-A3B | #28 | qwen3-6-35b-a3b | Alibaba Cloud | 49.7 | percent |
| Claude Opus 5 | #29 | claude-opus-5 | Anthropic | 50.1 | percent |
| Kimi K3 | #30 | kimi-k3 | Moonshot AI | 50.9 | percent |
| Llama 3.1 405B | #31 | llama-3-1-405b | Meta | 51 | percent |
| GPT-5.1 | #32 | gpt-5-1 | OpenAI | 51.3 | percent |
| Gemini 3.6 Flash | #33 | gemini-3-6-flash | 53.5 | percent | |
| Grok 4.5 | #34 | grok-4-5 | xAI | 53.5 | percent |
| Claude Fable 5 | #35 | claude-fable-5 | Anthropic | 54.9 | percent |
| Claude Opus 4.5 Thinking | #36 | claude-opus-4-5-thinking | Anthropic | 59.8 | percent |
| Gemini 3.5 Flash | #37 | gemini-3-5-flash | 60.7 | percent | |
| Mistral Medium 3 | #38 | mistral-medium-3 | Mistral AI | 60.9 | percent |
| Claude Opus 4.6 | #39 | claude-opus-4-6 | Anthropic | 61.3 | percent |
| GLM-5-Turbo | #40 | glm-5-turbo | Z.AI | 62.2 | percent |
| Inkling | #41 | inkling | Thinking Machines Lab | 63.1 | percent |
| Grok 4 | #42 | grok-4 | xAI | 64.2 | percent |
| Kimi K2.5 | #43 | kimi-k2-5 | Moonshot AI | 64.6 | percent |
| Claude Sonnet 4.6 | #44 | claude-sonnet-4-6 | Anthropic | 65.9 | percent |
| Grok 4 Fast (Reasoning) | #45 | grok-4-fast-reasoning | xAI | 66 | percent |
| GLM-4.6 | #46 | glm-4-6 | Z.AI | 66.1 | percent |
| Mistral Small 4 | #47 | mistral-small-4 | Mistral AI | 66.8 | percent |
| Mistral Large 2 | #48 | mistral-large-2 | Mistral AI | 67.8 | percent |
| GLM-5V-Turbo | #49 | glm-5v-turbo | Z.AI | 67.9 | percent |
| o1 | #50 | o1 | OpenAI | 69.3 | percent |
| Grok 4.1 Fast (Reasoning) | #51 | grok-4-1-fast-reasoning | xAI | 72.4 | percent |
| GPT-5.2 Codex | #52 | gpt-5-2-codex | OpenAI | 72.8 | percent |
| Hy3 | #53 | hy3 | Tencent | 73 | percent |
| Hy3 Preview | #54 | hy3-preview | Tencent | 73 | percent |
| Muse Spark | #55 | muse-spark | Meta | 73.2 | percent |
| GPT-5.4 nano | #56 | gpt-5-4-nano | OpenAI | 73.6 | percent |
| Kimi K2 | #57 | kimi-k2 | Moonshot AI | 74.2 | percent |
| GPT-5.1-Codex | #58 | gpt-5-1-codex | OpenAI | 74.4 | percent |
| GPT-5.1-Codex-Max | #59 | gpt-5-1-codex-max | OpenAI | 74.4 | percent |
| Claude Opus 4.5 | #60 | claude-opus-4-5 | Anthropic | 75.4 | percent |
| Granite-4.0-350M | #61 | granite-4-0-350m | IBM | 77.8 | percent |
| Nova Pro | #62 | nova-pro | Amazon | 77.9 | percent |
| Claude 3 Haiku | #63 | claude-3-haiku | Anthropic | 78.2 | percent |
| Llama 4 Scout | #64 | llama-4-scout | Meta | 78.3 | percent |
| Grok Code Fast 1 | #65 | grok-code-fast-1 | xAI | 78.5 | percent |
| GPT-4.1 | #66 | gpt-4-1 | OpenAI | 79.6 | percent |
| GPT-5.2 | #67 | gpt-5-2 | OpenAI | 79.7 | percent |
| Qwen3.5 27B | #68 | qwen3-5-27b | Alibaba Cloud | 79.7 | percent |
| GPT-5 | #69 | gpt-5 | OpenAI | 80.1 | percent |
| DeepSeek V3.1 | #70 | deepseek-v3-1 | DeepSeek | 80.3 | percent |
| Kimi K2.7 Code | #71 | kimi-k2-7-code | Moonshot AI | 80.3 | percent |
| GPT-4.1 nano | #72 | gpt-4-1-nano | OpenAI | 80.4 | percent |
| Phi-4 | #73 | phi-4 | Microsoft | 80.5 | percent |
| Gemma 4 12B Unified | #74 | gemma-4-12b | 80.8 | percent | |
| Gemma 4 26B A4B | #75 | gemma-4-26b-a4b | 80.9 | percent | |
| Exaone 4.0 32B | #76 | exaone-4-0-32b | LG AI Research | 81 | percent |
| Gemini 3.1 Flash-Lite | #77 | gemini-3-1-flash-lite | 81.6 | percent | |
| Gemma 4 31B | #78 | gemma-4-31b | 81.6 | percent | |
| Nemotron Ultra 253B | #79 | nemotron-ultra-253b | NVIDIA | 81.7 | percent |
| Grok 4.1 Fast | #80 | grok-4-1-fast | xAI | 81.8 | percent |
| GPT-4.1 mini | #81 | gpt-4-1-mini | OpenAI | 82 | percent |
| Mistral Medium 3.5 128B | #82 | mistral-medium-3-5-128b | Mistral AI | 82 | percent |
| Nemotron 3 Nano 30B | #83 | nemotron-3-nano-30b | NVIDIA | 82.9 | percent |
| Nemotron 3 Nano Omni 30B A3B | #84 | nemotron-3-nano-omni-30b-a3b | NVIDIA | 83.1 | percent |
| Granite-4.0-H-1B | #85 | granite-4-0-h-1b | IBM | 83.4 | percent |
| Mistral Large 3 | #86 | mistral-large-3 | Mistral AI | 83.7 | percent |
| DeepSeek-R1 | #87 | deepseek-r1 | DeepSeek | 84 | percent |
| Qwen3.5-35B-A3B | #88 | qwen3-5-35b-a3b | Alibaba Cloud | 84 | percent |
| Step 3.7 Flash | #89 | step-3-7-flash | StepFun | 84.4 | percent |
| GPT-5.6 Terra | #90 | gpt-5-6-terra | OpenAI | 85.2 | percent |
| GPT-5.5 | #91 | gpt-5-5 | OpenAI | 85.5 | percent |
| Qwen3.5-122B-A10B | #92 | qwen3-5-122b-a10b | Alibaba Cloud | 85.5 | percent |
| Trinity-Large-Preview | #93 | trinity-large-preview | Arcee AI | 86.6 | percent |
| Trinity-Large-Thinking | #94 | trinity-large-thinking | Arcee AI | 86.6 | percent |
| GPT-5.3-Codex | #95 | gpt-5-3-codex | OpenAI | 86.9 | percent |
| o3 | #96 | o3 | OpenAI | 87.1 | percent |
| Llama 4 Maverick | #97 | llama-4-maverick | Meta | 87.3 | percent |
| Gemini 2.5 Pro | #98 | gemini-2-5-pro | 87.4 | percent | |
| DeepSeek V4 Pro | #99 | deepseek-v4-pro | DeepSeek | 88.6 | percent |
| GPT-5.4 | #100 | gpt-5-4 | OpenAI | 88.6 | percent |
| GPT-5.6 Sol | #101 | gpt-5-6-sol | OpenAI | 88.8 | percent |
| K-Exaone | #102 | k-exaone | LG AI Research | 89.1 | percent |
| Qwen3.5 397B A17B | #103 | qwen3-5-397b | Alibaba Cloud | 89.1 | percent |
| DeepSeek V3 | #104 | deepseek-v3 | DeepSeek | 89.4 | percent |
| Qwen3 Max | #105 | qwen3-max | Alibaba Cloud | 89.4 | percent |
| Gemma 3 27B | #106 | gemma-3-27b | 89.5 | percent | |
| DeepSeek V4 Flash | #107 | deepseek-v4-flash | DeepSeek | 89.7 | percent |
| GPT-5.4 mini | #108 | gpt-5-4-mini | OpenAI | 89.8 | percent |
| GPT-5.6 Luna | #109 | gpt-5-6-luna | OpenAI | 90.1 | percent |
| Gemini 3 Flash | #110 | gemini-3-flash | 90.2 | percent | |
| GLM-4.7 | #111 | glm-4-7 | Z.AI | 90.3 | percent |
| GPT-OSS 120B | #112 | gpt-oss-120b | OpenAI | 91.2 | percent |
| Exaone 4.0 1.2B | #113 | exaone-4-0-1-2b | LG AI Research | 91.5 | percent |
| Solar Pro 2 | #114 | solar-pro-2 | Upstage | 91.5 | percent |
| GLM-4.5-Air | #115 | glm-4-5-air | Z.AI | 92.3 | percent |
| Gemini 2.5 Flash | #116 | gemini-2-5-flash | 93.3 | percent | |
| DeepSeek V3.2 | #117 | deepseek-v3-2 | DeepSeek | 93.5 | percent |
| Granite-4.0-1B | #118 | granite-4-0-1b | IBM | 93.5 | percent |
| Sarvam 105B | #119 | sarvam-105b | Sarvam | 93.5 | percent |
| LFM2.5-VL-1.6B-Extract | #120 | lfm2-5-vl-1-6b-extract | LiquidAI | 94 | percent |
| GPT-OSS 20B | #121 | gpt-oss-20b | OpenAI | 94.1 | percent |
| Granite-4.0-H-350M | #122 | granite-4-0-h-350m | IBM | 94.4 | percent |
| Ling 2.6 Flash | #123 | ling-2-6-flash | InclusionAI | 95.8 | percent |
| Sarvam 30B | #124 | sarvam-30b | Sarvam | 97 | percent |
One best compatible score per canonical product · lower is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | Command A+Cohere | Open weights | Source carrier | 14.1% |
| 02 | MiniMax M3MiniMax | Open weights | Source carrier | 16.1% |
| 03 | Qwen3.7-MaxAlibaba Cloud | Closed weights | Source carrier | 22.9% |
| 04 | MiMo-V2.5-ProXiaomi | Open weights | Source carrier | 24.5% |
| 05 | Grok 4.3xAI | Closed weights | Source carrier | 25% |
| 06 | Qwen3.7-PlusAlibaba Cloud | Closed weights | Source carrier | 25.5% |
| 07 | GLM-5.2Z.AI | Open weights | Source carrier | 28.1% |
| 08 | Nemotron 3 UltraNVIDIA | Open weights | Source carrier | 28.5% |
| 09 | GLM-5.1Z.AI | Open weights | Source carrier | 29.4% |
| 10 | MiMo-V2-ProXiaomi | Closed weights | Source carrier | 29.9% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Exact BenchLM registry variant Sarvam 30B; bulk export does not retain a complete upstream harness configuration.Sarvam 30BExact identitySource label without a registered configuration ID Canonical product: sarvam-30b | 97% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant DeepSeek V4 Flash (Max); bulk export does not retain a complete upstream harness configuration.DeepSeek V4 FlashExact identitydeepseek-v4-flash-max Canonical product: deepseek-v4-flash | 95.8% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Ling 2.6 Flash; bulk export does not retain a complete upstream harness configuration.Ling 2.6 FlashExact identitySource label without a registered configuration ID Canonical product: ling-2-6-flash | 95.8% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Granite-4.0-H-350M; bulk export does not retain a complete upstream harness configuration.Granite-4.0-H-350MExact identitySource label without a registered configuration ID Canonical product: granite-4-0-h-350m | 94.4% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-OSS 20B; bulk export does not retain a complete upstream harness configuration.GPT-OSS 20BExact identitySource label without a registered configuration ID Canonical product: gpt-oss-20b | 94.1% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant DeepSeek V4 Pro (Max); bulk export does not retain a complete upstream harness configuration.DeepSeek V4 ProExact identitydeepseek-v4-pro-max Canonical product: deepseek-v4-pro | 94% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant LFM2.5-VL-1.6B-Extract; bulk export does not retain a complete upstream harness configuration.LFM2.5-VL-1.6B-ExtractExact identitySource label without a registered configuration ID Canonical product: lfm2-5-vl-1-6b-extract | 94% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant DeepSeek V3.2; bulk export does not retain a complete upstream harness configuration.DeepSeek V3.2Exact identitySource label without a registered configuration ID Canonical product: deepseek-v3-2 | 93.5% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Granite-4.0-1B; bulk export does not retain a complete upstream harness configuration.Granite-4.0-1BExact identitySource label without a registered configuration ID Canonical product: granite-4-0-1b | 93.5% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Sarvam 105B; bulk export does not retain a complete upstream harness configuration.Sarvam 105BExact identitySource label without a registered configuration ID Canonical product: sarvam-105b | 93.5% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
From result to context
A display-only Artificial Analysis factuality metric for the rate of incorrect answers among non-correct responses.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.