Coding / Benchmark profile
A benchmark for agentic software engineering tasks executed in real terminal environments. DeepSeek reports it in the agentic section, while BenchLM also mirrors it in coding for models that publish it as a developer-task signal.
Observed results
20 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| GPT-5.6 Sol | #1 | gpt-5-6-sol | OpenAI | 91.9 | percent |
| Kimi K3 | #2 | kimi-k3 | Moonshot AI | 88.3 | percent |
| Claude Mythos 5 | #3 | claude-mythos-5 | Anthropic | 88 | percent |
| GPT-5.6 Terra | #4 | gpt-5-6-terra | OpenAI | 87.4 | percent |
| GPT-5.6 Luna | #5 | gpt-5-6-luna | OpenAI | 84.7 | percent |
| Claude Fable 5 | #6 | claude-fable-5 | Anthropic | 84.3 | percent |
| Grok 4.5 | #7 | grok-4-5 | xAI | 83.3 | percent |
| Sakana Fugu-Ultra | #8 | sakana-fugu-ultra | Sakana AI | 82.1 | percent |
| GPT-5.5 | #9 | gpt-5-5 | OpenAI | 82 | percent |
| SWE-1.7 | #10 | swe-1-7 | Cognition | 81.5 | percent |
| GLM-5.2 | #11 | glm-5-2 | Z.AI | 81 | percent |
| Claude Sonnet 5 | #12 | claude-sonnet-5 | Anthropic | 80.4 | percent |
| Sakana Fugu | #13 | sakana-fugu | Sakana AI | 80.2 | percent |
| Muse Spark 1.1 | #14 | muse-spark-1-1 | Meta | 80 | percent |
| Ornith-1.0-397B | #15 | ornith-1-0-397b | DeepReinforce AI | 77.5 | percent |
| GPT-5.3-Codex | #16 | gpt-5-3-codex | OpenAI | 77.3 | percent |
| Gemini 3.5 Flash | #17 | gemini-3-5-flash | 76.2 | percent | |
| GPT-5.4 | #18 | gpt-5-4 | OpenAI | 75.1 | percent |
| Claude Opus 4.8 | #19 | claude-opus-4-8 | Anthropic | 74.6 | percent |
| Qwen3.7-Plus | #20 | qwen-3-7-plus | Alibaba Cloud | 70.3 | percent |
| Laguna S 2.1 | #21 | laguna-s-2-1 | Poolside | 70.2 | percent |
| Qwen3.7-Max | #22 | qwen-3-7-max | Alibaba Cloud | 69.7 | percent |
| Claude Opus 4.7 | #23 | claude-opus-4-7 | Anthropic | 69.4 | percent |
| Composer 2.5 | #24 | composer-2-5 | Cursor | 69.3 | percent |
| MiMo-V2.5-Pro | #25 | mimo-v2-5-pro | Xiaomi | 68.4 | percent |
| DeepSeek V4 Pro | #26 | deepseek-v4-pro | DeepSeek | 67.9 | percent |
| Kimi K2.6 | #27 | kimi-k2-6 | Moonshot AI | 66.7 | percent |
| MiniMax M3 | #28 | minimax-m3 | MiniMax | 66 | percent |
| MiMo-V2.5 | #29 | mimo-v2-5 | Xiaomi | 65.8 | percent |
| Claude Opus 4.6 | #30 | claude-opus-4-6 | Anthropic | 65.4 | percent |
| Qwen 3.6 Max (preview) | #31 | qwen3-6-max-preview | Alibaba Cloud | 65.4 | percent |
| Inkling-Small | #32 | inkling-small | Thinking Machines Lab | 64.7 | percent |
| Ornith-1.0-35B | #33 | ornith-1-0-35b | DeepReinforce AI | 64.2 | percent |
| Inkling | #34 | inkling | Thinking Machines Lab | 63.8 | percent |
| GLM-5.1 | #35 | glm-5-1 | Z.AI | 63.5 | percent |
| Composer 2 | #36 | composer-2 | Cursor | 61.7 | percent |
| Qwen3.6 Plus | #37 | qwen3-6-plus | Alibaba Cloud | 61.6 | percent |
| GPT-5.4 mini | #38 | gpt-5-4-mini | OpenAI | 60 | percent |
| Step 3.7 Flash | #39 | step-3-7-flash | StepFun | 59.5 | percent |
| Claude Opus 4.5 | #40 | claude-opus-4-5 | Anthropic | 59.3 | percent |
| Qwen3.6 27B | #41 | qwen3-6-27b | Alibaba Cloud | 59.3 | percent |
| Claude Sonnet 4.6 | #42 | claude-sonnet-4-6 | Anthropic | 59.1 | percent |
| Muse Spark | #43 | muse-spark | Meta | 59 | percent |
| MiniMax M2.7 | #44 | minimax-m2-7 | MiniMax | 57 | percent |
| DeepSeek V4 Flash | #45 | deepseek-v4-flash | DeepSeek | 56.9 | percent |
| Nemotron 3 Ultra | #46 | nemotron-3-ultra | NVIDIA | 56.4 | percent |
| GLM-5 | #47 | glm-5 | Z.AI | 56.2 | percent |
| Hy3 Preview | #48 | hy3-preview | Tencent | 54.4 | percent |
| Gemini 3.5 Flash-Lite | #49 | gemini-3-5-flash-lite | 54 | percent | |
| Qwen3.5 397B A17B | #50 | qwen3-5-397b | Alibaba Cloud | 52.5 | percent |
| Qwen3.6-35B-A3B | #51 | qwen3-6-35b-a3b | Alibaba Cloud | 51.5 | percent |
| Kimi K2.5 | #52 | kimi-k2-5 | Moonshot AI | 50.8 | percent |
| Claude Sonnet 4.5 | #53 | claude-sonnet-4-5 | Anthropic | 50 | percent |
| Qwen3.5-122B-A10B | #54 | qwen3-5-122b-a10b | Alibaba Cloud | 49.4 | percent |
| Grok 4.20 | #55 | grok-4-20 | xAI | 47.1 | percent |
| GPT-5.4 nano | #56 | gpt-5-4-nano | OpenAI | 46.3 | percent |
| MAI-Thinking-1 | #57 | mai-thinking-1 | Microsoft | 46 | percent |
| Laguna M.1 | #58 | laguna-m-1 | Poolside | 45.8 | percent |
| Ornith-1.0-9B | #59 | ornith-1-0-9b | DeepReinforce AI | 43.1 | percent |
| Qwen3.5 27B | #60 | qwen3-5-27b | Alibaba Cloud | 41.6 | percent |
| GLM-4.7 | #61 | glm-4-7 | Z.AI | 41 | percent |
| Qwen3.5-35B-A3B | #62 | qwen3-5-35b-a3b | Alibaba Cloud | 40.5 | percent |
| Laguna XS 2.1 | #63 | laguna-xs-2-1 | Poolside | 37.5 | percent |
| Laguna XS.2 | #64 | laguna-xs-2 | Poolside | 35.7 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | GPT-5.6 SolOpenAI | Closed weights | Source carrier | 91.9% |
| 02 | Kimi K3Moonshot AI | Open weights | Source carrier | 88.3% |
| 03 | Claude Mythos 5Anthropic | Closed weights | Source carrier | 88% |
| 04 | GPT-5.6 TerraOpenAI | Closed weights | Source carrier | 87.4% |
| 05 | GPT-5.6 LunaOpenAI | Closed weights | Source carrier | 84.7% |
| 06 | Claude Fable 5Anthropic | Closed weights | Source carrier | 84.3% |
| 07 | Grok 4.5xAI | Closed weights | Source carrier | 83.3% |
| 08 | Sakana Fugu-UltraSakana AI | Closed weights | Source carrier | 82.1% |
| 09 | GPT-5.5OpenAI | Closed weights | Source carrier | 82% |
| 10 | SWE-1.7Cognition | Closed weights | Source carrier | 81.5% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Exact BenchLM registry variant GPT-5.6 Sol; bulk export does not retain a complete upstream harness configuration.GPT-5.6 SolExact identitySource label without a registered configuration ID Canonical product: gpt-5-6-sol | 91.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Kimi K3; bulk export does not retain a complete upstream harness configuration.Kimi K3Exact identitySource label without a registered configuration ID Canonical product: kimi-k3 | 88.3% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Mythos 5; bulk export does not retain a complete upstream harness configuration.Claude Mythos 5Exact identitySource label without a registered configuration ID Canonical product: claude-mythos-5 | 88% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.6 Terra; bulk export does not retain a complete upstream harness configuration.GPT-5.6 TerraExact identitySource label without a registered configuration ID Canonical product: gpt-5-6-terra | 87.4% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.6 Luna; bulk export does not retain a complete upstream harness configuration.GPT-5.6 LunaExact identitySource label without a registered configuration ID Canonical product: gpt-5-6-luna | 84.7% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Fable 5; bulk export does not retain a complete upstream harness configuration.Claude Fable 5Exact identitySource label without a registered configuration ID Canonical product: claude-fable-5 | 84.3% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Grok 4.5; bulk export does not retain a complete upstream harness configuration.Grok 4.5Exact identitySource label without a registered configuration ID Canonical product: grok-4-5 | 83.3% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Sakana Fugu-Ultra; bulk export does not retain a complete upstream harness configuration.Sakana Fugu-UltraExact identitySource label without a registered configuration ID Canonical product: sakana-fugu-ultra | 82.1% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.5; bulk export does not retain a complete upstream harness configuration.GPT-5.5Exact identitySource label without a registered configuration ID Canonical product: gpt-5-5 | 82% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant SWE-1.7; bulk export does not retain a complete upstream harness configuration.SWE-1.7Exact identitySource label without a registered configuration ID Canonical product: swe-1-7 | 81.5% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
From result to context
A benchmark for agentic software engineering tasks executed in real terminal environments. DeepSeek reports it in the agentic section, while BenchLM also mirrors it in coding for models that publish it as a developer-task signal.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.
