Agents / Benchmark profile
A transparent real-world autonomous-agent benchmark with 300 human-verified tasks, 2,159 rubric items, and Pass^3 scoring across general, multi-turn, and native multimodal agent tasks.
Observed results
20 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| Ornith-1.0-397B | #1 | ornith-1-0-397b | DeepReinforce AI | 77.1 | percent |
| MiniMax M3 | #2 | minimax-m3 | MiniMax | 74.5 | percent |
| Qwen3.6 27B | #3 | qwen3-6-27b | Alibaba Cloud | 72.4 | percent |
| Claude Opus 4.6 | #4 | claude-opus-4-6 | Anthropic | 70.4 | percent |
| Ornith-1.0-35B | #5 | ornith-1-0-35b | DeepReinforce AI | 69.8 | percent |
| Qwen3.6-35B-A3B | #6 | qwen3-6-35b-a3b | Alibaba Cloud | 68.7 | percent |
| Claude Sonnet 4.6 | #7 | claude-sonnet-4-6 | Anthropic | 67.8 | percent |
| Step 3.7 Flash | #8 | step-3-7-flash | StepFun | 67.1 | percent |
| Qwen3.7-Max | #9 | qwen-3-7-max | Alibaba Cloud | 65.2 | percent |
| MiMo-V2.5-Pro | #10 | mimo-v2-5-pro | Xiaomi | 63.8 | percent |
| Muse Spark | #11 | muse-spark | Meta | 63.8 | percent |
| Ornith-1.0-9B | #12 | ornith-1-0-9b | DeepReinforce AI | 63.1 | percent |
| Qwen3.7-Plus | #13 | qwen-3-7-plus | Alibaba Cloud | 62.7 | percent |
| GLM-5.1 | #14 | glm-5-1 | Z.AI | 62.3 | percent |
| Kimi K2.6 | #15 | kimi-k2-6 | Moonshot AI | 62.3 | percent |
| MiMo-V2.5 | #16 | mimo-v2-5 | Xiaomi | 62.3 | percent |
| GPT-5.4 | #17 | gpt-5-4 | OpenAI | 60.3 | percent |
| DeepSeek V4 Pro | #18 | deepseek-v4-pro | DeepSeek | 59.8 | percent |
| Claude Opus 4.5 | #19 | claude-opus-4-5 | Anthropic | 59.6 | percent |
| Qwen3.6 Plus | #20 | qwen3-6-plus | Alibaba Cloud | 58.8 | percent |
| DeepSeek V4 Flash | #21 | deepseek-v4-flash | DeepSeek | 57.8 | percent |
| MiMo-V2-Pro | #22 | mimo-v2-pro | Xiaomi | 57.8 | percent |
| GLM-5 | #23 | glm-5 | Z.AI | 57.7 | percent |
| Qwen3.8-27B | #24 | qwen-3-8-27b | Alibaba Cloud | 57.4 | percent |
| Qwen3.5 397B A17B | #25 | qwen3-5-397b | Alibaba Cloud | 56.8 | percent |
| GLM-5-Turbo | #26 | glm-5-turbo | Z.AI | 55.8 | percent |
| GLM-5V-Turbo | #27 | glm-5v-turbo | Z.AI | 53.8 | percent |
| Kimi K2.5 | #28 | kimi-k2-5 | Moonshot AI | 52.3 | percent |
| Gemini 3 Flash | #29 | gemini-3-flash | 49.2 | percent | |
| MiniMax M2.7 | #30 | minimax-m2-7 | MiniMax | 48.7 | percent |
| MiMo-V2-Omni | #31 | mimo-v2-omni | Xiaomi | 45.2 | percent |
| DeepSeek V3.2 | #32 | deepseek-v3-2 | DeepSeek | 40.2 | percent |
| Nemotron 3 Super 100B | #33 | nemotron-3-super-100b | NVIDIA | 5.5 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | Ornith-1.0-397BDeepReinforce AI | Open weights | Source carrier | 77.1% |
| 02 | MiniMax M3MiniMax | Open weights | Source carrier | 74.5% |
| 03 | Qwen3.6 27BAlibaba Cloud | Open weights | Source carrier | 72.4% |
| 04 | Claude Opus 4.6Anthropic | Closed weights | Source carrier | 70.4% |
| 05 | Ornith-1.0-35BDeepReinforce AI | Open weights | Source carrier | 69.8% |
| 06 | Qwen3.6-35B-A3BAlibaba Cloud | Open weights | Source carrier | 68.7% |
| 07 | Claude Sonnet 4.6Anthropic | Closed weights | Source carrier | 67.8% |
| 08 | Step 3.7 FlashStepFun | Open weights | Source carrier | 67.1% |
| 09 | Qwen3.7-MaxAlibaba Cloud | Closed weights | Source carrier | 65.2% |
| 10 | MiMo-V2.5-ProXiaomi | Open weights | Source carrier | 63.8% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Qwen3.7-Plus (provider-published configuration) as published in Qwen's Qwen3.8-Flash-Next comparison tableQwen3.7-PlusExact identityqwen-3-7-plus-unspecified Canonical product: qwen-3-7-plus | 60.1% | Public referenceprovider-reportedVersion & systemAverage system:qwen3-8-flash-next-comparison:cell:vision:claweval-average:2 | Qwen3.8-Flash-Next launch and official provider evaluationsObserved Checked |
Qwen3.8-27B (xhigh default thinking configuration) as published in Qwen's Qwen3.8-Flash-Next comparison tableQwen3.8-27BExact identityqwen-3-8-27b-xhigh Canonical product: qwen-3-8-27b | 57.4% | Public referenceprovider-reportedVersion & systemPass@3 system:qwen3-8-flash-next-comparison:cell:vision:claweval-pass3:1 | Qwen3.8-Flash-Next launch and official provider evaluationsObserved Checked |
Qwen3.7-Plus (provider-published configuration) as published in Qwen's Qwen3.8-Flash-Next comparison tableQwen3.7-PlusExact identityqwen-3-7-plus-unspecified Canonical product: qwen-3-7-plus | 57.4% | Public referenceprovider-reportedVersion & systemPass@3 system:qwen3-8-flash-next-comparison:cell:vision:claweval-pass3:2 | Qwen3.8-Flash-Next launch and official provider evaluationsObserved Checked |
Qwen3.8-27B (xhigh default thinking configuration) as published in Qwen's Qwen3.8-Flash-Next comparison tableQwen3.8-27BExact identityqwen-3-8-27b-xhigh Canonical product: qwen-3-8-27b | 56.9% | Public referenceprovider-reportedVersion & systemAverage system:qwen3-8-flash-next-comparison:cell:vision:claweval-average:1 | Qwen3.8-Flash-Next launch and official provider evaluationsObserved Checked |
Claude Opus 4.6 (Max) as published in Qwen's Qwen3.8-Flash-Next comparison tableClaude Opus 4.6Exact identityclaude-opus-4-6-max Canonical product: claude-opus-4-6 | 54.7% | Public referenceprovider-reportedVersion & systemAverage system:qwen3-8-flash-next-comparison:cell:vision:claweval-average:3 | Qwen3.8-Flash-Next launch and official provider evaluationsObserved Checked |
Claude Opus 4.6 (Max) as published in Qwen's Qwen3.8-Flash-Next comparison tableClaude Opus 4.6Exact identityclaude-opus-4-6-max Canonical product: claude-opus-4-6 | 52.5% | Public referenceprovider-reportedVersion & systemPass@3 system:qwen3-8-flash-next-comparison:cell:vision:claweval-pass3:3 | Qwen3.8-Flash-Next launch and official provider evaluationsObserved Checked |
Qwen3.7-Plus as published by QwenQwen3.7-PlusExact identityqwen-3-7-plus-unspecified Canonical product: qwen-3-7-plus | 60.1% | Public referenceprovider-reportedVersion & systemAverage Qwen3.8-27B official VL table | Qwen3.8-27B official model cardObserved Checked |
Qwen3.7-Plus as published by QwenQwen3.7-PlusExact identityqwen-3-7-plus-unspecified Canonical product: qwen-3-7-plus | 57.4% | Public referenceprovider-reportedVersion & systemPass@3 Qwen3.8-27B official VL table | Qwen3.8-27B official model cardObserved Checked |
Qwen3.8-27B (xhigh)Qwen3.8-27BExact identityqwen-3-8-27b-xhigh Canonical product: qwen-3-8-27b | 57.4% | Public referenceprovider-reportedVersion & systemPass@3 Qwen3.8-27B official VL table | Qwen3.8-27B official model cardObserved Checked |
Qwen3.8-27B (xhigh)Qwen3.8-27BExact identityqwen-3-8-27b-xhigh Canonical product: qwen-3-8-27b | 56.9% | Public referenceprovider-reportedVersion & systemAverage Qwen3.8-27B official VL table | Qwen3.8-27B official model cardObserved Checked |
From result to context
A transparent real-world autonomous-agent benchmark with 300 human-verified tasks, 2,159 rubric items, and Pass^3 scoring across general, multi-turn, and native multimodal agent tasks.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.
