A task with its own rules.
Dual-control conversational agent benchmark for telecom support where agent and user must coordinate tool actions.
- Organisation
- Sierra / Artificial Analysis
- Version
- 2
Loading benchmark header
Loading benchmark registry
Agents / Benchmark profile
Dual-control conversational agent benchmark for telecom support where agent and user must coordinate tool actions.
Observed results
20 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| GLM-5.2 | #1 | glm-5-2 | Z.AI | 99.1228 | percent |
| GPT-5.4 | #2 | gpt-5-4 | OpenAI | 98.9 | percent |
| Claude Fable 5 | #3 | claude-fable-5 | Anthropic | 98.538 | percent |
| GLM-5-Turbo | #4 | glm-5-turbo | Z.AI | 98.538 | percent |
| GLM-5V-Turbo | #5 | glm-5v-turbo | Z.AI | 98.538 | percent |
| Step 3.7 Flash | #6 | step-3-7-flash | StepFun | 98.5 | percent |
| GLM-5 | #7 | glm-5 | Z.AI | 98.2456 | percent |
| GPT-5.5 | #8 | gpt-5-5 | OpenAI | 98 | percent |
| GLM-5.1 | #9 | glm-5-1 | Z.AI | 97.7 | percent |
| Grok 4.3 | #10 | grok-4-3 | xAI | 97.7 | percent |
| Qwen3.6 Plus | #11 | qwen3-6-plus | Alibaba Cloud | 97.7 | percent |
| DeepSeek V4 Pro | #12 | deepseek-v4-pro | DeepSeek | 96.2 | percent |
| GLM-4.7 | #13 | glm-4-7 | Z.AI | 95.9064 | percent |
| Kimi K2.5 | #14 | kimi-k2-5 | Moonshot AI | 95.9064 | percent |
| Kimi K2.6 | #15 | kimi-k2-6 | Moonshot AI | 95.9064 | percent |
| Qwen3.6 Max | #16 | qwen3-6-max | Alibaba Cloud | 95.9064 | percent |
| Qwen 3.6 Max (preview) | #17 | qwen3-6-max-preview | Alibaba Cloud | 95.9 | percent |
| Gemini 3.1 Pro Preview | #18 | gemini-3-1-pro-preview | 95.614 | percent | |
| Qwen3.5 397B A17B | #19 | qwen3-5-397b | Alibaba Cloud | 95.614 | percent |
| DeepSeek V4 Flash | #20 | deepseek-v4-flash | DeepSeek | 95.6 | percent |
| Gemini 3.5 Flash | #21 | gemini-3-5-flash | 95.3216 | percent | |
| MiniMax M2.5 | #22 | minimax-m2-5 | MiniMax | 95.3216 | percent |
| Qwen3.6-35B-A3B | #23 | qwen3-6-35b-a3b | Alibaba Cloud | 95.3 | percent |
| MiMo-V2-Pro | #24 | mimo-v2-pro | Xiaomi | 95.0292 | percent |
| Qwen3.7-Max | #25 | qwen-3-7-max | Alibaba Cloud | 94.7368 | percent |
| Claude Opus 4.8 | #26 | claude-opus-4-8 | Anthropic | 94.4444 | percent |
| MiMo-V2.5-Pro | #27 | mimo-v2-5-pro | Xiaomi | 94.2 | percent |
| Mistral Medium 3.5 128B | #28 | mistral-medium-3-5-128b | Mistral AI | 94.2 | percent |
| Qwen3.6 27B | #29 | qwen3-6-27b | Alibaba Cloud | 94.2 | percent |
| Mistral Medium 3.5 | #30 | mistral-medium-3-5 | Mistral AI | 94.152 | percent |
| Qwen3.5 27B | #31 | qwen3-5-27b | Alibaba Cloud | 93.9 | percent |
| Qwen3.5-122B-A10B | #32 | qwen3-5-122b-a10b | Alibaba Cloud | 93.6 | percent |
| Qwen3.5 122B | #33 | qwen3-5-122b | Alibaba Cloud | 93.5673 | percent |
| GPT-5.4 mini | #34 | gpt-5-4-mini | OpenAI | 93.4 | percent |
| Grok 4.1 Fast (Reasoning) | #35 | grok-4-1-fast-reasoning | xAI | 93.3 | percent |
| Qwen3.7-Plus | #36 | qwen-3-7-plus | Alibaba Cloud | 93 | percent |
| Grok 4.20 | #37 | grok-4-20 | xAI | 92.9825 | percent |
| Kimi K2 Thinking | #38 | kimi-k2-thinking | Moonshot AI | 92.9825 | percent |
| Nova 2 Pro | #39 | nova-2-pro | Amazon | 92.6901 | percent |
| GPT-5.4 nano | #40 | gpt-5-4-nano | OpenAI | 92.5 | percent |
| Claude Opus 4.6 | #41 | claude-opus-4-6 | Anthropic | 92.1053 | percent |
| GPT-5.2 Codex | #42 | gpt-5-2-codex | OpenAI | 92.1053 | percent |
| Muse Spark | #43 | muse-spark | Meta | 91.5205 | percent |
| MiMo-V2-Omni | #44 | mimo-v2-omni | Xiaomi | 91.2281 | percent |
| DeepSeek V3.2 | #45 | deepseek-v3-2 | DeepSeek | 90.6433 | percent |
| MiMo-V2.5 | #46 | mimo-v2-5 | Xiaomi | 90.6433 | percent |
| Grok 3 mini | #47 | grok-3-mini | xAI | 90.3509 | percent |
| Kimi K2.7 Code | #48 | kimi-k2-7-code | Moonshot AI | 90.1 | percent |
| Trinity-Large-Preview | #49 | trinity-large-preview | Arcee AI | 90.1 | percent |
| Trinity-Large-Thinking | #50 | trinity-large-thinking | Arcee AI | 90.1 | percent |
| Ling 2.6 1T | #51 | ling-2-6-1t | InclusionAI | 89.7661 | percent |
| Claude Opus 4.5 Thinking | #52 | claude-opus-4-5-thinking | Anthropic | 89.5 | percent |
| Claude Opus 4.5 | #53 | claude-opus-4-5 | Anthropic | 89.4737 | percent |
| Qwen3.5-35B-A3B | #54 | qwen3-5-35b-a3b | Alibaba Cloud | 89.2 | percent |
| Qwen3.5 35B | #55 | qwen3-5-35b | Alibaba Cloud | 89.1813 | percent |
| MiniMax M3 | #56 | minimax-m3 | MiniMax | 88.9 | percent |
| Claude Opus 4.7 | #57 | claude-opus-4-7 | Anthropic | 88.6 | percent |
| Qwen3.5 Omni Plus | #58 | qwen3-5-omni-plus | Alibaba Cloud | 88.3041 | percent |
| LFM2.5-8B-A1B | #59 | lfm2-5-8b-a1b | LiquidAI | 88.07 | percent |
| GPT-5 Codex | #60 | gpt-5-codex | OpenAI | 86.8421 | percent |
| MiniMax M2 | #61 | minimax-m2 | MiniMax | 86.8421 | percent |
| GPT-5 | #62 | gpt-5 | OpenAI | 86.5 | percent |
| GPT-5.6 Terra | #63 | gpt-5-6-terra | OpenAI | 86.3 | percent |
| GPT-5.3-Codex | #64 | gpt-5-3-codex | OpenAI | 86 | percent |
| Ling 2.6 Flash | #65 | ling-2-6-flash | InclusionAI | 86 | percent |
| MiniMax M2.1 | #66 | minimax-m2-1 | MiniMax | 85.3801 | percent |
| GPT-5.6 Sol | #67 | gpt-5-6-sol | OpenAI | 85.1 | percent |
| Command A+ | #68 | command-a-plus | Cohere | 85 | percent |
| GPT-5.2 | #69 | gpt-5-2 | OpenAI | 84.8 | percent |
| MiniMax M2.7 | #70 | minimax-m2-7 | MiniMax | 84.8 | percent |
| Qwen3 Max | #71 | qwen3-max | Alibaba Cloud | 83.6257 | percent |
| Nemotron 3 Ultra | #72 | nemotron-3-ultra | NVIDIA | 83.3333 | percent |
| GPT-5.1-Codex | #73 | gpt-5-1-codex | OpenAI | 83 | percent |
| GPT-5.1-Codex-Max | #74 | gpt-5-1-codex-max | OpenAI | 83 | percent |
| GPT-5.1 | #75 | gpt-5-1 | OpenAI | 81.9 | percent |
| o3 | #76 | o3 | OpenAI | 80.7018 | percent |
| Gemini 3 Flash | #77 | gemini-3-flash | 80.4094 | percent | |
| Claude Sonnet 4.6 | #78 | claude-sonnet-4-6 | Anthropic | 79.5 | percent |
| Claude Sonnet 4.5 | #79 | claude-sonnet-4-5 | Anthropic | 78.0702 | percent |
| GLM-4.6 | #80 | glm-4-6 | Z.AI | 76.9 | percent |
| Grok Code Fast 1 | #81 | grok-code-fast-1 | xAI | 75.731 | percent |
| Grok 4 | #82 | grok-4 | xAI | 74.9 | percent |
| K-Exaone | #83 | k-exaone | LG AI Research | 74.3 | percent |
| Claude Opus 4 | #84 | claude-opus-4 | Anthropic | 73.3918 | percent |
| Kimi K2 0905 | #85 | kimi-k2-0905 | Moonshot AI | 73.3918 | percent |
| Claude 4.1 Opus Thinking | #86 | claude-4-1-opus-thinking | Anthropic | 71.4 | percent |
| Claude Opus 4.1 | #87 | claude-opus-4-1 | Anthropic | 71.3661 | percent |
| GPT-5 mini | #88 | gpt-5-mini | OpenAI | 71.0526 | percent |
| Nemotron 3 Super | #89 | nemotron-3-super | NVIDIA | 67.8363 | percent |
| GPT-OSS 120B | #90 | gpt-oss-120b | OpenAI | 65.8 | percent |
| Grok 4 Fast (Reasoning) | #91 | grok-4-fast-reasoning | xAI | 65.8 | percent |
| Grok 4 Fast | #92 | grok-4-fast | xAI | 65.7895 | percent |
| Claude Sonnet 4 | #93 | claude-sonnet-4 | Anthropic | 64.6199 | percent |
| Grok 4.1 Fast | #94 | grok-4-1-fast | xAI | 63.7 | percent |
| o1 | #95 | o1 | OpenAI | 62.6 | percent |
| Kimi K2 | #96 | kimi-k2 | Moonshot AI | 61.1 | percent |
| GPT-OSS 20B | #97 | gpt-oss-20b | OpenAI | 60.2 | percent |
| Gemma 4 31B | #98 | gemma-4-31b | 59.9415 | percent | |
| o4-mini | #99 | o4-mini | OpenAI | 55.5556 | percent |
| Claude Haiku 4.5 | #100 | claude-haiku-4-5 | Anthropic | 54.6784 | percent |
| Claude Sonnet 3.7 | #101 | claude-sonnet-3-7 | Anthropic | 54.6784 | percent |
| Gemini 2.5 Pro | #102 | gemini-2-5-pro | 54.1 | percent | |
| GPT-4.1 mini | #103 | gpt-4-1-mini | OpenAI | 52.9 | percent |
| Claude 4 Sonnet | #104 | claude-4-sonnet | Anthropic | 52.3 | percent |
| GPT-4.1 | #105 | gpt-4-1 | OpenAI | 47.1 | percent |
| Sarvam 105B | #106 | sarvam-105b | Sarvam | 46.8 | percent |
| GLM-4.5-Air | #107 | glm-4-5-air | Z.AI | 46.5 | percent |
| Gemini 2.5 Flash | #108 | gemini-2-5-flash | 45.614 | percent | |
| Nemotron 3 Nano Omni 30B A3B | #109 | nemotron-3-nano-omni-30b-a3b | NVIDIA | 45.3 | percent |
| Gemma 4 26B A4B | #110 | gemma-4-26b-a4b | 43.6 | percent | |
| Gemma 4 26B | #111 | gemma-4-26b | 43.5673 | percent | |
| Mistral Small 4 | #112 | mistral-small-4 | Mistral AI | 41.2281 | percent |
| Nemotron 3 Nano 30B | #113 | nemotron-3-nano-30b | NVIDIA | 40.9 | percent |
| DeepSeek V3.1 | #114 | deepseek-v3-1 | DeepSeek | 37.4 | percent |
| DeepSeek V3.1 Terminus | #115 | deepseek-v3-1-terminus | DeepSeek | 37.1345 | percent |
| DeepSeek-R1 | #116 | deepseek-r1 | DeepSeek | 36.5 | percent |
| Gemma 4 12B Unified | #117 | gemma-4-12b | 36.3 | percent | |
| Sarvam 30B | #118 | sarvam-30b | Sarvam | 34.5 | percent |
| Solar Pro 2 | #119 | solar-pro-2 | Upstage | 31.9 | percent |
| Gemini 3.1 Flash-Lite | #120 | gemini-3-1-flash-lite | 31.3 | percent | |
| Mistral Large 2 | #121 | mistral-large-2 | Mistral AI | 30.7 | percent |
| o3-mini | #122 | o3-mini | OpenAI | 28.7 | percent |
| GPT-4o | #123 | gpt-4o | OpenAI | 25.1 | percent |
| Mistral Large 3 | #124 | mistral-large-3 | Mistral AI | 24.6 | percent |
| Mistral Medium 3 | #125 | mistral-medium-3 | Mistral AI | 24.3 | percent |
| DeepSeek V3 | #126 | deepseek-v3 | DeepSeek | 22.8 | percent |
| Granite-4.0-1B | #127 | granite-4-0-1b | IBM | 22.8 | percent |
| Claude 3 Haiku | #128 | claude-3-haiku | Anthropic | 21.1 | percent |
| Gemma 4 E2B | #129 | gemma-4-e2b | 20.8 | percent | |
| Gemma 4 E4B | #130 | gemma-4-e4b | 20.8 | percent | |
| Exaone 4.0 1.2B | #131 | exaone-4-0-1-2b | LG AI Research | 20.5 | percent |
| Granite-4.0-H-1B | #132 | granite-4-0-h-1b | IBM | 19.6 | percent |
| Llama 3.1 405B | #133 | llama-3-1-405b | Meta | 19 | percent |
| Llama 4 Maverick | #134 | llama-4-maverick | Meta | 17.8 | percent |
| GPT-4.1 nano | #135 | gpt-4-1-nano | OpenAI | 17.3 | percent |
| Llama 4 Scout | #136 | llama-4-scout | Meta | 15.5 | percent |
| Granite-4.0-H-350M | #137 | granite-4-0-h-350m | IBM | 14.6 | percent |
| Nova Pro | #138 | nova-pro | Amazon | 14 | percent |
| Granite-4.0-350M | #139 | granite-4-0-350m | IBM | 13.2 | percent |
| Nemotron Ultra 253B | #140 | nemotron-ultra-253b | NVIDIA | 11.4 | percent |
| Gemma 3 27B | #141 | gemma-3-27b | 10.5 | percent | |
| LFM2.5-VL-1.6B-Extract | #142 | lfm2-5-vl-1-6b-extract | LiquidAI | 8.5 | percent |
| Exaone 4.0 32B | #143 | exaone-4-0-32b | LG AI Research | 4.1 | percent |
| Phi-4 | #144 | phi-4 | Microsoft | 0 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | GLM-5.2Z.AI | Open weights | Public reference | 99.1% |
| 02 | GPT-5.4OpenAI | Closed weights | Source carrier | 98.9% |
| 03 | Claude Fable 5Anthropic | Closed weights | Public reference | 98.5% |
| 04 | GLM-5-TurboZ.AI | Closed weights | Public reference | 98.5% |
| 05 | GLM-5V-TurboZ.AI | Closed weights | Public reference | 98.5% |
| 06 | Step 3.7 FlashStepFun | Open weights | Source carrier | 98.5% |
| 07 | GLM-5Z.AI | Open weights | Public reference | 98.2% |
| 08 | GPT-5.5OpenAI | Closed weights | Source carrier | 98% |
| 09 | GLM-5.1Z.AI | Open weights | Source carrier | 97.7% |
| 10 | Grok 4.3xAI | Closed weights | Source carrier | 97.7% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Exact BenchLM registry variant GLM-5.2; bulk export does not retain a complete upstream harness configuration.GLM-5.2Exact identitySource label without a registered configuration ID Canonical product: glm-5-2 | 99.1% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.4; bulk export does not retain a complete upstream harness configuration.GPT-5.4Exact identitySource label without a registered configuration ID Canonical product: gpt-5-4 | 98.9% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Fable 5; bulk export does not retain a complete upstream harness configuration.Claude Fable 5Exact identitySource label without a registered configuration ID Canonical product: claude-fable-5 | 98.5% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GLM-5-Turbo; bulk export does not retain a complete upstream harness configuration.GLM-5-TurboExact identitySource label without a registered configuration ID Canonical product: glm-5-turbo | 98.5% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GLM-5V-Turbo; bulk export does not retain a complete upstream harness configuration.GLM-5V-TurboExact identitySource label without a registered configuration ID Canonical product: glm-5v-turbo | 98.5% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Step 3.7 Flash; bulk export does not retain a complete upstream harness configuration.Step 3.7 FlashExact identitySource label without a registered configuration ID Canonical product: step-3-7-flash | 98.5% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GLM-5; bulk export does not retain a complete upstream harness configuration.GLM-5Exact identitySource label without a registered configuration ID Canonical product: glm-5 | 98.2% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GPT-5.5; bulk export does not retain a complete upstream harness configuration.GPT-5.5Exact identitySource label without a registered configuration ID Canonical product: gpt-5-5 | 98% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant GLM-5.1; bulk export does not retain a complete upstream harness configuration.GLM-5.1Exact identitySource label without a registered configuration ID Canonical product: glm-5-1 | 97.7% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Grok 4.3; bulk export does not retain a complete upstream harness configuration.Grok 4.3Exact identitySource label without a registered configuration ID Canonical product: grok-4-3 | 97.7% | Source carriersource-checkedVersion & system2025 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
From result to context
Dual-control conversational agent benchmark for telecom support where agent and user must coordinate tool actions.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Low. Lifecycle: Active.