A task with its own rules.
A function-calling benchmark for tool selection, schema adherence, and argument correctness.
- Organisation
- Arcee AI
- Version
- 2026
Loading benchmark header
Loading benchmark registry
Agents / Benchmark profile
A function-calling benchmark for tool selection, schema adherence, and argument correctness.
Observed results
9 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| Qwen3.7-Max | #1 | qwen-3-7-max | Alibaba Cloud | 75 | percent |
| Qwen3.7-Plus | #2 | qwen-3-7-plus | Alibaba Cloud | 72.9 | percent |
| LFM2.5-8B-A1B | #3 | lfm2-5-8b-a1b | LiquidAI | 49.73 | percent |
| Mellum2-12B-A2.5B-Thinking | #4 | mellum2-12b-a2-5b-thinking | JetBrains | 45.6 | percent |
| Mellum2-12B-A2.5B-Instruct | #5 | mellum2-12b-a2-5b-instruct | JetBrains | 44.2 | percent |
| ZAYA1-8B | #6 | zaya1-8b | Zyphra | 39.22 | percent |
| MiniCPM5-1B | #7 | minicpm5-1b | OpenBMB | 25.15 | percent |
| LFM2.5-VL-450M | #8 | lfm2-5-vl-450m | LiquidAI | 21.08 | percent |
| LFM2.5-230M | #9 | lfm2-5-230m | LiquidAI | 21.03 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | Qwen3.7-MaxAlibaba Cloud | Closed weights | Source carrier | 75% |
| 02 | Qwen3.7-PlusAlibaba Cloud | Closed weights | Source carrier | 72.9% |
| 03 | LFM2.5-8B-A1BLiquidAI | Open weights | Source carrier | 49.7% |
| 04 | Mellum2-12B-A2.5B-ThinkingJetBrains | Open weights | Source carrier | 45.6% |
| 05 | Mellum2-12B-A2.5B-InstructJetBrains | Open weights | Source carrier | 44.2% |
| 06 | ZAYA1-8BZyphra | Open weights | Source carrier | 39.2% |
| 07 | MiniCPM5-1BOpenBMB | Open weights | Source carrier | 25.1% |
| 08 | LFM2.5-VL-450MLiquidAI | Open weights | Source carrier | 21.1% |
| 09 | LFM2.5-230MLiquidAI | Open weights | Source carrier | 21.0% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Exact BenchLM registry variant Qwen3.7 Max; bulk export does not retain a complete upstream harness configuration.Qwen3.7-MaxExact identitySource label without a registered configuration ID Canonical product: qwen-3-7-max | 75% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Qwen3.7 Plus; bulk export does not retain a complete upstream harness configuration.Qwen3.7-PlusExact identitySource label without a registered configuration ID Canonical product: qwen-3-7-plus | 72.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant LFM2.5-8B-A1B; bulk export does not retain a complete upstream harness configuration.LFM2.5-8B-A1BExact identitySource label without a registered configuration ID Canonical product: lfm2-5-8b-a1b | 49.7% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Mellum2-12B-A2.5B-Thinking; bulk export does not retain a complete upstream harness configuration.Mellum2-12B-A2.5B-ThinkingExact identitySource label without a registered configuration ID Canonical product: mellum2-12b-a2-5b-thinking | 45.6% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Mellum2-12B-A2.5B-Instruct; bulk export does not retain a complete upstream harness configuration.Mellum2-12B-A2.5B-InstructExact identitySource label without a registered configuration ID Canonical product: mellum2-12b-a2-5b-instruct | 44.2% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant ZAYA1-8B; bulk export does not retain a complete upstream harness configuration.ZAYA1-8BExact identitySource label without a registered configuration ID Canonical product: zaya1-8b | 39.2% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant MiniCPM5-1B; bulk export does not retain a complete upstream harness configuration.MiniCPM5-1BExact identitySource label without a registered configuration ID Canonical product: minicpm5-1b | 25.1% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant LFM2.5-VL-450M; bulk export does not retain a complete upstream harness configuration.LFM2.5-VL-450MExact identitySource label without a registered configuration ID Canonical product: lfm2-5-vl-450m | 21.1% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant LFM2.5-230M; bulk export does not retain a complete upstream harness configuration.LFM2.5-230MExact identitySource label without a registered configuration ID Canonical product: lfm2-5-230m | 21.0% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Qwen3.7 Max; bulk export does not retain a complete upstream harness configuration.Qwen3.7-MaxExact identitySource label without a registered configuration ID Canonical product: qwen-3-7-max | 75% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-07-27Observed Checked |
From result to context
A function-calling benchmark for tool selection, schema adherence, and argument correctness.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.