A task with its own rules.
An OpenClaw agent benchmark from Kilo that measures successful task completion across standardized real-world agent workflows.
- Organisation
- Kilo Code
- Version
- 2026
Loading benchmark header
Loading benchmark registry
Agents / Benchmark profile
An OpenClaw agent benchmark from Kilo that measures successful task completion across standardized real-world agent workflows.
Observed results
2 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| Nemotron 3 Ultra | #1 | nemotron-3-ultra | NVIDIA | 90 | percent |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | #2 | nemotron-3-5-lightning-30b-a3b | NVIDIA | 85.37 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | Nemotron 3 UltraNVIDIA | Open weights | Source carrier | 90% |
| 02 | Open weights | Public reference | 85.4% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16; NVIDIA release evaluation; temperature=1.0; top_p=0.95NVIDIA Nemotron 3.5 Lightning 30B-A3BExact identitynemotron-3-5-lightning-30b-a3b-default Canonical product: nemotron-3-5-lightning-30b-a3b | 85.4% | Public referenceprovider-reportedVersion & system2026 NVIDIA NeMo Gym / NeMo Evaluator SDK consistent release harness | NVIDIA Nemotron 3.5 Lightning 30B-A3B BF16 model cardObserved Checked |
Exact BenchLM registry variant Nemotron 3 Ultra; bulk export does not retain a complete upstream harness configuration.Nemotron 3 UltraExact identitySource label without a registered configuration ID Canonical product: nemotron-3-ultra | 90% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Nemotron 3 Ultra; bulk export does not retain a complete upstream harness configuration.Nemotron 3 UltraExact identitySource label without a registered configuration ID Canonical product: nemotron-3-ultra | 90% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-07-27Observed Checked |
Exact BenchLM registry variant Nemotron 3 Ultra; bulk export does not retain a complete upstream harness configuration.Nemotron 3 UltraExact identitySource label without a registered configuration ID Canonical product: nemotron-3-ultra | 90% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 21 July 2026Observed Checked |
From result to context
An OpenClaw agent benchmark from Kilo that measures successful task completion across standardized real-world agent workflows.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.