A task with its own rules.
Tool-free variant of CharXiv that isolates raw visual reasoning ability without code execution or tool augmentation.
- Organisation
- CharXiv authors
- Version
- 2024
Loading benchmark header
Loading benchmark registry
Multimodal / Benchmark profile
Tool-free variant of CharXiv that isolates raw visual reasoning ability without code execution or tool augmentation.
Observed results
7 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| Claude Mythos 5 | #1 | claude-mythos-5 | Anthropic | 88.9 | percent |
| Kimi K3 | #2 | kimi-k3 | Moonshot AI | 84.8 | percent |
| Claude Opus 4.7 | #3 | claude-opus-4-7 | Anthropic | 82.1 | percent |
| Claude Opus 4.8 | #4 | claude-opus-4-8 | Anthropic | 80.5 | percent |
| Inkling | #5 | inkling | Thinking Machines Lab | 78.1 | percent |
| Inkling-Small | #6 | inkling-small | Thinking Machines Lab | 77.4 | percent |
| Claude Sonnet 5 | #7 | claude-sonnet-5 | Anthropic | 77 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | Claude Mythos 5Anthropic | Closed weights | Source carrier | 88.9% |
| 02 | Kimi K3Moonshot AI | Open weights | Source carrier | 84.8% |
| 03 | Claude Opus 4.7Anthropic | Closed weights | Source carrier | 82.1% |
| 04 | Claude Opus 4.8Anthropic | Closed weights | Source carrier | 80.5% |
| 05 | InklingThinking Machines Lab | Open weights | Source carrier | 78.1% |
| 06 | Inkling-SmallThinking Machines Lab | Open weights | Source carrier | 77.4% |
| 07 | Claude Sonnet 5Anthropic | Closed weights | Source carrier | 77% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Exact BenchLM registry variant Claude Mythos 5; bulk export does not retain a complete upstream harness configuration.Claude Mythos 5Exact identitySource label without a registered configuration ID Canonical product: claude-mythos-5 | 88.9% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Kimi K3; bulk export does not retain a complete upstream harness configuration.Kimi K3Exact identitySource label without a registered configuration ID Canonical product: kimi-k3 | 84.8% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Opus 4.7 (Adaptive); bulk export does not retain a complete upstream harness configuration.Claude Opus 4.7Exact identityclaude-opus-4-7-max Canonical product: claude-opus-4-7 | 82.1% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Opus 4.8; bulk export does not retain a complete upstream harness configuration.Claude Opus 4.8Exact identitySource label without a registered configuration ID Canonical product: claude-opus-4-8 | 80.5% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Inkling; bulk export does not retain a complete upstream harness configuration.InklingExact identitySource label without a registered configuration ID Canonical product: inkling | 78.1% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Inkling-Small; bulk export does not retain a complete upstream harness configuration.Inkling-SmallExact identitySource label without a registered configuration ID Canonical product: inkling-small | 77.4% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Sonnet 5; bulk export does not retain a complete upstream harness configuration.Claude Sonnet 5Exact identitySource label without a registered configuration ID Canonical product: claude-sonnet-5 | 77% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Mythos 5; bulk export does not retain a complete upstream harness configuration.Claude Mythos 5Exact identitySource label without a registered configuration ID Canonical product: claude-mythos-5 | 88.9% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-07-27Observed Checked |
Exact BenchLM registry variant Kimi K3; bulk export does not retain a complete upstream harness configuration.Kimi K3Exact identitySource label without a registered configuration ID Canonical product: kimi-k3 | 84.8% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-07-27Observed Checked |
Exact BenchLM registry variant Claude Opus 4.7 (Adaptive); bulk export does not retain a complete upstream harness configuration.Claude Opus 4.7Exact identityclaude-opus-4-7-max Canonical product: claude-opus-4-7 | 82.1% | Source carriersource-checkedVersion & system2024 Source-native system | BenchLM public datasets — 2026-07-27Observed Checked |
From result to context
Tool-free variant of CharXiv that isolates raw visual reasoning ability without code execution or tool augmentation.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.