Case study

How we proved it.

The hard part of benchmarking a reasoning engine is not running it. It is finding a test the competition could not have already memorized. Here is how we got to a clean, blind measurement.

We started with the hardest public math.

Our first measurements used the toughest public competition mathematics from 2024 and 2025. ALEXIS Fusion answered them at 100% accuracy. Several frontier models returned their answers faster than we did.

The frontier's speed was recall, not reasoning.

When we looked closely at those fast frontier answers, the speed did not come from solving. These problems already live in public training data, so a model can retrieve a memorized result instead of deriving one. Fast, but contaminated. On a public set, absolute score measures memorization as much as capability.

So we built tasks no model could have seen.

We authored five original, complex tasks from scratch, after the training cutoffs of every model under test. The set was frozen and fingerprinted with a SHA-256 hash before any run. No gold answer ever appeared in any prompt. There was no path to the answer except to actually work it out.

Then we ran everyone blind.

Every arm - ALEXIS Fusion and every frontier comparator - received the same inputs and was graded by the same automated extraction and the same equivalence rules. Runs were recorded to signed archives. Cost and latency are read back from those records, never estimated.

One arm solved all five, at the lowest cost.

Under the blind protocol, ALEXIS Fusion was the only arm to answer all five tasks correctly, and it did so at the lowest cost per correct answer of any arm in the field. When memorization is removed, the gap between recall and reasoning shows up in the numbers.

The blind result

Every arm, the same five frozen tasks, the same grader. Read the numbers straight from the archived run.

Arm	Accuracy	Correct	Cost	Cost / correct	Mean latency
ALEXIS FusionOurs	100%	5/5	$0.204	$0.041	70s
Qwen3.7 Max	60%	3/5	≥ $0.407	$0.136	556s
Grok 4.5	60%	3/5	$0.883	$0.294	255s
GPT-5.6 Sol	60%	3/5	≥ $1.043	$0.348	226s
GPT-5.5	60%	3/5	$1.094	$0.365	98s
Gemini 3.1 Pro	60%	3/5	$1.486	$0.495	163s
Fugu Ultra	60%	3/5	≥ $1.913	$0.638	495s
Opus 4.8	60%	3/5	$2.154	$0.718	161s
Fable 5	60%	3/5	$2.492	$0.831	88s
DeepSeek V4 Pro	40%	2/5	$0.093	$0.046	246s
Grok 4.3	40%	2/5	$0.280	$0.140	208s
GLM-5.2	40%	2/5	≥ $0.297	$0.148	444s

n = 5frozen tasks. Rows marked ≥ are lower bounds: the arm hit the wall-clock cap on at least one task, so its true cost and latency are higher. Sorted by accuracy, then cost.

Sanitized evidence pack

Check our work.

Every arm, every task. Preview or download each model’s full per-task log, the five frozen tasks, the answer key, and the protocol. Frontier logs include the model’s own worked solution. The ALEXIS Fusion log shows the outcome only: no model names, no routing, no providers. That line is deliberate.

Benchmark tasks

The five frozen tasks and the answer key.

tasks.json

2.4 KB

The five original tasks (statements only), with the dataset fingerprint.