# ALEXIS Fusion - Post-Cutoff Pilot: Methodology

Benchmark:      postcutoff_pilot_v1
Freeze tag:     postcutoff-pilot-v1-freeze-20260705
Dataset SHA-256: b5ef06357d5c0077ed5dbf35508eeee46a73dfb22dbcc81f7510989601ca4595
Tasks:          5

## Why we built a private set

Our first measurements used the hardest public competition mathematics from
2024 and 2025. ALEXIS Fusion answered them at 100% accuracy, but several
frontier models returned their answers faster than we could.

When we inspected those fast frontier answers, the speed did not come from
reasoning. On problems that already live in public training data, a model can
recall a memorized result instead of deriving it. Fast, but contaminated.

Absolute scores on public sets therefore measure memorization as much as
capability. To measure capability, the tasks have to be ones no model could
have seen.

## The blind protocol

1. Five original tasks were authored from scratch, after the training cutoffs
   of every model under test.
2. The task set was frozen and fingerprinted (SHA-256 above) before any run.
3. No gold answer ever appeared in any prompt, on any arm.
4. Every arm - ALEXIS Fusion and every frontier comparator - received the same
   inputs and was graded by the same automated answer extraction and the same
   equivalence rules.
5. Runs were recorded to signed archives. Cost and latency are read back from
   those records, not estimated.

## What we measure

- Accuracy: fraction of the five tasks answered correctly.
- Cost (USD): total metered spend to run the arm end to end.
- Latency: wall-clock time per task (mean and max).

The pilot is intentionally small (n = 5) and directional. It is arm-versus-arm
evidence under a contamination-controlled protocol, not a public leaderboard.
A larger held-out set is the next benchmark.