Research

Which parts actually carry the result.

A system with three escalation layers, a memory and an elimination rule can improve a number for reasons its authors did not intend. Ablations are how we find out which parts are load-bearing — by removing one at a time and re-running the same campaign against the same split.

Nothing here is published yet. The arms below are fixed, and most of them are one config file away from running; the runs are queued behind the primary benchmark work. Results land here one arm at a time.

The arms

Config only — runnable as shipped, no patch. Needs code — the switch does not exist yet.

Full ladder Config only nothing to set — this is the default

What does the whole system do when nothing is removed?

The reference every other arm is measured against. Without it the rest are numbers with nothing to subtract.
L1 + L2 Config only l2_patience: null

What does reframing the task add once L1 stalls?

Reframing the task is the expensive escalation. If L2 carries the gain, the cheap layer is not the one doing the work.
L1 only Config only l1_patience: 99

How far does critique-guided generation get on its own?

Critique-guided generation is the part most people already do by hand. This is the honest floor for whether the rest of it is worth anything.

Caveat Not hermetic. Patience is off, but three structural rules still fire L2 regardless — l1_generate_unusable, l1_evidence_starved, l2_axis_yield_drought. Read this arm as L1 with L2 as a fault handler, not as L1 alone.
No critique Needs code no switch exists yet

Is the round critique load-bearing, or is it the population size?

Separates the loop’s actual argument — evidence steers the next round — from the boring explanation that more candidates find more.

Caveat Needs a small patch: a new campaign-config flag threaded through to the round loop. The switch exists in the code, but today only sweep-batch mode reaches it.
No early dropping Config only mechanisms.elimination.epsilon_elimination: false

What does statistical elimination buy in spend, not accuracy?

Elimination is sold on spend, not accuracy. This arm is the only way to price it.

Caveat Turns off the ε-stop only. Its sibling degradation_fatal_fastpath is a separate stop and stays on by default, so a candidate can still end early on a fatal sample.
No block catalogue Config only prompt_block_catalogue: "off"

Does the reusable block library find the gain, or does the loop?

The library is borrowed prior art. If it carries the gain, the loop is a retrieval system with extra steps.
No search memory Needs code no switch exists yet

How much of the gain is carried across runs rather than found fresh?

Tells you whether a fresh run on a fresh machine gets what the demo got, or whether the demo was standing on months of banked measurements.

Caveat Needs a design decision before it needs code. The axis index does not only feed the prompts — scoring’s stale-data recovery reads it too. Removing it removes two things, and the arm has to say which one it is testing.

There is no per-flag switch on the command line. An arm is a campaign config, handed to new <dataset> --config <path>, and the keys above all sit under optimization: in that file — which also means every arm on this page is one file you can write yourself and run on your own key.

Every arm holds the model, the split and the scoring fixed, and reports the same four numbers as the benchmarks — accuracy alone would hide most of what an ablation is for. How we score