Ablations. Which parts actually carry the result.

A system with three escalation layers, a memory and an elimination rule can improve a number for reasons its authors did not intend. Ablations are how we find out which parts are load-bearing: by removing one at a time and re-running the same campaign against the same split.

Benchmarks

The arms. 0 of 7 have run.

Nothing here is published yet. The arms below are fixed; the runs are queued behind the primary benchmark work. Results land here one arm at a time.

Each ablation arm, what it asks, why it matters and whether it has run
Arm	What it asks	Why it matters	Status
Full ladder nothing to set: this is the default	What does the whole system do when nothing is removed?	The reference every other arm is measured against. Without it the rest are numbers with nothing to subtract.	Not run Config only
L1 + L2 `escalation.config.l2_patience: null`	What does refining the context add once L1 stalls?	Refining the context is the expensive escalation. If L2 carries the gain, the cheap layer is not the one doing the work.	Not run Config only
L1 only `escalation.config.l1_patience: 99`	How far does critique-guided generation get on its own?	Critique-guided generation is the part most people already do by hand. This is the honest floor for whether the rest of it is worth anything. Caveat. Not hermetic. Patience is off, but three structural rules still fire L2 regardless: l1_generate_unusable, l1_evidence_starved, l2_axis_yield_drought. Read this arm as L1 with L2 as a fault handler, not as L1 alone.	Not run Config only
No critique no switch exists yet	Is the round critique load-bearing, or is it the population size?	Separates the loop’s actual argument, that evidence steers the next round, from the boring explanation that more candidates find more. Caveat. Needs a small patch: a new campaign-config flag threaded through to the round loop. The switch exists in the code, but today only sweep-batch mode reaches it.	Not run Needs code
No early dropping `pobb.config.epsilon_elimination: false`	What does statistical elimination buy in spend, not accuracy?	Elimination is sold on spend, not accuracy. This arm is the only way to price it. Caveat. Turns off the ε-stop only. Its sibling degradation_fatal_fastpath is a separate stop and stays on by default, so a candidate can still end early on a fatal sample.	Not run Config only
No block catalogue `l1_generate.config.prompt_block_catalogue: "off"`	Does the reusable block library find the gain, or does the loop?	The library is borrowed prior art. If it carries the gain, the loop is a retrieval system with extra steps.	Not run Config only
No search memory no switch exists yet	How much of the gain is carried across runs rather than found fresh?	Tells you whether a fresh run on a fresh machine gets what the demo got, or whether the demo was standing on months of banked measurements. Caveat. Needs a design decision before it needs code. The axis index does not only feed the prompts: scoring’s stale-data recovery reads it too. Removing it removes two things, and the arm has to say which one it is testing.	Not run Needs code

Config only means the arm is runnable as shipped, with no patch. Needs code means the switch does not exist yet.

There is no per-flag switch on the command line. An arm is a campaign config, handed to new <dataset> --config <path>, and the keys above all sit under optimization.nodes in that file, which also means every arm on this page is one file you can write yourself and run on your own key.

Every arm holds the model, the split and the scoring fixed, and reports the same three numbers as the benchmarks. Accuracy alone would hide most of what an ablation is for. How we score

In the optimizer repo: Candidate elimination , Roadmap