Bring your dataset
Drop in labelled examples — the input/output pairs every candidate prompt is scored against.
PromptPotter is LLM-driven program evolution for prompts and pipeline parameters — running on your machine, against your backend. Declare the tunable params, hand it a labelled dataset, run one command, and walk away: the Potter generates candidates, scores them on your data, critiques the weak ones, and regenerates — round after round — until it converges on a prompt that holds up under measurement. It never edits your code. The whole thing is open source, and the contracts below are all you need to build on it.
Drop in labelled examples — the input/output pairs every candidate prompt is scored against.
Point PromptPotter at the pipeline you want to improve. It reads the tunable params and never edits your code.
Watch the loop generate, score and critique — round after round — until your numbers move.
Every round, L1 reads the prior winners and the critique log, then proposes a fresh population of candidate prompts. PoBB elimination (ε=0.05, n_min=4) culls the weak so spend tracks signal — no flat sweeps, no wasted queries.
round_0004/candidates.json winner_id "6a674185" composite 0.94 // +0.31 vs origin parent "3c1d9020" (round 3 winner) eliminated 11 of 16 // PoBB ε=0.05, n_min=4 critique "add explicit combinatorial count cue"
When L1 plateaus across rounds, L2 fires once. It reads the stalled critique stream and rewrites task_context — never pipeline_params. The next L1 round inherits the new framing and goes again, often with a step-change in fitness.
// L2 fired after 3 stalled rounds on hard samples task_context "This dataset rewards step-by-step combinatorial enumeration. Candidates that skip the count step consistently miss multi-hop entries — make the counting explicit before the answer." pipeline_params (unchanged — L2 never edits these)
L3 fires only when both L1 and L2 have stalled. It scraps the plan and writes a new one — different sample budget, different scoring composite, different exploration policy. The new plan becomes the contract for round five and on.
round_0007/l3_plan.json verdict "replan" budget samples_per_round: 64 → 128 composite accuracy + latency_penalty * 0.15 policy "prefer wide exploration over fine sweep" rationale "L2's task_context edit didn't move composite past 0.71 — the scoring shape is wrong, not the prompt"
promptpotter connector/ui polls every two seconds