Critique-guided generation
Each round an LLM proposes a population of candidates, scores them on your real dataset, and critiques the result. The next round builds on that evidence — never a random guess.
Concepts
Three ideas, one loop.
Almost everything PromptPotter does follows from three ideas. It never guesses — each round is steered by a critique of the last. It only escalates when the evidence says the current approach has stalled. And it leaves your backend untouched, so the thing being tuned is the prompt, never your code. The rest is detail.
Three nested layers
L1 generates every round. When it stalls, L2 refines how the task is framed; if that stalls too, L3 replans the whole strategy. Each layer fires only on real evidence.
Pluggable, backend untouched
Any backend that publishes a pipeline definition is optimizable — model, retrieval, params and all. PromptPotter never edits the backend; it tunes a per-call overlay.
What that actually takes
It doesn’t think it improved. It measured.
You already have the input
Whatever your AI does today, plus examples where you know what the answer should have been. Nothing new to build.
The output is a number, not an opinion
A version that scores higher on those same examples, and the before-and-after that shows how much it gained.
It stops when it stops helping
Usually about five minutes. Once the score flattens it stops on its own rather than spending more of your budget.
Nothing of yours is touched
It changes only what you said it may change. Walk away at any point and there is nothing to undo.
Manual
Six chapters, install to going deeper.
The full walkthrough lives in the optimizer repo, where it stays in step with the code. Each chapter opens on GitHub.
Docs · Connectors
Point it at the stack you already run.
A connector is how PromptPotter reaches your pipeline. It reads the definition your backend already publishes — the nodes, the parameters, the prompts — runs its loop against your own labelled data, and hands back a per-call overlay. Your code is never edited, never forked, never redeployed to be optimized.
Reads
- your pipeline and its settings
- the prompts you run today
- your labelled examples
Writes
- one per-call overlay
Nothing else is touched. Your backend runs exactly as you deployed it.