It runs your examples, changes one thing, and keeps what scored higher.
You get something that does the same job better, and a clear before-and-after score. If you can describe a good answer, you can use it.
What happens on each round. The winner starts the next one.
It stops on a rule, not on a fixed number of tries: the goal is reached, the budget is spent, or the score has stopped moving.
- 01
It writes the versions, so you don’t have to.
Every round, PromptPotter writes a fresh batch of variations and tries each one.
- 02
“Better” means measurably better.
Every version is graded against your own examples of a good answer. You don’t get a hunch. You get a number, before and after, so you can see exactly how much it gained.
- 03
It stops spending on versions that are clearly behind.
In one round from a real run, three versions were scored on the same 34 examples. The third was dropped after 7. The other 27 were never paid for.
- 04
It reads its own results, in plain English.
After each round it writes a short note on what helped and what to try next, then uses that note to write the next batch. When the score stops moving, a second layer changes what the first one looks at, and a third rewrites the plan.
| Layer | Fires | What it does |
|---|---|---|
| L1 generate, score, critique | every round | Writes a batch of versions, scores each on your examples, and reads the misses to steer the next batch. |
| L2 refine context | when L1 stalls | Changes which evidence L1 sees and how hard it explores. The task framing stays as you wrote it. |
| L3 modify plan | when L2 stalls | Rewrites the plan L1 works within, or proposes returning to an earlier round. |
Point it at a prompt, a skill, an agent, or at itself.
Whatever you point it at, the job is the same: write better versions, score each one on examples where you know the right answer, and keep only what measurably wins.
-
Prompt
Available today
One instruction, or every step of a pipeline at once.
The smallest target is a single model call: the instruction, the model answering it, and the settings around it. The larger one is a pipeline of several steps, where the wording, the model at each step and the thresholds between them are all adjustable together, because a better first step can change what the second step should say.
-
Skill
Available today, one skill at a time
The standing instructions an agent carries into every task.
A skill is a file an agent reads before it acts: how to approach an unfamiliar machine, when to check its work, how to decide it is done. It is scored by what the agent actually achieves with it, graded by the finished result rather than by anything the agent says.
-
Agent
Part shipped
An agent with many skills, each one kept honest.
The strongest agent is not one long instruction tuned as a whole. It is an agent equipped with many skills, where each skill is kept, measured and improved on its own, and the set is judged together: improving one skill must not quietly break another.
An agent run is scored today, and single skills are improved today. What is still to come is looking after the whole collection at once, including checking each skill against what the agent would do with no skill at all.
-
Itself
Runs today, from the command line
The optimizer is a target too, so it can improve itself.
The optimizer runs on prompts and settings of its own, so it can be pointed at itself: an outer run scores versions of the optimizer by how well each one improves something else. It is the same loop run one level up, not a fourth layer. It can never change how it is scored: the scorer and the held-out examples sit outside what it can edit.
Today that covers one design, and no result from it is published yet.
What is shipped and what is next, in the source: the roadmap
One loop. A ladder when it stalls. A memory underneath.
Your dataset and the prompt you have now go in; a better prompt and a measured gain against the origin come out. What happens between is one loop, an escalation it climbs only when stuck, and every measurement it has ever taken.
- 01 Dataset in Your dataset and the prompt you have now.
- 02 Check-in Settles the framing and sets the origin: your current score, measured once, so every later claim is against a real number.
- 03 The loop · L1, every round Generate n candidates, score them on your dataset, critique what to fix. Losing candidates are cut early, and the round winner becomes the next parent. Then it goes again.
- 04 Best prompt out Prompt and pipeline params, with a measured gain against the origin. Your models are never touched.
- flows forward
- loops back
- escalates & rewinds
- feeds every layer
The optimizer can also be pointed at itself. Off‑map.