Point it at the stack you already run.
Three things to do here: connect a pipeline, run the optimizer on your machine, and find or become a host.
What a connector is. It reads everything and writes one thing.
A connector is how PromptPotter reaches your pipeline. It reads the definition your backend already publishes: the nodes, the parameters, the prompts. It runs its loop against your own labelled data, and hands back a per-call overlay. Your code is never edited, never forked, never redeployed to be optimized.
- Reads
- your pipeline and its settings
- the prompts you run today
- your labelled examples
- Writes
- one per-call overlay
- Nothing else is touched. Your backend runs exactly as you deployed it.
| Connector | What it reaches | Source |
|---|---|---|
harbor | An agent run in a sandbox, graded by the task’s own verifier. | GitHub |
dspy | Your dspy.Module. PromptPotter also runs as a DSPy optimizer, PromptPotterOpt. | GitHub |
termnorm | A pipeline served over HTTP. It publishes its nodes at /pipeline and answers one sample at a time at /matches. | GitHub |
dbllmbench | A query-writing benchmark, run in a container. | GitHub |
promptpotter | The optimizer pointed at itself, so its own prompts and settings are the thing being tuned. | GitHub |
The list is whatever the registry holds on your install. This prints it, with where each one came from:
python -c "from promptpotter.connectors import connector_origins as o; print(*o().items(), sep='\n')"
A connector of your own is a Python package that registers under the
entry-point group promptpotter.connectors. A connector is
trusted code, not sandboxed.
Write a connector
Connect a pipeline. Three endpoints, then one command.
Any backend that publishes a pipeline definition is optimizable: model, retrieval, params and all. PromptPotter never edits the backend. Over HTTP, that means your backend serves these three.
| Endpoint | Method | What it does |
|---|---|---|
/pipeline | GET | The full schema: nodes, parameters, allowed values and the prompt templates of the LLM nodes. Read once, to find what can be tuned. |
/matches | POST | Run one query under the given parameters and return the prediction. |
/status | GET | Liveness. 200 when up. |
Bring a labelled dataset, input and output pairs as CSV, TSV, JSON,
JSONL or .xlsx, and say what the job is. A campaign refuses to start
without a backend to score against; the default address is
http://127.0.0.1:8000.
The backend contract
# your backend answers
curl http://127.0.0.1:8000/status
# start a campaign on your own examples
python -m promptpotter new tickets.csv --set task_description="classify support tickets by urgency"
python -m promptpotter resume
Run it on your machine. Or on someone else’s.
PromptPotter runs on someone’s machine: ours, yours, or your team’s. Free and open source, with no license keys and no seats.
- Hosted Sign in Sign in and use the public instance. Nothing to install.
- Your machine Run it on your machine Your examples and your API keys stay on your own machine, and there is no account to create before you can try it.
- Your server Become a host The same app as a service on a Linux box, reached through a tunnel and behind sign-in.
The command line runs where your data already is. It needs Python 3.13 or newer and a provider key. Start a campaign, close the laptop, pick it up where it left off. Calls go to the model provider you configure, on your own key.
The same steps, one at a time# install
pip install "promptpotter[all]"
# or from source
git clone https://github.com/PromptPotter/prompt-potter-optimizer.git
cd prompt-potter-optimizer
pip install -e ".[all]"
# your provider key
echo "OPENROUTER_API_KEY=your_key_here" > .env
# the app, at http://127.0.0.1:8001/
python -X utf8 -m uvicorn promptpotter.main:app --port 8001
What you download.
What makes a round cheap. Five settings you own.
None of these is a discount. Each one is a decision about where the next dollar buys the most evidence, and each is a setting you own, on your own machine, billed to your own key.
- sp_budget_origin
- Defaults to 40 samples against 20 for a candidate round. Origin rows are content-addressed, so they replay free into every candidate, every fork and every resume. Origin breadth is the one breadth you pay for once.
- Scoring and memory
- epsilon
- Defaults to 0.15: stop measuring a candidate once its probability of being the round’s best drops under 15%.
- Candidate elimination
- lives
- Opt-in. A campaign banks a life for every round that improves and loses one for every round that does not; at zero it stops itself.
- potter/pipeline.yaml
- Prior results
- Measurements are content-addressed by config and dataset. Resume a campaign and every sample it already paid for replays at zero spend. Close the laptop mid-run without buying those rows twice.
- Persistence and state
- set-limits
- Two hard ceilings, one in dollars and one in tokens; whichever trips first halts the run. The token ceiling exists because a free backend reports $0, and the dollar ceiling would never see it coming.
- Your first campaign
python -m promptpotter set-limits --max-usd N --max-tokens NFind or become a host. Pick who runs it.
Join a public instance below, or take the source and run the whole stack on your own keys.
| Host | Where | Address | Spots open | Join |
|---|---|---|---|---|
| David Streuli | Switzerland | app.promptpotter.com | 992 of 1,000 | Sign in |
PromptPotter is open source and whitelabel-ready: run the full stack on your own machine, put your brand on it, invite people you know, then request a listing here. The Linux deploy runs the app on 127.0.0.1 under systemd and reaches it through a Cloudflare tunnel, with no open ports on your router and no static IP.
# on the box, in the deploy-linux folder
cp deploy.config.example deploy.config
./bootstrap.sh