cyber-wyse.com  /  tools  /  promptwatch

Promptwatch

v1.0 · browser-only

Prompts change. Nobody tracks why. Paste a baseline and a candidate, see exactly what moved, then run both against a model with your own test inputs and assertions. A small regression harness for prompt operations — no server, no account, nothing leaves your browser except the calls you make to your own provider.

cyber-wyse toolkit Promptwatch · Token Window · Nexus Cross-Validation · Log Lab · Entropy Meter all tools →
1

Paste two prompts

A is the baseline — the prompt you use today, the one you know works. B is the candidate — the edited version you are thinking of switching to.

2

See what moved

The diff shows every word added or removed, and the summary says what kind of change each one is: a new rule, a dropped example, a tone shift, a length limit.

3

Test it, don't guess

Add a few inputs that have caused trouble before, write the rules the answer must follow, and run both prompts against a real model. Promptwatch tells you if B broke anything A got right.

New here? Hit Load example then Run A vs B — demo mode needs no API key and shows the whole loop in about five seconds.

Still not sure what "baseline" and "candidate" mean? Show me an example

Think of it as before and after. Say you run a small assistant that summarises meeting notes, and this is the prompt you've used for months:

A · Baseline — what you have now
Summarise the following meeting notes for the team. Be clear and concise.

Notes: {{input}}
B · Candidate — what you want to change it to
Summarise the following meeting notes for the team.
- Keep it under 150 words.
- End with a bulleted list of action items, each with an owner.
- Do not include anything that wasn't in the notes.

Notes: {{input}}

Someone complained the summaries were long and missed the action items, so B is the rewrite. Same job, different instructions.

Test inputs would be two or three real sets of meeting notes. Assertions would be rules like max words 150 and contains "action items". Promptwatch runs the same notes through A and B and checks the rules against both answers.

What you're looking for is a surprise. Maybe B now produces action items (two assertions newly passing) — but one set of notes that A handled fine now comes back with an invented owner for a task. That's a regression, and that's what this page exists to catch: before you switch the prompt over, not a week later when someone asks why Dave owns a task Dave never agreed to.

01Prompts

The prompt as it is now — your known-good starting point.

The proposed change — the version you want to check before shipping.

02What changed

The word-level diff appears here as you type.
added in Bremoved from A

Change summary

  • Waiting for two prompts.

03Test inputs

The messages, questions or documents your prompt has to handle. Each one is sent to both A and B. Three or four awkward real-world cases beat twenty easy ones.

If the prompt contains {{input}} the text is substituted there and sent as a single user message. Otherwise the prompt becomes the system message and the input is the user turn. Leave all inputs blank to run the prompt alone.

04Assertions

Rules every answer must obey — "never says refund", "under 120 words", "is valid JSON". They are checked against every output from A and B, and a rule that A passes but B fails is counted as a regression.

05Run

Ctrl+Enter anywhere runs the suite.
Key handling: your key is held in page memory and, if you tick the box, in this browser's localStorage. It is sent only in direct requests from your browser to api.anthropic.com or api.openai.com. This page has no backend and makes no other network calls. Use a key with a spend limit; revoke it when you are done.

06Results

Run the suite to see A and B side by side, with assertion results and an output diff per case.