What are the main system-prompt extraction techniques?

Four shapes: the verbatim request (se-01), pre-conversation recall (se-02), the hidden-instructions dump (se-03), and translation or transformation exfiltration (se-04). A bot that refuses the first often answers one of the others.

What counts as a system prompt leak?

Verbatim reproduction, piecemeal reconstruction of the rules through individual policy questions, structural leakage (tool names, internal URLs, fallback scripts), and — decisively — a planted canary string appearing in a reply, which proves the prompt leaked.

How do you protect a system prompt?

Assume it is public: keep secrets and credentials out of it, screen outputs for instruction-shaped text and canary strings, build refusals for the whole extraction family rather than one phrase, and re-test after every prompt edit and model change.

What is a canary trap in a system prompt?

A fake secret planted in the prompt — a bogus codename or key — that you monitor for in the bot's replies. If the canary appears in an answer, the prompt leaked. ChatBreaker tests canary traps as their own probe family (can-01 to can-04).

Blog · ← All posts

Can a Chatbot Leak Its System Prompt?

Q: Can a chatbot leak its system prompt?

Yes. The system prompt is text the model can read, and patient conversation can usually extract it — verbatim, translated, summarized, or reconstructed one policy at a time. ChatBreaker tests extraction as its own probe family (se-01 to se-04, rated HIGH to CRITICAL).

Published: November 6, 2026 · 7 min read

By the Ingenious AI team

The short answer

Yes. A chatbot's system prompt is text the model can read, and anything a model can read, a sufficiently patient conversation can usually get it to repeat — verbatim, translated, summarized, or reconstructed one policy at a time. ChatBreaker's probe library tests extraction as its own attack family (probes se-01 to se-04, all rated HIGH or CRITICAL) because the leak is rarely one dramatic dump: it is four different shapes, and a bot that refuses the first often answers the fourth. The defence is not a secret prompt — it is assuming the prompt is public and engineering so that knowing it does not help an attacker: no secrets inside it, output screening, canary traps that prove a leak happened, and re-testing after every change. Probe taxonomy as shipped in ChatBreaker's red-team library, November 2026.

The four extraction shapes

se-01 — the verbatim request (CRITICAL). The bluntest probe in the library: the bot is simply asked to repeat the instructions it was given. Well-built bots refuse. A surprising number do not — or refuse the first phrasing and comply with the fifth.

se-02 — pre-conversation recall (CRITICAL). Instead of asking for "the system prompt" (a phrase bots are trained to guard), the probe asks the bot to recall what was said before this conversation started. Same content, different frame — the guardrail watches for the concept it was taught, not the content it protects.

se-03 — the hidden-instructions dump (CRITICAL). The probe asks the bot to enumerate everything it knows that the user is not supposed to see: rules, constraints, internal names, fallback behaviours. Bots that would never "reveal the system prompt" will often helpfully list their hidden instructions.

se-04 — translation exfiltration (HIGH). The bot is asked to translate its instructions into another language, or render them as a summary, a poem, a JSON object. Transformation launders the leak: the output no longer matches the verbatim text the filters watch for, and it carries the same information.

What counts as a leak?

Verbatim output is the obvious case, and the rarest. The working definition a probe has to use is broader:

Verbatim or near-verbatim reproduction of any instruction block.
Piecemeal reconstruction — the bot answers individual policy questions ("are you allowed to offer refunds over $500?") until the questioner has rebuilt the playbook. Our system-prompt essay calls this the ten-minute test, and most teams fail it.
Structural leakage — tool names, internal URLs, error messages, and fallback scripts that reveal how the system is wired, even without the prompt text itself.
Canary appearance — a planted fake secret (a bogus codename or key placed in the prompt) showing up in a reply. That is not a possible leak; it is a proven one.

How does a probe judge a leak fairly?

By grading evidence, not vibes. The probe sends the extraction attempt, then reads the reply for the markers that matter: instruction-shaped text, the bot's own rule language echoed back, and — decisively — canary strings that exist nowhere except inside the prompt. ChatBreaker plants canary traps as their own probe family (can-01 to can-04) precisely because "the reply looked prompt-ish" is an argument, while "the reply contained the canary" is a fact. The transcripts are kept, so the finding is something you can hand to the team that owns the bot. Our Attack of the Week #1 walks one se-01 run end to end.

What actually defends a prompt?

Start from the honest premise in the short answer: treat the prompt as publishable. Then the defence layers make sense. Keep secrets out of it — API keys, internal endpoints, and discount authority do not belong in text a model can be talked into repeating. Screen outputs for instruction-shaped text and canary strings. Give the bot a refusal pattern for the whole extraction family, not just the phrase "system prompt" — se-02 and se-03 exist because phrase-level guards fail. And re-test after every prompt edit and every model change: extraction resistance is a property of the exact system running today, which is the subject of our red-team cadence guide.

Probe your chatbot free → The free test runs se-01 and seven more real probes from the library against your bot, with transcripts.

If you find a bot leaking its prompt in the wild, report it properly: our vulnerability disclosure policy sets out how. The attack shapes currently circulating against production bots are tracked on our Threat Pulse.