Two Trust Boundaries; One System.
Security & Privacy

Treat model output like user input

When strangers write the prompt and the answer renders on your page, you have two untrusted boundaries back to back. The layered posture we settled on.

Most guidance about prompt injection is written for one boundary: untrusted text goes into the model, and you want the model to keep behaving. Our public AI feature has that boundary and a second one right behind it. An anonymous visitor supplies both texts the model reads. The model then writes a summary and a list of suggestions. And that prose renders on a public page of ours.

So a stranger can try to make the model say something, and whatever the model says gets displayed under our name, on our domain. That is two attack surfaces, and defending only the first one leaves the second wide open.

The posture we landed on is layered, six things deep, and the reason it is six things rather than one is that we could not name a single layer we would bet the page on.

Boundary one: input into the prompt

Bound it, then render it once. Public inputs are capped tighter than the authenticated paths (30k and 6k characters). The template is filled in a single pass over both placeholders with a block-form substitution, so inserted text is inserted verbatim and is never re-scanned. (Two of our early bugs lived exactly here: a replacement-string backreference that let \' clone the template, and a two-pass fill that let a literal placeholder in input one pull input two into itself. Both are their own posts.) A hard ceiling on the assembled prompt precedes any billed call.

Delimit it as data. The prompt wraps each input in explicit fences and says, in plain words, that the fenced content is material to analyse, not instructions, and that anything imitating the fences is still content. This is guidance to the model, and we treat it as exactly that: helpful, not a security boundary. It raises the bar. It does not hold the line alone.

Gate it before spend. A vocabulary and density check rejects text that is not plausibly what it claims to be, before the model is called. This is partly a quality control and partly a cost control, and it happens to remove a whole class of "paste instructions in the JD field" attempts for free.

Boundary two: model output onto the page

Force structured output. The model is required to answer through a tool call with a fixed schema (a summary string, a list of matches, a list of suggestions), not free prose. An injected instruction that would like to append a paragraph of its own has no field to put it in. Structure is not a guarantee (a summary field is still a string) but it shrinks the target from "anything the model might say" to "a handful of bounded strings".

Sanitise at the render boundary. Right before the analysis is cached and sent, a small sanitiser runs over every free-text field. It unwraps markdown links to their text, strips bare URLs and angle brackets, and hard-caps each field (summary 500, each suggestion 300, each keyword 120). A summary that empties out after stripping falls back to a neutral default rather than a blank. This is where the "no links in output" prompt rule gets enforced instead of requested.

Display only. Model output drives React-escaped text and nothing else. It is never placed in an href, a src, a style, or anything that renders markup. So even a URL that survives every previous layer arrives as inert characters on the screen. This is the layer we would keep if we could keep only one, and it costs nothing.

Why six and not one

Walk an attack through it. Suppose the input persuades the model to include click https://evil.example/claim in a suggestion. It has to get past the delimiter framing (maybe). Then it has to fit inside a tool-call field (possible). Then the sanitiser strips the URL, leaving click. Then, had that failed, the string would render as text with no link. Four independent things have to fail for a live link to appear, and each of them was cheap to add.

The cost is real but small: a genuinely useful URL in a suggestion gets stripped, and long suggestions are clipped at 300 characters. On a public teaser page, that trade is easy.

The same rule, pointed at ourselves

The rule generalises past the product. While reviewing this work we found a sibling problem in our own tooling: a content workflow that asked a model to return a complete HTML document, saved it, and rendered it in a headless browser with a flag that grants local-file access. That is model-authored markup, executed, with permission to read the disk. A malicious or merely confused response could run script and read files.

The fix was the same shape as the product's last two layers. Have the model return text fields. Fill a trusted template ourselves. Render over a local HTTP server without the file-access flag. Model output is data; the markup is ours; the two never meet in a way that executes.

The one-line version

If a model reads untrusted text and its output reaches a page, a file, or a shell, treat the output with the same suspicion you would give a form field from a stranger, because that is, transitively, what it is. Bound and delimit on the way in, structure and sanitise on the way out, and never let the output become markup or an attribute. Six layers sounds like a lot until you count how many of them are a single function each.