LLM Output Handling: How to Render AI Responses Safely

To handle LLM output safely, treat every model response as untrusted user input. Render it as text or sanitized markdown, restrict where its links and images can point, validate structured output in code, and never pass it unchecked to a shell, eval, a SQL query or a file path. LLM output handling matters because anyone who can influence the model's context can influence what it writes, and your app then delivers that text to browsers, databases and other systems.

Why LLM output is untrusted input

The user typing into the chat controls part of the prompt. So does every document, web page, email and tool result the model reads. An instruction hidden in any of those is indirect prompt injection, and its goal is usually to shape the output.

OWASP lists this risk as LLM05:2025 Improper Output Handling: insufficient validation, sanitization and handling of model output before it reaches downstream components. The results it names include cross-site scripting (XSS) in browsers, and server-side request forgery (SSRF) or remote code execution on backend systems. Its first mitigation is to treat the model like any other user.

Models also produce malformed JSON and wrong record IDs with no attacker involved. The same controls catch both.

Map every place model output goes

Start with an inventory of sinks: every place a response is displayed, stored, executed or forwarded. Each sink needs its own control.

The last row gets missed most often. Text written into a ticket, CRM note or log is rendered later by an app nobody built for model output, and the staff member opening it usually has more privileges than the customer who wrote it.

How to render LLM output safely in a chat UI

Default to text. React escapes strings in JSX, so the risk appears when someone reaches for dangerouslySetInnerHTML to make formatting work. The OWASP Cross Site Scripting Prevention Cheat Sheet recommends safe APIs that treat data as text, context-specific encoding where syntax is interpreted, and a maintained sanitizer when you must accept HTML. It also notes that React 19 blocks javascript: URLs in src and href, which doesn't replace your own URL validation.

Chat interfaces usually render markdown, so the renderer's defaults decide your exposure. The react-markdown documentation says the library is secure by default and escapes HTML, warns that an insecure urlTransform or plugins such as rehype-raw can reintroduce XSS, and recommends rehype-sanitize. Check whichever library you use for the same points:

• Is raw HTML in the markdown escaped or dropped?
• Which URL schemes are allowed in links and images?
• Do your plugins or custom components bypass those defaults?

If the UI streams tokens, sanitize every partial render, not only the finished message.

Stop data leaks through markdown images and links

Escaping HTML doesn't stop exfiltration. Suppose a summarizer reads a web page with a hidden instruction to put the user's earlier messages into an image URL. The browser fetches the image automatically, and the data reaches the other server without a click. LLM05's second attack scenario describes the same summarizer setup sending data to an attacker's server.

The fix belongs in the renderer. react-markdown's default transform allows any http or https image URL, so restrict image sources to hosts you control:

import Markdown, { defaultUrlTransform } from "react-markdown";
 
const IMAGE_HOSTS = new Set(["cdn.example.com"]);
 
function urlTransform(url: string, key: string) {
  const safe = defaultUrlTransform(url);
  if (key !== "src") return safe;
  try {
    return IMAGE_HOSTS.has(new URL(safe).hostname) ? safe : undefined;
  } catch {
    return undefined;
  }
}
 
export function Answer({ text }: { text: string }) {
  return <Markdown urlTransform={urlTransform}>{text}</Markdown>;
}

Back it up with a Content Security Policy (CSP), which LLM05 also recommends. An img-src directive limited to your own domains blocks the request even if a renderer change lets an image through.

Links need a lighter touch, because users click them on purpose. Show the full destination, open external links with rel="noopener noreferrer", and consider a confirmation step for domains outside an allowlist.

Validate structured output before you act on it

When output drives code, such as a JSON tool call or a record update, validate it against a schema before anything happens, and reject on failure. Use enums for fields with a fixed set of values.

Structured output features that constrain decoding guarantee the shape. They don't make the values safe. A valid {"invoiceId": "inv_123"} still needs a check that invoice inv_123 belongs to the current user. Look up records with the session's identity, and let the model choose only from options your code already filtered. Our guide to securing AI agents that can use tools covers scoping those tools.

Keep model output out of SQL, shells and file paths

LLM05's third scenario describes a user asking a text-to-SQL chat feature for a query that deletes every table. Run generated queries as a database role that can only read the views the feature needs, with a statement timeout and a row limit. Check the statement type with a SQL parser rather than a regular expression. When the model extracts values for a query you wrote, pass them as parameters.

Never pass output to a shell or eval. If the product must run generated code, for example for data analysis, use a throwaway sandbox with no network access, no credentials and CPU and time limits.

For files, give the model a list of IDs and resolve each ID to a path in your code. Never join model text into a path. For outbound requests, allowlist hosts and resolve the address before connecting, so a model-supplied URL can't reach 169.254.169.254 or another internal address.

Common LLM output handling mistakes

• Sanitizing in the web client while the mobile app, admin console or email renders the raw text.
• Adding rehype-raw or dangerouslySetInnerHTML to fix table formatting.
• Filtering <script> with a regular expression instead of using a maintained sanitizer.
• Telling the model "never output HTML" and treating that as a control.
• Allowing images from any host.
• Validating JSON shape but never checking that referenced records belong to the user.
• Testing only with friendly prompts, so dangerous output never appears.

How to test LLM output handling before launch

1. List the sinks. Start from the table above.
2. Ask for the dangerous form. Try to get the model to produce HTML with an inline event handler, a markdown image pointing to a test domain you control, a javascript: link, a write statement, a path with ../ or an internal URL. Use harmless marker strings.
3. Repeat it indirectly. Put the same requests in a document or web page the feature reads.
4. Check every surface. Open the result in the chat UI, mobile app, admin console and emails.
5. Watch the network. No request should reach a host outside your allowlist.
6. Keep the cases. Rerun them after every model, prompt, renderer or plugin change.

Our guide on how to test an LLM feature before launch shows how to write these as repeatable cases with pass criteria. For an outside review, our LLM and AI agent security testing includes output handling checks on how model output is rendered, stored and passed to other systems, with a retest after fixes. The AI app security checklist covers the wider basics.

Frequently asked questions

Is improper output handling the same as prompt injection?

No. Prompt injection is how an attacker steers the model, and output handling limits what the steered output can do. OWASP lists them separately, as LLM01 and LLM05. Injection can't be fully prevented, so you need both.

Can a moderation API make LLM output safe?

Not for this problem. Moderation classifiers flag categories such as harassment or violence. They can't tell whether a string is safe inside your HTML, SQL statement or file path.

Should we sanitize LLM output when we store it or when we display it?

Encode at the point of use, for that context. Storing the raw response and encoding at render covers every display surface, including ones added later. Sanitizing once at storage misses sinks that need different encoding, such as SQL or URLs.

Is it safe to show AI-generated code or commands to users?

Displaying them as text is safe if the renderer escapes them. The risk is running them automatically. Offer a copy button, and never a "run" button wired to your servers unless execution happens in a sandbox.