LLM Guardrails at the Gateway: Block and Mask PII in Prompts and Responses

Apply one guardrail policy to every LLM app from the agentgateway UI: reject prompts that contain SSNs or card numbers, mask emails and phone numbers in responses, call cloud moderation services, and cap output tokens.

Users paste customer records into chat assistants, and models repeat contact details they find in retrieved documents. If each team writes its own PII filter, you end up with as many filters as teams, each with different rules and gaps. An AI gateway applies one guardrail policy to the prompts and responses of all your apps and models, with no app changes. With agentgateway, you:

  • Reject prompts that contain a Social Security number or credit card number
  • Mask email addresses and phone numbers in model responses
  • Send prompts to a cloud moderation service and block harmful content
  • Cap output tokens with a request transformation
  • Control whether blocked prompts are stored in the gateway's logs
One policy on every prompt and response: requests pass a PII regex (400 if an SSN or card number is found), OpenAI moderation (403 if harmful), and a max_tokens cap before reaching OpenAI; responses pass a mask step that replaces phone numbers and emails before reaching the app.
One policy on every prompt and response: requests pass a PII regex (400 if an SSN or card number is found), OpenAI moderation (403 if harmful), and a max_tokens cap before reaching OpenAI; responses pass a mask step that replaces phone numbers and emails before reaching the app.

Guard types

Open LLM → Guardrails in the UI and click Add guard under Request guards or Response guards. The Guard type list offers:

Guard type in the UI
YAML key
Checks with
Request
Response
Built-in detectors
regex with builtin rules
Ready-made patterns for email, phone number, credit card, US SSN, and Canadian SIN
Yes
Yes
Custom regex
regex with your own patterns
Regular expressions you write
Yes
Yes
OpenAI Moderation
openAIModeration
OpenAI's moderation model
Yes
Bedrock Guardrails
bedrockGuardrails
Amazon Bedrock Guardrails
Yes
Yes
Google Model Armor
googleModelArmor
Google Cloud Model Armor
Yes
Yes
Azure Content Safety
azureContentSafety
Azure AI Content Safety
Yes
Yes
Webhook
webhook
Your own guardrail service
Yes
Yes

Solo Enterprise for agentgateway adds a Microsoft Purview DLP guard that applies your existing Purview policies.

Guards are saved under llm.policies.guardrails, with separate lists for requests and responses, and agentgateway runs each list in order. They apply to every model.

Each task below shows two ways to make the change: Option A in the agentgateway UI, or Option B by editing config.yaml directly. Both produce the same config, so use whichever you prefer.

Reject PII in prompts, mask it in responses

Option A: In the UI

  1. Open LLM → Guardrails. Under Request guards, click Add guard and fill in the step 1 values below. Click Add guard.
  2. Under Response guards, click Add guard and fill in the step 2 values below. Click Add guard.
  3. The guards stay pending until you save them. Click View diff to see the YAML, then Save guardrails.

Step 1

Field
Value
Guard type
Built-in detectors
Action
Reject request
Built-in detectors
SSN and Credit card. Email starts selected; click it to deselect.
Rejection status
400
Rejection body
Request blocked: remove sensitive data (SSN or card number) and try again.

Step 2

Field
Value
Guard type
Built-in detectors
Action
Mask matched text
Built-in detectors
Email (already selected) and Phone
LLM Guardrails in the agentgateway UI after saving: a request guard that rejects 2 built-in detectors with 400, and a response guard that masks 2 built-in detectors.
LLM Guardrails in the agentgateway UI after saving: a request guard that rejects 2 built-in detectors with 400, and a response guard that masks 2 built-in detectors.

Option B: Edit config.yaml directly‍

llm:
  policies:
    guardrails:
      request:
      - regex:
          action: reject
          rules:
          - builtin: ssn
          - builtin: creditCard
        rejection:
          status: 400
          body: "Request blocked: remove sensitive data (SSN or card number) and try again."
      response:
      - regex:
          action: mask
          rules:
          - builtin: phoneNumber
          - builtin: email

A prompt with an SSN gets the 400 and the custom message, and the gateway never sends it to the provider. In a response, the gateway replaces matches with placeholders such as <PHONE_NUMBER> and <EMAIL_ADDRESS> before the app receives it.

Block harmful content with a moderation service

A moderation guard sends each prompt to a classification service. This one uses OpenAI's moderation model, so it needs the same key reference as the models.

Option A: In the UI

  1. On LLM → Guardrails, under Request guards, click Add guard and choose OpenAI Moderation.
  2. Fill in the form with the values in the table below.
  3. Click Add guard, then Save guardrails.
Field
Value
Moderation model
omni-moderation-latest
Backend policies
The two YAML lines below
Rejection body
Content blocked by moderation policy
backendAuth:
  key: "$OPENAI_API_KEY"

Paste the Backend policies lines rather than typing them; the editor's autocomplete drops the : when you press Enter.

The page now lists both request guards and the response guard:

LLM Guardrails in the agentgateway UI: request guards with built-in detectors (reject, 400) and OpenAI Moderation, and a response guard that masks with built-in detectors.
LLM Guardrails in the agentgateway UI: request guards with built-in detectors (reject, 400) and OpenAI Moderation, and a response guard that masks with built-in detectors.

Option B: Edit config.yaml directly

Add a second entry to the request list:

llm:
  policies:
    guardrails:
      request:
      - openAIModeration:
          model: omni-moderation-latest
          policies:
            backendAuth:
              key: "$OPENAI_API_KEY"
        rejection:
          body: "Content blocked by moderation policy"

A prompt the moderation model flags gets 403 by default. Choose Bedrock Guardrails, Google Model Armor, or Azure Content Safety instead to use the safety service your cloud already provides.

Cap output tokens

A transformation rewrites fields of the request before the gateway sends it. This one, on a single model, limits max_tokens to 50 whatever the app asks for.

Option A: In the UI

  1. Open LLM → Models and click Edit on the gpt-4o-mini row.
  2. Open Advanced, then Model policies, then Transformation.
  3. Under LLM request fields, set the field name to max_tokens and the CEL expression to min(llmRequest.max_tokens, 50). Click Add, then Save model.
The Edit model dialog for gpt-4o-mini in the agentgateway UI: Model policies → Transformation with max_tokens set to min(llmRequest.max_tokens, 50).
The Edit model dialog for gpt-4o-mini in the agentgateway UI: Model policies → Transformation with max_tokens set to min(llmRequest.max_tokens, 50).

Option B: Edit config.yaml directly

Add a transformation block to the model:

llm:
  models:
  - name: gpt-4o-mini
    provider: openai
    params:
      model: gpt-4o-mini
      apiKey: "$OPENAI_API_KEY"
    transformation:
      max_tokens: "min(llmRequest.max_tokens, 50)"

Results

Prompt
Result
Contains an SSN
400, blocked before the provider is called
Asks for help hurting someone
403, blocked by moderation
Asks the model to reply with a phone number and email
200, with both masked
Asks for 2,000 output tokens
Capped at 50 tokens, finish_reason: length

Decide what your logs keep

Guardrails protect what reaches the model. The gateway's own logging decides what your systems store. By default, agentgateway logs metadata for each request: status, model, tokens, cost, user, and which guardrail fired.

Option A: In the UI

Open LLM → Logs and click a blocked request. The entry shows the 400, the user, and the guardrail that rejected it:

The Logs entry for a blocked request in the agentgateway UI: HTTP 400, user alice, and the regex request guardrail marked Rejected.
The Logs entry for a blocked request in the agentgateway UI: HTTP 400, user alice, and the regex request guardrail marked Rejected.

To store prompt and completion text too, click Settings on the same page, select Include prompts and completions in logs, and click Save settings:

Log settings in the agentgateway UI with Include prompts and completions in logs selected.
Log settings in the agentgateway UI with Include prompts and completions in logs selected.

Option B: Edit config.yaml directly

Add a top-level frontendPolicies block:

frontendPolicies:
  accessLog:
    database:
      llm: full

With llm: full, the log database also stores the text of rejected prompts, including the PII the guard blocked. Decide where prompt text may be stored before you select that box.

Frequently asked questions

What are LLM guardrails?

LLM guardrails are checks on prompts and responses that block, mask, or flag content such as PII, secrets, jailbreak attempts, or harmful requests. Running them in an AI gateway applies one policy to all apps and models.

Can an AI gateway redact PII from LLM responses?

Yes. agentgateway's response guards mask matches from built-in detectors (email, phone number, credit card, SSN, Canadian SIN) or your own regular expressions before the response reaches the app.

Can I use Amazon Bedrock Guardrails or Google Model Armor with agentgateway?

Yes. agentgateway has built-in guard types for Amazon Bedrock Guardrails, Google Cloud Model Armor, Azure AI Content Safety, and OpenAI moderation, and a webhook type for your own service.

Do guardrails add latency?

Regex guards run inside the gateway and add little latency. Moderation services add a network call per request, so measure them against your latency budget.

Do guardrails also apply to MCP tool calls?

agentgateway has separate MCP guardrails for tool calls and results. See What is an MCP gateway?.