Two separate problems

A language model tries to be helpful, and it can’t reliably tell helpful from safe. Wrap a request in a story or claim to be a security researcher, and many models will explain things they should have refused.

With tools the risk grows. The model can now read a repository, open a pull request or call an API, so a prompt that manipulates it can make it act.

Part 1 described the foundation: one hardened base for an employee assistant and engineering agents. This post covers the two guardrails closest to the model:

  • Content safety controls what the model may receive and produce. That is the RAI content filter.
  • Tool access controls what the model may do. That is the governed MCP gateway.

Teams often add the first and forget the second. You need both, and they sit in different layers.

Content filter with harm thresholds and shields

Microsoft Foundry comes with a default responsible-AI policy. I use a custom policy instead and enforce it with Azure Policy so projects can’t opt out.

It adds three things to the base policy:

  • Harm categories (hate, sexual, violence, self-harm) filtered at Medium, on both prompt and completion. Medium and high severity are blocked, benign content passes. Stricter settings made the assistant refuse normal work; looser ones let real harm through.
  • Jailbreak shield on the prompt, for the “ignore your previous instructions” type of attack.
  • Indirect-attack shield on the prompt. This one is often underestimated. It catches instructions hidden in content the model reads, for example a malicious line in a document the RAG pipeline just retrieved. For a platform built on answering from the knowledge base, this is the most relevant attack.

PII redaction is deliberately not part of the model policy. It happens at the gateway and app layer, before and after the model.

How every message gets filtered Shields and harm filters sit on both sides of the model User prompt Prompt shields Jailbreak + Indirect-Attack (block on prompt) Harm filter · Medium checked on the prompt Model generates Harm filter · Medium checked on the completion Response to user Harm categories Hate · Sexual · Violence · Self-harm, each blocked at Medium+ severity PII redaction happens at the gateway and app layer, not in the model policy
Shields guard the prompt, harm filters guard both the prompt and the completion, and PII is handled outside the model policy.

As mentioned in Part 1, the RAI policy is account-scoped. If workloads are split across regions, each account needs its own copy of the policy before a deployment can reference it.

MCP gateway with access tiers

The content filter does nothing about what the model does with a tool. That needs a separate control in front of every tool call.

I put API Management in front of all MCP (Model Context Protocol) traffic and use the subscription as identity. Every tool request from an agent goes through the gateway, which does five things before the call reaches the actual MCP server.

How an agent reaches a tool Agent acts as an Entra identity Governed MCP Gateway (APIM) 1 · Map identity to a product tier (read / write) 2 · Enforce the tool allow-list 3 · Strip client-supplied X-MCP-* headers 4 · Inject a short-lived token (per call) 5 · Emit an audit event Tool MCP server receives a scoped, short-lived token Every call audited who · which tool · when, to Log Analytics
The gateway sits in front of every tool call, tiers access by identity, and never hands the agent a long-lived secret.

Two of these steps need more explanation.

No static tokens behind the gateway. The gateway doesn’t store a long-lived personal access token for the tool server. For each call it gets a new short-lived token from an internal token endpoint and injects it into the request. The agent still authenticates to the gateway, but it never holds a token for the tool itself. A leaked agent credential only gets as far as the gateway, where the allow-list and audit still apply, and a captured request expires within minutes. It is the same approach as the managed identities in Part 1, applied to tool traffic.

Tiers are enforced as APIM products. An Entra group maps to an API Management product, and the product defines which tools are reachable. The read tier can search and read. The write tier can create branches, add files and open pull requests, but it does not include merge or delete. The agent proposes, a human merges. The next post covers this in detail; the point here is that the gateway enforces it, independent of how the agent behaves.

Read tier Write tier
Purpose answer questions from repos and docs propose changes as pull requests
Allowed search, get, list create branch, add file, open PR
Excluded every write merge, delete
Typical identity ops and analyst agents the authoring agent
Token short-lived, minted per call short-lived, minted per call
Audited every call every call

Downsides

  • Medium needs tuning. At first a few legitimate engineering questions triggered the harm filter. Medium was still the right setting, but plan time to review what gets blocked.
  • Duplicated policy per account. Every model account needs its own copy. If you forget the second one, the deployment fails without an obvious reason.
  • APIM policy takes practice. Header stripping, token injection and per-tier routing all live in policy XML, which is not pleasant to debug and took several iterations.
  • Slight latency. Fetching a token per call adds a hop. I accept that to avoid long-lived secrets.

If you are building something similar or would set these thresholds differently, let me know.

Resources

  1. Microsoft, “Content filtering for Microsoft Foundry Models,” Microsoft Learn
  2. Microsoft, “Prompt Shields for jailbreak and indirect attacks,” Microsoft Learn
  3. Microsoft, “GenAI gateway capabilities in Azure API Management,” Microsoft Learn
  4. Anthropic, “Model Context Protocol,” modelcontextprotocol.io
  5. Microsoft, “Control AI model deployment with built-in policies,” Microsoft Learn