A helpful assistant will help you right out of a locked door
Out of the box, a language model is relentlessly helpful. That is the whole product. It is also the problem, because “helpful” and “safe” are not the same thing, and the model cannot tell the difference on its own. Ask it nicely, wrap the request in a story, tell it you are a security researcher, and a lot of models will happily walk you through something they should have refused.
Then you give that same model tools, and the stakes change completely. Now it is not just producing text. It can read a repository, open a pull request, call an API. A prompt that talks the model into misbehaving is now a prompt that talks the model into doing something.
In Part 1 I laid out the foundation: one hardened base carrying an employee assistant and engineering agents. This post is about the two guardrails that sit closest to the model, and they solve two genuinely different problems:
- Content safety decides what the model is allowed to say and be told. This is the RAI content filter.
- Tool access decides what the model is allowed to do. This is the governed MCP gateway.
People tend to bolt on the first and forget the second. You need both, and they work in completely different places.
The content filter: harm thresholds and shields
Microsoft Foundry ships a default responsible-AI policy, but the default is a starting point, not a finished answer. The useful move is a custom policy you control and enforce with Azure Policy so no project can quietly opt out.
Mine layers three things on top of the base policy:
- Harm categories (hate, sexual, violence, self-harm) filtered at a Medium threshold, on both the prompt and the completion. Medium means medium and high severity get blocked while genuinely benign content passes. Set it too aggressive and the assistant refuses normal work; too loose and it lets real harm through. Medium is the setting I keep coming back to.
- A jailbreak shield on the prompt, which catches the “ignore your previous instructions” family of attacks.
- An indirect-attack shield on the prompt, which is the one people underestimate. This catches instructions smuggled in through content the model is reading, like a malicious line hidden in a document that the RAG pipeline just retrieved. For a platform whose whole point is answering from your knowledge base, that is exactly the attack surface that matters.
One thing the model policy deliberately does not do: PII redaction. That belongs at the gateway and app layer, in front of and behind the model, not inside the content filter. Keeping those concerns separate means each does one job well.
A detail that carries over from Part 1: the RAI policy is account-scoped. If you split workloads across regions, each account needs its own copy of the same policy before a deployment can bind to it. Same rules, applied in every place a model lives.
The MCP gateway: tiering what an agent can touch
Content safety keeps the model from saying the wrong thing. It does nothing about what the model can do once it holds a tool. That is a separate control, and it lives in front of every tool call.
The pattern I settled on puts API Management in front of all MCP (Model Context Protocol) traffic and treats the subscription as identity. Think of it as a bouncer for tool calls. Every request an agent makes to a tool goes through the gateway, and the gateway does five things before the call ever reaches the real MCP server.
Two parts of that are worth pausing on.
No static tokens past the gateway. The gateway does not store a long-lived personal access token for the tool server. For each call it mints a fresh, short-lived token from an internal token endpoint and injects it into the request. The agent still authenticates to the gateway, so it is not credential-free — but it never holds a token for the tool itself. That is the point: a leak stops at the gateway, where the allow-list and the audit trail still apply, and a captured request is useless minutes later. This is the same secret-less principle from Part 1, applied to tool traffic.
Tiers are products, not prompts. Access is enforced by mapping an Entra group to an API Management product, and the product decides which tools are reachable. A read tier can search and read. A write tier can create branches, add files, and open pull requests. Critically, the write tier does not include merge or delete. The agent proposes; a human disposes. That single exclusion is the whole subject of the next post, so I will not spoil it here beyond saying the gateway, not the agent’s good behaviour, is what enforces it.
| Read tier | Write tier | |
|---|---|---|
| Purpose | answer questions from repos and docs | propose changes as pull requests |
| Allowed | search, get, list | create branch, add file, open PR |
| Excluded | every write | merge, delete |
| Typical identity | ops and analyst agents | the authoring agent |
| Token | short-lived, minted per call | short-lived, minted per call |
| Audited | every call | every call |
What this costs you
As in Part 1, the honest tradeoffs:
- Medium is a real tuning decision. At first a couple of legitimate engineering questions tripped the harm filter. Medium was still the right call, but “set the threshold and walk away” is not how it goes. Budget time to watch what gets blocked and confirm it should have been.
- Account-scoped duplication. Every model account needs its own copy of the policy. Easy to forget the second one and watch a deployment fail with no obvious reason.
- Gateway policy is its own skill. APIM policies are powerful and not especially fun to debug. Header stripping, token injection, and per-tier routing all live in policy XML, and getting them right takes iteration.
- A tiny bit of latency. Minting a token per call adds a small hop. In exchange you never hold a long-lived secret. For anything touching real systems, that is a trade I take every time.
None of it makes the demo better. All of it makes the platform safe to hand real tools.
Where this fits in the series
- Part 1: Building a Governed AI Platform on Microsoft Foundry sets up the foundation these guardrails sit on.
- Part 3 goes deep on the write tier: how you let an agent author Terraform without ever giving it the right to merge, and why the human stays the single, enforced merge path.
- Part 4 covers the approval cockpit where those human decisions actually happen.
Content safety and tool access are different problems, solved in different layers. Get both in place and you can let a model talk to your users and your systems without lying awake about it.
If you are building something similar, or you would tune these thresholds differently, I would like to hear it.
Resources
- Microsoft, “Content filtering for Microsoft Foundry Models,” Microsoft Learn
- Microsoft, “Prompt Shields for jailbreak and indirect attacks,” Microsoft Learn
- Microsoft, “GenAI gateway capabilities in Azure API Management,” Microsoft Learn
- Anthropic, “Model Context Protocol,” modelcontextprotocol.io
- Microsoft, “Control AI model deployment with built-in policies,” Microsoft Learn