5. Layer 3: Secure Your AI Infrastructure

Introduction

Chapter 2 Section 4 ends by converting an unanswerable question – are we exposed to infrastructure attacks? – into four answerable ones:

Who can get an image scheduled on our GPU nodes? Who can edit a workflow in our orchestrator? Who shares an inference backend with our tenants? What does our public endpoint return besides text?

It then says those are “what Layer 2 and Layer 3 exist to close.” This section is the Layer 3 half of that answer, and the four questions are a fair test of it: if you finish here unable to answer them about a system you are handed, the section has failed regardless of how much posture-management vocabulary it taught you.

Layer 3 owns the stack underneath the model – GPU nodes, the serving container, the inference server’s internal sockets, the caches it shares between requests, the orchestrator that wires tools to the model, and the identities all of it runs as. It is also the layer most often reduced to a single product purchase, which is the failure this section is written against.

What Layer 3 Owns – and What It Cannot Stop

Section 2’s comparison table records Layer 3 as detective, with a flat No under “stops a live request,” and the second-broadest coverage in the Blueprint at 10 of 20 OWASP categories. Both halves of that are correct, and taken alone they mislead, because Layer 3 is really two different kinds of control wearing one number.

The detective half – AI-SPM – reports and blocks nothing. It tells you a GPU node is over-permissioned, an endpoint is unauthenticated, an API key has no rotation policy, an MCP server appeared last Tuesday. Every one of those is a real finding and none of them is an interception. Section 2 puts it bluntly: a layer with high coverage and no stopping power is telling you that a control somewhere else failed or was never configured.

The configured-boundary half enforces continuously and is still never in the request path. A seccomp profile denies a syscall. An egress rule drops a packet. A scoped credential refuses an API call. A per-tenant cache never holds another tenant’s prefix. These stop things – but none of them inspects a prompt, a response, or a tool argument. They are the permission shape, decided before the run, which is the phrasing Chapter 2 Section 6 already uses for excessive agency. Section 2’s “No” is a statement about the request path, and it is exactly right: nothing here reads the request.

The trade Layer 3 makes is the inverse of Layer 2’s. Layer 2 acts once, before anything runs, and is finished. Layer 3 acts continuously and never gets to see what the system is actually being asked to do.

Which Layer 3 Controls Are Yours

Chapter 1 Section 3 promised that “the layers stay constant, but which ones are yours is set by the choice you make here.” Layer 3 is where that promise is most expensive to ignore, because most of this section describes machinery that a cloud-API consumer will never touch – and a smaller, unglamorous subset that is theirs no matter what they chose.

Read this against the five deployment patterns from Chapter 1.

The fifth pattern, hybrid, has no column because it inherits the row of whichever path a request takes. A router that can fail over from a cloud API to a self-hosted fallback has silently handed you the self-hosted column.

Three consequences, and they are the point of the table:

1 · On a Cloud API, Layer 3 does not shrink to nothing – it moves. You own no GPU, no container, no socket, no cache. What remains is the orchestrator, the identity scope of every credential your agents hold, and discovery of what your organisation is actually calling. Those are not leftovers: the orchestrator is Chapter 2’s highest-value target in the stack and it runs on your infrastructure in every pattern here.

2 · Self-hosting takes the whole column at once, and the rows differ in how obvious they are. Endpoint authentication is obvious and gets done. The inference server’s internal ZeroMQ sockets, and whether the prefix cache is shared across tenants, are not obvious and do not get done – which is precisely why ShadowMQ and CVE-2025-46570 are in Chapter 2.

3 · A finding written against a control you do not own is not a finding. Run this table before the checklist at the end of the section. An organisation consuming a managed API that opens a finding about missing seccomp profiles on its serving containers has written a finding about someone else’s containers.

AI Security Posture Management (AI-SPM)

Traditional CSPM monitors cloud resources for misconfigurations: open storage buckets, permissive security groups, unencrypted databases. AI-SPM extends that to AI-specific resources, and the extension that matters most is discovery – many organisations cannot enumerate their own AI footprint.

Standard CSPM sees a GPU instance as a virtual machine. It does not know the instance is serving an unauthenticated inference API, that the vector database behind it has no access controls, or that an MCP server was connected to the agent framework last week.

Discovery scope is the row that does the most work, and it is also the row where the category’s marketing and its products diverge, so read it carefully. What an AI-SPM tool reliably gives you is AI-aware classification of assets it can already see – it knows the GPU instance is serving a model and the bucket holds embeddings. What it does not automatically give you is reach beyond its connection scope.

An organisation can hold a clean CSPM score across every managed account while running AI assets that were never in one – and, importantly, a clean AI-SPM score has the same blind spot, bounded by the accounts you connected.

The AI-SPM Workflow

graph TB
    DIS["<b>Discover</b><br/><small>Inventory the AI assets<br/>in connected scope:<br/>model endpoints, GPU clusters,<br/>vector stores, orchestrators</small>"]
    ASS["<b>Assess</b><br/><small>Evaluate configuration<br/>against AI security<br/>baselines: authentication,<br/>tenancy, credential scope</small>"]
    SCR["<b>Score</b><br/><small>Risk scoring based on<br/>exposure, data sensitivity,<br/>tool access, and<br/>blast radius</small>"]
    REM["<b>Remediate</b><br/><small>Prioritized guidance:<br/>what to fix first,<br/>how to fix it,<br/>expected risk reduction</small>"]

    DIS --> ASS --> SCR --> REM
    REM -.->|"Continuous<br/>reassessment"| DIS

    NOTE["<b>Every arrow here is a report.</b><br/><small>Nothing in this loop is in the<br/>request path. Remediation is a<br/>change someone makes, not a block.</small>"]
    REM -.- NOTE

    style DIS fill:#2d5016,color:#fff
    style ASS fill:#2d5016,color:#fff
    style SCR fill:#2d5016,color:#fff
    style REM fill:#2d5016,color:#fff
    style NOTE fill:#a85800,color:#fff

The loop is continuous because the estate is. A developer spins up a test endpoint, a team connects an MCP server, an internal tool starts calling an external AI API. AI-SPM’s job is to notice each of those that falls inside the scope it is connected to, and to notice without waiting for a review cycle.

Its limits follow from the same two properties. Discovery is triggered by an asset existing, so it reports after the fact. For a resource that sits there being misconfigured – an unauthenticated endpoint, an over-scoped role – after the fact is early enough, and this is where AI-SPM earns its place. For an attack that completes in one action, it is not, and no configuration of the tool changes that.

Posture Management for AI Resources

Different AI resources need different baselines, and the four Chapter 2 infrastructure cases land on four different ones.

GPU Cluster Security

GPU clusters are expensive, powerful, and shared across teams, which makes multi-tenancy the dominant risk. Several controls below act on the individual node, because a node is what an image gets scheduled onto.

Runtime Policy for the Serving Container

Layer 2 ends when the image is signed and admitted to the registry. What the running container may then reach is Layer 3’s, and it is the half of container security that a scanning pipeline does not provide.

Admission control is what makes the other five rows real. Each is a property of a workload manifest, and a property nobody enforces at deploy time is a property some manifest will lack.

Model Serving Endpoint Hardening

Every model serving endpoint is an API, and needs what any API needs:

• Authentication required – no unauthenticated endpoints in production. This is the single most common AI-SPM finding, and “internal” is not authentication.
• TLS in transit, including on internal hops.
• Rate limiting per user and per endpoint, coordinated with Layer 5 rather than duplicated.
• Request size caps – maximum tokens and payload size enforced at the endpoint.
• Health and latency monitoring, which is also the signal that a side channel is being probed.

Shared State Between Requests

To serve many users economically, inference servers share the GPU, the batch, and the prefix cache. Caching the attention state for a prompt prefix means a repeated opening is not recomputed – an enormous performance win that makes response time depend on what other people have asked.

CVE-2025-46570 is that dependency turned into an oracle: cache hits drop time-to-first-token measurably, and published research reports hit/miss detection at 99% accuracy and system-prompt recovery at roughly 111 queries per token. Chapter 2’s conclusion is the one to carry here – the benefit and the leak are the same mechanism, so it cannot be patched away, only scoped.

The Layer 3 controls are tenancy decisions, and each costs performance:

Chapter 2’s generalisation is worth applying before the next CVE exists: any cross-request optimisation in a multi-tenant AI system is a candidate side channel. Batch scheduling that lets one tenant’s request length shape another’s latency is the same shape as the prefix cache, and has no CVE yet.

Cloud AI Service Considerations

Organisations using managed services – Amazon Bedrock, Microsoft Foundry, Google Vertex AI – inherit the provider’s infrastructure security and keep the configuration:

• IAM policies: are service roles least-privilege? Chapter 2 records a Vertex AI service agent that inherited excessive default permissions and was used to pivot to restricted internal artifacts. Default permissions on a managed platform are a decision someone else made for you – inheriting them is a choice you are making by not looking.
• Network configuration: are endpoints internet-facing when they should be VPC-internal?
• Logging: are the provider’s audit logs enabled for AI service operations, and does anyone read them?
• Data residency: are training jobs and inference endpoints in approved regions?
• Model registry access: who can deploy to a managed endpoint, and is there an approval gate?

Risk Insights and Prioritization

Findings are not equal. An unauthenticated endpoint handling PII outranks a missing audit log on a test cluster, and an AI-SPM console that presents both as “findings” has moved the triage problem rather than solved it.

What Raises an AI Finding’s Priority

• Data sensitivity – what the system can reach. PII, financial and health data raise the score.
• Tool access – whether the system can take real-world actions. This is the largest single multiplier, because it converts a wrong answer into an effect.
• Exposure surface – internet-facing or internal, and how many paths reach it.
• Regulatory context – GDPR, HIPAA, the EU AI Act.
• Blast radius – what else is reachable from here if this is compromised. For an agent, this is the union of every tool granted to any agent it can delegate to, not its own tool list; Chapter 2 Section 5 shows a research agent and a writer agent that are each safe alone and hold the lethal trifecta together.

Prioritization Framework

Response times below are an example SLA, not a standard – the tiers are the transferable part.

The MEDIUM row is the one to argue with. An internal GPU cluster reachable from the enterprise network is one scheduled image away from host root and every tenant on it. “Internal-only” caps a score on exposure while the blast radius says otherwise – which is the standing weakness of exposure-weighted scoring, and the reason a scoring model is an input to triage rather than a substitute for it.

Securing the AI Orchestration Layer

The orchestration layer – MCP servers, function-calling frameworks, agent orchestrators such as n8n and LangGraph, custom tool integrations – is the bridge between “AI that generates text” and “AI that takes actions.” Chapter 2 calls it the highest-value target in the stack and usually the least hardened, because it is treated as internal tooling rather than production infrastructure.

The Orchestrator Is a Credential Store

An orchestrator’s job is to hold every credential the AI system needs: model API keys, database connection strings, internal service tokens, cloud roles. Remote code execution there is not one compromised application – it is the credential store for the whole AI estate, plus the network position those credentials are normally used from.

The Tool-Call Pipeline

graph TB
    REQ["Agent Request<br/><small>Tool call from<br/>LLM or agent</small>"]
    AUTH["Authenticate<br/><small>Which agent, acting<br/>for which user?</small>"]
    ALLOW["Allowlist<br/>Check<br/><small>Tool approved for<br/>this agent?</small>"]
    ENT["Entitlement<br/>Check<br/><small>May <b>this user</b> do this,<br/>not just this server?</small>"]
    SANDBOX["Sandbox<br/>Execution<br/><small>No credentials,<br/>no egress</small>"]
    VALIDATE["Output<br/>Handling<br/><small>Treat result as<br/>untrusted data</small>"]
    RESP["Response<br/><small>Returned to<br/>the agent</small>"]
    BLOCK["Blocked<br/><small>Unauthorized or<br/>out of scope</small>"]

    REQ --> AUTH
    AUTH -->|"Verified"| ALLOW
    AUTH -->|"Failed"| BLOCK
    ALLOW -->|"Approved"| ENT
    ALLOW -->|"Denied"| BLOCK
    ENT -->|"Entitled"| SANDBOX
    ENT -->|"Not entitled"| BLOCK
    SANDBOX --> VALIDATE
    VALIDATE --> RESP

    style REQ fill:#1565c0,color:#fff
    style AUTH fill:#2d5016,color:#fff
    style ALLOW fill:#2d5016,color:#fff
    style ENT fill:#2d5016,color:#fff
    style SANDBOX fill:#2d5016,color:#fff
    style VALIDATE fill:#a85800,color:#fff
    style RESP fill:#1565c0,color:#fff
    style BLOCK fill:#8b0000,color:#fff

Two nodes here are the ones usually missing. The entitlement check is the subject of the next section. The output handling step is orange rather than green deliberately: a tool result is untrusted input to the next model turn, and Chapter 2 is clear that there is no reliable detector for injected instructions in it. Layer 3’s contribution is to ensure the result arrives as data the agent cannot be tricked into treating as authority; the filtering itself is Layer 5’s.

MCP Server Verification That Survives a Rug Pull

The intuitive control – review the server before you connect it – fails against the incident Chapter 2 actually documents. postmark-mcp shipped fifteen clean versions before v1.0.16 added a BCC on every outbound message. A code read, a reputation check and a one-time review would all have passed, because at review time there was nothing to find.

So the controls that work are the ones that act on change, not on first sight:

• Pin the version, never a floating tag. An unpinned dependency re-decides trust on every install.
• Require a reviewed diff on change – this is the control the rug pull is designed to defeat, and the only one that catches it.
• Bind approval to contents, not to a name. Cursor MCPoison (CVE-2025-54136) bound approval to the server’s name, so replacing the command it ran never re-prompted. An approval that is not bound to a content hash is an approval of a name.
• Maintain an inventory of which servers are connected to what. This is AI-SPM’s contribution, and it is what makes the other three auditable.
• Read tool descriptions as untrusted text. They enter the context window and the user never sees them – reviewing a server’s code does not review its prose.

Per-Agent Identity and Mediated Delegation

Multi-agent systems fail in a way single agents do not: a message from a named internal component arrives in the system’s own format, describing a task, carrying no privilege marker that distinguishes it from a web page. The trust is in the channel’s appearance.

Two Layer 3 controls address it:

• Per-agent identity. Each agent authenticates as itself. Without this, an orchestrator’s logs record one identity performing every action and nothing in them distinguishes a delegated task from an injected one.
• Mediated delegation. A subtask request carries the original user’s entitlements, not the orchestrator’s. An orchestrator that delegates under its own authority is a confused deputy with a task queue.

Identity and Access Management for AI

AI systems need identities. A serving endpoint needs credentials for its weights. An agent needs keys for external services. An orchestrator needs database credentials. How those identities are shaped determines whether a single compromise stays contained.

The Confused Deputy Is the Default

Chapter 2 Section 5 sends the confused deputy here, and it is the most important thing in this section because it is not a misconfiguration – it is what you get by building the obvious way.

Most MCP servers hold their own service credentials, not the user’s. The database in Chapter 2’s privilege escalation chain does not see “an agent acting for Alice”; it sees the service account in the .env file. The agent is a component that acts on your behalf while holding broader privileges than you have, so persuading it is enough to exercise privileges you were never granted.

The consequence Chapter 2 states, and the requirement Layer 3 has to meet:

Authorization has to be evaluated against the requesting user’s entitlements, not the server’s. An agent that correctly refuses to read another user’s records is still exploitable if the tool underneath it can read them.

That is a design requirement, not a policy setting, and it rules out the arrangement teams reach for first:

The third row is on-behalf-of delegation, and existing standards cover it: OAuth 2.0 token exchange, or the equivalent in your identity provider. The work is not inventing a mechanism – it is refusing the shortcut of a shared service account during prototyping, because that is the decision that is never revisited before production.

IAM Controls for AI

Layer 3 → OWASP Mapping

Layer 3 is named as a primary defense in ten of the twenty categories across the LLM Top 10 and the Agentic AI Top 10 – the second-broadest coverage in the Blueprint, and, as Section 2 insists, coverage is not stopping power.

Read the fourth column down and Layer 3’s boundary is one sentence: it decides what each component may reach, and never what any component was asked to do. That is also why its coverage is so broad and its stopping power so narrow – almost every attack in the course crosses a boundary this layer defines, and none of them is recognisable as an attack at the moment it crosses.

AI Scanner Cross-Reference

AI Scanner assesses vulnerabilities in the model – prompt injection susceptibility, system prompt leakage, adversarial robustness – which no infrastructure control changes. AI-SPM assesses the infrastructure the model runs on, which no model-level scan sees. They are complementary because their blind spots are: a model that resists injection served from an unauthenticated endpoint is exposed, and a hardened endpoint serving a model that leaks its system prompt is also exposed. Section 9 covers how the two feed the scan-protect-validate-improve loop.

TrendAI Vision One’s AI-SPM capability provides the discovery and assessment half of this layer for the cloud accounts you connect to it – classifying AI assets by role, assessing configuration against AI baselines, surfacing threat alerts, misconfigurations and potential attack paths across them – with the Cyber Risk Exposure Management module correlating those findings against enterprise risk posture. Note the scope limit stated above: assets outside a connected account, including the developer estate, are not in that picture, and shadow usage of external AI services is a Layer 5 question rather than a posture one. The integration argument is specific rather than general: the ownership table above shows Layer 3 findings spanning components with different owners, so a console that reports an over-scoped agent credential and the unpatched container toolkit under it is answering a question that two separate tools each answer halfway. What no console supplies is the configured-boundary half – runtime policy, cache tenancy, user-derived credentials. Those are design decisions, and buying the dashboard does not make them.