1. The AI Attack Surface

Introduction

In Chapter 1, you built the mental models: the context window as a flat trust boundary, retrieval as a pipeline with eight writable stages, agency as the difference between a system that answers and a system that acts. This chapter attacks every one of them.

The people who maintain the industry’s threat list opened their 2026 edition with the posture that should frame everything you read here:

Stop trying to build a model that cannot be fooled. Build the system around it, so that when the model is fooled, and it will be, nothing important breaks.

– Steve Wilson and Rock Lambros, project leads, OWASP GenAI LLM Top 10 (2026)

That is not resignation. It is the reason this chapter is organised the way it is. You are not learning attacks so you can enumerate them; you are learning them so you can find the place in an architecture where a successful attack stops being contained and starts being expensive.

This section gives you the map and the vocabulary. By the end you should be able to take a system you have never seen, place its components on a lifecycle, and name both the category that attacks each one and the Chapter 3 layer that defends it.

The AI Lifecycle as an Attack Surface

Every stage of an AI system’s lifecycle – from initial training to production inference and ongoing monitoring – presents opportunities for attackers. The diagram below maps these stages and names the attack that lands at each one.

graph LR
    subgraph "Training Phase"
        A["Data Collection<br/>& Curation"]
        B["Model Training<br/>& Pre-training"]
    end

    subgraph "Customization Phase"
        C["Fine-tuning<br/>& Alignment"]
        D["RAG Pipeline<br/>Setup"]
    end

    subgraph "Deployment Phase"
        E["Model Packaging<br/>& Distribution"]
        F["Infrastructure<br/>& API Setup"]
    end

    subgraph "Inference Phase"
        G["Prompt Processing<br/>& Generation"]
        H["Tool Use &<br/>Agentic Actions"]
    end

    subgraph "Monitoring Phase"
        I["Logging &<br/>Feedback Loops"]
    end

    A --> B --> C --> D --> E --> F --> G --> H --> I
    I -.->|"retraining feedback"| A

    A -.- PA["Data Poisoning<br/>LLM05"]
    B -.- PB["Backdoor Insertion<br/>LLM05"]
    C -.- PC["Malicious LoRA Adapter<br/>LLM04"]
    D -.- PD["RAG Corpus Poisoning<br/>LLM09"]
    E -.- PE["Artifact Substitution<br/>LLM04"]
    F -.- PF["Credential Theft<br/>LLM06"]
    G -.- PG["Prompt Injection<br/>LLM01"]
    H -.- PH["Excessive Agency<br/>LLM03"]
    I -.- PI["Memory Poisoning<br/>ASI06"]

    style PA fill:#8b0000,color:#fff
    style PB fill:#8b0000,color:#fff
    style PC fill:#8b0000,color:#fff
    style PD fill:#8b0000,color:#fff
    style PE fill:#8b0000,color:#fff
    style PF fill:#8b0000,color:#fff
    style PG fill:#8b0000,color:#fff
    style PH fill:#8b0000,color:#fff
    style PI fill:#8b0000,color:#fff

The key insight: there is no safe stage. Attackers target training data months before a model ships. They poison RAG corpora without touching the model itself. They craft prompts that override system instructions in real time. They steal models through inference APIs.

Notice also that the feedback arrow closes the loop. A monitoring pipeline that feeds production interactions back into retraining turns an inference-time attack into a training-time one, which is why the last stage points back at the first.

Here is the map for the rest of the course. Every row names a stage, the section of this chapter that attacks it, and the Chapter 3 layer that owns the control:

Two stages sit outside the training-to-inference chain and are easy to miss. Output consumption is a surface because the consumer – a human or a downstream parser – makes decisions on content the model generated. Edge deployment is a surface because a model running on a device you do not control has no server-side guardrail in front of it.

Who Are the Threat Actors?

Not all attackers are the same. The last column is the one that matters operationally: knowing who realistically targets your deployment tells you which control to fund first.

OWASP LLM Top 10 (2026)

The OWASP Top 10 for LLM Applications is the industry-standard framework for categorizing LLM vulnerabilities. The 2026 edition, published on 4 August 2026 by the OWASP GenAI Security Project, is the current release and the reference framework for the rest of this chapter.

graph TB
    subgraph "Input Attacks"
        LLM01["LLM01<br/>Prompt Injection"]
        LLM08["LLM08<br/>Hidden Context<br/>Exposure"]
    end

    subgraph "Data & Training"
        LLM05["LLM05<br/>Data and Model<br/>Poisoning"]
        LLM04["LLM04<br/>Supply Chain"]
        LLM09["LLM09<br/>Vector and Embedding<br/>Weaknesses"]
    end

    subgraph "Output & Behavior"
        LLM02["LLM02<br/>Sensitive Information<br/>Disclosure"]
        LLM10["LLM10<br/>Improper Output<br/>Handling"]
        LLM07["LLM07<br/>Misinformation"]
    end

    subgraph "System & Operations"
        LLM03["LLM03<br/>Excessive Agency"]
        LLM06["LLM06<br/>Unbounded Consumption"]
    end

    style LLM01 fill:#8b0000,color:#fff
    style LLM02 fill:#8b0000,color:#fff
    style LLM03 fill:#8b0000,color:#fff
    style LLM04 fill:#8b0000,color:#fff
    style LLM05 fill:#8b0000,color:#fff
    style LLM06 fill:#8b0000,color:#fff
    style LLM07 fill:#8b0000,color:#fff
    style LLM08 fill:#8b0000,color:#fff
    style LLM09 fill:#8b0000,color:#fff
    style LLM10 fill:#8b0000,color:#fff

What Changed in 2026, and Why It Matters

The 2026 edition was the first built on evidence as well as opinion. Alongside the community vote, the project classified 7,714 real incidents from public vulnerability and AI-harm databases, using the 6,639 that carried enough detail. The vote carried 75% of the weight and the incident data 25%.

The disagreements between the two are more instructive than the list itself:

Both numbering schemes will be in circulation for some time – tooling, audit reports and vendor mappings written before August 2026 all cite the 2025 identifiers. This table is your translation key:

Choosing a Framework

Four frameworks cover this ground, and they are not interchangeable. Each answers a different question:

The first three describe threats; AISVS describes what you check. Published 24 June 2026 and modelled on the ASVS that web application security already runs on, it is positioned by OWASP as supplying “the detailed controls to mitigate” the two Top 10 lists – so a category from either list becomes a set of AISVS requirements to verify. Chapter 3 Section 10 works through it, including the identifier discipline that keeps a citation readable across versions.

The boundary between the two OWASP lists is the one to internalise. In the project’s own words: the LLM list owns the risk when the model is a component inside your application; the moment that model becomes an actor – with tools it can call, memory it carries between sessions, and consequences it sets in motion – the risk moves to the Agentic list. Many real incidents sit right on that line, and neither list covers it alone.

MITRE ATLAS

MITRE ATLAS (Adversarial Threat Landscape for AI Systems) maps adversarial tactics and techniques for AI and machine learning systems in a matrix modelled on MITRE ATT&CK. As of release 2026.07 it catalogs 16 tactics and 178 techniques (101 techniques with 77 sub-techniques), alongside 37 mitigations and 68 real-world case studies.

OWASP Top 10 for Agentic Applications (2026)

Agentic AI – systems that use tools, make decisions, and act autonomously – created attack surfaces different enough from ordinary LLM usage that the OWASP GenAI Security Project published a dedicated list, released 9 December 2025. Unlike the LLM list it has not yet been revised, so the December 2025 edition is current – do not confuse it with the project’s separate State of Agentic AI Security and Governance report, which reached version 2.01 in June 2026.

In Chapter 1 Section 7 you learned the agent loop: plan, select tool, execute, observe, decide. Each stage of that loop is a target, and this list maps them. The ten categories are covered in depth in Section 5: Agentic AI Attack Vectors; here is the preview.

Case Study: Microsoft v. Storm-2139 (2024-2025)