1. The AI Attack Surface

Introduction

In Chapter 1, you explored the remarkable capabilities of modern AI systems – from how LLMs process language to how agentic AI systems autonomously execute complex tasks. Every one of those capabilities – prompt processing, RAG retrieval, tool use, code execution, model fine-tuning – represents a surface that attackers can target.

This section maps the complete AI attack surface. You’ll learn the frameworks security professionals use to categorize and communicate about AI threats, understand who the threat actors are, and build a mental model that connects every vulnerability covered in the rest of this chapter back to a specific stage of the AI lifecycle.

Think of this section as your attack vocabulary. By the end, you’ll be able to walk into any conversation about AI security and speak the language.

The AI Lifecycle as an Attack Surface

Every stage of an AI system’s lifecycle – from initial training to production inference and ongoing monitoring – presents opportunities for attackers. The diagram below maps these stages and highlights where the most significant attack vectors exist.

graph LR
    subgraph "Training Phase"
        A["Data Collection<br/>& Curation"]
        B["Model Training<br/>& Pre-training"]
    end

    subgraph "Customization Phase"
        C["Fine-tuning<br/>& Alignment"]
        D["RAG Pipeline<br/>Setup"]
    end

    subgraph "Deployment Phase"
        E["Model Packaging<br/>& Distribution"]
        F["Infrastructure<br/>& API Setup"]
    end

    subgraph "Inference Phase"
        G["Prompt Processing<br/>& Generation"]
        H["Tool Use &<br/>Agentic Actions"]
    end

    subgraph "Monitoring Phase"
        I["Logging &<br/>Feedback Loops"]
    end

    A --> B --> C --> D --> E --> F --> G --> H --> I
    I -.->|"feedback"| A

    A ---|"Data Poisoning<br/>LLM04"| PA["Attack Vector"]
    B ---|"Backdoor Insertion<br/>LLM04"| PB["Attack Vector"]
    C ---|"LoRA Adapter Attacks<br/>LLM03"| PC["Attack Vector"]
    D ---|"RAG Poisoning<br/>LLM08"| PD["Attack Vector"]
    E ---|"Supply Chain<br/>LLM03"| PE["Attack Vector"]
    F ---|"Infrastructure Exploits<br/>LLM10"| PF["Attack Vector"]
    G ---|"Prompt Injection<br/>LLM01"| PG["Attack Vector"]
    H ---|"Excessive Agency<br/>LLM06"| PH["Attack Vector"]
    I ---|"Memory Poisoning<br/>LLM01"| PI["Attack Vector"]

    style PA fill:#8b0000,color:#fff
    style PB fill:#8b0000,color:#fff
    style PC fill:#8b0000,color:#fff
    style PD fill:#8b0000,color:#fff
    style PE fill:#8b0000,color:#fff
    style PF fill:#8b0000,color:#fff
    style PG fill:#8b0000,color:#fff
    style PH fill:#8b0000,color:#fff
    style PI fill:#8b0000,color:#fff

The key insight: there is no safe stage. Attackers target training data months before a model ships. They poison RAG corpora without touching the model itself. They craft prompts that override system instructions in real time. They steal models through inference APIs. Each section of this chapter maps to one or more of these lifecycle stages.

Who Are the Threat Actors?

Not all attackers are the same. Understanding who targets AI systems helps you assess risk and prioritize defenses.

OWASP LLM Top 10 (2025)

The OWASP Top 10 for LLM Applications is the industry-standard framework for categorizing LLM vulnerabilities. The 2025 edition reflects the current threat landscape, including risks specific to RAG systems, agentic AI, and model supply chains.

This is your reference framework for the rest of the chapter. Every attack technique covered in Sections 2-7 maps back to one or more of these categories.

graph TB
    subgraph "Input Attacks"
        LLM01["LLM01<br/>Prompt Injection"]
        LLM07["LLM07<br/>System Prompt Leakage"]
    end

    subgraph "Data & Training"
        LLM04["LLM04<br/>Data and Model Poisoning"]
        LLM03["LLM03<br/>Supply Chain"]
        LLM08["LLM08<br/>Vector and Embedding<br/>Weaknesses"]
    end

    subgraph "Output & Behavior"
        LLM02["LLM02<br/>Sensitive Information<br/>Disclosure"]
        LLM05["LLM05<br/>Improper Output<br/>Handling"]
        LLM09["LLM09<br/>Misinformation"]
    end

    subgraph "System & Operations"
        LLM06["LLM06<br/>Excessive Agency"]
        LLM10["LLM10<br/>Unbounded Consumption"]
    end

    style LLM01 fill:#8b0000,color:#fff
    style LLM02 fill:#8b0000,color:#fff
    style LLM03 fill:#8b0000,color:#fff
    style LLM04 fill:#8b0000,color:#fff
    style LLM05 fill:#8b0000,color:#fff
    style LLM06 fill:#8b0000,color:#fff
    style LLM07 fill:#8b0000,color:#fff
    style LLM08 fill:#8b0000,color:#fff
    style LLM09 fill:#8b0000,color:#fff
    style LLM10 fill:#8b0000,color:#fff

MITRE ATLAS: A Supplementary Framework

While OWASP provides the vulnerability categories, MITRE ATLAS (Adversarial Threat Landscape for AI Systems) offers a complementary framework that maps adversarial tactics and techniques specifically for AI and machine learning systems.

ATLAS currently catalogs 15 tactics and 66 techniques organized in a matrix similar to the MITRE ATT&CK framework familiar to cybersecurity professionals. It covers the full adversarial lifecycle from reconnaissance to impact.

OWASP Agentic AI Top 10 (2026)

The rise of agentic AI systems – AI that uses tools, makes decisions, and acts autonomously – created attack surfaces so different from traditional LLM usage that OWASP published a dedicated framework in 2026. Where the LLM Top 10 covers what can go wrong with model inputs, outputs, and data, the Agentic AI Top 10 covers what can go wrong when AI systems take actions in the real world.

In Chapter 1 Section 7, you learned about the agent loop: plan, select tool, execute, observe, decide. Each stage of that loop is a target, and the Agentic AI Top 10 maps those targets.

These categories will be covered in depth in Section 5: Agentic Attack Vectors. Here’s the preview:

Case Study: Microsoft LLMjacking Lawsuit (2024)