1. Introduction to AI and LLMs

Introduction

Welcome to the first step on the road to understanding Generative AI and LLMs! This section provides a technical foundation for understanding how AI has evolved into today’s landscape of multimodal, reasoning-capable, and agentic systems – focusing on the architectural breakthroughs and implementation patterns that enable the frontier models built by OpenAI, Anthropic, Google, Meta, DeepSeek, Alibaba, and Mistral.

In 1950, Alan Turing opened his paper "Computing Machinery and Intelligence" with a profound question: “Can machines think?” He then set it aside as too ill-defined to answer, and proposed instead that we judge machines by what they can do – an instinct that turned out to be a good one. Today we’re building systems that process natural language, generate code, work through complex problems step by step, interpret images, and take autonomous action.

The Evolution of AI: A Technical Perspective

From Rules to Learning

Early AI systems relied on rigid, rule-based programming. For example, teaching a computer to recognize a cat required painstakingly writing rules like “If it has pointy ears and whiskers, it’s probably a cat.” However, this approach quickly fell short in handling real-world complexity. Intelligence isn’t about following static rules – it’s about learning and adapting.

Consider an early AI system designed to play chess. It would follow a set of predefined rules for each possible move. However, it couldn’t adapt to new strategies or learn from its mistakes, making it less effective against skilled human players.

The Machine Learning Breakthrough

The turning point came when researchers shifted from programming rules to teaching machines how to learn. Instead of manually coding every rule, they fed computers massive datasets – like millions of cat images – and let algorithms discover patterns on their own. This marked the birth of machine learning.

Instead of writing rules to identify spam emails, machine learning algorithms can be trained on a large dataset of emails labeled as spam or not spam, learning to identify spam based on patterns in the data.

The Neural Revolution

Deep learning took this further using neural networks – layers of interconnected nodes, loosely inspired by biological neurons, that pass signals forward through weighted connections. Each layer learns to detect increasingly abstract features: early layers might pick out edges in an image, later layers combine those into shapes, and later still into objects. The “deep” simply refers to having many such layers.

Voice assistants like Siri and Alexa use deep learning to understand and respond to spoken language, even with different accents and speech patterns.

Each era is defined less by its date than by what stopped being hand-written. That is the through-line worth holding on to:

graph TB
    accTitle: The evolution of AI, from hand-written rules to agentic systems
    accDescr: Six eras in sequence. Rule-based systems, 1950s to 1980s, where humans write every rule. Machine learning, 1980s to 2000s, where the rules are learned from labelled data. Deep learning, 2010s, where the features are learned too, in layers. Transformers, 2017, where every token is weighed against every other. Large language models at scale, 2020s, where scale turns next-token prediction into general capability. And multimodal, reasoning and agentic systems from 2023 onward, where the model acts through real tools. Each step moves work from the human to the learned model, and the final step is what creates the attack surface this course addresses.

    A["<b>Rule-based systems</b><br/><small>1950s-1980s<br/>Humans write every rule</small>"]
    B["<b>Machine learning</b><br/><small>1980s-2000s<br/>The rules are learned<br/>from labelled data</small>"]
    C["<b>Deep learning</b><br/><small>2010s<br/>The features are learned<br/>too, layer by layer</small>"]
    D["<b>Transformers</b><br/><small>2017<br/>Every token weighed<br/>against every other</small>"]
    E["<b>LLMs at scale</b><br/><small>2020s<br/>Scale turns next-token<br/>prediction into<br/>general capability</small>"]
    F["<b>Multimodal, reasoning, agentic</b><br/><small>2023 onward<br/>The model acts:<br/>tools, steps, many modalities</small>"]

    A --> B --> C --> D --> E --> F

    style A fill:#4a4a4a,color:#fff
    style B fill:#1a3a5c,color:#fff
    style C fill:#1565c0,color:#fff
    style D fill:#1565c0,color:#fff
    style E fill:#1565c0,color:#fff
    style F fill:#a85800,color:#fff

Read the last box carefully, because the rest of this course lives there. Everything up to LLMs at scale describes a system that produces text. The moment a model also takes actions – calls a tool, reads a file, sends a request – the consequences of a wrong output stop being confined to the answer. Section 7 builds that shift properly, and it is the reason Chapter 2 has a whole section on agentic attack vectors.

A Language Leap: Transformers

In 2017, a paper with the memorable title "Attention Is All You Need" introduced the Transformer architecture, and it reshaped natural language processing. Transformers are exceptionally good at using context. Consider the word “bank” in these two sentences:

• “I walked along the bank and watched the current pull the leaves downstream.”
• “I walked into the bank and asked to speak to someone about a mortgage.”

Same word, entirely different meaning – and the only thing that distinguishes them is the surrounding text. Earlier architectures processed sentences one word at a time and struggled to carry that context. Transformers weigh every word against every other word at once, so “current” and “downstream” can pull “bank” toward one meaning while “mortgage” pulls it toward another.

How an LLM Actually Works, in Outline

Three ideas do most of the work. You will meet all of them properly in Section 4, but the outline matters now because almost everything else in this course builds on it.

1. Tokens. A model does not see letters or words. Text is first chopped into tokens – roughly word-sized pieces, though common words are usually a single token and unusual ones get split into several. “Unbelievable” might become un + believ + able. Every input and every output is a sequence of these tokens.

2. Attention. For each token, the model computes how much every other token in the context should influence it. This is the attention mechanism, and it is the Transformer’s central innovation. Because attention looks at the whole sequence simultaneously rather than walking through it in order, Transformers both capture long-range context far better than their predecessors and train efficiently on modern hardware.

3. Next-token prediction. Here is the part that surprises people. An LLM does exactly one thing: given the sequence of tokens so far, it predicts a probability distribution over what the next token should be. It picks one, appends it, and repeats. Sentence by sentence, essay by essay, an entire response is produced one token at a time.

The Transformer is also what made Generative AI (GenAI) – systems that create new content such as text, images, audio, or code – work at scale for language. Generative models existed before 2017, but the Transformer, combined with enormous training datasets, is what turned generation from a research curiosity into something practically useful.

That combination produced the Large Language Models this course is about. They are strikingly capable: they hold context across long documents, connect ideas across domains, and produce fluent, useful output. Just keep in mind what is underneath – a very sophisticated engine for predicting the next token, not a mind that comprehends.

Today’s Landscape: Beyond Text Generation

The AI landscape has evolved dramatically beyond the early days of text-only chatbots. Today’s systems span several capabilities, each with different trade-offs. Note that these are not mutually exclusive categories – a single modern model is often multimodal and capable of extended reasoning and available in several sizes.

Capabilities and Limitations

Modern LLMs can assist in a remarkable range of tasks, including:

• Writing essays, code, and creative content
• Translating languages and summarizing reports
• Analyzing images, charts, and documents (multimodal capability)
• Reasoning through math, logic, and science problems (with reasoning effort turned up)
• Automating cybersecurity workflows such as analyzing threat reports or detecting phishing attempts
• Generating and debugging code across dozens of programming languages

Understanding Prompts: The Basic Building Block

At their core, LLMs work by responding to “prompts” – text inputs that tell the model what we want it to do. Think of a prompt as a conversation starter or instruction that guides the AI’s response. The way we phrase these prompts significantly impacts the quality and usefulness of the AI’s response. For example:

Prompt:   Explain quantum computing like I'm 10 years old.

Response: You know how a light switch is either on or off? Regular computers
          think like that -- everything is a 1 or a 0. A quantum computer uses
          bits that can be a bit of both at the same time, which lets it try
          lots of answers at once instead of one after another...

Change the framing and you change the output entirely:

Prompt:   Explain quantum computing to a systems engineer evaluating
          post-quantum cryptography migration. Be concise.

Response: Quantum computers exploit superposition and entanglement to evaluate
          certain problem structures in parallel. The migration-relevant result
          is Shor's algorithm, which factors large integers in polynomial time
          and therefore breaks RSA and elliptic-curve cryptography...

Same topic, same model. The prompt did all the work.

A well-structured prompt typically includes:

• Clear instructions about what you want
• Relevant context or background information
• Any specific requirements for the format or style of the response

This careful crafting of prompts has evolved into its own discipline known as Prompt Engineering – both an art and a science that involves creating effective instructions for AI models. A skilled prompt engineer knows how to break down complex tasks into clear directives, provide the right context, and set appropriate constraints that guide the model toward producing accurate and useful responses.

Limitations and Vulnerabilities

However, these models are not infallible. While they excel at generating human-like text, they also have critical limitations and vulnerabilities that must be understood for safe, responsible, and effective use. We’ll cover these in more extensive detail later in this course, but here is a quick overview:

Inherent limitations

These are properties of how LLMs work, not bugs to be patched:

• Knowledge cutoff: A model’s training data stops at a fixed date. Ask about anything after it and the model will either say it doesn’t know or, worse, confidently answer from stale knowledge. This is why the model roster in this very section carries a verification date.
• Context window limits: A model can only attend to a finite number of tokens at once. Exceed it and the earliest content falls out of view – the model doesn’t warn you, it simply loses the thread.
• Non-determinism: The same prompt can produce different outputs on different runs. This complicates testing, reproducibility, and incident investigation. Note that setting temperature to 0 does not remove it: inference batching makes the arithmetic depend on what else was running concurrently, so a replay is never guaranteed to be exact.
• No self-verification: As established earlier, the model predicts likely next tokens. Nothing in that process checks whether the output is true.

Biases

Models may reflect biases present in their training data, leading to unfair or inappropriate outputs. This kind of bias – output bias – is learned and distributed across the whole parameter set. It shares a name with the bias terms that Section 4 covers as part of a network’s arithmetic, and nothing else: output bias is not located in those numbers and cannot be audited through them.

Hallucinations and erroneous outputs

Models can generate content that appears convincing but has no basis in reality or their training data, and state it in a confident manner. This can be a serious issue in high-stakes applications, such as in agentic workflows where the model is used to make decisions and take actions, beyond just providing information.

Overreliance

A limitation of the system, not the model: because output is fluent and confident, people tend to under-scrutinize it. The failure mode is a human accepting a wrong answer they would have caught from a less articulate source. Measured rather than assumed – METR’s randomized trial found experienced developers were 19% slower with AI tooling while estimating they had been 20% faster. METR labels that result historical, having measured early-2025 tooling, so read the 39-point gap between measured and felt performance as the durable finding rather than the percentages.

The security reading is the one to carry forward: automation bias does not stay constant, it accumulates with every output that turns out to be fine. A reviewer who checked an agent’s first hundred actions is markedly less likely to check the hundred-and-first, which is why a good track record is an attack precondition rather than a reassurance – OWASP’s agentic list names it directly. Section 7 returns to it once agents can act, and sets out the METR evidence in both directions.

Vulnerabilities to Threats

• Prompt Injection: Crafted input that makes the model follow the attacker’s instructions instead of yours. It has no complete fix, because it exploits an architectural property rather than a coding error: everything assembled for a request arrives as one flat token sequence with no privilege levels, so an attacker’s text and your instructions carry equal weight. Chapter 2, Section 2 is built on this.
• Jailbreaking: Prompts that bypass safety alignment to get output the model was trained to refuse. Worth keeping apart from injection by whose instructions are subverted: injection defeats your application logic, which you can remedy; jailbreaking defeats the provider’s alignment, which you did not build and cannot patch.
• Data Poisoning: Corrupting what a model learns from, so the defect is baked into the weights and reaches every future interaction. Dilution is not a defence – the research measures a near-constant number of malicious documents rather than a percentage of the corpus, which is a quantity an attacker can produce.
• Adversarial Inputs: Inputs perturbed just far enough to cross a model’s decision boundary while looking unchanged to a human. The mathematics is identical for any differentiable model, which is the consequence to carry forward: your input filter is a model too, with a decision boundary and therefore adversarial examples of its own.
• Sensitive Information Disclosure: Confidential data reaching a user through model output. The route that gets most organisations needs no attacker at all – content your own retrieval pipeline fetched, passed straight through to a reader who was never entitled to see it.

These vulnerabilities highlight the importance of understanding not just what LLMs can do, but also where they fall short – and how they can be exploited. We’ll explore these threats in detail in Chapter 2, where we focus on LLM vulnerabilities, and again in Chapter 3 as we explore ways to mitigate risks.

Understanding AI Hallucinations

Hallucinations in AI represent a complex challenge that goes beyond simple mistakes or errors. They occur when an AI system generates content that appears convincing but has no basis in reality or its training data.

Types of Hallucinations

• Content Fabrication:

  • Complete invention of facts, figures, or narratives
  • Generation of non-existent sources or references
  • Creation of false relationships between real entities

• Contextual Confusion:

  • Mixing accurate information with false details
  • Temporal inconsistencies (mixing up timelines)
  • Inappropriate transfer of attributes between subjects

• Confidence Misrepresentation:

  • Presenting speculative information as factual
  • Maintaining false certainty despite contradictory evidence
  • Generating precise but incorrect details

What impact are LLMs having on industries and modern society?

Imagine a world where your doctor consults not just their years of medical training but also an AI assistant that has analyzed millions of medical studies in seconds. Picture a classroom where every student has access to a personalized tutor who adapts to their learning pace, or a newsroom where journalists collaborate with AI to fact-check stories in real time. These scenarios are no longer distant dreams – they are already in production, powered by the rapid adoption of Large Language Models.

Transforming Industries

Across industries, LLMs are acting as catalysts for innovation. In healthcare, for instance, clinicians use them to draft discharge summaries, condense a patient’s history into a briefing before an appointment, and synthesize findings across thousands of published studies – work that is language-heavy and enormously time-consuming. But this isn’t without risks: what happens when a summary confidently states a medication the patient was never prescribed? The stakes are high, and so is the potential.

Education

Education is another domain undergoing seismic shifts. Teachers now have tools that can summarize complex topics, generate lesson plans, or even provide instant feedback on student essays. Yet, there’s an ongoing debate: Will students become overly reliant on AI, losing the ability to think critically? Educators must strike a balance between leveraging these tools and fostering independent thought.

Creative Industries

In the creative industries, LLMs are both a boon and a challenge. They can draft marketing copy, write scripts, or even compose music. However, questions about originality and intellectual property loom large. Is a song co-written by an AI truly creative? And who owns the rights to it – the user or the developer of the model?

Cybersecurity

Even cybersecurity professionals are finding new allies in LLMs. These models can analyze threat reports or triage phishing attempts at scale. But ironically, they also introduce new vulnerabilities: adversarial inputs can manipulate outputs, and attackers use models to lower the effort involved in social engineering – producing fluent, well-targeted phishing in languages they don’t speak, or iterating on malware variants faster than they could by hand. Frontier providers do apply safety measures against the most direct forms of this abuse, but open-weight models can be fine-tuned to remove them.

Shaping Society

Beyond individual industries, LLMs are reshaping societal structures in profound ways. Automation is one of the most contentious issues. As repetitive tasks are handed over to AI – whether it’s customer service chatbots or data entry systems – significant numbers of existing roles are exposed to displacement, though credible estimates of the scale vary widely and should be treated with caution. Yet history shows that technological revolutions often create new opportunities. The rise of AI has already spurred demand for roles in model development, ethical oversight, and AI governance.

Bias amplification

Bias amplification is another critical issue. LLMs trained on biased datasets risk perpetuating societal inequalities. For example, an AI used in hiring could favor certain demographics if its training data reflects historical biases. The challenge lies in designing systems that not only reflect but also improve upon human fairness.

Ethical concerns

Ethical concerns extend beyond bias. Hallucinations – a hallmark limitation of LLMs – pose risks in high-stakes applications like legal advice or medical recommendations. Imagine an AI confidently citing non-existent laws or recommending harmful treatments; the consequences could be catastrophic.

Regulation and Public Perception

Regulation is emerging as a key area of focus. Governments worldwide are grappling with how to govern these powerful tools without stifling innovation.

The European Union’s AI Act is the most developed example. It entered into force on 1 August 2024 and applies in stages rather than all at once:

The Act classifies systems by risk level – from minimal risk through high risk to outright prohibited practices – and scales obligations accordingly. The GPAI provisions are the ones that bind the model providers discussed in this chapter, covering transparency, documentation, and systemic-risk assessment.

Elsewhere the picture is uneven. China already has binding rules in force, including measures governing generative AI services and filing requirements for algorithms and deep synthesis. The United States has no comprehensive federal AI statute; it operates through executive action, sector-specific regulators, and a growing patchwork of state laws. We return to governance and compliance in Chapter 3.

The public’s perception of AI oscillates between awe and fear. On one hand, there’s excitement about its potential to solve humanity’s biggest challenges – from climate modeling to disease eradication. On the other hand, misconceptions about “sentient” machines fuel dystopian fears.

Transparency will be crucial in building trust. Users need to understand not just what these models can do but also their limitations – why they sometimes hallucinate or fail spectacularly at tasks humans find trivial.

The Road Ahead

As we stand at this crossroads, one thing is clear: LLMs are not just tools; they are mirrors reflecting our own ingenuity and flaws. Their impact will depend on how responsibly we wield them – balancing innovation with ethics, efficiency with fairness, and automation with humanity.