Section 6 Quiz

--- shuffle_answers: true shuffle_questions: false --- ## A finance manager joins a video call with the CFO and two colleagues, all of whom are synthetic, and is asked to authorise an urgent confidential transfer. Which control holds even if every detector on the call reports clean? > Hint: One of these does not depend on being right about the media. - [ ] Real-time video analysis of facial landmarks and temporal coherence during the call > This is the control the premise has already defeated. Live face-swap quality has outpaced detector retraining, and real-time analysis adds latency to the very call it is protecting. Detection is a triage signal, not an authorisation decision. - [x] A callback to the requester on a number taken from the directory, not from the request > Correct! Out-of-band confirmation on a channel the requester did not supply is independent of the fake's quality. The call can be flawless and the control still holds, because it verifies the *request* through a path the attacker does not control. This is the control that would have stopped the Arup fraud, and it costs nothing to deploy. - [ ] Behaviour analytics flagging the payment as unusual for that manager's account > Analytics may flag the transaction, but it does so after authorisation. The control has to intervene at the decision point, before money moves, not produce an alert afterwards. - [ ] Email security tuned to catch the AI-generated phishing that set up the call > Worth having, and it addresses a different link in the chain. Once the target is on the call, the mail gateway has no further part to play in the decision they are about to make. ## Your vendor's deepfake detector reports 99% accuracy on an industry benchmark. What does that number actually license you to do? > Hint: What population was the accuracy measured over? - [ ] Treat a clean detector result as sufficient verification for high-value requests > This inverts the control. A benchmark score is not a statement about the next piece of media you receive, and treating "unflagged" as "authorised" removes the only defense that does not depend on the detector being right. - [ ] Retire the out-of-band callback procedure for media that passes the detector > The callback exists precisely because the detector will sometimes be wrong in the attacker's favour. Retiring it converts a layered defense into a single point of failure with a good benchmark score. - [ ] Rely on the detector for live calls and reserve procedure for recorded media > The split runs the wrong way. Pre-recorded fakes get unlimited post-processing passes to remove detection signals, and live analysis is the more expensive and less mature case. Neither justifies dropping procedure. - [x] Use it to triage which inbound media gets escalated, and nothing beyond that > Correct! A benchmark score describes the synthesis pipelines the detector was trained on. Against a generator it has not seen -- or a newer version of one it has -- reported accuracy falls by tens of percentage points, and recompression or resizing degrades it further. The attacker's choice of tool is not drawn from the benchmark set, so the score supports triage and never authorisation. ## An inbound video arrives with no C2PA Content Credentials manifest attached. What does its absence tell you? > Hint: Consider how much legitimate media carries a manifest today. - [x] Nothing useful -- most legitimate media carries no manifest either > Correct! Provenance works asymmetrically. A valid manifest from a trusted signer raises confidence and can shorten a verification path; its absence is the normal state for legacy files, screenshots, re-encodes, and anything through a platform that strips metadata. A control that treats "no credential" as "suspicious" fires on the majority of real content and gets switched off within a week. - [ ] Probably synthesised -- generators do not sign the output they produce > Several major generators do attach credentials, and stripping a manifest is trivial for anyone who wants to. Absence is not evidence of synthesis, which is exactly why it cannot be used as one. - [ ] Altered since capture, which broke the signature chain that was there > A broken or invalid manifest would suggest that. No manifest at all means no claim was ever made, which is a different and far more common condition. - [ ] Sent from a platform outside C2PA, so it should be treated as untrusted > Membership of the sender's platform is not what a manifest records, and treating every non-participating platform as untrusted would exclude most of the internet. ## An engineering team has standardised on a local model runtime on their laptops. Network traffic analysis, DNS monitoring, CASB discovery, and expense review all report a clean estate for that team. What has happened? > Hint: Where does the inference actually run? - [ ] The team is compliant, since local inference keeps data inside the organisation > Local inference does keep prompts off a third-party API, which removes one risk and adds others -- unreviewed weights, no central interaction log, and a full inference stack on an unhardened endpoint. "No network signal" is not the same finding as "no risk." - [x] Every technique listed is network-side; local inference crosses no boundary > Correct! A local runtime resolves no AI domain, opens no connection to a hosted endpoint, appears on no invoice, and needs no procurement. Chapter 2 Section 7 makes exactly this point about SLMs on endpoints -- this is shadow AI with weights. The completing technique is runtime detection on the device: the process, the listening port, the weights on disk, and accelerator load with no attributable application. - [ ] Encrypted DNS is hiding the resolutions from the monitoring pipeline > DoH and DoT do blind DNS monitoring, and that is a real gap in the discovery table. It is not this gap -- a local model performs no resolution to hide in the first place. - [ ] The discovery tooling needs a longer baseline window before it reports usage > A longer window collects more of the same signal. No amount of observation time will surface traffic that is never generated. ## An agent's design requires human approval before every action it takes. Six weeks in, approval latency has collapsed and the rejection rate is near zero across several thousand gates. What is the right correction? > Hint: The problem is the number of gates, not the reviewers. - [ ] Add a second approver to high-volume gates so that two people must agree > Doubling the requests doubles the fatigue. Two reviewers clicking through the same undifferentiated queue produce two rubber stamps rather than one considered decision. - [ ] Retrain reviewers on the importance of reading each request carefully > Approval fatigue is a predictable result of asking for attention more often than attention exists, not a discipline problem. Training against it fails the same way every six weeks. - [x] Gate only the irreversible subset, and let reversible actions run logged > Correct! A gate on everything is a gate on nothing. Gate what moves money, changes access, contacts third parties, writes to systems of record, or grants the system new capability -- and let reversible actions execute with logging. Fewer gates that are actually read is a stronger control than universal gates that are not. - [ ] Replace the human approver with a reviewing agent to eliminate the fatigue entirely > This removes the fatigue and the control together. A reviewing agent built on the same model, reading the actor's output as authoritative, is one control wearing two hats -- it cannot fail differently from the thing it reviews. ## A team consuming a managed AI API argues that most of Chapter 3 does not apply to them, so their Layer 4 obligations should be small too. Where does that reasoning break? > Hint: What asset is Layer 4 protecting? - [ ] Layer 4 obligations do shrink, but Layer 5 grows to compensate for the loss > Layer 5 does not absorb Layer 4's work; it acts in the request path of services you run. The approval gate, the verification procedure, and the device estate have no Layer 5 equivalent. - [x] Layer 4's asset is the person and the device, and neither changes with the deployment > Correct! The reasoning is sound for Layer 3, where consuming a managed API genuinely removes the GPU, the container, the sockets, and the cache. Layer 4 removes nothing: the same people on the same devices approve the same actions. The axis that varies Layer 4 ownership is the device and identity estate -- managed, BYOD, contractor, personal -- not the model deployment pattern. - [ ] Their obligations shift to the provider under the shared responsibility model > A provider can secure infrastructure it operates. It cannot gate an action inside your application, run your joiner/mover/leaver process, or train your staff to verify a request. - [ ] Layer 4 only applies to organisations that self-host and fine-tune their models > This has it backwards. The fewer components you operate, the larger the share of your remaining risk that is a person pasting something or approving something. ## Why can user behaviour analytics not detect ASI09, human-agent trust exploitation? > Hint: What actually changes during the incident? - [ ] The interaction volumes are too low for a statistical baseline to form > Volume is not the obstacle. Trust exploitation typically occurs in systems with heavy, well-established usage -- which is precisely how the track record that enables it gets built. - [x] Nothing anomalous happens -- the shift is in the reviewer's attention > Correct! The agent behaves as it always has, volumes are normal, queries are role-appropriate, and the human approves as they have a hundred times before. Chapter 2 records ASI09's runtime signal as "not technically detectable" and the glossary is blunter: no technical signal exists for a trust gradient. What analytics *can* see is the gate going quiet -- collapsing approval latency and a near-zero rejection rate. - [ ] The exploitation occurs inside the model, where endpoint analytics has no visibility > The exploitation occurs in the human, not the model. Better instrumentation of the model would not surface it either. - [ ] Attackers deliberately pace their activity to stay under the alerting thresholds > Evasion of thresholds is a real technique against volume-based detection. It is not what makes ASI09 undetectable, since here there is no anomalous activity to pace in the first place. ## A finding for an executive voice-cloning fraud is filed under `LLM07: Misinformation`. What does that mislabel cost you in practice? > Hint: A category name is a routing decision. - [ ] Nothing material, since both concern AI-generated content reaching a person > Both involve AI output reaching a human, and that similarity is what makes the mislabel easy. It is also where the resemblance ends -- the mechanisms and the owning controls have nothing in common. - [x] It routes you to grounding and output validation, which cannot affect a cloned voice > Correct! OWASP defines LLM07 as the model's *own* incorrect or misleading output, so the label sends you to review grounding, citations, and response validation. A cloned voice on a phone call is unaffected by every one of those. Deepfake impersonation has no OWASP LLM category at all; its identifier is MITRE ATLAS `AML.T0052.001`, with `AML.M0034` and `AML.M0018` as the named mitigations. When OWASP has no slot for a technique, you are holding the wrong reference work. - [ ] It overstates the severity, since LLM07 ranks higher than the ATLAS equivalent > ATLAS techniques and OWASP categories are not ranked against each other, and severity is not what the mislabel distorts. It distorts which team gets the work. - [ ] It understates the risk, because impersonation is not covered by any framework > Impersonation is covered, and specifically so -- ATLAS `AML.T0052.001` even cites the Arup case among its references. The problem is the wrong framework, not the absence of one. ## Which single fact about `ASI09` makes a Layer 4 program built entirely from detectors and dashboards a coverage gap rather than a partial success? > Hint: Look at the "completed by" column of the Layer 4 → OWASP mapping. - [ ] It is the highest-severity category in the Agentic AI Top 10 for 2026 > ASI09 is not ranked highest, and severity is not the issue. The issue is that nothing else is positioned to cover it. - [ ] It requires no attacker, so no control anywhere in the Blueprint applies to it > The no-attacker-required category is ASI10, rogue agents. ASI09 does involve an adversary exploiting accumulated trust, and controls do apply -- procedural ones. - [x] Layer 4 is its only owning layer -- what Layer 4 drops, nothing picks up > Correct! Every other category in the course names a second layer that completes the first. ASI09 does not -- and its Layer 4 controls are the procedural ones a detector-first program skips: the gate on irreversible actions, confidence calibration so the reviewer has something to weigh, and rotation or sampling so verification does not rest on one person's sustained vigilance. - [ ] Detection tools for it exist but are immature and not yet worth deploying > This suggests the gap is temporary and closes with better products. It does not: the change during exploitation is in a human's attention, which no future detector will instrument.