Section 1 Quiz

--- shuffle_answers: true shuffle_questions: false --- ## A fintech company discovers that their AI customer service chatbot has been revealing internal pricing formulas when users craft specific requests. Which OWASP LLM Top 10 (2025) category does this incident map to? > Hint: Think about what type of information is being exposed and through what mechanism. - [ ] LLM04: Data and Model Poisoning > Data and Model Poisoning targets the training pipeline. This incident involves information being disclosed through the inference interface, not a training-time compromise. - [ ] LLM06: Excessive Agency > Excessive Agency covers systems taking unauthorized actions. Here the model is disclosing information, not performing unauthorized actions. - [x] LLM07: System Prompt Leakage > Correct! This maps to LLM07: System Prompt Leakage. The chatbot is revealing hidden system-level configuration (pricing formulas embedded in its instructions). Leaked system prompts expose business logic, guardrail configurations, and sometimes API keys -- giving attackers a roadmap for further exploitation. - [ ] LLM10: Unbounded Consumption > Unbounded Consumption covers resource exhaustion attacks like prompt flooding and API cost exploitation, not information disclosure. ## An attacker publishes a corrupted LoRA fine-tuning adapter on Hugging Face that appears to improve a model's medical knowledge. When loaded, it subtly steers recommendations toward a specific pharmaceutical brand. Which stage of the AI lifecycle is being attacked? > Hint: Consider when in the AI lifecycle LoRA adapters are applied. - [ ] Inference Phase -- the attack happens when users ask questions > While the effect manifests at inference time, the compromise occurs when the adapter is loaded during the customization phase. - [x] Customization Phase -- the adapter modifies model behavior during fine-tuning > Correct! LoRA adapters are applied during the Customization Phase. This is a supply chain attack (LLM03: Supply Chain) combined with data poisoning (LLM04: Data and Model Poisoning). The attacker targets the fine-tuning stage to introduce bias that persists through deployment and inference. - [ ] Monitoring Phase -- the attack exploits feedback loops > The Monitoring Phase involves logging and feedback. The compromise here happens when the malicious adapter is loaded, not during monitoring. - [ ] Deployment Phase -- the attack targets the packaging process > Deployment Phase covers model packaging and distribution infrastructure. While LoRA adapters are distributed, they are applied during the Customization Phase. ## Which of the following is NOT one of the OWASP LLM Top 10 (2025) categories? > Hint: Review the ten official categories -- one of these options doesn't belong. - [ ] LLM01: Prompt Injection > LLM01: Prompt Injection is a real category covering both direct and indirect prompt manipulation. - [ ] LLM08: Vector and Embedding Weaknesses > LLM08: Vector and Embedding Weaknesses is a real category covering RAG and vector store vulnerabilities. - [x] LLM11: Model Hallucination Exploitation > Correct! There is no LLM11 -- the OWASP LLM Top 10 has exactly 10 categories (LLM01 through LLM10). Hallucination-related risks are covered under LLM09: Misinformation, which addresses fabricated content and overreliance on model outputs. - [ ] LLM10: Unbounded Consumption > LLM10: Unbounded Consumption is a real category covering resource exhaustion and cost exploitation attacks. ## The Microsoft LLMjacking case (2024) resulted in costs exceeding $100,000 per day for victims. What was the primary attack vector? > Hint: Think about what the attackers needed to steal in order to run up charges on someone else's account. - [ ] They poisoned the training data of Azure OpenAI models > The attackers didn't tamper with training data. They exploited the operational infrastructure. - [ ] They deployed adversarial inputs to crash the inference servers > This would be a denial-of-service attack, but the LLMjacking case focused on unauthorized usage, not crashing services. - [x] They stole API credentials from public repositories and resold access to GPT-4 and DALL-E > Correct! This maps to LLM10: Unbounded Consumption. Attackers harvested API keys from leaked credentials and public code repositories, then built proxy services reselling access. Victims discovered the attack only when unexpected charges appeared on their bills. API credential security protects not just data but budgets. - [ ] They exploited a prompt injection vulnerability to extract model weights > Model weight extraction is a different attack category. The LLMjacking case was about unauthorized resource consumption through stolen credentials. ## Why did OWASP create a separate Agentic AI Top 10 (2026) framework rather than expanding the LLM Top 10? > Hint: Consider what is fundamentally different about agentic AI systems compared to standard LLM applications. - [ ] The LLM Top 10 was too old and needed a complete replacement > The Agentic AI Top 10 is a companion to the LLM Top 10 (2025), not a replacement. Both frameworks remain current and relevant. - [ ] The Agentic AI Top 10 only applies to multi-agent systems, not single agents > The Agentic AI Top 10 covers both single-agent and multi-agent risks, including tool misuse, goal hijacking, and code execution. - [x] Agentic systems create entirely new risk categories around tool use, autonomous actions, and multi-agent coordination that go beyond model input/output vulnerabilities > Correct! A chatbot that gets prompt-injected can say something wrong. An agent that gets prompt-injected can do something wrong -- run code, send emails, modify files, or purchase services. The Agentic AI Top 10 maps risks specific to autonomous AI actions, tool exploitation, inter-agent communication, and cascading failures that have no equivalent in the LLM Top 10. - [ ] The Agentic AI Top 10 covers only the most advanced, cutting-edge threats > Both frameworks address current, real-world threats. The distinction is about the type of system (text-only LLM vs. action-taking agent), not the sophistication of the threat. ## A security team is conducting a formal threat modeling exercise for a new AI deployment. They need to map the complete attacker journey from initial reconnaissance to final impact. Which framework is most appropriate for this task? > Hint: One framework categorizes vulnerabilities, while the other maps attacker tactics and techniques across the full attack lifecycle. - [x] MITRE ATLAS, because it maps adversarial tactics and techniques across the full attack lifecycle, similar to MITRE ATT&CK > Correct! MITRE ATLAS catalogs 15 tactics and 66 techniques organized in a matrix that maps the attacker's journey from reconnaissance to impact. It answers "how does an attacker get from initial access to impact, step by step?" -- exactly what formal threat modeling requires. - [ ] OWASP LLM Top 10, because it has the most comprehensive vulnerability categories > The OWASP LLM Top 10 categorizes vulnerabilities (what can go wrong) but doesn't map the step-by-step attacker journey needed for formal threat modeling. - [ ] OWASP Agentic AI Top 10, because it covers the most dangerous modern threats > The Agentic AI Top 10 covers agentic risks specifically but doesn't provide the structured attacker-journey framework needed for formal threat modeling. - [ ] All three frameworks are interchangeable for threat modeling > Each framework serves a different purpose. MITRE ATLAS maps attacker journeys, while OWASP frameworks categorize vulnerability types. They complement each other but are not interchangeable. ## An organization uses OWASP LLM Top 10 category LLM06: Excessive Agency as the starting point when assessing their agentic AI system. What is the relationship between LLM06 and the OWASP Agentic AI Top 10? > Hint: Consider how LLM06's scope expands when systems have agency. - [ ] LLM06 replaces all 10 ASI categories -- they cover the same risks > LLM06 is a single category focused on preventing over-permission. The Agentic AI Top 10 expands this into 10 detailed attack categories. - [ ] LLM06 and the ASI categories are completely unrelated frameworks > They are explicitly connected. LLM06 is described as the foundation that the Agentic AI Top 10 expands upon. - [x] LLM06 is the foundation that the entire Agentic AI Top 10 expands upon -- it says "don't give excess agency," while the ASI categories map what happens when systems have that agency > Correct! LLM06: Excessive Agency focuses on preventing over-permission ("don't give the model a tool it doesn't need"). The OWASP Agentic AI Top 10 (ASI01-ASI10) maps what happens when systems do have agency -- the 10 specific attack categories that emerge, from goal hijacking (ASI01) to rogue agents (ASI10). - [ ] LLM06 only applies to non-agentic systems and should be ignored for agent assessments > LLM06 is the entry point for agentic assessments. Start with "does this system have excessive agency?" then use ASI categories to map specific risks. ## A cybercriminal group targets a public-facing AI chatbot using well-known jailbreak templates found on social media. According to the threat actor classification, which category best describes this group? > Hint: Consider the sophistication level of using pre-made jailbreak templates. - [x] Script Kiddies -- they use published jailbreaks and prompt injection templates with low capability > Correct! Script kiddies use readily available, pre-published attack tools and templates. The OWASP threat actor classification places them at low capability, motivated by curiosity and bragging rights. The democratization of AI attack tools means even low-skill attackers can threaten AI systems -- jailbreak prompts spread on social media within hours and require no coding ability to use. - [ ] Cybercriminals -- they are attacking for financial gain > While cybercriminals may use jailbreaks, the scenario describes using publicly available templates on a public chatbot -- characteristic of script kiddies. Cybercriminals typically have medium-to-high capability and target API credits, compute resources, and customer data. - [ ] Nation-States -- they target public-facing systems as part of reconnaissance > Nation-state actors operate at very high capability with supply chain attacks and zero-days. Using public jailbreak templates on a chatbot is far below nation-state sophistication. - [ ] Security Researchers -- they are testing the chatbot's defenses > Security researchers typically practice responsible disclosure and often discover novel techniques. Using well-known templates from social media is characteristic of script kiddies, not researchers.