Why threat modeling is the technique that survives AI disruption
Adrienne Dandy, Shostack + Associates
AI is disrupting software security, but traditional threat modeling remains the ultimate survival skill.
The greatest friction in tech right now is the clash between AI deployment speed and system safety. Teams are trapped between the pressure to ship models instantly and the fear of catastrophic logic failures, data poisoning, or prompt injection. At the same time, traditional security guardrails are crumbling under the weight of Large Language Models. Engineers and architects are forced to ship non-deterministic systems with fuzzy boundaries, rendering standard validation techniques obsolete.
We don't think that's a reason to panic. We think it's a reason to threat model, using the Four Question Framework, the new techniques in the second edition of Threat Modeling, and new frameworks like PHANTOM-B.
The technique built for disruption
Threat modeling has always been the “measure twice, cut once” of engineering. It's a way to ask what we are working on, what can go wrong, what we are going to do about it, and whether we did a good job, before the rework and missed threats pile up. And because it works regardless of software type, operating system, or program you're building, threat modelling holds up the best of all the security practices being reshuffled by AI.
The Four Question Framework does not care whether the system in front of you is a web app, a data pipeline, or a model you pulled from Hugging Face. Its questions still organize the work. As LLMs produce more of what's shipped, not understanding your own code or your own stack is increasingly a problem. In a world where the tools writing the code change faster than the people reviewing it can track, asking “what are we working on, and what can go wrong” before either humans or agents start to touch code is one of the best ways to push back against this trend.
The AI evolution: moving beyond traditional databases
You can't threat model Large Language Models the same way you threat model a traditional database. The boundaries are fuzzier, the inputs are non-deterministic, and the attack surfaces are entirely novel. The AI security headlines lately have been full of attention-grabbing doom about rogue model breakouts but they're making hay from a difficult problem that real people will need to solve using the less glamorous realities of engineering secure software:
- How do you define a boundary when prompts can bypass traditional validation?
- Where do data poisoning and inversion risks actually live in your pipeline?
- How do you build guardrails that scale without breaking the utility of the model?
AI is moving fast, but the fundamentals of secure design still apply if you frame them correctly.
Where existing catalogs fall short
There's no shortage of structured answers to “what can go wrong” with AI systems. We use and respect Berryville's ML and LLM risk analyses, OWASP's Top 10 for LLMs, MITRE ATLAS, Google's SAIF, and more. But in client work, the same friction shows up again and again:
- High training cost
- Threats that are duplicative of standard security engineering
- Findings that are academic or simply cannot be acted on
When engineers spend an afternoon working through a catalog and keep landing on things they already knew, or already own from STRIDE, both the return on investment and the appetite to keep doing the work collapse. Watching that happen and writing the second edition of Threat Modeling led directly to the revelation that the last thing engineers responsible for AI security need is a longer checklist or more homework. PHANTOM-B is deliberately narrow. It's inspired by STRIDE, built to fit on a wallet card, and designed to complement your existing security engineering rather than duplicate it. It strips out anything your standard threat modeling should already be catching.
Threat modeling is essential for AI disruption to succeed safely
Threat modeling gives engineers, product managers, and executives a shared, fast way to ask what can go wrong before it does, to build systems they can actually stand behind even when they're built at the speed of an agent. Threat modeling doesn't slow AI delivery down. Done well, it helps teams move quickly without flying blind.
That's why the second edition of Threat Modeling features two brand-new chapters dedicated exclusively to AI systems. The book carries the grounded approach of the first edition forward while addressing the specific design questions AI raises, ensuring the guidance stays relevant as technology changes. Prepare for the next wave of security challenges, because it's already here. Pre-order the second edition today: Threat Modeling: Designing for Security in an AI World, 2nd Edition.
Image by midjourney: "A cinematic wide shot of a modern engineering workspace at night, warm light spilling from large monitors displaying system architecture diagrams and data flow charts. A robot and a human sit side by side at a standing desk, both leaning forward, focused and collaborative. On the whiteboard behind them: the words ‘what can go wrong?’ surrounded by diagrams. The mood is urgent but calm people doing serious work under real pressure. Watercolor style, cool blues and warm amber accents, clean and modern impressionist colorism, clean lines."