Appsec Roundup - April 2025
Threat modeling. So much threat modeling, and so much more, including foreshadowing of new rules from FDA.
Threat Modeling
- Threat Modeling Connect has new in person groups.
- There’s a new human harms focused threat modeling approach, covered in an academic paper, Threat Me Right: A Human HARMS Threat Model for Technical Systems.
- Linwood Jones and Pawan Suresh blogged about Scaling Your Threat Modeling Program using GenAI at Adobe. I’d appreciate hearing about ‘what could go wrong’ and ‘what they did about those things.’ (From September, but I’d missed it.)
- In January, CISA (along with DARPA, NSA and DoD) released Closing the Software Understanding Gap. “Understanding” is interesting as a framing, but I think some of what they want is ‘understand the software’ and another part is ‘control what the software can do,’ and they might do well by reducing that ambiguity.
- The UK’s Ministry of Defense released a Secure by Design Problem Book (The announcement has context.)
- Matt Blaze testified about CALEA and Salt Typhoon. He says “while the legally-mandated CALEA capability requirements have changed little over the last three decades, the infrastructure that must implement and protect it has changed radically.” This is useful as a lens into “keeping threat models up to date.” The changes happened over decades. Most of our processes aren’t designed to work on those sorts of scales: we rely on people and their memories.
Appsec
- Lauren Zabierek and Bob Lord have both left CISA. I appreciate all the hard work they did, and hope that the incredible international team they built continues the important work. (Posts on Linkedin from Lauren, Bob.)
- SOSecure: Safer Code Generation with RAG and StackOverflow Discussions is an academic paper that builds a security knowledgebase from Stack Overflow, and delivers fix rates from 71% to 96%, depending on the evaluation.
LLM Security
- Defeating Prompt Injections by Design is a fascinating paper by a team from Google and ETH Zurich. Simon Willison has a good discussion. I agree with Simon, this is a very important development. I have lots of questions about completeness of coverage, what security properties we can expect, and impact on LLM quality for metrics other than security, but none of those take away from the fact that this is the first time someone has published a principled way to address code/data confusion in LLMs.
Regulation
- According to Estee Orani, FDA has made a major announcement that “Quality System Inspection Technique (QSIT) - that trusted roadmap for FDA inspections since 1999 - will be retired completely. No "QSIT 2.0" is coming. Instead, inspection processes aligned with ISO 13485:2016” will be rolled out. (FDA often previews plans like this before formal announcements, I don’t have a perspective on how official this might be, and there’s no skepticism in thread.)
Shostack + Associates updates
- Adam will be co-presenting with Tanya Janca at RSA: Red Teaming AI: 50 Years of Failure, But This Time, For Sure! - [IAIS-R03].
- Adam and Erik will be training at OWASP Global Appsec Barcelona (May 27-28), and seats are roughly half-gone.
- Adam will be giving the opening talk at ThreatModCon Barcelona, right after OWASP Global Appsec.
- Adam will be training at Blackhat USA, Aug 2-3 and 4-5.
Image by Midjourney: “a photograph of a robot, sitting in a library, working on a jigsaw puzzle”