Application Security Roundup - March
A few tools, some thoughts on injection, some standards, and some of Adam’s appsec news.
The March appsec roundup includes few tools, some thoughts on injection, some standards, and some of my own appsec news.
Tools
- Semgrep now integrates GPT4. What could go wrong?
- Introducing Proxy Enriched Sequence Diagrams (PESD) is a new tool from Doyensec that builds sequence diagrams for your extant systems, which is pretty neat.
Injection and Parsing
Indirect Prompt Injection on Bing Chat is an interesting and powerful attack which relies on a mix of unclear boundaries and the unique programming model of LLMs, which is that ‘everything is part of the prompt.’ Bob Gourley took advantage of that to create Unrestricted Intelligence, where he submits a carefully crafted pre-prompt to ChatGPT, followed by your submission. These attacks are worked through in more depth in More than you’ve asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models. (Speaking of which, did you know you can swap ar5iv for arxiv and get readable HTML? )
All of these injection attacks can be seen as parsing attacks, where code and data intermingle. Another example of that is in The Most Dangerous Codec in the World: Finding and Exploiting Vulnerabilities in H.264 Decoder, which presents a new toolset for “analyzing, generating, and manipulating syntactically correct but semantically spec-non-compliant video files.” Video decoding has always been intensely dangerous. People mocked Microsoft for putting graphics into the NT kernel, but note footnote 2, “Some Twitter commentary about CVE-2022-22675 assumed that Apple only recently moved video parsing into the iOS kernel. Not so. In fact, the first bug we identified was present in the kernel as far back as iOS 10.” (Also, what do we expect of Twitter commentary?)
Standards
- A US Agency Rejected Face Recognition—and Landed in Big Trouble is a Wired story about a challenge of standards, which is that they may require you to do certain things which may be sub-optimal in your situation.
- The EU's new Cyber Resilience Act is about to tell us how to code by Burt Hubert and his update post.
- Waymo Makes A New Safety Case by Brad Templeton which asks some really good questions about the fuller Safety Case, which takes an interesting perspective of “the absence of unreasonable risk.”
- Whitehall wiring: The Communications-Electronics Security Group and the struggle for secure speech is a fascinating look back into the UK’s struggle to ship a secure communications system, including politics, economics, and the struggle to get funding for defense. Some really interesting details such as, even after the UK had broken Enigma, they declined to look at their own ciphers. It’s interesting how far back the bias goes: offensive success is easily seen and celebrated, defense has long played second fiddle. (Sadly, paywalled.)
Training and Adam Notes
- Today’s the last day to get the early bird discount for our May Threat Modeling Intensive!
- My fellow Star Wars geeks at Panther are giving away copies of Threats as part of both an April 11 webinar registration required, and a signing at RSA at their booth (#228 in the South Expo Hall) on Wednesday, April 26 at 11 am).
- I’m keynoting Appsec PNW, with a working title of “From Tacoma Narrows to West Seattle...Lessons from a century of PNW bridges.”
- Last, but not least, my article “Nothing is Good Enough” got a callout on the cover of the Jan/Feb 2023 IEEE S+P all about how “nothing” is often seen as “good enough” and how we should not ignore that in process design. (Paywalled, sorry)