Publish your threat model!
We think you should publish your threat model, and we’re publishing our arguments.
At ThreatModCon, I gave a talk titled “Publish Your Threat Model!” In it, I discussed work that Loren Kohnfelder and I have been doing to explore the idea, and today I want to share the slides and an essay form of the idea. We invite comments on the essay form, which is the most fleshed out.
Many people have strong emotional responses to the idea, and we can’t argue with those emotions. We’ve heard the cries du coeur of the secrecy crowd, and are open to new arguments. Today, secrecy about software design has given way to SBOMs, and .. checks notes ... the world has not ended. The world will also not end when we publish our threat models.
As I gave my talk, the nice folks at Cursor were kind enough to help illustrate why publishing your threat models can be a good idea, and it's not even an LLM problem! Karol Mazurek of Afine documents a new Threat of TCC Bypasses on macOS: “I decided to disclose a TCC bypass vulnerability in Cursor.app because, despite responsible disclosure, developers stated this issue ‘falls outside their threat model’ and have no plans to fix it.” Now, maybe that was true before they got the bug report or maybe it’s a justification... but if Cursor had published their threat model, we could assess the situation with a lot more nuance. If their threat model says “we don’t consider local sandbox escapes or local privilege escalation important,” then their customers could decide if that’s acceptable. If it didn’t, then we can ask if that was hindsight or oversight, or possibly a result of moving quickly. If it’s moving quickly, then we learn something about LLM-driven coding in one of the most popular packages, because adding a coding rule of “avoid local privilege escalation” doesn’t seem infeasible. Perhaps I’m wrong, and that’s hard? Again, we learn something from the threat model being published.
Incidentally, if you’re not paying attention to the EU’s Cyber Resilience Act, it seems to require publishing threat models (as of the effective dates). Specifically, Article 13 states:
“3. The cybersecurity risk assessment shall be documented and updated as appropriate during a support period to be determined in accordance with paragraph 8 of this Article. That cybersecurity risk assessment shall comprise at least an analysis of cybersecurity risks based on the intended purpose and reasonably foreseeable use, as well as the conditions of use, of the product with digital elements, such as the operational environment or the assets to be protected, taking into account the length of time the product is expected to be in use. The cybersecurity risk assessment shall indicate whether and, if so in what manner, the security requirements set out in Part I, point (2), of Annex I are applicable to the relevant product with digital elements and how those requirements are implemented as informed by the cybersecurity risk assessment...Further, Annex 7, Part 2(a) specifies that:
4. When placing a product with digital elements on the market, the manufacturer shall include the cybersecurity risk assessment referred to in paragraph 3 of this Article in the technical documentation required pursuant to Article 31 and Annex VII.”
“The technical documentation referred to in Article 31 shall contain at least the following information, as applicable to the relevant product with digital elements: ... 2. a description of the design... including: necessary information on the design and development of the product with digital elements, including, where applicable, drawings and schemes and a description of the system architecture explaining how software components build on or feed into each other and integrate into the overall processing...”
I am confident that someone’s lawyers will disagree with me
regarding what that means, and drive a truck
through generously interpret “as
applicable” and “necessary,” claiming there are ways to comply
that don’t involve publishing your threat model. The rest of
the world ought to get used to the idea and start publishing
their threat models.
[Update: Matin Mavaddat has a thoughtful reply, A Critical Reflection. On Linkedin, I asked about shared responsibility models, and Matin replied “The Shared Responsibility Model (SRM) is an incredibly effective way to communicate who is accountable for different aspects of cloud security. While it’s likely informed by internal threat modeling, it doesn’t expose the detailed threat enumeration or system-specific mappings that a threat model would typically include. So rather than calling it a threat model in itself, I see SRM as a boundary-defining framework—one that enables and guides threat modeling, especially for customers building on cloud services.” I want to continue the conversation and add more questions:
- Are a list of boundaries (in text or graphic form) a threat model, or a threat model diagram?
- Do they become a threat model diagram if they include some of the internal code?
- If we add a list of threats, like information disclosure, does it become a threat model document?
- If we add a list of controls, like encryption, does it become a threat model?
- Does the place those controls are or aren't implemented matter?
- https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDurability.html :)
- Let me invent the T4 service, which is just like S3, but operated by HAL (“One step ahead of IBM”) so we can discuss it freely. Let's say T4 encrypts data internally, and it doesn't matter because the threats all come through the authorized clients, so HAL doesn't talk about the encryption.
- Does the T4 TM including that internal encryption change its status as a threat model or not?