Shostack + Friends Blog

 

Spatial Reasoning and Threat Modeling

Do diagrams leverage the brain in a different way? An abstract images for the header

Creating, refining, communicating, and working with models are all important parts of how I think about answering “what are we working on?” People often want to eliminate the diagramming or modeling step as “not required,” and that’s a mistake. The act of engaging with the higher order question of ‘what are we building working on’ is important, and diagramming acts as a forcing function. Committing to a specific representation opens the door to reflecting, criticism or even disagreement, sometimes in counter-productive ways, sometimes ‘merely’ letting the pursuit of perfection distract us.

So as I was reading Improving Computer Science Performance by Developing Spatial Skills, I was struck by one of the pull quotes: “Humans recruit spatial structures in the brain for pseudo-spatial representations of complex, abstract ideas.” The paper is focused on how spatial reasoning can help people learn computer science, and that spatial reasoning is unusual because it’s can be taught and improves performance on a wide range of tasks. In this blog post, I’m engaging with that really interesting idea of “recruiting spatial structures to represent complex ideas.” Wow.

Threat modeling is, in part, an exercise in associative reasoning. “This thing looks like that thing, and that thing had these problems.” Structures like STRIDE per element give us value in part by helping focus our iteration, and in part by chunking security. (I don’t think anyone’s done an experiment, but it wouldn’t surprise me if STRIDE takes up only one chunk of working memory in experts.) But everything that we’re trying to do takes space in the brain. In a recent class, someone asked me about some very specific security impact of a thing, after a minute of stammering said something like: “I have no idea. I’m tracking how students are responding to prompts, overall participation and energy, time, and I don’t worry about the security of this fake system.”

But short term memory is tiny and cramped. (It’s famously described as “seven plus or minus 2.”) If we can trick our brains into using additional memory systems beyond our normal short term and long term memory, we absolutely should. And as I think about it, one of the most famous and enduring tricks for memorization is the ‘memory palace,’ which may be using some of these same mechanisms.

This also ties into how we might use LLMs to help us threat model. On the one hand, they can act as giant contextual retrieval engines. That requires that we set the context well. It may be possible to do that without a diagram, but a picture is not only worth a thousand words, but it’s easier to review. And if it really uses a different part of the brain, then it can help us keep more information at hand, to better see how an LLM is doing.

Midjourney: : water color clean line graphic illustrates Creating, refining, models, important, think, answering, People, diagramming --v 6.0 --ar 8:3