Reflective Practice and Threat Modeling (Threat Model Thursday)
[no description provided]Lately, I've been asking what takes threat modeling from a practice to a mission. If you're reading this blog, you may have seen that some people are nearly mad about threat modeling. The ones who say "you're never done threat modeling." The ones who've made it the center of their work practice. What distinguishes those people from those who keep trying to teach developers about the difference between a hactivist and a script kiddie?
A book I've read recently, "The Reflective Practitioner: How Professionals Think In Action," gives some useful perspective. It's about how practitioners use the cases and issues before them to grapple with questions like 'is this the best way to approach this problem?' It's not an easy read by any stretch. It engages in analysis of both what makes a profession, and how several professions including architect, psychologist, and town planner engage with their work.
They may ask themselves, for example, “What features do I notice when I recognize this thing? What are the criteria by which I make this judgment? What procedures am I enacting when I perform this skill? How am I framing the problem that I am trying to solve?” Usually reflection on knowing-in-action goes together with reflection on the stuff at hand. There is some puzzling, or troubling, or interesting phenomenon with which the individual is trying to deal. As he tries to make sense of it, he also reflects on the understandings which have been implicit in his action, understandings which he surfaces, criticizes, restructures, and embodies in further action. It is this entire process of reflection-in-action which is central to the “art” by which practitioners sometimes deal well with situations of uncertainty, instability, uniqueness, and value conflict.
Those seeking to advance their practice of threat modeling would do well to pick up a copy and use it as a lens into reflecting on their practice of the arts.
After the jump, I'm going to quote more bits that struck me as I read, and offer some reflection on them.
Alfred Schultz and his intellectual descendants have analyzed the tacit, everyday know-how that we bring to social interactions such as the rituals of greeting, ending a meeting, or standing in a crowded elevator. Birdwhistell has made comparable contributions to a description of the tacit knowledge embodied in our use and recognition of movement and gesture. In these domains, too, we behave according to rules and procedures that we cannot usually describe and of which we are often unaware.
This was absolutely the situation in my own threat modeling (circa 1999). Being shocked by the cumbersome emergent processes descriptions helped me realized that the rules and procedures that were beneath our consciousness were so important. In fact it was an accumulation of rules and advice that were appropriate in some but not all situations that led me to the four-question framework.
Many practitioners, locked into a view of themselves as technical experts, find nothing in the world of practice to occasion reflection. They have become too skillful at techniques of selective inattention, junk categories, and situational control, techniques which they use to preserve the constancy of their knowledge-in-practice. For them, uncertainty is a threat; its admission is a sign of weakness. Others, more inclined toward and adept at reflection-in-action, nevertheless feel profoundly uneasy because they cannot say what they know how to do...
Yes. We need to consider ourselves as learning experts. That feeling that things are not quite right is not a sign of weakness, but of weakness leaving ones practice. At the same time, we need to practice. Real cases. Real systems. Real threat models.
When design terms are ambiguous in this way, they may create confusion, but they also call attention to multiple consequences. Terms like “stair,” “ramp,” and “wall” refer both to particular building elements and to formal functions such as “marking” and “relating in.” “Gallery” refers both to an organization of space and to a particular precedent (“the sort of thing Aalto would invent”). Aspiring members of the linguistic community of design learn to detect multiple reference, distinguish particular meanings in context, and use multiple reference as an aid to vision across design domains.
This relates closely to the question of "is this the right model?" Many models are layered, and understanding those layers, prying them apart and gluing them back together is a task along the way to expertise.
As he reflects-in-action on the situation created by his earlier moves, the designer must consider not only the present choice but the tree of further choices to which it leads, each of which has different meanings in relation to the systems of implications set up by earlier moves. Quist’s virtuosity lies in his ability to string out design webs of great complexity. But even he cannot hold in mind an indefinitely expanding web. At some point, he must move from a “what if?” to a decision which then becomes a design node with binding implications for further moves. Thus there is a continually evolving system of implications within which the designer reflects-in-action.
This touches on the virtue of the whiteboard and the "pointable model." The whiteboard holds part of the web, and as choices appear binding in unacceptable ways, we can wipe them away and explore anew
It would be a mistake to attribute to the inquirer at the beginning of such a process the articulated description which he achieves later on—to say, for example, that Quist must have known unconsciously at the beginning just how this site is screwy and just how the geometry of parallels can be successfully imposed on it.
Each real system has unique problems. The architect sees a site as 'screwy,' and must respond to its challenges. Sometimes a solution seems to appear in a flash, and sometimes that flash even fleshes out, but more frequently, the fleshing out alters our understanding as we go. When I ask what can go wrong, my understanding is guided and shaped by conversations and inquiry, and then grounded in listed threats, and guided again by a framework like STRIDE or a kill chain as a particular thread dries up. I almost never see all the problems all at once, although I frequently think I see the biggest ones quickly. Sometimes, I'm even right.
When a civil engineer worries over what road to build rather than how to build it, he comes up against the politics of land taking and the organized resistance of neighborhoods. Indeed, he comes up against the whole economic, social, and political life of the region upon which the road may be imposed. And when, having designed a road, he begins to convert his design to reality, he encounters such additional problems as the constraints on city budgets, the reactions of organized labor, and the political machinations of contractors. The engineer may deal with these messy factors by placing them beyond the boundaries of his professional life; he may try to clear a space for narrowly defined professional work, treating the rest of the situation as a necessary evil. Or he may accept the intrusions of the larger situation as a part of his legitimate professional concern, opening himself to complexity, instability, and uncertainty.
This is important on two levels. First, considering what road to build is an important investigation, and the technology sector as a whole is not doing enough of it. (This relates to Rogaway's "The Moral Character of Cryptographic Work," and is amplified greatly by the impact of social media on our social systems.) Second, threat modeling gives us a way to abstract the road we're building, and to engage in dialogue about the tradeoffs we face, and so to engage with those messy factors which surround our work.