Shostack + Friends Blog

 

Valorizing Rule-breaking

An AI produced image four clowns building a submarine in a large, bright, well-lit hanger. The submarine dominates the scene and is much larger than the clowns. It has badly-fitted parts. The clowns are wearing baggy polka-dotted clown suits, bright clown makeup and wigs.
[Stockton Rush] understood that his approach “flies in the face of the submersible orthodoxy, but that is the nature of innovation,” he wrote. “We have heard the baseless cries of ‘you are going to kill someone’ way too often. I take this as a serious personal insult.” (An Accident Waiting to Happen, The New Yorker)

I’ve been reading about Oceangate with increasing horror and anger, and the confusion over the relationship between engineering and — frankly — bullshit. I don’t think anyone has ever told me ‘you are going to kill someone.’ Certainly not ‘way too often.’ If you’re hearing that, the right response is to slow down, not take it as an insult.

Engineering is about making tradeoffs in complex situations. Sometimes, as you do so, you have to violate rules. But the causal relationship is rulebreaking sometimes allows innovation, not vice versa. What Stocktom Rush says here is not “just” upsetting, it literally killed people:

“If you’re not breaking things, you’re not innovating,” Rush said, at the GeekWire Summit last fall. “If you’re operating within a known environment, as most submersible manufacturers do—they don’t break things. To me, the more stuff you’ve broken, the more innovative you’ve been.”

This is so very dangerously wrong that I want to write something equally outrageous, like: the most useful thing Mr. Rush ever did was to die spectacularly so we can judge his perspective.

“To me, the more stuff you’ve broken, the more innovative you’ve been.” If it’s not unclear to you why that’s wrong, pick up a hammer and hit things until you’ve gotten clear about the amount of innovation you’re producing.

The trouble is the confusion over what comes first. The trouble is exacerbated by valorizing rule-breaking. That is, we ascribe value to the rule breaking. For example, the New York Times story, The Maverick Design Choices That May Have Doomed Titan actually does a good job of explaining some of the engineering tradeoffs they wanted to make. There were admirable goals being pursued, such as more people in the sub. But the headline? “Maverick?”

These were not admirable choices. Using expired carbon fiber to save money was, simply, dangerous. Manufacturers set expiration dates for a reason, and it may have been relevant. Of course, carbon fiber works better in tension than compression, so it may have not mattered. And when that’s the direction of your conversation, it’s time to quit.

The video The Questionable Engineering of Oceangate is by someone who wrote his Master’s thesis on predicting failures in composites. The decisions being made by Oceansgate were foolhardy. That’s why Rush’s peers sent repeated warnings. It’s why people quit and then blew the whistle.

As the New Yorker article points out, the most innovative thing that Oceansgate did was to carefully structure their operations to minimize liability: “It is truly hard to imagine the discernment it took for Stockton to string together each of the links in the chain,” Patrick noted. “ ‘How do I avoid liability in Washington State? How do I avoid liability with an offshore corporate structure? How do I keep the U.S. Coast Guard from breathing down my neck?’”

Again, sometimes rule-breaking is required to achieve new results. But good engineering, by definition, includes an awareness that you’re venturing into the unknown, and caution and respect for the environment as you do so.

One of the reasons that I teach people to ask “what can go wrong” is that it’s expansive. (Expansive, not expensive! It can be very inexpensive.) It creates space to bring up concerns. And if what you’re building is a website, maybe moving fast and breaking things is ok. I don’t have a one size fits all approach to “how to threat model” because you can, and should, adjust your engineering effort and due care to the scenario in which you expect to operate.

Anything less is worth a response, from a raised eyebrow to outraged scorn.

Image: Midjourney four clowns building a submarine in a large, bright, well-lit hanger. The submarine dominates the scene and is much larger than the clowns. It has badly-fitted parts. The clowns are wearing baggy polka-dotted clown suits, bright clown makup and wigs. ultrarealistic, photorealistic, documentary. --ar 8:3