The Costs of Security and Algorithms
I was struck by this quote in the Economist special report on international banking:
There were navigational aids to help investors but they often gave false comfort. FICO scores, the most widely used credit score in America, were designed to assess the creditworthiness of individual borrowers, not the quality of pools of mortgages. “’Know your customer’ is a staple of banking that has largely been forgotten because of the disaggregation of the supply chain,” says Mark Greene, the chief executive of Fair Isaac, the company behind FICO scores. (“Ruptured credit)
“Know your customer” actually hasn’t been forgotten, it’s been co-opted. It’s been co-opted by the “AML” (Anti-Money Laundering) crowd. (The Google search is also fascinating. Look at all those ads!) But “know your customer” has been co-opted by the surveillance state. The people who want to know where your money is going in case they need to investigate you.
Bruce Schneier has a 5 step process for evaluating security:
- What problem does it solve?
- How well does it solve the problem?
- What new problems does it add?
- What are the economic and social costs?
- Given the above, is it worth the costs?
To be clear, the whole idea of AML doesn’t pass this test. But let’s set that aside, and test the re-definition of knowing your customer. We can then look at step 2 and 3, and ask “is re-defining a known element of good advice worthwhile?” I don’t think it is. I think it’s an example of how we let process and algorithms replace clear thinking.
It used to be that part of getting a mortgage was talking to a banker. You talked to an officer of the bank who was going to be collecting money from you for twenty years. And he made a call. That’s been replaced by the FICO algorithms and checking your ID. There’s now a process and an audit trail. And there’s no common sense. There’s no senior person who can see trends. To be fair, with common sense, it’s become harder to impose racist lending standards. That senior person can’t imagine trends.
Back to the topic at hand, we’ve moved from “know your customer” as sage advice to trite bits of checklist faux diligence. We’ve lost something important.
Really, what we’ve done is substituted a knowing a person with a knowing their data shadow. That’s not the only problem, but it’s one of a set of synergistic changes that will cost us hundreds of billions to clean up.
(Data shadows is a great term, defined by Alan Westin. Bruce Schneier used it recently in his excellent essay “Our Data, Ourselves,” which I hope to shadow shortly.)
Image: “Sinister,” by Adactio.
Actually it’s still “know the customer” and “an officer makes the decision” but on the other end it’s software not a human 🙂
Great post! I think Greene is fundamentally right, the key change is securitisation: it caused the end of banking as we know it, because it moved banks into risk-controlled trading space, and out of that very special “risky loan to person” space which required the special subsidies that we see now bringing turmoil to wall street. As Greene puts it, the supply chain imposed by securitisation not only broke the KYC, it took away the banker’s needs for it.
Only later on did it got co-opted by the AML people. It is good to see analysis of the failure and cost of AML and the related war on drugs. It may be too late, many economies, especially latin american countries have suffered tremendously under it, and that damage will never be repaired.