The cost of false positives in detection (lessons from public health)
More is not always better. This is especially true for screening and detection systems.
False positives can be very costly in a sneaky way. For example, they can cause users, administrators, or managers to go around or turn off the detection/protection mechanism. Here are a few publicized examples of false positives in information security:
- invalid self-signed SSL certificate warningsin web browsers
- false positives in intrusion detection and prevention
- over-classification of data or documents
- false positives in software static code analysis (For in-depth discussion, read this empirical research on Google’s experience with open-sourceFindBugs tool)
- Windows Vista “Nagware” (a.k.a.“Security Through Endless Warning Dialogs“)
We need to be able to steer away from policies, designs,or controlswhere the detection/prevention costs are greater than the benefits. No security measurement or management program can be considered complete unless it includes assessment for the likely costs of false positives.
We can learn lessons from recent pronouncements from public health organizations: one on mammograms for breast cancer screening, and the other on pap tests for cervical cancer screening. Both are a result of statistical analysis of the total costs and total benefits of testing. Both reports recommend less frequent and/or later testing in most cases, basically because the cost of frequent testing (including false positives) exceeds the benefits in risk reduction. Here are quotes from summary articles:
On Mammograms: “While many women do not think a screening test can be harmful, medical experts say the risks are real. A test can trigger unnecessary further tests, like biopsies, that can create extreme anxiety. And mammograms can find cancers that grow so slowly that they never would be noticed in a woman’s lifetime, resulting in unnecessary treatment. Over all, the report says, the modest benefit of mammograms — reducing the breast cancer death rate by 15 percent — must be weighed against the harms. And those harms loom larger for women in their 40s, who are 60 percent more likely to experience them than women 50 and older but are less likely to have breast cancer, skewing the risk-benefit equation.” [emphasis added]
On Pap testing: “The tradition of doing a Pap test every year has not been supported by recent scientific evidence,” Alan G. Waxman, MD, of the University of New Mexico in Albuquerque, said in a statement. “A review of the evidence to date shows that screening at less frequent intervals prevents cervical cancer just as well, has decreased costs, and avoids unnecessary interventions that could be harmful.” [emphasis added]
Similar conclusions have been reached regarding other medical screening tests, including colonoscopy, PSA test (for prostate cancer), chest X-ray (lung cancer screening for smokers), and full body scan (for everything!). In nearly all of these situations, the forces that were promoting more frequent and earlier testing were ignoring or downplaying the consequences of false positives.
If only the information security community had as much well-organized data and well-controlled tests and experiments as our publichealth brethren, we would be able to make better informed decisions based on evidence and not prevalent beliefs. This is the direction we need to go.
[Update: Here’s a good article from the Wall Street Journal on the cost aspects of risk/benefit analysis in these cases. Great quote: “Americans feel that in health care, more is always better and more means better outcomes,” she said. “That’s just not true, but it’s counterintuitive to a lot of people.”]
[Update 2: Bruce Schneier has a good post on the significance of false positives in evaluating detection mechanisms. In the second half of the post, he gives a fairly clear example of how even a “high quality” detection system (= very low false-positive rate) can still yield poor results when the underlying phenomena are very rare, even if you have huge piles of data. Great line: “It’s a needle-in-a-haystack problem, and throwing more hay on the pile doesn’t make that problem any easier.”]
Here are some letters written to WSJ on this topic that gives more substance to the cost of false positives in the case of mammograms:
=============================================================
Breast Cancer: Radiologists Need to Do More Reading
Here is what has substantially changed in the clinical evidence regarding breast-cancer screening since 2002: The British randomized Age Trial limited to younger women under 50 and beginning at age 40 found a relative risk reduction of 17% and absolute risk reduction of 0.4/1000, but the results were not statistically significant (Lancet, 2006). Perhaps radiologists do not best know the medical literature, since the Lancet study has been cited six times in PubMed and 67 times in Google Scholar, but never by a radiology journal.
Furthermore, according to the 2006 Cochrane Review, about 10 women receive harmful overtreatment including mastectomies for pseudodisease found by “false true-positive” exams (not the false-positive evaluations) for every life saved. Finally, the relevant statistic with screening is not the lifetime development risk but the 5/1000 (0.5%) screen-free death risk from breast cancer for 40-year-old women over 15 years (Keen, 2009).
What is now clear is that the American College of Radiology knows that $3.3 billion is spent annually on mammography (The Wall Street Journal, Nov. 17). There is an obvious conflict of interest with screening mammography for radiologists and other doctors, including oncologists, which is a good reason for the USPSTF to be made up of independent experts using evidence-based methods clearly described in the Annals. I encourage my radiologist colleagues to read the study before condemning it, and to support informed decision-making regarding screening.
John D. Keen, M.D., M.B.A.
Brookfield, Ill.
=============================================================
Regarding the Dec. 2 letter from Dr. John Keen: I am a 60-year-old, board-certified radiologist who used to teach mammography. I got retrained at the ripe old age of 58 and changed careers again to specialize in PET/CT.
Mammography is the number one modality source of malpractice suits for radiologists. Many radiologists my age are discontinuing this public service (reading mammograms) because they are tired of dreading the lawsuit that could wipe out their retirement plans. A significant percentage of younger radiologists don’t even deal with mammography, mainly because of low reimbursement for RVU (relative value unit of work) and the malpractice exposure.
I can read a CT exam in the same time that it takes to read a four-view screening mammogram with comparisons. The CT evaluation typically pays two-and-a-half times the reimbursement received for reading a screening mammogram. I could read two MRIs in the time it takes to read one multiple-view diagnostic mammogram. The MRI pays about six times the mammogram reimbursement.
David S. Archie, M.D.
Memphis, Tenn.
=============================================================
Much of the argument against early routine breast cancer screening is related to the increased cost and morbidity of follow-up evaluations related to routine screening mammograms.
As an obstetrician-gynecologist I witness the high frequency of more in-depth mammograms, ultrasounds and biopsies that result from screening mammograms. I also witness how much needless anxiety and expense this causes my patients. I agree that the benefit of all of this follow-up testing is hard to justify; in the vast majority of cases no true abnormality is found. What is missing from the debate so far is why all the follow-up testing is being performed in the first place.
I would contend that most radiologists would agree that the majority of follow-up testing is done because of the high liability associated with reading mammograms.
The vulnerability of radiologists in reading mammograms is great; miss one breast cancer on a screening mammogram and a radiologist may be out of business. If the liability concern was eliminated from the equation in how mammograms were interpreted, there would be a dramatic decline in unnecessary follow-up testing and biopsies; this would allow mammography to resume its appropriate role as a screening test. This is another expensive example, to the detriment of patients and society, of practicing defensive medicine. This is also another egregious example of the glaring omission of malpractice reform from the current health-care legislation being considered in Washington.
Patrick J. Naples, M.D.
Medina, Ohio
=============================================================
Russell,
Care to elaborate on why you think SSL warnings for self-signed certs are actually a bad idea? Given what we’ve seen coming out as tools such as sslstrip, and the hacking of wifi hotspots, it isn’t clear to me why browsers are incorrect in not just sending a user along to a site.
SSL without cert warnings amounts to no SSL, which on public networks is a recipe for disaster.
What are yo proposing instead?
Hi Andy,
My point wasn’t that SSL warnings are bad, as such. My point is that the false positive rate for SSL warnings seems to be high, so that the average user could be led to make mistakes of comission (blocking otherwise valid sites) or omission (turning off the SSL warning and then opening them up to fraudulent sites).
I’m not an expert in this particular area so I have no recommended modifications or alternatives. But the people who *are* experts should be designing such systems (including the whole certificate process) to reduce the false positive rate.
This article plus the comments provide interesting insights into the situation: http://royal.pingdom.com/2008/08/19/new-ssl-policy-in-firefox-hurting-tens-of-thousands-of-sites/
From one comment: “the whole point here is that inexperienced users are *supposed* to be scared away be the new error pages. Think about it: how can an inexperienced user tell the difference between a site that, while legitimate, just happens to use an invalid cert, versus a site that is trying to attack them?
This policy was known long before FF3 shipped and anyone using self-signed certs should have known that the world’s second-largest browser was heading in this direction. The key is educating and preparing users by letting them know how to install the self-signed cert beforehand.”
Here’s Bruce Schneier’s video rant on this topic: http://searchsecurity.techtarget.com/video/0,297151,sid14_gci1376328,00.html , starting at 4:20
Here’s another insightful article on this topic: http://www.nytimes.com/2009/12/20/business/20view.html?_r=1
“Here is a quiz: Suppose that there is a one-in-1,000 chance that a woman in her 40s with no symptoms has breast cancer, and that 90 percent of the time a mammogram correctly classifies women as having cancer or not. If a woman in this group tests positive on her mammogram, what is the chance that she has cancer? The answer is not 90 percent. It is less than 1 percent, because of the large number of false positive results.”
You only have to look at Finland, they have the lowest rates of cervical cancer in the world and send the fewest women for colposcopy and biopsies (fewer false positives)
They don’t offer screening before age 30 and then only test 5 yearly – most women only have 5-7 tests in total and some low risk women don’t test at all.
The risk of cancer is not exaggerated and informed consent is respected. Over-screening and inappropriate screening is harmful
Even this schedule sends somewhere between 30%-55% of women at some point for colposcopy, about 95% of all referrals are unnecessary and excessive. It’s a very unreliable test for an uncommon cancer, that would usually rule it out for mass screening because you harm vast numbers of women to help a small number. (fewer than 1% of women are helped by smears…0.65%)
99.35% derive no benefit at all (incl the 0.35% who get false negatives) but 2 yearly screening sends 77% for colposcopy/biopsies.
Annual – a shocking 95%
Yet there is no informed consent in women’s cancer screening, women are not given risk information and are virtually demanded to have screening…also many women are coerced to screen with doctors refusing them birth control “until” they test…these things are unrelated; this is an unethical tactic to force women to screen.
Women under 25 – screening has NO effect on the tiny death rate, but sends very high numbers of women for treatment after getting false positives. It is unethical to screen women under 25…some say 30.
Statistics taken from Richard DeMay, American pathologist, “Should we abandon pap smear testing” available on line, Laura Koutsky’s research and Dr Angela Raffle – the latter in 2003, “1000 women need regular smears to save ONE woman from cervical cancer”….BMJ
All these ref’s also appear at Dr Joel Sherman’s medical privacy blog under women’s privacy concerns.
I meant to say, Dr Raffle’s research from 2003, found in the BMJ..
“1000 women need regular smears FOR 35 YEARS to save ONE woman from cervical cancer”.
Hardly the rampant cancer, the huge threat we’re led to believe is the case…a tiny risk which doctors think justifies massive and harmful over-treatment for false positives…and all with no informed consent and in many cases, no consent at all.
Also note Australian and UK doctors receive target payments when they screen a % of their patients, 80% or 70% or 65% and bonuses for screening unscreened or under-screened women. This is in direct conflict with the legal requirement of informed consent…if this is ethical and legal, why are these payments hidden from women?