A test that is right 99% of the time sounds like it settles the question. For a rare disease, most of its positive results are still false alarms. This page builds the reason three ways: by counting people, by measuring areas, and by multiplying odds. Then it shows the trade-off every test makes between catching the sick and alarming the healthy.The disease and tests here are hypothetical. This page explains math and is not medical advice.
Picture 10,000 people. Prevalence is the share who actually have the disease. Sensitivity is the share of sick people the test flags as positive. Specificity is the share of healthy people it correctly clears as negative. A sick person who tests positive is a true positive; a sick person who tests negative is a false negative. A healthy person who tests positive is a false positive; one who tests negative is a true negative. The question a patient cares about is the positive predictive value (PPV): if my test is positive, how likely is it that I'm sick?
Here p is the prevalence. Sensitivity and specificity describe the test. PPV and NPV describe what a result means, and they depend on how common the disease is in the people being tested.
# expected counts in a population of n people def screen(n, p, sens, spec): sick = n * p healthy = n - sick tp = sick * sens # sick, flagged fn = sick - tp # sick, missed fp = healthy * (1 - spec) # healthy, flagged tn = healthy - fp # healthy, cleared return tp / (tp + fp), tn / (tn + fn) ppv, npv = screen(10_000, 1/1000, 0.990, 0.990) # ppv = 0.0902, npv = 0.999990
The healthy group is huge, so even a 1% error rate inside it produces more false alarms than there are sick people in total. "99% accurate" describes the test's errors within each group. It says nothing about how big the groups are. That missing piece is the base rate, and ignoring it is called the base-rate fallacy. The random-draw mode shows the same pattern with chance variation, the kind of sampling noise covered in D00 · Polls & margins of error.
Draw everyone as a square of area 1. Split it into a thin column for the sick (width = prevalence) and a wide column for the healthy. Inside each column, shade the part that tests positive. A positive result means you are somewhere in the shaded area, and your chance of being sick is the sick share of that shaded area. That share is Bayes' theorem. Your prior is your chance before the test; your posterior is your chance after it. A second test starts from the first test's posterior.
How to read this: bright areas are the people who got this test result, and dimmed areas are the results you didn't get. The amber arrow points to a bar that zooms in on just the bright areas; its red share is your chance of being sick (the small "A ÷ (A+B)" numbers work it out).
P(D|+) reads "the probability of disease given a positive result." P(+|D) is the sensitivity. Odds are sick-to-healthy: a probability of 10% is odds of 1 : 9. The likelihood ratio says how many times more likely this result is in a sick person than in a healthy one. Multiplying odds and dividing areas give the same answer; the Node check on this page confirms it to 12 decimal places.
def update(p, sens, spec, positive=True):
odds = p / (1 - p)
lr = sens / (1 - spec) if positive else (1 - sens) / spec
odds *= lr
return odds / (1 + odds)
p = 1/1000
p = update(p, 0.990, 0.990, True) # 0.0902
p = update(p, 0.990, 0.990, True) # 0.9075
Multiplying a second likelihood ratio assumes the two results are independent given disease status: among the sick, test 2 is as likely to be positive whether or not test 1 was, and the same among the healthy. Rerunning the same test on the same sample usually breaks this, because whatever fooled it once (a cross-reacting substance, a contaminated sample) can fool it again. A different kind of test is closer to independent. With correlated errors the true posterior after two positives is lower than this page shows. The same counting logic runs through 006 · Probability.
Many tests measure a number, a score, and call anything above a cutoff, the threshold, positive. Sick people tend to score higher, but the two groups overlap. Raise the threshold and you clear more healthy people (higher specificity) while missing more sick ones (lower sensitivity). The ROC curve traces every threshold at once: false-positive rate across, sensitivity up. The area under it (AUC) is the chance that a randomly chosen sick person scores higher than a randomly chosen healthy one. Toy model: both groups' scores follow bell curves with the same spread.
Healthy scores follow a standard bell curve centered at 0, sick scores the same curve shifted by d. Φ is the bell curve's cumulative area. The page integrates the curve with the trapezoid rule, compares it with Φ(d/√2), and also ranks 1,000 seeded synthetic sick scores against 1,000 healthy ones to get the sample AUC.
from statistics import NormalDist import random d, t = 2.00, 1.50 # separation, threshold healthy, sick = NormalDist(0, 1), NormalDist(d, 1) sens = 1 - sick.cdf(t) spec = healthy.cdf(t) auc = NormalDist().cdf(d / 2**0.5) def ppv(p): return sens*p / (sens*p + (1 - spec)*(1 - p)) print(sens, spec, auc) # 0.6915 0.9332 0.9214 print(ppv(1/1000), ppv(1/5)) # 0.0103 0.7213 # AUC is the chance a random sick score beats a random healthy one rng = random.Random(1) n = 200_000 wins = sum(rng.gauss(d, 1) > rng.gauss(0, 1) for _ in range(n)) print(wins / n) # close to auc
The test is identical in both columns. Only the prior changes. Among people who walk in with symptoms the disease is far more common, so the same positive result is far more likely to be real. That is why mass screening for a rare condition usually confirms positives with a second, different test, as in Lab 2. AUC measures how well scores separate the groups and ignores prevalence entirely, so a high AUC alone can't tell you what a positive means. Classifiers in C05 · Machine learning are scored with the same curve.