Sensitivity & Specificity — Diagnostic Metrics

Dr. R. Düsing · Osnabrück University
Running Example
TGold standard: ADHD diagnosis by clinical interview (DSM-5) XTest: score on a parent-report screening instrument cCut-off: test-score threshold for the screening decision ADHD+Affected: slightly elevated test scores ADHD−Unaffected: unremarkable test scores
At what test score should a screening be considered positive — and how many children does that miss (FN) versus how many are needlessly referred on (FP)? Distribution overlap, prevalence, and the cost of errors determine the optimal cut-off.
① Distribution Plot
② Confusion Matrix & Metrics
Test +Test −Σ
Crit. ADHD+ TPtrue positive— FNfalse negative— —
Crit. ADHD− FPfalse positive— TNtrue negative— —
Σ — — —
MetricFormulaCalculationValue
③ ROC Analysis
Optimal Cut-Off
Concepts
Sensitivity vs. Specificity — the Trade-Off
Sensitivity (SE) = TP / (TP + FN) — the proportion of affected people correctly identified as positive (few missed cases, FN). Specificity (SP) = TN / (TN + FP) — the proportion of unaffected people correctly excluded (few false alarms, FP). With overlapping distributions, both cannot be maximized at once: a lower cut-off raises SE and lowers SP — and vice versa. Which side matters more is decided by the clinical consequences.
ROC Curve & AUC
The ROC curve plots sensitivity against 1−specificity for all cut-offs. A perfect test runs through the top-left corner (0,1); pure guessing corresponds to the diagonal. The AUC summarizes discriminative power, independent of any cut-off, in a single number (0.5 = chance, 1.0 = perfect). The farther apart and narrower the distributions, the higher the AUC.
Predictive Values & Prevalence
PPV = TP / (TP + FP) and NPV = TN / (TN + FN) answer the question for the specific person: how reliable is a positive or negative finding? Unlike SE and SP, they depend strongly on prevalence: for a rare condition, PPV drops low even with a good test, because among many unaffected people even a small percentage of false alarms (FP) outnumbers the few true cases. Prevalence form: PPV = (SE·prev) / (SE·prev + (1−SP)·(1−prev)).
Likelihood Ratios
LR+ = SE/(1−SP) and LR− = SP/(1−SE) are prevalence-independent measures of discriminative power. They are deliberately defined here so that for both, higher is better: LR+ supports the diagnosis given a positive finding, LR− rules it out given a negative finding. The variant common in many sources, LR− = (1−SE)/SP, is the reciprocal — needed for converting to odds (post-odds = pre-odds · LR), but less intuitive to interpret (there, "smaller is better"). Always pay attention to which definition is being used.
Choosing the Cut-Off
There is no single correct cut-off — it follows from the goal: Youden index (max SE+SP−1, equally weighted), cost-benefit (one error type is costlier → shift the cut-off), misclassification cost (prevalence-weighted total error), or predictive (max PPV/NPV). Try out the methods in the ROC panel and adopt the cut-off.
Property vs. Utility
SE and SP are properties of the test — prevalence-independent and transferable to other populations. PPV/NPV, by contrast, are situation-dependent. In selection contexts, PPV is called the success rate — the same quantity, a different field of application. → Diagnostic Validity · → Taylor-Russell Tables
Sensitivity & Specificity — Help
Example

A parent-report screening instrument should distinguish children with ADHD from unaffected children. We first want to examine the test — the location and spread of the distributions are initially unknown and are explored via the sliders. A clinical interview based on DSM-5 serves as the gold standard. Sought: the optimal cut-off c for the test decision.

The distributions overlap — no cut-off is error-free. At c=105: children with a test score ≥ 105 are screened positive (T=1), all others negative (T=0).

Sensitivity & Specificity

Sensitivity (SE) = the proportion of truly ADHD+ children correctly identified as positive (TP rate). High SE → few missed diagnoses (FN). Lowering the cut-off raises SE, lowers specificity.

SE = TP / (TP + FN)

Specificity (SP) = the proportion of truly ADHD− children correctly excluded (TN rate). High SP → few false alarms (FP). Raising the cut-off increases SP, lowers sensitivity.

SP = TN / (TN + FP)

In the example: with overlapping distributions, a perfect separation is impossible — the trade-off is unavoidable and depends on clinical priorities.

Predictive Values & Prevalence

PPV (positive predictive value) = P(ADHD+ | test+): how likely is the diagnosis given a positive test? Strongly prevalence-dependent: for rare ADHD (5% prevalence), PPV drops sharply — many FP among few true positives.

PPV = TP / (TP + FP) = (SE · prev) / (SE · prev + (1−SP) · (1−prev))

NPV = P(ADHD− | test−): how certain is a negative finding? In the example: with high sensitivity and moderate ADHD prevalence, a negative result is very reliable.

Likelihood ratios: LR+ = SE/(1−SP) — how much more likely is a positive result given ADHD+ than given ADHD−? LR− = SP/(1−SE) — how much more likely is a negative result given ADHD− than given ADHD+? (Deliberately defined this way round so that, as with LR+, "higher is better"; the standard variant (1−SE)/SP is the reciprocal.) LRs are prevalence-independent.

ROC Curve & AUC

The ROC curve shows sensitivity against 1−specificity for all possible cut-offs. A perfect test runs through the point (0,1): SE=1, 1−SP=0. A diagonal curve corresponds to guessing.

The AUC (Area Under the Curve) summarizes overall discriminative power in a single number (0.5=chance, 1.0=perfect). AUC is independent of the chosen cut-off — the farther apart and narrower the distributions, the higher the AUC.

Optimal Cut-Off Methods

Youden index (Youden, 1950): maximizes SE + SP − 1. The default choice with no specific assumptions, treats FP and FN as equally weighted, no prevalence influence. For normally distributed groups, the optimal cut-off sits midway between the distribution means — adjustable via the sliders.

Cost-benefit: sensible when one error type is clearly costlier. In the screening example: a missed ADHD diagnosis (FN) can be costly in the long run — a lower cut-off, favoring higher SE. The factor c = cost(FP)/cost(FN) controls this directly: c<1 → FN costlier → cut-off decreases.

Misclassification cost: incorporates prevalence — population-weighted total error. For rare ADHD (5%), FP errors occur more often; the model shifts the cut-off upward (more SP).

Prevalence-adjusted (PPV/NPV): when communicability is decisive. Maximizing PPV: for an invasive follow-up examination, a positive finding should be reliable. Maximizing NPV: for a serious condition, a negative result should reliably rule it out.

References

Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1), 32–35.