Survivorship Bias — where to put the armor?

Dr. R. Düsing · Osnabrück University
Survivorship Bias — Help
The story (Abraham Wald, 1943)

In World War II, the US Air Force examined returning bombers and mapped their bullet holes. The obvious conclusion: armor where most of the holes are (wings, fuselage). Statistician Abraham Wald disagreed: armor where the returning planes have almost no holes — cockpit, engines, fuel tanks. Because planes hit there never came back. They're missing from the dataset.

Does / does NOT

Does: shows how a sample made entirely of "survivors" systematically deceives — the crucial information lies in the missing cases.
Does NOT: the returning planes' holes aren't "wrong" — they're just half the truth. It's a selection on the outcome (survival), not measurement error.

The mechanism

In truth, all zones get hit about equally often. But a hit in a critical zone causes a crash → that plane doesn't return → in that zone, the returning planes show almost no holes. So what you observe isn't "where planes get hit," but "where you can get hit and still make it home."

How to use it

1. Look at the returning planes. 2. Click the zones in the image that you would armor. 3. Click ▸ Reveal — the shot-down planes (the missing data) appear, and your choice is evaluated. Lethality controls how reliably a critical hit causes a crash.

Everyday life

"Successful dropouts," "this fund has beaten the market for 10 years" (you don't see the funds that went bust), "old buildings/bands/products were built better" — wherever only the survivors make it into the sample, survivorship bias threatens. Related to selection and collider bias.

References

Wald, A. (1943). A method of estimating plane vulnerability based on damage of survivors. Statistical Research Group, Columbia University (CRC 432). · Mangel, M. & Samaniego, F. J. (1984). Abraham Wald's work on aircraft survivability. JASA, 79(386), 259–267.

📋 1943 — Where to Put the Armor?
The returning bombers show their bullet holes. Armor costs weight — you can only protect a few zones. Question: where do you armor? Click the zones in the image and decide — then "▸ Reveal."
① The Bomber Fleet
Returning planes only · click zones to armor
Click cockpit, engines, fuel tanks, wings, or fuselage/tail — they light up green when you armor them.
Choose your armor zones and click "▸ Reveal."
Concepts
What is survivorship bias?
A bias that arises when only the survivors (successes, returners) make it into the sample. The losers are missing — and with them, often the crucial information.
Wald's insight
Armor where the returning planes have no holes. Those zones aren't safe — quite the opposite: hits there were fatal, which is why they're missing from the survivors. Absence of damage = location of fatal damage.
Selection on the outcome
The sample is filtered by the outcome (survival). That's the same pattern as collider/selection bias: conditioning on "returned" biases the relationship between hit location and frequency.
In everyday life
Successful dropouts, "this fund beats the market," "things used to be built to last" — you don't see the failures. Always ask: who's missing from the data? → Berkson's Paradox → Range Restriction