Effect Sizes — understanding & interpreting

Dr. R. Düsing · University of Osnabrück

Help — Effect Sizes

What does this tool show?

A guided introduction to effect sizes for mean comparisons — using a running example whose parameters can be changed via sliders. For exact calculations with your own data (CSV upload, confidence intervals via Bonett/Rosenthal/NCT, all corrections) see the Effect Size Calculator.

The running example

A psychologist is investigating whether an 8-week mindfulness training improves well-being. Between: training group vs. waitlist control group, n people each. Within: the same people before (pre) and after (post) the training, with pre-post correlation r. All means, spreads, n (and r) are freely adjustable — the numbers in the example are a plausible starting point, not a fixed given.

What does the plot show?

The two curves are the assumed population distributions of the two groups on the actual measurement scale (x-axis) — with their respective set means and spreads. A narrower, taller curve has a smaller spread; that's not a distortion but correct: the area under each curve is always the same size (= 100% of that group). The gray shaded area is the region where the two distributions overlap — the more overlap, the harder it is to tell which group a person belongs to from a single value alone. The arrow shows the raw mean difference on the original scale; the number d next to it is the same difference, standardized by a spread (see below) — hence unitless and comparable across studies.

Cohen's d in detail

d = (M₁ − M₂) / SD. The sign shows the direction (here: positive = increase), the magnitude shows the size. Cohen's (1988) informal convention: |d| ≈ 0.2 small, 0.5 medium, 0.8 large — meant as a rough orientation, not a fixed threshold; practical significance always depends on the field and the research question.

Why several standardizations for repeated measures?

For independent groups there's an obvious denominator: the pooled spread of both groups. For repeated measures, however, it's unclear whose spread should be standardized against. Three conventions have become established:

dz — difference-score SD as the denominator; directly usable for t-test power and Wuensch's CLES.
drm — dz · √(2(1−r)) (Becker 1988): equals dav under variance homogeneity, equals dz at r = .5.
dav — average SD of pre and post as the denominator (Lakens 2013); no r in the denominator.

The three values diverge more the higher the pre-post correlation r is — you can observe this directly on the slider in within mode.

d_av and comparability with between designs

dav is the most comparable to between designs. Since the pre-post correlation doesn't enter the denominator, dav is — ceteris paribus — identical to Cohen's d from an independent-groups comparison. This allows a direct comparison across designs: a within-dav of 0.5 means the same as a between-ds of 0.5. dz and drm, by contrast, depend on r and are design-specific — that's why this tutorial uses dav as the effect size in the plot and the conversion table.

Corrections for small samples

Cohen's d is an upwardly biased estimator of the true population effect size δ — the bias depends on the degrees of freedom, not on the true effect. Three common correction factors turn d into a "Hedges' g":

j (Hedges 1981) — j = 1 − 3/(4df−1); the most common.
nag (Nakagawa & Cuthill 2007) — its own df count, recommended specifically for repeated measures.
Bon (Bonett 2015) — Bon = √((n−2)/(n−1)); based on variance unbiasedness rather than the expected value of d, tends to be more conservative.

At n ≥ 40 the three barely differ; at very small n (e.g. < 15) they can diverge noticeably — try it with the n slider.

Cohen's U₃ and CLES

Both translate d into a probability and are often easier to communicate than d itself.

U₃ = Φ(d) — the proportion of the reference group (M₁) that the other group's mean (M₂) exceeds (at d = 0.5, that's 69.1%).
CLES = Φ(d/√2) — the probability that a randomly drawn person from group 2 has a higher value than a randomly drawn person from group 1.

Further conversions

r = d/√(d²+4) — point-biserial correlation between group membership and the measured value.
r² or η² — proportion of variance explained by the grouping.
OR = exp(d·π/√3) — odds ratio under a normal-distribution assumption, common in medicine and epidemiology.

Sign: r, OR, U₃, and CLES all follow the same sign as d. If d is negative (M₂ < M₁, here: a decrease), U₃ and CLES fall below 50 %, OR falls below 1, and r becomes negative — that's not an error, it consistently indicates the same direction as d. Only r² and η² are sign-independent, because they express explained variance (squared quantities). Whether to report effect sizes with sign or take the magnitude is itself a convention: the sign should be kept when the direction is substantively informative (e.g. increase vs. decrease) — only when comparing purely against the small/medium/large thresholds (which are by definition magnitudes) does one look at the magnitude.

For your own study

This tutorial works with a fixed (but freely adjustable) example and shows no confidence intervals. For exact calculations with your own data — CSV upload, confidence intervals via Bonett, Rosenthal, and the exact noncentral t-distribution — see the Effect Size Calculator.

What is an effect size? A p-value only says whether a difference is distinguishable from zero — nothing about its practical significance. An effect size expresses how large a difference is, independent of the scale used and the sample size. The most common effect size for mean comparisons is Cohen's d: the difference between two means, standardized by a spread. d = 0.5 means: the two distributions are, on average, half a standard deviation apart — whether that's about test scores or height. This makes effects comparable across studies and measures.
Visualization: how much do the groups differ?
x-axis: well-being score (fictional scale, as in the example) · curves = assumed population distributions of both groups
Point estimate — Cohen's d (n−1 pooled SD)
What do these numbers mean?
①  Uncorrected vs. corrected
The "uncorr." column is the raw Cohen's d — a slightly upward-biased estimator, especially at small n. The three correction columns (g·j, g·nag, g·Bon) pull the value a bit closer to zero. At n = 40 the difference is small; at n < 15 it can become relevant — try it with the n slider.
②  Translating effect size into other measures
Cohen's d can be converted into other common measures — depending on what's customary in your field or communicates better. r and η² are familiar from variance explained, OR from the logistic regression/epidemiology world, U₃ and CLES are probability-based measures ("how often does a person from group 1 exceed one from group 2?") and often easier to convey than d itself.
③  Conventions are not a law of nature
The "convention" column in the table above (0.2 / 0.5 / 0.8 for small/medium/large) comes from Cohen (1988) — a rough rule of thumb, averaged across very different behavioral and social sciences, intended for situations where no other guidance was available at all. It is not a fixed threshold and not an empirically derived boundary. The same value d = 0.3 can be negligible in one context (e.g. an inconsequential personality trait) and enormously significant in another (e.g. a mortality reduction in a drug trial, or a small effect scaled across millions of users). Whether an effect is practically meaningful cannot be read off from statistics alone — it's always a substantive judgment: depending on the field, the costs and risks of an intervention, the comparison with already-known effects in that exact research area, and the practical consequences for the people affected.
Concepts & limitations
Why is Cohen's d biased?
The raw d systematically overestimates the true population effect size δ — more strongly at small n. The bias depends on the degrees of freedom, not on the size of the true effect. That's why there are several correction formulas instead of just one: they make slightly different assumptions about the sampling distribution.
Hedges' j correction
j = 1 − 3/(4df−1) (Hedges 1981). The most common correction, asymptotically exact and set as the default in most statistics software (e.g. the R package effsize). The result is called Hedges' g.
Nakagawa & Cuthill correction
Its own df count, derived specifically for repeated-measures designs (Nakagawa & Cuthill 2007). Accounts for the fact that within designs effectively work with pairs, not independent observations.
Bonett correction
Bon = √((n−2)/(n−1)) (Bonett 2015). Not based on the expected value of d, but on variance unbiasedness — a different statistical target. Tends to be somewhat more conservative than j or nag. Typically used for within designs, where it's offered alongside j and nag as a third correction option (see tables above).
Cohen's U₃
U₃ = Φ(d): the proportion of the reference group that the other group's mean exceeds. At d = 0, U₃ is 50%; at d = 0.8 (large effect), around 78.8%. Easy to convey as "what percentage of the control group falls below the training group's mean?"
CLES — Common Language Effect Size
CLES = Φ(d/√2) (McGraw & Wong 1992): the probability that a randomly drawn person from one group has a higher value than a randomly drawn person from the other. Unlike U₃, CLES compares two individual people rather than one person against an entire distribution.
r, r² and η²
d can be expressed as the point-biserial correlation r between group membership and the measured value. r² (or η² for more than two groups) is the proportion of variance explained by the grouping — the same concept as in linear regression, just with a binary rather than continuous predictor.
Odds ratio
OR = exp(d·π/√3) approximates the conversion between Cohen's d and the odds ratio, assuming an underlying normal distribution was dichotomized logistically. Common in medicine and epidemiology, where results are often reported as odds ratios anyway.
Limits of standardized effect sizes
d is only comparable when the population spreads are similar. With highly selective samples (Range Restriction), d underestimates the true effect. Measurement error additionally pulls d downward via attenuation.
For your own study
The Effect Size Calculator computes exact values from your own data (CSV upload) — including confidence intervals via Bonett, Rosenthal, and the exact noncentral t-distribution.
Unstandardized is also an effect size
Standardization makes d unitless and comparable across studies — but that's exactly what loses the original scale. An unstandardized mean difference (in the plot above: "raw diff. = …") or an unstandardized regression coefficient b are often more directly interpretable, because they're expressed in the metric of the dependent variable itself — a metric you usually know well in your own field (points on a familiar test scale, days of hospital stay, euros of revenue …). A d of 0.4 says little on its own; a difference of "6 points on the well-being score," by contrast, can be placed directly against known reference values. It's usually most informative to report both: the raw difference for substantive interpretation, the standardized version for comparability across studies.