📋 Example — clinical antidepressant trial
A researcher is planning a study: expects d = 0.50 (from the literature),
n = 64 per group → planned power:
80%.
A pilot study points to d ≈ 0.30 —
Section 3 shows what that means.
Every conceivable true δ has a different power. The often-cited "80% power" applies only for exactly one assumed δ — and you never know that value for certain before the study.
α and power move in opposite directions
A stricter α (e.g. 0.01 instead of 0.05) lowers power for the same n and δ — you make it harder to reject H₀, and thereby also miss more real effects.
Cohen: rules of thumb, not laws
Cohen (1988) derived d = 0.2/0.5/0.8 from the behavioral-science studies of his time — as a stopgap for when no domain knowledge is available. In medicine, d = 0.1 can be clinically meaningful; in basic research, d = 0.8 can be trivial. The relevant ES is always context-specific.
The MDES (Minimum Detectable Effect Size) should come from clinical/practical relevance, not from statistical conventions. Ask: "What effect would be small enough to ignore?"
Post-hoc power is misleading
Computing power after a non-significant study using the observed d is circular: it just reflects the p-value back at you and says nothing about the original planning. Use only a-priori power (before the study).
Power & Sample Size — Help
Example
A clinician is planning an RCT for an antidepressant. From the literature they expect Cohen's d = 0.50 on the HAM-D scale. With n = 64 per group and α = 0.05 (two-tailed), the planned power comes out to about 80%. A pilot study, however, points to d ≈ 0.30 — Section 3 shows what that means for the actual power.
Power (1−β) is the probability of detecting a true effect — that is, correctly rejecting H₀ when H₁ holds. Power depends on four quantities: effect size δ, sample size n, significance level α, and test direction.
Power = P(reject H₀ | H₁ true) = 1 − β
Why is power a curve?
Every possible true δ yields a different power. At δ = 0, power equals the α level (false positive). As δ grows, power rises toward 1. Section 1 shows this curve — and how it shifts with n. The often-cited "80% power" always applies only for one specific δ.
Normal approximation of power
Power ≈ Φ(|d| · √(n/2) − z_{α/2}) [two-tailed, 2 groups]
Power ≈ Φ(|d| · √n − z_{α/2}) [two-tailed, 1 group]
Φ = standard normal distribution (cumulative), z_{α/2} = quantile for α/2. The approximation is very accurate for n ≥ 20.
Standardized effect size: d = (μ₁ − μ₂) / σ. The small/medium/large (0.2/0.5/0.8) conventions are historical rules of thumb from 1960s–70s behavioral research — not universal standards. Cohen himself warned against adopting them uncritically. In medicine, d = 0.1 can be clinically meaningful (blood-pressure reduction × millions of patients); in basic research, d = 0.8 can be trivial.
SESOI — how do you determine the relevant effect size?
The core problem of sample-size planning isn't the computation — it's the substantive question: What effect do I want to reliably detect?Lakens (2022) suggests anchoring planning on the SESOI — the Smallest Effect Size of Interest: the smallest effect that would still be practically or clinically meaningful.
The SESOI is fundamentally different from the question "what did earlier studies find?" — because published effect sizes are systematically overestimated by selection bias and publication bias (the winner's curse). Anyone who bases their planning on the mean published d risks an inflated effect expectation and therefore chronic underpowering.
Four ways to determine the SESOI:
1. Clinical/practical anchor (recommended): Which effect would change a decision — e.g. treatment guidelines, approval, reimbursement? In medicine, many scales already have empirically determined minimum thresholds (MCID, Minimal Clinically Important Difference). Example: on the HAM-D, a reduction of ≥ 3 points is considered clinically meaningful — which corresponds to a specific d.
2. Resource-based justification: Work backward: what n is realistically achievable? What MDES does that give you at 80% power? Is that MDES still substantively relevant — or so large that only large, implausible effects would be detected?
3. Lower bound from theory: What does theory say about the minimum size of the effect? If a mechanism is postulated, you can often argue below which threshold it practically doesn't exist.
4. Prior literature — with caution: Meta-analytic estimates from pre-selected samples overestimate the true effect. Better: uncorrected raw data from registry studies or preregistered replications. If only published effects are available, set the planning value noticeably more conservatively (e.g. 50–70% of the published d).
Connecting SESOI and MDES: Once the SESOI has been substantively determined, Section 3 of this tool provides the answer: is my planned n large enough to reliably detect the SESOI (MDES ≤ SESOI)? If not, n must be increased or the design changed.
Reference:Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1), 33267.
doi:10.1525/collabra.33267 — open access, includes R code and interactive examples. · Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Erlbaum.
Terms
α (Type I error): probability of incorrectly rejecting H₀. β (Type II error): probability of missing a real effect. Power = 1−β. NCP (non-centrality parameter): δ · √(n/2) — indicates how far H₁ is from H₀ (in SD units of the test statistic). MDES (Minimum Detectable Effect Size): smallest effect that can be detected with given n and α at target power. Readable from Section 3. SESOI (Smallest Effect Size of Interest): the smallest effect that would still be substantively practically or clinically relevant. Planning should ensure MDES ≤ SESOI. Post-hoc power: power computed after the study using the observed d — circular and uninformative, because it just reflects the p-value back at you.