N = 30 students
X₁ = study time (h/week)
X₂ = sleep hours (h/night)
Y = exam grade (0–100 pts.)
Seed: —
What you're seeing
30 simulated students: study time X₁ (h/week) → exam grade Y (points). The orange line is your estimated line — draggable via two round anchors at the left and right edge. Turn on ● Residuals: green marks = positive residuals (point above the line), red marks = negative residuals (point below it). A residual eᵢ = yᵢ − ŷᵢ is the deviation of the observed value from the predicted value.
What to do
Drag the anchors until your RSS (orange bar, absolute value) is as small as possible. Then click ◎ Show OLS — the blue OLS line appears. The second bar shows its RSS. Compare: how close did you get to the minimum?
What is RSS? (Residual Sum of Squares)
RSS = Σeᵢ² = Σ(yᵢ − ŷᵢ)². OLS finds analytically the exact b₀ and b₁ for which RSS is globally minimal — unique and provable. No trial and error: b₁ = Cov(X,Y)/Var(X), b₀ = ȳ − b₁·x̄.
Why squares, not absolute values?
(1) For every OLS line, Σeᵢ = 0 holds — positive and negative residuals trivially cancel out. (2) Squaring penalizes large residuals disproportionately: a residual of 10 costs 100, not 10. (3) The square yields a differentiable function → a closed-form, unique solution.
What to observe
· Wrong slope → RSS rises faster than a wrong intercept.
· Points at the edge of X (high leverage) swing b₁ strongly if they deviate far from the line.
· Deliberately try bad lines (steep, flat, negative) — the RSS value explodes.
· Gauss-Markov: under the 5 OLS assumptions (→ Module ③) OLS is BLUE — Best Linear Unbiased Estimator. No other linear unbiased estimator has smaller variance.
What you're seeing
The OLS line through all 30 data points. Enable △ Slope for the slope triangle and ✛ Mean for the centroid (x̄, ȳ) on the canvas.
b₁ = — — slope (unstandardized)
Per +1 h of study time, the expected grade rises by — points. Unit: points/hour. Interpretation: descriptive, not a causal effect. b₁ is scale-dependent — a b₁ from another study with different scales isn't directly comparable.
The slope triangle on the canvas shows this concretely: +3 h of study time → +— points (= 3 · b₁).
β = — — standardized coefficient
Per +1 SD in X₁, Y rises by β SD. Bivariately, β = r (Pearson correlation) always holds — check this in the coefficient cell! β allows comparing predictors on different scales within a study, but not across studies (it depends on SD(X) and SD(Y)).
Important: β does not directly indicate the importance or causal influence of a predictor — with correlated predictors, β can deviate strongly from the partial effect (→ Module ④).
b₀ = — — intercept
Expected grade at X₁ = 0 h of study time. Since 0 h lies outside the observed range, this is an extrapolation — usually not meaningfully interpretable in substantive terms. b₀ is mathematically necessary but rarely has stand-alone meaning.
Centroid (x̄ = — h | ȳ = — pts)
The OLS line always passes through (x̄, ȳ) — mathematically necessary, since b₀ = ȳ − b₁·x̄. Enable ✛ Mean and check this visually. OLS estimates E[Y|X] — the conditional expectation, not an individual prediction.
What you're seeing — three modes (TSS = SSM + RSS)
TSS (gray marks): distances of all points from the mean ȳ — the total spread in Y, independent of the model. SSM (green marks): distances of the model (regression line) from the mean ȳ — the share the model explains over "just ȳ as the prediction". RSS (red marks): distances of the points from the model — the residuals, i.e. what remains unexplained. The bar on the left shows the SSM : RSS ratio. Switch modes using the buttons.
TSS = — — Total Sum of Squares
TSS = Σ(yᵢ − ȳ)². Total variance in Y — how much do grades vary in general? TSS does not depend on the model. It's the starting point that is to be explained.
SSM = — (—% of TSS) — Sum of Squares Model
SSM = Σ(ŷᵢ − ȳ)². The share explained by the model — the distance of the regression line from the mean ȳ at each data point (one contribution per observation), i.e. the improvement over "just ȳ". The larger SSM is relative to TSS, the better the fit. OLS maximizes SSM (equivalent to minimizing RSS).
RSS = — (—% of TSS) — Residual Sum of Squares
RSS = Σeᵢ² = Σ(yᵢ − ŷᵢ)² — the squared sum of all residuals, what the model doesn't explain. TSS = SSM + RSS holds exactly. RSS = 0 would be a perfect fit — impossible in practice and not desirable (overfitting).
R² and f² (Cohen's effect size)
R² = SSM/TSS (→ stats on the left): proportion of explained variance. Caution: R² always increases with every additional predictor — even useless ones! Adjusted R² corrects this with a penalty term per predictor.
f² = R²/(1−R²) = SSM/RSS = — → a — effect. f² is the ratio of explained to unexplained variance. Cohen (1988): .02 small · .15 medium · .35 large.
Model comparisons: AIC and BIC penalize complexity more strongly than adj. R² — recommended when comparing several competing models.
What's good / problematic
· Good: large green bar (SSM ≫ RSS), R² close to 1, f² ≥ .15.
· R² = 0: the model explains nothing — no better than ȳ as a forecast.
· R² alone is not sufficient — a good R² despite violated assumptions is misleading. Residual diagnostics are mandatory (→ OLS assumptions accordion below).
| Model | b₁ | β₁ | R² |
|---|---|---|---|
| Bivariate | — | — | — |
| Partial | — | — | — |
From the line to the plane
Bivariately (Y ~ X₁), the solution is a line in 2D space. With two predictors (Y ~ X₁ + X₂), it becomes a regression surface (plane) in 3D space. Picture it: X₁ on the x-axis, X₂ on the z-axis, Y on the y-axis — the OLS minimum is the plane that minimizes RSS across all points. b₁ is the slope in the X₁ direction, b₂ in the X₂ direction — each holding the other predictor constant.
Why multiple regression?
X₁ (study time) and X₂ (sleep) correlate with r₁₂ = —. The bivariate model Y ~ X₁ does not measure the pure X₁ effect — b₁ also contains the influence of X₂, since X₂ is intertwined with X₁. The multiple model solves this: it holds X₂ statistically constant.
Reading the coefficients (table on the left)
Bivariate b₁: slope from Y ~ X₁ alone — contains confounding by X₂.
Partial b₁: slope from Y ~ X₁ + X₂ — adjusted for X₂. This is the value the regression surface has in the X₁ direction.
Δb₁: the difference quantifies the confounding. At r₁₂ = 0, Δb₁ = 0.
Step ① — bivariate starting point
What you're seeing: the bivariate OLS line Y ~ X₁ (study time → grade) — identical to Module ①.
Problem: when r₁₂ ≠ 0, b₁ contains the influence of X₂ as a confound. The bivariate estimator is biased as long as X₂ correlates with X₁ and also affects Y.
Tip: note the bivariate b₁ (table on the left), then change r₁₂ and compare.
Confounding compared
bivariate b₁ = — · partial b₁ = — · Δb₁ = —
The larger r₁₂, the more the bivariate deviates from the partial b₁. At r₁₂ = 0 both are identical — X₂ confounds nothing. Try the four r₁₂ scenarios on the left.
min Σ(yᵢ − ŷᵢ)².
Squaring has two reasons: signs are neutralized,
and large residuals are penalized disproportionately more than small ones.
The solution is unique and always yields Σeᵢ = 0.
b₁ = Σ(xᵢ−x̄)(yᵢ−ȳ) / Σ(xᵢ−x̄)².
The numerator is the covariance, the denominator the variance of X.
ȳ = b₀ + b₁·x̄.
R² = SSM/SST ∈ [0, 1] — proportion of explained variance.
R² increases with every added predictor, even useless ones.
Adjusted R² corrects for this:
adj.R² = 1 − (1−R²)·(n−1)/(n−k−1).
adj.R² decreases when a new predictor explains less than expected by chance.
e(Y|X₂) on e(X₁|X₂).
This value corresponds to the partial slope: the effect of X₁ on Y,
after the shared portion of X₂ has been removed from both.
β = b · (SD_X / SD_Y) — the standardized regression coefficient.
β indicates how many standard deviations Y rises
when X rises by one SD. Allows comparison of predictors
on different scales, but only within a sample.
Across studies, β values are not directly comparable because of differing SDs.
Ordinary Least Squares (OLS) is the standard method for estimating
linear regression models. It finds the coefficients b₀ and b₁ that minimize the
sum of squared residuals:
min Σ(yᵢ − ŷᵢ)² = min Σ(yᵢ − b₀ − b₁·xᵢ)².
The solution follows from the normal equations:
b₁ = Σ(xᵢ−x̄)(yᵢ−ȳ) / Σ(xᵢ−x̄)²b₀ = ȳ − b₁·x̄b = (XᵀX)⁻¹ Xᵀyb₁ (unstandardized): "Per +1 unit of X, Ŷ rises by b₁ units, holding all other predictors constant." The "ceteris paribus" is crucial — in the multiple model, b₁ is a partial effect, not a marginal raw effect.
β (standardized): comparison of predictors on different scales. Caution: β is sample-specific and must not be compared across studies.
R² = SSM/TSS = 1 − RSS/TSS. In the bivariate case, R² = r².
R² increases with every predictor, even random ones (Freedman's paradox).
Adjusted R² corrects for the number of predictors k:
adj.R² = 1 − (1−R²)·(n−1)/(n−k−1)
f² = R²/(1−R²) — effect size for multiple regression.
Conventions (Cohen 1988): small ≥ .02, medium ≥ .15, large ≥ .35.
Better than conventions: SESOI — define the smallest
substantively meaningful effect before the study. → See
Power & Sample Size.