📋 What This Is About
From the 8-item cognitive test (factor analysis, measurement models, CFA), we pick out the verbal
subtest V1–V4 and use it to compare two groups — say, a German and a translated test version, or two
testing time points. Before you compare means, you must be sure the test measures the same thing in the
same way in both groups. If that's violated (non-invariance / DIF), a "group
difference" can be a pure measurement artifact. Set the true difference and a violation —
and see how much of the observed difference is real.
What Is Measurement Invariance — and What Is It For?
Measurement invariance (factorial invariance) holds when an instrument measures a construct the same way across different groups (gender, countries) or different time points — formally: when the measurement model's parameters (loadings λ, intercepts τ, error variances θ) are equal across groups. What's it for? Only then does a difference in the test score also mean a difference in the construct — and not merely in the instrument. Without invariance, you're comparing apples to oranges: a "group difference" can be a pure measurement artifact. It is tested as a nested hierarchy in a multi-group SEM (diagram ② below): step by step, more parameters are set equal across groups (green paths) and it is checked whether the model still fits.
| Level | what is equal? | allows comparing … | in the path diagram (②), recognizable by |
|---|---|---|---|
| Configural | only the structure: the same items load on the same factor | — (only: "same construct, same pattern") | All four items hang on the same factor ξ in both groups. This basic structure always holds here. |
| Metric (weak) | + loadings λ equal | associations, covariances, regressions | All λ paths green. A red λ path (Δλ) breaks this level. |
| Scalar (strong) | + intercepts τ equal | latent & observed means | Additionally all τ in the item boxes green. A red τ (Δτ) breaks this level. |
| Strict | + error variances θ (residuals ε) equal | observed sum scores directly (equal measurement precision) | Additionally the residuals ε equal across groups (satisfied here by design once scalar holds). |
① What Would You Conclude?
—
Observed Score Difference
—
avg. sum score B − avg. A
of which real (latent)
—
Δμ · (Σλ / k)
of which artifact
—
(Δτ + Δμ·Δλ) / k
② The Measurement Model as a Multi-Group CFA
Left group A, right group B — a complete path diagram each (one factor ξ, four items), with the parameter values of that group. Green = the parameter is equal in both groups (invariant), red = it differs between the groups (non-invariant). In a multi-group SEM, parameters are tentatively set equal; if the model still fits, invariance holds at this level.
③ Observed Item Means by Group
Gray bars = group A, blue = group B. The violated item is outlined in red.
④ Decomposition of the Observed Difference
The blue portion is the legitimate (latent) difference, the orange portion the artifact. Only when orange = 0 does the observed difference truly measure the construct.
⑤ Full Calculation: Items Only → Sum Score vs. SEM
Flashcards
① The Invariance Hierarchy
Configural ⊂ metric ⊂ scalar ⊂ strict —
each level additionally requires equality: configural only the same structure, metric equal loadings λ,
scalar additionally equal intercepts τ, strict additionally equal error variances θ.
Tested in a nested fashion, usually via a multi-group CFA model.
② Why Scalar for Means?
An observed item mean is τ + λ·μ. To compare latent means μ,
intercept τ and loading λ must be equal across groups — otherwise it's not separable whether a higher
score comes from a higher construct level (μ) or a shifted item (τ). That's
exactly what the orange portion makes visible.
③ Two Error Directions: Illusion vs. Masking
Non-invariance can mislead in both directions. Illusory difference ("scalar violated"):
Δμ = 0, but a DIF creates a difference where none exists → false positive.
Masking ("masked diff."): Δμ is real (+0.5), but an opposing DIF (Δτ = −0.7)
hides it — the sum score shows only +0.175 instead of the true difference → false negative.
In both cases the raw mean comparison is misleading; the SEM recovers it via the anchors, κ_B = 0.5.
④ Score Scale ≠ Latent Scale
The latent difference Δμ and the observed score difference live on
different metrics. Even under full invariance, the observed difference is only Δμ·(Σλ/k) —
the latent difference scaled by the mean loading (here ×0.70). They'd only be equal at a mean
loading of 1. That's why "of which real" is not Δμ itself, but its image on the score scale — and in
an SEM you compare the latent means directly instead of raw sum scores. The chain above the diagrams shows this
translation.
⑤ Uniform vs. Non-Uniform DIF
Δτ shifts the item equally for everyone (uniform DIF) → violates only the
scalar level. Δλ changes how strongly the item measures the construct (non-uniform DIF)
→ already violates the metric level and hence also association comparisons. The same phenomenon is
called item DIF in IRT.
→ DIF (IRT)
⑥ Partial Invariance & Anchors
Rarely are all items invariant. With enough invariant anchor items, the
latent scale can still be identified, and individual items are allowed free parameters (partial
invariance, Byrne/Shavelson/Muthén 1989). Rule of thumb: at least two invariant items per factor.
⑦ How It's Tested in Practice
Nest and compare models: the Δχ² difference test (often too strict with large n) or,
better, ΔCFI ≤ .01 / ΔRMSEA ≤ .015 (Cheung & Rensvold 2002; Chen 2007). If a level doesn't
hold, the corresponding group comparison is not valid — or one moves to partial invariance.
⑧ Related Tools
Measurement invariance is the group comparison of measurement models: the same
CFA building blocks as in the Measurement Model and CFA tools, just restricted across groups. Shifted
intercepts are reminiscent of norm shifts.
→ Confirmatory Factor Analysis (CFA)
→ Measurement Models
→ Structural Equation Model (SEM)
→ Norming