Written and maintained by CASRAI Editorial Board
Last updated
Games-Howell is the post hoc test for the all-pairwise ANOVA follow-up when group variances aren’t equal. Tukey HSD, Tukey-Kramer, and the Scheffé test all assume homogeneity of variance across groups; when Levene’s test comes back significant, that assumption has failed, and running Tukey HSD anyway inflates the Type I error rate for whichever pairs involve the higher-variance group. Games-Howell fixes this by computing a separate Welch-style standard error and a separate Welch-Satterthwaite adjusted degrees of freedom for each pairwise comparison, rather than pooling variance across the whole design the way Tukey does. This guide covers how the test works, why the per-pair degrees of freedom matter, how it differs from Dunnett’s test and Tukey HSD, and how to run and report it, with a fully reproducible worked example.
When to use Games-Howell instead of Tukey or Scheffé
Games-Howell belongs in the same slot Tukey HSD or the Scheffé test would otherwise fill: the all-pairwise follow-up after a significant omnibus ANOVA, where every group is compared against every other group. The difference is the assumption each test is willing to make about variance:
- Tukey HSD and Tukey-Kramer pool variance across all groups into a single error term and assume that pooled estimate applies equally everywhere. Valid when variances are roughly equal (Levene’s test non-significant); increasingly wrong as variances diverge.
- Scheffé’s test also relies on a pooled error term (it’s built for testing arbitrary linear contrasts, not just the standard pairwise set), and inherits the same equal-variance assumption Tukey does.
- Games-Howell drops the pooled-variance assumption entirely. Each pairwise comparison gets its own standard error, computed from only the two groups being compared, and its own degrees of freedom via the Welch-Satterthwaite approximation. It doesn’t require equal group sizes either, which matters in practice since unequal-n designs and unequal variance often show up together.
The practical routing rule: run Levene’s test (or Brown-Forsythe) first. If it’s non-significant, Tukey HSD is the more standard, more widely reported choice. If it’s significant, Games-Howell is the correct all-pairwise post hoc test — not Tukey HSD run anyway, and not a Bonferroni correction on a set of separate Welch t-tests, which is a defensible alternative but not the same procedure or the same critical value.
Games-Howell vs. Dunnett’s test with unequal variances
Dunnett’s test solves a different problem: comparing several treatment groups against one shared control, not every group against every other group. Standard Dunnett’s test also assumes equal variances, and when that assumption fails, some software and packages offer a Games-Howell-style variance correction applied specifically to the many-to-one comparison set. That variant is still answering Dunnett’s narrower question (treatment vs. control only); it is not the same computation as the all-pairwise Games-Howell test described here, which corrects for the full set of k(k−1)/2 comparisons. Use the many-to-one variant only when the research design genuinely has a single reference group and no interest in treatment-vs-treatment differences; use standard Games-Howell for the general all-pairwise case.
How the per-pair adjustment works
For two groups i and j with means Mi, Mj, sample variances s2i, s2j, and sample sizes ni, nj, Games-Howell computes:
Standard error (Welch-style, unpooled): SE = √(s2i/ni + s2j/nj)
Welch-Satterthwaite adjusted degrees of freedom, specific to this one pair:
df = (s2i/ni + s2j/nj)2 ÷ [ (s2i/ni)2/(ni−1) + (s2j/nj)2/(nj−1) ]
This is the same Welch-Satterthwaite approximation used by Welch’s unequal-variances t-test and by Welch’s ANOVA — it down-weights the contribution of whichever group has the smaller, less reliable variance estimate, which is why the resulting df is rarely a whole number and is almost always lower than the pooled df Tukey HSD would use for the same pair.
Test statistic and p-value: the studentized-range statistic for the pair is q = |Mi − Mj| ÷ (SE/√2), evaluated against the studentized range distribution with k (the total number of groups in the design) and the pair’s own Welch-Satterthwaite df — not the design’s overall error df, which is exactly what makes the test’s degrees of freedom different for every pair unless every group happens to share identical variance and size. Because each pair gets its own df, two pairs in the same analysis can end up with visibly different critical values even though they’re using the same k and the same alpha.
A worked example
The values below are a seeded, reproducible simulation (illustrative composite, not data from a real study) run in Python with numpy/scipy (seed=42), computing each step directly rather than asserting a plausible-looking result. Three groups were generated with deliberately unequal size and unequal spread — the exact scenario that makes Tukey HSD invalid and Games-Howell the correct choice:
| Group | n | Mean | SD |
|---|---|---|---|
| A | 20 | 49.84 | 4.35 |
| B | 15 | 55.17 | 6.36 |
| C | 10 | 59.36 | 9.74 |
Levene’s test (median-centered) on these three groups: W = 4.041, p = .0248 — significant, so the equal-variance assumption Tukey HSD needs is violated and Games-Howell is the appropriate follow-up. Running the pairwise comparisons above:
| Comparison | Mean diff. | SE | Welch-Satterthwaite df | q | p | 95% CI |
|---|---|---|---|---|---|---|
| A − B | −5.33 | 1.91 | 23.44 | 3.953 | .0265 | [−10.10, −0.56] |
| A − C | −9.52 | 3.23 | 10.83 | 4.168 | .0332 | [−18.27, −0.78] |
| B − C | −4.19 | 3.49 | 14.10 | 1.698 | .4719 | [−13.32, 4.94] |
Notice the three comparisons carry three different degrees of freedom (23.44, 10.83, 14.10) despite all three coming from the same three-group design — that variability is the whole point of computing df per pair instead of pooling it. Group A differs significantly from both B and C; B and C do not differ significantly from each other, even though the raw mean gap between them (4.19) is not much smaller than the A−B gap (5.33) — Group C’s much larger variance and smaller sample size widen its standard error enough to erase that comparison’s significance.
Running it in software
Most general statistics packages expose Games-Howell directly rather than requiring a manual per-pair computation:
- SPSS: Analyze › Compare Means › One-Way ANOVA › Post Hoc, under the “Games-Howell” checkbox in the group not assuming equal variances (alongside Tamhane’s T2, Dunnett’s T3, and Dunnett’s C — SPSS groups all four unequal-variance post hoc options together, separate from the Tukey/Scheffé/Bonferroni options that assume equal variances).
- R:
rstatix::games_howell_test()andPMCMRplus::gamesHowellTest()both implement the standard procedure; base R has no built-in Games-Howell function, so one of these packages (or an equivalent) is required. - Python: the
pingouinpackage’spairwise_gameshowell()function implements it; basescipydoes not include a packaged Games-Howell function, though the underlying pieces (Welch-Satterthwaite df, the studentized range distribution) are all inscipy.statsand can be assembled manually, as in the worked example above.
How to report it
State the omnibus test, confirm Levene’s test flagged unequal variances (the reason Games-Howell rather than Tukey was used), and report each significant pairwise comparison with its own degrees of freedom:
“A one-way ANOVA showed a significant effect of group on the outcome, and Levene’s test indicated unequal variances across groups, W = 4.04, p = .025. Games-Howell post hoc comparisons (which do not assume equal variances) showed Group A scored significantly lower than both Group B (mean difference = −5.33, 95% CI [−10.10, −0.56], p = .027) and Group C (mean difference = −9.52, 95% CI [−18.27, −0.78], p = .033); Groups B and C did not differ significantly (p = .472).”
Reporting the per-pair degrees of freedom (or at minimum noting that they vary by pair) signals to a reviewer that the correct unequal-variances procedure was used rather than a standard Tukey HSD run without checking the Levene’s test result first.
Frequently asked questions
Does Games-Howell require equal group sizes?
No. Unlike classic Tukey HSD (which assumes equal n, with Tukey-Kramer as the standard unequal-n extension), Games-Howell was built to handle unequal group sizes and unequal variances together, since the two problems frequently co-occur in real data — a naturally occurring subgroup that’s both smaller and more variable than the rest.
What if Levene’s test is non-significant — should I still use Games-Howell to be safe?
Not as a default. When variances are genuinely equal, Tukey HSD is the more standard, more widely taught, and marginally more powerful choice for that condition, and reviewers generally expect it unless there’s a stated reason to deviate. Switching to Games-Howell “just in case” without a significant Levene’s result reads as an unmotivated departure from convention rather than a methodologically justified one; let the Levene’s test result drive the choice.
Is Games-Howell the same as Welch’s ANOVA?
No, though they share the same Welch-Satterthwaite logic. Welch’s ANOVA is the omnibus test — it replaces the standard F-test as the first step, correcting for unequal variances across the whole design and answering only “does group membership matter at all.” Games-Howell is a post hoc procedure that answers “which specific pairs differ,” run after an omnibus test (classic or Welch) has already indicated a significant effect. The two are commonly paired: Welch’s ANOVA for the omnibus step, Games-Howell for the pairwise follow-up, when Levene’s test has flagged unequal variances.
Can Games-Howell be used with only two groups?
It can be computed, but with only two groups it reduces to essentially the same comparison a two-sample Welch’s t-test already provides — there’s no multiple-comparison correction to gain from a post hoc procedure when there’s only one comparison to make. Games-Howell’s actual value is in the three-or-more-group all-pairwise case.
See also the Research Methods & Statistics hub for the rest of the post hoc test family, effect size reporting, and study design guidance.
