Written and maintained by CASRAI Editorial Board
Last updated
The Hosmer-Lemeshow test is a goodness-of-fit test for a logistic regression model: it asks whether the model’s predicted probabilities match the observed outcome rates closely enough to trust, by grouping subjects into deciles of predicted risk and comparing observed to expected event counts within each group. It was, for years, the default calibration statistic reported alongside a fitted logistic model. It is also, on its own terms, a limited way to summarize calibration — its sensitivity to sample size, its dependence on an arbitrary group count, and its low power in smaller samples are all well documented in the methodological literature, and alternative statistics and calibration-plot-based reporting have been developed in response. This guide covers the test’s actual mechanics with a worked, reproducible example, the large-sample-oversensitivity problem that is its most consequential limitation, and what to report instead.
What the Hosmer-Lemeshow test actually does
The test starts from a fitted logistic regression model that has already produced a predicted probability for every subject. The mechanics are a fixed sequence:
- Rank every subject by predicted probability, from lowest to highest.
- Split into g groups of roughly equal size — conventionally deciles, so g = 10, though the test can be run with fewer or more groups.
- Within each group, compute the observed number of events (O) — the actual count of subjects with outcome = 1 — and the expected number of events (E), which is the sum of the predicted probabilities of everyone in that group, not a count derived from the outcome data at all.
- Compute the statistic: for each group, (O − E)² divided by E(1 − E/n), summed across all g groups.
That sum is approximately chi-square distributed with g − 2 degrees of freedom under the null hypothesis that the model is correctly calibrated — for the standard g = 10 version, that is 8 degrees of freedom. A large statistic (and a correspondingly small p-value) means the observed and expected counts diverge more than sampling variation alone would explain: the model’s predicted probabilities don’t track reality well in at least one risk stratum. A non-significant result is conventionally read as “no detected evidence of miscalibration” — language worth taking literally, for reasons the next section covers.
This is the same logistic-regression fitting process covered in CASRAI’s guide to logistic regression (the logit model); if you haven’t already produced fitted probabilities from a glm() call in R or an equivalent procedure, that’s the prerequisite step this test is applied after.
A worked example (illustrative, reproducible)
The table below is generated from a simulated dataset, not a real study — disclosed explicitly because the numbers exist purely to show the mechanics, computed from a known data-generating process rather than presented as a finding. n = 100 subjects were simulated from a single-predictor logistic model, logit(p) = −2 + 1.4x with x drawn uniformly from 0 to 3, using a seeded random-number generator so the exact result is reproducible. The “model” here is the same process that generated the outcomes — a correctly specified model — which is why the test comes back non-significant.
| Decile | n | Observed (O) | Expected (E) | Predicted-risk range |
|---|---|---|---|---|
| 1 | 10 | 1 | 1.34 | 0.120–0.156 |
| 2 | 10 | 0 | 1.88 | 0.157–0.231 |
| 3 | 10 | 2 | 2.75 | 0.247–0.317 |
| 4 | 10 | 5 | 3.51 | 0.318–0.373 |
| 5 | 10 | 6 | 4.25 | 0.379–0.487 |
| 6 | 10 | 6 | 5.55 | 0.501–0.601 |
| 7 | 10 | 6 | 6.62 | 0.620–0.717 |
| 8 | 10 | 7 | 7.60 | 0.719–0.785 |
| 9 | 10 | 9 | 8.25 | 0.785–0.858 |
| 10 | 10 | 10 | 8.76 | 0.859–0.900 |
Summing (O − E)² / [E(1 − E/n)] across the ten rows gives a Hosmer-Lemeshow statistic of 7.181 on 8 degrees of freedom (g − 2 = 10 − 2), for a p-value of 0.517. Nowhere close to significant, which is the expected result for a model whose predicted probabilities were the actual generating process for the outcomes — there’s no real miscalibration to detect. In practice you’d get this table and statistic directly from software rather than computing it by hand: ResourceSelection::hoslem.test() in R, estat gof after logistic in Stata, or the Hosmer-Lemeshow table SPSS prints automatically as part of its binary logistic regression procedure output.
The known limitation: oversensitivity at large sample sizes
The (O − E)² term in the Hosmer-Lemeshow statistic scales with how far apart observed and expected counts are in absolute terms, and that gap tends to grow with the size of each decile group — which grows with total sample size once the number of groups is fixed at g = 10. The practical consequence: a fixed, small, arguably clinically negligible degree of miscalibration produces a non-significant test in a modest sample and a highly significant test in a large one, even though the actual calibration error hasn’t changed at all.
That isn’t a hypothetical concern — it reproduces directly from a controlled simulation. Take a model whose stated predicted probabilities are shifted from the true outcome-generating probabilities by a fixed, small amount (a 0.15 shift on the logit scale, roughly a couple of percentage points of predicted risk in the middle of the range) and run the identical Hosmer-Lemeshow procedure at two sample sizes, holding the miscalibration constant:
| Sample size (n) | HL statistic | df | p-value |
|---|---|---|---|
| 200 | 7.492 | 8 | 0.485 |
| 20,000 | 113.191 | 8 | <0.0001 |
Same model, same fixed 0.15-logit miscalibration, same test — non-significant at n = 200, overwhelmingly significant at n = 20,000. A test whose verdict flips from “fine” to “reject” purely as a function of how many subjects happened to be available, with the actual calibration error held constant, is not answering “is this model usable” in any sample-size-independent sense. This is precisely the failure mode the prediction-modeling literature has pushed away from over the past decade, in favor of calibration curves and calibration-slope reporting that don’t collapse “how big is the miscalibration” and “how much data do we have” into one number.
The companion problem — that the test’s result also depends on the arbitrary choice of how many groups to use, and that a non-significant result in a small sample often reflects low statistical power rather than genuinely good calibration — is covered in CASRAI’s guide to calibration plots for prediction models, which cites Van Calster and colleagues’ STRATOS-initiative case against relying on Hosmer-Lemeshow as the primary calibration statistic. The two limitations reinforce each other: a test whose behavior already depends on an arbitrary grouping choice becomes additionally unreliable once its significance also depends on sample size in a way that doesn’t track the thing it’s meant to measure.
What the test can’t tell you, even when it “passes”
A non-significant Hosmer-Lemeshow p-value is a single number. It doesn’t say:
- Where in the risk range calibration is good or bad — a model can be well-calibrated in the middle deciles and badly off at the extremes, and the omnibus statistic can still come back non-significant because the deviations partly cancel across groups.
- Which direction any miscalibration runs — systematically over-predicting risk (calibration intercept below zero) versus under-predicting it, or predictions that are too extreme versus too conservative (calibration slope away from 1) look identical to a single chi-square p-value but require very different fixes.
- How large the practical miscalibration is — statistical significance and clinical or decision-relevant magnitude are different questions, and the large-sample example above shows the test conflating them directly.
A calibration plot with the calibration intercept and slope reported alongside it answers all three of those questions in one figure: the plot shows where on the risk range the model’s predictions and observed outcomes diverge, the intercept shows the direction of any overall miscalibration, and the slope shows whether predictions are too extreme (slope < 1) or too conservative (slope > 1). See CASRAI’s calibration plots for prediction models guide for how to build and read that plot, including the calibration hierarchy (mean, weak, moderate, strong calibration) that has largely superseded a single Hosmer-Lemeshow p-value in current prediction-model reporting guidance such as TRIPOD+AI, covered in CASRAI’s STARD and TRIPOD+AI guide.
Where you’ll still see it, and how to read it responsibly
The Hosmer-Lemeshow test hasn’t disappeared from practice — it remains the default output of several statistical packages’ logistic regression procedures, and reviewers at some journals still expect to see it reported, particularly in older subfields or where a calibration plot wasn’t part of the original analysis plan. If you’re reporting it because it’s expected rather than because it’s the best available evidence, a few practices limit the damage:
- Report it alongside a calibration plot, not instead of one. The p-value alone, in either direction, is not sufficient evidence of good or bad calibration on its own.
- State the group count used (g). A Hosmer-Lemeshow result without the number of groups isn’t fully reproducible — different g values can produce different p-values from the same fitted model and data.
- Don’t treat a non-significant result as proof of good calibration in a small-to-moderate sample — absence of a significant result there is at least as likely to reflect the test’s low power as genuinely close observed-expected agreement.
- Don’t treat a significant result as proof the model is unusable in a large sample without checking the calibration plot first — the large-sample example above shows a trivial, likely inconsequential shift producing an overwhelming rejection.
The test originates from Hosmer and Lemeshow’s 1980 paper “Goodness of Fit Tests for the Multiple Logistic Regression Model” in Communications in Statistics. Statistician Frank Harrell’s widely cited critique of the test lists essentially the same weaknesses covered above — sensitivity to the arbitrary choice of bins, power that is often too low to be useful, and no accounting for overfitting — and points to the Osius-Rojek test and the Stukel test as alternative omnibus goodness-of-fit statistics with better theoretical properties. None of these alternatives has displaced calibration-plot-plus-slope reporting as the primary recommendation in current prediction-model guidance; they’re a better version of the same single-number approach, not a substitute for seeing where and how a model is miscalibrated.
Frequently asked questions
What counts as a good Hosmer-Lemeshow p-value?
Conventionally, p > 0.05 is read as “no detected evidence of miscalibration,” using the same significance-threshold logic as any other goodness-of-fit test. Given the large-sample-oversensitivity problem above, that threshold means something different in a sample of 200 than in a sample of 20,000 — treat it as one input alongside a calibration plot, not a standalone pass/fail criterion.
How many groups (deciles) should the test use?
Ten (deciles) is the conventional default and what most software uses out of the box. The choice is genuinely arbitrary in the sense that different group counts can produce different results from identical data — one of the documented weaknesses of the test, not a setting with a single objectively correct value.
Why does SPSS/Stata/R report Hosmer-Lemeshow automatically but not a calibration plot?
The test predates routine calibration-plot reporting by decades and became the default output before the more informative alternative was standard practice; software output conventions tend to lag methodological consensus. A calibration plot with slope and intercept isn’t automatic output in most packages and generally has to be built from the model’s predicted probabilities directly, as covered in CASRAI’s calibration plots guide.
Is the Hosmer-Lemeshow test the same as a chi-square goodness-of-fit test?
They’re built on the same observed-versus-expected chi-square logic covered in CASRAI’s general chi-square test guide, but the Hosmer-Lemeshow test is a specific application of that logic to grouped predicted probabilities from a fitted logistic model, with its own degrees-of-freedom convention (g − 2) and its own documented weaknesses — it isn’t interchangeable with a standard categorical chi-square goodness-of-fit test.
Should I still run it at all?
It’s reasonable to report as a secondary, familiar statistic if your audience expects it, but it shouldn’t be the only calibration evidence in a paper reporting a new prediction model. A calibration plot with slope and intercept, reported per the calibration hierarchy in TRIPOD+AI-era guidance, is the stronger and now generally expected primary evidence.
