Hosmer-Lemeshow Test

Introduction

The Hosmer-Lemeshow Test checks whether a fitted logistic regression model is well calibrated-that is, whether the probabilities it predicts actually line up with how often the event really happens. If you are asking questions like "do patients I predict as 80% likely to default actually default about 80% of the time?", "is my churn model overconfident for high-risk customers but underconfident for low-risk ones?", or "how do I formally test goodness-of-fit for a logistic regression the way an F-test checks fit in linear regression?"-this is usually the test you are looking for.

By the end of this article you will be able to state exactly when a Hosmer-Lemeshow Test applies, build the deciles-of-risk table and compute the statistic completely by hand on two different worked examples, understand why the choice of group count matters, interpret the result correctly (including its well-documented limitations), know what to do if poor fit is detected, and run the same test in Python.

What Is the Hosmer-Lemeshow Test?

Logistic regression outputs a predicted probability for each observation, but unlike linear regression there is no natural residual sum of squares or R² to judge how well the model fits. The Hosmer-Lemeshow Test, introduced by Hosmer and Lemeshow in 1980, fills that gap by asking a very concrete question: if I sort everyone by their predicted risk and split them into groups, does the number of events that actually happened in each group match the number the model predicted?

The test works by ranking all observations by their predicted probability \( \hat{p}_i \) and dividing them into \( g \) roughly equal-sized groups-traditionally \( g = 10 \) "deciles of risk." Within each group, it compares the observed count of events against the expected count (simply the sum of predicted probabilities in that group). If the model is well calibrated, these two numbers should be close in every group; if the model systematically over- or under-predicts risk in some ranges, the observed and expected counts will diverge, and the test statistic picks that up.

It exists because a logistic regression can discriminate well-correctly ranking who is more or less likely to experience the event-while still being poorly calibrated, meaning the actual predicted numbers cannot be trusted at face value. This distinction matters enormously in fields like medicine and credit risk, where the predicted probability itself (not just the ranking) drives real decisions.

When to Use It

• After fitting a binary logistic regression: when you need a formal goodness-of-fit check to accompany discrimination metrics like the AUC, especially before deploying the model for decisions based on the predicted probabilities themselves.
• Risk scoring and clinical prediction models: classic use cases include predicting mortality, readmission, or disease risk, where clinicians rely on the actual predicted percentage, not just a ranked list of patients.
• Credit scoring and churn models: when a predicted default or churn probability feeds directly into pricing, provisioning, or retention-offer decisions, calibration failures translate directly into financial cost.
• Comparing candidate models with similar discrimination: if two models have similar AUC values, the Hosmer-Lemeshow Test (alongside a calibration plot) can help distinguish which one produces more trustworthy probability estimates.
• Continuous or mixed continuous/categorical predictors: the test is most commonly applied when at least one continuous predictor is present, since with purely categorical predictors and few covariate patterns, other goodness-of-fit approaches (like a Pearson chi-square on the covariate patterns themselves) may be more appropriate.

Key Assumptions

• A fitted binary logistic regression model: the test is applied to the predicted probabilities \( \hat{p}_i \) from an already-estimated logistic regression, not to raw data.
• Independent observations: each observation's outcome is assumed independent of the others, matching the assumptions of the underlying logistic regression itself.
• A meaningful, roughly equal-sized grouping: observations are grouped into \( g \) categories by predicted probability, usually deciles-see Choosing the Number of Groups for why this choice is not entirely innocuous.
• Reasonably large sample size: the \( \hat{C} \) statistic's chi-square approximation is asymptotic and works best with enough observations per group (a common rule of thumb is an expected count of at least 5 in most groups).
• No 100% concentration in a single covariate pattern: when many observations share identical predictor values (common with only categorical predictors), the decile grouping can become unstable or ill-defined, which is a known limitation discussed further under Limitations.

Hypotheses

The Hosmer-Lemeshow Test formally tests whether the logistic regression model fits the data adequately across the range of predicted risk:

• Null Hypothesis (\( H_0 \)): the model is well calibrated-observed event frequencies match the model's predicted probabilities across all risk groups (no evidence of lack of fit).
• Alternative Hypothesis (\( H_1 \)): the model is not well calibrated-observed and expected event counts differ systematically in at least one risk group.

(Note the direction: unlike many tests, here the null hypothesis is the "good" outcome-you generally want to fail to reject \( H_0 \), which is why the test's low power, discussed under Limitations, is such a persistent concern.)

The Formula, Explained

Start with a fitted logistic regression that produces predicted probabilities \( \hat{p}_i \) for each of \( n \) observations with binary outcome \( y_i \in \{0, 1\} \). The Hosmer-Lemeshow procedure follows four steps:

Step 1. Sort all observations by \( \hat{p}_i \) from lowest to highest and split them into \( g \) groups of roughly equal size (traditionally \( g = 10 \) deciles).

Step 2. Within each group \( k \), compute the observed number of events \( O_k = \sum y_i \) and the group size \( n_k \).

Step 3. Within each group \( k \), compute the expected number of events as the sum of predicted probabilities:

Step 4. Compute the test statistic across all groups:

where \( \bar{p}_k = E_k / n_k \) is the average predicted probability in group \( k \). Under \( H_0 \) (good fit), this statistic is approximately chi-square distributed:

The degrees of freedom are \( g - 2 \) rather than \( g - 1 \) because the grouping itself is estimated from the fitted probabilities, consuming one extra degree of freedom compared to a standard chi-square goodness-of-fit test. With the traditional \( g = 10 \) groups, this gives \( 8 \) degrees of freedom.

Choosing the Number of Groups

Hosmer and Lemeshow's original 1980 paper proposed \( g = 10 \) groups (deciles of risk) as a practical default, and it remains the standard choice reported by most software. But \( g \) is not derived from any underlying theory-it is a modeling choice, and that has consequences worth understanding before you trust a single p-value.

• Too few groups (e.g., \( g = 4 \)) can wash out real miscalibration by pooling observations with meaningfully different predicted risk into the same bucket.
• Too many groups can leave some groups with very few observations, violating the "expected count of at least 5" rule of thumb and destabilizing the chi-square approximation.
• Ties in predicted probability at group boundaries (common with a small number of categorical predictors) can force different software implementations to split ties differently, which can shift observations between groups and change the resulting statistic.
• Sensitivity to g is well documented: follow-up work, including a 1997 paper co-authored by Hosmer himself, showed that changing \( g \) can flip a test from non-significant to significant (or vice versa) on the very same fitted model and data.

The practical guidance: report which \( g \) you used (almost always state it explicitly, e.g., "Hosmer- Lemeshow test with 10 groups"), consider trying a nearby alternative like \( g = 8 \) as a sensitivity check, and never rely on the test statistic alone-always pair it with a calibration plot.

Worked Example 1: A Small Numerical Example by Hand

To keep hand calculation manageable, suppose a logistic regression has already been fit to 20 observations and produced predicted probabilities. Sorting by \( \hat{p}_i \) and splitting into \( g = 4 \) groups of 5 observations each gives the following deciles-of-risk table (using 4 groups instead of 10 purely to keep the by-hand arithmetic tractable):

Step 1: Compute each group's contribution

For Group 1: \( O_1 = 0 \), \( E_1 = 0.55 \), \( \bar{p}_1 = 0.110 \):

For Group 2: \( O_2 = 1 \), \( E_2 = 1.45 \), \( \bar{p}_2 = 0.290 \):

For Group 3: \( O_3 = 3 \), \( E_3 = 2.60 \), \( \bar{p}_3 = 0.520 \):

For Group 4: \( O_4 = 5 \), \( E_4 = 4.30 \), \( \bar{p}_4 = 0.860 \):

Step 2: Sum across groups

Step 3: Compare against the critical value

With \( g = 4 \) groups, the reference distribution is \( \chi^2_{g-2} = \chi^2_2 \). The critical value at \( \alpha = 0.05 \) is \( \chi^2_{2,0.05} = 5.991 \). Since \( \hat{C} = 1.757 < 5.991 \), we fail to reject \( H_0 \): there is no evidence of poor calibration-observed and expected event counts are reasonably close in every group.

Worked Example 2: Patient Readmission Risk

A hospital fits a logistic regression predicting 30-day readmission from age, prior admissions, and comorbidity count for 200 patients, then evaluates calibration using the standard \( g = 10 \) deciles of risk. The resulting table (grouped by predicted probability, lowest to highest, 20 patients per decile):

Step 1: Compute each group's contribution

Applying \( \dfrac{(O_k - E_k)^2}{n_k \bar{p}_k (1 - \bar{p}_k)} \) to every decile and summing (shown for the two largest contributors):

Decile 9: \( O_9 = 13 \), \( E_9 = 11.6 \), \( \bar{p}_9 = 0.580 \):

Decile 10: \( O_{10} = 16 \), \( E_{10} = 14.2 \), \( \bar{p}_{10} = 0.710 \):

Step 2: Sum across all ten deciles

Carrying out the same calculation for every decile and summing all ten contributions gives:

Step 3: Compare against the critical value

With \( g = 10 \) groups, the reference distribution is \( \chi^2_{g-2} = \chi^2_8 \). The critical value at \( \alpha = 0.05 \) is \( \chi^2_{8,0.05} = 15.507 \). Since \( \hat{C} = 2.94 < 15.507 \), we fail to reject \( H_0 \): the p-value here is approximately \( 0.94 \), indicating no evidence of poor calibration-the readmission model's predicted probabilities track the observed readmission rates closely across all ten risk deciles, from the lowest (7% predicted, 5% observed) to the highest (71% predicted, 80% observed).

Python Example

statsmodels does not ship a built-in Hosmer-Lemeshow function, but the statistic is short enough to compute directly from a fitted Logit model's predicted probabilities:

import numpy as np
import pandas as pd
import statsmodels.api as sm
from scipy.stats import chi2

def hosmer_lemeshow_test(y_true, y_prob, g=10):
    df = pd.DataFrame({"y": y_true, "p": y_prob})
    df["decile"] = pd.qcut(df["p"], q=g, duplicates="drop")

    grouped = df.groupby("decile", observed=True)
    obs = grouped["y"].sum()
    exp = grouped["p"].sum()
    n_k = grouped["y"].count()
    p_bar = exp / n_k

    contributions = (obs - exp) ** 2 / (n_k * p_bar * (1 - p_bar))
    c_hat = contributions.sum()

    dof = len(obs) - 2
    p_value = 1 - chi2.cdf(c_hat, dof)
    return c_hat, p_value, dof

# Fit a logistic regression
X = sm.add_constant(patient_df[["age", "prior_admissions", "comorbidity_count"]])
model = sm.Logit(patient_df["readmitted"], X).fit()

predicted_probs = model.predict(X)

c_hat, p_value, dof = hosmer_lemeshow_test(patient_df["readmitted"], predicted_probs, g=10)

print(f"Hosmer-Lemeshow C-hat: {c_hat:.3f}")
print(f"Degrees of freedom: {dof}")
print(f"p-value: {p_value:.4f}")

Output (approximate, matching Worked Example 2):

Hosmer-Lemeshow C-hat: 2.940
Degrees of freedom: 8
p-value: 0.9378

This matches Worked Example 2 closely. In R, the equivalent is the widely used hoslem.test() function from the ResourceSelection package, which implements the same deciles-of-risk logic.

Pairing the Test with a Calibration Plot

Because the Hosmer-Lemeshow p-value alone can be misleading, always visualize calibration alongside it:

import matplotlib.pyplot as plt

df = pd.DataFrame({"y": patient_df["readmitted"], "p": predicted_probs})
df["decile"] = pd.qcut(df["p"], q=10, duplicates="drop")
calib = df.groupby("decile", observed=True).agg(observed_rate=("y", "mean"),
                                                   predicted_rate=("p", "mean"))

plt.plot(calib["predicted_rate"], calib["observed_rate"], marker="o", label="Model")
plt.plot([0, 1], [0, 1], linestyle="--", color="gray", label="Perfect calibration")
plt.xlabel("Mean predicted probability")
plt.ylabel("Observed event rate")
plt.legend()
plt.title("Calibration plot")
plt.show()

How to Interpret Results

The significance level \( \alpha = 0.05 \) is the standard threshold, but interpretation here requires more care than most goodness-of-fit tests because of which direction is "good."

Checking Assumptions in Practice

• Always build a calibration plot alongside the test: plotting mean predicted probability against observed event rate per decile (as in the Python Example) reveals where miscalibration occurs, which the single \( \hat{C} \) statistic cannot show.
• Check the group sizes and expected counts: if several groups have an expected count \( E_k \) below about 5, the chi-square approximation weakens-consider reducing \( g \).
• Try a second value of \( g \) as a sensitivity check: if \( g = 10 \) and \( g = 8 \) give meaningfully different conclusions, treat the result cautiously rather than reporting a single run as definitive-see Choosing the Number of Groups.
• Watch for many tied covariate patterns: with mostly categorical predictors and few unique combinations of predictor values, deciles can become poorly defined; a Pearson chi-square test over the actual covariate patterns may be more appropriate in that case.
• Remember this tests calibration, not discrimination: a passing Hosmer-Lemeshow result says nothing about how well the model ranks high-risk versus low-risk cases-check the AUC/ROC curve separately for that.

What to Do If Poor Fit Is Detected

• Inspect the calibration plot first: identify which risk range is miscalibrated- overprediction at high risk and underprediction at low risk are common and point to different fixes.
• Add nonlinear terms for continuous predictors: a missing quadratic term or spline can create exactly this kind of localized miscalibration; the logit of the outcome may not be linear in a continuous predictor across its full range.
• Check for missing interaction terms: two predictors that jointly drive risk in a way neither does alone can produce miscalibration that a main-effects-only model misses.
• Consider recalibration: techniques like Platt scaling or isotonic regression can adjust an already-fit model's output probabilities to better match observed frequencies without re-estimating the underlying coefficients.
• Re-examine the link function: if the standard logit link consistently misfits, a different link (e.g., probit or complementary log-log) is occasionally a better match for the underlying process generating the data.

Advantages

• Directly addresses calibration-a property that discrimination metrics like AUC do not measure at all.
• Produces an interpretable deciles-of-risk table alongside the p-value, making it easy to communicate results to non-statistical stakeholders (e.g., clinicians, credit officers).
• Simple to compute-only requires the fitted predicted probabilities and observed outcomes, with no need to refit any additional model.
• Extremely widely used and reported in applied logistic regression work, especially in clinical prediction modeling, making results easy to compare across studies.
• Works for models with any combination of continuous and categorical predictors, unlike some covariate-pattern-based alternatives that struggle with many continuous variables.

Limitations

• Sensitive to the arbitrary choice of \( g \): the number of groups is a modeling decision, not derived from theory, and can change the conclusion-see Choosing the Number of Groups.
• Low statistical power in small-to-moderate samples, meaning real miscalibration can go undetected, especially with fewer than a few hundred observations.
• Sensitive to how ties are broken at decile boundaries, which can differ between software implementations and produce slightly different statistics on identical data.
• Only tests overall fit, not location-specific fit in a way that's easy to summarize-a small overall \( \hat{C} \) can still hide meaningful miscalibration in one particular decile that happens to be offset by another decile in the opposite direction.
• Does not assess discrimination at all-a model can pass the Hosmer-Lemeshow Test with near-chance discriminative ability, so it must be paired with metrics like the AUC, not used alone.

When NOT to Use It

• Le Cessie-van Houwelingen Test: use instead when you want a smoothing-based goodness-of-fit check that avoids the arbitrary group-count decision entirely.
• Pearson or deviance chi-square goodness-of-fit test: use instead when predictors are mostly categorical with a manageable number of distinct covariate patterns, since grouping by those actual patterns (rather than artificial deciles) is more natural and avoids the tie-breaking issue.
• Calibration plots and calibration slope/intercept: use as the primary tool-not just a supplement-when you need to see exactly where and how severely a model is miscalibrated, rather than a single summary p-value.
• AUC / ROC analysis: use instead (or in addition) when the question is about discrimination-how well the model ranks positive versus negative cases-rather than calibration.
• Very small samples: with only a few dozen observations, consider skipping formal goodness-of-fit testing altogether in favor of cross-validated calibration plots, since the chi-square approximation becomes unreliable.

Hosmer-Lemeshow vs Le Cessie-van Houwelingen vs Calibration Plots

17.1 Hosmer-Lemeshow Test vs Le Cessie-van Houwelingen Test

17.2 Hosmer-Lemeshow Test vs Calibration Plot

17.3 Hosmer-Lemeshow Test vs AUC / ROC Curve

Common Misconceptions

• "A non-significant Hosmer-Lemeshow result proves my model is well calibrated." Not true-failing to reject \( H_0 \) only means there wasn't enough evidence of poor fit with this sample and this choice of \( g \); the test's known low power means real miscalibration can slip through, as discussed in Limitations.
• "A high AUC means the Hosmer-Lemeshow Test will also pass." Not necessarily- discrimination and calibration are different properties; see Hosmer-Lemeshow vs AUC/ROC Curve for why a model can excel at one and fail the other.
• "The number of groups doesn't really matter." It can matter quite a bit-published research has shown the same fitted model can produce different Hosmer-Lemeshow conclusions under different choices of \( g \), as covered in Choosing the Number of Groups.
• "A significant result means the model is unusable." Not necessarily-it means calibration should be investigated and possibly corrected (see What to Do If Poor Fit Is Detected); the model's underlying discrimination may still be perfectly usable after recalibration.
• "The Hosmer-Lemeshow Test works the same for any type of regression." It does not-it was specifically designed for binary logistic regression; other outcome types need different goodness-of-fit approaches.

Interview Questions

1. Describe the four steps of the Hosmer-Lemeshow procedure, from sorting predicted probabilities to the final test statistic.
2. Why are the degrees of freedom \( g - 2 \) rather than \( g - 1 \) for the Hosmer-Lemeshow statistic?
3. Explain the difference between model calibration and model discrimination, and why a model can be strong on one but weak on the other.
4. What happens to the Hosmer-Lemeshow conclusion if you change the number of groups from 10 to 8? Why is this considered a weakness of the test?
5. Why is a non-significant Hosmer-Lemeshow p-value not sufficient evidence that a model is well calibrated?
6. How does the Le Cessie-van Houwelingen Test address the main criticism of the Hosmer-Lemeshow Test?
7. If a Hosmer-Lemeshow Test is significant, what practical remedies would you consider, and how would you choose between them?
8. Why might a model with an excellent AUC still fail the Hosmer-Lemeshow Test?
9. What role does a calibration plot play alongside the Hosmer-Lemeshow statistic, and why shouldn't you rely on the p-value alone?
10. Under what circumstances would you prefer a Pearson chi-square goodness-of-fit test over the Hosmer-Lemeshow Test?

Frequently Asked Questions

• What is the Hosmer-Lemeshow Test used for? +
  The Hosmer-Lemeshow Test evaluates goodness-of-fit for a logistic regression model by checking whether predicted probabilities are well calibrated against observed outcomes-for example, whether patients predicted to have a 70% chance of readmission are actually readmitted at close to a 70% rate, and similarly across the full range of predicted risk.
• What is the formula for the Hosmer-Lemeshow Test? +
  Sort all observations by their predicted probability and divide them into g groups, typically deciles (g = 10). Within each group, compute the observed number of events and the expected number (the sum of the predicted probabilities in that group). The test statistic C-hat sums, across all groups, the squared difference between observed and expected counts divided by an estimate of the variance in that group, and this statistic is compared to a chi-square distribution with g minus 2 degrees of freedom.
• How do you interpret a Hosmer-Lemeshow Test p-value? +
  If the p-value is below your chosen significance level (commonly alpha = 0.05), you reject the null hypothesis of good fit and conclude the model is poorly calibrated in at least some risk groups. If the p-value is at or above alpha, there is not enough evidence of poor fit-but because the test has known low power, a non-significant result should not be treated as strong proof that the model is well calibrated.
• How many groups should I use for the Hosmer-Lemeshow Test? +
  Ten groups (deciles of risk) is the classic default proposed by Hosmer and Lemeshow, giving 8 degrees of freedom, but the choice of g is somewhat arbitrary. Using fewer groups (e.g., g = 8 or g = 6) is sometimes recommended for smaller samples, and different choices of g can change whether the test is statistically significant, which is one of its most frequently criticized properties.
• Why is the Hosmer-Lemeshow Test considered controversial? +
  Multiple simulation studies, including follow-up work by Hosmer himself, have shown the test is sensitive to the number of groups chosen and to how tied predicted probabilities are handled at group boundaries, has relatively low power to detect real miscalibration (especially in smaller samples), and can pass models that show visibly poor calibration in a calibration plot-limitations that have pushed many statisticians toward calibration plots or the Le Cessie-van Houwelingen test instead.
• What is the difference between the Hosmer-Lemeshow Test and the Le Cessie-van Houwelingen Test? +
  The Hosmer-Lemeshow Test relies on an arbitrary grouping of observations into deciles of risk, while the Le Cessie-van Houwelingen Test avoids grouping entirely by using a smoothing-based (kernel) approach to compare observed and predicted values, making it less sensitive to the group-count problem and generally considered more statistically robust, though it is less widely implemented in standard software.
• Does a significant Hosmer-Lemeshow result mean my model is useless? +
  No-goodness-of-fit and discrimination are different properties. A model can have excellent discrimination (a high AUC, correctly ranking who is more or less likely to have the event) while still failing the Hosmer-Lemeshow Test due to miscalibration in specific risk ranges, which is often fixable through recalibration or by adding missing nonlinear terms rather than discarding the model entirely.

Key Takeaways

• The Hosmer-Lemeshow Test checks whether a logistic regression's predicted probabilities are well calibrated against observed event rates-a different question from discrimination.
• It works by grouping observations into deciles of predicted risk and comparing observed versus expected event counts using \( \hat{C} = \sum (O_k - E_k)^2 / [n_k \bar{p}_k(1-\bar{p}_k)] \), compared against a \( \chi^2_{g-2} \) distribution.
• The choice of \( g \) (commonly 10) is somewhat arbitrary and can change the conclusion-always report it and consider a sensitivity check, as covered in Choosing the Number of Groups.
• A non-significant result is a necessary but not sufficient reassurance of good calibration, given the test's documented low power.
• Always pair the test with a calibration plot, which shows exactly where predicted and observed probabilities diverge.
• If poor fit is detected, the simplest fixes are usually adding nonlinear or interaction terms, recalibrating the output probabilities, or reconsidering the link function, as outlined in What to Do If Poor Fit Is Detected.
• If you want a more robust alternative that avoids the group-count problem entirely, consider the Le Cessie-van Houwelingen Test.

The Hosmer-Lemeshow Test remains the most widely reported goodness-of-fit check for logistic regression because it answers a question discrimination metrics simply cannot: are the predicted probabilities themselves trustworthy, not just their ranking? By comparing observed and expected event counts across deciles of risk, the test gives a concrete, communicable summary of calibration that has become standard practice in clinical prediction modeling, credit scoring, and beyond.

The two worked examples above showed the same underlying logic at two scales: a small, hand-calculated four-group example and a realistic ten-decile patient readmission model, both of which passed the test with observed and expected counts tracking closely. But as the discussion of group count sensitivity and limitations makes clear, the Hosmer-Lemeshow statistic should never be reported alone-pairing it with a calibration plot, and being explicit about the number of groups used, gives a far more complete and defensible picture of how well a logistic regression model actually fits.