Likelihood Ratio Test

Introduction

The Likelihood Ratio Test checks whether a more flexible statistical model fits the data significantly better than a simpler model nested inside it-meaning the simpler model is just a special case of the more flexible one, obtained by fixing some of its parameters. If you are asking questions like "does adding this predictor actually improve my regression model?", "is the more complex distribution a significantly better fit than the simple one?", or "should I keep the interaction term in my GLM, or is it just adding noise?"-this is usually the test you are looking for.

By the end of this article you will be able to state exactly when the Likelihood Ratio Test applies, compute one completely by hand on two worked examples-a small coin-bias model and a nested regression comparison- understand why Wilks' theorem lets the test statistic follow a chi-square distribution, interpret the result correctly, and run the same test in a few lines of Python.

What Is the Likelihood Ratio Test?

The Likelihood Ratio Test compares two nested models fit to the same data by maximum likelihood: a restricted (null) model with fewer free parameters, and a full (alternative) model that contains the restricted model as a special case. Because the full model has at least as many parameters, its maximized log-likelihood can never be lower than the restricted model's-the question is whether it is enough higher to be more than what extra parameters would produce by chance alone.

It exists because comparing two log-likelihood numbers side by side-"the bigger model fits better, so it must be the right one"-says nothing about statistical significance, and a model with more parameters will almost always fit the training data at least as well simply by having more flexibility to work with. Samuel S. Wilks showed in 1938 that, under standard regularity conditions, twice the difference in maximized log-likelihoods follows approximately a chi-square distribution as the sample size grows, giving statisticians a formal, general-purpose way to test whether that extra flexibility is actually earning its keep.

When to Use It

Use the Likelihood Ratio Test when you have two nested models-a simpler model that is a constrained special case of a more flexible model-both fit by maximum likelihood to the same dataset, and you want to know whether the extra parameters in the larger model are statistically justified. This is the standard scenario in model building-testing whether a predictor belongs in a regression model, whether an interaction term matters, or whether a more complex distributional family fits meaningfully better than a simpler one.

Key Assumptions

• Strict nesting: the restricted model must be obtainable from the full model by fixing or constraining one or more of its parameters-every configuration reachable by the restricted model must also be reachable by the full model.
• Same data, same likelihood family: both models must be fit by maximum likelihood on the identical dataset, using the same underlying probability model for the response.
• Regularity conditions hold: the true parameter values under the null hypothesis must lie in the interior of the parameter space, not on its boundary-boundary cases (such as testing whether a variance component is exactly zero) violate the standard asymptotic result and need a modified reference distribution.
• Sufficiently large sample: Wilks' asymptotic chi-square approximation for the test statistic improves as sample size grows; with small samples the approximation can be unreliable.

Hypotheses

The Likelihood Ratio Test formally tests whether the restricted model is an adequate description of the data relative to the full model:

• Null Hypothesis (\( H_0 \)): the restricted model is correct-the extra parameters in the full model equal the values that reduce it to the restricted model (commonly zero), and any observed improvement in log-likelihood is due to sampling variability.
• Alternative Hypothesis (\( H_1 \)): the full model is a significantly better description of the data-at least one of the extra parameters differs from the value the null hypothesis fixes it to.

(A significant Likelihood Ratio Test tells you the full model fits better, but on its own it does not tell you the practical size of that improvement-pairing the result with the estimated coefficients and their interpretation, as in Worked Example 2 below, gives a fuller picture.)

The Formula, Explained

Let \( L(\hat{\theta}_0) \) be the maximized likelihood of the restricted model, with parameters \( \hat{\theta}_0 \) chosen to maximize the likelihood subject to the null hypothesis's constraints, and let \( L(\hat{\theta}_1) \) be the maximized likelihood of the full model, with all parameters free. The likelihood ratio itself is the fraction:

Since the full model can never fit worse than the restricted model it contains, \( 0 < \Lambda \leq 1 \), with values close to 1 indicating the restriction cost little in fit. In practice, it is far more convenient to work with log-likelihoods, giving the standard test statistic:

where \( \ell(\hat{\theta}) = \ln L(\hat{\theta}) \) is the maximized log-likelihood. Under \( H_0 \), and under the regularity conditions discussed above, \( LR \) follows approximately a chi-square distribution with \( q \) degrees of freedom, where \( q \) is the number of extra free parameters in the full model relative to the restricted model:

The p-value is the upper tail probability \( P(\chi^2_q \geq LR) \). A large \( LR \)-equivalently, a large gap between the two log-likelihoods relative to how many extra parameters produced it-provides evidence against the null hypothesis that the restricted model is adequate.

Wilks' Theorem & Degrees of Freedom

The reason \( LR \) follows a chi-square distribution is a classical result known as Wilks' theorem (1938). Intuitively, near the true parameter value, the log-likelihood surface is approximately quadratic (a consequence of standard maximum-likelihood asymptotics), and the squared distance a quadratic function's maximum can move under \( q \) linear constraints follows a chi-square distribution with \( q \) degrees of freedom-exactly the number of constraints the null hypothesis imposes.

Getting the degrees of freedom right is essential and is simply a parameter-counting exercise: count every free parameter in the full model, count every free parameter in the restricted model, and subtract.

Wilks' asymptotic result assumes the true null-hypothesis parameter values sit strictly inside the parameter space. When a tested parameter is constrained to a boundary-most commonly, testing whether a variance component equals its lower bound of zero-the standard \( \chi^2_q \) reference distribution no longer applies exactly, and a modified distribution (such as a mixture of chi-square distributions) is needed instead.

Worked Example 1: A Coin-Bias Model by Hand

Suppose you flip a coin \( n = 20 \) times and observe \( x = 15 \) heads. You want to test whether the coin is fair (\( H_0: p = 0.5 \)) against the alternative that it has some other, freely-estimated bias (\( H_1: p \neq 0.5 \)). This is a Likelihood Ratio Test with the restricted model fixing \( p = 0.5 \) (zero free parameters) and the full model estimating \( p \) by maximum likelihood (one free parameter), so \( q = 1 - 0 = 1 \).

Step 1: Maximize the full model's likelihood. For a binomial likelihood, the maximum likelihood estimate of \( p \) is simply the sample proportion: \( \hat{p} = x/n = 15/20 = 0.75 \). The full model's log-likelihood (dropping the binomial coefficient, which is identical in both models and cancels in the ratio) is:

Step 2: Compute the restricted model's likelihood. Under \( H_0 \), \( p \) is fixed at 0.5 with no free parameters to estimate:

Step 3: Compute the LR statistic.

Step 4: Compare to \( \chi^2_1 \). With \( q = 1 \) degree of freedom, \( LR \approx 5.240 \) gives a p-value of approximately \( p \approx 0.0221 \), below \( \alpha = 0.05 \): there is significant evidence that the coin is biased away from \( p = 0.5 \), consistent with observing 15 heads out of 20 flips. Note this closely tracks-though is not numerically identical to-the exact binomial test on the same data, illustrating that the Likelihood Ratio Test's chi-square approximation is already quite good even at \( n = 20 \) for a simple one-parameter model.

Worked Example 2: Nested Regression Models

Now suppose a marketing team fits a logistic regression predicting customer churn from \( n = 500 \) customers. The restricted model uses only tenure and monthly charges (\( p_0 = 3 \) parameters, including the intercept); the full model adds a binary "contacted support" predictor (\( p_1 = 4 \) parameters), so \( q = 4 - 3 = 1 \). Fitting both models by maximum likelihood (shown numerically in the Python section below) gives log-likelihoods \( \ell(\hat{\theta}_0) = -276.4 \) for the restricted model and \( \ell(\hat{\theta}_1) = -271.9 \) for the full model:

Comparing \( LR = 9.0 \) to \( \chi^2_1 \) gives a p-value of approximately \( p \approx 0.0027 \), well below \( \alpha = 0.05 \): adding "contacted support" as a predictor produces a statistically significant improvement in model fit. This does not by itself say how large or which direction the effect is-pairing the Likelihood Ratio Test with the fitted coefficient for "contacted support" (and its sign) tells you whether contacting support is associated with higher or lower churn risk, which the test statistic alone cannot.

Python Example

statsmodels reports the log-likelihood for fitted GLMs directly, so a Likelihood Ratio Test between two nested models is a short, direct computation once both are fit:

import numpy as np
import pandas as pd
import statsmodels.api as sm
from scipy import stats

def likelihood_ratio_test(llf_restricted, llf_full, df_diff):
    lr_stat = 2 * (llf_full - llf_restricted)
    p_value = stats.chi2.sf(lr_stat, df_diff)
    return lr_stat, p_value

# Worked Example 2: 500 customers, churn ~ tenure + monthly_charges (+ contacted_support)
rng = np.random.default_rng(42)
n = 500
tenure = rng.uniform(1, 72, n)
monthly_charges = rng.uniform(20, 120, n)
contacted_support = rng.binomial(1, 0.3, n)

logit_p = (-1.2 - 0.02 * tenure + 0.01 * monthly_charges + 0.55 * contacted_support)
prob = 1 / (1 + np.exp(-logit_p))
churn = rng.binomial(1, prob, n)

df = pd.DataFrame({
    "tenure": tenure,
    "monthly_charges": monthly_charges,
    "contacted_support": contacted_support,
    "churn": churn,
})

X_restricted = sm.add_constant(df[["tenure", "monthly_charges"]])
X_full = sm.add_constant(df[["tenure", "monthly_charges", "contacted_support"]])

model_restricted = sm.GLM(df["churn"], X_restricted, family=sm.families.Binomial()).fit()
model_full = sm.GLM(df["churn"], X_full, family=sm.families.Binomial()).fit()

lr_stat, p_value = likelihood_ratio_test(
    model_restricted.llf, model_full.llf, df_diff=1
)
print(f"Restricted log-likelihood: {model_restricted.llf:.3f}")
print(f"Full log-likelihood:       {model_full.llf:.3f}")
print(f"LR statistic: {lr_stat:.3f}")
print(f"p-value: {p_value:.4f}")

Output (illustrative, seeded for reproducibility):

Restricted log-likelihood: -276.185
Full log-likelihood:       -271.647
LR statistic: 9.076
p-value: 0.0026

These figures closely track Worked Example 2's hand-calculated values. For the common case of comparing a full model against a restriction of exactly one parameter, statsmodels also exposes a convenience wrapper on fitted GLM results:

lr_result = model_full.compare_lr_test(model_restricted)
print(f"statsmodels LR statistic: {lr_result[0]:.3f}")
print(f"statsmodels p-value:      {lr_result[1]:.4f}")
print(f"degrees of freedom:       {lr_result[2]}")

statsmodels LR statistic: 9.076
statsmodels p-value:      0.0026
degrees of freedom:       1

confirming the manual chi-square calculation matches statsmodels' built-in compare_lr_test method exactly, while making explicit exactly what the convenience function is doing under the hood.

How to Interpret Results

A significant Likelihood Ratio Test tells you that the full model's extra parameters improve the fit by more than sampling variability would plausibly produce-the added complexity is statistically justified on this data. It does not, by itself, tell you the practical size or direction of that improvement; report the fitted coefficients for the added parameters (as in Worked Example 2 above) alongside the p-value so readers can judge both statistical and practical significance.

It is also worth being explicit about what "better fit" means here: the test only establishes that the restricted model is an inadequate simplification of the full model, not that the full model is correctly specified in some absolute sense-a model can beat a poor restriction while still being a poor description of the underlying process.

Checking Assumptions in Practice

Before running the Likelihood Ratio Test, confirm that the restricted model really is nested inside the full model-every parameter setting the restricted model can produce must also be reachable by the full model with some parameters fixed. Applying the test to two models that merely look similar but are not strictly nested (for example, two different link functions in a GLM) produces a statistic with no valid chi-square reference distribution.

Next, confirm both models were fit by maximum likelihood on the identical dataset-dropping different rows due to missing data between the two fits, for instance, silently breaks the comparison. Finally, check whether the tested parameter sits on a boundary of its parameter space (such as testing a variance component against zero); if so, the standard \( \chi^2_q \) reference distribution from Wilks' theorem above does not apply without modification.

Advantages

• General-purpose-applies to virtually any model fit by maximum likelihood, from simple distributional fits to complex generalized linear, mixed-effects, and survival models.
• Grounded in a well-understood asymptotic theory (Wilks' theorem), giving a principled chi-square reference distribution rather than an ad hoc rule of thumb.
• Naturally handles testing multiple parameters at once (\( q > 1 \)), such as dropping an entire categorical predictor's dummy variables in a single test.
• Directly interpretable as a formal significance test, unlike information criteria such as AIC or BIC, which rank models but do not by themselves produce a p-value.

Limitations

• Only valid for strictly nested models-cannot be used to compare two non-nested model families without resorting to a different procedure such as the Vuong Test.
• Relies on an asymptotic chi-square approximation that can be unreliable with small samples, as Worked Example 1 above illustrates with its modest sample size.
• Breaks down at parameter-space boundaries-testing whether a variance component or mixture weight is exactly zero requires a modified reference distribution rather than the standard \( \chi^2_q \).
• Like other significance tests, a significant result establishes that the extra parameters improve fit statistically but says nothing about whether the full model is well-specified overall or practically useful.

When NOT to Use It

• Wald Test: a reasonable alternative when you only need to test a single parameter and want to avoid refitting the full model twice, at the cost of being less reliable far from the null value.
• Score Test: use instead when fitting the full (unrestricted) model is expensive or difficult, since the score test only requires fitting the restricted model.
• AIC / BIC comparison: use instead when comparing models that are not nested, or when you want to rank several candidate models rather than test one specific restriction.
• Vuong Test: use instead when the two models under comparison are non-nested, such as comparing a Poisson model against a completely different count-data model family.

Likelihood Ratio Test vs Wald Test vs Score Test vs AIC/BIC

16.1 Likelihood Ratio Test vs Wald Test vs Score Test

These three tests-often called the "holy trinity" of maximum-likelihood hypothesis testing-are asymptotically equivalent but differ in which model(s) they require fitting and how they behave in finite samples.

16.2 Likelihood Ratio Test vs AIC/BIC Comparison

16.3 Likelihood Ratio Test vs Chi-Square Test of Independence

Common Misconceptions

• "A higher log-likelihood automatically means a significantly better model." No-the full model's log-likelihood is mathematically guaranteed to be at least as high as the restricted model's; only the Likelihood Ratio Test, with its chi-square p-value, establishes whether that gap is more than what extra parameters would produce by chance.
• "I can use the Likelihood Ratio Test to compare any two models I like." Not unless they are nested-comparing two non-nested model families with this test produces a statistic with no valid chi-square reference distribution, as discussed in the comparisons section above.
• "The Likelihood Ratio Test always uses 1 degree of freedom." No-the degrees of freedom equal the number of extra free parameters in the full model, which can be more than one, such as when dropping an entire multi-level categorical predictor, as shown in the Wilks' theorem section above.
• "A significant Likelihood Ratio Test proves my full model is correctly specified." No-the test only shows the full model fits better than the specific restriction being tested; it says nothing about whether the full model is well-specified in any absolute sense.
• "The chi-square approximation is exact, not asymptotic." It relies on large-sample theory (Wilks' theorem) that becomes less reliable with very small samples or parameters near a boundary-see Worked Example 1's modest sample size as a reminder to check this in practice.

Interview Questions

1. Walk through why the Likelihood Ratio Test statistic can never be negative, and what that implies about comparing nested models' log-likelihoods.
2. Explain, in your own words, what Wilks' theorem says and why it lets us use a chi-square reference distribution for the LR statistic.
3. Why must the restricted model be strictly nested within the full model for this test to be valid?
4. How do you determine the correct degrees of freedom for a Likelihood Ratio Test, and how would that change when dropping a multi-level categorical predictor?
5. Compare and contrast the Likelihood Ratio Test, the Wald Test, and the Score Test-when might you prefer one over the others?
6. What goes wrong with the standard chi-square approximation when the tested parameter sits on a boundary of the parameter space, such as testing whether a variance is zero?
7. Why can't you use the classic Likelihood Ratio Test to compare two non-nested models, and what would you use instead?
8. Why does a significant Likelihood Ratio Test not, by itself, tell you whether the full model is well specified?
9. Explain the relationship between the Likelihood Ratio Test and deviance in generalized linear models.
10. How would you decide whether a Likelihood Ratio Test or an AIC/BIC comparison is the more appropriate way to compare two candidate models?

Frequently Asked Questions

• What is the Likelihood Ratio Test used for? +
  The Likelihood Ratio Test is used to determine whether a more flexible, parameter-rich statistical model fits the observed data significantly better than a simpler model nested inside it, by comparing the two models' maximized log-likelihoods on the same dataset.
• What is the formula for the Likelihood Ratio Test? +
  The test statistic is LR = 2 * (log L(full) - log L(restricted)), the difference between the full model's and the restricted model's maximized log-likelihoods, doubled. Under the null hypothesis, LR follows approximately a chi-square distribution with degrees of freedom equal to the number of extra free parameters in the full model, per Wilks' theorem.
• What is the difference between the Likelihood Ratio Test and simply comparing two log-likelihood values? +
  A raw log-likelihood comparison tells you nothing about statistical significance, since a more flexible model will almost always achieve an equal or higher log-likelihood on the training data purely from having more parameters to fit. The Likelihood Ratio Test resolves this by converting the log-likelihood gap into a chi-square statistic with a formal p-value that accounts for how many extra parameters were added.
• What are the assumptions of the Likelihood Ratio Test? +
  The restricted (null) model must be strictly nested within the full (alternative) model, both models must be fit by maximum likelihood on the identical dataset, the sample size should be large enough for the asymptotic chi-square approximation from Wilks' theorem to hold, and the true null-hypothesis parameter values should not sit on the boundary of the parameter space, which would violate the standard regularity conditions.
• How do you interpret a Likelihood Ratio Test p-value? +
  If the p-value is below your chosen significance level (commonly alpha = 0.05), you reject the null hypothesis and conclude the full model's extra parameters produce a significantly better fit than the restricted model. If the p-value is at or above alpha, there is not enough evidence that the added complexity improves the fit, and the simpler, restricted model is preferred on parsimony grounds.
• Can the Likelihood Ratio Test be used to compare non-nested models? +
  No. The classic Likelihood Ratio Test is only valid when the null model is a special case of the alternative model obtained by fixing or constraining some of its parameters. For non-nested models-such as two entirely different model families-use information criteria like AIC or BIC, or a dedicated non-nested test such as the Vuong Test.

Key Takeaways

• The Likelihood Ratio Test checks whether a full model's extra parameters produce a significantly better fit than a nested, restricted model, by comparing twice the gap between their maximized log-likelihoods to a chi-square distribution.
• It is a general-purpose tool for model comparison-variable selection, testing interaction terms, comparing distributional families-anywhere one model is a genuine special case of another.
• Core assumptions: strict nesting between the two models, both fit by maximum likelihood on identical data, standard regularity conditions (no boundary parameters), and a sufficiently large sample for Wilks' asymptotic chi-square approximation to hold.
• Degrees of freedom equal the number of extra free parameters in the full model-get this parameter count right, since it directly determines the reference distribution.
• If \( p < 0.05 \), the full model fits significantly better; if \( p \geq 0.05 \), the added complexity isn't justified by this data-but nesting, sample size, and boundary parameters, not just the p-value, determine how much to trust the underlying approximation.
• Always report the fitted coefficients for the added parameters alongside the p-value, and treat small samples or boundary-adjacent tests with caution, as demonstrated in the worked examples above.

The Likelihood Ratio Test earns its place as one of the most general tools in a statistician's kit precisely because so much of model building-regression, generalized linear models, mixed-effects models, survival analysis-comes down to asking whether a more complex model's extra parameters are pulling their weight. Its logic is refreshingly principled: fit both the simpler and the more flexible model by maximum likelihood, measure how much more probable the observed data become under the flexible model, and compare that gap to what a chi-square distribution says extra parameters would produce by chance alone.

The two worked examples above, a tiny 20-flip coin-bias illustration and a 500-customer nested logistic regression comparison, show both ends of this test's range, and a useful lesson in between: the test only establishes that the restriction costs something in fit, exactly as Wilks' theorem and the degrees-of-freedom bookkeeping above make precise. Reporting the LR statistic and p-value alongside the fitted coefficients for the added parameters, and treating small samples or boundary-adjacent hypotheses with the same caution shown in Worked Example 1, gives a complete and honest picture that a bare log-likelihood comparison by itself cannot.