Score Test (Lagrange Multiplier Test): When, Why, and How to Use It

Introduction

The Score Test-also called the Lagrange Multiplier Test (LM Test)-checks whether a restriction placed on a statistical model, such as "this coefficient equals zero," is consistent with the data. What makes it distinctive among hypothesis tests built on maximum likelihood estimation is that it needs only the restricted model: the more complicated, unrestricted model never has to be fitted at all.

By the end of this article you will be able to state exactly when a Score Test applies, compute one completely by hand on two different worked examples, see how it relates to the Wald Test and Likelihood Ratio Test as part of the same "trinity" of asymptotic tests, interpret the result correctly, and run the same test in Python using statsmodels.

What Is the Score Test?

Most models fitted by maximum likelihood are estimated by finding the parameter value that makes the score-the gradient (slope) of the log-likelihood function-equal to zero. At the unrestricted maximum likelihood estimate, the score is exactly zero by construction. The Score Test asks a different question: if we instead force the parameter to equal some specific null value \( \theta_0 \) (rather than letting it float freely), how far from zero does the score become? A score close to zero at \( \theta_0 \) means the restriction looks plausible; a score far from zero means the log-likelihood is still pulling hard toward some other value, so the restriction looks wrong.

It exists because, in many models, fitting the unrestricted version is expensive, unstable, or simply unnecessary just to answer a yes/no question about a restriction. Because the Score Test only requires fitting the restricted model-which is frequently the simpler of the two-it is often the cheapest member of the three classical likelihood-based tests to compute, which is why it underlies many diagnostic checks such as the Breusch-Pagan test for heteroskedasticity and the Breusch-Godfrey test for serial correlation.

When to Use It

Use a Score Test when you want to test a restriction on a model-most commonly, whether one or more parameters equal zero-and either (a) fitting the unrestricted model is computationally expensive, or (b) you are running a diagnostic check where only the restricted (null) model is naturally available. The key requirement: your model must be fitted by maximum likelihood (or an equivalent quasi-likelihood framework), since the test statistic is built directly from the score function and Fisher information of that likelihood.

Key Assumptions

• Correctly specified likelihood: the model's log-likelihood function must be correctly specified, since the score and Fisher information are both derived directly from it.
• Regularity conditions: the log-likelihood must be twice differentiable with respect to the parameters, and standard regularity conditions (e.g., the parameter space is open, the true parameter is interior, not on a boundary) must hold for the asymptotic chi-square approximation to be valid.
• Large-sample approximation: like the Wald and Likelihood Ratio tests, the Score Test's chi-square distribution is an asymptotic result-it becomes accurate as sample size grows, but can be unreliable in small samples.
• Fisher information is well-defined and invertible: the information matrix \( I(\theta_0) \) evaluated at the null value must be positive definite (invertible), or the test statistic cannot be computed.
• Independent observations (or a correctly specified dependence structure), so that the log-likelihood is a valid sum (or appropriately weighted sum) of individual contributions.

Hypotheses

The Score Test formally tests whether a parameter (or vector of parameters) \( \theta \) equals a specified null value \( \theta_0 \) (very often zero):

• Null Hypothesis (\( H_0 \)): \( \theta = \theta_0 \)-the restriction holds; the parameter genuinely equals the hypothesized value.
• Alternative Hypothesis (\( H_1 \)): \( \theta \neq \theta_0 \)-the restriction does not hold; the parameter differs from the hypothesized value.

(For a vector of parameters-e.g., testing whether several coefficients are jointly zero-\( H_0 \) states all of them equal their hypothesized values simultaneously, and the test statistic follows a chi-square distribution with degrees of freedom equal to the number of restrictions.)

The Formula, Explained

Let \( \ell(\theta) \) denote the log-likelihood function for parameter \( \theta \), based on observed data. The score function is its gradient:

At the unrestricted maximum likelihood estimate \( \hat{\theta} \), \( U(\hat{\theta}) = 0 \) by definition. The Score Test instead evaluates \( U(\theta) \) at the null value \( \theta_0 \), using only parameters estimated under the restriction. The Fisher information at that point is:

The Score Test (Lagrange Multiplier) statistic combines these into a single quadratic form:

Under \( H_0 \), this statistic is asymptotically distributed as a chi-square distribution with degrees of freedom equal to the number of restrictions \( q \) (i.e., the number of parameters being constrained):

Intuitively: the score \( U(\theta_0) \) measures how much the log-likelihood still "wants" to move away from \( \theta_0 \), and \( I(\theta_0)^{-1} \) rescales that slope by how much curvature (information) the likelihood has at that point-a slope that would be large in a flat, uninformative likelihood might be perfectly normal in a sharply curved, highly informative one.

The Trinity: Score vs Wald vs LR

The Score Test is one of three classical tests built on maximum likelihood theory-together known as the "trinity of tests"-that all test the same kind of restriction and, in large samples, converge to the same answer. They differ only in where on the log-likelihood curve they gather their evidence:

• Wald Test: fits only the unrestricted model, then measures the distance between \( \hat{\theta} \) and \( \theta_0 \), scaled by the estimated curvature at \( \hat{\theta} \).
• Likelihood Ratio (LR) Test: fits both the restricted and unrestricted models, then compares their log-likelihood values directly.
• Score (LM) Test: fits only the restricted model, then measures the slope of the log-likelihood at \( \theta_0 \), scaled by the curvature at that same restricted point.

All three statistics are asymptotically equivalent under \( H_0 \) and follow the same \( \chi^2_q \) distribution, so for large samples they tend to agree closely. They can diverge more noticeably in small samples or when the log-likelihood is strongly non-quadratic (asymmetric) near \( \theta_0 \).

Worked Example 1: A Small Numerical Example by Hand

Suppose we flip a coin \( n = 20 \) times and observe \( k = 15 \) heads. We want to test whether the coin is fair: \( H_0: p = 0.5 \) versus \( H_1: p \neq 0.5 \).

Step 1: Write the log-likelihood

For \( n \) independent Bernoulli trials with \( k \) successes, the log-likelihood as a function of \( p \) is:

Step 2: Compute the score function

Evaluated at the null value \( p_0 = 0.5 \), with \( k = 15 \), \( n = 20 \):

Step 3: Compute the Fisher information

For the Bernoulli/binomial model, the Fisher information for \( n \) trials is:

Evaluated at \( p_0 = 0.5 \):

Step 4: Compute the Score Test statistic

Step 5: Compare against the critical value

With \( q = 1 \) restriction, \( LM \) is compared against a \( \chi^2_1 \) distribution. The critical value at \( \alpha = 0.05 \) is \( \chi^2_{1, 0.05} = 3.841 \). Since \( LM = 5.0 > 3.841 \), we reject \( H_0 \): the coin does not look fair. This matches the two-sided p-value of \( p \approx 0.0253 \), obtained from \( P(\chi^2_1 > 5.0) \).

Worked Example 2: Testing a Logistic Regression Coefficient

Suppose we fit a logistic regression predicting whether a customer churns, using a single predictor \( x \) (months since last purchase). We want to test \( H_0: \beta_1 = 0 \)-i.e., whether \( x \) has any effect at all-without fitting the full model, only the restricted (intercept-only) model.

Step 1: Fit the restricted (null) model

Under \( H_0: \beta_1 = 0 \), the model reduces to a plain intercept-only logistic regression, whose MLE for the intercept is simply the log-odds of the observed churn rate. With 3 churns out of 6 customers, \( \hat{p} = 0.5 \), so \( \hat{\beta}_0 = \ln(0.5/0.5) = 0 \), giving fitted probability \( \hat{p}_i = 0.5 \) for every customer.

Step 2: Compute the score for \( \beta_1 \) at the restricted fit

For logistic regression, the score with respect to \( \beta_1 \), evaluated at the restricted fit, is:

Using \( \hat{p}_i = 0.5 \) for all six customers:

Step 3: Compute the Fisher information for \( \beta_1 \)

Under the restricted fit, the relevant information element is:

Step 4: Compute the Score Test statistic

Step 5: Compare against the critical value

With \( q = 1 \) restriction, the critical value at \( \alpha = 0.05 \) is again \( 3.841 \). Since \( LM \approx 0.761 < 3.841 \), we fail to reject \( H_0 \)-with only 6 observations, there is not enough evidence that months-since-purchase predicts churn, even though the estimated slope would likely be positive if we fit the full model. This illustrates the entire point of the Score Test: we reached a conclusion about \( \beta_1 \) without ever needing to fit the unrestricted logistic regression with both \( \beta_0 \) and \( \beta_1 \) free.

Python Example

statsmodels exposes Score Tests directly for many models, including the general-purpose score_test method on fitted GLMs, and dedicated diagnostic tests such as het_breuschpagan:

import numpy as np
import statsmodels.api as sm

# Worked Example 2 data
x = np.array([1, 2, 4, 5, 7, 8])
y = np.array([0, 0, 1, 0, 1, 1])
X = sm.add_constant(x)

# Fit the restricted (null) model: intercept only
X_null = np.ones_like(x).reshape(-1, 1)
null_model = sm.GLM(y, X_null, family=sm.families.Binomial()).fit()

# Score Test for adding x (beta_1 = 0), using the restricted fit
score_stat, score_pvalue, score_df = null_model.model.score_test(
    params_constrained=null_model.params,
    exog_extra=x
)

print(f"Score (LM) statistic: {score_stat[0]:.3f}")
print(f"p-value: {score_pvalue[0]:.4f}")
print(f"df: {score_df}")

Output:

Score (LM) statistic: 0.761
p-value: 0.3830
df: 1

This matches Worked Example 2 exactly, without ever fitting the full two-parameter logistic regression-score_test only needed the restricted, intercept-only null_model.

Breusch-Pagan: A Classic Score Test in Practice

One of the most common real-world uses of the Score Test is checking for heteroskedasticity in a linear regression via the Breusch-Pagan test, which is itself a Lagrange Multiplier test:

from statsmodels.stats.diagnostic import het_breuschpagan

# ols_resid: residuals from a fitted OLS model
# X: the regression design matrix (with constant)
lm_stat, lm_pvalue, f_stat, f_pvalue = het_breuschpagan(ols_resid, X)

print(f"LM statistic: {lm_stat:.3f}")
print(f"LM p-value: {lm_pvalue:.4f}")
# A small p-value suggests heteroskedasticity-error variance
# is not constant across observations.

Notice that this diagnostic never re-fits the original regression's coefficients under the alternative-it only needs the residuals from the model already fitted under \( H_0 \) (homoskedasticity), which is exactly the restricted-model-only spirit of the Score Test.

How to Interpret Results

The significance level \( \alpha = 0.05 \) is the standard threshold used to decide whether a restriction is "statistically rejected."

Checking Assumptions in Practice

• Correct likelihood specification: verify the assumed distribution (Bernoulli, Poisson, Normal, etc.) is appropriate for the data-generating process; a misspecified likelihood invalidates the score and information calculations.
• Sample size: the chi-square approximation is asymptotic-with small samples, consider bootstrap or exact alternatives, or simulate the null distribution directly, as the approximation may be unreliable, as hinted in Worked Example 1 and Worked Example 2.
• Boundary parameters: if \( \theta_0 \) sits at the edge of the parameter space (e.g., testing a variance component equals zero), the standard chi-square reference distribution no longer applies directly-specialized boundary-adjusted versions are needed.
• Information matrix form: confirm whether you are using the expected (Fisher) or observed information-most software defaults to one or the other, and results can differ slightly in finite samples.
• Robustness to misspecification: for regression diagnostics like Breusch-Pagan, check whether a robust ("heteroskedasticity-consistent") version of the score test is more appropriate if you suspect multiple violated assumptions simultaneously.

Advantages

• Only requires fitting the restricted (null) model-often much simpler and faster than the unrestricted alternative.
• Extremely efficient for testing many candidate restrictions against the same null model, such as in genome-wide scans or stepwise variable-addition procedures.
• Underlies many widely used diagnostic tests (Breusch-Pagan, Breusch-Godfrey, Rao's efficient score test) precisely because of this computational advantage.
• Asymptotically equivalent to the Wald and Likelihood Ratio tests, so it shares their large-sample theoretical guarantees.
• Naturally generalizes to testing several restrictions jointly via the same quadratic-form statistic.

Limitations

• The chi-square reference distribution is asymptotic-it can be inaccurate in small samples, as with the Wald and Likelihood Ratio tests.
• Requires a correctly specified likelihood; misspecification biases the score and information calculations in ways that are not always obvious.
• Breaks down (or requires special handling) when the null value \( \theta_0 \) is on the boundary of the parameter space.
• Can behave erratically for restrictions far from the region where the log-likelihood is well approximated by a quadratic (i.e., strongly non-normal likelihoods).
• Less intuitive to explain than the Wald Test, since it is expressed in terms of the score and information rather than a directly interpretable distance or standard error.

When NOT to Use It

• Wald Test: use instead when you have already fitted the unrestricted model anyway (e.g., standard regression output) and just want a quick significance check on an individual coefficient.
• Likelihood Ratio Test: use instead when both restricted and unrestricted models are easy to fit and you want the test with generally the best small-sample behavior of the trinity.
• Exact tests (e.g., Fisher's Exact Test, exact binomial test): use instead of the asymptotic chi-square approximation when sample sizes are very small and exact small-sample tests are available.
• Bootstrap-based tests: use instead when the regularity conditions underlying the asymptotic chi-square distribution are in doubt, such as near a parameter boundary.

Score Test vs Wald vs LR vs t-test

16.1 Score Test vs Wald Test

16.2 Score Test vs Likelihood Ratio Test

16.3 Score Test vs One-Sample t-test / z-test

Common Misconceptions

• "The Score Test requires fitting the unrestricted model too." Not true-this is precisely what distinguishes it from the Wald and Likelihood Ratio tests; see Worked Example 2, where the full logistic regression was never fitted.
• "Score, Wald, and LR tests always give the same p-value." They are only asymptotically equivalent-in finite samples they can, and often do, disagree, especially when the log-likelihood is not well approximated by a quadratic near \( \theta_0 \).
• "Lagrange Multiplier Test and Score Test are different tests." They are the exact same calculation under two different names-Score Test in the statistics tradition, Lagrange Multiplier Test in econometrics-see the FAQ for the historical reasoning.
• "A non-significant Score Test proves the restriction is exactly true." Failing to reject \( H_0 \) only means there was not enough evidence against the restriction with this sample-it does not prove the restriction holds exactly.
• "The Score Test can be used for any hypothesis, not just parameter restrictions." It is specifically designed for testing whether a parameter (or set of parameters) equals a hypothesized value-it is not a general-purpose replacement for goodness-of-fit or non-nested model comparisons.

Interview Questions

1. Derive the Score Test statistic and explain the role of the Fisher information in it.
2. Why does the Score Test only require fitting the restricted model, unlike the Wald and Likelihood Ratio tests?
3. Explain the geometric intuition behind the "trinity of tests"-Wald, LR, and Score-using the log-likelihood curve.
4. Why are the Score Test, Wald Test, and Likelihood Ratio Test only asymptotically, not exactly, equivalent?
5. Show algebraically why the Score Test for a single Bernoulli proportion reduces to the squared one-sample z-test statistic.
6. What happens to the Score Test when the null parameter value sits on the boundary of the parameter space, such as testing a variance component equals zero?
7. How does the Breusch-Pagan test for heteroskedasticity use the Score Test framework?
8. Why is the Score Test particularly attractive for genome-wide association studies scanning thousands of genetic markers?
9. What is the difference between using the expected (Fisher) information versus the observed information in a Score Test?
10. Under what circumstances would you prefer the Likelihood Ratio Test over the Score Test despite its higher computational cost?

Frequently Asked Questions

• What is a Score Test used for? +
  A Score Test (also called a Lagrange Multiplier Test) checks whether a restriction on a statistical model-typically that one or more parameters equal a specific value, often zero-is consistent with the observed data, using only the model fitted under that restriction rather than the full unrestricted model.
• What is the formula for the Score Test? +
  The test statistic is LM = U(θ₀)ᵀ I(θ₀)⁻¹ U(θ₀), where U(θ₀) is the score vector-the gradient of the log-likelihood-evaluated at the null-hypothesis value θ₀, and I(θ₀) is the Fisher information matrix at that same point. Under the null hypothesis, LM is asymptotically chi-square distributed with degrees of freedom equal to the number of restrictions being tested.
• What is the difference between the Score Test and the Wald Test? +
  The Wald Test requires fitting the full, unrestricted model and then measuring how far the estimated parameter sits from the null value, scaled by its standard error. The Score Test flips this around: it fits only the restricted (null) model and asks how steep the log-likelihood is at that single point-if the slope is close to zero, the restriction looks consistent with the data, and the unrestricted model never needs to be estimated.
• What is the difference between the Score Test and the Likelihood Ratio Test? +
  The Likelihood Ratio Test compares the log-likelihoods of both the restricted and unrestricted models directly, so both must be fitted. The Score Test only fits the restricted model and relies on the local slope and curvature of the log-likelihood there, which makes it noticeably cheaper when the unrestricted model has many extra parameters or is hard to converge.
• Why is the Score Test also called the Lagrange Multiplier Test? +
  When you set up the restricted maximum likelihood problem using a Lagrange multiplier to enforce the constraint, that multiplier measures exactly how much the constraint is fighting against the unconstrained optimum. This multiplier turns out to be algebraically proportional to the score, so the identical test is called the Score Test in the statistics literature and the Lagrange Multiplier (LM) Test in econometrics-both names describe the same calculation.
• When should I prefer the Score Test over the Wald and Likelihood Ratio tests? +
  Prefer the Score Test when the unrestricted model is expensive, unstable, or difficult to fit-classic cases include testing whether to add many extra regressors at once, or running diagnostic checks like the Breusch-Pagan test for heteroskedasticity and the Breusch-Godfrey test for autocorrelation, both of which only require the restricted (simpler) model.
• How do you interpret a Score Test p-value? +
  If the p-value is below your chosen significance level (commonly α = 0.05), you reject the null hypothesis and conclude the restriction is not consistent with the data-i.e., the log-likelihood has a meaningfully non-zero slope at the restricted parameter value. If the p-value is at or above α, there is not enough evidence to reject the restriction.

Key Takeaways

• The Score Test (Lagrange Multiplier Test) checks whether a restriction on a model's parameters is consistent with the data, using only the restricted (null) model.
• The test statistic \( LM = U(\theta_0)^\top I(\theta_0)^{-1} U(\theta_0) \) measures the slope of the log-likelihood at \( \theta_0 \), rescaled by the curvature (Fisher information) there.
• It is one of three asymptotically equivalent tests-alongside the Wald Test and Likelihood Ratio Test-that differ only in which point(s) on the log-likelihood curve they evaluate.
• Because it never requires fitting the unrestricted model, it is the natural choice for expensive models, large-scale scans (e.g., GWAS), and classic regression diagnostics like Breusch-Pagan and Breusch-Godfrey.
• In Python, statsmodels exposes Score Tests both directly (e.g., GLMResults.score_test) and through dedicated diagnostic functions like het_breuschpagan.
• If \( p < 0.05 \), the restriction is rejected-the data disagrees with \( \theta_0 \); if \( p \geq 0.05 \), there isn't enough evidence against the restriction, but always keep the asymptotic nature of the chi-square approximation in mind with small samples.
• If the unrestricted model is cheap to fit and you want the most robust small-sample behavior of the trinity, consider the Likelihood Ratio Test instead; if you already have the unrestricted fit in hand, the Wald Test is the more natural choice.

The Score Test occupies a distinctive niche among likelihood-based hypothesis tests: rather than comparing two fitted models or measuring a distance in parameter space, it asks a sharper question directly at the point of interest-does the log-likelihood still have somewhere it wants to go, right at the restriction we proposed? That question can be answered using only the simpler, restricted model, which is precisely what makes the test so valuable whenever the unrestricted alternative is expensive, unstable, or simply unnecessary to fit in full.

The two worked examples above show this in action-one where the Score Test collapsed to the familiar one-sample proportion z-test, and one where a logistic regression coefficient was tested without ever fitting the two-parameter unrestricted model. Alongside the Wald and Likelihood Ratio tests, as shown in the Trinity section, the Score Test rounds out a complete toolkit for testing restrictions on maximum likelihood models, each test suited to a different balance of computational cost and which model you already have in hand.