Lilliefors Test

Introduction

The Lilliefors Test checks whether a sample of data plausibly came from a normally distributed population when the mean and standard deviation used for the comparison are estimated from the sample itself rather than known in advance. It's the correction that makes a Kolmogorov-Smirnov-style normality check valid in the situation almost everyone is actually in: wanting to test normality of data whose true population parameters were never known to begin with.

By the end of this article you will be able to state exactly when a Lilliefors Test applies and why the ordinary Kolmogorov-Smirnov table doesn't work here, compute the D statistic completely by hand on two contrasting worked examples, read the corrected critical value table Lilliefors published, know when to reach for the Shapiro-Wilk Test instead, and run the same test in one line of Python with statsmodels.

What Is the Lilliefors Test?

Introduced by Hubert Lilliefors in 1967, the Lilliefors Test is a modification of the Kolmogorov-Smirnov Test built for a specific, very common problem. The ordinary Kolmogorov-Smirnov Test compares a sample's empirical distribution function against a fully specified reference distribution-one whose mean and standard deviation are fixed before looking at the data. In practice, though, people almost always want to test whether their sample is normal using the mean and standard deviation of that same sample, since the true population values are rarely known. Plugging estimated parameters into the standard Kolmogorov-Smirnov table makes the test overly conservative-the fitted curve is pulled toward the data by construction, which shrinks the typical gap between the two curves and understates how unusual a given D really is.

Lilliefors solved this by simulation: he generated many samples from a true normal distribution, estimated the mean and standard deviation from each sample the same way an analyst would, computed the Kolmogorov-Smirnov D statistic against that fitted curve, and tabulated the resulting critical values. The result is a test that keeps the familiar Kolmogorov-Smirnov logic-largest gap between empirical and theoretical cumulative distributions-but judges that gap against critical values that are actually valid when the comparison distribution's parameters come from the data.

When to Use It

Use a Lilliefors Test whenever you need to check whether a continuous sample plausibly comes from a normal distribution, and the mean and standard deviation you'd compare against are estimated from that same sample-which is nearly always the case outside of textbook problems with a known population.

The key requirement: continuous data, an estimated (not independently known) mean and standard deviation, and a genuine question about whether normality can be assumed. If your population parameters really are known in advance, the plain Kolmogorov-Smirnov Test is the correct choice instead-see Why Estimated Parameters Need a Correction.

Key Assumptions

• Continuous data. The test is designed for continuous measurements; heavily rounded, discrete, or count data can distort the empirical distribution function and produce misleading results.
• Independent observations. Each observation is assumed independent of the others, which can be violated by repeated measures or clustered data.
• Identically distributed observations. All observations are assumed to be drawn from the same underlying distribution-mixing two different populations in one sample can produce an apparent non-normality that has nothing to do with either population individually.
• Parameters estimated from the same sample. The test's corrected critical values are only valid when the mean and standard deviation plugged into the comparison distribution come from the sample under test-if they're known independently, use the plain Kolmogorov-Smirnov Test instead.
• Reasonably sized sample. Very small samples have limited power to detect real departures; very large samples make the test, like most goodness-of-fit tests, sensitive to trivial deviations-see Common Pitfalls.

Hypotheses

• Null Hypothesis (\( H_0 \)): the sample was drawn from a normally distributed population.
• Alternative Hypothesis (\( H_1 \)): the sample was not drawn from a normally distributed population.

(Note the asymmetry: failing to reject \( H_0 \) never proves normality-it only means the test did not find enough evidence, in this sample, to rule it out.)

The Formula, Explained

Let \( x_{(1)} \le x_{(2)} \le \cdots \le x_{(n)} \) be the sample values sorted in ascending order, with sample mean \( \bar{x} \) and sample standard deviation \( s \). First, standardize each sorted value:

Let \( \Phi(z) \) denote the standard normal cumulative distribution function. The empirical distribution function \( F_n \) is the step function that jumps by \( 1/n \) at each sorted observation. The Lilliefors statistic is the largest absolute vertical gap between \( F_n \) and the fitted normal curve \( \Phi(z_{(i)}) \), which is computed as the larger of two one-sided maximums:

\( D^+ \) captures the largest amount the empirical function sits above the fitted normal curve just before each jump, and \( D^- \) captures the largest amount it sits below the curve just after each jump. \( D \) is always between 0 and 1; the closer it is to 0, the more closely the sample's shape tracks the fitted normal curve. A p-value for \( D \) is obtained from Lilliefors's corrected critical value table-see below-rather than the standard Kolmogorov-Smirnov table.

The Lilliefors Critical Value Table

Because the fitted normal curve is pulled toward the sample by using its own mean and standard deviation, the sampling distribution of \( D \) under \( H_0 \) is different-and generally smaller-than under the plain Kolmogorov-Smirnov Test with fixed parameters. Lilliefors derived approximate critical values by simulation for several common significance levels:

For sample sizes beyond a small table, the commonly used large-sample approximations for the \( \alpha = 0.05 \) critical value are \( D_{0.05} \approx \dfrac{0.886}{\sqrt{n}} \), with similarly scaled constants for other significance levels. In practice, software computes an interpolated or asymptotic p-value directly rather than requiring a manual table lookup-see the Python Example below.

Worked Example 1: An Approximately Normal Sample

Suppose 12 measurements are collected and sorted in ascending order:

The sample mean is \( \bar{x} = 25.692 \) and the sample standard deviation is \( s = 2.018 \). Standardizing each sorted value and evaluating \( \Phi(z) \):

Computing \( i/n - \Phi(z_{(i)}) \) for every row, the largest value is at \( i = 3 \): \( 0.2500 - 0.2009 = 0.0491 \), so \( D^+ = 0.0491 \). Computing \( \Phi(z_{(i)}) - (i-1)/n \) for every row, the largest value is at \( i = 10 \): \( 0.8014 - 0.7500 = 0.0514 \), so \( D^- = 0.0514 \). Therefore:

For \( n = 12 \), even the loosest tabulated critical value at \( \alpha = 0.20 \) sits well above \( 0.15 \) (interpolating between the \( n = 10 \) and \( n = 15 \) rows in the table above), and \( 0.0514 \) falls far below it-software gives \( p \approx 0.94 \). We fail to reject \( H_0 \): this sample's empirical distribution tracks the fitted normal curve closely.

Worked Example 2: A Clearly Skewed Sample

Now suppose 12 values are collected that are obviously skewed, with one large outlier pulling the tail far to the right:

The sample mean is \( \bar{x} = 10.333 \) and the sample standard deviation is \( s = 7.797 \)-already inflated by the outlier. Standardizing and evaluating \( \Phi(z) \):

Computing \( i/n - \Phi(z_{(i)}) \) for every row, the largest value is at \( i = 10 \): \( 0.8333 - 0.5341 = 0.2993 \), so \( D^+ = 0.2993 \). Computing \( \Phi(z_{(i)}) - (i-1)/n \) for every row, the largest value is at \( i = 1 \): \( 0.2470 - 0.0000 = 0.2470 \), so \( D^- = 0.2470 \). Therefore:

For \( n = 12 \), interpolating the table above puts the \( \alpha = 0.05 \) critical value at roughly \( 0.24 \)-\( 0.25 \), and \( D = 0.2993 \) clears it, with software returning \( p \approx 0.004 \). We reject \( H_0 \): the sample departs significantly from normality, driven mostly by the same outlier that inflated the sample standard deviation in the first place.

Python Example

In practice you'll almost never interpolate a printed table by hand. Python's statsmodels.stats.diagnostic.lilliefors computes the corrected statistic and p-value directly from the raw sample:

from statsmodels.stats.diagnostic import lilliefors

data = [22.1, 23.4, 24.0, 24.6, 25.1, 25.5, 25.9, 26.3, 26.8, 27.4, 28.0, 29.2]

D, p_value = lilliefors(data, dist='norm')

print(f"D statistic: {D:.4f}")
print(f"p-value: {p_value:.4f}")

Output:

D statistic: 0.0514
p-value: 0.9900

This matches Worked Example 1's \( D \) exactly; statsmodels caps very high p-values at \( 0.99 \) since Lilliefors's simulated table doesn't resolve p-values more precisely than that in the far tail. Note that dist='norm' is what tells the function to use the normal-distribution correction rather than plain Kolmogorov-Smirnov critical values-passing the wrong distribution silently gives an invalid p-value.

How to Interpret Results

Because \( D \) is defined as the single largest gap anywhere in the distribution, it's worth checking where that gap occurred-a big gap in one tail, as in Worked Example 2, points to an outlier or skew, while a gap in the middle of the distribution suggests a different kind of shape mismatch.

Why Estimated Parameters Need a Correction

The plain Kolmogorov-Smirnov Test's critical values are derived under the assumption that the reference distribution is fixed independently of the data being tested. When you instead compute the sample mean and sample standard deviation and plug them into the normal curve you're comparing against, the fitted curve is no longer independent of the data-it was built to fit the sample as well as possible given only two parameters. That fitting process mechanically pulls the fitted curve closer to the empirical distribution function than an independently specified curve would be, which shrinks \( D \) on average even when the true population isn't normal at all.

If you naively looked up that shrunken \( D \) in the standard Kolmogorov-Smirnov table-built for the independent-parameter case-you'd systematically fail to reject \( H_0 \) too often, understating real departures from normality. Lilliefors's contribution was to simulate the actual sampling distribution of \( D \) under this estimated-parameter setup and tabulate smaller, corrected critical values that restore the test's intended \( \alpha \)-level accuracy.

Common Pitfalls and How to Avoid Them

• Using the standard Kolmogorov-Smirnov table with estimated parameters. This is the single most common mistake-see Why Estimated Parameters Need a Correction for why it silently understates real departures from normality.
• Treating a non-significant result as proof of normality. A large p-value means insufficient evidence of non-normality was found, not that normality is confirmed-especially relevant with small samples that simply lack power to detect real departures.
• Blindly trusting the p-value at large sample sizes. Like most goodness-of-fit tests, power increases with \( n \), so even negligible departures can become significant once samples grow very large.
• Running the test on data that includes multiple subgroups. If a sample actually mixes two different populations, the combined data can look non-normal even if each subgroup is individually normal-check subgroups separately when relevant.
• Ignoring where the maximum gap occurred. A significant \( D \) alone doesn't say whether the data is skewed, heavy-tailed, or has isolated outliers-locate whether \( D^+ \) or \( D^- \) drove the result, as in Worked Example 2.

Advantages

• Correctly handles the common real-world case where the mean and standard deviation are estimated from the sample being tested.
• Shares the intuitive, visual empirical-distribution-function logic of the widely taught Kolmogorov-Smirnov Test.
• Produces a single, easy-to-report statistic (\( D \)) with a well-established corrected table.
• Works for continuous data without needing published order-statistic coefficients, unlike Shapiro-Wilk.
• Extends naturally to testing against a fitted exponential distribution, using Lilliefors's separate 1969 correction.

Limitations

• Generally less powerful than Shapiro-Wilk. Most simulation studies find the Shapiro-Wilk Test detects real departures from normality more reliably, especially in small to moderate samples.
• Sensitive to sample-mean and sample-variance inflation from outliers. A single extreme value, as in Worked Example 2, both pulls the fitted curve and can drive a large \( D \) at the same time.
• No diagnosis of the type of departure. The test only produces a p-value, not information about whether the data is skewed, heavy-tailed, or has outliers, beyond noting where the maximum gap occurred.
• Designed for continuous data. Heavily rounded or discrete data, with many tied values, can distort the empirical distribution function.
• Correction is distribution-specific. The classic tables apply to the normal (and, separately, exponential) case-there isn't a single universal Lilliefors table for arbitrary estimated distributions.

When NOT to Use It

• When the population mean and standard deviation are genuinely known independently of the sample-use the plain Kolmogorov-Smirnov Test with its standard table instead.
• When maximum statistical power for detecting non-normality in a small sample is the priority-the Shapiro-Wilk Test is typically the stronger choice.
• When the data is discrete, heavily rounded, or contains many tied values.
• When you need goodness-of-fit against a distribution other than normal or exponential-a distribution-specific corrected table would be needed, which generally doesn't exist off the shelf.
• When the sample actually mixes multiple distinct subpopulations that should be analyzed separately.

Lilliefors vs Kolmogorov-Smirnov vs Shapiro-Wilk vs Anderson-Darling

Common Misconceptions

• "The Lilliefors Test is just the Kolmogorov-Smirnov Test with a different name." They compute the same D statistic, but the Lilliefors Test uses a different, corrected critical value table specifically for when parameters are estimated from the sample-see Why Estimated Parameters Need a Correction.
• "A non-significant Lilliefors Test proves the data is normal." It only means no significant departure was detected at the given sample size-see Common Pitfalls. Small samples in particular often lack the power to detect real, moderate departures.
• "You can use the regular Kolmogorov-Smirnov table if you're careful." No amount of care fixes this-the standard table is simply invalid once the comparison distribution's parameters come from the same data being tested.
• "The Lilliefors Test tells you what kind of non-normality you have." It only tells you whether the data departs from normal and roughly where the biggest gap occurred, not whether it's skewed, heavy-tailed, or has outliers in a fully characterized way.
• "Shapiro-Wilk and Lilliefors will always agree." They usually agree on clear-cut cases, but because they measure different things-order-statistic weighting versus maximum distribution-function gap-borderline samples can produce a significant result on one test and not the other.

Interview Questions

1. Explain in your own words what problem the Lilliefors Test solves that the plain Kolmogorov-Smirnov Test does not.
2. Why does estimating the mean and standard deviation from the same sample shrink the Kolmogorov-Smirnov D statistic on average, and why does that matter for choosing critical values?
3. Walk through how \( D^+ \) and \( D^- \) are computed from the sorted sample and the fitted normal CDF.
4. Compare the Lilliefors Test and the Shapiro-Wilk Test in terms of what each one actually measures.
5. What does a non-significant Lilliefors result actually tell you, and what does it not tell you?
6. Why can a single outlier drive both the fitted normal curve and the D statistic at the same time, as in Worked Example 2?
7. If you saw a large D driven by a gap at the very start or end of the sorted sample, what would that suggest about the shape of the departure?
8. Under what circumstances would you use the plain Kolmogorov-Smirnov Test instead of the Lilliefors Test?
9. How would you explain to a colleague why looking up a Lilliefors D in a standard Kolmogorov-Smirnov table would be a mistake?
10. Describe a situation where mixing two subpopulations in one sample could produce a misleading Lilliefors result.

Frequently Asked Questions

• What is the Lilliefors Test used for? +
  The Lilliefors Test is a hypothesis test that checks whether a sample plausibly comes from a normal distribution when the mean and variance used for the comparison are estimated from that same sample rather than known ahead of time. It is the go-to correction whenever someone wants to run a Kolmogorov-Smirnov-style normality check without independently specified population parameters.
• What is the formula for the Lilliefors Test statistic? +
  The D statistic is the maximum absolute vertical distance between the sample's empirical distribution function and a normal cumulative distribution function built from the sample mean and sample standard deviation. Formally, D equals the larger of two one-sided maximums: the largest amount the empirical function sits above the fitted normal curve, and the largest amount it sits below it.
• How do you interpret the Lilliefors Test result? +
  A small p-value, conventionally below 0.05, indicates the sample significantly departs from a normal distribution. A large p-value means the test did not find sufficient evidence of non-normality; this never proves the data is normal, it only means no statistically significant departure was detected given the available sample.
• What is the difference between the Lilliefors Test and the Kolmogorov-Smirnov Test? +
  The plain Kolmogorov-Smirnov Test assumes the comparison distribution's parameters are fully specified before looking at the data. When the mean and standard deviation are instead estimated from the same sample being tested-by far the more common real-world case-the ordinary Kolmogorov-Smirnov critical values are too conservative, making the test less likely to detect real departures. The Lilliefors Test supplies a separate, smaller set of critical values derived specifically for that estimated-parameter scenario, restoring the test's intended sensitivity.
• What is the difference between the Lilliefors Test and the Shapiro-Wilk Test? +
  Both are normality tests designed for the case where parameters are estimated from the sample, but they use different mechanics. The Lilliefors Test measures the largest gap between the sample's empirical distribution function and a fitted normal curve, while the Shapiro-Wilk Test compares a weighted combination of the sorted sample values against the sample variance. In most simulation studies the Shapiro-Wilk Test has somewhat higher power, particularly for small samples, but the Lilliefors Test remains popular because its logic mirrors the widely taught Kolmogorov-Smirnov Test.
• Can the Lilliefors Test be used for distributions other than normal? +
  Lilliefors originally derived corrected critical values for the normal distribution and, in a follow-up paper, for the exponential distribution with an estimated mean. It is not a general drop-in replacement for the plain Kolmogorov-Smirnov Test across arbitrary distributions with estimated parameters-each such case in principle needs its own corrected critical value table.

Key Takeaways

• The Lilliefors Test checks whether a sample plausibly came from a normal distribution when the mean and standard deviation used for comparison are estimated from that same sample.
• The statistic \( D = \max(D^+, D^-) \) measures the largest vertical gap between the sample's empirical distribution function and a fitted normal curve, exactly like Kolmogorov-Smirnov-but judged against a corrected table.
• Using the standard Kolmogorov-Smirnov critical values with estimated parameters understates real departures from normality-see Why Estimated Parameters Need a Correction.
• It is generally somewhat less powerful than the Shapiro-Wilk Test for detecting non-normality, particularly in small samples, but shares its logic with the widely known Kolmogorov-Smirnov Test.
• In Python, statsmodels.stats.diagnostic.lilliefors computes the corrected statistic and p-value directly from raw data in one line.
• A non-significant result never proves normality-it only means no significant departure was detected given the sample size and data at hand.
• For goodness-of-fit against a distribution with genuinely known parameters, use the plain Kolmogorov-Smirnov Test instead.

The Lilliefors Test fills a gap that trips up a lot of people the first time they run a normality check: the standard Kolmogorov-Smirnov table simply isn't valid once the comparison distribution's mean and standard deviation come from the sample being tested-which, in real analysis work, is almost always the case. By re-deriving the critical values under that estimated-parameter scenario, Lilliefors turned a familiar, intuitive test into one that actually behaves the way people expect when they use it.

The two worked examples above show the mechanics clearly-a sample that closely tracks its own fitted normal curve produces a small \( D \) and a large p-value, while a single dominant outlier inflates both the sample standard deviation and the maximum gap, producing a \( D \) that clears the corrected critical value by a wide margin. For raw power against small to moderate samples the Shapiro-Wilk Test usually edges it out, but the Lilliefors Test remains a natural, well-understood choice whenever the Kolmogorov-Smirnov framework is already the mental model in play.