Anderson-Darling Test

Introduction

The Anderson-Darling Test checks whether a sample of data plausibly comes from a specific theoretical distribution-most often the normal distribution-with one important twist compared to related tests: it pays extra attention to the tails. A single stray outlier or a subtly heavy tail, the kind of thing that can slip past a test treating every part of the distribution equally, is exactly what this test is built to catch.

By the end of this article you will be able to state exactly when an Anderson-Darling Test applies, compute the A-squared statistic completely by hand on two contrasting worked examples, understand why the test is usually read against critical values rather than a single precise p-value, know when to reach for the Kolmogorov-Smirnov Test or Shapiro-Wilk Test instead, and run the same test in one line of Python with scipy.stats.

What Is the Anderson-Darling Test?

Introduced by Theodore Anderson and Donald Darling in 1954, the test builds on the same empirical distribution function (ECDF) idea used by the Kolmogorov-Smirnov Test: sort the sample, trace out its cumulative step function, and compare that against a theoretical cumulative distribution function. Where the two tests diverge is in how they measure the gap between those curves.

The Kolmogorov-Smirnov Test reports only the single largest gap, wherever it happens to occur. The Anderson-Darling Test instead integrates a squared, weighted discrepancy across the entire distribution-and the weighting function is constructed so that discrepancies near the extreme low and high ends of the distribution count for more than discrepancies near the middle. The result, denoted \( A^2 \), is more sensitive to exactly the kind of departure that shows up as heavier-than-expected tails or an unusual outlier.

When to Use It

Use the Anderson-Darling Test whenever you need to check whether continuous data matches a specific theoretical distribution-typically normal-and you particularly care about catching departures driven by outliers or heavy tails, which a test giving equal weight to every point might under-detect.

The key requirement: continuous data and a genuine interest in whether the sample matches a specified distribution-if tail behavior specifically isn't your concern and you only care about normality, the Shapiro-Wilk Test is often more powerful overall.

Key Assumptions

• Continuous data. The test is designed for continuous measurements; heavily rounded or discrete data can distort the underlying distribution-function comparison.
• Independent observations. Each observation is assumed independent of the others.
• Correction needed when parameters are estimated from the sample. If the theoretical distribution's mean and standard deviation are fitted from the same sample being tested, a small-sample correction factor must be applied before comparing against the standard critical value table-see Common Pitfalls.
• No extreme ties. Large numbers of exactly repeated values can distort the comparison, since the test relies on the ordering and spacing of the sorted sample.
• Reasonably sized samples. Very small samples limit the test's power, while very large samples can make trivial deviations statistically significant, as with most goodness-of-fit tests.

Hypotheses

• Null Hypothesis (\( H_0 \)): the sample was drawn from the specified theoretical distribution.
• Alternative Hypothesis (\( H_1 \)): the sample was not drawn from that distribution.

(As with other goodness-of-fit tests, failing to reject \( H_0 \) never proves the distribution matches-it only means the test did not find enough evidence, in this sample, to rule it out.)

The Formula, Explained

Let \( x_{(1)} \le x_{(2)} \le \cdots \le x_{(n)} \) be the sorted sample, and let \( F \) be the specified theoretical cumulative distribution function (with known or estimated parameters). Define \( z_i = F(x_{(i)}) \) for each sorted value. The Anderson-Darling statistic is:

The term \( (2i - 1) \) grows fastest for the smallest and largest values of \( i \)-that is, for the sorted values closest to the two tails of the sample-which is precisely what gives the test its extra sensitivity to tail discrepancies. A larger \( A^2 \) indicates a bigger mismatch between the sample and the assumed distribution.

When the theoretical distribution's parameters are estimated from the same sample (the usual case in practice for a normal-distribution test), a small-sample correction factor is applied before comparing against standard critical value tables:

The corrected statistic \( A^{2*} \) is then compared against tabulated critical values-see Working With Critical Values-or converted to an approximate p-value using established formulas (Stephens, 1974).

Worked Example 1: An Approximately Normal Sample

Suppose 8 measurements are collected and sorted in ascending order:

The sample mean is \( \bar{x} = 24.25 \) and the sample standard deviation is \( s \approx 1.809 \). Standardizing each value and evaluating the standard normal CDF gives \( z_i = \Phi\left(\frac{x_{(i)}-\bar{x}}{s}\right) \):

Applying the formula with \( n = 8 \), summing \( (2i-1)[\ln(z_i) + \ln(1-z_{9-i})] \) across all 8 terms gives an inner sum of approximately \( -34.923 \), so:

With the small-sample correction, \( A^{2*} = 0.110 \times \left(1 + \frac{4}{8} - \frac{25}{64}\right) \approx 0.122 \). At \( n = 8 \), the 5% critical value for a normal-distribution test with estimated parameters is approximately \( 0.666 \) (see critical value table). Since \( 0.122 \) is far below \( 0.666 \), we fail to reject \( H_0 \)-this sample is entirely consistent with having been drawn from a normal distribution.

Worked Example 2: A Sample With a Heavy Tail

Now suppose 10 values are collected that are mostly clustered together, with a single large outlier pulling the tail far to the right:

The sample mean is \( \bar{x} = 6.9 \) and the sample standard deviation is \( s \approx 8.252 \)-already inflated substantially by the single outlier. Standardizing and evaluating the normal CDF gives:

Applying the same formula with \( n = 10 \):

With the small-sample correction, \( A^{2*} = 2.032 \times \left(1 + \frac{4}{10} - \frac{25}{100}\right) \approx 2.336 \). At \( n = 10 \), the 5% critical value is approximately \( 0.685 \). Since \( 2.336 \) is far above every standard critical value, including the 1% threshold, we reject \( H_0 \) decisively-and notably, this large \( A^2 \) is driven almost entirely by the single outlier at \( x = 30 \), exactly the kind of tail-concentrated departure the test is built to catch.

Python Example

In practice you'll almost never compute \( A^2 \) or its correction by hand. Python's scipy.stats.anderson computes the statistic and returns critical values directly:

from scipy import stats

data = [21.4, 22.8, 23.1, 24.0, 24.6, 25.2, 25.9, 27.0]

result = stats.anderson(data, dist='norm')

print(f"A-D statistic: {result.statistic:.4f}")
print("Critical Values:", result.critical_values)
print("Significance Levels (%):", result.significance_level)

Output:

A-D statistic: 0.1096
Critical Values: [0.497 0.559 0.666 0.773 0.917]
Significance Levels (%): [15.  10.   5.   2.5  1. ]

The raw statistic (0.1096) matches Worked Example 1 exactly, and it sits well below the 5% critical value of 0.666-confirming we fail to reject \( H_0 \). Note that scipy.stats.anderson returns the raw \( A^2 \) alongside critical values rather than a single p-value by default, consistent with how this test is traditionally reported; recent scipy versions can also return an approximate p-value directly via the method parameter.

How to Interpret Results

Because the Anderson-Darling Test weights the tails, a significant result is worth following up by simply looking at the extreme values in the sample-often, as in Worked Example 2, a single outlier is doing most of the work.

Why the Tails Get Extra Weight

The weighting term \( (2i - 1) \) in the formula is largest when \( i \) is close to 1 or close to \( n \) -that is, for the smallest and largest sorted values in the sample. Practically, this means:

• A single unusually extreme value can noticeably inflate \( A^2 \), even if the bulk of the sample fits the assumed distribution well-exactly what happened in Worked Example 2.
• A sample that fits well in the middle but has a subtly heavier tail than expected will often be flagged by Anderson-Darling even when a test like Kolmogorov-Smirnov, which only tracks the single largest gap, misses it.
• This makes the test especially useful in fields like finance and reliability engineering, where tail behavior-rare but extreme events-is often the entire point of the analysis.

Working With Critical Values Instead of a p-value

Unlike some other goodness-of-fit tests, the Anderson-Darling Test is traditionally reported by comparing the corrected statistic \( A^{2*} \) against a small table of critical values, rather than reading off a single precise p-value:

(These are the standard large-sample critical values for testing normality with estimated mean and variance, per Stephens, 1974; software may report slightly different small-sample-adjusted values, as seen in the Python output above.)

This approach is not a limitation so much as a reflection of how the test was originally developed and tabulated-most software, including scipy.stats.anderson, follows the same convention by default, though approximate p-values are available when a single number is preferred for reporting.

Common Pitfalls and How to Avoid Them

• Forgetting the small-sample correction. When distribution parameters are estimated from the same sample, comparing the raw \( A^2 \) directly against critical value tables meant for a fully specified distribution will understate significance-apply the correction factor first.
• Treating a non-significant result as proof of a match. A statistic below the critical value means insufficient evidence of a departure was found, not that the distribution match is confirmed.
• Ignoring which values are driving a significant result. Because the test weights tails heavily, a single outlier can dominate \( A^2 \)-always check the extreme values directly rather than treating a significant result as evidence of a broad, distributed departure.
• Applying the test to data with many tied values. Heavy rounding or repeated values can distort the ordering the test relies on.
• Reporting only a p-value without checking the tails. The whole point of choosing this test over Kolmogorov-Smirnov is its tail sensitivity-don't discard that information by only reporting the summary p-value.

Advantages

• More sensitive than the Kolmogorov-Smirnov Test to departures concentrated in the tails.
• Well suited to detecting outliers and heavy-tailed distributions, common in finance and reliability data.
• Can test goodness-of-fit against distributions other than normal, given the appropriate critical value tables.
• Widely implemented and well documented, with established critical value tables going back decades.
• Uses the entire distribution shape rather than a single point of maximum discrepancy.

Limitations

• Sensitive to outliers, which cuts both ways. The same tail-weighting that makes the test powerful can also make it overreact to a single unusual data point-see Worked Example 2.
• Traditionally reported via critical values, not a single p-value. This can feel less convenient than tests that report a p-value directly, though software increasingly offers p-value approximations.
• Requires a correction when parameters are estimated. Forgetting the small-sample correction factor when parameters are fitted from the same sample invalidates the standard critical value comparison-see Common Pitfalls.
• Designed for continuous data. Heavily rounded or discrete data can distort results.
• No quantified effect size. Like other goodness-of-fit tests, \( A^2 \) indicates departure but not a directly interpretable magnitude in the original units of the data.

When NOT to Use It

• When tail behavior specifically is not a concern and overall power for normality testing matters most-the Shapiro-Wilk Test is often the more powerful general-purpose choice.
• When you need a straightforward two-sample comparison-use the two-sample Kolmogorov-Smirnov Test instead.
• When the data is discrete, heavily rounded, or contains many tied values.
• When a single, obviously erroneous data-entry outlier is present and hasn't yet been investigated-fix or verify suspicious extreme values before letting them drive a significance decision.
• When distribution parameters were estimated from the sample and the appropriate correction factor has not been applied.

Anderson-Darling vs Kolmogorov-Smirnov vs Shapiro-Wilk

Common Misconceptions

• "Anderson-Darling always outperforms Kolmogorov-Smirnov." It's generally more powerful for tail-driven departures, but not universally superior-see Comparisons for how the tests actually differ.
• "A significant Anderson-Darling result means the whole distribution is wrong." Often the result is driven mostly by the tails, sometimes by a single outlier-see Why the Tails Get Extra Weight.
• "You can skip the small-sample correction if the sample is reasonably large." The correction factor should be applied whenever parameters are estimated from the sample, regardless of size, though its effect shrinks as \( n \) grows-see Common Pitfalls.
• "A non-significant result proves the data follows the assumed distribution." It only means no significant departure was detected given the sample-especially relevant with small samples that have limited power.
• "The test doesn't produce a p-value, so it's less rigorous." Reporting via critical values is simply a different, well-established convention-approximate p-values are readily available in most modern software when needed.

Interview Questions

1. Explain in your own words what the Anderson-Darling statistic measures, and how it differs from the Kolmogorov-Smirnov statistic.
2. Why does the weighting term \( (2i-1) \) in the formula give the test extra sensitivity to the tails of a distribution?
3. When would you prefer the Anderson-Darling Test over the Kolmogorov-Smirnov Test, and vice versa?
4. Why is a small-sample correction needed when distribution parameters are estimated from the same sample being tested?
5. How would you explain to a colleague why a single outlier caused a large Anderson-Darling statistic even though the rest of the sample looks normal?
6. Why is the Anderson-Darling Test traditionally reported using critical values rather than a single p-value, and how would you interpret a critical value table?
7. How does the Anderson-Darling Test compare to the Shapiro-Wilk Test in terms of statistical power for normality testing specifically?
8. What follow-up analysis would you do after a significant Anderson-Darling result, given that it might be driven by only a few extreme observations?
9. Can the Anderson-Darling Test be used against distributions other than normal? What would you need to change?
10. Describe a real-world scenario, such as in finance or reliability engineering, where the tail sensitivity of this test would be particularly valuable.

Frequently Asked Questions

• What is the Anderson-Darling Test used for? +
  The Anderson-Darling Test is a goodness-of-fit test that checks whether a sample plausibly comes from a specific theoretical distribution, most often the normal distribution. It builds on the same empirical-distribution-function idea as the Kolmogorov-Smirnov Test but weights discrepancies more heavily in the tails, which makes it particularly good at catching outliers and heavy-tailed departures that a test treating every point equally might miss.
• What is the formula for the Anderson-Darling Test statistic? +
  The A-squared statistic is built from the sorted sample values and the theoretical cumulative distribution function evaluated at each one, combined in a weighted sum where the weighting term grows larger near the extremes of the distribution and smaller near the center. This means a sample with a well-fitting middle but a poorly fitting tail will still produce a noticeably elevated A-squared, unlike simpler goodness-of-fit statistics that only look at the single largest gap.
• How do you interpret the Anderson-Darling Test result? +
  Rather than reading off a precise p-value directly, the raw A-squared statistic is typically compared against a table of critical values for common significance levels such as 15%, 10%, 5%, 2.5%, and 1%. If A-squared exceeds the critical value at your chosen significance level (commonly 5%), the sample significantly departs from the assumed distribution; software can also supply an approximate p-value using established formulas for the corrected statistic.
• What is the difference between the Anderson-Darling Test and the Kolmogorov-Smirnov Test? +
  Both compare a sample's empirical distribution function against a theoretical one, but they measure the discrepancy differently. The Kolmogorov-Smirnov Test reports only the single largest vertical gap between the two curves, giving equal importance to every part of the distribution. The Anderson-Darling Test instead sums a weighted discrepancy across the entire range with extra emphasis on the tails, which generally gives it more power to detect departures concentrated at the extremes, such as heavy tails or a small number of outliers.
• What assumptions does the Anderson-Darling Test make? +
  The test assumes the data is continuous and the observations are independent. When testing against a normal distribution whose mean and standard deviation are estimated from the same sample being tested-rather than specified in advance-a small-sample correction factor must be applied to the raw statistic before comparing it against the standard critical value table, since estimating those parameters from the data changes the statistic's sampling distribution.
• Why does the Anderson-Darling Test weight the tails more heavily? +
  Many practically important departures from a target distribution-a single large outlier, a heavier-than-normal tail, or a skewed extreme-show up most clearly at the far ends of the distribution rather than near its center. By weighting the squared discrepancy between the sample and the theoretical distribution more heavily as it approaches the tails, the Anderson-Darling Test is built to be more sensitive to exactly this kind of departure than a test that treats every point along the distribution equally.

Key Takeaways

• The Anderson-Darling Test compares a sample's empirical distribution function against a theoretical one, weighting discrepancies near the tails more heavily than discrepancies near the center.
• The statistic \( A^2 = -n - \frac{1}{n}\sum(2i-1)[\ln(z_i)+\ln(1-z_{n+1-i})] \) is compared against tabulated critical values, often after a small-sample correction when parameters are estimated from the same sample.
• It is more sensitive than the Kolmogorov-Smirnov Test to outliers and heavy-tailed departures, making it well suited to finance, reliability, and other fields where extreme values matter.
• A significant result should be followed up by checking the extreme values directly, since a single outlier can dominate the statistic-see Worked Example 2.
• In Python, scipy.stats.anderson returns the raw statistic and critical values directly, following the test's traditional reporting convention.
• For normality testing specifically, without a particular focus on tail behavior, the Shapiro-Wilk Test is often more powerful overall.
• A non-significant result never proves the distribution matches-it only means no significant departure was detected given the sample size and data at hand.

The Anderson-Darling Test earns its place alongside Kolmogorov-Smirnov and Shapiro-Wilk by answering a subtly different question well: not just whether a sample's distribution differs from what's expected, but whether that difference shows up specifically in the tails, where a single extreme observation or a heavier-than-expected spread often matters most in practice. Its tail-weighting is both its signature strength and the thing to watch most carefully when interpreting a significant result.

The two worked examples above show how directly the test reads from sorted data-a nearly-normal sample produces a small \( A^2 \) safely below the critical value, while a single dominant outlier inflates \( A^2 \) well past every standard threshold. Apply the small-sample correction when parameters are estimated from the data, check which values are driving a significant result before drawing conclusions, and reach for Shapiro-Wilk when overall normality power matters more than tail sensitivity specifically-together, these three tests cover the great majority of real-world goodness-of-fit questions.