Anderson-Darling Test

A single extreme observation can be more informative than the rest of the sample combined

Consider a sample of 1,000 measurements that closely follows a normal distribution, with one exception: a single observation far beyond the range of the rest. Goodness-of-fit tests based on the maximum deviation between empirical and theoretical distribution functions are largely insensitive to this single point.

The Anderson-Darling Test is constructed specifically to detect it. By weighting discrepancies near the tails of the distribution more heavily than discrepancies near the center, it remains sensitive to exactly this class of departure-a single outlier or a heavier-than-expected tail.

• Purpose: test whether data follows a specified distribution.
• Use when: outliers or heavy-tailed departures are of particular concern.
• Avoid when: tail behavior is not of specific interest-Shapiro-Wilk is generally more powerful for normality testing alone.
• Input: a continuous sample of independent observations.
• Output: the \( A^2 \) statistic (larger values indicate greater departure).
• Formula: \( A^2 = -n - \frac{1}{n}\sum(2i-1)[\ln(z_i)+\ln(1-z_{n+1-i})] \)
• Related tests:Kolmogorov-Smirnov, Shapiro-Wilk.

Why the test is sensitive to departures that other tests overlook

Consider two samples. Sample A is drawn from a normal distribution. Sample B is identical to Sample A except for one observation displaced far into the tail.

A test based on the single largest gap between the empirical and theoretical distribution functions is largely unaffected by Sample B's outlier: one point among many does not substantially shift the maximum gap. The Anderson-Darling Test instead integrates the squared discrepancy across the entire distribution, weighting points near the extremes more heavily than points near the center.

This is the principal distinction from the Kolmogorov-Smirnov Test: both compare cumulative distribution functions, but Anderson-Darling sums the weighted discrepancy across the full range rather than reporting only its maximum.

Worked Example 2 demonstrates this numerically: a single outlier among ten otherwise unremarkable observations drives the statistic well past every standard threshold.

Applications where tail sensitivity matters

Any time you need to check whether continuous data matches a theoretical distribution-and you especially care about catching outliers or heavy tails.

Key requirement: continuous data, and a genuine interest in tail behavior. For normality alone, see Shapiro-Wilk instead.

Conditions required for valid inference

• Continuous data. The test is designed for continuous measurements; heavily rounded or discrete data can distort the underlying distribution-function comparison.
• Independent observations. Each observation is assumed independent of the others.
• Correction needed when parameters are estimated from the sample. If the theoretical distribution's mean and standard deviation are fitted from the same sample being tested, a small-sample correction factor must be applied before comparing against the standard critical value table-see Common Pitfalls.
• No extreme ties. Large numbers of exactly repeated values can distort the comparison, since the test relies on the ordering and spacing of the sorted sample.
• Reasonably sized samples. Very small samples limit the test's power, while very large samples can make trivial deviations statistically significant, as with most goodness-of-fit tests.

Definition of the test statistic

Let \( x_{(1)} \le x_{(2)} \le \cdots \le x_{(n)} \) denote the sorted sample, and let \( F \) be the theoretical cumulative distribution function under test. Define \( z_i = F(x_{(i)}) \) for each sorted value.

The Anderson-Darling statistic is:

The term \( (2i - 1) \) grows fastest for the smallest and largest values of \( i \)-the sorted values closest to the tails-which produces the weighting described above. Larger \( A^2 \) indicates a greater discrepancy between the sample and the assumed distribution.

The corrected statistic \( A^{2*} \) is compared against tabulated critical values-see Critical Values-or converted to an approximate p-value (Stephens, 1974).

Null and alternative hypotheses

• Null Hypothesis (\( H_0 \)): the sample was drawn from the specified theoretical distribution.
• Alternative Hypothesis (\( H_1 \)): the sample was not drawn from that distribution.

Failing to reject \( H_0 \) never proves the distribution matches-it only means the test found no sufficient evidence to rule it out.

Why the weighting survives averaging

The test sorts the sample and evaluates the theoretical CDF at each point, producing the \( z_i \) values used above. Values near 0 or 1 correspond to the tails; values near 0.5 correspond to the center.

The \( (2i-1) \) weight means the first and last few sorted points contribute the most to the sum. A sample that fits well everywhere except at one extreme still produces a large \( A^2 \).

Worked example: a sample consistent with normality

Eight measurements, sorted:

The sample mean is \( \bar{x} = 24.25 \), the sample standard deviation is \( s \approx 1.809 \). Standardizing and evaluating the normal CDF gives \( z_i = \Phi\left(\frac{x_{(i)}-\bar{x}}{s}\right) \):

Summing \( (2i-1)[\ln(z_i) + \ln(1-z_{9-i})] \) across all 8 terms gives an inner sum of approximately \( -34.923 \), so:

With the small-sample correction, \( A^{2*} \approx 0.122 \). The 5% critical value at \( n = 8 \) is approximately \( 0.666 \). Since \( 0.122 \ll 0.666 \), the null hypothesis is not rejected.

Worked example: a single outlier driving a significant result

Ten values, mostly clustered together, with one large outlier pulling the tail far to the right:

The sample mean jumps to \( \bar{x} = 6.9 \) and \( s \approx 8.252 \)-already inflated by that single point. Standardizing and evaluating the normal CDF gives:

Applying the same formula with \( n = 10 \):

With the small-sample correction, \( A^{2*} \approx 2.336 \). The 5% critical value at \( n = 10 \) is only \( 0.685 \); \( 2.336 \) exceeds every standard threshold, including the 1% level, and the null hypothesis is rejected decisively.

A single displaced observation was sufficient to reverse the conclusion, illustrating the mechanism described above in numerical terms.

Implementation in Python

In practice, \( A^2 \) is rarely computed by hand. Python's scipy.stats.anderson computes the statistic directly:

from scipy import stats

data = [21.4, 22.8, 23.1, 24.0, 24.6, 25.2, 25.9, 27.0]

result = stats.anderson(data, dist='norm')

print(f"A-D statistic: {result.statistic:.4f}")
print("Critical Values:", result.critical_values)
print("Significance Levels (%):", result.significance_level)

Output:

A-D statistic: 0.1096
Critical Values: [0.497 0.559 0.666 0.773 0.917]
Significance Levels (%): [15.  10.   5.   2.5  1. ]

This value, 0.1096, matches Worked Example 1 exactly, and it falls well below the 5% critical value of 0.666, confirming that the null hypothesis is not rejected.

Interpreting the statistic

Because the test weights the tails, a significant result warrants inspection of the extreme values in the sample. As in Worked Example 2, a single outlier is often responsible for the majority of the statistic.

Origin of the tail weighting

The \( (2i - 1) \) term is largest when \( i \) is close to 1 or close to \( n \)-the smallest and largest sorted values. In practice, this has three consequences:

• A single unusually extreme value can noticeably inflate \( A^2 \), even if the bulk of the sample fits the assumed distribution well-as demonstrated in Worked Example 2.
• A sample that fits well in the middle but has a subtly heavier tail than expected is often flagged by Anderson-Darling even when a test based on the single largest gap, such as Kolmogorov-Smirnov, does not detect it.
• This property makes the test particularly useful in fields such as finance and reliability engineering, where tail behavior-rare but extreme events-is often the primary object of analysis.

Reporting via critical values rather than a p-value

Unlike some goodness-of-fit tests, the Anderson-Darling Test is traditionally reported by comparing \( A^{2*} \) against a table of critical values:

These are the standard large-sample critical values (Stephens, 1974); software may report slightly different small-sample-adjusted values, as seen in the Python output above.

Common pitfalls in application

• Forgetting the small-sample correction. When distribution parameters are estimated from the same sample, comparing the raw \( A^2 \) directly against critical value tables meant for a fully specified distribution will understate significance-apply the correction factor first.
• Treating a non-significant result as proof of a match. A statistic below the critical value means insufficient evidence of a departure was found, not that the distribution match is confirmed.
• Ignoring which values are driving a significant result. Because the test weights tails heavily, a single outlier can dominate \( A^2 \)-the extreme values should be checked directly rather than treating a significant result as evidence of a broad, distributed departure.
• Applying the test to data with many tied values. Heavy rounding or repeated values can distort the ordering the test relies on.
• Reporting only a p-value without checking the tails. The principal advantage of this test over Kolmogorov-Smirnov is its tail sensitivity-that information should not be discarded by reporting only a summary p-value.

Advantages relative to alternative tests

• More sensitive than the Kolmogorov-Smirnov Test to departures concentrated in the tails.
• Well suited to detecting outliers and heavy-tailed distributions, common in finance and reliability data.
• Can test goodness-of-fit against distributions other than normal, given the appropriate critical value tables.
• Widely implemented and well documented, with established critical value tables going back decades.
• Uses the entire distribution shape rather than a single point of maximum discrepancy.

Limitations

• Sensitivity to outliers is bidirectional. The same tail-weighting that makes the test powerful can also cause it to overreact to a single unusual data point-see Worked Example 2.
• Traditionally reported via critical values, not a single p-value. This can be less convenient than tests that report a p-value directly, though software increasingly offers p-value approximations.
• Requires a correction when parameters are estimated. Omitting the small-sample correction when parameters are fitted from the same sample invalidates the standard critical value comparison-see Common Pitfalls.
• Designed for continuous data. Heavily rounded or discrete data can distort results.
• No quantified effect size. Like other goodness-of-fit tests, \( A^2 \) indicates departure but not a directly interpretable magnitude in the original units of the data.

When an alternative test is preferable

• When tail behavior specifically is not a concern and overall power for normality testing matters most-the Shapiro-Wilk Test is often the more powerful general-purpose choice.
• When a straightforward two-sample comparison is required-the two-sample Kolmogorov-Smirnov Test is more appropriate.
• When the data is discrete, heavily rounded, or contains many tied values.
• When a single, obviously erroneous data-entry outlier is present and has not yet been investigated- suspicious extreme values should be verified before they are allowed to drive a significance decision.
• When distribution parameters were estimated from the sample and the appropriate correction factor has not been applied.

Comparison with related goodness-of-fit tests

Common misconceptions

• "Anderson-Darling always outperforms Kolmogorov-Smirnov." It's generally more powerful for tail-driven departures, but not universally superior-see Comparisons for how the tests actually differ.
• "A significant Anderson-Darling result means the whole distribution is wrong." Often the result is driven mostly by the tails, sometimes by a single outlier-see Why the Tails Get Extra Weight.
• "You can skip the small-sample correction if the sample is reasonably large." The correction factor should be applied whenever parameters are estimated from the sample, regardless of size, though its effect shrinks as \( n \) grows-see Common Pitfalls.
• "A non-significant result proves the data follows the assumed distribution." It only means no significant departure was detected given the sample-especially relevant with small samples that have limited power.
• "The test doesn't produce a p-value, so it's less rigorous." Reporting via critical values is simply a different, well-established convention-approximate p-values are readily available in most modern software when needed.

Interview Questions

1. Explain in your own words what the Anderson-Darling statistic measures, and how it differs from the Kolmogorov-Smirnov statistic.
2. Why does the weighting term \( (2i-1) \) in the formula give the test extra sensitivity to the tails of a distribution?
3. When would you prefer the Anderson-Darling Test over the Kolmogorov-Smirnov Test, and vice versa?
4. Why is a small-sample correction needed when distribution parameters are estimated from the same sample being tested?
5. How would you explain to a colleague why a single outlier caused a large Anderson-Darling statistic even though the rest of the sample looks normal?
6. Why is the Anderson-Darling Test traditionally reported using critical values rather than a single p-value, and how would you interpret a critical value table?
7. How does the Anderson-Darling Test compare to the Shapiro-Wilk Test in terms of statistical power for normality testing specifically?
8. What follow-up analysis would you do after a significant Anderson-Darling result, given that it might be driven by only a few extreme observations?
9. Can the Anderson-Darling Test be used against distributions other than normal? What would you need to change?
10. Describe a real-world scenario, such as in finance or reliability engineering, where the tail sensitivity of this test would be particularly valuable.

Frequently Asked Questions

• What is the Anderson-Darling Test used for? +
  The Anderson-Darling Test is a goodness-of-fit test that checks whether a sample plausibly comes from a specific theoretical distribution, most often the normal distribution. It builds on the same empirical-distribution-function idea as the Kolmogorov-Smirnov Test but weights discrepancies more heavily in the tails, which makes it particularly good at catching outliers and heavy-tailed departures that a test treating every point equally might miss.
• What is the formula for the Anderson-Darling Test statistic? +
  The A-squared statistic is built from the sorted sample values and the theoretical cumulative distribution function evaluated at each one, combined in a weighted sum where the weighting term grows larger near the extremes of the distribution and smaller near the center. This means a sample with a well-fitting middle but a poorly fitting tail will still produce a noticeably elevated A-squared, unlike simpler goodness-of-fit statistics that only look at the single largest gap.
• How do you interpret the Anderson-Darling Test result? +
  Rather than reading off a precise p-value directly, the raw A-squared statistic is typically compared against a table of critical values for common significance levels such as 15%, 10%, 5%, 2.5%, and 1%. If A-squared exceeds the critical value at your chosen significance level (commonly 5%), the sample significantly departs from the assumed distribution; software can also supply an approximate p-value using established formulas for the corrected statistic.
• What is the difference between the Anderson-Darling Test and the Kolmogorov-Smirnov Test? +
  Both compare a sample's empirical distribution function against a theoretical one, but they measure the discrepancy differently. The Kolmogorov-Smirnov Test reports only the single largest vertical gap between the two curves, giving equal importance to every part of the distribution. The Anderson-Darling Test instead sums a weighted discrepancy across the entire range with extra emphasis on the tails, which generally gives it more power to detect departures concentrated at the extremes, such as heavy tails or a small number of outliers.
• What assumptions does the Anderson-Darling Test make? +
  The test assumes the data is continuous and the observations are independent. When testing against a normal distribution whose mean and standard deviation are estimated from the same sample being tested-rather than specified in advance-a small-sample correction factor must be applied to the raw statistic before comparing it against the standard critical value table, since estimating those parameters from the data changes the statistic's sampling distribution.
• Why does the Anderson-Darling Test weight the tails more heavily? +
  Many practically important departures from a target distribution-a single large outlier, a heavier-than-normal tail, or a skewed extreme-show up most clearly at the far ends of the distribution rather than near its center. By weighting the squared discrepancy between the sample and the theoretical distribution more heavily as it approaches the tails, the Anderson-Darling Test is built to be more sensitive to exactly this kind of departure than a test that treats every point along the distribution equally.

Key Takeaways

• The Anderson-Darling Test compares a sample's empirical distribution function against a theoretical one, weighting discrepancies near the tails more heavily than discrepancies near the center.
• The statistic \( A^2 = -n - \frac{1}{n}\sum(2i-1)[\ln(z_i)+\ln(1-z_{n+1-i})] \) is compared against tabulated critical values, often after a small-sample correction when parameters are estimated from the same sample.
• It is more sensitive than the Kolmogorov-Smirnov Test to outliers and heavy-tailed departures, making it well suited to finance, reliability, and other fields where extreme values matter.
• A significant result should be followed up by checking the extreme values directly, since a single outlier can dominate the statistic-see Worked Example 2.
• In Python, scipy.stats.anderson returns the raw statistic and critical values directly, following the test's traditional reporting convention.
• For normality testing specifically, without a particular focus on tail behavior, the Shapiro-Wilk Test is often more powerful overall.
• A non-significant result never proves the distribution matches-it only means no significant departure was detected given the sample size and data at hand.

Conclusion

The Anderson-Darling Test addresses a question distinct from simple goodness-of-fit: not merely whether a sample's distribution differs from what is expected, but whether that difference is concentrated in the tails, where a single extreme observation or a heavier-than-expected spread is often of greatest practical consequence.

The two worked examples illustrate this directly: a nearly-normal sample produced a statistic well below the critical threshold, while a single dominant outlier drove the same statistic well past it, using identical code and an identical formula.

The small-sample correction should be applied whenever parameters are estimated from the data, and the observations driving a significant result should be checked before conclusions are drawn. Where tail behavior is not the primary concern, Shapiro-Wilk remains the more powerful choice for normality testing alone.