Bartlett's Test

Introduction

Bartlett's Test checks whether two or more normally distributed groups have equal variances-a property called homogeneity of variance-or whether some groups are genuinely more spread out than others. If you are asking questions like "do these three production batches vary by about the same amount, assuming their measurements are roughly normal?", "is it safe to run a standard ANOVA on this data, or do I need a variance-robust version instead?", or "which equal-variance test gives me the most statistical power when I'm confident my data are normal?"-this is usually the test you are looking for.

By the end of this article you will be able to state exactly when Bartlett's Test applies, compute the statistic completely by hand on two different worked examples, understand the correction factor that distinguishes it from a plain chi-square calculation, interpret the result correctly (including why normality matters so much here), know what to do if unequal variances are detected, and run the same test in one line of Python with scipy.stats.

What Is Bartlett's Test?

Many classical procedures-the independent-samples t-test, one-way ANOVA, and various regression diagnostics-assume that the groups being compared have the same variance. Introduced by Maurice Bartlett in 1937, Bartlett's Test was one of the earliest formal ways to check this assumption, built directly from the theory of the normal distribution rather than relying on eyeballing box plots.

The test works by comparing each group's own sample variance against a single pooled variance computed by combining all the groups together. If the groups truly share a common variance, each individual group's variance should be a reasonably close estimate of that shared pooled value, and the resulting test statistic-built from the logarithms of these variances-should stay small. If one or more groups are genuinely more variable than the others, their sample variance will diverge from the pooled estimate, inflating the statistic.

It exists because unequal variances-called heteroskedasticity when discussed across groups-can seriously distort the standard errors used by a t-test or ANOVA, making the reported p-values untrustworthy even when the actual group means are correctly estimated. Catching this before trusting a comparison of means is the entire purpose of running Bartlett's Test as a preliminary check, provided the normality assumption it relies on is credible.

When to Use It

• Before an independent-samples t-test: to decide whether the standard equal-variance t-test is appropriate, when you are confident the data within each group are approximately normal.
• Before a one-way ANOVA: to check the homogeneity-of-variance assumption underlying the standard F-test comparing several group means, again when normality is well established.
• Quality control with approximately normal measurements: classic use cases include comparing measurement consistency across instruments, machines, or production batches where the underlying process is known to be roughly normal.
• When maximum statistical power matters and normality is credible: Bartlett's Test is more powerful than Levene's Test at detecting real differences in variance specifically when the normality assumption holds.
• Two or more groups: like Levene's Test, Bartlett's Test naturally extends to any number of groups \( k \geq 2 \), unlike the two-sample F-test for variances.

Key Assumptions

• Normally distributed data within each group: this is the single most important assumption-Bartlett's Test is built directly on normal-distribution theory and is known to be sensitive to violations of it, discussed further under Limitations.
• Independent observations: observations within and across groups must be independent of one another.
• Two or more groups: the test compares variances across \( k \geq 2 \) independent groups.
• Continuous data: the measured variable should be continuous so that a sample variance is a meaningful summary of spread.
• Reasonably sized groups: the chi-square approximation underlying the test statistic improves with larger group sizes-very small groups (e.g., fewer than 5 observations) make the approximation less reliable, even under normality.

Hypotheses

Bartlett's Test formally tests whether the population variances are equal across all groups:

• Null Hypothesis (\( H_0 \)): all group variances are equal, \( \sigma_1^2 = \sigma_2^2 = \cdots = \sigma_k^2 \) (homogeneity of variance).
• Alternative Hypothesis (\( H_1 \)): at least one group variance differs from the others.

(As with Levene's Test, the null hypothesis here is the "convenient" outcome-you generally want to fail to reject \( H_0 \) so that a standard t-test or ANOVA remains valid. The key difference is that Bartlett's conclusion is only trustworthy to the extent the normality assumption itself holds.)

The Formula, Explained

Suppose there are \( k \) groups, with group \( i \) containing \( n_i \) observations, sample variance \( s_i^2 \), and \( N = \sum n_i \) total observations. Bartlett's procedure follows three steps:

Step 1. Compute the pooled variance across all groups, weighted by each group's degrees of freedom:

Step 2. Compute the uncorrected test statistic, comparing the log of the pooled variance against the weighted sum of each group's log variance:

Step 3. Divide by the correction factor \( C \) to obtain the final statistic:

Under \( H_0 \) (equal variances) and assuming normally distributed data, this statistic follows a chi-square distribution:

Intuitively: if all groups truly share the same variance, each \( s_i^2 \) should be a reasonably close estimate of the pooled variance \( s_p^2 \), keeping the weighted sum of log variances close to \( (N-k) \ln(s_p^2) \) and \( T \) small. If one or more groups are genuinely more variable, their \( s_i^2 \) departs from \( s_p^2 \), inflating the difference in the numerator and therefore \( T \).

The Correction Factor, Explained

Without the correction factor \( C \), the raw statistic \( \chi^2_{\text{raw}} \) tends to be too large in small samples, making the test reject the null hypothesis more often than the nominal significance level would suggest-even when the variances truly are equal. Bartlett derived \( C \) specifically to correct this small-sample bias:

Notice that \( C \) is always slightly greater than 1, so dividing by it always shrinks the raw statistic somewhat, correcting for the fact that \( \chi^2_{\text{raw}} \) alone is a biased estimator in finite samples. As every \( n_i \) grows large, \( C \) approaches 1, and the correction becomes negligible-which is why some textbook presentations of very-large-sample Bartlett's Test omit it, though virtually all modern software applies it by default regardless of sample size.

Worked Example 1: A Small Numerical Example by Hand

Suppose two groups of 5 observations each are measured, and we want to check whether they share a common variance:

Step 1: Compute each group's sample variance

Group A mean \( = 11 \), sample variance \( s_A^2 = 2.5 \). Group B mean \( = 12 \), sample variance \( s_B^2 = 21.0 \) (using the unbiased \( n - 1 \) divisor for both).

Step 2: Compute the pooled variance

With \( n_A = n_B = 5 \), \( N = 10 \), \( k = 2 \):

Step 3: Compute the raw chi-square statistic

Step 4: Compute the correction factor and final statistic

Step 5: Compare against the critical value

With \( k - 1 = 1 \) degree of freedom, the critical value at \( \alpha = 0.05 \) is \( \chi^2_{1,0.05} = 3.841 \). Since \( T = 3.44 < 3.841 \), we fail to reject \( H_0 \): there is not enough evidence that the two groups have different variances-even though Group B's sample variance is roughly 8.4 times larger than Group A's, the small sample size limits the test's ability to detect the difference with confidence.

Worked Example 2: Yield Consistency Across Three Fertilizers

An agricultural researcher compares crop yield consistency (measured in normally distributed yield units) across three fertilizer treatments, each tried on 6 plots, suspecting that one fertilizer may produce far more variable yields than the other two.

Step 1: Compute each group's sample variance

Fertilizer 1: mean \( = 49.33 \), \( s_1^2 \approx 4.67 \). Fertilizer 2: mean \( = 51.67 \), \( s_2^2 \approx 4.67 \). Fertilizer 3: mean \( = 49.67 \), \( s_3^2 \approx 96.27 \).

Step 2: Compute the pooled variance

With \( n_1 = n_2 = n_3 = 6 \), \( N = 18 \), \( k = 3 \):

Step 3: Compute the raw chi-square statistic

Step 4: Compute the correction factor and final statistic

Step 5: Compare against the critical value

With \( k - 1 = 2 \) degrees of freedom, the critical value at \( \alpha = 0.05 \) is \( \chi^2_{2,0.05} = 5.991 \). Since \( T = 13.97 \gg 5.991 \), we reject \( H_0 \): the three fertilizer treatments do not share a common variance-Fertilizer 3 produces far more variable yields than the other two, exactly as the researcher suspected, with a p-value below \( 0.001 \).

Python Example

scipy.stats computes Bartlett's Test directly from raw group data with bartlett:

import numpy as np
from scipy import stats

# Worked Example 2 data: crop yields by fertilizer
fertilizer_1 = np.array([48, 51, 46, 50, 49, 52])
fertilizer_2 = np.array([52, 49, 55, 51, 53, 50])
fertilizer_3 = np.array([40, 58, 45, 62, 38, 55])

stat, p_value = stats.bartlett(fertilizer_1, fertilizer_2, fertilizer_3)

print(f"Bartlett's T statistic: {stat:.3f}")
print(f"p-value: {p_value:.6f}")

Output:

Bartlett's T statistic: 13.968
p-value: 0.000925

This matches Worked Example 2 closely (small differences from rounding during the hand calculation). scipy.stats.bartlett applies the correction factor \( C \) automatically, so no extra step is needed to get the standard corrected statistic.

Checking Normality First

Since Bartlett's Test result is only trustworthy when the normality assumption holds, it is good practice to check normality within each group before relying on it:

from scipy.stats import shapiro

for name, group in zip(["Fertilizer 1", "Fertilizer 2", "Fertilizer 3"],
                        [fertilizer_1, fertilizer_2, fertilizer_3]):
    stat, p = shapiro(group)
    print(f"{name}: Shapiro-Wilk p-value = {p:.4f}")

# If any group's normality is doubtful, prefer Levene's Test instead:
levene_stat, levene_p = stats.levene(fertilizer_1, fertilizer_2, fertilizer_3, center='median')
print(f"Levene's Test (as a robustness check): W = {levene_stat:.3f}, p = {levene_p:.4f}")

How to Interpret Results

The significance level \( \alpha = 0.05 \) is the standard threshold used to decide whether unequal variances are "statistically detected."

Checking Assumptions in Practice

• Check normality within each group first: use a Shapiro-Wilk test or Q-Q plot per group before trusting Bartlett's result-this single assumption drives most of the test's known weaknesses, as covered in Limitations.
• Plot box plots or histograms per group: a quick visual comparison of spread often reveals unequal variance before running any formal test-both worked examples above would show this pattern clearly.
• Cross-check with Levene's Test: running both tests side by side is a common and sensible robustness check-if they agree, you can be more confident in the conclusion; if they disagree, suspect non-normality is driving the difference.
• Watch group sizes: very small or highly unequal group sizes make the chi-square approximation less reliable, even when normality holds-treat borderline p-values cautiously in that case.
• Don't treat the test as a strict gatekeeper: given how well Welch's t-test and Welch's ANOVA perform under both equal and unequal variances, many modern statisticians recommend using them directly rather than conditioning the choice of test purely on a Bartlett's Test result.

What to Do If Unequal Variances Are Detected

• Use Welch's t-test instead of Student's t-test: the most common and simplest fix for two groups-it adjusts the degrees of freedom to remain valid without assuming equal variances, as shown in the Python Example.
• Use Welch's ANOVA instead of the standard one-way ANOVA: for three or more groups, Welch's ANOVA provides the same variance-robust adjustment.
• Re-check the result with Levene's Test: if the two tests disagree, the non-normality sensitivity discussed in Limitations is a likely culprit-trust Levene's Test more in that case.
• Apply a variance-stabilizing transformation: a log or square-root transform of the outcome variable can sometimes equalize variances across groups, particularly for count or money-based data.
• Consider a non-parametric alternative: the Kruskal-Wallis Test compares group distributions using ranks and can be a reasonable alternative when both variances and normality are in doubt.

Advantages

• More statistically powerful than Levene's Test at detecting real differences in variance, specifically when the normality assumption holds.
• Built on well-established normal-distribution theory, giving it a long history of use and a simple, closed-form chi-square approximation.
• Naturally extends to any number of groups \( k \geq 2 \), not just two, unlike the classical F-test for variances.
• The correction factor \( C \) makes the test reasonably accurate even in moderate sample sizes, not just asymptotically.
• Widely implemented across statistical software, making results easy to compute and compare across studies and fields.

Limitations

• Highly sensitive to non-normality: this is by far the most cited weakness-heavy-tailed or skewed data can make the test reject the null hypothesis far too often, even when variances are truly equal, an issue traced back to Box's classic 1953 analysis.
• Like other chi-square-based tests, the chi-square reference distribution is approximate-can be less reliable in very small samples even under normality.
• Does not, by itself, indicate which groups differ in variance when there are more than two groups-only that at least one does.
• A significant result does not tell you how to fix unequal variances-only that they are present; see What to Do If Unequal Variances Are Detected.
• Requires an extra normality-checking step before the result can be trusted, adding a layer of complexity that Levene's Test largely avoids.

When NOT to Use It

• Levene's Test: use instead whenever normality is doubtful or unverified-it is far more robust to non-normal data at only a modest cost in power when normality does hold.
• F-test for two variances: use instead only when comparing exactly two groups under a strict normality assumption and you specifically want the classical two-sample variance-ratio test.
• Welch's t-test or Welch's ANOVA directly: use instead of running Bartlett's Test at all if you'd rather skip the variance-equality question entirely and use a test that performs well under both equal and unequal variances.
• Paired or repeated-measures data: use a different variance-comparison approach when observations within groups are not independent, since Bartlett's Test assumes independent samples.
• Very small samples with uncertain normality: consider skipping formal variance testing altogether and using a variance-robust test by default, since neither the normality check nor the chi-square approximation are reliable at very small \( n \).

Bartlett's vs Levene's vs F-Test

17.1 Bartlett's Test vs Levene's Test

17.2 Bartlett's Test vs Classical F-Test for Two Variances

17.3 Bartlett's Test vs Brown-Forsythe (Levene's Median Version)

Common Misconceptions

• "A significant Bartlett's Test always means the variances are truly unequal." Not necessarily-if the data are non-normal, a significant result may reflect that non-normality rather than a genuine variance difference; cross-check with Levene's Test.
• "Bartlett's Test and Levene's Test always give the same answer." Not necessarily-their differing sensitivity to non-normality can make them disagree, especially with skewed or heavy-tailed samples.
• "A non-significant Bartlett's result proves the variances are exactly equal." Failing to reject \( H_0 \) only means there was not enough evidence against equal variances with this sample-it does not prove homogeneity holds exactly.
• "Bartlett's Test doesn't need any assumptions about the data's shape." It very much does-normality is central to the test's validity, unlike Levene's Test, as covered in Key Assumptions.
• "You should always default to Bartlett's Test since it's more powerful." Only when normality is confirmed-otherwise its inflated false-positive rate under non-normal data can outweigh the power advantage entirely.

Interview Questions

1. Describe the three steps of Bartlett's procedure, from group variances to the final test statistic.
2. Why does Bartlett's Test use the logarithm of each group's variance rather than the raw variances themselves?
3. Explain the purpose of the correction factor \( C \), and what happens to it as sample sizes grow large.
4. Why is Bartlett's Test more sensitive to non-normality than Levene's Test?
5. If Bartlett's Test and Levene's Test disagree on the same dataset, which would you trust more, and why?
6. How would you decide whether to use Bartlett's Test or Levene's Test before comparing two group means?
7. What does it mean, in practice, if Bartlett's Test is significant but the group means still need to be compared?
8. Compare Bartlett's Test to the classical two-sample F-test for variances-when would you use each?
9. Why might a very small or highly unequal sample size across groups make Bartlett's Test results unreliable, even under normality?
10. Describe a practical workflow for checking the equal-variance assumption before running a one-way ANOVA, including where Bartlett's Test fits in.

Frequently Asked Questions

• What is Bartlett's Test used for? +
  Bartlett's Test checks whether two or more independent, normally distributed groups have the same population variance-for example, whether crop yield variability is the same across three fertilizer treatments, or whether measurement spread is the same across several instruments-a check commonly run before trusting a t-test or ANOVA that assumes equal variances.
• What is the formula for Bartlett's Test? +
  Compute the sample variance of each group and a pooled (weighted average) variance across all groups. The test statistic compares the log of the pooled variance, weighted by total degrees of freedom, against the sum of each group's log variance weighted by its own degrees of freedom, then divides by a correction factor C that adjusts for small-sample bias. The resulting statistic follows a chi-square distribution with k-1 degrees of freedom under the null hypothesis of equal variances, where k is the number of groups.
• How do you interpret a Bartlett's Test p-value? +
  If the p-value is below your chosen significance level (commonly alpha = 0.05), you reject the null hypothesis and conclude the groups have significantly different variances. If the p-value is at or above alpha, there is not enough evidence against equal variances, meaning it is reasonable to proceed with a standard t-test or ANOVA that assumes homogeneity of variance-provided the normality assumption underlying Bartlett's Test itself is credible.
• What is the difference between Bartlett's Test and Levene's Test? +
  Bartlett's Test works directly with group variances using a formula derived under normal-distribution theory, giving it noticeably more statistical power when the data really are normal. Levene's Test instead converts the data into absolute deviations from each group's center and runs an ordinary ANOVA on those deviations, which sacrifices some power under normality but is much less likely to falsely flag unequal variances when the data are not normally distributed.
• Why is Bartlett's Test sensitive to non-normality? +
  The test statistic is built from the natural logarithm of each group's sample variance, and sample variances are strongly affected by the tail behavior (kurtosis) of the underlying distribution, not just its true spread. With heavy-tailed or skewed data, sample variances can differ substantially by chance alone even when the population variances are equal, which inflates Bartlett's false-positive rate-a well-documented issue traced back to Box (1953).
• What should I do if Bartlett's Test detects unequal variances? +
  Switch to a variance-robust alternative such as Welch's t-test (instead of Student's t-test) or Welch's ANOVA (instead of the standard one-way ANOVA), both of which remain valid without assuming equal variances. It is also worth checking normality first with a test like Shapiro-Wilk or a Q-Q plot, since a significant Bartlett's result driven by non-normal data rather than truly unequal variances should be interpreted with Levene's Test instead.

Key Takeaways

• Bartlett's Test checks whether two or more normally distributed groups have equal variances (homogeneity of variance), a common assumption behind the t-test and ANOVA.
• It works by comparing each group's log variance against the log of a pooled variance, corrected by a small-sample factor \( C \), producing a chi-square statistic \( T \).
• Bartlett's Test is more powerful than Levene's Test when normality genuinely holds, but far more sensitive to violations of that assumption.
• Always pair Bartlett's Test with a normality check (e.g., Shapiro-Wilk) and consider cross-checking with Levene's Test, especially when results are borderline.
• In Python, scipy.stats.bartlett computes the corrected test statistic directly from raw group data in a single line.
• If unequal variances are detected, the simplest fix is usually switching to Welch's t-test or Welch's ANOVA, as outlined in What to Do If Unequal Variances Are Detected.
• If normality is doubtful, prefer Levene's Test instead, since it delivers similar protection with far less sensitivity to non-normal data.

Bartlett's Test remains a foundational tool in classical statistics because, when its normality assumption genuinely holds, it offers more statistical power than any of its more robust alternatives at detecting real differences in variance. By comparing the log of each group's variance against a shared pooled estimate, the test reduces a subtle question about spread into a compact, well-understood chi-square calculation that has remained in standard use since 1937.

The two worked examples above showed the same underlying logic from two angles: a small, hand-calculated two-group case where the difference in variance wasn't quite strong enough to be statistically significant, and a three-group agricultural comparison where one fertilizer's much higher yield variability was detected clearly. Reporting the corrected \( T \) statistic alongside an explicit normality check, and switching to Levene's Test or Welch's variance-robust procedures whenever normality is in doubt, gives a reliable foundation for trusting a comparison of group means.