Kruskal-Wallis Test: When, Why, and How to Use It

Introduction

The Kruskal-Wallis Test checks whether three or more independent groups differ significantly from one another, without assuming those groups are normally distributed. If you are comparing three or more separate treatment arms, product versions, or manufacturing processes-different subjects in each group-but your sample is small, skewed, or has outliers that make a one-way ANOVA risky, this is usually the test you are looking for.

By the end of this article you will be able to state exactly when the Kruskal-Wallis Test applies, compute one completely by hand on two different worked examples, follow up a significant result with Dunn's post-hoc test, interpret everything alongside an effect size, and run the same test in a few lines of Python with scipy.stats.kruskal.

What Is the Kruskal-Wallis Test?

The Kruskal-Wallis Test compares three or more groups made up of different, unrelated subjects-just like one-way ANOVA. The difference is in how it uses the data: instead of comparing the group means directly, it pools every observation from all groups, ranks them together, and asks whether the resulting rank sums are balanced across groups, or whether one or more groups systematically dominate.

It exists because real between-group data does not always behave-distributions can be skewed, contain outliers, violate the equal-variance assumption, or simply come from too small a sample to trust normality. By working with ranks rather than raw values, the test trades a small amount of statistical power (when the data genuinely are normal and homoscedastic) for considerable robustness to violations of those assumptions. It is, in effect, the natural extension of the Mann-Whitney U Test to more than two groups.

When to Use It

Use the Kruskal-Wallis Test when you are comparing three or more separate, independent groups and either cannot verify normality within each group, know it does not hold, or are working with a small or ordinal dataset. The key requirement is the same as one-way ANOVA: the groups are independent-no subject appears in more than one group.

Key Assumptions

• Independence between and within groups: no subject appears in more than one group, and observations within each group do not influence one another.
• At least ordinal data: the values must be capable of being meaningfully ranked.
• Similar distribution shape (for a median interpretation): a significant result is cleanly interpretable as "the medians differ" only when the groups' distributions have roughly the same shape and spread-otherwise the result is better described as a difference in overall distribution.

Hypotheses

The Kruskal-Wallis Test formally tests whether the \( k \) independent groups share the same distribution:

• Null Hypothesis (\( H_0 \)): all \( k \) groups come from the same distribution-any observed differences in rank sums are due to chance, i.e., the groups have no real effect on the outcome.
• Alternative Hypothesis (\( H_1 \)): at least one group's distribution differs systematically from at least one other group's-i.e., there is a real effect somewhere among the \( k \) groups.

(The Kruskal-Wallis Test is inherently omnibus-like a one-way ANOVA's F-test, a significant result tells you a difference exists somewhere, not which specific pair of groups differs. That question is answered by Dunn's post-hoc test.)

The Formula, Explained

For \( k \) independent groups with sizes \( n_1, n_2, \dots, n_k \) and total sample size \( N = \sum_{j=1}^{k} n_j \), the procedure is:

1. Combine all \( N \) observations across every group and rank them together from 1 (smallest) to \( N \) (largest), assigning the average rank to any tied values.
2. Sum the ranks for each group \( j \) to obtain the rank sum \( R_j = \sum \text{rank}(x_{ij}) \) for all observations \( i \) in group \( j \).
3. Compare how spread out the \( R_j \) values are-scaled by group size-relative to what would be expected if every group were equally likely to receive any rank.

The test statistic is:

Under \( H_0 \), \( H \) follows approximately a chi-square distribution with \( k - 1 \) degrees of freedom:

When ties occur across groups, a tie correction is applied to the denominator, dividing \( H \) by \( 1 - \dfrac{\sum (t^3 - t)}{N^3 - N} \), where \( t \) is the number of tied observations in each tied group of values. For small samples, exact Kruskal-Wallis tables are preferred over the chi-square approximation, which can be conservative when group sizes are small.

Worked Example 1: A Small Numerical Example by Hand

Suppose a researcher recruits three separate, unrelated groups of 5 adults each to test three different diets and records weight loss (in kg) after 8 weeks-extending the two-diet data used in the Mann-Whitney U Test guide with a third diet, so you can compare both approaches directly.

Step 1: Combine and rank all 15 values

Sorted combined values: 18(C), 19(C), 20(B), 20(C), 21(B), 22(A), 22(C), 23(B), 23(C), 24(A), 24(B), 25(A), 25(B), 27(A), 28(A). The tied 20's (B and C) share ranks 3 and 4, averaging to 3.5; the tied 22's (A and C) share ranks 6 and 7, averaging to 6.5; the tied 23's (B and C) share ranks 8 and 9, averaging to 8.5; the tied 24's (A and B) share ranks 10 and 11, averaging to 10.5; the tied 25's (A and B) share ranks 12 and 13, averaging to 12.5.

Step 2: Sum the ranks for each group

Check: \( R_A + R_B + R_C = 120 = \dfrac{15 \times 16}{2} \), confirming the ranking is correct.

Step 3: Compute H

Step 4: Compare against the critical value

With \( k = 3 \) groups, degrees of freedom \( = k - 1 = 2 \). The chi-square critical value at \( \alpha = 0.05 \) with 2 degrees of freedom is \( \chi^2_{0.05,2} = 5.991 \). Since our computed \( H = 6.845 \) exceeds 5.991, we reject \( H_0 \) at the 5% level-the corresponding p-value works out to approximately 0.0326, below 0.05.

(Note: the H computed above uses the uncorrected formula. With five tied pairs of values, scipy.stats.kruskal applies its standard tie correction and returns a very slightly larger \( H \approx 6.907 \), \( p \approx 0.0316 \)-the same conclusion, just a marginally different exact value.)

Worked Example 2: Comparing Three Training Methods (Unequal Group Sizes)

Suppose 7 employees are trained with a new onboarding method, 6 different employees are trained with an older method, and 6 more employees receive no formal training at all (self-guided)-extending the two-method data used in the Mann-Whitney U Test guide's second example with a third, self-guided group. Note the group sizes are unequal-this is completely fine for the Kruskal-Wallis Test.

Step 1: Combine and rank all 19 values

Sorted combined values: 68(SG), 70(SG), 71(SG), 72(SG), 74(SG), 75(Old), 76(SG), 77(Old), 78(Old), 79(Old), 80(Old), 82(Old), 84(New), 85(New), 88(New), 89(New), 90(New), 91(New), 92(New). There are no ties here, so every rank is a whole number-notice all six Self-Guided values occupy nearly the lowest ranks, and all seven New Method values occupy the highest seven ranks.

Step 2: Sum the ranks for each group

Check: \( R_{\text{New}} + R_{\text{Old}} + R_{\text{SG}} = 190 = \dfrac{19 \times 20}{2} \).

Step 3: Compute H

Step 4: Decision

With \( k = 3 \) groups, degrees of freedom \( = 2 \). The chi-square critical value at \( \alpha = 0.05 \) is again \( 5.991 \). Since our computed \( H \approx 15.64 \) is far greater than this critical value, we reject \( H_0 \) decisively. The corresponding p-value is approximately 0.0004, far below 0.05-the three training approaches produce clearly different outcomes, consistent with the strong separation visible in the ranks above.

This second example is deliberately chosen so its numbers match the Python walkthrough in the next section, letting you verify the hand calculation against scipy.stats.kruskal directly.

Effect Size (Epsilon-Squared)

A p-value tells you whether an effect is likely to be real; it says nothing about how large that effect is. For the Kruskal-Wallis Test, a common effect size is epsilon-squared (\( \epsilon^2 \)), computed directly from H and the total sample size:

Using Worked Example 2: \( \epsilon^2 = \dfrac{15.64}{19 - 1} = \dfrac{15.64}{18} \approx 0.869 \). This ranges from 0 to 1, with values near 1 indicating the groups are almost completely separated by rank-here, a very large effect, consistent with the near-total separation seen in the ranks above.

Some software instead reports eta-squared (\( \eta^2_H \)), a closely related rank-based effect size computed as \( \eta^2_H = \dfrac{H - k + 1}{N - k} \); both scale from roughly 0 to 1 and are interpreted the same way.

Post-Hoc: Dunn's Test

A significant Kruskal-Wallis result only establishes that at least one group differs from the others-it does not say which pair. The standard follow-up is Dunn's test, which performs all pairwise comparisons using the same combined rank sums already computed for the omnibus test, with a p-value adjustment (commonly Bonferroni or Holm) to control the family-wise error rate across the multiple comparisons being made.

For each pair of groups \( i \) and \( j \), Dunn's test compares the difference in mean ranks \( \bar{R}_i - \bar{R}_j \) (where \( \bar{R}_j = R_j / n_j \)) against a z-score built from the pooled variance of ranks across all \( N \) observations, then adjusts the resulting p-values for the number of pairwise comparisons made.

Applying Dunn's test with a Bonferroni correction to Worked Example 2 (3 pairwise comparisons, so \( \alpha_{\text{adj}} = 0.05 / 3 \approx 0.0167 \)) gives:

The same site also covers the Nemenyi Test, which uses a related rank-based logic but is designed specifically as the post-hoc follow-up to the Friedman Test for repeated-measures data-a Nemenyi variant exists for Kruskal-Wallis designs too, but Dunn's test with a Bonferroni or Holm correction remains the more commonly reported choice for independent-groups designs in most software.

Python Example

You can run this test in one line using scipy.stats.kruskal:

from scipy import stats
import numpy as np

new_method = [85, 90, 88, 92, 84, 91, 89]
old_method = [78, 82, 75, 80, 79, 77]
self_guided = [70, 74, 68, 72, 76, 71]

h_stat, p_value = stats.kruskal(new_method, old_method, self_guided)

print(f"H-statistic: {h_stat:.3f}")
print(f"p-value: {p_value:.4f}")

# Effect size: epsilon-squared
N = len(new_method) + len(old_method) + len(self_guided)
epsilon_sq = h_stat / (N - 1)
print(f"Epsilon-squared: {epsilon_sq:.3f}")

Output:

H-statistic: 15.639
p-value: 0.0004
Epsilon-squared: 0.869

These figures match Worked Example 2 closely (small differences come from rounding in the hand calculation). For the post-hoc comparisons, scikit-posthocs provides Dunn's test directly:

import scikit_posthocs as sp
import pandas as pd

data = (
    [("New", v) for v in new_method]
    + [("Old", v) for v in old_method]
    + [("Self-Guided", v) for v in self_guided]
)
df = pd.DataFrame(data, columns=["group", "score"])

dunn_result = sp.posthoc_dunn(df, val_col="score", group_col="group", p_adjust="bonferroni")
print(dunn_result)

This returns a matrix of Bonferroni-adjusted pairwise p-values, matching the table in the post-hoc section above. By default, scipy.stats.kruskal applies the standard tie correction automatically whenever tied values are present across groups.

Checking Distribution Shape

Before interpreting a significant result as a difference in medians specifically, it is good practice to visually compare the shape of the groups' distributions:

import matplotlib.pyplot as plt

plt.boxplot([new_method, old_method, self_guided],
            tick_labels=["New Method", "Old Method", "Self-Guided"])
plt.title("Distribution comparison by training method")
plt.show()
# Substantially different spreads or shapes across boxplots suggest
# interpreting the result strictly as a median difference may be misleading.

With only 19 total observations spread across three groups, a formal shape-comparison test has very low power-in practice, small between-group studies like this are usually evaluated with a visual check rather than relying on a formal statistical test alone.

How to Interpret Results

The significance level \( \alpha = 0.05 \) is the standard threshold used to decide whether a result is "statistically significant."

Checking Assumptions in Practice

• Independence between and within groups: a design question, not something visible in the data itself-confirm no subject appears in more than one group and observations don't influence one another.
• Distribution shape: a set of boxplots or histograms comparing all groups-substantially different shapes or spreads complicate interpreting a significant result strictly as a median difference.
• Ties: check how many tied values span across groups; they receive averaged ranks and typically require a tie correction to the H statistic.
• Group sizes: unequal group sizes are handled naturally by the formula, but very small groups (roughly \( n_j < 5 \)) reduce the reliability of the chi-square approximation-exact Kruskal-Wallis tables are preferred in that case.

Advantages

• Does not require any group's data to be normally distributed, making it reliable with small or skewed samples.
• Does not require homogeneity of variance across groups, unlike one-way ANOVA.
• Much less sensitive to outliers than one-way ANOVA, since a single extreme value can shift its rank by only one position.
• Works with ordinal data (e.g., Likert-scale ratings), where ANOVA's mean-based logic is questionable.
• Handles unequal group sizes naturally, exactly like one-way ANOVA.

Limitations

• Generally less statistically powerful than one-way ANOVA when all groups genuinely are normally distributed with equal variances, since it discards exact magnitude information.
• A significant result is only cleanly interpretable as a difference in medians when all distributions have a similar shape-under differing shapes, the result is better described as evidence that the distributions differ overall, not necessarily in central tendency alone.
• A significant omnibus result does not identify which specific groups differ-a post-hoc test such as Dunn's test is required for that.
• Large numbers of tied values reduce the test's effective resolution and require a tie correction to the H statistic.

When NOT to Use It

• One-way ANOVA: use instead when all groups' data are approximately normally distributed with roughly equal variances-it offers somewhat higher statistical power in that case.
• Mann-Whitney U Test: use instead when you only have two independent groups rather than three or more.
• Friedman Test: use instead when the groups are actually related-the same subjects measured under every condition-rather than separate, independent groups.
• Dunn's test (or Nemenyi): use as a follow-up, not a replacement-run it only after the Kruskal-Wallis Test itself returns a significant result, to identify which specific pairs differ.

Kruskal-Wallis vs One-Way ANOVA vs Mann-Whitney vs Friedman

17.1 Kruskal-Wallis Test vs One-Way ANOVA

17.2 Kruskal-Wallis Test vs Mann-Whitney U Test

17.3 Kruskal-Wallis Test vs Friedman Test

Common Misconceptions

• "The Kruskal-Wallis Test has no assumptions at all." Not quite-it still requires independent groups and at least ordinal data; what it relaxes specifically is the normality and equal-variance assumptions of one-way ANOVA.
• "A significant Kruskal-Wallis result tells me which group is best." No. The omnibus test only tells you a difference exists somewhere-see Dunn's test section, where one of the three pairwise comparisons in Worked Example 2 narrowly missed significance despite the overall test being highly significant.
• "The Kruskal-Wallis Test and the Friedman Test are interchangeable." They are related but distinct-Kruskal-Wallis assumes independent groups, while the Friedman Test specifically requires the same subjects measured under every condition.
• "This test works for exactly two groups too." Mathematically yes-the formula reduces to a monotonic function of the Mann-Whitney U statistic at \( k = 2 \)-but in practice the Mann-Whitney U Test is used directly for the two-group case.
• "A non-significant result proves there is no effect." Failing to reject \( H_0 \) only means there was not enough evidence to detect an effect with this sample-it does not prove the true effect is zero.

Interview Questions

1. Walk through the ranking procedure of the Kruskal-Wallis Test and explain why all groups are ranked together rather than within each group separately.
2. Why is the Kruskal-Wallis Test generally more robust to outliers than one-way ANOVA?
3. What does it mean for the Kruskal-Wallis Test to be an "omnibus" test, and why is a post-hoc test needed after a significant result?
4. Given a dataset with several tied ranks spanning multiple groups, how are ranks assigned, and how does this affect the chi-square approximation of \( H \)?
5. Explain the relationship between the Kruskal-Wallis Test and the Mann-Whitney U Test at \( k = 2 \) groups.
6. How would you compute and interpret epsilon-squared for a Kruskal-Wallis Test result?
7. What does Dunn's test add beyond the Kruskal-Wallis p-value alone, and how does it control for multiple comparisons?
8. Under what circumstances would you switch from a Kruskal-Wallis Test back to one-way ANOVA?
9. Why does scipy.stats.kruskal(new, old, self_guided) allow the input arrays to have different lengths?
10. What happens to statistical power if you mistakenly apply a Kruskal-Wallis Test to normally distributed, homoscedastic independent-group data instead of a one-way ANOVA?

Frequently Asked Questions

• What is the Kruskal-Wallis Test used for? +
  The Kruskal-Wallis Test is used to check whether three or more independent groups-such as several treatment arms, several product versions, or several manufacturing processes, each tested on different subjects-differ significantly from one another, without assuming the data follow a normal distribution.
• What is the formula for the Kruskal-Wallis Test? +
  Combine every observation from all k groups and rank them together from smallest to largest, averaging tied ranks. Sum the ranks for each group j to get R_j, then compute H = [12 / (N(N+1))] · Σ (R_j² / n_j) − 3(N+1), where N is the total sample size across all groups. Under H₀, H follows approximately a chi-square distribution with k − 1 degrees of freedom.
• What is the difference between the Kruskal-Wallis Test and one-way ANOVA? +
  One-way ANOVA compares the means of three or more independent groups and requires each group to be approximately normally distributed with roughly equal variances. The Kruskal-Wallis Test compares groups using only the ranks of the combined observations, discarding exact magnitude information, which makes it more robust to outliers and non-normality but slightly less powerful when the parametric assumptions genuinely hold.
• What are the assumptions of the Kruskal-Wallis Test? +
  The groups should be independent of one another, observations within each group should be independent, and the data should be at least ordinal. No assumption of normality or equal variance is required, which is the test's main advantage over one-way ANOVA. A significant result is cleanly interpretable as a difference in medians only when the groups' distributions have a similar shape.
• How do you interpret a Kruskal-Wallis Test p-value? +
  If the p-value is below your chosen significance level (commonly α = 0.05), you reject the null hypothesis and conclude that at least one group differs significantly from the others. If the p-value is at or above α, there is not enough evidence that the groups differ. A significant Kruskal-Wallis result only tells you a difference exists somewhere among the k groups-it does not say which specific pairs differ, which is why a post-hoc test such as Dunn's test is typically run next.
• What should I use instead of the Kruskal-Wallis Test if my data are approximately normal? +
  Use one-way ANOVA, the parametric counterpart of the Kruskal-Wallis Test. It uses the actual magnitude of each observation rather than just its rank, which gives it somewhat higher statistical power when normality and homogeneity of variance genuinely hold.
• What post-hoc test follows a significant Kruskal-Wallis Test? +
  Dunn's test is the standard post-hoc procedure-it performs all pairwise comparisons between groups using the same combined rank sums computed for the Kruskal-Wallis Test, with a p-value adjustment (commonly Bonferroni or Holm) to control the family-wise error rate across the multiple comparisons being made.
• How many independent groups does the Kruskal-Wallis Test require? +
  At least three. With exactly two independent groups, the Kruskal-Wallis Test formula still works mathematically and reduces to the Mann-Whitney U Test-in fact, the Kruskal-Wallis H statistic and the Mann-Whitney U statistic are mathematically related in the two-group case, so the Mann-Whitney U Test is used in practice for two groups.

Key Takeaways

• The Kruskal-Wallis Test compares three or more separate, independent groups by ranking all observations together and comparing how those ranks split across groups-a non-parametric counterpart to one-way ANOVA.
• It is ideal for multi-group treatment or A/B/C comparisons where normality or equal variance cannot be assumed, the sample is small, or the data are ordinal.
• Core assumptions: independence between and within groups, and at least ordinal data-no normality or equal variance is required; a similar distribution shape is needed to interpret the result as a difference in medians.
• In Python, scipy.stats.kruskal(...) gets you the H-statistic and p-value in one call; pair it with epsilon-squared and, when significant, Dunn's post-hoc test for a complete picture.
• If \( p < 0.05 \), at least one group differs significantly; if \( p \geq 0.05 \), there isn't enough evidence of a real difference-but a significant omnibus result alone never tells you which pair differs, as both worked examples above demonstrate.
• If all your groups are approximately normal with equal variances, one-way ANOVA will generally be more powerful; if you only have two groups, use the Mann-Whitney U Test instead; if the same subjects were measured across all conditions, use the Friedman Test instead.

The Kruskal-Wallis Test earns its place as the standard non-parametric tool for independent-groups designs precisely because real between-group data rarely arrives with guaranteed normality and equal variance-small clinical pilots, ordinal survey scales across multiple product versions, and skewed measurement distributions are common, and each one puts one-way ANOVA's core assumptions at risk. Its logic extends the Mann-Whitney U Test's philosophy from two groups to three or more: pool every group's observations, rank them together, then ask whether the resulting rank sums are too uneven across groups to be explained by chance.

The two worked examples above, built on the same diet and training-method data used in the Mann-Whitney U Test guide, show both the omnibus test and its natural follow-up in practice-a small five-per-group dataset that reached significance once a third diet sharpened the overall pattern, and a larger three-method training dataset where Dunn's post-hoc test pinpointed exactly which pairs of methods actually differed. Reporting the p-value alongside epsilon-squared and, whenever the result is significant, a proper post-hoc comparison, as demonstrated throughout this guide, gives a complete and honest picture that the omnibus p-value by itself cannot.