Dunn's Post-Hoc Test: When, Why, and How to Use It

Introduction

Dunn's Post-Hoc Test answers the question a significant Kruskal-Wallis Test deliberately leaves open: which specific pair of independent groups actually differs? The Kruskal-Wallis Test is an omnibus test-it tells you a difference exists somewhere among three or more independent groups, but not where. Dunn's Test picks up exactly where the Kruskal-Wallis Test leaves off, using the same combined ranks to compare every pair of groups directly.

By the end of this article you will be able to state exactly when Dunn's Test applies, compute one completely by hand on two worked examples that continue directly from the Kruskal-Wallis Test guide, understand the difference between Bonferroni and Holm p-value adjustment, and run the same comparison in a few lines of Python with scikit-posthocs.

What Is Dunn's Post-Hoc Test?

Dunn's Post-Hoc Test compares every possible pair of groups from an independent-groups design-the same one analyzed by the Kruskal-Wallis Test-using the mean rank each group received across the combined, pooled sample. Rather than running a separate, uncorrected test for each pair (which would inflate the chance of a false positive as the number of comparisons grows), it converts each pairwise mean-rank difference into a z-statistic and then adjusts the resulting p-values, commonly with a Bonferroni or Holm correction, so the overall risk of a false pairwise claim stays under control.

It exists because a significant Kruskal-Wallis Test, on its own, is an incomplete answer. Knowing that "the groups are not all the same" is rarely the question a researcher actually cares about-the real question is almost always "which groups differ, and from which others?" Dunn's Test converts the Kruskal-Wallis Test's omnibus signal into a concrete, pair-by-pair verdict.

When to Use It

Use Dunn's Post-Hoc Test immediately after a Kruskal-Wallis Test returns a significant result (\( p < 0.05 \)) on three or more independent groups, and you need to know specifically which groups are responsible for that significance-not just that a difference exists somewhere. It is the natural next step whenever the Kruskal-Wallis Test itself was the right choice: independent groups, normality not assumed, and at least ordinal data.

Key Assumptions

• Prior Kruskal-Wallis Test structure. Dunn's Test operates on the same independent-groups design as the Kruskal-Wallis Test-three or more groups, with observations in each group unrelated to observations in any other group-and reuses the ranking step already performed for that omnibus test.
• Independence within and across groups: no subject appears in more than one group, and observations within a group don't influence one another.
• At least ordinal data: the values must be capable of being meaningfully ranked.
• Significant omnibus result (conventionally): most analysts run Dunn's Test only after the Kruskal-Wallis Test itself rejects \( H_0 \), to avoid stacking an unnecessary layer of correction on top of an already non-significant finding.

Hypotheses

Unlike the single omnibus hypothesis pair tested by the Kruskal-Wallis Test, Dunn's Test evaluates a separate pair of hypotheses for every combination of two groups:

• Null Hypothesis (\( H_0 \)), for each pair \( (i, j) \): groups \( i \) and \( j \) come from the same distribution-any observed difference in their mean ranks is due to chance.
• Alternative Hypothesis (\( H_1 \)), for each pair \( (i, j) \): groups \( i \) and \( j \) come from different distributions-their mean ranks differ systematically.

With \( k \) groups, there are \( \binom{k}{2} \) such pairs to test simultaneously-for example, 3 pairwise comparisons with \( k = 3 \) groups, or 6 comparisons with \( k = 4 \) groups. This is exactly why a p-value adjustment is needed: testing many pairs at the uncorrected \( \alpha = 0.05 \) level would let the family-wise false-positive rate climb well above 5%.

The Formula, Explained

Starting from the same combined ranks used by the Kruskal-Wallis Test, with rank sums \( R_j \) already computed for each group \( j \) of size \( n_j \), out of a total sample size \( N \):

1. Convert each rank sum to a mean rank: \( \bar{R}_j = R_j / n_j \).
2. For every pair of groups \( i \) and \( j \), compute a z-statistic comparing their mean ranks.
3. Convert each z-statistic to a two-tailed p-value, then adjust all the p-values together for the number of comparisons made.

The z-statistic for comparing groups \( i \) and \( j \) is:

This \( z_{ij} \) is compared against the standard normal distribution to obtain a two-tailed p-value. When ties are present across groups, the denominator is adjusted with a standard tie-correction factor, the same correction used in the underlying Kruskal-Wallis \( H \) statistic. The raw p-value for each pair is then passed through a multiple-comparison adjustment-almost never reported unadjusted-before being compared against \( \alpha \).

P-Value Adjustment: Bonferroni vs Holm

With \( m = \binom{k}{2} \) pairwise comparisons computed from the same data, the raw p-values from the z-statistics above cannot be compared directly against \( \alpha = 0.05 \)-doing so would let the probability of at least one false positive across all comparisons climb well above 5%. Two adjustment methods are most common:

• Bonferroni correction: multiply every raw p-value by \( m \) (capping at 1.0), or equivalently compare each raw p-value against \( \alpha / m \). Simple and widely used, but conservative- it treats every comparison identically regardless of how strong the evidence for each one is.
• Holm correction: sort the \( m \) raw p-values from smallest to largest, then compare the \( r \)-th smallest p-value against \( \alpha / (m - r + 1) \), stopping the first time a comparison fails to clear its threshold (all remaining, larger p-values are then treated as non-significant too). Holm's method controls the same family-wise error rate as Bonferroni but is uniformly more powerful-it never rejects fewer hypotheses than Bonferroni would, and often rejects more.

Worked Example 1: A Small Numerical Example by Hand

This continues Worked Example 1 from the Kruskal-Wallis Test guide: weight loss (in kg) for three diets, 5 subjects per diet, where the Kruskal-Wallis Test already returned \( H \approx 6.85 \), \( p \approx 0.033 \)-a significant result that justifies running Dunn's Test as a follow-up.

Recall the rank sums computed there: \( R_A = 58.5 \), \( R_B = 40 \), \( R_C = 21.5 \), with \( n_A = n_B = n_C = 5 \) and \( N = 15 \).

Step 1: Convert to mean ranks

Step 2: Compute the z-statistic for each pair

With \( N = 15 \) and equal group sizes \( n_i = n_j = 5 \), the standard error term \( \sqrt{\dfrac{N(N+1)}{12}\left(\dfrac{1}{n_i}+\dfrac{1}{n_j}\right)} = \sqrt{\dfrac{15 \times 16}{12}\left(\dfrac{1}{5}+\dfrac{1}{5}\right)} = \sqrt{20 \times 0.4} = \sqrt{8} \approx 2.828 \) is the same for all three pairs:

Step 3: Convert to raw p-values, then adjust

With \( m = 3 \) pairwise comparisons, Bonferroni multiplies each raw p-value by 3:

Worked Example 2: Comparing Three Training Methods

This continues Worked Example 2 from the Kruskal-Wallis Test guide: 7 employees trained with a new method, 6 with an older method, and 6 self-guided, where the Kruskal-Wallis Test returned \( H \approx 15.64 \), \( p \approx 0.0004 \)-a strongly significant result.

Recall the rank sums: \( R_{\text{New}} = 112 \), \( R_{\text{Old}} = 56 \), \( R_{\text{SG}} = 22 \), with \( n_{\text{New}} = 7 \), \( n_{\text{Old}} = n_{\text{SG}} = 6 \), and \( N = 19 \).

Step 1: Convert to mean ranks

Step 2: Compute the z-statistic for each pair

Unlike Worked Example 1, the group sizes here are unequal, so the standard error term must be recomputed for each pair. The shared factor \( \sqrt{N(N+1)/12} = \sqrt{19 \times 20 / 12} \approx 5.627 \) is reused across all three:

Step 3: Convert to raw p-values, then adjust

With \( m = 3 \) pairwise comparisons, Bonferroni multiplies each raw p-value by 3:

This second example is deliberately chosen so its numbers match the Python walkthrough in the next section, letting you verify the hand calculation against scikit-posthocs directly.

Python Example

The Kruskal-Wallis Test itself runs through scipy.stats.kruskal, and Dunn's post-hoc comparison runs through scikit-posthocs:

from scipy import stats
import pandas as pd
import scikit_posthocs as sp

new_method  = [85, 90, 88, 92, 84, 91, 89]
old_method  = [78, 82, 75, 80, 79, 77]
self_guided = [70, 74, 68, 72, 76, 71]

# Step 1: confirm the omnibus Kruskal-Wallis Test is significant
h_stat, p_value = stats.kruskal(new_method, old_method, self_guided)
print(f"Kruskal-Wallis H: {h_stat:.3f}, p-value: {p_value:.4f}")

# Step 2: build a long-format DataFrame for the post-hoc test
data = (
    [("New", v) for v in new_method]
    + [("Old", v) for v in old_method]
    + [("Self-Guided", v) for v in self_guided]
)
df = pd.DataFrame(data, columns=["group", "score"])

# Step 3: run Dunn's Test with Bonferroni correction
dunn_bonf = sp.posthoc_dunn(df, val_col="score", group_col="group", p_adjust="bonferroni")
print(dunn_bonf)

# Step 4: compare against Holm correction
dunn_holm = sp.posthoc_dunn(df, val_col="score", group_col="group", p_adjust="holm")
print(dunn_holm)

Output (Bonferroni-adjusted):

Kruskal-Wallis H: 15.639, p-value: 0.0004
                 New       Old  Self-Guided
New          1.00000  0.099660     0.000245
Old          0.09966  1.000000     0.243392
Self-Guided  0.00025  0.243392     1.000000

Each off-diagonal cell is the adjusted pairwise p-value for that comparison. Reading the New/Self-Guided cell, \( p \approx 0.0002 \) is well below 0.05, matching the "significantly different" conclusion from the z-statistic calculation in Worked Example 2. The New/Old cell sits at \( p \approx 0.10 \)-above 0.05, confirming that pair narrowly fails to reach significance once corrected.

Switching to p_adjust="holm" generally recovers a little more power: because Holm's method only divides by the full comparison count \( m \) for the single smallest p-value, and by progressively smaller divisors for the rest, the New/Old comparison's adjusted p-value drops slightly under Holm relative to Bonferroni-though in this particular dataset it still does not cross the 0.05 threshold.

Visualizing Results with a Significance Plot

import matplotlib.pyplot as plt

# scikit-posthocs includes a built-in heatmap-style significance plot
sp.sign_plot(dunn_bonf)
plt.title("Dunn's Test: Pairwise Significance (Bonferroni-Adjusted)")
plt.show()

This produces a heatmap where each cell is shaded by significance level, making it easy to scan which pairs cleared the adjusted threshold at a glance-particularly useful when comparing more than three or four groups, where a plain table of p-values becomes harder to read quickly.

How to Interpret Results

The significance level \( \alpha = 0.05 \) is the standard threshold used to decide whether a specific pairwise comparison is "statistically significant," applied to the adjusted p-value for each pair, not the raw one.

Checking Assumptions in Practice

• Confirm the Kruskal-Wallis Test was significant first: running Dunn's comparisons after a non-significant omnibus result is generally discouraged, since the correction step is calibrated assuming it follows a signal worth chasing.
• Independent groups: the same requirement as the Kruskal-Wallis Test-no subject should appear in more than one group.
• Number of groups: as \( k \) grows, the number of pairwise comparisons \( \binom{k}{2} \) grows quickly, and the adjustment (especially Bonferroni) becomes progressively more conservative- consider whether all pairs are truly of interest, or only a subset.
• Choice of correction: decide between Bonferroni and Holm before looking at the results, not after-switching methods after seeing which one produces a significant finding undermines the correction's purpose.
• Ties: check for tied values spanning multiple groups; most software applies the standard tie correction to the z-statistic's denominator automatically.

Advantages

• Directly answers the natural follow-up question after a significant Kruskal-Wallis Test: which groups specifically differ.
• Reuses the ranks already computed for the Kruskal-Wallis Test, requiring no additional data collection or re-ranking.
• Works naturally with unequal group sizes, unlike some simpler post-hoc procedures that assume balanced designs.
• Flexible choice of p-value adjustment (Bonferroni, Holm, and others), letting analysts trade off simplicity against statistical power.

Limitations

• Conservative by design-the multiple-comparison correction that protects against false positives also makes it harder to detect real but moderate pairwise differences, especially with few subjects, as seen in both worked examples above.
• Only meaningful as a follow-up to a prior Kruskal-Wallis Test on the same independent-groups design-it is not a general-purpose pairwise comparison tool for arbitrary unrelated samples.
• Statistical power drops as the number of groups \( k \) increases, since more comparisons demand a stricter adjusted threshold.
• Bonferroni correction specifically can be noticeably conservative with many groups-Holm's correction is usually a better default when available.

When NOT to Use It

• Mann-Whitney U Test: use instead when there are only two independent groups to begin with-there is no pairwise ambiguity to resolve.
• Nemenyi Test: use instead when the groups are actually related-the same subjects measured under every condition-and you are following up a significant Friedman Test rather than a Kruskal-Wallis Test.
• Tukey's HSD Test: use instead when the parametric assumptions hold and you are following up a significant one-way ANOVA rather than a Kruskal-Wallis Test.
• A single, pre-specified pairwise comparison: if you only care about one specific pair decided in advance, a single Mann-Whitney U Test may be more appropriate and more powerful than running the full Dunn's Test correction across all pairs.

Dunn's Test vs Nemenyi vs Tukey's HSD vs Bonferroni

16.1 Dunn's Test vs Nemenyi Test

16.2 Dunn's Test vs Tukey's HSD Test

16.3 Dunn's Test vs Plain Bonferroni-Corrected Mann-Whitney Tests

Common Misconceptions

• "A significant Kruskal-Wallis Test means every pair of groups differs." Not necessarily-see Worked Example 1 and Worked Example 2, where the omnibus test was significant but only one of the three possible pairs cleared Dunn's adjusted significance bar in each case.
• "Dunn's Test can be run on its own, without a Kruskal-Wallis Test." It is designed as a post-hoc procedure that reuses the Kruskal-Wallis Test's rank structure and is conventionally interpreted alongside a prior significant omnibus result, not as a standalone test.
• "A non-significant pairwise comparison proves those two groups are identical." Failing to clear the adjusted significance threshold only means there was not enough evidence to distinguish that pair at this sample size-not proof that no real difference exists.
• "Bonferroni and Holm always give the same conclusion." Not always-Holm's correction is uniformly at least as powerful as Bonferroni, so a pair that is significant under Bonferroni will also be significant under Holm, but the reverse is not guaranteed.
• "More groups always mean more significant pairwise findings." The opposite is often true-as \( k \) grows, the number of comparisons \( \binom{k}{2} \) grows too, tightening the adjusted significance threshold and making individual pairwise comparisons harder to distinguish, even with the same underlying effect size.

Interview Questions

1. Why is a post-hoc test like Dunn's Test necessary after a significant Kruskal-Wallis Test, rather than just inspecting which group has the highest mean rank?
2. Walk through how the z-statistic for a single pairwise comparison is derived, and explain the role of each term in the standard error.
3. Why does Dunn's Test require a p-value adjustment across all pairs, rather than comparing each raw p-value directly to 0.05?
4. Explain the difference between Bonferroni and Holm correction, and why Holm is generally preferred when available.
5. How does an unequal group size affect the standard error used in a specific pairwise z-statistic, compared to the equal-group-size case?
6. Compare Dunn's Test to simply running repeated Mann-Whitney U Tests with no correction at all.
7. Under what circumstances might you prefer the Nemenyi Test over Dunn's Test for a multi-group comparison?
8. Is it appropriate to run Dunn's Test when the Kruskal-Wallis Test itself was not significant? Why or why not?
9. Explain what each off-diagonal cell in scikit-posthocs's posthoc_dunn output represents.
10. How does statistical power for a specific pairwise comparison change as the total number of groups \( k \) increases, holding each group's sample size constant?

Frequently Asked Questions

• What is Dunn's Post-Hoc Test used for? +
  Dunn's Post-Hoc Test is used after a significant Kruskal-Wallis Test to pin down exactly which pairs of three or more independent groups differ from one another. The Kruskal-Wallis Test alone only tells you that a difference exists somewhere among the groups-it does not say which specific pair is responsible.
• What is the formula for Dunn's Post-Hoc Test? +
  Starting from the same combined ranks used by the Kruskal-Wallis Test, compute the mean rank for each group (rank sum divided by group size, R̄ⱼ = Rⱼ / nⱼ). For every pair of groups i and j, compute z = (R̄ᵢ − R̄ⱼ) / sqrt[(N(N+1)/12)(1/nᵢ + 1/nⱼ)], convert z to a two-tailed p-value from the standard normal distribution, then adjust that p-value for the total number of comparisons made, typically with a Bonferroni or Holm correction.
• What is the difference between Dunn's Test and the Kruskal-Wallis Test? +
  The Kruskal-Wallis Test is an omnibus test-like a one-way ANOVA's F-test-that only establishes whether at least one group differs from the others across the whole set of k independent groups. Dunn's Test is the pairwise follow-up: it reuses the same combined rank sums to test every pair of groups individually while adjusting for the overall false-positive risk across all those comparisons.
• What are the assumptions of Dunn's Post-Hoc Test? +
  Since it operates on the same ranks as the Kruskal-Wallis Test, it inherits that test's assumptions: independent groups, independent observations within each group, and at least ordinal data. It is conventionally run only after the Kruskal-Wallis Test itself returns a significant p-value, though some analysts run it regardless and rely on its own correction for multiple comparisons.
• How do you interpret Dunn's Post-Hoc Test results? +
  Each pairwise comparison is judged independently: if the adjusted p-value falls below your significance level, that pair of groups is significantly different. It is entirely possible-and common-for the omnibus Kruskal-Wallis Test to be significant while several individual pairs are not, as shown in the worked examples on this page.
• Should I use Bonferroni or Holm correction with Dunn's Test? +
  Holm's correction is generally preferred over Bonferroni whenever you have a choice-it controls the same family-wise error rate but is uniformly more powerful (never more conservative), since it adjusts each p-value based on its rank among all the comparisons rather than dividing every p-value by the same fixed number.
• Can Dunn's Test be used without running the Kruskal-Wallis Test first? +
  It is designed as a post-hoc procedure for independent-groups designs and its z-statistic reuses the same combined rank structure the Kruskal-Wallis Test produces, so in practice it is typically paired with a prior Kruskal-Wallis Test-conventionally run only when that omnibus test is significant, to avoid an extra, unnecessary layer of multiple-comparison correction.

Key Takeaways

• Dunn's Post-Hoc Test is the standard follow-up to a significant Kruskal-Wallis Test, identifying exactly which pairs of independent groups differ rather than just confirming a difference exists somewhere.
• It compares every pair of mean ranks using a z-statistic, \( z_{ij} = (\bar{R}_i - \bar{R}_j) / \sqrt{(N(N+1)/12)(1/n_i + 1/n_j)} \), then adjusts the resulting p-values-commonly with Bonferroni or Holm-to control the family-wise error rate across all comparisons at once.
• Core assumptions are inherited directly from the Kruskal-Wallis Test: independent groups and at least ordinal data-no normality is required.
• In Python, scikit_posthocs.posthoc_dunn(...) returns a full matrix of adjusted pairwise p-values that matches the hand-calculated conclusions exactly, as shown in both worked examples above.
• A significant omnibus Kruskal-Wallis result does not imply every pair differs-both worked examples on this page found only one of three possible pairs significant under Dunn's Test, underscoring why this step matters.
• Holm's correction is uniformly at least as powerful as Bonferroni and is generally the better default when your software supports both, as demonstrated in the Python example above.

Dunn's Post-Hoc Test completes the story the Kruskal-Wallis Test starts. An omnibus test answering only "does something differ?" is rarely the end goal of an independent-groups analysis-researchers and analysts almost always want to know specifically which groups moved, and by how confidently. Dunn's Test supplies that answer using the same rank-based machinery already built for the Kruskal-Wallis Test, adding an explicit, adjustable p-value correction that keeps the overall risk of a false pairwise claim under control even as the number of comparisons grows.

The two worked examples above, continuing directly from the same diet and training-method data used throughout the Kruskal-Wallis Test guide, show a consistent pattern: a significant omnibus result driven by one clear extreme-to-extreme difference, while the intermediate, smaller gaps did not individually clear the stricter pairwise bar. Reporting the Kruskal-Wallis \( H \) and p-value alongside the full Dunn's Test pairwise comparison table, as demonstrated throughout this guide, gives a complete and honest account of an independent-groups result that an omnibus p-value alone cannot provide.