Mann-Whitney U Test: When, Why, and How to Use It

Introduction

The Mann-Whitney U Test (also called the Wilcoxon Rank-Sum Test) checks whether two separate, unrelated groups differ significantly, without assuming the data are normally distributed. If you are comparing a treatment group against a control group, version A against version B with two different sets of users, or any two unrelated groups-but your sample is small, skewed, or has outliers that make an independent samples t-test risky-this is usually the test you are looking for.

By the end of this article you will be able to state exactly when the Mann-Whitney U Test applies, compute one completely by hand on two different worked examples, interpret the result correctly alongside an effect size and confidence interval, and run the same test in a few lines of Python with scipy.stats.mannwhitneyu.

What Is the Mann-Whitney U Test?

The Mann-Whitney U Test compares two groups made up of different, unrelated subjects-just like the independent samples t-test. The difference is in how it uses the data: instead of comparing the two group means directly, it pools every observation from both groups, ranks them together, and asks whether one group's ranks tend to sit higher or lower than the other's.

It exists because real between-group data does not always behave-distributions can be skewed, contain outliers, or come from too small a sample to trust a normality assumption on either group. By working with ranks rather than raw values, the test trades a small amount of statistical power (when the data genuinely are normal) for considerable robustness to violations of that assumption.

When to Use It

Use the Mann-Whitney U Test when you are comparing two separate groups of different subjects and either cannot verify normality within each group, know it does not hold, or are working with a small or ordinal dataset. The key requirement is the same as the independent samples t-test: the two samples are independent-no subject appears in both groups.

Key Assumptions

• Independent groups. The two groups must consist of different, unrelated subjects-no subject appears in both.
• Independence within groups. Observations within each group should not influence one another.
• At least ordinal data. The values must be at least rankable-continuous or ordinal measurements both work, unlike the independent t-test which requires interval or ratio data.
• Similar distribution shape (for the median interpretation). To interpret a significant result specifically as a difference in medians, the two groups' distributions should have roughly the same shape and spread, differing mainly in location.
• No requirement of normality. This is the defining relaxation relative to the independent samples t-test-neither group's data need follow any particular parametric distribution.

Hypotheses

The Mann-Whitney U Test formally tests whether the two groups' distributions are identical:

• Null Hypothesis (\( H_0 \)): the two groups come from the same distribution (equal probability that a randomly chosen value from either group exceeds the other)-i.e., there is no real difference between groups.
• Alternative Hypothesis (\( H_1 \)): the two distributions differ-one group's values tend to be systematically larger than the other's.

(For one-tailed tests, \( H_1 \) can instead state that Group 1 is specifically stochastically greater than or less than Group 2, depending on the direction being tested-for example, if you only care whether the new method outperformed the old one.)

The Formula, Explained

For a group of size \( n_1 \) and a group of size \( n_2 \), the procedure is:

1. Combine all \( n_1 + n_2 \) observations from both groups into a single list.
2. Rank every observation from smallest (rank 1) to largest (rank \( n_1 + n_2 \)), assigning the average rank to any ties, regardless of which group each value belongs to.
3. Sum the ranks belonging to Group 1 to obtain \( R_1 \), and the ranks belonging to Group 2 to obtain \( R_2 \).
4. Compute the two U statistics:

Note that \( U_1 + U_2 = n_1 n_2 \), which serves as a useful check on the arithmetic. The test statistic is usually taken as:

For small \( n_1, n_2 \) (roughly both \( \leq 20 \)), \( U \) is compared directly against exact critical values from Mann-Whitney tables. For larger samples, a normal approximation is used. Under \( H_0 \), \( U \) has known mean and variance:

giving the z-score (with a continuity correction of 0.5, and a tie correction to \( \sigma_U \) when ties are present):

This \( z \) value is compared against the standard normal distribution to obtain a p-value. Because the test discards the exact magnitude of each observation and works only with ranks, it makes no assumption about the shape of either group's underlying distribution.

Worked Example 1: A Small Numerical Example by Hand

Suppose a researcher recruits two separate, unrelated groups of 5 adults each to test two different diets and records weight loss (in kg) after 8 weeks-the same data used in the independent samples t-test guide, so you can compare both approaches directly.

Step 1: Combine and rank all 10 values

Sorted combined values: 20(B), 21(B), 22(A), 23(B), 24(A), 24(B), 25(A), 25(B), 27(A), 28(A). The tied 24's (one from each group) share ranks 5 and 6, averaging to 5.5; the tied 25's similarly share ranks 7 and 8, averaging to 7.5.

Step 2: Sum the ranks for each group

Check: \( R_A + R_B = 55 = \dfrac{10 \times 11}{2} \), confirming the ranking is correct.

Step 3: Compute U for each group

Check: \( U_A + U_B = 25 = n_A \times n_B = 5 \times 5 \).

Step 4: Compare against the critical value

With \( n_1 = n_2 = 5 \) and a two-tailed test at \( \alpha = 0.05 \), the exact critical value from Mann-Whitney tables is \( U_{0.05, 5, 5} = 2 \) (the observed \( U \) must be at or below this value to reject \( H_0 \)). Since our computed \( U = 5 \) is greater than 2, we fail to reject \( H_0 \) at the 5% level-the corresponding exact two-tailed p-value works out to approximately 0.222, well above 0.05.

Worked Example 2: Comparing Two Training Methods (Unequal Group Sizes)

Suppose 7 employees are trained with a new onboarding method and 6 different employees are trained with the old method-again, the same dataset used in the independent samples t-test guide's second example, so the Python walkthrough below can be checked against both tests directly. Note the group sizes are unequal-this is completely fine for the Mann-Whitney U Test.

Step 1: Combine and rank all 13 values

Sorted combined values: 75(Old), 77(Old), 78(Old), 79(Old), 80(Old), 82(Old), 84(New), 85(New), 88(New), 89(New), 90(New), 91(New), 92(New). There are no ties here, so every rank is a whole number-notice all six Old Method values occupy the lowest six ranks.

Step 2: Sum the ranks for each group

Check: \( R_{\text{New}} + R_{\text{Old}} = 91 = \dfrac{13 \times 14}{2} \).

Step 3: Compute U for each group

Check: \( U_{\text{New}} + U_{\text{Old}} = 42 = n_1 \times n_2 = 7 \times 6 \).

Step 4: Decision

With \( n_1 = 7 \), \( n_2 = 6 \), the exact two-tailed critical value at \( \alpha = 0.05 \) is \( U_{0.05, 7, 6} = 6 \). Since our computed \( U = 0 \) is at or below this critical value, we reject \( H_0 \). This is in fact the most extreme possible outcome-every single Old Method score ranks below every single New Method score-so the exact two-tailed p-value is at its minimum for this sample size, approximately 0.0011, far below 0.05. The new training method appears to have produced a statistically significant improvement, consistent with the independent t-test result on the same data.

This second example is deliberately chosen so its numbers match the Python walkthrough in the next section, letting you verify the hand calculation against scipy.stats.mannwhitneyu directly.

Effect Size (Rank-Biserial Correlation)

A p-value tells you whether an effect is likely to be real; it says nothing about how large that effect is. For the Mann-Whitney U Test, a common effect size is the rank-biserial correlation, computed directly from U and the group sizes:

Using Worked Example 2: \( r = 1 - \dfrac{2 \times 0}{7 \times 6} = 1 - 0 = 1.0 \). This ranges from \( -1 \) to \( +1 \), with values near the extremes indicating one group almost entirely dominates the other in rank-here, a perfect separation between the two training methods, consistent with the minimal p-value found above.

Confidence Interval via the Hodges-Lehmann Estimator

Because the Mann-Whitney U Test does not estimate a mean difference, its natural confidence interval is not built around \( \bar{x}_1 - \bar{x}_2 \) the way the independent t-test's is. Instead, the standard approach uses the Hodges-Lehmann estimator for two independent samples: compute every possible pairwise difference \( x_{1i} - x_{2j} \) between an observation from Group 1 and an observation from Group 2, then take the median of all those differences as the point estimate of the location shift. A distribution-free confidence interval is then built from the ordered pairwise differences, using rank positions determined by the Mann-Whitney distribution rather than a t-distribution.

In practice, this calculation is rarely done by hand-statistical software computes it directly. For Worked Example 2, R's wilcox.test(new, old, conf.int = TRUE) or equivalent tools report a Hodges-Lehmann estimate of the location shift close to 10.0, with a 95% confidence interval of roughly [7.0, 13.0]-a range consistent with the complete separation between groups seen in the ranks above.

Python Example

You can run this test in one line using scipy.stats.mannwhitneyu:

from scipy import stats
import numpy as np

new_method = [85, 90, 88, 92, 84, 91, 89]
old_method = [78, 82, 75, 80, 79, 77]

u_stat, p_value = stats.mannwhitneyu(new_method, old_method, alternative="two-sided")

print(f"U-statistic: {u_stat:.3f}")
print(f"p-value: {p_value:.4f}")

# Effect size: rank-biserial correlation
n1, n2 = len(new_method), len(old_method)
r_rank_biserial = 1 - (2 * u_stat) / (n1 * n2)
print(f"Rank-biserial correlation: {r_rank_biserial:.3f}")

# Exact p-value for small samples
res = stats.mannwhitneyu(new_method, old_method, alternative="two-sided", method="exact")
print(f"Exact p-value: {res.pvalue:.4f}")

Output:

U-statistic: 42.000
p-value: 0.0012
Rank-biserial correlation: 1.000
Exact p-value: 0.0012

Note that scipy.stats.mannwhitneyu(new_method, old_method) reports \( U \) relative to the first array passed in-here \( U_{\text{New}} = 42 \), matching Worked Example 2 exactly, while the p-value itself is symmetric and identical regardless of which group is listed first. By default, scipy.stats.mannwhitneyu automatically switches between the exact distribution and a normal approximation depending on sample size and the presence of ties.

Checking Distribution Shape

Before interpreting a significant result as a difference in medians specifically, it is good practice to visually compare the shape of the two distributions:

import matplotlib.pyplot as plt

plt.boxplot([new_method, old_method], tick_labels=["New Method", "Old Method"])
plt.title("Distribution comparison by training method")
plt.show()
# Substantially different spreads or shapes between the two boxplots
# suggest interpreting the result strictly as a median difference may be misleading.

With only 13 total observations, a formal shape-comparison test has very low power-in practice, small between-group studies are usually evaluated with a visual check rather than relying on a formal statistical test alone.

How to Interpret Results

The significance level \( \alpha = 0.05 \) is the standard threshold used to decide whether a result is "statistically significant."

Checking Assumptions in Practice

• Independence between and within groups: a design question, not something visible in the data itself-confirm no subject appears in both groups and observations don't influence one another.
• Distribution shape: a pair of boxplots or histograms comparing both groups-substantially different shapes or spreads complicate interpreting a significant result strictly as a median difference.
• Ties: check how many tied values span across groups; they receive averaged ranks and typically require a tie correction in the normal-approximation variance.
• Sample size and exact vs. approximate p-values: for small \( n_1, n_2 \) (roughly both \( \leq 20 \)), use the exact distribution; for larger samples, the normal approximation is generally reliable, particularly with a continuity correction.

Advantages

• Does not require either group's data to be normally distributed, making it reliable with small or skewed samples.
• Much less sensitive to outliers than the independent t-test, since a single extreme value can shift its rank by only one position.
• Works with ordinal data (e.g., Likert-scale ratings), where an independent t-test's mean-based logic is questionable.
• Handles unequal group sizes naturally, exactly like the independent t-test, with exact p-values available for small samples.

Limitations

• Generally less statistically powerful than the independent samples t-test when both groups genuinely are normally distributed, since it discards exact magnitude information.
• A significant result is only cleanly interpretable as a difference in medians when both distributions have a similar shape-under differing shapes, the result is better described as evidence of stochastic dominance rather than a clean median-shift statement.
• Large numbers of tied values reduce the test's effective resolution and require variance corrections in the normal approximation.
• Only compares two independent groups-not three or more (use the Kruskal-Wallis test instead).

When NOT to Use It

• Independent samples t-test: use instead when both groups' data are approximately normally distributed-it offers somewhat higher statistical power in that case.
• Wilcoxon Signed-Rank Test: use instead when your two samples are actually related-the same subjects measured twice-rather than two separate, unrelated groups.
• Kruskal-Wallis Test: use instead when you have more than two independent groups (e.g., comparing three different training methods) and cannot assume normality.
• One-way ANOVA: use instead when you have more than two independent groups and normality genuinely holds within each.

Mann-Whitney vs Independent t-test vs Wilcoxon vs Kruskal-Wallis

17.1 Mann-Whitney U Test vs Independent Samples t-test

17.2 Mann-Whitney U Test vs Wilcoxon Signed-Rank Test

17.3 Mann-Whitney U Test vs Kruskal-Wallis Test

Common Misconceptions

• "The two groups must be the same size." Not true-the formula works perfectly well with \( n_1 \neq n_2 \), as shown in Worked Example 2.
• "A significant Mann-Whitney result always means the medians differ." Not necessarily- strictly, the test detects stochastic dominance. It cleanly implies a median difference only when both groups' distributions share a similar shape and spread.
• "A significant p-value means the effect is large." No. Statistical significance and effect size are different questions-see Worked Example 1, where a visible rank advantage for Diet A was not significant at \( n = 5 \) per group, and compare with the rank-biserial correlation for how to quantify magnitude separately.
• "Related or repeated data can just be analyzed with a Mann-Whitney U Test." Doing so ignores the correlation between paired observations and is generally the wrong test-use the Wilcoxon Signed-Rank Test instead when subjects are measured twice.
• "A non-significant result proves there is no difference." Failing to reject \( H_0 \) only means there was not enough evidence to detect a difference with this sample-it does not prove the true difference is zero.

Interview Questions

1. Walk through the ranking procedure of the Mann-Whitney U Test and explain how \( U_1 \) and \( U_2 \) relate to each other.
2. Why is the Mann-Whitney U Test generally more robust to outliers than the independent samples t-test?
3. What does a significant Mann-Whitney result strictly tell you, and when does that collapse to "the medians differ"?
4. Given a dataset with several tied values spanning both groups, how are ranks assigned, and how does this affect the normal-approximation p-value?
5. Explain the relationship between the Mann-Whitney U Test and the Kruskal-Wallis Test.
6. How would you compute and interpret the rank-biserial correlation for a Mann-Whitney U Test?
7. What does the Hodges-Lehmann estimator add beyond the p-value alone, and how does it differ conceptually from a t-test confidence interval?
8. Under what circumstances would you switch from a Mann-Whitney U Test back to an independent samples t-test?
9. Why does scipy.stats.mannwhitneyu(a, b) not require the two arrays to be the same length?
10. What happens to statistical power if you mistakenly apply a Mann-Whitney U Test to two normally distributed groups instead of an independent t-test?

Frequently Asked Questions

• What is the Mann-Whitney U Test used for? +
  The Mann-Whitney U Test (also called the Wilcoxon Rank-Sum Test) is used to check whether the distributions of two separate, unrelated groups-such as a treatment group and a control group, or users on version A versus version B-differ significantly, without assuming the data are normally distributed.
• What is the formula for the Mann-Whitney U Test? +
  Combine both groups' observations and rank them together from smallest to largest, averaging tied ranks. Sum the ranks belonging to each group to get R₁ and R₂, then compute U₁ = R₁ − n₁(n₁+1)/2 and U₂ = R₂ − n₂(n₂+1)/2, noting U₁ + U₂ = n₁n₂. The test statistic U is typically the smaller of the two, compared against exact tables (small samples) or a normal approximation (larger samples).
• What is the difference between the Mann-Whitney U Test and the independent samples t-test? +
  The independent samples t-test compares the means of two unrelated groups and requires each group to be approximately normally distributed. The Mann-Whitney U Test compares the ranks of the combined observations from both groups, discarding exact magnitude information, which makes it more robust to outliers and skewed distributions but slightly less powerful when the data genuinely are normal.
• What are the assumptions of the Mann-Whitney U Test? +
  The two groups must be independent of one another, observations within each group should be independent, and the data should be measured on at least an ordinal scale. No assumption of normality is required. A significant result is cleanly interpretable as a difference in medians only when the two distributions have a similar shape and spread.
• How do you interpret a Mann-Whitney U Test p-value? +
  If the p-value is below your chosen significance level (commonly α = 0.05), you reject the null hypothesis and conclude the two groups' distributions differ significantly-typically interpreted as a difference in medians. If the p-value is at or above α, there is not enough evidence to say the observed difference reflects a real effect rather than random variation.
• What should I use instead of the Mann-Whitney U Test if my data is approximately normal? +
  Use the independent samples t-test, the parametric counterpart of the Mann-Whitney U Test. It uses the actual magnitude of each observation rather than just its rank, which gives it somewhat higher statistical power when the normality assumption genuinely holds.
• Does the Mann-Whitney U Test require equal group sizes? +
  No. The Mann-Whitney U Test works perfectly well with unequal group sizes (n₁ ≠ n₂), just like the independent samples t-test-the rank-sum formula accounts for each group's size separately.
• What does a significant Mann-Whitney U Test result actually tell you? +
  Strictly, it tells you that one group is stochastically larger than the other-that a randomly chosen observation from one group is more likely to exceed a randomly chosen observation from the other group than chance alone would predict. This is commonly summarized as a difference in medians, but that specific interpretation requires the two distributions to have a similar shape.

Key Takeaways

• The Mann-Whitney U Test compares two separate, unrelated groups by ranking all observations together and comparing how those ranks split between the groups-a non-parametric counterpart to the independent samples t-test.
• It is ideal for treatment-vs-control or A/B comparisons where normality of the data cannot be assumed, the sample is small, or the data are ordinal.
• Core assumptions: independence between and within groups, and at least ordinal data-no normality is required; a similar distribution shape is needed to interpret the result as a difference in medians.
• In Python, scipy.stats.mannwhitneyu(a, b) gets you the U-statistic and p-value in one call; pair it with the rank-biserial correlation and, where needed, a Hodges-Lehmann estimate for a complete picture.
• If \( p < 0.05 \), the observed difference is statistically significant; if \( p \geq 0.05 \), there isn't enough evidence of a real difference-but always check the effect size too, since the two questions are different, as both worked examples above demonstrate.
• If both groups are approximately normal, the independent samples t-test will generally be more powerful; if your two samples are actually related (same subjects), use the Wilcoxon Signed-Rank Test instead; if you have more than two independent groups, use the Kruskal-Wallis Test.

The Mann-Whitney U Test earns its place as one of the most widely used non-parametric tests precisely because between-group comparisons rarely arrive with guaranteed normality-small clinical pilots, ordinal survey scales, and skewed measurement distributions are common, and each one puts the independent samples t-test's core assumption at risk. Its logic mirrors the t-test's simplicity while trading raw magnitude for ranks: pool both groups, rank every observation, then ask whether the ranks are too imbalanced between groups to be explained by chance.

The two worked examples above, built on the same data as the independent samples t-test guide, show both sides of that logic in practice-one where a visible rank advantage fell short of significance at a very small sample size, and one where a complete separation between groups produced an unambiguous, highly significant result under both tests. Reporting the p-value alongside the rank-biserial correlation and, where relevant, a Hodges-Lehmann interval, as demonstrated throughout this guide, gives a complete and honest picture that a p-value by itself cannot.