Friedman Test: When, Why, and How to Use It

Introduction

The Friedman Test checks whether three or more related measurements differ significantly from one another, without assuming those measurements are normally distributed. If you have a repeated measures design-the same subjects measured under three or more conditions, or at three or more time points-but your sample is small, skewed, or has outliers that make a repeated measures ANOVA risky, this is usually the test you are looking for.

By the end of this article you will be able to state exactly when the Friedman Test applies, compute one completely by hand on two different worked examples, follow up a significant result with the Nemenyi post-hoc test, interpret everything alongside an effect size, and run the same test in a few lines of Python with scipy.stats.friedmanchisquare.

What Is the Friedman Test?

The Friedman Test compares three or more sets of measurements taken from the same subjects-across three or more conditions or time points-much like a repeated measures ANOVA. The difference is in how it uses the data: instead of working with the actual magnitude of each subject's scores, it converts the scores within each subject into ranks and asks whether the rank sums for each condition are balanced across subjects, or whether one or more conditions systematically dominate.

It exists because real repeated-measures data does not always behave-scores can be skewed, contain outliers, violate sphericity, or simply come from too small a sample to trust a normality assumption. By ranking within each subject rather than working with raw values across the whole dataset, the test trades a small amount of statistical power (when the data genuinely are normal and spherical) for considerable robustness to violations of those assumptions.

When to Use It

Use the Friedman Test when the same subjects are measured under three or more related conditions-repeated time points, different treatments applied in sequence to the same participants, or ratings of three or more items by the same judges-and you either cannot verify normality and sphericity, know they do not hold, or are working with a small or ordinal dataset. The key requirement is the same as the Wilcoxon Signed-Rank Test: the measurements are not independent-each subject contributes one value per condition, forming a "block."

Key Assumptions

• Related samples in blocks. Each subject (block) must be measured under all \( k \) conditions being compared-the design is a complete, one-observation-per-condition-per-subject layout.
• At least ordinal data. The values must be at least rankable-continuous or ordinal measurements both work, unlike repeated measures ANOVA which requires interval or ratio data.
• Independence across subjects. One subject's set of ranks should not influence another subject's set of ranks.
• \( k \geq 3 \) related conditions. With exactly two related conditions, use the Wilcoxon Signed-Rank Test instead; the Friedman Test's added value comes from handling three or more.
• No requirement of normality or sphericity. This is the defining relaxation relative to repeated measures ANOVA-the scores need not follow any particular parametric distribution, and the variances of pairwise differences between conditions need not be equal.

Hypotheses

The Friedman Test formally tests whether the \( k \) related conditions share the same distribution of ranks:

• Null Hypothesis (\( H_0 \)): there is no systematic difference among the \( k \) related conditions-any observed differences in rank sums are due to chance, i.e., the treatments or time points have no real effect.
• Alternative Hypothesis (\( H_1 \)): at least one condition differs systematically from at least one other condition-i.e., there is a real effect somewhere among the \( k \) conditions.

(The Friedman Test is inherently omnibus-like a one-way ANOVA, a significant result tells you a difference exists somewhere, not which specific pair of conditions differs. That question is answered by the Nemenyi post-hoc test.)

The Formula, Explained

For \( n \) subjects (blocks) each measured under \( k \) related conditions, let \( x_{ij} \) be the score for subject \( i \) under condition \( j \). The procedure is:

1. Within each subject \( i \), rank the \( k \) scores \( x_{i1}, \dots, x_{ik} \) from 1 (smallest) to \( k \) (largest), assigning the average rank to any ties within that subject.
2. Sum the ranks for each condition \( j \) across all \( n \) subjects to obtain the rank sum \( R_j = \sum_{i=1}^{n} \text{rank}(x_{ij}) \).
3. Compare how spread out the \( R_j \) values are relative to what would be expected if every condition were equally likely to receive any rank.

The test statistic is:

Under \( H_0 \), \( Q \) follows approximately a chi-square distribution with \( k - 1 \) degrees of freedom:

For small \( n \) and \( k \), exact tables of the Friedman distribution are preferred over the chi-square approximation, which can be conservative when the number of subjects is small. When ties occur within a subject's ranks, most software applies a tie correction to the denominator of \( Q \) to keep the chi-square approximation accurate.

Worked Example 1: A Small Numerical Example by Hand

Suppose a physiotherapist measures grip strength (in kg) for 5 patients at three time points-baseline, and after 4 and 8 weeks of a hand exercise program-extending the same patients used in the Wilcoxon Signed-Rank Test guide with one additional follow-up measurement, so you can compare both approaches directly.

Step 1: Rank within each patient

For each patient, rank the three scores from 1 (lowest) to 3 (highest). Patient 3 is the only case where Week 0 exceeds Week 4, so their within-patient ranking differs from the rest.

Step 2: Sum the ranks for each condition

Check: \( R_{\text{Week 0}} + R_{\text{Week 4}} + R_{\text{Week 8}} = 30 = n \cdot \dfrac{k(k+1)}{2} = 5 \times 6 \), confirming the ranking is correct.

Step 3: Compute the test statistic

With \( n = 5 \) and \( k = 3 \):

Step 4: Compare against the critical value

With \( k - 1 = 2 \) degrees of freedom, the chi-square critical value at \( \alpha = 0.05 \) is \( \chi^2_{0.05, 2} = 5.991 \). Since our computed \( Q = 8.4 \) exceeds 5.991, we reject \( H_0 \)-the corresponding p-value works out to approximately 0.015, below 0.05.

Worked Example 2: A Training Program (Six Students, Three Checkpoints)

Suppose 6 students take a mock test, then are re-tested after 2 weeks and again after 4 weeks of a training program-extending the same students used in the paired t-test and Wilcoxon Signed-Rank Test guides with one additional checkpoint, so the Python walkthrough below can be checked against all three tests directly.

Step 1: Rank within each student

Student 2 dipped slightly from Week 2 to Week 4, and Student 4 dipped from Pre-training to Week 2, so their rankings are the two exceptions to an otherwise steadily increasing pattern.

Step 2: Sum the ranks for each condition

Check: \( R_{\text{Pre}} + R_{\text{Wk2}} + R_{\text{Wk4}} = 36 = n \cdot \dfrac{k(k+1)}{2} = 6 \times 6 \).

Step 3: Test statistic

With \( n = 6 \) and \( k = 3 \):

Step 4: Decision

With \( k - 1 = 2 \) degrees of freedom, \( \chi^2_{0.05, 2} = 5.991 \). Since our computed \( Q \approx 8.33 \) is above this critical value, we reject \( H_0 \). The p-value at \( Q = 8.33 \) works out to approximately 0.0156, below 0.05-the training program appears to have produced a statistically significant change across the three checkpoints, consistent with the paired t-test and Wilcoxon results on the related two-point subsets of this same data.

This second example is deliberately chosen so its numbers match the Python walkthrough in the next section, letting you verify the hand calculation against scipy.stats.friedmanchisquare directly.

Effect Size (Kendall's W)

A p-value tells you whether an effect is likely to be real; it says nothing about how large that effect is. For the Friedman Test, the standard effect size is Kendall's coefficient of concordance, \( W \), which rescales the same \( Q \) statistic to a fixed 0-to-1 range:

Using Worked Example 2: \( W = \dfrac{8.33}{6 \times 2} = \dfrac{8.33}{12} \approx 0.69 \). \( W \) ranges from 0 (no agreement in ranking across subjects-conditions are essentially interchangeable) to 1 (perfect agreement-every subject ranks the conditions in exactly the same order). A value of 0.69 indicates strong, though not perfect, agreement that scores increased across the three checkpoints, consistent with the low p-value found above.

Post-Hoc: The Nemenyi Test

A significant Friedman Test only establishes that at least one condition differs from the others-it is an omnibus test, just like a one-way ANOVA's F-test. To find out which specific pairs of conditions differ, the standard follow-up is the Nemenyi test, which compares every pair of rank sums while controlling the family-wise error rate across all pairwise comparisons.

Two condition rank sums \( R_a \) and \( R_b \) (from Worked Example 2, using average ranks \( \bar{R}_j = R_j / n \)) are declared significantly different if:

where \( q_\alpha \) is a critical value from the Studentized range distribution (for \( k = 3 \) conditions at \( \alpha = 0.05 \), \( q_\alpha \approx 2.343 \), following the convention used by most statistical packages). For Worked Example 2, the average ranks are \( \bar{R}_{\text{Pre}} = 7/6 \approx 1.17 \), \( \bar{R}_{\text{Wk2}} = 12/6 = 2.00 \), and \( \bar{R}_{\text{Wk4}} = 17/6 \approx 2.83 \), and the critical difference is:

In other words, the overall training effect detected by the omnibus Friedman Test is driven mainly by the difference between the very first and very last checkpoint-the two-week gaps individually are not large enough, at this sample size, to clear the stricter pairwise threshold.

Python Example

You can run the omnibus test in one line using scipy.stats.friedmanchisquare, and the post-hoc comparison using scikit-posthocs:

from scipy import stats
import numpy as np

pre  = [62, 74, 58, 81, 69, 70]
wk2  = [68, 79, 65, 80, 75, 77]
wk4  = [71, 78, 69, 83, 77, 79]

q_stat, p_value = stats.friedmanchisquare(pre, wk2, wk4)

print(f"Q-statistic: {q_stat:.3f}")
print(f"p-value: {p_value:.4f}")

# Effect size: Kendall's W
n, k = 6, 3
kendalls_w = q_stat / (n * (k - 1))
print(f"Kendall's W: {kendalls_w:.3f}")

Output:

Q-statistic: 8.333
p-value: 0.0156
Kendall's W: 0.694

Every number here matches the hand calculation in Worked Example 2 closely-friedmanchisquare(pre, wk2, wk4) internally ranks each subject's row, sums the ranks per condition, and applies precisely the formula from the Formula section.

Post-Hoc Nemenyi Test in Python

import pandas as pd
import scikit_posthocs as sp

data = np.array([pre, wk2, wk4]).T  # shape: (n_subjects, k_conditions)
df = pd.DataFrame(data, columns=["Pre", "Wk2", "Wk4"])

nemenyi_result = sp.posthoc_nemenyi_friedman(df)
print(nemenyi_result)
# Each cell is a p-value for that pairwise comparison;
# values below 0.05 indicate a significant pairwise difference.

This reproduces the pairwise conclusions from the Nemenyi section above: the Pre-training vs Week 4 comparison comes back significant, while the other two pairs do not.

How to Interpret Results

The significance level \( \alpha = 0.05 \) is the standard threshold used to decide whether a result is "statistically significant."

Checking Assumptions in Practice

• Complete blocks: confirm every subject has a value for every condition-Friedman requires a complete design, and subjects with missing conditions are typically dropped entirely rather than partially used.
• Ties within subjects: check how many subjects have tied scores across conditions (they receive averaged ranks and may require a tie correction to \( Q \)).
• Independence across subjects: a design question, not something visible in the data itself-confirm one subject's ranking pattern did not influence another's.
• Number of conditions and subjects: for small \( n \) or \( k \), the chi-square approximation to \( Q \) can be conservative-exact Friedman tables or an exact-test option in software are preferable when available.

Advantages

• Does not require the scores to be normally distributed or satisfy sphericity, making it reliable with small or skewed repeated-measures samples.
• Much less sensitive to outliers than repeated measures ANOVA, since within-subject ranking dampens the influence of any single extreme value.
• Works with ordinal data (e.g., Likert-scale ratings across multiple conditions), where an ANOVA's mean-based logic is questionable.
• Naturally extends the two-condition logic of the Wilcoxon Signed-Rank Test to three or more related conditions, using the same underlying ranking philosophy.

Limitations

• Generally less statistically powerful than repeated measures ANOVA when the data genuinely are normal and spherical, since it discards exact magnitude information.
• Is an omnibus test-a significant result does not identify which specific conditions differ; a post-hoc test such as the Nemenyi test is required for that.
• Requires a complete design-every subject must have a value for every condition, so missing data forces either dropping that subject entirely or using a more complex imputation approach.
• The chi-square approximation for \( Q \) can be conservative with a small number of subjects or conditions, sometimes understating true significance.

When NOT to Use It

• Repeated measures ANOVA: use instead when the scores are approximately normally distributed and satisfy sphericity-it offers somewhat higher statistical power in that case.
• Wilcoxon Signed-Rank Test: use instead when you have exactly two related conditions rather than three or more, since it incorporates magnitude information the Friedman Test's two-condition case does not.
• Kruskal-Wallis Test: use instead when comparing three or more independent, unrelated groups rather than the same subjects measured repeatedly.
• Cochran's Q Test: use instead when the repeated outcome is strictly binary (success/ failure) rather than continuous or ordinal.

Friedman vs Repeated Measures ANOVA vs Wilcoxon vs Kruskal-Wallis

17.1 Friedman Test vs Repeated Measures ANOVA

17.2 Friedman Test vs Wilcoxon Signed-Rank Test

17.3 Friedman Test vs Kruskal-Wallis Test

Common Misconceptions

• "The Friedman Test has no assumptions at all." Not quite-it still requires a complete block design and independence across subjects; what it relaxes specifically is the normality and sphericity assumptions of repeated measures ANOVA.
• "A significant Friedman result tells me which condition is best." No. The omnibus test only tells you a difference exists somewhere-see the Nemenyi post-hoc section, where two of the three pairwise comparisons in Worked Example 2 were not individually significant despite the overall test being significant.
• "The Friedman Test and the Kruskal-Wallis Test are interchangeable." They are related but distinct-Kruskal-Wallis assumes independent groups, while the Friedman Test specifically requires the same subjects measured under every condition.
• "This test works for exactly two repeated measurements too." Mathematically yes, but in practice the Wilcoxon Signed-Rank Test is used for two related measurements since it incorporates magnitude information the Friedman ranking scheme discards at \( k = 2 \).
• "A non-significant result proves there is no effect." Failing to reject \( H_0 \) only means there was not enough evidence to detect an effect with this sample-it does not prove the true effect is zero.

Interview Questions

1. Walk through the ranking procedure of the Friedman Test and explain why ranking happens within each subject rather than across the whole dataset.
2. Why is the Friedman Test generally more robust to outliers than repeated measures ANOVA?
3. What does it mean for the Friedman Test to be an "omnibus" test, and why is a post-hoc test needed after a significant result?
4. Given a dataset with several tied ranks within a subject, how are ranks assigned, and how does this affect the chi-square approximation of \( Q \)?
5. Explain the relationship between the Friedman Test and the Wilcoxon Signed-Rank Test at \( k = 2 \) conditions.
6. How would you compute and interpret Kendall's W for a Friedman Test result?
7. What does the Nemenyi test add beyond the Friedman p-value alone, and how does it control for multiple comparisons?
8. Under what circumstances would you switch from a Friedman Test back to repeated measures ANOVA?
9. Why does scipy.stats.friedmanchisquare(pre, wk2, wk4) require all input arrays to be the same length and in matched subject order?
10. What happens to statistical power if you mistakenly apply a Friedman Test to normally distributed, spherical repeated-measures data instead of a repeated measures ANOVA?

Frequently Asked Questions

• What is the Friedman Test used for? +
  The Friedman Test is used to check whether three or more related measurements taken from the same subjects-such as scores at three or more time points, or ratings under three or more conditions applied to the same people-differ significantly from one another, without assuming the data follow a normal distribution.
• What is the formula for the Friedman Test? +
  Within each of the n subjects (blocks), rank the k related scores from 1 to k (averaging tied ranks). Sum the ranks for each condition across all subjects to get R_j. The test statistic is Q = [12 / (n·k·(k+1))] · Σ R_j² − 3·n·(k+1), which follows approximately a chi-square distribution with k − 1 degrees of freedom under the null hypothesis.
• What is the difference between the Friedman Test and repeated measures ANOVA? +
  Repeated measures ANOVA compares the means of three or more related conditions and requires the data to be approximately normally distributed with sphericity (equal variances of the pairwise differences). The Friedman Test compares conditions using only within-subject ranks, discarding exact magnitude information, which makes it more robust to outliers and non-normality but slightly less powerful when the parametric assumptions genuinely hold.
• What are the assumptions of the Friedman Test? +
  The subjects (blocks) should be independent of one another, every subject must be measured under all k conditions being compared, and the data should be at least ordinal. No assumption of normality or sphericity is required, which is the test's main advantage over repeated measures ANOVA.
• How do you interpret a Friedman Test p-value? +
  If the p-value is below your chosen significance level (commonly α = 0.05), you reject the null hypothesis and conclude that at least one condition differs significantly from the others. If the p-value is at or above α, there is not enough evidence that the conditions differ. A significant Friedman result only tells you a difference exists somewhere among the k conditions-it does not say which specific pairs differ, which is why a post-hoc test such as the Nemenyi test is typically run next.
• What should I use instead of the Friedman Test if my data are approximately normal? +
  Use repeated measures ANOVA, the parametric counterpart of the Friedman Test. It uses the actual magnitude of each score rather than just its within-subject rank, which gives it somewhat higher statistical power when normality and sphericity genuinely hold.
• What post-hoc test follows a significant Friedman Test? +
  The Nemenyi test is the standard post-hoc procedure-it performs all pairwise comparisons between conditions using the same rank sums computed for the Friedman Test, while controlling the family-wise error rate across the multiple comparisons being made.
• How many related conditions does the Friedman Test require? +
  At least three. With exactly two related conditions, the Friedman Test formula still works mathematically but reduces to a monotonic function of the Sign Test-the Wilcoxon Signed-Rank Test is used in practice for the two-condition case because it incorporates magnitude information the Friedman Test does not.

Key Takeaways

• The Friedman Test compares three or more related measurements from the same subjects by analyzing within-subject ranks-a non-parametric counterpart to repeated measures ANOVA.
• It is ideal for repeated time-point or multi-condition designs where normality or sphericity cannot be assumed, the sample is small, or the data are ordinal.
• Core assumptions: a complete block design (every subject measured under every condition) and independence across subjects-no normality or sphericity is required.
• In Python, scipy.stats.friedmanchisquare(...) gets you the Q-statistic and p-value in one call; pair it with Kendall's W and, when significant, the Nemenyi post-hoc test for a complete picture.
• If \( p < 0.05 \), at least one condition differs significantly; if \( p \geq 0.05 \), there isn't enough evidence of a real difference-but a significant omnibus result alone never tells you which pair differs, as both worked examples above demonstrate.
• If your data are approximately normal and spherical, repeated measures ANOVA will generally be more powerful; if you only have two related measurements, use the Wilcoxon Signed-Rank Test instead.

The Friedman Test earns its place as the standard non-parametric tool for repeated-measures designs precisely because real longitudinal and multi-condition data rarely arrives with guaranteed normality and sphericity-small clinical follow-ups, ordinal survey scales across multiple conditions, and skewed measurement distributions are common, and each one puts repeated measures ANOVA's core assumptions at risk. Its logic extends the Wilcoxon Signed-Rank Test's philosophy from two conditions to three or more: rank each subject's scores within their own row, sum those ranks by condition, then ask whether the resulting rank sums are too uneven to be explained by chance.

The two worked examples above, built on the same patients and students used in the paired t-test and Wilcoxon Signed-Rank Test guides, show both the omnibus test and its natural follow-up in practice-a small five-patient dataset that still reached significance thanks to a highly consistent ranking pattern, and a six-student training dataset where the Nemenyi post-hoc test pinpointed exactly which checkpoints actually differed. Reporting the p-value alongside Kendall's W and, whenever the result is significant, a proper post-hoc comparison, as demonstrated throughout this guide, gives a complete and honest picture that the omnibus p-value by itself cannot.