Fisher's Exact Test

Introduction

Fisher's Exact Test checks whether two categorical variables are associated in a contingency table by computing an exact p-value, rather than relying on a large-sample approximation. If you are asking questions like "is a rare side effect more common in the treatment group?", "does a small pilot study show a real preference between two options?", or "are two categories associated when your total sample is only 20 or 30 subjects?"-and your table has small or sparse cell counts-this is usually the test you are looking for.

By the end of this article you will be able to state exactly when Fisher's Exact Test applies, compute one completely by hand on two worked examples-a small classic 2x2 table and a rare-event clinical table where the chi-square approximation would mislead-interpret the result alongside the odds ratio effect size, understand the difference between one-sided and two-sided versions, and run the same test in one line of Python with scipy.stats.fisher_exact.

What Is Fisher's Exact Test?

Fisher's Exact Test asks a very direct question: given the row and column totals of a contingency table are fixed, exactly how likely is the observed table-and every table even more extreme-to arise purely by chance, if the two categorical variables were truly independent? Rather than approximating this probability with the chi-square distribution, it computes it exactly, using the hypergeometric distribution, which describes the probability of drawing a given number of successes when sampling without replacement from a fixed population.

It exists because the Chi-Square Test of Independence's approximation breaks down precisely when it matters most-small studies, rare events, pilot data-situations common throughout clinical research, quality control, and early-stage A/B testing. Fisher's Exact Test fills that gap directly, remaining valid no matter how small the sample or how sparse the table.

When to Use It

Use Fisher's Exact Test when you have two categorical variables, typically arranged in a 2x2 table (though larger tables are possible), and either the total sample size is small or one or more expected cell frequencies fall below 5-the same threshold at which the Chi-Square Test of Independence's approximation becomes unreliable. Like that test, both variables here are categorical counts, not numeric measurements.

Key Assumptions

• Independence of observations: each subject or unit contributes to exactly one cell of the contingency table-no subject is counted twice, and no subject's category depends on another subject's.
• Categorical data: both variables must be categorical (nominal or ordinal), recorded as frequency counts rather than continuous measurements.
• Mutually exclusive, exhaustive categories: every subject falls into exactly one category for each variable.
• Fixed margins (classical framing): the exact probability calculation conditions on the observed row and column totals being fixed-a mathematical device for enumerating possible tables, not a requirement that the researcher literally fixed both margins in advance.

Hypotheses

Fisher's Exact Test formally tests whether the two categorical variables are related, exactly as the Chi-Square Test of Independence does, only using an exact rather than approximate calculation:

• Null Hypothesis (\( H_0 \)): the two categorical variables are independent-a subject's category on one variable gives no information about their category on the other; any observed association in the sample is due to chance.
• Alternative Hypothesis (\( H_1 \)): the two categorical variables are associated- knowing a subject's category on one variable does give information about their likely category on the other.

(As with the omnibus tests elsewhere on this site, a significant Fisher's Exact Test tells you an association exists, but the odds ratio, discussed in the effect size section below, is needed to describe its size and direction.)

The Formula, Explained

For a 2x2 table with observed cell counts \( a, b, c, d \), row totals \( a+b \) and \( c+d \), column totals \( a+c \) and \( b+d \), and grand total \( N = a+b+c+d \):

Treating the margins as fixed, the exact probability of this particular table arising under \( H_0 \) is the hypergeometric probability:

To obtain a two-sided p-value, this same formula is applied to every possible table with the identical row and column totals-varying \( a \) across its full valid range while \( b, c, d \) adjust to keep the margins fixed-and the probabilities of all tables that are no more likely than the observed table (i.e., at least as extreme) are summed together:

Because every term is computed exactly from factorials rather than approximated, no minimum sample size or cell-frequency requirement is needed for the result to be valid.

Worked Example 1: A 2x2 Table by Hand

Suppose a small trial gives 10 patients a new drug and 10 patients a placebo, then records whether each patient improved:

Step 1: Compute the observed table's exact probability

Step 2: Enumerate every table with the same margins

With row totals of 10 and 10, and column totals of 12 and 8, \( a \) can range from 2 to 10. Computing the exact probability for every possible value of \( a \) gives:

Check: all nine probabilities sum to 1.000000, confirming every possible table has been enumerated.

Step 3: Sum the probabilities of tables at least as extreme

The observed table (\( a=9 \)) has probability 0.009526. Two tables are at least this extreme: \( a=9 \) itself and its mirror-image tail, \( a=2 \) (probability 0.000357), plus the far tail \( a=10 \) (probability 0.000357) and \( a=3 \) (probability 0.009526), which are equally or less likely than the observed table:

Step 4: Conclusion

Since \( p \approx 0.0198 < 0.05 \), we reject \( H_0 \). The drug and placebo groups differ significantly in improvement rate-notably, with expected cell frequencies of only \( \frac{10 \times 12}{20} = 6 \) and \( \frac{10 \times 8}{20} = 4 \), the Chi-Square Test of Independence's reliability threshold is already borderline here, making the exact calculation the safer choice.

Worked Example 2: A Rare-Event Clinical Table

Now suppose a clinical study monitors 30 patients for a rare adverse event across two treatment arms, where the event occurs in only a handful of patients overall:

Step 1: Check expected cell frequencies

Under independence, the expected frequency for the "New Treatment, Event Occurred" cell is \( \frac{15 \times 9}{30} = 4.5 \)-already below the conventional threshold of 5, and the same holds for the "Standard Treatment, Event Occurred" cell. This is exactly the situation where the chi-square approximation should not be trusted.

Step 2: Compute the exact p-value

Enumerating every table with row totals 15 and 15 and column totals 9 and 21, and summing the hypergeometric probabilities of every table at least as extreme as the observed one, gives:

(Verified against Python in the Python section below.)

Step 3: Compare against the chi-square approximation

Running the (uncorrected) Chi-Square Test of Independence on this same table gives \( p \approx 0.00529 \)- noticeably smaller, and noticeably more "significant-looking," than the exact p-value of 0.01419. This gap is precisely why Fisher's Exact Test is preferred whenever expected cell frequencies fall this low: the chi-square approximation systematically understates the p-value in small, sparse tables like this one.

Step 4: Conclusion

Since \( p \approx 0.0142 < 0.05 \) even under the more conservative exact test, we still reject \( H_0 \)- the adverse event rate differs significantly between the two treatment arms. But had the true effect been slightly weaker, relying on the chi-square approximation's inflated significance could have led to a false positive that the exact test would have correctly avoided.

Effect Size: Odds Ratio

Like the chi-square-based tests elsewhere on this site, a Fisher's Exact Test p-value alone says nothing about the size of the association. For a 2x2 table, the standard companion effect size is the odds ratio:

An odds ratio of 1 indicates no association; values further from 1 in either direction indicate a stronger effect. For Worked Example 1:

Patients on the drug had 21 times higher odds of improving than patients on placebo. For Worked Example 2:

Patients on the new treatment had roughly one-sixteenth the odds of the adverse event compared to the standard treatment-a strong protective effect, consistent with the significant result above. scipy.stats.fisher_exact reports a conditional maximum likelihood estimate of the odds ratio alongside the p-value, shown in the Python section below.

One-Sided vs Two-Sided Tests

Because Fisher's Exact Test enumerates every possible table explicitly, it can report either a two-sided p-value (the default and standard choice for most research questions, summing both tails of extremity) or a one-sided p-value (summing only the tail in one pre-specified direction).

For Worked Example 1, the one-sided p-value testing specifically whether the drug group has a higher improvement rate is:

which only sums the tables at least as extreme in the drug-favoring direction (\( a = 9 \) and \( a = 10 \)). Using a one-sided test roughly halves the p-value compared to the two-sided version, but is only valid when the direction of the expected effect was specified before looking at the data-choosing the direction after seeing which way the data leans is a form of p-hacking and inflates the false-positive rate.

Python Example

You can run this test in one line using scipy.stats.fisher_exact:

import numpy as np
from scipy import stats

# Worked Example 1: Drug (row 1) vs Placebo (row 2), Improved vs Did Not Improve
table = np.array([
    [9, 1],   # Drug
    [3, 7]    # Placebo
])

odds_ratio, p_value = stats.fisher_exact(table, alternative='two-sided')
print(f"Odds ratio: {odds_ratio:.3f}")
print(f"Two-sided p-value: {p_value:.6f}")

# One-sided version (direction: drug improves outcomes)
_, p_greater = stats.fisher_exact(table, alternative='greater')
print(f"One-sided p-value (greater): {p_greater:.6f}")

Output:

Odds ratio: 21.000
Two-sided p-value: 0.019767
One-sided p-value (greater): 0.009883

These figures match Worked Example 1 and the one-sided section above exactly. For Worked Example 2, comparing against the chi-square approximation:

table_2 = np.array([
    [1, 14],   # New Treatment
    [8, 7]     # Standard Treatment
])

odds_ratio_2, p_fisher = stats.fisher_exact(table_2, alternative='two-sided')
print(f"Fisher's exact odds ratio: {odds_ratio_2:.4f}")
print(f"Fisher's exact p-value: {p_fisher:.5f}")

# Compare against the (less reliable, here, given small expected frequencies) chi-square approximation
chi2_stat, p_chi2, dof, expected = stats.chi2_contingency(table_2, correction=False)
print(f"Chi-square expected frequencies:\n{np.round(expected, 2)}")
print(f"Chi-square-approx p-value: {p_chi2:.5f}")

Fisher's exact odds ratio: 0.0625
Fisher's exact p-value: 0.01419
Chi-square expected frequencies:
[[ 4.5 10.5]
 [ 4.5 10.5]]
Chi-square-approx p-value: 0.00529

This confirms both the exact p-value and the meaningful gap versus the chi-square approximation discussed in Worked Example 2-with expected frequencies of only 4.5, the chi-square test understates the true p-value by roughly a factor of 2.7 in this case.

How to Interpret Results

A significant Fisher's Exact Test tells you the two categorical variables are associated, exactly as a significant Chi-Square Test of Independence would-the difference lies entirely in how rigorously that p-value was computed, not in what it means. Always report the odds ratio alongside the p-value, as in both worked examples above, so readers can judge both the statistical and practical significance of the association.

When expected cell frequencies are borderline (roughly 5-10), it is good practice to report both the chi-square and Fisher's Exact p-values, as Worked Example 2 does-showing the exact figure builds confidence that the conclusion does not hinge on an approximation that may not hold.

Checking Assumptions in Practice

Before running Fisher's Exact Test, confirm that observations are independent and that the data are genuine frequency counts arranged in a contingency table-the same basic checks used for the Chi-Square Test of Independence. Unlike that test, there is no expected-frequency threshold to verify here; Fisher's Exact Test can be applied safely regardless of how small or sparse the table is.

As a practical workflow, many analysts compute the expected cell frequencies first (as in Worked Example 2) purely to decide whether the faster chi-square approximation is trustworthy enough to report on its own, or whether Fisher's Exact Test should be run (or reported alongside it) instead.

Advantages

• Computes an exact p-value with no minimum sample size or expected-frequency requirement, making it valid for the smallest and sparsest tables.
• Avoids the systematic understatement of p-values that the chi-square approximation can produce when expected cell frequencies are low, as demonstrated in Worked Example 2.
• Directly interpretable: the p-value is literally the probability of observing a table this extreme or more, under independence, with no approximating distribution involved.
• Supports both one-sided and two-sided versions naturally, since every possible table's exact probability is already being enumerated.

Limitations

• Computationally expensive for large tables or large sample sizes, since it must enumerate (or otherwise account for) every possible table with the same margins-the Chi-Square Test of Independence is faster and just as reliable once sample sizes are adequate.
• The classical version conditions on both margins being fixed, which some statisticians consider overly conservative when neither margin was truly fixed by the study design; unconditional alternatives such as Barnard's test can have more power in that situation.
• Like the Chi-Square Test of Independence, a significant result only establishes that an association exists-it says nothing about causation.
• Extending the test to tables larger than 2x2 (the Freeman-Halton extension) becomes computationally demanding quickly, and is not implemented identically across all statistical software.

When NOT to Use It

• Chi-Square Test of Independence: use instead when expected cell frequencies are comfortably above 5 and the sample is reasonably large, since it is computationally cheaper and gives essentially the same conclusion in that regime.
• Barnard's Test: consider instead when you want an unconditional exact test with somewhat higher statistical power than Fisher's Exact Test, particularly for 2x2 tables where neither margin was genuinely fixed by design.
• McNemar's Test: use instead when the two categorical measurements come from the same subjects measured twice (paired data) rather than independent subjects cross-classified on two variables.
• Logistic regression: consider instead when you need to control for additional covariates or model the association alongside other predictor variables.

Fisher's Exact Test vs Chi-Square Test of Independence vs Barnard's Test

17.1 Fisher's Exact Test vs Chi-Square Test of Independence

17.2 Fisher's Exact Test vs Barnard's Test

17.3 Fisher's Exact Test vs McNemar's Test

Common Misconceptions

• "Fisher's Exact Test should always be used instead of the Chi-Square Test of Independence." Not necessarily-once expected cell frequencies are comfortably above 5, both tests give essentially the same conclusion, and the chi-square test is computationally cheaper and just as valid.
• "Fixed margins means the researcher must have literally fixed both totals before collecting data." No-conditioning on both margins is a mathematical device for enumerating possible tables, used even when neither total was fixed by the actual study design.
• "A significant Fisher's Exact Test result proves causation." No-like every association test on this site, it only detects that an association exists, not that one variable causes the other.
• "Fisher's Exact Test only works for 2x2 tables." The classical formula is defined for 2x2 tables, but the underlying hypergeometric logic extends to general \( r \times c \) tables via the Freeman-Halton extension, albeit at greater computational cost.
• "A non-significant Fisher's Exact result proves the variables are truly independent." Failing to reject \( H_0 \) only means there was not enough evidence to detect an association with this sample-it does not prove the true association is exactly zero, and small samples in particular have limited power to detect real but modest effects.

Interview Questions

1. Walk through how the hypergeometric probability of a single 2x2 table is computed, and explain the intuition behind the formula.
2. Why does Fisher's Exact Test not require a minimum expected cell frequency, unlike the Chi-Square Test of Independence?
3. Explain conceptually what it means to sum the probabilities of "tables at least as extreme" when computing a two-sided p-value.
4. What does it mean for the chi-square approximation to "understate" the true p-value in a sparse table, and why does this happen?
5. Explain the difference between a one-sided and two-sided Fisher's Exact Test, and when a one-sided test is (and isn't) appropriate.
6. How would you compute and interpret the odds ratio for a Fisher's Exact Test result, and why is it necessary alongside the p-value?
7. What does it mean for Fisher's Exact Test to "condition on fixed margins," and why is this sometimes criticized as overly conservative?
8. Under what circumstances would you switch from Fisher's Exact Test to Barnard's Test?
9. Why does Fisher's Exact Test become computationally expensive for large tables or large sample sizes, and what test would you use instead once expected frequencies are large enough?
10. How would you extend Fisher's Exact Test logic to a table larger than 2x2, and what is this extension called?

Frequently Asked Questions

• What is Fisher's Exact Test used for? +
  Fisher's Exact Test is used to determine whether two categorical variables-such as treatment group and outcome, or exposure and disease status-are associated in a contingency table, and is especially valuable when the sample size is small or the table contains rare events, situations where the Chi-Square Test of Independence's large-sample approximation is unreliable.
• What is the formula for Fisher's Exact Test? +
  For a 2x2 table with cells a, b, c, d, row totals (a+b) and (c+d), column totals (a+c) and (b+d), and grand total N, the exact probability of that specific table arising by chance, given the margins are fixed, is the hypergeometric probability p = [(a+b)! (c+d)! (a+c)! (b+d)!] / [N! a! b! c! d!]. The two-sided p-value is the sum of this probability over every table with the same margins whose probability is no greater than the observed table's probability.
• What is the difference between Fisher's Exact Test and the Chi-Square Test of Independence? +
  Fisher's Exact Test computes an exact p-value from the hypergeometric distribution and is valid at any sample size. The Chi-Square Test of Independence uses a large-sample approximation to the chi-square distribution that requires expected cell frequencies of roughly 5 or more to be trustworthy; below that threshold, Fisher's Exact Test is the standard alternative.
• What are the assumptions of Fisher's Exact Test? +
  Observations must be independent of one another, the data must be categorical frequency counts arranged in a contingency table, and the classical version of the test treats both the row totals and column totals as fixed when enumerating all possible tables-this is a mathematical convenience rather than a requirement that the researcher literally fixed both margins during data collection.
• How do you interpret a Fisher's Exact Test p-value? +
  If the p-value is below your chosen significance level (commonly alpha = 0.05), you reject the null hypothesis of independence and conclude the two categorical variables are significantly associated. If the p-value is at or above alpha, there is not enough evidence of an association given the data collected.
• When should I use Fisher's Exact Test instead of the Chi-Square Test of Independence? +
  Use Fisher's Exact Test whenever your contingency table-especially a 2x2 table-has one or more expected cell frequencies below 5, or whenever the total sample size is small. It is also a safe default any time you are uncertain whether the chi-square approximation's sample-size requirement is met, since Fisher's Exact Test remains valid regardless.
• Can Fisher's Exact Test be used for tables larger than 2x2? +
  Yes-the hypergeometric logic extends to general r x c tables (sometimes called the Freeman-Halton extension), though the computation grows substantially more expensive as the table grows, since every possible table with the same margins must be enumerated or approximated.
• What is the difference between one-sided and two-sided Fisher's Exact Test p-values? +
  A two-sided p-value sums the probabilities of every table at least as extreme as the observed table in either direction of association, and is the standard default for most research questions. A one-sided p-value only sums tables at least as extreme in one specified direction, and should only be used when the direction of the expected effect was decided before looking at the data.

Key Takeaways

• Fisher's Exact Test checks whether two categorical variables are associated, computing an exact p-value from the hypergeometric distribution rather than approximating it.
• It is ideal for small studies, rare-event data, and any 2x2 (or small) contingency table where expected cell frequencies fall below 5-the same threshold at which the Chi-Square Test of Independence becomes unreliable.
• Core assumptions: independent observations, mutually exclusive categories, and the classical fixed-margins framing-there is no minimum sample size requirement at all.
• In Python, scipy.stats.fisher_exact(...) gets you the odds ratio and p-value in one call, with alternative='two-sided', 'greater', or 'less' to control the direction of the test.
• If \( p < 0.05 \), the variables are significantly associated; if \( p \geq 0.05 \), there isn't enough evidence of an association-but as Worked Example 2 shows, the chi-square approximation can meaningfully understate the true p-value once expected frequencies drop below 5.
• Always report an effect size such as the odds ratio alongside the p-value, and default to the two-sided test unless a directional hypothesis was specified before looking at the data.

Fisher's Exact Test earns its place as the small-sample safeguard for categorical association precisely because so much real-world research-early clinical trials, rare-event monitoring, pilot studies-simply cannot gather the sample sizes the Chi-Square Test of Independence's approximation was built to handle. Its logic is refreshingly direct, if computationally heavier: enumerate every table that could have arisen with the same margins, and sum the exact probabilities of every one at least as extreme as what was actually observed.

The two worked examples above, a small classic 2x2 drug trial and a rare-event clinical table, show both ends of this test's range, and a useful lesson in between: the rare-event table's expected cell frequency of 4.5 was already below the conventional chi-square threshold, and the resulting gap between the exact p-value (0.01419) and the chi-square approximation (0.00529) was substantial enough to matter for a borderline decision. Reporting the exact p-value alongside the odds ratio, and defaulting to Fisher's Exact Test whenever expected cell frequencies are in doubt, as demonstrated throughout this guide, gives a complete and honest picture that an unchecked approximation cannot guarantee.