Chi-Square Test of Independence

Introduction

The Chi-Square Test of Independence checks whether two categorical variables are related to one another, or whether they behave independently. If you are asking questions like "does gender influence product preference?", "is education level associated with voting choice?", or "does treatment group affect which outcome category a patient falls into?"-and your data are counts in categories rather than measured numbers-this is usually the test you are looking for.

By the end of this article you will be able to state exactly when the Chi-Square Test of Independence applies, compute one completely by hand on two worked examples-a small 2x2 table and a larger 3x3 table-interpret the result alongside Cramer's V, understand when Yates' continuity correction matters, and run the same test in a few lines of Python with scipy.stats.chi2_contingency.

What Is the Chi-Square Test of Independence?

The Chi-Square Test of Independence compares the observed counts in a two-way contingency table against the counts you would expect if the two categorical variables had absolutely no relationship-that is, if knowing a subject's category on one variable told you nothing about their category on the other. The bigger the gap between observed and expected counts, summed appropriately across every cell of the table, the stronger the evidence that the variables are actually associated.

It exists because a great deal of real-world data is categorical rather than numeric-gender, product choice, pass/fail outcome, region, treatment arm, survey response category-and comparing means or ranks, the tools used by most other tests on this site, simply does not apply. The Chi-Square Test of Independence fills that gap directly, working entirely with frequency counts.

When to Use It

Use the Chi-Square Test of Independence when you have two categorical variables measured on the same set of subjects, and you want to know whether they are associated. Unlike most other tests on this site, there is no numeric outcome to speak of-both variables are categories, and the raw data are simply counts of how many subjects fall into each combination of categories.

Key Assumptions

• Independence of observations: each subject or unit contributes to exactly one cell of the contingency table-no subject is counted twice, and no subject's category depends on another subject's.
• Categorical data: both variables must be categorical (nominal or ordinal), recorded as frequency counts rather than continuous measurements.
• Mutually exclusive, exhaustive categories: every subject falls into exactly one category for each variable.
• Sufficiently large expected frequencies: every expected cell frequency should generally be 5 or greater for the chi-square distribution to approximate the test statistic reliably-see checking assumptions in practice below.

Hypotheses

The Chi-Square Test of Independence formally tests whether the two categorical variables are related:

• Null Hypothesis (\( H_0 \)): the two categorical variables are independent-a subject's category on one variable gives no information about their category on the other; any observed association in the sample is due to chance.
• Alternative Hypothesis (\( H_1 \)): the two categorical variables are associated- knowing a subject's category on one variable does give information about their likely category on the other.

(Like the omnibus tests elsewhere on this site, a significant Chi-Square Test of Independence tells you an association exists somewhere in the table, but not necessarily which specific category combinations are driving it-inspecting the standardized residuals for each cell, mentioned in interpreting results below, helps pinpoint that.)

The Formula, Explained

For a contingency table with \( r \) rows and \( c \) columns, built from a sample of \( N \) subjects cross-classified on two categorical variables, the procedure is:

1. Tally the observed frequency \( O_{ij} \) in each cell-the count of subjects falling into row \( i \) and column \( j \).
2. Compute the expected frequency for each cell under independence: \( E_{ij} = \dfrac{(\text{row } i \text{ total}) \times (\text{column } j \text{ total})}{N} \).
3. Compare observed against expected in every cell, and sum the squared, scaled differences across the whole table.

The test statistic is:

Under \( H_0 \), \( \chi^2 \) follows approximately a chi-square distribution with degrees of freedom:

Intuitively, each cell contributes more to \( \chi^2 \) the further its observed count strays from what independence would predict, with that contribution scaled down by \( E_{ij} \) so that a given absolute gap matters more in a cell that was expected to be small.

Worked Example 1: A 2x2 Table by Hand

Suppose a researcher surveys 100 employees, recording their gender and whether they prefer working from home. The raw counts are:

Step 1: Compute expected frequencies

Step 2: Compute the chi-square statistic

Step 3: Compare against the critical value

With \( r = 2 \) and \( c = 2 \), degrees of freedom \( = (2-1)(2-1) = 1 \). The chi-square critical value at \( \alpha = 0.05 \) with 1 degree of freedom is \( \chi^2_{0.05,1} = 3.841 \). Since our computed \( \chi^2 = 9.091 \) exceeds 3.841, we reject \( H_0 \)-the corresponding p-value works out to approximately 0.0026, well below 0.05.

Worked Example 2: A 3x3 Table (Education Level vs Voting Preference)

Suppose a political survey of 320 voters records education level and which of three parties they support:

Step 1: Compute expected frequencies

Each expected cell uses \( E_{ij} = (\text{row total}_i \times \text{column total}_j) / 320 \):

Step 2: Compute the chi-square statistic

Summing \( (O-E)^2/E \) across all nine cells:

Step 3: Decision

With \( r = 3 \) and \( c = 3 \), degrees of freedom \( = (3-1)(3-1) = 4 \). The chi-square critical value at \( \alpha = 0.05 \) with 4 degrees of freedom is \( \chi^2_{0.05,4} = 9.488 \). Since our computed \( \chi^2 \approx 24.327 \) is far greater than this critical value, we reject \( H_0 \) decisively. The corresponding p-value is approximately 0.00007, far below 0.05-education level and voting preference are clearly associated in this sample.

Looking at the individual cell contributions, High School/Party A (7.102), Graduate/Party A (6.011), and Graduate/Party C (6.050) contribute the most to the total-High-school-educated respondents favor Party A more than expected, while graduate-educated respondents favor Party C more and Party A less than expected.

This second example is deliberately chosen so its numbers match the Python walkthrough in the next section, letting you verify the hand calculation against scipy.stats.chi2_contingency directly.

Effect Size: Phi and Cramer's V

A p-value tells you whether an association is likely to be real; it says nothing about how strong that association is, and it grows more easily "significant" simply by collecting more data, even for a trivial effect. For 2x2 tables, the standard effect size is the phi coefficient:

For larger tables, Cramer's V generalizes phi to any table size:

Using Worked Example 1 (a 2x2 table, where phi and Cramer's V are identical): \( \phi = \sqrt{9.091/100} \approx 0.302 \). Using Worked Example 2 (a 3x3 table, so \( k = 3 \)): \( V = \sqrt{24.327/(320 \times 2)} = \sqrt{24.327/640} \approx 0.195 \).

By these conventional benchmarks (Cohen, 1988), Worked Example 1's \( \phi \approx 0.302 \) sits right at the boundary between a medium and large effect for a 2x2 table, while Worked Example 2's \( V \approx 0.195 \) is a small-to-medium effect for a 3x3 table-despite Example 2 having the far larger, more "impressive-looking" chi-square statistic and smaller p-value, its underlying association is proportionally weaker once table size and sample size are accounted for.

Yates' Continuity Correction

For 2x2 tables specifically, the chi-square distribution is a continuous approximation to what is actually a discrete counting process, and this approximation can overstate significance in small samples. Yates' continuity correction subtracts 0.5 from the absolute difference between each observed and expected count before squaring, making the test slightly more conservative:

Applying this to Worked Example 1:

This is noticeably smaller than the uncorrected \( \chi^2 = 9.091 \), and the corresponding p-value rises from about 0.0026 to about 0.0049-still comfortably significant here, but the correction can flip borderline results from significant to non-significant in smaller samples.

Python Example

You can run this test in one line using scipy.stats.chi2_contingency:

import numpy as np
from scipy import stats

# Worked Example 2: Education level (rows) vs Party preference (columns)
observed = np.array([
    [50, 30, 20],   # High School
    [40, 45, 35],   # Bachelor's
    [20, 35, 45]    # Graduate
])

chi2_stat, p_value, dof, expected = stats.chi2_contingency(observed, correction=False)

print(f"Chi-square statistic: {chi2_stat:.3f}")
print(f"Degrees of freedom: {dof}")
print(f"p-value: {p_value:.6f}")
print("Expected frequencies:")
print(np.round(expected, 3))

# Effect size: Cramer's V
N = observed.sum()
k = min(observed.shape)
cramers_v = np.sqrt(chi2_stat / (N * (k - 1)))
print(f"Cramer's V: {cramers_v:.3f}")

Output:

Chi-square statistic: 24.327
Degrees of freedom: 4
p-value: 0.000069
Expected frequencies:
[[34.375 34.375 31.25 ]
 [41.25  41.25  37.5  ]
 [34.375 34.375 31.25 ]]
Cramer's V: 0.195

These figures match Worked Example 2 exactly. For the 2x2 case from Worked Example 1, scipy.stats.chi2_contingency applies Yates' correction automatically:

observed_2x2 = np.array([
    [30, 20],  # Male
    [15, 35]   # Female
])

# Default: Yates' correction applied automatically for 2x2 tables
chi2_yates, p_yates, dof, expected = stats.chi2_contingency(observed_2x2)
print(f"With Yates' correction: chi2={chi2_yates:.3f}, p={p_yates:.4f}")

# Explicitly disable it to match the uncorrected hand calculation
chi2_raw, p_raw, dof, expected = stats.chi2_contingency(observed_2x2, correction=False)
print(f"Without Yates' correction: chi2={chi2_raw:.3f}, p={p_raw:.4f}")

With Yates' correction: chi2=7.919, p=0.0049
Without Yates' correction: chi2=9.091, p=0.0026

Both match the hand-calculated values above exactly. For post-hoc inspection of which specific cells drive a significant result, standardized residuals are useful:

residuals = (observed - expected) / np.sqrt(expected)
print("Standardized residuals:")
print(np.round(residuals, 2))
# Cells with |residual| > ~2 are typically flagged as notable contributors

This returns a matrix the same shape as the contingency table; cells with a standardized residual larger than roughly 2 in absolute value are conventionally treated as meaningful contributors to the overall association, echoing the cell-by-cell breakdown already discussed in Worked Example 2.

How to Interpret Results

The significance level \( \alpha = 0.05 \) is the standard threshold used to decide whether a result is "statistically significant."

Checking Assumptions in Practice

• Independence of observations: a design question, not something visible in the data itself-confirm each subject contributes to only one cell and that subjects don't influence one another's category (e.g., repeated measurements on the same person violate this).
• Expected cell frequencies: inspect the expected-frequency table (shown automatically by scipy.stats.chi2_contingency)-if more than about 20% of cells have an expected frequency below 5, or any single expected frequency is below 1, consider Fisher's Exact Test instead.
• Category definitions: categories should be mutually exclusive and exhaustive-every subject fits into exactly one category per variable, with no overlap or leftover "other" bucket unless explicitly defined.
• Table size: larger tables with more categories increase degrees of freedom and can dilute a real but localized association across many small cells-inspecting standardized residuals, as shown in the Python section, helps identify where the association is actually concentrated.

Advantages

• Works directly with categorical data, requiring no assumption of normality, equal variance, or any underlying numeric distribution at all.
• Extends naturally to any number of rows and columns-2x2, 3x3, or much larger tables all use the same formula.
• Simple to compute by hand for small tables, and simple to interpret: bigger observed-vs-expected gaps mean stronger evidence of association.
• Widely implemented and reported, making results easy to communicate to non-technical audiences via a contingency table.

Limitations

• Requires sufficiently large expected cell frequencies (conventionally 5 or more) to trust the chi-square approximation-unreliable with small samples or many sparse categories.
• A significant result only establishes that an association exists somewhere in the table-it does not by itself indicate which specific category combinations are driving it, nor the direction of the relationship.
• Says nothing about causation-an association between two categorical variables does not imply either one causes the other.
• The chi-square statistic alone grows with sample size even for a fixed, small underlying effect, making an accompanying effect size such as Cramer's V essential for honest reporting.

When NOT to Use It

• Fisher's Exact Test: use instead when expected cell frequencies are small (commonly below 5), particularly in 2x2 tables, where the chi-square approximation becomes unreliable.
• Chi-Square Goodness-of-Fit Test: use instead when you have only one categorical variable and want to compare its observed distribution to a specific hypothesized distribution, rather than testing the relationship between two variables.
• McNemar's Test: use instead when the two categorical measurements come from the same subjects measured twice (e.g., before/after a treatment on a binary outcome) rather than independent subjects cross-classified on two variables.
• Logistic regression or log-linear models: consider instead when you need to control for additional variables, model interaction effects, or work with more than two categorical variables jointly.

Chi-Square Test of Independence vs Fisher's Exact Test vs Goodness-of-Fit Test

17.1 Chi-Square Test of Independence vs Fisher's Exact Test

17.2 Chi-Square Test of Independence vs Chi-Square Goodness-of-Fit Test

17.3 Chi-Square Test of Independence vs McNemar's Test

Common Misconceptions

• "A significant chi-square result proves one variable causes the other." No-the test only detects association, not causation. Confounding variables, reverse causation, or coincidence can all produce a significant result.
• "A larger chi-square statistic always means a stronger relationship." Not necessarily- see the effect size section, where Worked Example 2's larger \( \chi^2 \) and smaller p-value corresponded to a weaker Cramer's V than Worked Example 1's smaller, simpler 2x2 table.
• "The Chi-Square Test of Independence works fine with any sample size." It requires sufficiently large expected cell frequencies-typically 5 or more-to trust the chi-square approximation; smaller samples call for Fisher's Exact Test instead.
• "This test can be used on raw, continuous data directly." The Chi-Square Test of Independence requires categorical frequency counts. Continuous variables must first be binned into categories before this test applies, and that binning choice can itself affect the result.
• "A non-significant result proves the variables are truly independent." Failing to reject \( H_0 \) only means there was not enough evidence to detect an association with this sample-it does not prove the true association is exactly zero.

Interview Questions

1. Walk through how expected frequencies are computed for a contingency table cell, and explain the intuition behind the formula.
2. Why does the Chi-Square Test of Independence require expected cell frequencies of at least 5, and what happens to the test's reliability when this is violated?
3. Explain what the degrees of freedom \( (r-1)(c-1) \) represent conceptually for a contingency table.
4. What does it mean for the Chi-Square Test of Independence to detect "an association exists somewhere" without pinpointing exactly which cells are responsible, and how would you identify those cells?
5. Explain the difference between the Chi-Square Test of Independence and the Chi-Square Goodness-of-Fit Test.
6. How would you compute and interpret Cramer's V for a chi-square test result, and why is it necessary alongside the p-value?
7. What does Yates' continuity correction adjust for, and why is it applied specifically to 2x2 tables?
8. Under what circumstances would you switch from the Chi-Square Test of Independence to Fisher's Exact Test?
9. Why can a chi-square test become "significant" purely by increasing the sample size, even with a fixed, small underlying effect?
10. How would you handle a contingency table where several expected cell frequencies fall below 5-would you combine categories, switch tests, or collect more data, and what are the trade-offs of each?

Frequently Asked Questions

• What is the Chi-Square Test of Independence used for? +
  The Chi-Square Test of Independence is used to determine whether two categorical variables-such as gender and product preference, education level and voting choice, or treatment group and outcome category-are statistically associated, or whether observing one variable gives you no information about the other.
• What is the formula for the Chi-Square Test of Independence? +
  Arrange the data in a contingency table of observed frequencies. For each cell, compute the expected frequency as E = (row total x column total) / grand total-the count you would expect if the two variables were perfectly independent. The test statistic is chi-square = sum[(O - E)^2 / E] across every cell, following approximately a chi-square distribution with (r - 1)(c - 1) degrees of freedom, where r and c are the number of rows and columns.
• What is the difference between the Chi-Square Test of Independence and the Chi-Square Goodness-of-Fit Test? +
  The Chi-Square Test of Independence examines whether two categorical variables are related, using a two-dimensional contingency table. The Chi-Square Goodness-of-Fit Test instead checks whether a single categorical variable's observed frequencies match a specific hypothesized distribution (such as "equal proportions" or a known population distribution), using only one row of category counts.
• What are the assumptions of the Chi-Square Test of Independence? +
  Observations must be independent-each subject or unit contributes to exactly one cell of the table. The data must be frequency counts of categories, not means or raw scores. Every expected cell frequency should generally be at least 5 for the chi-square approximation to hold reliably; when many cells fall below that, Fisher's Exact Test is the standard alternative.
• How do you interpret a Chi-Square Test of Independence p-value? +
  If the p-value is below your chosen significance level (commonly alpha = 0.05), you reject the null hypothesis of independence and conclude the two variables are significantly associated. If the p-value is at or above alpha, there is not enough evidence of an association-the variables can reasonably be treated as independent given the data collected.
• What should I use instead of the Chi-Square Test of Independence with a small sample? +
  Use Fisher's Exact Test, particularly for 2x2 tables, whenever one or more expected cell frequencies fall below 5. Fisher's Exact Test computes an exact p-value directly from the hypergeometric distribution rather than relying on the chi-square distribution's large-sample approximation.
• What does Cramer's V tell you that the chi-square p-value doesn't? +
  The chi-square p-value only indicates whether an association is statistically detectable; it says nothing about how strong that association is, and grows automatically with larger sample sizes even for a trivial effect. Cramer's V rescales the chi-square statistic to a value between 0 and 1, giving a standardized measure of association strength that is comparable across tables of different sizes and sample sizes.
• Can the Chi-Square Test of Independence be used with more than two categorical variables? +
  The standard Chi-Square Test of Independence handles exactly two categorical variables arranged in a two-way contingency table. For three or more categorical variables analyzed jointly, log-linear models or a series of stratified chi-square tests are typically used instead.

Key Takeaways

• The Chi-Square Test of Independence checks whether two categorical variables are associated, by comparing observed counts in a contingency table against the counts expected under independence.
• It is ideal for survey data, demographic breakdowns, and any two-variable categorical comparison where neither variable can be meaningfully averaged or ranked.
• Core assumptions: independent observations, mutually exclusive categories, and expected cell frequencies generally at or above 5-when violated, Fisher's Exact Test is the standard fallback.
• In Python, scipy.stats.chi2_contingency(...) gets you the chi-square statistic, degrees of freedom, p-value, and expected frequencies in one call; pair it with Cramer's V for a complete effect size picture.
• If \( p < 0.05 \), the variables are significantly associated; if \( p \geq 0.05 \), there isn't enough evidence of an association-but a larger chi-square statistic and smaller p-value do not automatically mean a stronger effect, as both worked examples above demonstrate.
• For 2x2 tables specifically, consider whether Yates' continuity correction is appropriate for your sample size, and always report an effect size such as phi or Cramer's V alongside the p-value.

The Chi-Square Test of Independence earns its place as the standard tool for categorical association precisely because so much real-world data-survey responses, demographic categories, treatment outcomes, pass/fail results-comes in categories rather than numbers. Its logic is refreshingly direct: build a contingency table, compute what independence would predict for every cell, and measure how far reality strays from that prediction.

The two worked examples above, a small 2x2 gender-versus-preference table and a larger 3x3 education-versus-voting table, show both ends of this test's range-and a useful lesson in between: the larger table produced a bigger chi-square statistic and a far smaller p-value, yet its Cramer's V revealed a proportionally weaker association than the smaller table's phi coefficient. Reporting the chi-square statistic and p-value alongside an appropriate effect size, and inspecting standardized residuals when a result is significant, as demonstrated throughout this guide, gives a complete and honest picture that the omnibus p-value by itself cannot.