One-way ANOVA: comparing multiple group means
A one-way ANOVA (analysis of variance) tests whether at least one group mean differs across
\(k\) independent groups. It is a generalization of the two-sample t-test to more than two groups.
1) Model and hypotheses
For group \(i\) with observations \(y_{ij}\) (where \(j=1,\dots,n_i\)), a classic one-way ANOVA model is:
\[
y_{ij}=\mu_i+\varepsilon_{ij},\qquad \mathbb{E}[\varepsilon_{ij}]=0,\quad \mathrm{Var}(\varepsilon_{ij})=\sigma^2.
\]
The null hypothesis states all means are equal:
\[
H_0:\mu_1=\mu_2=\cdots=\mu_k
\qquad\text{vs}\qquad
H_1:\text{not all }\mu_i\text{ are equal}.
\]
2) Intuition: between-group vs within-group variation
ANOVA compares two kinds of variability:
- Within-group variation: how spread out the data are inside each group (noise).
- Between-group variation: how far group means are from the overall mean (signal).
If between-group variation is large relative to within-group variation, the data support different means.
3) Definitions: means and totals
Let \(N=\sum_{i=1}^{k} n_i\) be the total sample size. Define group means and the grand mean:
\[
\bar y_i=\frac{1}{n_i}\sum_{j=1}^{n_i} y_{ij},
\qquad
\bar y=\frac{1}{N}\sum_{i=1}^{k}\sum_{j=1}^{n_i} y_{ij}
=\frac{1}{N}\sum_{i=1}^{k} n_i\bar y_i.
\]
4) Sums of squares
ANOVA is built from sums of squares:
\[
SS_B=\sum_{i=1}^{k} n_i(\bar y_i-\bar y)^2
\quad\text{(between groups)}
\]
\[
SS_W=\sum_{i=1}^{k}\sum_{j=1}^{n_i} (y_{ij}-\bar y_i)^2
\quad\text{(within groups)}
\]
\[
SS_T=\sum_{i=1}^{k}\sum_{j=1}^{n_i} (y_{ij}-\bar y)^2
\quad\text{(total)}
\]
With standard definitions, the decomposition holds: \(\;SS_T = SS_B + SS_W.\)
5) Degrees of freedom and mean squares
Each sum of squares has an associated degrees of freedom (df):
\[
df_B = k-1,\qquad df_W = N-k,\qquad df_T=N-1.
\]
Convert SS to mean squares (MS) by dividing by df:
\[
MSB=\frac{SS_B}{df_B},\qquad MSW=\frac{SS_W}{df_W}.
\]
6) F statistic and p-value
The one-way ANOVA test statistic is:
\[
\boxed{F=\frac{MSB}{MSW}}.
\]
Under \(H_0\) (and assumptions), \(F\) follows an F distribution with \((df_B, df_W)\) degrees of freedom:
\[
F \sim F_{df_B,\;df_W}.
\]
The p-value is the upper-tail probability:
\[
p=P\!\left(F_{df_B,\;df_W}\ge F_{\text{obs}}\right).
\]
7) ANOVA table
Results are usually displayed in an ANOVA table:
8) Post-hoc tests (Tukey HSD teaser)
A significant ANOVA tells you that at least one mean differs, but not which groups differ.
Post-hoc multiple comparison methods (like Tukey’s HSD) compare pairs of groups while controlling family-wise error.
University note: Tukey HSD uses the studentized range distribution and depends on \(MSW\), group sizes, and a critical value \(q\).
9) Assumptions and common alternatives
- Independence of observations (by design).
- Normality of residuals within groups (ANOVA is fairly robust for moderate \(n\)).
- Equal variances across groups (homoscedasticity).
If equal variances are doubtful, consider Welch’s ANOVA. If distributions are strongly non-normal or ordinal, consider
Kruskal–Wallis.
10) University extensions
More advanced designs include two-way ANOVA (two factors), ANCOVA (covariates), and MANOVA (multiple responses).
These use similar “variance partitioning” ideas but with richer model structure.
