Key idea: decreasing \( \alpha \) usually increases \( \beta \) when everything else is fixed
The phrase decreasing alpha from .05 to .01 effect on beta refers to a standard trade-off in hypothesis testing: making a test stricter against Type I errors generally makes it easier to miss a real effect (Type II error).
Definitions
- \( \alpha \): probability of a Type I error (rejecting \(H_0\) when \(H_0\) is true).
- \( \beta \): probability of a Type II error (failing to reject \(H_0\) when \(H_1\) is true).
- \( 1-\beta \): power (probability of correctly rejecting \(H_0\) when \(H_1\) is true).
Why \( \beta \) tends to rise when \( \alpha \) is lowered
Holding the sample size \(n\), variability, and the alternative effect size fixed, lowering \( \alpha \) shrinks the rejection region. Equivalently, the critical value becomes more extreme, so rejecting \(H_0\) requires stronger evidence. Under the alternative \(H_1\), that makes rejection less likely, increasing \( \beta \) and reducing power.
Concrete demonstration with a one-sided z-test
Assume a right-tailed test with known \( \sigma \): \[ H_0:\mu=\mu_0 \quad\text{vs}\quad H_1:\mu>\mu_0, \qquad Z=\frac{\bar{X}-\mu_0}{\sigma/\sqrt{n}}. \] The rejection rule is \( Z \ge z_{1-\alpha} \), equivalently \[ \bar{X} \ge \mu_0 + z_{1-\alpha}\cdot\frac{\sigma}{\sqrt{n}}. \]
If the true mean under the alternative is \( \mu_1>\mu_0 \), then \[ \beta =P_{\mu_1}\!\left(\bar{X}<\mu_0+z_{1-\alpha}\cdot\frac{\sigma}{\sqrt{n}}\right) =\Phi\!\left(\frac{\mu_0+z_{1-\alpha}\cdot\frac{\sigma}{\sqrt{n}}-\mu_1}{\sigma/\sqrt{n}}\right) =\Phi\!\left(z_{1-\alpha}-\frac{\mu_1-\mu_0}{\sigma/\sqrt{n}}\right). \] Lowering \( \alpha \) increases \( z_{1-\alpha} \), which increases the argument of \( \Phi(\cdot) \), so \( \beta \) increases.
Numerical example comparing \( \alpha=0.05 \) vs \( \alpha=0.01 \)
Use \( \mu_0=100 \), \( \mu_1=103 \), \( \sigma=10 \), \( n=25 \). Then \( \sigma/\sqrt{n}=10/5=2 \).
Critical values: \[ z_{1-0.05}=z_{0.95}\approx 1.645, \qquad z_{1-0.01}=z_{0.99}\approx 2.326. \]
| Significance level | Critical z | Cutoff for \( \bar{X} \) | \( \beta \) when \( \mu=\mu_1=103 \) | Power \( 1-\beta \) |
|---|---|---|---|---|
| \( \alpha=0.05 \) | \( z_{0.95}\approx 1.645 \) | \( 100 + 1.645\cdot 2 = 103.290 \) | \( \beta=\Phi\!\left(\frac{103.290-103}{2}\right) =\Phi(0.145)\approx 0.558 \) | \( 1-0.558=0.442 \) |
| \( \alpha=0.01 \) | \( z_{0.99}\approx 2.326 \) | \( 100 + 2.326\cdot 2 = 104.652 \) | \( \beta=\Phi\!\left(\frac{104.652-103}{2}\right) =\Phi(0.826)\approx 0.795 \) | \( 1-0.795=0.205 \) |
Interpretation: decreasing \( \alpha \) from \(0.05\) to \(0.01\) made rejection harder, so \( \beta \) increased from about \(0.558\) to about \(0.795\), and power dropped from about \(0.442\) to about \(0.205\) (with \(n\), \( \sigma \), and \( \mu_1-\mu_0 \) fixed).
How to reduce \( \alpha \) without inflating \( \beta \)
If the goal is to decrease \( \alpha \) while keeping \( \beta \) (or power) about the same, the test must gain sensitivity—most commonly by increasing \(n\). For the one-sided z-test above, setting a target \( \beta \) leads to the planning identity: \[ \beta=\Phi\!\left(z_{1-\alpha}-\frac{\mu_1-\mu_0}{\sigma/\sqrt{n}}\right) \quad\Longrightarrow\quad n=\left(\frac{\bigl(z_{1-\alpha}-z_{\beta}\bigr)\cdot\sigma}{\mu_1-\mu_0}\right)^2. \] Using the example’s target \( \beta\approx 0.558 \) (so \( z_{\beta}\approx 0.145 \)) with \( \alpha=0.01 \) gives \[ n\approx \left(\frac{(2.326-0.145)\cdot 10}{3}\right)^2 \approx 52.875, \] so a practical choice is \( n=53 \), larger than the original \( n=25 \).
Bottom line
Decreasing \( \alpha \) from \(0.05\) to \(0.01\) generally increases \( \beta \) and decreases power because the rejection threshold becomes more extreme; maintaining power requires compensating changes such as increasing sample size or effect size, or reducing variability.