p-values for z-tests and t-tests
A hypothesis test compares a null hypothesis \(H_0\) to an alternative \(H_1\) using a test statistic.
The p-value measures how extreme the observed test statistic is assuming \(H_0\) is true.
1) The basic ingredients
- Null hypothesis \(H_0\): a reference value \(\theta=\theta_0\) (e.g., \(\mu=\mu_0\)).
- Alternative \(H_1\): right-tailed \((\theta>\theta_0)\), left-tailed \((\theta<\theta_0)\), or two-tailed \((\theta\ne\theta_0)\).
- Test statistic: a standardized quantity like a z or t value.
- p-value: tail area(s) beyond the observed statistic under the \(H_0\) distribution.
- Decision rule: reject \(H_0\) if \(p \le \alpha\).
2) z-test (standard normal)
A one-sample z test for a mean is used when the population standard deviation \(\sigma\) is known:
\[
z=\frac{\bar x-\mu_0}{\sigma/\sqrt{n}}.
\]
Under \(H_0\), the statistic is approximately standard normal \(Z\sim\mathcal N(0,1)\). Let \(\Phi(\cdot)\) be the standard normal CDF.
Then the p-values are:
\[
\begin{aligned}
\text{Right-tailed: } & p = P(Z \ge z) = 1-\Phi(z) \\
\text{Left-tailed: } & p = P(Z \le z) = \Phi(z) \\
\text{Two-tailed: } & p = 2\min\big(\Phi(z),\,1-\Phi(z)\big)
\end{aligned}
\]
3) t-test (Student t)
When \(\sigma\) is unknown (common in practice), the one-sample mean test uses:
\[
t=\frac{\bar x-\mu_0}{s/\sqrt{n}}, \qquad df=n-1.
\]
Under \(H_0\), the statistic follows a Student t distribution with \(df\) degrees of freedom.
The p-value formulas are the same as above, but \(\Phi(\cdot)\) is replaced by the t CDF \(F_t(\cdot)\):
\[
\begin{aligned}
\text{Right-tailed: } & p = 1-F_t(t) \\
\text{Left-tailed: } & p = F_t(t) \\
\text{Two-tailed: } & p = 2\min\big(F_t(t),\,1-F_t(t)\big)
\end{aligned}
\]
4) What does “reject at \(\alpha\)” mean?
The significance level \(\alpha\) is a threshold chosen before seeing the data. If we reject \(H_0\) when \(p\le\alpha\),
then \(\alpha\) controls the long-run probability of a Type I error (rejecting a true null).
5) Visual interpretation: tail shading
The calculator shades the tail area corresponding to the p-value:
- right-tailed: area to the right of the statistic,
- left-tailed: area to the left of the statistic,
- two-tailed: both tails beyond \(\pm|stat|\).
6) University note: power and effect size
The p-value is not the probability that \(H_0\) is true. Power is the probability of rejecting \(H_0\) when a specific alternative is true.
Power depends on the effect size, sample size \(n\), variability (via \(SE\)), and the chosen \(\alpha\).
