Bayesian probabilistic framework
bayesian probabilistic reasoning represents uncertainty about events and unknown parameters with probability. Evidence updates those probabilities through conditional probability, producing a posterior probability (for events) or a posterior distribution (for parameters).
Central identity. \[ \text{posterior} \;\propto\; \text{likelihood} \cdot \text{prior}. \]
Normalization converts proportionality into a proper probability or probability density.
Bayes’ theorem for events
For events \(A\) and \(B\) with \(P(B) > 0\), Bayes’ theorem expresses a conditional probability in terms of the reverse conditioning:
\[ P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}. \]The denominator \(P(B)\) is the marginal probability of \(B\), often expanded by the law of total probability across a partition \(\{A, A^{c}\}\):
\[ P(B) = P(B \mid A)\,P(A) + P(B \mid A^{c})\,P(A^{c}). \]Worked example with posterior probability
A screening test has sensitivity \(P(+ \mid D) = 0.99\) and specificity \(P(- \mid D^{c}) = 0.95\). Disease prevalence is \(P(D) = 0.01\), and a positive test result \(+\) is observed.
| Quantity | Value | Meaning |
|---|---|---|
| \(P(D)\) | 0.01 | Prior probability of disease |
| \(P(+ \mid D)\) | 0.99 | Sensitivity |
| \(P(+ \mid D^{c})\) | \(1 - 0.95 = 0.05\) | False-positive rate |
The marginal probability of a positive test is \[ P(+) = 0.99 \cdot 0.01 + 0.05 \cdot 0.99 = 0.0099 + 0.0495 = 0.0594. \] The posterior probability of disease given a positive test is \[ P(D \mid +) = \frac{0.99 \cdot 0.01}{0.0594} \approx 0.1667. \]
A large sensitivity does not guarantee a large posterior probability when the prior prevalence is small and false positives are non-negligible.
Bayes’ theorem for parameters
For an unknown parameter \(\theta\) and observed data \(y\), the Bayesian update uses densities:
\[ p(\theta \mid y) = \frac{p(y \mid \theta)\,p(\theta)}{p(y)}, \qquad p(y) = \int p(y \mid \theta)\,p(\theta)\,d\theta. \]- Prior density \(p(\theta)\): uncertainty about \(\theta\) before observing \(y\).
- Likelihood \(p(y \mid \theta)\): data model viewed as a function of \(\theta\).
- Posterior density \(p(\theta \mid y)\): updated uncertainty after observing \(y\).
- Evidence \(p(y)\): normalizing constant (marginal likelihood).
Conjugate normal example and credible interval
A normal prior and a normal likelihood yield a normal posterior. Assume \(\theta\) is a mean parameter with prior \(\theta \sim N(\mu_0,\tau_0^2)\), and one observation \(y\) with known \(\sigma^2\) satisfies \(y \mid \theta \sim N(\theta,\sigma^2)\).
\[ \tau_1^2 = \left(\frac{1}{\tau_0^2} + \frac{1}{\sigma^2}\right)^{-1}, \qquad \mu_1 = \tau_1^2\left(\frac{\mu_0}{\tau_0^2} + \frac{y}{\sigma^2}\right). \]With \(\mu_0 = 0\), \(\tau_0^2 = 1\), \(y = 1.5\), and \(\sigma^2 = 1\), \[ \tau_1^2 = \left(1 + 1\right)^{-1} = 0.5, \qquad \mu_1 = 0.5\left(0 + 1.5\right) = 0.75. \] The posterior is \(\theta \mid y \sim N(0.75, 0.5)\).
A central 95% credible interval for \(\theta\) under this posterior is \[ 0.75 \pm 1.96 \cdot \sqrt{0.5} \approx 0.75 \pm 1.96 \cdot 0.7071 \approx 0.75 \pm 1.3859, \] giving \((-0.636,\; 2.136)\) after rounding.
Visualization of Bayesian updating
Common pitfalls
Posterior probability depends on the prior and the likelihood; ignoring the prior base rate can produce severely distorted interpretations of evidence. A posterior distribution describes uncertainty about a parameter conditioned on the observed data and the chosen model, while model misspecification can dominate the update regardless of algebraic correctness. Correlation between variables and causal claims belong to a different inferential layer than Bayesian updating of probabilities.