Bayesian Inference is a framework for updating uncertainty about an unknown quantity after observing data.

Given parameter $\theta$ and observed data $D$,

$$ p(\theta \mid D)=\frac{p(D \mid \theta)\,p(\theta)}{p(D)} $$
  • $p(\theta)$: prior distribution
  • $p(D \mid \theta)$: likelihood
  • $p(\theta \mid D)$: posterior distribution
  • $p(D)$: evidence or marginal likelihood

The evidence is obtained by integrating over all possible parameter values:

$$ p(D)=\int p(D \mid \theta)p(\theta)\,d\theta $$

For discrete $\theta$, replace the integral with a sum.

A proportional form is often used:

$$ p(\theta \mid D)\propto p(D \mid \theta)p(\theta) $$

This is sufficient when the normalizing constant $p(D)$ is difficult to compute.