Normalization
Definition
Normalization in probability refers to ensuring that a function integrates (or sums) to 1 so that it defines a valid probability distribution.
- Continuous: $\displaystyle\int_{-\infty}^{\infty} f(x)\, dx = 1$.
- Discrete: $\displaystyle\sum_{x} p(x) = 1$.
Normalizing Constant
Given an unnormalized density $\tilde{p}(x) \geq 0$, the normalizing constant is
$$ Z = \int \tilde{p}(x)\, dx, \qquad p(x) = \frac{\tilde{p}(x)}{Z}. $$Computing $Z$ is often the central difficulty in Bayesian Inference, motivating approximations like Variational Inference (VI) and Monte Carlo Estimation.