Stationary Measure

Definition

Define stationary measure, denoted as $\pi$, such that

  • for all $i \in S$, $\pi_i \ge 0$
  • $\pi \ne 0$
  • for all $j \in S$,
$$ \pi_j=\sum_{i\in S}\pi_i p_{ij} $$

which means

$$ \pi P=\pi $$

Stationary Distribution

Definition

Stationary distribution refers to a stationary measure such that

$$ \sum_i \pi_i=1 $$

In other words, a stationary measure that can be normalized to $1$ is a stationary distribution.

A stationary measure that is not a stationary distribution can be infinite.

Explicit Form

In Markov Chain, stationary measure $\pi$ is defined as a row vector.

Assume the state space is

$$ S=\{1,2,\dots,N\} $$

Then the explicit form of

$$ \pi P=\pi $$

is

$$ [\pi_1\ \pi_2\ \cdots\ \pi_N] \begin{bmatrix} p_{11} & p_{12} & \cdots & p_{1N}\ p_{21} & p_{22} & \cdots & p_{2N}\ \vdots & \vdots & \ddots & \vdots\ p_{N1} & p_{N2} & \cdots & p_{NN} \end{bmatrix}

[\pi_1\ \pi_2\ \cdots\ \pi_N] $$

Componentwise, this becomes

$$ \pi_j=\sum_{i\in S}\pi_i p_{ij} $$

So the system can be written as

$$ \begin{cases} \pi_1=\pi_1p_{11}+\pi_2p_{21}+\cdots+\pi_Np_{N1}\\ \pi_2=\pi_1p_{12}+\pi_2p_{22}+\cdots+\pi_Np_{N2}\\ \vdots\\ \pi_N=\pi_1p_{1N}+\pi_2p_{2N}+\cdots+\pi_Np_{NN} \end{cases} $$

This is essentially a linear system, or equivalently, the eigenvector problem for $P^\top$ with eigenvalue $1$.

Example: Measure vs Distribution

Let the state space be

$$ S=\{1,2\} $$

with transition matrix

$$ P= \begin{bmatrix} 0.8 & 0.2\\ 0.4 & 0.6 \end{bmatrix} $$

To solve

$$ \pi P=\pi $$

write

$$ [\pi_1\ \pi_2] \begin{bmatrix} 0.8 & 0.2\ 0.4 & 0.6 \end{bmatrix}

[\pi_1\ \pi_2] $$

Expanding gives

$$ \begin{cases} 0.8\pi_1+0.4\pi_2=\pi_1\\ 0.2\pi_1+0.6\pi_2=\pi_2 \end{cases} $$

which simplifies to

$$ \pi_1=2\pi_2 $$

Stationary Measure

Any nonzero vector satisfying $\pi_1=2\pi_2$ and $\pi_i\ge 0$ is a stationary measure.

Examples:

  • $\pi=[2,1]$
  • $\pi=[4,2]$
  • $\pi=[200,100]$

These vectors remain unchanged after left multiplication by $P$.

Stationary Distribution

To be a stationary distribution, one must additionally satisfy

$$ \pi_1+\pi_2=1 $$

Substitute $\pi_1=2\pi_2$ into the normalization condition:

$$ 2\pi_2+\pi_2=1 $$$$ 3\pi_2=1 $$$$ \pi_2=\frac13,\quad \pi_1=\frac23 $$

Hence the unique stationary distribution is

$$ \pi=\begin{bmatrix}\frac23 & \frac13\end{bmatrix} $$

Stationary distribution is obtained by normalizing a stationary measure by the sum of its entries.

Learned

Any convex combination of stationary distributions is also a stationary distribution.

There also exists Markov Chain that has infinitely many stationary distributions.