Assumption: Let $P$ be a Markov chain with stationary measure $\pi$.

Theorem:

  1. Convergence condition: Only when the chain is irreducible, aperiodic, and positive recurrent does the limit of the transition probabilities exist and equal the stationary distribution:

    $$ \lim_{n \to \infty} p_{ij}^{(n)} = \tilde{\pi}_j, \quad \text{where } \sum_j \tilde{\pi}_j = 1. $$
  2. Vanishing probabilities: If the chain is transient or null recurrent, then regardless of whether a stationary measure exists,

    $$ \lim_{n \to \infty} p_{ij}^{(n)} = 0, \quad \forall i,j \in S. $$

Therefore, even if

$$ \lim_{n \to \infty} \left|p_{ij}^{(n)} - p_{ik}^{(n)}\right| = 0 $$

and a stationary measure exists, if the chain is null recurrent, that measure cannot be normalized into a limiting probability distribution $\tilde{\pi}_j$.