Recurrence and Transience

Definition

Recurrence state:

$$ f_{ii}=1 $$

Transient state:

$$ f_{ii}<1 $$

$$ E_i\!\left[\sum_{n=0}^{\infty}\mathbf{1}(X_n=i)\right]=\sum_{n=0}^{\infty}P_i(X_n=i) $$

every departure from $i$ results possibility of $1-f_{ii}>0$ that $i$ never returns.

The amount of $N_i$ times visiting follows geometric distribution, which

$$ P_i(N_i=k)=f_{ii}^{\,k-1}(1-f_{ii}) $$

and

$$ \mathbb{E}_i[N_i]=\sum_{n=0}^{\infty}P_{ii}^{(n)}=\frac{1}{1-f_{ii}}<\infty $$

by strong markov property, if returning to itself in Prob. of $1$, then it’s equivalent to a fresh start. So,

$$ \sum_{n=0}^{\infty}P_{ii}^{(n)}=\infty $$

in this case.