Molecular Note: AR(1) Three Regimes

Linked Atomic Concepts: Autoregressive Process, AR(1) — Causal Case, AR(1) — Random Walk, AR(1) — Non-causal Stationary Case, AR(1) — Causality vs Stationarity Summary

Scene-Setting

You’re given $X_t = \phi X_{t-1} + Z_t$. Depending on $|\phi|$, this single equation produces three completely different processes. You need to know what happens in each regime and why.

Concept Chain

The Model: AR(p) Model with $p = 1$

$$X_t = \phi X_{t-1} + Z_t, \qquad Z_t \sim \text{WN}(0, \sigma^2)$$

Operator form: $(1 - \phi B)X_t = Z_t$.

Regime 1: $|\phi| < 1$ → AR(1) — Causal Case

Representation: $X_t = \sum_{j=0}^{\infty} \phi^j Z_{t-j}$ (causal MA(∞)).

Properties: $E(X_t) = 0$, $\gamma(0) = \sigma^2/(1-\phi^2)$, $\gamma(h) = \phi^h \gamma(0)$, $\rho(h) = \phi^h$.

ACF: Exponential decay. $\phi > 0$ → monotone; $\phi < 0$ → alternating sign.

Both causal and stationary.

Regime 2: $|\phi| = 1$ → AR(1) — Random Walk

$X_t = X_0 + \sum_{j=1}^t Z_j$. $\text{Var}(X_t) = \text{Var}(X_0) + t\sigma^2$ → diverges.

Not stationary, not causal.

Sample ACF shows very slow decay → visual indicator of non-stationarity → apply differencing.

Regime 3: $|\phi| > 1$ → AR(1) — Non-causal Stationary Case

Solve forward: $X_t = -\sum_{j=1}^{\infty}\phi^{-j}Z_{t+j}$. Converges since $|\phi^{-1}| < 1$.

Stationary but not causal (depends on future noise).

Comparison Table

| $|\phi|$ | Causal? | Stationary? | Representation | ACF | |—|—|—|—|—| | $< 1$ | ✓ | ✓ | $\sum_{j=0}^{\infty}\phi^j Z_{t-j}$ | $\phi^h$ (exp. decay) | | $= 1$ | ✗ | ✗ | $X_0 + \sum_{j=1}^t Z_j$ | Slow linear decay | | $> 1$ | ✗ | ✓ | $-\sum_{j=1}^{\infty}\phi^{-j}Z_{t+j}$ | (exists but non-causal) |

Worked Pipeline

Given: $X_t = 0.6 X_{t-1} + Z_t$, $\sigma^2 = 4$.

  1. $|\phi| = 0.6 < 1$ → causal, stationary.
  2. $\gamma(0) = 4/(1 - 0.36) = 4/0.64 = 6.25$.
  3. $\gamma(1) = 0.6 \times 6.25 = 3.75$; $\gamma(2) = 0.36 \times 6.25 = 2.25$.
  4. $\rho(h) = 0.6^h$: $\rho(1) = 0.6$, $\rho(2) = 0.36$, $\rho(3) = 0.216$, …
  5. One-step prediction: $P_n X_{n+1} = 0.6 X_n$.
  6. MSE for one-step prediction: $\sigma^2 = 4$ (noise variance).
  7. 95% PI: $0.6 X_n \pm 1.96\sqrt{4} = 0.6 X_n \pm 3.92$.

Exam Patterns

Pattern 1: “Prove $X_t = \phi X_{t-1} + Z_t$ with $|\phi|<1$ is stationary”

  • Key move: Show causal representation $\sum \phi^j Z_{t-j}$. Then verify $E(X_t) = 0$, $\text{Var}(X_t) = \sigma^2/(1-\phi^2)$ (independent of $t$), $\gamma(h) = \phi^h\gamma(0)$ (depends only on $h$).

Pattern 2: “Random walk: find mean, ACVF, prove non-stationary, prove differenced series is stationary” (MT2 Problem 1)

  • Key move for non-stationarity: $\text{Var}(X_t) = t\sigma^2$ depends on $t$.
  • Key move for differenced: $\nabla X_t = \mu + Z_t$ → iid (shifted), clearly stationary.

Pattern 3: “Compute prediction and prediction interval for AR(1)”

  • Key move: $P_n X_{n+1} = \phi X_n$, MSE $= \sigma^2$, PI $= \phi X_n \pm 1.96\sigma$.