ARMA Process
An ARMA($p,q$) process combines an autoregressive part and a moving average part.
1. Definition
$\{X_t\}$ is an ARMA($p,q$) process if
$$X_t - \phi_1 X_{t-1} - \cdots - \phi_p X_{t-p} = Z_t + \theta_1 Z_{t-1} + \cdots + \theta_q Z_{t-q}$$where $\{Z_t\} \sim \text{WN}(0, \sigma^2)$.
In operator form:
$$\phi(B) X_t = \theta(B) Z_t$$where $\phi(B) = 1 - \phi_1 B - \cdots - \phi_p B^p$ and $\theta(B) = 1 + \theta_1 B + \cdots + \theta_q B^q$.
2. Special Cases
| Model | $p$ | $q$ |
|---|---|---|
| White noise | 0 | 0 |
| MA($q$) | 0 | $q$ |
| AR($p$) | $p$ | 0 |
| ARMA($p,q$) | $p$ | $q$ |
3. Stationarity and Invertibility
- Stationary iff all roots of $\phi(z) = 0$ lie outside the unit circle
- Invertible iff all roots of $\theta(z) = 0$ lie outside the unit circle
These conditions depend only on the AR and MA polynomials respectively.
4. ARMA(1,1) Example
$$X_t - \phi X_{t-1} = Z_t + \theta Z_{t-1}$$ACVF:
$$\gamma_X(0) = \sigma^2 \frac{1 + 2\phi\theta + \theta^2}{1 - \phi^2}$$$$\gamma_X(1) = \sigma^2 \frac{(1 + \phi\theta)(\phi + \theta)}{1 - \phi^2}$$$$\gamma_X(h) = \phi \cdot \gamma_X(h-1) \quad \text{for } h \geq 2$$5. ACF Behavior
- The ACF of ARMA($p,q$) tails off (like AR) but starts with $q$ “irregular” values before settling into the AR-like decay pattern
- Neither cuts off nor has pure exponential decay from lag 0
6. ARMA Plus Noise
If $\{X_t\}$ is ARMA($p,q$) and $\{W_t\} \sim \text{WN}(0, \sigma_W^2)$ is uncorrelated with $\{Z_t\}$, then $Y_t = X_t + W_t$ is stationary with
$$\gamma_Y(h) = \gamma_X(h) + \sigma_W^2 \cdot \mathbf{1}_{h=0}$$