STA457 Formula Index
White Noise WN$(0, \sigma^2)$
| Property | Formula |
|---|---|
| Mean | $E(Z_t) = 0$ |
| Variance | $\text{Var}(Z_t) = \sigma^2$ |
| ACVF | $\gamma(h) = \sigma^2 \cdot \mathbf{1}_{h=0}$ |
| ACF | $\rho(h) = \mathbf{1}_{h=0}$ |
| Stationarity | Weakly stationary always |
MA(1): $X_t = Z_t + \theta Z_{t-1}$
| Property | Formula |
|---|---|
| Mean | $0$ |
| $\gamma(0)$ | $(1 + \theta^2)\sigma^2$ |
| $\gamma(1)$ | $\theta\sigma^2$ |
| $\gamma(h),\; \|h\| \geq 2$ | $0$ |
| $\rho(1)$ | $\dfrac{\theta}{1+\theta^2}$ |
| ACF signature | Cutoff after lag 1 |
| Stationarity | Always |
MA(2): $X_t = Z_t + \theta_1 Z_{t-1} + \theta_2 Z_{t-2}$
| Property | Formula |
|---|---|
| Mean | $0$ |
| $\gamma(0)$ | $(1 + \theta_1^2 + \theta_2^2)\sigma^2$ |
| $\gamma(1)$ | $(\theta_1 + \theta_1\theta_2)\sigma^2$ |
| $\gamma(2)$ | $\theta_2\sigma^2$ |
| $\gamma(h),\; \|h\| \geq 3$ | $0$ |
| $\rho(1)$ | $\dfrac{\theta_1 + \theta_1\theta_2}{1+\theta_1^2+\theta_2^2}$ |
| $\rho(2)$ | $\dfrac{\theta_2}{1+\theta_1^2+\theta_2^2}$ |
| ACF signature | Cutoff after lag 2 |
MA(q) General: $X_t = \theta(B)Z_t$
| Property | Formula |
|---|---|
| Mean | $0$ |
| $\gamma(0)$ | $(1 + \theta_1^2 + \cdots + \theta_q^2)\sigma^2$ |
| $\gamma(h),\; 1 \leq h \leq q$ | $\sigma^2 \sum_{j=0}^{q-h}\theta_j\theta_{j+h}$ where $\theta_0 = 1$ |
| $\gamma(h),\; h > q$ | $0$ |
| ACF signature | Cutoff after lag $q$ |
| Stationarity | Always |
AR(1): $X_t = \phi X_{t-1} + Z_t$, $\|\phi\| < 1$
| Property | Formula |
|---|---|
| Mean | $0$ |
| Causal form | $X_t = \sum_{j=0}^{\infty}\phi^j Z_{t-j}$ |
| $\gamma(0)$ | $\dfrac{\sigma^2}{1 - \phi^2}$ |
| $\gamma(h)$ | $\phi^h \cdot \dfrac{\sigma^2}{1-\phi^2}$ |
| $\rho(h)$ | $\phi^h$ |
| ACF signature | Exponential decay (monotone if $\phi>0$, alternating if $\phi<0$) |
| Stationarity | Causal + stationary |
| $P_n X_{n+1}$ | $\phi X_n$ |
| 1-step MSE | $\sigma^2$ |
AR(1) Random Walk: $\|\phi\| = 1$
| Property | Formula |
|---|---|
| Expansion | $X_t = X_0 + \sum_{j=1}^t Z_j$ |
| Mean | $E(X_t) = E(X_0)$ (or $t\mu$ with drift) |
| $\text{Var}(X_t)$ | $\text{Var}(X_0) + t\sigma^2$ |
| Stationarity | Not stationary |
| Fix | Differencing: $\nabla X_t = \mu + Z_t$ (stationary) |
AR(p) General: $\phi(B)X_t = Z_t$
| Property | Formula |
|---|---|
| Stationarity condition | All roots of $\phi(z) = 0$ outside unit circle |
| $P_n X_{n+1}$ (causal, $n \geq p$) | $\phi_1 X_n + \phi_2 X_{n-1} + \cdots + \phi_p X_{n+1-p}$ |
| 1-step MSE | $\sigma^2$ |
| ACF signature | Tails off (exponential/damped oscillation decay) |
ARMA(1,1): $X_t = \phi X_{t-1} + Z_t + \theta Z_{t-1}$, $\|\phi\| < 1$
| Property | Formula |
|---|---|
| Causal form | $X_t = Z_t + (\phi+\theta)\sum_{j=1}^{\infty}\phi^{j-1}Z_{t-j}$ |
| $\psi_0$ | $1$ |
| $\psi_j\;(j \geq 1)$ | $(\phi+\theta)\phi^{j-1}$ |
| ACF signature | Tails off |
| Stationarity | Causal + stationary (root of $\phi(z)$ outside unit circle) |
ARMA(p,q) General: $\phi(B)X_t = \theta(B)Z_t$
| Property | Formula |
|---|---|
| Stationarity/causality | All roots of $\phi(z) = 0$ outside unit circle |
| ACF signature | Tails off |
Best Linear Prediction
| Item | Formula |
|---|---|
| General BLP | $P(U\|\mathbf{W}) = E(U) + \mathbf{a}'(\mathbf{W} - E(\mathbf{W}))$, solve $\Gamma\mathbf{a} = \text{Cov}(U, \mathbf{W})$ |
| Stationary series | $P_n X_{n+h} = \mu + \mathbf{a}_n'(\mathbf{X} - \mu\mathbf{1})$, solve $\Gamma_n \mathbf{a}_n = \boldsymbol{\gamma}_n(h)$ |
| $\Gamma_n$ structure | $(\Gamma_n)_{ij} = \gamma(\|i-j\|)$ (Toeplitz) |
| $\boldsymbol{\gamma}_n(h)$ | $(\gamma(h),\;\gamma(h+1),\;\ldots,\;\gamma(h+n-1))'$ |
| MSE | $\gamma(0) - \mathbf{a}_n'\boldsymbol{\gamma}_n(h)$ |
| $(1-\alpha)$ PI | $P_N X_{N+h} \pm z_{\alpha/2}\sqrt{\text{MSE}}$ |
| 95% PI | $P_N X_{N+h} \pm 1.96\sqrt{\text{MSE}}$ |
Trend Estimation / Elimination
| Method | Formula |
|---|---|
| Regression | $\hat{m}_t = \hat{a} + \hat{b}t$ |
| MA filter (odd $d=2q+1$) | $\hat{m}_t = \frac{1}{d}\sum_{j=-q}^{q}x_{t+j}$ |
| MA filter (even $d=2q$) | $\hat{m}_t = \frac{0.5x_{t-q} + x_{t-q+1}+\cdots+x_{t+q-1}+0.5x_{t+q}}{d}$ |
| Exp. smoothing | $\hat{m}_t = \alpha x_t + (1-\alpha)\hat{m}_{t-1}$ |
| Differencing | $\nabla X_t = (1-B)X_t$ |
| Lag-$d$ differencing | $\nabla_d X_t = (1-B^d)X_t$ |
Sample Statistics
| Statistic | Formula |
|---|---|
| Sample mean | $\bar{x} = \frac{1}{n}\sum_{t=1}^n x_t$ |
| Sample ACVF | $\hat{\gamma}(h) = n^{-1}\sum_{t=1}^{n-h}(x_{t+h}-\bar{x})(x_t - \bar{x})$ |
| Sample ACF | $\hat{\rho}(h) = \hat{\gamma}(h)/\hat{\gamma}(0)$ |
| 95% bounds (iid) | $\pm 1.96/\sqrt{n}$ |
Stationarity Quick Reference
| Process | Weakly Stationary? | Strictly Stationary? |
|---|---|---|
| IID$(0,\sigma^2)$ | ✓ | ✓ |
| WN$(0,\sigma^2)$ | ✓ | Not necessarily |
| MA(q) | ✓ always | If Gaussian WN |
| AR(1), $\|\phi\|<1$ | ✓ | If Gaussian WN |
| AR(1), $\|\phi\|=1$ | ✗ | ✗ |
| AR(1), $\|\phi\|>1$ | ✓ (non-causal) | If Gaussian WN |
| Weakly stat. + Gaussian | ✓ | ✓ |