Molecular Note: ARMA from Definition to Prediction

Linked Atomic Concepts: ARMA Process, ARMA(1,1) Causal Representation, AR(1) — Causal Case, Prediction for Causal AR(p), Prediction Intervals

Scene-Setting

You’re given $X_t = 0.5 X_{t-1} + Z_t + 0.3 Z_{t-1}$ with $\sigma^2 = 1$. You need to understand its structure, show it’s causal, find its linear process representation, and use it for prediction.

Concept Chain

The model → ARMA(p,q) Model

$\phi(B)X_t = \theta(B)Z_t$ with $\phi(B) = 1 - 0.5B$, $\theta(B) = 1 + 0.3B$. ARMA(1,1).

Causality check

Root of $\phi(z) = 1 - 0.5z = 0$ is $z = 2$. Since $|2| > 1$ → causal.

Causal representation → ARMA(1,1) Causal Representation

$$X_t = Z_t + (\phi + \theta)\sum_{j=1}^{\infty}\phi^{j-1}Z_{t-j} = Z_t + 0.8\sum_{j=1}^{\infty}0.5^{j-1}Z_{t-j}$$

Coefficients: $\psi_0 = 1$, $\psi_j = 0.8 \cdot 0.5^{j-1}$ for $j \geq 1$.

So $\psi_1 = 0.8$, $\psi_2 = 0.4$, $\psi_3 = 0.2$, …

ACVF from causal representation

$\gamma(h) = \sigma^2 \sum_{j=0}^{\infty}\psi_j\psi_{j+h}$.

Prediction

For one-step ahead using the ARMA recursion: $P_n X_{n+1} = \phi_1 X_n + \theta_1 (X_n - P_{n-1}X_n)$.

This involves the innovations $X_n - P_{n-1}X_n$ — a more advanced computation than pure AR. For exam purposes on current material, the key result is: if ACVF is known, set up $\Gamma_n \mathbf{a} = \boldsymbol{\gamma}(h)$ and solve.

Worked Pipeline

ARMA(1,1): $\phi = 0.5$, $\theta = 0.3$, $\sigma^2 = 1$.

  1. Causal: root of $\phi(z)$ at $z = 2$ (outside unit circle). ✓
  2. $\psi_j$: $\psi_0 = 1, \psi_1 = 0.8, \psi_2 = 0.4, \psi_3 = 0.2, \ldots$
  3. $\gamma(0) = \sum \psi_j^2 = 1 + 0.64 + 0.16 + 0.04 + \cdots = 1 + 0.64/(1-0.25) = 1 + 0.853 = 1.853$
  4. ACF tails off (not cutoff) — consistent with ARMA, not pure MA.

Exam Patterns

Pattern: “Show ARMA(1,1) can be written as causal linear process” (MT1 Review Problem 3)

  • Template: $X_t = \phi X_{t-1} + Z_t + \theta Z_{t-1}$, $|\phi| < 1$. Show $X_t = Z_t + (\phi+\theta)\sum_{j=1}^{\infty}\phi^{j-1}Z_{t-j}$.
  • Key move: Apply $(1-\phi B)^{-1}$ to both sides: $X_t = \frac{1+\theta B}{1-\phi B}Z_t$. Expand $(1-\phi B)^{-1} = \sum \phi^j B^j$, distribute, collect terms.