Forecasting
Forecasting uses observed history $X_1, \dots, X_n$ to predict future values $X_{n+h}$.
1. Best Linear Predictor
The best linear predictor of $X_{n+1}$ given $X_1, \dots, X_n$ is
$$P_n X_{n+1} = a_1 X_n + a_2 X_{n-1} + \cdots + a_n X_1$$where the coefficients $a_1, \dots, a_n$ minimize the mean squared error $\mathbb{E}[(X_{n+1} - P_n X_{n+1})^2]$.
The coefficients are determined by the prediction equations:
$$\mathbb{E}[(X_{n+1} - P_n X_{n+1}) X_j] = 0 \quad \text{for } j = 1, \dots, n$$2. Mean Squared Error (MSE)
The prediction error is $e_{n+1} = X_{n+1} - P_n X_{n+1}$.
The MSE of the best linear predictor is
$$\text{MSE} = \mathbb{E}[e_{n+1}^2] = \gamma_X(0) - \sum_{i=1}^n a_i \gamma_X(n+1-i)$$3. AR(1) Forecasting
For AR(1) with $X_t = \phi X_{t-1} + Z_t$:
$$P_n X_{n+1} = \phi X_n$$$$\text{MSE} = \sigma^2$$4. Prediction Intervals
Assuming Gaussian errors, a 95% prediction interval for $X_{n+1}$ is
$$P_n X_{n+1} \pm 1.96 \sqrt{\text{MSE}}$$5. Fitted vs True Predictive Performance
A common trap is confusing fitted values (in-sample) with true forecasting ability (out-of-sample). Overfitting reduces in-sample error but increases out-of-sample error.