STA447: Stochastic Process MOC

This map indexes every atomic note for STA447 along Markov chains, martingales, and Brownian motion. It also includes phase-based reading guides that are not yet split into atoms.

Phase Reading Orders

Phase-by-phase study guides with syllabus structure and reading order.

• Phase 5.2-5.5 Atomic Reading Order — Stopping times and the optional stopping theorem
• Phase 6 Atomic Reading Order — Brownian motion foundations
• Phase 7 Atomic Reading Order — Itô integral: definition and properties
• Phase 8 Atomic Reading Order — Itô’s formula and applications

Phase Teaching Documents

Full lecture-style notes with detailed derivations and examples.

• Phase 5.2-5.5 — Stopping times, OST, martingale convergence
• Phase 6 — Brownian motion foundations
• Phase 7 — Itô integral: definition and properties
• Phase 8 — Itô’s formula and applications

Markov Chain Basics

Markov chains: definitions, transition matrices, and Chapman–Kolmogorov equations.

• State Space
• Discrete-Time Markov Chain
• Transition Matrix
• 1-Step and n-Step Transition Probability
• Chapman–Kolmogorov Equations
• Markov Property

Accessibility and Communication

Accessibility between states and communication classes (Markov chain sense).

• Accessibility
• Communication
• Communication Class
• Class Property

Periodicity and Aperiodicity

Periodicity.

• Periodicity
• Aperiodicity

Recurrence and Transience

Recurrence and transience.

• Recurrence and Transience
• First Return Time
• Hitting Time
• Return Probability
• Reaching Probability
• Positive vs Null Recurrence
• Asymmetry of Reachability in Transience
• Reducibility and Irreducibility

Stationary Distributions and Convergence

Stationary distributions and convergence theorems.

• Stationary Measure and Distribution
• Fundamental Theorem of Markov Chain
• Convergence Theorem
• Convergence Theorem and Vanishing Probabilities
• Detailed Balance and Reversibility

Special States

Special types of states.

• Absorbing State

Martingales

Martingale foundations.

• Martingale
• Exponential Martingale
• Product Martingale
• Partial Sum Martingale
• Simple Random Walk Martingale
• Brownian Motion as a Martingale

Stopping Times and Optional Stopping

Stopping times and applications.

• Stopping Time
• Geometric Stopping Time
• Operations on Stopping Times
• Bounded Before Stopping Criterion
• Non-Stopping Time Examples

Optional Stopping Theorem

Optional stopping theorems.

• Optional Stopping Theorem
• Bounded Optional Stopping Theorem
• Continuous-Time Optional Stopping Theorem
• Optional Stopping Application Template

Martingale Convergence and Related Theorems

Martingale convergence and related results.

• Martingale Convergence Theorem
• Doob’s Maximal Inequality
• Doob’s Martingale
• Uniform Integrability
• Uniform Integrability Criteria
• UI, L1 Convergence, and OST

Wald’s Identity

Wald’s identity.

• Wald’s Identity

Conditional Expectation

Conditional expectation.

• Conditional Expectation Calculation Rules

Coupling

Coupling.

• Coupling
• Independent vs General Coupling

Random Walks and Related Processes

Random walks and related processes.

• Coupon Collector Problem
• Exponential Tail Bound for Hitting Times

MCMC and Random Walks

How MCMC relates to random walks (as needed).

Brownian Motion

Brownian motion and its properties.

• Brownian Motion
• Brownian Motion as the Scaling Limit of SRW
• Brownian Motion as a Markov Process
• Strong Markov Property of Brownian Motion
• Brownian Motion as a Martingale
• Brownian Motion as a Gaussian Process
• Brownian Motion Is Nowhere Differentiable
• Brownian Motion Square Martingale

Brownian Motion Applications

Applications of Brownian motion.

• Brownian Motion Expected Exit Time
• Brownian Motion Exit Probability
• Maximum of Brownian Motion
• Reflection Principle for Brownian Motion
• Quadratic Variation of Brownian Motion

Filtration

Filtrations and stopping times in continuous time.

• Filtration
• Continuous-Time Stopping Time

Itô Integral

The Itô integral: definition, properties, and computation.

• Why Ordinary Stochastic Integration Fails for Brownian Motion
• Construction of the Ito Integral
• Ito Integral for Simple Processes
• Left Endpoint Rule in the Ito Integral
• Ito Integral Is a Martingale
• Ito Integral as a Continuous Martingale Transform
• Ito Isometry
• Linearity of the Ito Integral
• Computing the Ito Integral of Brownian Motion
• Ito vs Stratonovich Integral

Quadratic Variation

Quadratic variation.

• Quadratic Variation Heuristic for Brownian Motion
• Quadratic Variation of an Ito Process

Itô Formula and Applications

Itô’s formula and applications.

• Classical Chain Rule Fails for Brownian Motion
• Proof Idea of Ito Formula
• Ito Formula for Brownian Motion
• Ito Formula for Ito Processes
• Ito Product Rule
• Time-Inhomogeneous Ito Formula
• Ito Differential Rules

Martingales from Itô Formula

Martingales built using Itô’s formula.

• Constructing Martingales with Ito Formula
• Exponential Brownian Martingale

Branching Processes

Branching processes.

• Branching Process Extinction

Product Measure

Product measures.

• Product Measure
• Product Measure of Joint Chains

Coupling and Other Advanced Topics

Advanced topics (as needed).

Reference Materials

• Week 1 Molecular Structure

Last updated: 2026-04-01