Exam Pattern: Paired Randomization Test

This note abstracts the makeup midterm question where treatment labels are randomized within matched pairs and inference is based on paired differences.

Source Question

  • STA305 Makeup W2026, Question 2

Setup

For each pair or subject, define the within-pair difference

$$ D_i = Y_{i,A} - Y_{i,B}. $$

The hypotheses can be written as

$$ H_0: \mu_D = 0 \quad \text{versus} \quad H_1: \mu_D \ne 0. $$

The observed statistic is often

$$ T_{\text{obs}} = \bar{D}. $$

Core Idea

Because the assignment is randomized within each pair, under $H_0$ the two labels inside a pair are exchangeable. That means each $D_i$ can have its sign flipped without changing its null distribution.

With $m$ pairs, there are

$$ 2^m $$

possible sign-flip assignments.

Workflow

Step 1: Recognize the design

This is a paired randomized design, not an independent two-group design.

Step 2: State the exchangeability assumption

Under $H_0$, swapping the two treatment labels inside a pair does not change the outcome structure, so the sign of each $D_i$ is exchangeable.

Step 3: Build the randomization distribution

Compute the mean difference for every sign pattern

$$ (\pm D_1, \pm D_2, \dots, \pm D_m). $$

Step 4: Compute the randomization $p$-value

For a two-sided test,

$$ p\text{-value}

\Pr\left(|\bar{D}^{,*}| \ge |\bar{D}_{\text{obs}}| \mid H_0\right). $$

Step 5: Compare with the paired $t$-test

The correct parametric comparison is the paired $t$-test, equivalently a one-sample $t$-test on the differences.

R Template

A <- c(-4.1, -3.2, -2.8, -3.6)
B <- c(-2.7, -3.5, -1.9, -2.4)
d <- A - B

obs <- mean(d)

signs <- expand.grid(rep(list(c(-1, 1)), length(d)))
stats <- rowMeans(as.matrix(signs) * matrix(d, nrow = nrow(signs), ncol = length(d), byrow = TRUE))

mean(abs(stats) >= abs(obs))

t.test(A, B, paired = TRUE)

Validity Conditions for the Paired $t$-Test

The paired $t$-test assumes the differences

$$ D_1, \dots, D_m $$

are independent and normally distributed with mean $\mu_D$.

Exam Checklist

  • Name the design as paired or matched
  • Define $D_i$ clearly
  • Use $2^m$ sign flips, not $\binom{n}{n_1}$ assignments
  • Use a paired $t$-test, not an independent-samples $t$-test
  • State the normality assumption on the differences