Exam Pattern: Simulation-Based Power for Comparing Proportions

This note abstracts the normal midterm question where power is estimated from repeated simulated studies rather than from a closed-form formula.

Source Question

  • STA305 Midterm W2026, Question 5

Setup

Suppose a simulation generates $B$ hypothetical studies under a specific alternative, and the hypothesis test is run on each simulated dataset.

If the number of rejections is $R$, then the estimated power is

$$ \widehat{\text{power}} = \frac{R}{B}. $$

Workflow

Step 1: Identify the rejection rule

For a two-sided level-$\alpha$ test, reject when

$$ |Z| > z_{1-\alpha/2}. $$

Step 2: Count the simulated rejections

Go through the simulation output and count how many of the $B$ studies reject $H_0$.

Step 3: Estimate power

Compute

$$ \widehat{\text{power}} = \frac{R}{B}. $$

In the source problem, this was

$$ \frac{20}{30} = 0.6667. $$

Step 4: Interpret in context

Interpret the estimate as the approximate probability that the planned study would reject the null under the stated alternative.

R Template

reject <- c(TRUE, FALSE, TRUE)  # replace with simulated decisions
mean(reject)

Exam Checklist

  • Identify whether the simulation is under the null or under the alternative
  • Count rejections, not just extreme statistics
  • Divide by the number of simulated studies
  • Give a contextual interpretation, not just a decimal