Simulation-Based Power for ANOVA

Simulation-based power estimates the probability that ANOVA rejects $H_0$ by repeatedly generating data under a chosen alternative and recording how often the test is significant.

This is especially useful when the classical assumptions may fail, for example under unequal variances or non-normal data.

R

B <- 5000
reject <- replicate(B, {
  y <- c(rnorm(n, mu1, sigma),
         rnorm(n, mu2, sigma),
         rnorm(n, mu3, sigma))
  treatment <- gl(3, n)
  pval <- summary(aov(y ~ treatment))*1**"Pr(>F)"*[1]
  pval < 0.05
})

mean(reject)