William J. Powers, M.D.-- Departments of
Neurology, Neurosurgery and Radiology
Washington University School of Medicine -- Presented July
11, 2006
The Perils of Subgroup Analysis
Negative
Trials with “Positive” Subgroups
Increased Type I error due to multiple
comparisons
•
The conventional criterion for statistical
significance isthat there is less than a 1 in 20 chance
(5% probability, p < .05) that an observeddifference between two experimental
groups could have occurred by chance alone even
if there is really no difference.
•
The more statistical comparisons that are made
between two groups, the greater is the
likelihood that one will yieldp < .05. With 20 comparisons, the
likelihood that one comparison yield p< .05
approaches 100%.
•
Statistically “positive” subgroups in overall
negative trials should be viewed as unproven
hypotheses unless the subgroup analyses were
specified a prioriand corrected for the effect of
multiple comparisons.