Confounds of insensitivity and blind luck: Statistical conclusion validity in stroke rehabilitation clinical trials

We present theoretical analyses of some infrequently considered consequences of low power and quantitatively review power in 36 stroke rehabilitation clinical trials with multiple outcome variables. We demonstrate that low power affects the proportion of false alarms (type 1 errors) in the literature and probability of replication, not only the ability to detect true intervention effects; and that these consequences of low power are modulated by the probability that effective interventions have been preselected for study via theory and prior evidence. Rehabilitation trials showed low power to detect small, medium, or large treatment effects at α = 0.05 (0.09, 0.33, and 0.69, respectively). Effect sizes were generally small with adverse consequences on power. As expected, sample size was an important determinant of power. Power was lower for motor/reflex measures due to differences in sample sizes rather than effect sizes. These results and the conceptual analyses predict and address conflicting conclusions from stroke rehabilitation clinical trials.