The pervasiveness of global changes requires that ecologists accurately quantify the consequences of global change on community structure and ecosystem function. To obtain such estimates, ecologists simulate one or multiple aspects of global change (e.g., warming, biodiversity loss, or drought) using manipulative field experiments. Yet field experiments often yield equivocal results. Biodiversity has been reported as having var- iable effects on ecosystem functioning. Likewise, warming can stimulate, reduce, or have no effect on aboveground net primary productivity (ANPP), whereas drought effect sizes vary from small to large. Undoubtedly, inconsistent results among studies arise partly from methodological differences, differing biotic/abiotic contexts, and temporal variation. However, we contend that contradictory results may also arise even in perfectly replicated studies as a consequence of low statistical power that by necessity plagues many global change experiments. Importantly, we focus on the recent realization that low-powered experiments examining processes that have small true effect sizes must observe an overestimated effect size in order to achieve statistical significance (Type M Error). Here, we describe this overlooked aspect of power, examine its prevalence in global change studies, and provide ways to remedy this important issue in ecological studies.
Results/Conclusions
Ecological field experiments consistently demonstrated low statistical power. Power was lowest for highly variable data, a common aspect of field experiments that is difficult to mitigate. Furthermore, increasing sample size can only partially alleviate the issue low power. Given that increased sample sizes can only partially offset the issues of low power, we advocate three approaches to rectifying the issue of Type M errors: (1) calculate Type M error and report it as you would a p-value, (2) emphasize effect sizes and confidence intervals over arbitrary cutoffs of any kind (e.g. p < 0.05, dAIC > 2, etc.), and (3) use Bayesian statistics with sensible priors to constrain effect sizes in the presence of small sample sizes. By adhering to these suggestions, ecologists can avoid the pitfalls of overstating results arising from underpowered experiments, which are common in ecology. Ecologists should debate the true effect size, which may differ among ecosystems, experimental methods, or study organisms. However, underpowered studies addressing issues with small true effect sizes must overestimate the size of the effect in order to find statistical significance. Ecologists need to be aware of this issue in order to avoid the pitfalls of irreproducible research.