The "decline effect" refers to the apparent decrease over time in effect size of many phenomena in various fields of science. The first published studies of the phenomenon report large effect sizes on average. But subsequent studies report smaller effect sizes than previous studies, with the grand mean effect size eventually converging to zero. The decline effect might arise if early studies tend to be conducted using study systems or methods likely to generate a large effect size. Alternatively, the decline effect might arise because of various sorts of publication biases and questionable research practices. In the presence of a decline effect, the time-averaged mean effect size might not be the best estimate of the true mean effect size. Identifying decline effects and the reasons for them can provide important insights into the conduct of science. Further, even in the absence of a decline effect, mean effect sizes might be slow to converge towards any stable value as more studies are published, if effect sizes are are highly variable. I compiled effect size data from over 80 published meta-analyses in ecology. For each meta-analysis, I plotted plotted cumulative mean and cumulative variance of effect sizes over time, to check for a decline effect.
Results/Conclusions
Preliminary results rarely suggest a decline effect; few plots of cumulative mean effect size converge towards a mean of zero. However, it is common for the first published effect sizes to be unusually large, compared to subsequent effect sizes. Preliminary results also reveal that cumulative mean effect sizes typically are slow to converge towards a stable value, if they converge at all. For that reason, variance among published effect sizes often increases over time as more effect size estimates are published, and the cumulative variance often fails to converge to a stable value. The trend of increasing variance over time often is noisy, but is too common across meta-analyses to be due to chance alone (p<0.05). These trends are not due to decreasing precision of effect size estimates over time. In some cases, these trends may reflect increasing among-study heterogeneity: over time, ecologists tend to increase the range of systems, locations, and methods used to study any given phenomenon. These results have implications for the interpretation of ecological meta-analyses, and provide novel insight into the conduct of ecological research.