Mon, Aug 15, 2022: 1:30 PM-1:45 PM
520E
Background/Question/MethodsWith an ever-growing number of scientific articles published each year, evidence synthesis has become essential to understand and summarize scientific progress. Evidence synthesis encompasses systematic literature reviews, which allow qualitatively synthesizing scientific evidence in a way that maximizes transparency, comprehensiveness and objectivity. Oftentimes, researchers combine a systematic review with a meta-analysis, which is a powerful statistical tool to quantitatively summarize the evidence for a hypothesis, estimate its generality and explore context-dependencies, as well as generating new hypotheses. Although multiple efforts have been made recently to evaluate the transparency and reproducibility of the primary studies that serve as the raw material for evidence syntheses, surprisingly little is known about the transparency and reproducibility of evidence syntheses themselves. Here, evaluate the prevalence of transparency- and reproducibility-related reporting practices in meta-analyses in ecology and evolution. We assessed the reporting standards of 178 meta-analyses published between 2010 and 2020 in 56 journals in ecology and evolution.
Results/ConclusionsOur results show that only 14.6% of meta-analyses followed any reporting guidelines and only 2.2% were pre-registered prior to analysis. Although data-sharing was comparatively high – 75.3% of meta-analyses shared either processed and/or raw data, computational reproducibility is presumably low as only 14.0% adhered to code-sharing. In addition, 7.9% and 35.4% of the meta-analyses failed to report the statistical software and packages used, respectively, and 54.5% also failed to report software and package versions, further reducing the likelihood of successful future computational reproduction attempts. Our meta-research suggests that there is ample room for improvement in the transparency and reproducibility of meta-analyses in ecology and evolution, problem that we consider essential to tackle given the influence that meta-analyses have on decision-making and future research.
Results/ConclusionsOur results show that only 14.6% of meta-analyses followed any reporting guidelines and only 2.2% were pre-registered prior to analysis. Although data-sharing was comparatively high – 75.3% of meta-analyses shared either processed and/or raw data, computational reproducibility is presumably low as only 14.0% adhered to code-sharing. In addition, 7.9% and 35.4% of the meta-analyses failed to report the statistical software and packages used, respectively, and 54.5% also failed to report software and package versions, further reducing the likelihood of successful future computational reproduction attempts. Our meta-research suggests that there is ample room for improvement in the transparency and reproducibility of meta-analyses in ecology and evolution, problem that we consider essential to tackle given the influence that meta-analyses have on decision-making and future research.