Evaluating complex theoretical propositions against data represents a major challenge for the quantitative sciences. For example, recent analyses have claimed to reveal the main mechanisms connecting ecological productivity and species diversity in grasslands. Such claims, of course, call for further evaluation, which raises the question, “What data and analyses are needed for scientists to further evaluate such conclusions?” It seems that one thing needed is a general set of guidelines for determining the data requirements for disentangling multiple simultaneous mechanisms. In this talk I will present results from considerations of the general requirements and assumptions for inference. I also use simulation studies to consider potential impacts of model-data discrepancies on results, as well as steps to overcome encountered difficulties.
Results/Conclusions
First, one key to success in testing complex theories is clear communication of the up-front assumptions and their linkage to the literature. Many debates in ecology have resulted from misunderstandings and this is not helpful. A formal approach to meta-modeling provides a useful and perhaps even essential contribution to theory testing, though it is nearly always ignored. Second, I find that it is critical to allow for competing expectations to avoid misattribution. Omitting competing expectations can either obscure or exaggerate focal effects, depending on whether competing expectations are positively or negatively correlated. Finally, there are ultimate limitations to obtaining unbiased causal estimates in the strict sense. Awareness of these can lead to the collection of more appropriate data as well as an avoidance of inappropriate interpretations.