Mon, Aug 02, 2021:On Demand
Background/Question/Methods
Uncertainty has always been a central issue confronting ecological management and decision making. While some uncertainty is irreducible due to the inherent variation of environment and demographics, much uncertainty arises from our limited knowledge of those processes, reflected in the choice of possible functions and parameters we use to model them. Consequently, a rich suite of techniques for addressing and reducing model uncertainty have been developed, often emphasizing the importance of an iterative or adaptive approach in which the choices of models and parameters are refined in light of new observations and management outcomes. Unfortunately, even the best of such adaptive approaches can push in the direction of worse decisions.
I consider two such techniques: (1) an iterative forecasting approach, a powerful technique which avoids the selection of models that over-fit the data that is becoming increasingly possible in real world applications thanks to the rapid growth of available ecological data, and (2) an adaptive management approach, which explicitly integrates over uncertainty to maximize expected real-world value rather than more abstract notion of forecast skill. I apply these to a very simple and familiar conservation decision problem: selecting a sustainable harvest policy, given uncertainty over two possible models.
Results/Conclusions In my example problem, I find that both iterative forecasting and adaptive management cycles lead to a rapid deterioration in management outcomes they are intending to optimize. Management under either approach leads to overfishing, depressed biomass, and depressed economic yields relative to a strategy that does not attempt to learn or reduce model uncertainty, or even one which merely assumes one of the two models is correct. The intuition for this result can be demonstrated on a napkin, which makes it clear that this is not a problem of model fitting and that no model choice technique would choose the model which gives the best outcomes. This result highlights a pitfall in how any available or emerging approaches which no amount of data can solve, but some napkin-sized theory can help.
Results/Conclusions In my example problem, I find that both iterative forecasting and adaptive management cycles lead to a rapid deterioration in management outcomes they are intending to optimize. Management under either approach leads to overfishing, depressed biomass, and depressed economic yields relative to a strategy that does not attempt to learn or reduce model uncertainty, or even one which merely assumes one of the two models is correct. The intuition for this result can be demonstrated on a napkin, which makes it clear that this is not a problem of model fitting and that no model choice technique would choose the model which gives the best outcomes. This result highlights a pitfall in how any available or emerging approaches which no amount of data can solve, but some napkin-sized theory can help.