Mon, Aug 02, 2021:On Demand
Background/Question/Methods
The detection of trends in population sizes is used to inform management decisions and risk assessments. Since population estimates are collected over time it is desirable to test for trends as data is collected to detect trends quickly. However, repeated testing of accumulating data using fixed sample size tests (the most common tests in ecology) inflates false detection of trends and biased parameter estimates. Proper usage of tests that assume fixed sample sizes require that all data be collected before testing to maintain nominal type I error and power. This disqualifies commonly used frequentist tests for repeatedly testing for population trends. Sequential testing, a procedure designed for repeatedly testing accumulating data while maintaining type I and II error rates are a frequentist solution to this problem, but have seen little development or application to population trend detection despite, on average, requiring fewer samples than fixed-sample tests.
In this study, we assume that managers wish to make a binary decision based on whether a population trend is greater or lesser than some predetermined trend size. We describe how such a manager can use sequential tests to detect stochastic linear or exponential population trends. The performance of sequential tests in terms of type I and II error rates, average sample number, and bias of trend estimates were evaluated using simulated data and compared to corresponding fixed-sample approaches.
Results/Conclusions We first show that repeated application of fixed-sample size tests is misleading due to inflated false detection rates and exaggerated trend estimates. These problems are exacerbated in situations where the true trend is non-zero, but smaller than the minimum relevant trend. Thus, to properly test for trends on accumulating data the frequentist manager must use sequential tests. If fixed-sample tests are used, the entire sample must be collected before analysis to maintain the pre-specified error probabilities. On average, sequential tests detect trends in as few as half the samples needed for fixed-sample tests. However, trend estimation following a sequential test is biased in the direction of whichever hypothesis is accepted, whereas trend estimation following properly conducted fixed-sample tests are not. If unbiased estimation of trends is desired, methods exist to reduce the bias of estimates following a sequential test but are complicated to implement. Sequential tests are a simple and superior approach for managers that want to quickly detect trends and do not require unbiased estimation of trends.
Results/Conclusions We first show that repeated application of fixed-sample size tests is misleading due to inflated false detection rates and exaggerated trend estimates. These problems are exacerbated in situations where the true trend is non-zero, but smaller than the minimum relevant trend. Thus, to properly test for trends on accumulating data the frequentist manager must use sequential tests. If fixed-sample tests are used, the entire sample must be collected before analysis to maintain the pre-specified error probabilities. On average, sequential tests detect trends in as few as half the samples needed for fixed-sample tests. However, trend estimation following a sequential test is biased in the direction of whichever hypothesis is accepted, whereas trend estimation following properly conducted fixed-sample tests are not. If unbiased estimation of trends is desired, methods exist to reduce the bias of estimates following a sequential test but are complicated to implement. Sequential tests are a simple and superior approach for managers that want to quickly detect trends and do not require unbiased estimation of trends.