Markov Decision Processes (MDPs) provide a convenient model for representing sequential decision-making optimization problems when the decision maker has complete information about the current state of the system and dynamics are non-deterministic. MDPs have been applied to help recover populations of threatened species under limited resources, to control invasive species, to perform adaptive management of natural resources, and to test behavioral ecology theories. These domains are human-operated systems, where MDP policies provide recommendations. Solutions computed for MDPs with thousands of states are difficult to understand. In human-operated systems, it is crucial that solutions provided by artificial intelligence algorithms can be interpreted and explained in order to increase uptake of MDP solutions. Explainable artificial intelligence, also known as the interpretability problem, aims to generate decisions in which one of the criteria is how easily a human can understand these decisions. We propose to increase the interpretability of MDPs by providing explainable artificial intelligence algorithms that can be used to solve conservation decision problems. We define the problem of solving K-MDPs, i.e., given an original MDP and a number of states (K), generate a reduced state space MDP that minimizes the difference between the original and reduced optimal solutions. Abstracting states aims to reduce the size of large state spaces by aggregating states which are equivalent given a metric.
Results/Conclusions
We evaluate our method to conserve two endangered species based on a problem from the literature. The sea otter Enhydra lutris kenyoni, and its preferred prey, northern abalone Haliotis kamtschatkana, are both listed as endangered in British Columbia, Canada. The conservation objective is to maximize the abundance of both species over time. The original MDP has 819 states representing the population of sea otters and the density of northern abalone. Managers can choose between 4 management actions: Introducing sea otters, antipoaching, control sea otters and investing half of the management resources in antipoaching and sea otter control respectively. We investigated the interpretability of both the model and the optimal K-MDP solution generated by our algorithms. We reduce the problem from 819 states to 10 states, with a loss of performance of less than 2.8%.
Computer-aided decision-making is not just about providing users with operations that they should blindly execute. It is critical for users to understand the plan at hand, including why it makes sense. Our algorithms provide compact models and solutions much needed to maximise uptake of AI decision tools.