Fast-tracking stationary MOMDPs for adaptive management problems

Peron, Martin and Bartlett, Peter and Becker, Kai Helge and Chades, Iadine; (2017) Fast-tracking stationary MOMDPs for adaptive management problems. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. Proceedings of the AAAI Conference on Artificial Intelligence, 31 (1). Association for the Advancement of Artificial Intelligence (AAAI), USA, pp. 4531-4537. (https://doi.org/10.1609/aaai.v31i1.11173)

[thumbnail of Peron-etal-AAAI2017-Fast-tracking-stationary-MOMDPs-for-adaptive-management-problems]
Preview
 Text. Filename: Peron_etal_AAAI2017_Fast_tracking_stationary_MOMDPs_for_adaptive_management_problems.pdf
Accepted Author Manuscript
Download (1MB)| Preview

Abstract

Adaptive management is applied in conservation and natural resource management, and consists of making sequential decisions when the transition matrix is uncertain. Informally described as ’learning by doing’, this approach aims to trade off between decisions that help achieve the objective and decisions that will yield a better knowledge of the true transition matrix. When the true transition matrix is assumed to be an element of a finite set of possible matrices, solving a mixed observability Markov decision process (MOMDP) leads to an optimal trade-off but is very computationally demanding. Under the assumption (common in adaptive management) that the true transition matrix is stationary, we propose a polynomial-time algorithm to find a lower bound of the value function. In the corners of the domain of the value function (belief space), this lower bound is provably equal to the optimal value function. We also show that under further assumptions, it is a linear approximation of the optimal value function in a neighborhood around the corners. We evaluate the benefits of our approach by using it to initialize the solvers MO-SARSOP and Perseus on a novel computational sustainability problem and a recent adaptive management data challenge. Our approach leads to an improved initial value function and translates into significant computational gains for both solvers.

  • Item type:Book Section
    ID code:59671
    Dates:
    Date
    Event
    12 February 2017
    Published
    12 November 2016
    Accepted
    Notes:Copyright © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Péron, M., Becker, K., Bartlett, P., & Chadès, I. (2017). Fast-Tracking Stationary MOMDPs for Adaptive Management Problems. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.11173
    Subjects:Science > Mathematics > Electronic computers. Computer science
    Department:Strathclyde Business School > Management Science
    Depositing user: Pure Administrator
    Date deposited:01 Feb 2017 12:48
    Last modified:11 Nov 2024 15:08
    Related URLs:
    URI:https://strathprints.strath.ac.uk/id/eprint/59671
CORE (COnnecting REpositories)