Continuous-observation partially observable semi-Markov decision processes for machine maintenance

Zhang, Mimi and Revie, Matthew (2017) Continuous-observation partially observable semi-Markov decision processes for machine maintenance. IEEE Transactions on Reliability, 66 (1). pp. 202-218. ISSN 0018-9529 (https://doi.org/10.1109/TR.2016.2626477)

[thumbnail of Zhang-Revie-TR-2016-continuous-observation-partially-observable-semi-Markov-decision-processes]
Preview
 Text. Filename: Zhang_Revie_TR_2016_continuous_observation_partially_observable_semi_Markov_decision_processes.pdf
Accepted Author Manuscript
Download (663kB)| Preview

Abstract

Partially observable semi-Markov decision processes (POSMDPs) provide a rich framework for planning under both state transition uncertainty and observation uncertainty. In this paper, we widen the literature on POSMDP by studying discrete-state, discrete-action yet continuous-observation POSMDPs. We prove that the resultant α-vector set is continuous and therefore propose a point-based value iteration algorithm. This paper also bridges the gap between POSMDP and machine maintenance by incorporating various types of maintenance actions, such as actions changing machine state, actions changing degradation rate, and the temporally extended action "do nothing''. Both finite and infinite planning horizons are reviewed, and the solution methodology for each type of planning horizon is given. We illustrate the maintenance decision process via a real industrial problem and demonstrate that the developed framework can be readily applied to solve relevant maintenance problems.

ORCID iDs

Zhang, Mimi ORCID logoORCID: https://orcid.org/0000-0002-3807-297X and Revie, Matthew ORCID logoORCID: https://orcid.org/0000-0002-0130-8109;
  • Item type:Article
    ID code:58511
    Dates:
    Date
    Event
    31 March 2017
    Published
    6 December 2016
    Published Online
    7 November 2016
    Accepted
    Notes:(c) 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.
    Subjects:Science > Mathematics > Probabilities. Mathematical statistics
    Department:Strathclyde Business School > Management Science
    Depositing user: Pure Administrator
    Date deposited:09 Nov 2016 12:01
    Last modified:11 Nov 2024 11:33
    URI:https://strathprints.strath.ac.uk/id/eprint/58511
CORE (COnnecting REpositories)