Leveraging optimal control demonstrations in reinforcement learning for powered descent

Wilson, Callum and Riccardi, Annalisa (2021) Leveraging optimal control demonstrations in reinforcement learning for powered descent. In: 8th International Conference on Astrodynamics Tools and Techniques, 2021-06-22 - 2021-06-25, Virtual.

[thumbnail of Wilson-Riccardi-ICATT-2021-Leveraging-optimal-control-demonstrations-in-reinforcement-learning-for-powered-descent]
Preview
 Text. Filename: Wilson_Riccardi_ICATT_2021_Leveraging_optimal_control_demonstrations_in_reinforcement_learning_for_powered_descent.pdf
Accepted Author Manuscript
Download (416kB)| Preview

Abstract

This work presents an approach to deriving a controller for spacecraft powered descent using reinforcement learning. To assist in the learning process, our approach uses optimal control demonstrations which provide open-loop control for optimal trajectories. Combining these approaches to use the optimal trajectories as demonstrations helps to overcome issues with convergence on desirable policies in the reinforcement learning problem. We demonstrate the applicability of this approach on a simulated 3-DOF Mars lander. The results show that the learned controller is capable of achieving a pinpoint soft landing from a range of initial conditions. Compared to the open-loop optimal trajectories alone, this controller generalises to more initial conditions and can cope with environmental uncertainties.

ORCID iDs

Wilson, Callum ORCID logoORCID: https://orcid.org/0000-0003-3736-1355 and Riccardi, Annalisa ORCID logoORCID: https://orcid.org/0000-0001-5305-9450;
CORE (COnnecting REpositories)