Enabling intelligent onboard guidance, navigation, and control using reinforcement learning on near-term flight hardware

Wilson, Callum and Riccardi, Annalisa (2022) Enabling intelligent onboard guidance, navigation, and control using reinforcement learning on near-term flight hardware. Acta Astronautica, 199. pp. 374-385. ISSN 0094-5765 (https://doi.org/10.1016/j.actaastro.2022.07.013)

[thumbnail of Wilson-Riccardi-AA-2022-Enabling-intelligent-onboard-guidance-navigation-and-control-using-reinforcement-learning]
Preview
 Text. Filename: Wilson_Riccardi_AA_2022_Enabling_intelligent_onboard_guidance_navigation_and_control_using_reinforcement_learning.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (1MB)| Preview

Abstract

Future space missions require technological advances to meet more stringent requirements. Next generation guidance, navigation, and control systems must safely operate autonomously in hazardous and uncertain environments. While these developments often focus on flight software, spacecraft hardware also creates computational limitations for onboard algorithms. Intelligent control methods combine theories from automatic control, artificial intelligence, and operations research to derive control systems capable of handling large uncertainties. While this can be beneficial for spacecraft control, such control systems often require substantial computational power. Recent improvements in single board computers have created physically lighter and less power-intensive processors that are suitable for spaceflight and purpose built for machine learning. In this study, we implement a reinforcement learning based controller on NVIDIA Jetson Nano hardware and apply this controller to a simulated Mars powered descent problem. The proposed approach uses optimal trajectories and guidance laws under nominal environment conditions to initialise a reinforcement learning agent. This agent learns a control policy to cope with environmental uncertainties and updates its control policy online using a novel update mechanism called Extreme Q-Learning Machine. We show that this control system performs well on flight suitable hardware, which demonstrates the potential for intelligent control onboard spacecraft.

ORCID iDs

Wilson, Callum ORCID logoORCID: https://orcid.org/0000-0003-3736-1355 and Riccardi, Annalisa ORCID logoORCID: https://orcid.org/0000-0001-5305-9450;
CORE (COnnecting REpositories)