A novel update mechanism for Q-Networks based on extreme learning machines

Wilson, Callum and Riccardi, Annalisa and Minisci, Edmondo; (2020) A novel update mechanism for Q-Networks based on extreme learning machines. In: 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, GBR. ISBN 9781728169262 (https://doi.org/10.1109/IJCNN48605.2020.9207098)

[thumbnail of Wilson-etal-WCCI2020-A-novel-update-mechanism-for-Q-Networks]
Preview
 Text. Filename: Wilson_etal_WCCI2020_A_novel_update_mechanism_for_Q_Networks.pdf
Accepted Author Manuscript
Download (319kB)| Preview

Abstract

Reinforcement learning is a popular machine learning paradigm which can find near optimal solutions to complex problems. Most often, these procedures involve function approximation using neural networks with gradient based updates to optimise weights for the problem being considered. While this common approach generally works well, there are other update mechanisms which are largely unexplored in reinforcement learning. One such mechanism is Extreme Learning Machines. These were initially proposed to drastically improve the training speed of neural networks and have since seen many applications. Here we attempt to apply extreme learning machines to a reinforcement learning problem in the same manner as gradient based updates. This new algorithm is called Extreme Q-Learning Machine (EQLM). We compare its performance to a typical Q-Network on the cart-pole task - a benchmark reinforcement learning problem - and show EQLM has similar long-term learning performance to a Q-Network.

ORCID iDs

Wilson, Callum ORCID logoORCID: https://orcid.org/0000-0003-3736-1355, Riccardi, Annalisa ORCID logoORCID: https://orcid.org/0000-0001-5305-9450 and Minisci, Edmondo ORCID logoORCID: https://orcid.org/0000-0001-9951-8528;
  • Item type:Book Section
    ID code:71948
    Dates:
    Date
    Event
    28 September 2020
    Published
    4 June 2020
    Published Online
    20 March 2020
    Accepted
    Notes:© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
    Subjects:Science > Mathematics > Electronic computers. Computer science
    Department:Faculty of Engineering > Mechanical and Aerospace Engineering
    Depositing user: Pure Administrator
    Date deposited:31 Mar 2020 13:31
    Last modified:11 Nov 2024 15:23
    Related URLs:
    URI:https://strathprints.strath.ac.uk/id/eprint/71948
CORE (COnnecting REpositories)