Scalable Evolutionary Hierarchical Reinforcement Learning

Abramowitz, Sasha and Nitschke, Geoff (2022) Scalable Evolutionary Hierarchical Reinforcement Learning, Proceedings of Genetic and Evolutionary Computation Conference (GECCO 2022), 9-13 July 2022, Boston, USA.

Text 2022-Scalable Evolutionary Hierarchical Reinforcement Learning.pdf - Published Version Download (573kB)

Abstract

This paper investigates a novel method combining Scalable Evolution Strategies (S-ES) and Hierarchical Reinforcement Learning (HRL). S-ES, named for its excellent scalability, was popularised with demonstrated performance comparable to state-of-the-art policy gradient methods. However, S-ES has not been tested in conjunction with HRL methods, which empower temporal abstraction thus allowing agents to tackle more challenging problems. We introduce a novel method merging S-ES and HRL, which creates a highly scalable and efficient (compute time) algorithm. We demonstrate that the proposed method benefits from S-ES’s scalability and indifference to delayed rewards. This results in our main contribution: significantly higher learning speed and competitive performance compared to gradient-based HRL methods, across a range of tasks.

Item Type:	Conference paper
Subjects:	Computing methodologies > Artificial intelligence Computing methodologies > Machine learning Computing methodologies > Modeling and simulation
Date Deposited:	23 Sep 2022 07:54
Last Modified:	23 Sep 2022 07:54
URI:	https://pubs.cs.uct.ac.za/id/eprint/1543

Actions (login required)

View Item