Curious Hierarchical Actor-Critic Reinforcement Learning

ion and curiosity-driven exploration are two common paradigms in current reinforcement learning approaches to break down difficult problems into a sequence of simpler ones and to overcome reward sparsity. However, there is a lack of approaches that combine these paradigms, and it is currently unknown whether curiosity also helps to perform the hierarchical abstraction. As a novelty and scientific contribution, we tackle this issue and develop a method that combines hierarchical reinforcement learning with curiosity. Herein, we extend a contemporary hierarchical actor-critic approach with a forward model to develop a hierarchical notion of curiosity. We demonstrate in several continuous-space environments that curiosity approximately doubles the learning performance and success rates for most of the investigated benchmarking problems.

@InProceedings\{RENW20,
  author       = "R{\"o}der, Frank and Eppe, Manfred and Nguyen, D.H. Phuong and Wermter, Stefan",
  title        = "Curious Hierarchical Actor-Critic Reinforcement Learning",
  booktitle    = "Artificial Neural Networks and Machine Learning – ICANN 2020",
  series       = "Lecture Notes in Computer Science",
  pages        = "408--419",
  month        = "May",
  year         = "2020",
  editor       = "Igor Farkaš, Paolo Masulli, Stefan Wermter",
  publisher    = "Springer",
  url          = "https://www2.informatik.uni-hamburg.de/wtm/publications/2020/RENW20/curious-hierarchical-ac-rl.pdf"
}