Cooperative Coevolution of Neural Networks in Sequential Decision Tasks

University Of Texas At Austin, Austin TX

Investigators

Abstract

In sequential decision tasks such as resource optimization, robot control, and game playing, several decisions must be made before the outcome can be evaluated. Such reinforcement feedback depends on the entire sequence of decisions, and it is difficult to determine which of the decisions were responsible for the outcome. This project aims at developing better techniques for learning in domains with such sparse feedback, based on evolving neural networks with genetic algorithms. The goal is both to be able to solve existing problems faster, and to be able to solve problems that have not been feasible as sequential decision tasks before. Our previous work showed that neuroevolution is most powerful when individual neurons are evolved to cooperate and form good networks. In this project, such cooperative coevolution methods are studied in depth. The research aims at answering three main questions: Where does the power of cooperative coevolution come from and what are the best ways of making use of it? How do the evolutionary reinforcement learning methods differ from the traditional value function methods in learning sequential decision tasks? Does evolutionary reinforcement learning have the accuracy and flexibility required in real-world applications? If successful, the project will result in cooperative coevolution algorithms that will solve existing sequential decision tasks faster, and will allow solving more difficult tasks than before. We will know how to decide between evolutionary and value function methods for a given reinforcement learning task, and also how to use each method most effectively. Finally, the project will demonstrate how learning in general, and cooperative coevolution of neural networks in particular, can be used to save resources and achieve complex behavior in challenging real-world tasks.