Learning Methods for Decentralized Control in Multi-Agent Systems

University Of Southern California, Los Angeles CA

Investigators

Abstract

Multi-agent systems (MAS) are expected to become increasingly prevalent in military and civilian domains. Decentralized control and decision-making by agents is a fundamental driver of the diverse applications of multi-agent systems. Agents are expected to act and make decisions without relying on a centralized command structure. Communication and coordination among agents may have to be carried out over sparse, intermittent, unreliable, low data rate and/or noisy communication networks that preclude the possibility of centralized information and decision-making. A key design challenge is to find efficient ways of computing decentralized control and decision strategies for a team of agents. The problem is further compounded by various kinds of uncertainties - uncertainties about the environment, noisy observations, unreliable communication as well as uncertainties in the system model. In this project, we aim to develop learning-based methods for decentralized control in multi-agent systems. Intellectual merit: The research develops the following: (i) learning-based practical methods for computing near-optimal decentralized control policies for multi-agent systems with known system model. (ii) online decentralized learning algorithms for control of multi-agent systems with unknown system model. We aim to develop decentralized algorithms that asymptotically find the optimal decentralized policy for such systems and learn in the most efficient way possible. The proposed research will lay the foundations for Learning-based Decentralized Optimal Control, which is expected to become increasingly important for emerging multi-agent system applications. Broader Impact: The research will significantly impact the science of multi-agent systems, autonomous robotic systems, and reinforcement learning. It will introduce a systematic and practical learning-based approach to design of multi-agent systems that has long been lacking in the literature. The educational impact of the proposed research will include: (i) providing graduate students with a multi-disciplinary training in stochastic control, online learning and optimization, (ii) involvement of undergraduate students during summer to perform computational and lab experiments (iii) efforts to recruit female and under-represented minority students in our projects; (iv) The research results will be incorporated in classes on reinforcement learning, stochastic systems, and decentralized control taught by the principal investigators. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.