CNS Core: Medium: Resource Constrained Reinforcement Learning for Computing Systems

Cornell University, Ithaca NY

Investigators

Abstract

Reinforcement Learning has surfaced as a promising algorithmic paradigm for online decision making and control. Recent efforts demonstrate impressive results, but with relatively unlimited computational resources, data, and training. However, many real-world decision-making and control problems require real-time decisions but are severely constrained in terms of throughput, latency, memory, and power. Existing algorithms are employed in software on general purpose processors, and do not translate well to real-timeresource-constrained computation in hardware. This project will develop new algorithmic paradigms as well as hardware acceleration primitives that work in sync to enable real-time reinforcement learning at scale in embedded applications. From the algorithmic perspective, this work will study efficiency in computational resource tradeoffs, and design reinforcement learning algorithms that efficiently adapt to given hardware constraints and hardware primitives. From the hardware perspective, this work will design new hardware primitives that support the developed reinforcement learning algorithms for real-time applications in a storage- and energy-efficient manner. Our reinforcement learning algorithms, analyses, and experiments will shed new insights about how to make optimal trade-offs between performance and different system constraints such as memory, power, and latency. The research will be guided by systems applications including switch scheduling for network routers, resource management for data centers, network congestion control, memory management for computers, and adaptive sensing for wireless networks and the Internet of things. The outcomes of this project have the potential to transform how data centers, communication networks, and wireless systems are managed. Applications beyond computer and network systems include operational challenges in large-scale systems, such as inventory management and resource allocation, or low latency control problems that arise in brain-machine interfaces. The project also includes an extensive outreach plan that involves women and underrepresented minorities in computing research in reinforcement learning from an algorithmic and architecture-level perspective. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.