CAREER: A deeper look at state-dependent noise in systems

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

Traditional models in control typically assume that any environmental noise or disturbance is independent of the state of the system itself. While this assumption greatly simplifies the models, it is not always true. Under a standard linear control perspective, some systems with state-dependent observation noise, (which often manifests as multiplicative noise), would be considered uncontrollable. This CAREER project focuses on discovering new non-linear control strategies for such systems and providing provable guarantees for their performance. This research has the potential to reduce conservativism of designs by interactively extracting more useful information from multiplicatively-corrupted measurements. The ability to analyze and give guarantees for refined models can better characterize the risk for safety-critical systems where state-dependent noise terms cannot be ignored. The education plan for this CAREER project includes three structured interventions to improve inclusion in higher education. These include: (1) Recruit new students through the PI’s novel lower-division engineering course that showcases real-world applications of linear-algebra through hands-on hardware labs. This CAREER project will build pathways to disseminate this new curriculum to two-year community colleges. (2) Develop explicit support structures for students and build community through an automated, scalable and feedback-based process for student study groups. (3) Integrate accessible and structured research-at-scale experiences into regular coursework, which can reduce the barrier-to-entry and broaden research access for all undergraduate students. State-dependent noise in systems can emerge through model linearization, parameter uncertainty or drift, and timing jitter. A key technical challenge posed by zero-mean multiplicative observation noise is that it can destroy sign information, and thus linear strategies can be unboundedly suboptimal. However, non-linear strategies that leverage the dual nature of control can do better. This CAREER project investigates such strategies through three steps. (1) Since the PI’s prior work suggests that control strategies with both exploration and exploitation elements can perform well, this project will first compute the probability distributions induced on the state by specific periodic non-linear control strategies. The shapes of these distributions will guide the search for new control strategies using computational approaches (e.g. maximum likelihood, policy gradient). (2) Regret-based formulations will be used to provide guarantees where optimality of strategies may be difficult to establish. Information-theoretic bit-level models will be used to extend to the case of slow-varying multiplicative noise and non-zero mean. (3) Finally, the project will explore approaches to augment local models with multiplicative noise in guided policy search for reinforcement learning frameworks. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.