Frugal, Lifelong-Learning Control Systems with Execution Guarantees
Abstract
RESEARCH PROBLEM: Recent advances in AI have given rise to unprecedented capabilities in autonomous decision making. Notable succes,ses include learning to solve challenging video,ms. While groundbreaking, the results have been largely empirical, and many fundamental challenges must be addressed before they can, reach their true potential in safety-critical autonomous applications. Real-world autonomous deployments require AI systems that ha,ve safety and execution guarantees, are robust to changes in the environment, can make decisions quickly in real-time, and are fruga,ar control, ultimately enabling large-scale deployment of intelligent autonomous systems in the real world. This project aims to hel,p transform learning-based control from an art that requires deep domain expertise to yield effective controllers, into an engineeri,ng discipline similar to optimization-based control, where each controller is equipped with explicit certificates and safety guarant,ees. RESEARCH OBJECTIVES AND TECHNICAL APPROACHES: To develop a theory of frugal, lifelong-learning control systems with execution, guarantees, our effort will have two research focus areas: Obj1: Lifelong Learning with Safety Guarantees. Reliable real-world op,eration requires lifelong learning, to adapt to the new situations encountered. While such learning has been studied, past works (e,.g. meta-reinforcement learning) tend to ignore the transitory performance. The brittle behaviors in the transitory period would ea,sily permanently damage the system. To ensure safe transitory performance, we will develop (and study relative merits of) three mai,n approaches: learning value function certificates, learning uncertainty awareness, using safe execution sets during adaptation. We, will also investigate how they might complement each other.Obj2: Frugal Control Systems. For real-world deployments it is crucial t,hat learning-based control methods provide execution guarantees in terms of run-time speed and energy efficiency. We propose to inve,ilizes different types of learning and control methods to solve challenging robotic manipulation tasks from raw sensory input, while, ensuring safety and robustness. We will also investigate frugality of different neural net architectures. This will include an in,vestigation of how to compile trained neural nets into high-performance code, inspired by similar advances in model-predictive contr,ol. Finally, we will investigate how frugality can contribute to better explain ability and analyzability of the learned control sys,tems. ANTICIPATED OUTCOME AND IMPACT ON DOD: Our effort will lead to the next generation of intelligent systems, which will harness, the strengths of learning-driven AI approaches and the rigor and guarantees of controls and optimization. We anticipate to transfo,rm learning for control from the art it is today into a true engineering discipline: Disciplined Learning for Control. This will en,able widespread adoption across many application domains and engineering disciplines. The need to establish a more rigorous approach, to learning for control is especially dire for the DoD. When lives are at stake, how can we not use learning components, if they gi,ve the best performance (by far)? But how can we use them if we can t establish confidence that they will not make an unacceptably c,ostly mistake? Autonomous systems that interact with complex, dynamic environments are critical in a wide variety of military and c,ivilian applications, from warfighting to surveillance to disaster relief to flexible manufacturing and logistics. Such systems must
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 14, 2022
- Source ID
- N000142212121
Entities
People
- Pieter Abbeel
Organizations
- Office of Naval Research
- United States Navy
- University of California Regents