Compositional Learning from an Imperfect Primitive Skill Sets for Solving Complex Tasks
Abstract
This proposal addresses the defense interests described in Topic 8 of Advanced Learning-Enabled Intelligent Cyber-Physical Systems (Program Officer- Dr. MaryAnne Fields). Specifically, it introduces the novel compositional learning paradigm with components ranging from learning and representing primitive skills to identifying missing skills and their composition for solving long-horizon tasks. Compositionality refers to combining basic functions to create complex ones that can be further combined to solve new problems. Although a few methods exist that attempt to solve the skill composition tasks, they rely on executing the primitive skills to interpret their behaviors since the primitive skills are treated as a black box. This makes the composition process data inefficient. On the other hand, the lack of skillset representation makes the process of identifying and learning missing skills necessary to solve a given task challenging. Therefore, we believe that a key aspect of solving composition learning problems is the representation of primitive skills such that their behavior can be interpreted without execution. The benefits of interpretability are twofold. First, it can allow the selection of skills crucial for solving the given task. Second, it can also be used to identify the missing skills necessary to achieve the given new task specifications. Hence, we propose that differentiable Signal Temporal Logic (STL) can provide a way to solve the challenge of representing primitive skills. The STL specifications concisely describe complex tasks and provide compact representations of learned skills. However, to leverage STLs for identifying and learning missing skills and composing skill ensembles to solve long-horizon, complex tasks, the following research questions (RQ) are yet to be investigated- (RQ1)- How to incorporate differentiable STLs into pol- icy gradients for efficient skill ensemble learning via reinforcement learning (RL). (RQ2)- How to leverage STL specifications of a given skill ensemble in synthesizing compositional controllers for solving complex, long-horizon tasks. (RQ3)- How new STL specifications can be generated automatically for learning missing, novel skills. To address these research questions, we propose the following three research thrusts. First, we provide an approach inspired by Lagrangian Optimization for incorporating STLs into policy gradients for skill learning. This framework can also incorporate complex collision avoidance and kinematic constraints using differentiable predicates. Second, we propose to use neural attention models to compose a given skill set based on their STL specifications. We will also demonstrate the hierarchical composition of skills to generate intricate behavior policies. Third, we provide a framework for identifying missing skills and learning them concurrently with a composition policy to accomplish a new task. A unified framework combining three research thrusts will be demonstrated in solving dynamic goal reaching problems in various cluttered, real, and simulated environments. These tasks appear in a variety of scenarios, such as a quadcopter landing on a moving ship or a robot chasing a moving target on the ground while avoiding collisions with the surrounding obstacles.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 06, 2025
- Source ID
- FA95502410233
Entities
People
- Ahmed Qureshi
Organizations
- Air Force Office of Scientific Research
- Office of the Secretary of Defense
- Purdue University