DURIP Mobile-manipulation robots for integrated intelligence research
Abstract
# ONR: Marc Steinberg # Code 351 # Naval Air Warfare and Weapons# ONR: Behzad Kamgar-Parsi # Code 311 # C5ISRTThis abstract is publicly releasable.Our goal is to design, implement, and test software for autonomous systems that can operate robustly in complex environments, ranging from supply depots to battlefields. Such systems must perceive their environments through their sensors, integratetheir perceptions over time into an understanding of the domain they are operating in, accept goal specifications from human operators, plan long-horizon courses of action to achieve those goals, gather information as necessary, re-plan when the domain or goal changes, and execute low-level sensory-motor controllers as needed for success. Implementing such systems requires a tightly integrated approach, knitting together many disparate sub-fields of artificial intelligence and robotics. To test and improve methods for integrated systems, it is necessary to have physical hardware with highly capable sensing and actuation that will operate robustly in avariety of environments with very little downtime.We propose to purchase two Boston Dynamics #Spot# robots, which have legs for locomotion and a versatile manipulator arm. These robots have a large suite of integrated visual and depth sensors and a very well-tested and highly robust underlying low-level control system. They will enable a focus on sense-making, planning, and autonomous operations that will substantially improve our research capacity in multiple projects supported by all three participating agencies.The robots will be used to solve problems that include finding desired objects, distributed over multiple crowded rooms, and moving them todesired locations or arrangements, clearing away piles of rubble that block a travel route, and using objects as tools. Specific tasks will include: packing multiple new objects into an already-crowded set of shelves in a supply depot; finding a particular part in a set of cupboardsand extracting it, possibly removing other objects in order to locate the desired one, and putting them back where they were found; scouting multiple rooms in multiple buildings, opening drawers and doors as necessary, to build up an inventory of what is there and being able to return later to locate an object, robustly handling the possibility that objects may have been rearranged by other parties in the meantime.This task suite will be used to demonstrate and evaluate progress in multiple aspects of intelligence, including:# Aggregating perceptual information into an object-based representation of the domain.# Reasoning about whatis known and not known, so that the robot can explicitly gather information, ranging from the mass or material of an object to how a door can be opened, as needed.# Planning safe courses of action while taking into account the inevitable uncertainty remaining about the domain.# Planning and executing joint courses of action using multiple robots, both for information gathering and for achieving domain objectives.# Breaking very large planning problems down into manageable sub-problems via hierarchical reasoning and learning to focus attention on relevant objects and other aspects of the problem at hand.# Gathering large amounts of information needed for training foundational generalpurpose neural-network models for behavior in physical domains.# Learning, during execution, from very few examples, to adapt the larger-scale more general-purpose models for new purposes. This work is critical to the success and competitiveness of the US Department of Defense and the acquisition of this equipment will significantly accelerate research progress.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 15, 2024
- Source ID
- N000142412174
Entities
People
- Leslie P. Kaelbling
Organizations
- Massachusetts Institute of Technology
- Office of Naval Research
- United States Navy