Multi-modal World-learning-model for robotics
Abstract
Underwater vehicles and Navy vessels are efficient and well-engineered machines, but due to the nature of their operation and environment, they can be susceptible to accidents and require crew rescue and escape. Due to the dangers around the disaster areas and the possible noxious environment, the use of robotic assistants is preferred to Navy Seals or other human operators in such crisis situations. Now suppose we want to send a robot to rescue a crew trapped in a capsized ship, for example. Or a robot that can find and free human buried in the collapsed debris of a building.Today robotic entities are not useful in this con text because of their lackof adequate environmental perception and of extensible controllers that can learn the complex sequences of actions required in a rescue operation. But why? Because we do not have the right algorithms that area trained to perceive the real 3D world and are able toplan motion and actions in 3D. What is missing is a world-model that can befoundational for robots to ground knowledge in, and withthat knowledge learn the ability to create sequences of actions that provide opportunistic results. In this proposal we propose to design advanced state-of-the-art learning algorithms and robotic controllers that can offer real-world like learning of 3D perception and manipulation needed to power successful crew rescue robotic entities. The product of the proposal will also augment current robotic controllers with the ability to extract complex knowledge graphs for real-world data and interactions.The proposed robotic controller and learning algorithm is designed to support several key applications, the most important of which are listed below:(1) learning a complex 3D scene understanding used for autonomous navigation and planning, operational target spatial recognition(2) learning object manipulation in 3D space, grasp assistance, learning to operate devices and objects(3) learning to plan complex action sequences for unforeseeable disaster scenarios.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Dec 15, 2023
- Source ID
- N000142412037
Entities
People
- Eugenio Culurciello
Organizations
- Office of Naval Research
- Purdue University
- United States Navy