A Real-Time, Markerless 3D Tracking and Perturbation System to Reverse Engineer Control of Insect, Bird, and Robotic Flight and Swimming
Abstract
Compared to the best robots, animal locomotion is remarkably robust. For instance, a fly can sustain wing damage due to predation and readily compensate for aerodynamic asymmetries, a feat with no current engineering analogue. To reverse engineer locomotion control in nature, it is essential to generate well-controlled perturbations to animals’ locomotory systems during natural behavior, both internally (e.g., optogenetic perturbations) and externally (e.g., gust perturbations).In this DURIP project, we will develop a real-time, markerless 3D tracking system to study insects, hummingbirds, and aerial and underwater robots by enabling spatially and temporally precise perturbations. These perturbations will be used to reverse engineer the control of animal and to perform comparative experiments between animals and robots. A variety of perturbation mechanisms will be developed and integrated with the tracking system; together they will be used to study how animals respond, compensate, and adapt to perturbations across multiple time scales. These perturbations include both internal origins- e.g., wing damage and optogenetic manipulation, and external origins- e.g., visual stimulus, magnetic force, and wind gust. Initially our effort will focus on flight, but it will extend to swimming. The proposed system will enable unparalleled research capabilities at Penn State, including but not limited to- 1) Perturbations of flight systems in free flight, such as visual perturbations during landing, wing damage in insects using a laser- based system and magnetic perturbations of hummingbirds, 2) Optogenetic perturbations of freely flying fruit flies enabling characterization of neural circuits underlying sensorimotor integration,3) Predator-prey interactions, social interactions and collision avoidance among bird, fish and insect groups. The tracking system will be camera-based and require millisecond delays to track body position and orientation (6 degrees of freedom), thus enabling markerless tracking of small animals to trigger precise perturbations at specific timing and spatial location and animal orientation. Acquisition of this instrumentation will deepen the scientific understanding of robust and adaptive control mechanisms in flying and swimming animals and robots.The proposed DURIP project will lead to a shared facility at Penn State and support collaborative research across the College of Engineering (Mechanical Engineering, Electrical and Computer Engineering, Aerospace Engineering) and Eberly College of Science (Biology and Entomology). It will support and augment multiple current DoD projects, including AFOSR Grant (No. FA9550- 20-1-0084) on Flying in an Uncertain World- Decoding Rules of Adaptive Neural Control in Insect Flight, ONR Grant (No. N00014-19-1-2540) on Integrative Studies of Escape Flight in Hummingbirds- From Perception and Aerodynamics to Maneuverability, ARO Grant (W911NF-20-1-0226) on Biologically-Inspired Sensing and Control for Shallow Water Robotic Locomotion.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 29, 2024
- Source ID
- FA95502310094
Entities
People
- Jean-Michel Mongeau
Organizations
- Air Force Office of Scientific Research
- Pennsylvania State University
- United States Air Force