Safety Against Latent Risks in Dynamic and Interactive Environments
Abstract
APPROVED FOR PUBLIC RELEASEDistributed multiagent systems must safely operate in unpredictable, interactive, and occluded environments with many latent risks. For instance, in a networked navy, ships cooperate among themselves and other navy assets. But, in adversarial situations, they can be surprised by unexpected events. For another example, drones flying through a crowded space should be prepared for the sudden appearance of an (unfriendly) agent hiding behind occluding obstacles. The intentions and actions of agents can change based on their interactions with others. Rules that govern safety (collision avoidance, safer detour, etc.) often directly conflict with those that can maximize performance (aggressive maneuvers, reaching goals quickly, etc.). The long-term impacts of all of these factors must be accounted for, despite the possible SWAP (size, weight, and power) constraints of some of the navy-deployed resources, when making real-time decisions in latency-critical environments. We envision our research will be useful in distributed maritime operations with agents (assets) communicating data and information and coordinating actions directly among themselves and the cloud to avoid delays and avoid breaking their covert operation.Existing methods are not sufficient to produce such capabilities due to the following challenges. First, there exists a multitude of latent risks arising from occlusions and unexpected interactions. The adversaries may hide purposefully or exploit low visibility, and the action of an agent can change the intentions/actions of other agents to produce unintended risk scenarios. Second, local failures can cascade and compromise system-wide performance. A successful attack on a large, centralized, unhardened system can disable the operations of the whole system. Further, accessing the cloud through satellites may require excessive radio power, risking denouncing the presence to others in covert operations. Third, there is a lack of safety guarantees in decentralized policy learning/updates. When agents learn their policies, the policies may not be safe during exploration (before the policy converges to an optimal one). When agents adapt their policies in a denied environmentwithout centralized information and cloud computation, the learned policy may not have gone through an exhaustive safety verification process. Finally, accounting for the long-term future can cause expensive computation and slow response, but a failure to do so will result in shortsighted behaviors that compromise safety and mission accomplishment.We will develop new techniques to ensure safety against latent risks in real-time in interactive environments. These techniques should be: 1) implementable in smaller dispersed agents; 2) transparent regarding hazardous scenarios; and 3) allow human operators to make informed decisions. To address these needs, we will develop a safe learning and control technique. Specifically, our proposed methods will be architected, developed, and implemented so they can be deployed with smaller dispersed distributed agents to ensure safety against latent risks in dynamic and interactive environments. The merits of the proposed methods will be demonstrated with simulated case studies of multiagent systems cooperating in adversarial and interactive environments with limited visibility. The proposed research will contribute to improving the safety of autonomous networked navy assets operating in adversarial settings. We will involve in the proposed research and supervise ROTC Midshipmen in the Pittsburgh region and seek feedback/collaboration with Navy personnel to better support Navy military needs.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 06, 2023
- Source ID
- N000142312252
Entities
People
- Yorie Nakahira
Organizations
- Carnegie Mellon University
- Office of Naval Research
- United States Navy