Trust in the Machine: AI, Autonomy, and Military Decision Making with Lethal Consequences
Abstract
The U.S. military has employed artificially intelligent and autonomous-capable weapons systems since the 1980s, but technology and capabilities have drastically changed in the past three decades. In order to remain competitive, the U.S. military leaders must reconsider AI and autonomous weapons employment doctrine across the spectrum of conflict, as well as work to improve trust in AI and autonomous technology. The time available to bring lethal force to bear has decreased while the amount of contextual information that enables decisions on the use of force has increased. At the same time, gray zone conflict activity is increasingly blurring the line between peacetime operations and warfare. U.S. military forces exerting forward, deterrent presence in areas prone to activity that is not in accordance with international norms or law are increasingly exposed to complex risk that may be misunderstood and lethally miscalculated. When operating in the gray zone, where the distinction between peace and war blurs and where technology has compressed reaction times, servicemembers face a moral gray zone, a potential dilemma between the duty to abide by the principle of distinction in the law of armed conflict (LOAC) and the inherent right to self-defense. The ambiguity inherent in the operational gray zone contributes to a higher-than-average likelihood of human judgment failures in the accompanying moral gray zone. AI and autonomous technology have the potential to improve both the success of self-defensive actions and the adherence to LOAC but only if humans and organizations are able to establish trust in the machine operating intelligently and autonomously. Establishing trust requires that humans perceive machine actions as predictable, transparent, and traceable, understanding how judgments of accountability, morality, and ethics differ between machine and human, and thus ensure that servicemember and societal trust in the Department of Defense (DoD) is preserved.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 05, 2019
- Accession Number
- AD1121116
Entities
People
- Christi S. Montgomery
Organizations
- Naval War College