Automated and Adaptive Coaching Using Large Language Models
Abstract
Project AbstractCAPTAIN: Coaching AI Platform Targeting Advanced Instruction in Naval LeadershipBAA Number: N0001424SB001Leadershipin the Navy is an intense and challenging domain that benefits from an embedded learning culture, yet such a culture is difficult to achieve. One promising avenue for facilitating a learning culture despite such challenges is a formal coaching program, which if implemented effectively can improve the competence, character, connections, and overall capabilities of current and future leaders across disparate subcultures. Leadership coaching programs are designed to develop the unique skills and characteristics vital for effective leadership and generally involve one-on-one sessions in which a trained coach facilitates for the leader a structured journeyof self-discovery and skill enhancement. The Navy#s current approach to professional development relies on a combination of mid-year and annual performance reviews alongside peer coaching via the MyNavy Coaching initiative. This coaching approach is effective butlimited by the expertise and engagement of peer coaches, as well as the availability of high-quality peer coaches across the Navy. Fortunately, recent research on the digitization of coaching suggests new technology has enormous promise to enhance the efficiency of coaching efforts, including through automation and artificial intelligence.A promising avenue to provide highly effective coaching without the typical personnel costs is artificial-intelligence-assisted coaching, an approach that simulates coaching sessions using automated chatbots. Large language models (LLMs), which are sophisticated predictive algorithms trained on vast datasets of humanlanguage, represent a significant leap in the capabilities of artificial intelligence to process and generate text to facilitate conversational exchanges that are believably human. A key gap in our understanding of existing AI-assisted leadership coaching, which is currently mostly restricted to fine-tuned LLMs used in commercial startups, is related to the use of idiosyncratic coaching models with limited empirical evidence supporting their effectiveness. As such, the core technical goal of the current project is to develop and empirically evaluate an LLM-based leadership coaching portal based upon validated coaching models for use by midshipmen at the US Naval Academy. To do this, an LLM-based agent, which we have named CAPTAIN, will be rigorously developed. This AI agent will balance several competing goals to facilitate coaching sessions in order to maximize its effectiveness for improving leadership capabilities among USNA midshipmen while simultaneously maintaining strict adherence to both the DOD AI Ethical Principles and core values of the Navy as explained in documents like the Uniform Code of Military Justice, the Naval Officer#s Guide, the United States CodeTitle 10 # Armed Forces, and the Navy Code of Ethics.Thus, over the course of the proposed project, the team will develop a prototype AI-assisted coaching platform customized to the leadership development needs of Navy midshipmen and built on LLMs. This technology will provide real-time coaching to them over the project span and a scientific basis for the development of a permanent technical solution, if desired. As one of the Navy#s primary advantages over competitors lies in its personnel, coaching research can foster amore agile and adaptable force to confront future challenges. In alignment with ONR#s mission, the proposed project directly contributes to the science of AI-assisted coaching, which has the potential to improve personnel development, a critical requirement for the maintenance of future naval power and preservation of national security. The integration of the state of the art in language-based AI and modern coaching solutions as proposed also extends into new, interdisciplinary frontiers of science and technology.Approvedfor public release
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 13, 2024
- Source ID
- N000142412336
Entities
People
- Richard Landers
Organizations
- Office of Naval Research
- Regents of the University of Minnesota
- United States Navy