Communicate Assistants for Grounded Information Aggregation and Summarization
Abstract
Approved for Public ReleaseOrganizations like ONR are constantly inundated with a large volume of documents related to research anddevelopment projects. These documents range from proposals outlining technical challenges and speculative ideas to solutions backedby supporting evidence. As the data volume increases, it becomes increasingly difficult for human experts to process and aggregate the findings manually. To enable efficient handling of the data, the development of natural language collaborative assistants is being motivated. These assistants must be able to handle new incoming data that is constantly evolving and discuss new frontiers and findings.To facilitate rapid information aggregation, our proposed solution is CAGIAS (Communicative Assistant for Grounded Information Aggregation and Summarization), a chatbot interface that utilizes Large Language Models as a building block. Users can engage in conversational experiences by asking queries and providing further clarifications or details, supporting complex reasoning, and handling uncertainty. CAGIAS is tailored to interact with a repository of ONR report documents, making it different from contemporary chatbots like ChatGPT or Bard. We will acquire the documents at the beginning of the project through our connections with ONR and create a suitable set of evaluation criteria tailored to the needs of this particular domain.The chatbot can find evidence related to theuser s active hypothesis, including evidence that strengthens or weakens their proposed hypothesis. Each time the chatbot responds to a user query, it has the ability to elaborate on evidence and reasoning for its arguments. Additionally, the system-generated arguments have associated uncertainty measures, reflecting the chatbot s reasoning, source reliability, and the frequency of a particular argument among existing reports.The chatbot can store user feedback in the form of interpretable symbolic memory, allowing it to focus on long-range dependencies that may not fit in its active memory but are necessary for the user s intent. Furthermore, a user may choose to convert significant segments of the conversation history into a report. At this point, they can request the chatbot togenerate a report incorporating the suggested elements from the conversation history.Overall, CAGIAS is designed to be an assistantto an expert who wishes to explore vast amounts of information through simple language queries. The system is equipped with features to find the needle in the haystack, enabling efficient handling of the data volume and facilitating rapid information aggregation.As the chatbot becomes more domain-specific over time, it will better align with the specific purposes and language used in this domain, potentially involving pretraining language models tailored to the documents and common queries in this area.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 08, 2024
- Source ID
- N000142412089
Entities
People
- Daniel Khashabi
Organizations
- Johns Hopkins University
- Office of Naval Research
- United States Navy