Deep Learning for Imaging Report Generation to Support Diagnosis of Military-Relevant Injury in a Deployed or Operational Environment
Abstract
The project will develop an artificial intelligence system that can read an X-ray image of a muscle or skeletal injury and generate a plain English text report in a quality as good as that written by experienced physicians. The generated report will be readily informative for care providers to treat the injured patient. The project will directly address the FY19 AIMM Focus Area on "Algorithms/tools for decision support in a deployed or operational environment to diagnose military-relevant disease, illness, or injury." The system, called AIRGO, will be a deep neural network system that can learn from training examples to perform the task. In this case, the training examples are pairs of X-ray images and their corresponding reports. AIRGO will learn how to read an image and generate a report from the training examples. The training examples to be used in this project to train AIRGO consist of more than 2.7 million real X-ray images of patients of muscle and skeletal injuries and their corresponding radiology reports written by board certified radiologists at Veteran Affairs medical centers. The patients and injuries reported in these training examples faithfully represent the injuries most likely for Service members and Veterans. The proposed automated imaging report generation system will benefit the care needs of Service members and Veterans who have combat-related muscle and skeletal injuries. The benefit is particularly eminent in battlefields when massive orthopedic injuries overwhelm available well-trained personnel. AIRGO can be deployed with the imaging equipment to generate a report right after a scanning of an injury. The report can be transferred to the care centers before the injured patient arrives, thus shortening the communication gap between the point of injury and the patient care, leading to earlier treatment and better outcomes. Improved communications between patients, radiologists, and trauma care teams by AIRGO s report generation service can also make a major impact in natural disasters and provide benefit to the general civilian population. For AIRGO to be effective, in addition to the training specialized to report generation, we will also need to train it to interpret images and possess general knowledge of the English language for radiology reporting. The project will develop systems and technologies required for AIRGO. In particular, AIRGO will learn from 150 million radiology reports written by board certified radiologists at Veteran Affairs medical centers to learn the language of radiology reporting. However, though the size of clinical documents that we will be using to develop the system is unprecedentedly large, the clinical documents are known to contain pervasive duplication due to the use of copy-pasting and templates during the process of clinical documentation. The duplication may mislead the contextualized word-embedding models to acquire bias that encodes spurious correlations in the system. Examples of spurious correlations include linking whether a patient should have a follow-up to the location that imaging is performed. Spurious correlations lead to systematic errors. The project will develop strategies to mitigate the bias based on previous work by the project team. Previously, the project team developed an efficient method to uncover duplicates in millions of clinical documents. This method can be used to guide the bias mitigation strategies to identify spurious correlations. These systems and technologies will be useful for other clinical applications of artificial intelligence. Examples of such clinical applications include automated coding of patient records and automated diagnosis of diseases from imaging. These applications can save substantial time and workload of clinicians, experienced or inexperienced alike, and benefit patients.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 10, 2021
- Source ID
- W81XWH2010693
Entities
People
- Chun-Nan Hsu
Organizations
- United States Army
- University of California, San Diego