Constructing a Clinical Gulf War Illness (GWI) Case Definition Using Natural Language Processing and Advanced Machine Learning Algorithms

Michael E Debakey Va Medical Center, Houston TX

Investigators

Abstract

The overarching goal of this application is to develop a case definition of Gulf War Illness (GWI) for application in the Veterans Health Administration (VA) electronic medical record (EMR) based on advanced data science techniques. Background: Veterans from the Gulf War (GW, 1990–1991) continue to experience medical issues, notably GWI. This ailment affects 15 to 45% of the 693,826 deployed Americans and is attributed to toxic exposures encountered during the war. Common symptoms of GWI include chronic fatigue, pain, respiratory issues, gastrointestinal problems, skin conditions, and neurological disturbances. The Institute of Medicine recommended two definitions for research: Kansas and Centers for Disease Control and Prevention (CDC) Chronic Multi-symptom Illness definition. There isn't a case definition for GWI that has been universally accepted for clinical use because the illness presents differently in different individuals, the etiology and pathophysiology are unclear, and there is no biomarker. Significance: The data collection for previous attempts to develop a case definition were based on prospectively elicited self-reported symptoms only; the rich information available in the EMR was not incorporated . Consequently, the definitions were created without the full range relevant information about the individuals’ health and bias or inter-individual variability in the self- reported symptom presence and severity affects the validity of the case definition. In this application, we will overcome these limitations by utilizing the full breadth of EMR (e.g., diagnosis and procedure codes, lab, medications, and free-form text). Innovation: Our study encompasses several highly novel techniques, including the development of NLP algorithms to extract signs and symptoms of interest from free-form text in the EMR. The NLP algorithm will utilize conventional named entity recognition approaches and transformer- based models. Additionally, we propose to use Theory of Belief to combine the EMR-based machine learning (ML) models and NLP models. Aim1- Case definition by ML on EMR structured data, (ML-EMR): Develop and validate a case definition for GWI utilizing structured data that can be applied in the VA EMR. Aim2- Case definition by NLP on unstructured data (ML-NLP): 2a. Create an innovative case definition of GWI using NLP algorithms on unstructured data, i.e., free text from the EMR. 2b. Explore the possible implementation of transformer-based large language model (LLM) on free-form texts of patients with GWI to improve case definition using NLP. Aim3- Case definition by combining ML-EMR and ML-NLP: Develop a hybrid model for the case definition of GWI using unstructured and structured data models. Methodology: for Aim 1, we will apply supervised learning methods (e.g., random forest and logistic regression) to the variables to train, validate and test the model. Variables will be constructed from diagnostic and procedure codes, lab results, and vital signs. For Aim 2a, we will use Medical Concept Annotation Tool (MedCAT) to automatically recognize and link concepts in free text to standard terminologies, such as unified medical language system (UMLS). Then, we apply ML algorithms to identify veterans with GWI. For Aim 2b, we will benefit from pre-trained LLMs by fine tuning them with the annotated free-form texts of veterans with GWI. For Aim 3, we use Theory of Belief to combine the output of the two proposed Aims (Aim 1 & Aim 2) from structured data and unstructured data. Additionally, we utilize the full list of variables from EMR and NLP. Then we apply “most important variable selection process” and construct the classifier to discriminate the veterans with GWI from those without. Deliverables: This study will produce a continuous score indicating the probability of GWI from 0 (very low) to 1 (very high) for every GW veteran who has used VHA services. It will also create an enhanced, highly interpretable case definition representing continuity with existing symptom-based criteria for GWI, such as Kansas and CDC. In practice, the score could be made visible to a clinician in the patient’s EMR, and the clinician could then apply the enhanced, interpretable algorithm to make a clinical determination of GWI.