Providing timely and effortless access to reliable health-related information for decision support and education

National Library Of Medicine

Investigators

Linked publications, trials & patents

Abstract

Our methods use text, image and video features extracted from relevant components in a document, database, or case description to achieve our objectives. To evaluate and demonstrate our techniques, we have developed Open-i (pronounced open eye, available at http://openi.nlm.nih.gov), a hybrid system combining text-based searching with an image similarity engine; an online radiology teaching tool MedPix (https://medpix.nlm.nih.gov/home); a clinical question answering system LHC CQA1.0, and a prototype consumer health question answering system https://chiqa.nlm.nih.gov/. The Open-i system enables users to search for and retrieve citations that are enriched with relevant images and bottom line (or take away) statements extracted from the Open Access subset of the PubMed Central repository maintained by the National Library of Medicine (NLM); as well as over 8,000 radiology images and 4,000 radiology examination reports from the Indiana University collection of chest x-rays; 67,517 images from NLM History of Medicine collection; and about 2,064 orthopedic anatomy illustrations provided by Norris Medical Library, University of Southern California. Our clinical question answering system is based on the framework for asking well-formed questions developed by the evidence-based medicine experts. Their analysis showed that presenting a clinical information need as four-part question frame: patient characteristics/problem; planned intervention; comparison; and desired outcome, helps formulate search engine queries that lead to relevant results. We developed methods for automatic extraction of question frames from information requests, automatic query formulation and automatic extraction of answers from retrieval results. The LHC CQA1.0 system extracts the bottom-line advice from biomedical publications and aligns the question frames and the answers to find the best answer. The CQA 1.0 system is currently used to provide bottom-line for retrieved images in the LHC Open-i system and to provide summaries of the biomedical articles in the LHC Open Summarizer. Using images from Open-i and MedPix, we have created several collections of clinically relevant question-answer pairs pertaining to images and used the collections in the biomedical Visual Question Answering (VQA) challenges, which we co-organized within the international ImageCLEF evaluations. In FY2022, we have developed several new approaches to facilitate understanding information requests sent to NLM customer services and long queries submitted to MedlinePlus search engine. Information requests sent to customer services are often several paragraphs long and provide the background and context that the customers believe will help understand their needs. For example, customers often describe several generations of their families affected by a disease and ask if their children will have it. The long MedlinePlus queries consist of one or two sentences and are often formed as questions. Both request forms are usually ungrammatical and rife with misspellings, abbreviations, and informal language. We have developed a spellchecker for consumer language that is performing adequately on the misspellings important to understanding of the needs. After correcting spelling, our system employs three modules: a knowledge-based and a supervised machine learning method to understand the main points of the request, such as the disease or a drug of interest and the type of information about it. The systems extract the main points, which we found are sufficient to automatically search MedlinePlus and find authoritative and relevant pages for 65% of the requests. The third approach is to find similar questions that already have authoritative answers, e.g., provided by NIH institutes. In FY2022, we explored several new directions to enhance the prototype consumer health question answering system CHiQA. First, we noticed that the answers to many questions asked by the public could be found only in scientific papers written in specialized professional language that is hard to understand. To explain these papers in plain language, we created a collection of close to 1000 PubMed abstracts translated into plain language. These collection enables training and testing Deep Learning approaches for automatic translation of the scientific language into plain. Second, we noticed that many long requests seek support beyond informational needs. People often need emotional and network support. To explore if these needs can be recognized and answered automatically, we labeled questions and answers in a publicly available collection with detailed emotional and social support needs labels. The preliminary results obtained using these data show that Deep Learning approaches can recognize emotional and social support needs. Finally, we believe that in many cases showing the answer is more helpful than text, e.g., if a patient is asking how to do a self-exam for breast cancer. In these cases, a video from a reliable source is the best answer. To see if automatically finding the video clips that answer the questions, which we call instructional, is possible, we developed the first medical video question answering collection, established baseline approaches for finding answers, and conducted a community-wide evaluation of the state-of-the-art approaches to medical video question answering. The MedVidQA 2022 challenge, associated with the BioNLP 2022, an Association for Computational Linguistics workshop, attracted wide participation of the international community.