Analyzing the SSA Disability Evaluation Process

Clinical Center

Investigators

Linked publications & trials

Abstract

Analytics (Objective 1) 1) Adjudicator Support Initial SSA decisions are made by examiners who have very little time to review evidence for a case, usually in the form of lengthy medical records from various healthcare providers. The objective of this focus area is to develop, adapt, and apply methods that aid SSA disability adjudicators to reach accurate, consistent, and timely decisions in accordance with SSA regulations and available medical evidence. This year, NIH deliverables encompassed the identification of functional terminology, specifically through a preliminary demonstration of named entity recognition and a demonstration of document classification. Identification of functional terminology: The aim of this subproject is to extract functional information from clinical documentation, which is an underdeveloped area of machine learning and natural language processing (NLP). In order to identify functional information, it is necessary to understand the variation in text used in clinical documentation, noting those rich in functional information compared to those focusing on health conditions. This year, we made progress on methods to characterize medical documentation with respect to functional language and extracting this information from samples of medical documents. To advance this work, we continued to develop annotation resources developed by content experts to serve as a gold standard for machine learning methods, models to automatically identify functional information, and resources to support the development and implementation of these models. We have developed models for SSA for labeling, ranking, and/or classifying medical records based on information related to mobility, self-care, domestic life, and interpersonal interactions and relationships to help improve the efficiency of SSAs case review processes. During this year, we have focused on furthering the development, testing, and refinement of these models. We have also continued to develop an ontology for mental functioning and corresponding terminology. Developing such models and resources is supported by a number of projects and research areas including processing medical records and exploring topics of temporality, variability, and bias. WD-FAB development (Objective 2) In collaboration with the SSA, the NIH and Boston University developed a comprehensive and efficient assessment instrument called the Work Disability Functional Assessment Battery (WD-FAB). Contemporary models of disability indicate that in order to assess work disability, what individuals can do and what they are expected to do for work must both be assessed. The WD-FAB is intended to assess what individuals can do. The WD-FAB is a 15-20-minute individualized assessment of functional activity that uses Item Response Theory (IRT), along with computer adaptive technology (CAT), to select the most relevant test items from a large pool of items to measure self-reported functional ability. Item-based scoring means respondents do not need to answer all items or the same items to obtain comparative scores and scores are obtained in a highly efficient manner. 2) Functional Assessment Tools The objective of this focus area is to develop new ways to collect, structure, and interpret functional data for use by SSA. This work will include development of the WD-FAB and methods to assist in interpreting WD-FAB results. WD-FAB instrument development: The aim of this subproject is to finalize the development of the WD-FAB so that it is ready for real-world, applied testing. The instrument now includes over 300 items across eight domains, four of which represent physical function (basic mobility, upper body function, fine motor function, community mobility) and four of which represent mental health function (communication & cognition, resilience & sociability, self-regulation, and mood & emotions). Functional stages (e.g., low, moderate, high functioning) were developed by content experts to aid score interpretation. To date, the reliability and validity of the WD-FAB have been supported by a variety of evidence from a continuum of studies. This year, we have developed a user guide for the WD-FAB intended for both SSA and external researcher use. We have also continued to develop and refine optimal methods in item response theory used in the WD-FAB software, as well as exploring relationships between versions of the WD-FAB as we implement these updated methods. During this year, we have also started analyzing WD-FAB data collected through various research efforts. One example of research efforts in this area is a collaboration with the University of New Hampshire to study how WD-FAB scores align with job demands. SSA has also provided initial data from their ongoing Supported Employment Demonstration (SED) where the WD-FAB is one of several measures collected as part of an effort to understand what supports help individuals with mental health disorders return to work and/or prevent application for disability benefits. We are using longitudinal WD-FAB data from the SED to explore changes in scores over time, which will then support future work with SSA that looks at the potential role of the WD-FAB in their continuing disability review process, which assesses whether beneficiaries continue to meet SSAs definition of disability. Publications generated by this year's research: Newman-Griffis D, Divita G, Desmet B, et al. Ambiguity in medical concept normalization: An analysis of types and coverage in electronic health record datasets, Journal of the American Medical Informatics Association, 2020, ocaa269. DOI: 10.1093/jamia/ocaa269. Jimnez-Silva R, Zhou C, Desmet B, et al. Curating Annotated Corpora for Functioning Information. Poster presentation, Rehabilitation Research 2020: Envisioning a Functional Future Conference. 2020 October. Zirikly A, Zhou C, Desmet B, et al. Capturing function information from clinical free text using NLP: fundamental research for decision support. Poster presentation, Rehabilitation Research 2020: Envisioning a Functional Future Conference. 2020 October. Sacco M, Divita G. Advancing a conceptual framework used in the development of an ontology of mental functioning. Presentation, Maryland Occupational Therapy Association centennial conference 2020. 2020 November. Camacho Maldonado J, Ho P-S, Jimnez-Silva R, et al. Experiences and Challenges in Manual Annotation of Functioning Information in Medical Records. Poster presentation, American Medical Informatics Association 2020 Virtual Annual Symposium. 2020 November. Divita G, Zirikly A, Breitfeller L, et al. Angelfish: Building a structurally diverse clinical document corpus. Poster presentation, American Medical Informatics Association 2020 Virtual Annual Symposium. 2020 November. Jimnez-Silva R, Zhou C, Desmet B, et al. Curating Annotated Corpora for Functioning Information. Poster presentation, American Medical Informatics Association 2020 Virtual Annual Symposium. 2020 November. Newman-Griffis D and Fosler-Lussier E. Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health. Frontiers in Digital Health, 2021; 3:620828. DOI: 10.3389/fdgth.2021.620828