Improving results and conclusions about harms in systematic reviews of drugs

Univ Of North Carolina Chapel Hill, Chapel Hill NC

Investigators

Abstract

PROJECT SUMMARY/ABSTRACT Systematic reviews (SRs) of interventions might report inconsistent results and conclusions about patient safety, and they might differ from other sources of patient safety information such as real-world evidence (RWE) and drug labels. SRs underpin clinical guidelines by the US Preventive Services Taskforce, professional societies, and national insurers. Guidance says that SRs should address both harms and benefits of interventions, yet most SRs are designed to assess benefits and consider harms as secondary outcomes. Moreover, guidance for conducting and reporting SRs includes scant advice about the different types of data, and different study designs, for harms compared with benefits. In a preliminary study, we found that 70 SRs reached different conclusions about the harms of a single drug, and conclusions in SRs were not consistent with the drug label. Inconsistency suggests that some conclusions about harms in SRs might be wrong. In this study, we propose to use innovative methods, including natural language processing (NLP), to assess the consistency of methods, results, and conclusions about patient safety in >19,000 SRs published from 2004— 2024. Advances in NLP have created new opportunities for evidence synthesis using models that can identify and interpret text for extraction (e.g., from journal articles). Specifically, we will leverage state-of-the-art NLP tools and methods, including those based on large language models, to extract and synthesize evidence on harms from SRs, RWE studies, and drug labels at a granularity that has not been attempted previously. Then, we will compare SRs that include the same drug (we estimate there are >4,000 overlapping SRs), and we will compare SRs with RWE and drug labels. We will investigate whether SRs use methods for assessing harms that might lead to inconsistent conclusions about patient safety. We will also investigate whether SRs use methods that lead to differences compared with RWE and drug labels. For example, SRs often exclude RWE. Moreover, SRs are usually limited to one health problem; by comparison, drug labels include harms that could occur when people take the drug to treat different health problems. Highlighting specific drugs for which SRs might misrepresent risk of harm, and identifying why certain conclusions might be wrong, could immediately help patients and healthcare providers make better-informed decisions. Using our findings, we will develop improved guidance for synthesizing and reporting harms in SRs, which should ultimately improve conclusions about patient safety in SRs and clinical guidelines. Specifically, we aim to answer the follow questions: Aim 1. Are results and conclusions about patient safety in SRs of the same drug consistent? Aim 2. Do SRs use RWE, and do the results and conclusions of SRs address harms found in RWE? Aim 3. Is information about harms in SRs consistent with information in drug labels? Aim 4. What recommendations could improve the synthesis and reporting of harms in SRs?