VA Office of Rural Health and QUERI Partnered Evaluation Center: Implementation, Sustainment, and Spread of Effective Programs to Improve the Health and Well-Being of Rural Veterans

Iowa City Va Medical Center, Iowa City IA

Investigators

Abstract

Since 2007, VA Office of Rural Health (ORH) has supported enterprise wide initiatives (EWIs) and promising practices to improve the health and well-being of rural Veterans. Program evaluation is required of projects to drive improvement and determine if goals are being met. A comprehensive review of ORH-funded programs is essential for identifying best practice and strategies to improve the quality and safety of care for rural Veterans and to inform dissemination and implementation (D&I) science. However, this review has not been conducted. We propose a partnership between QUERI and ORH?s Center for the Evaluation of Enterprise Wide Initiatives to conduct a systematic review of EWI evaluations. The partnership will result in a synthesis of the portfolio of ORH EWIs to support current projects and improve rural Veteran care by leveraging D&I science and CEEWI?s leaders in the fields of D&I, program evaluation, and health services research. In FY19, the EWI program consisted of 55 ORH-supported initiatives that reach 97% of VA medical centers in all 18 Veteran Integrated Service Networks (VISNs). A QUERI partnership with CEEWI will benefit ORH with systematic documentation and standardization of EWI implementation and evaluation across the country. This work will situate ORH as a leader in D&I by synthesizing its knowledge of ground-up program development and implementation to be more widely generalizable. ORH has never sought to comprehensively examine implementation strategies and outcomes across their portfolio. ORH?s CEEWI team, in partnership with QUERI, seeks to improve its evaluation program and strengthen the long-term sustainment of its projects. ORH began the process of building an infrastructure for systematic evaluation by asking EWIs to use RE- AIM as their evaluation framework. Recently, EWI evaluations have included Reach, Effectiveness, Adoption, Implementation, and Maintenance outcomes. However, ORH program analysts have encountered a number of barriers to the evaluation process and achieving full integration and operationalization of RE-AIM has been one of the challenges. The CEEWI team will work closely with the ORH program analysts to understand the specific challenges related to RE-AIM integration and to improve and standardize the EWI evaluation process. The specific aims of the QUERI partnership with CEEWI are to: 1) Identify implementation strategies and outcome measures described in ORH enterprise-wide initiatives (EWIs) evaluation reports and supplemental, qualitative interviews. Conduct a content analysis of the EWI evaluations submitted to ORH and supplemental, qualitative interviews to examine patterns of: a) Expert Recommendations for Implementing Change (ERIC) implementation strategies (e.g., which strategies are used most frequently, how they are operationalized and adapted); and 2) RE-AIM outcomes (e.g., how they are operationalized, key challenges). 2) Improve the ORH EWI evaluation process by insuring RE-AIM outcomes are effectively integrated into evaluations. Develop tailored toolkits and cyberseminars for EWI implementation and evaluation teams based on formative evaluation conducted with ORH program analysts. 3) Explore patterns of implementation strategies and outcomes within the quantitative and qualitative data associated with greater sustainment and spread of EWIs. Examine association of the number/type of implementation strategies with greater sustainment and spread. The long-term goal of the CEEWI team is to improve the EWI evaluation process and support sustainment and spread of effective ORH EWIs through effective implementation strategies. The approach and findings from the systematic review of evaluations will be disseminated to D&I researchers through presentations at professional meetings and publications.