CORE--EVALUATION

University Of California Los Angeles, Los Angeles CA

Investigators

Linked publications & trials

Abstract

The objective of the Evaluation Core is to provide the practical infrastructure and expertise for the coordinated effort of each Project's evaluation. Evaluation will measure the effectiveness of developed technologies and their impact on well defined patient populations and health care providers. To achieve this goal, the Evaluation Core will specifically: (1) assist the Projects in the laboratory evaluation of technologies by aiding in the implementation of evaluation instruments and tools; (2) assign and managed dedicated staff members to each Project, charged with collecting data from clinical sites; (3) perform data checks for consistency and range (i.e. quality control); (4) archive evaluation data into a central repository; and (5) analyze the collected data. These five tasks are inclusive of both technical and clinical evaluations. For each Project, technical, process, and satisfaction measurement endpoints are clearly described within the Project-specific sections of the Evaluation Core. Technical measures encompass the evaluation of information retrieval results, machine learning, end-user satisfaction, and other Project-specific technical metrics. Primary process measures include time to final disposition: time to stabilization; time spent by physicians for chart review; and the length of time for medical document coding. Secondary process measures include the number of primary care and specialty visits, and the number and types of procedures performed. Finally, physician and patient satisfaction are also evaluated in appropriate domains. The totality of these measures is designed to evaluate the impact of the described technologies in a real-world clinical environment. These initial measures, along with technical development described in this Program Project may also lay the appropriate groundwork for future cost-effectiveness evaluation by: (1) defining characterizations of appropriate imaging based "episodes" of care; (2) recording resource utilization; (3) gathering patient-specific information related to presenting symptoms and "final disposition"; and (4) storing patient-specific information in a structured format to facilitate recall. Three different designs (natural experiment, time series, and "off/on/off") are used. The choice of experimental design for each Project was influenced by its real-world clinical patient settings and the availability of data for a control group. Power calculations are defined and presented to determine the appropriate sample sizes required for meaningful evaluation. Interaction between the Projects and the Evaluation Core is enhanced by: (1) regular meetings of the Project leaders, Core directors, and the PI; and (2) the assignment of dedicated Evaluation Core is enhanced by: (1) regular meetings of the Project leaders; Core directors, and the PI; and (2) the assignment of dedicated Evaluation Core staff to each Project who are familiar with the details of the Project while working under the direct supervision of the Core faculty.