Testing Evaluation Metrics for Interface Terminologies

Vanderbilt University, Nashville TN

Investigators

Linked publications & trials

Abstract

DESCRIPTION: As a first step towards a career focusing on evaluation of computer-based clinical tools, the principal investigator will undertake to develop and test metrics for terminologies that support structured documentation tools. While documentation tools are proliferating in both public and commercial sectors, major U.S. healthcare delivery stakeholders frequently deploy them with little basis for determining suitability for given specific tasks. Improving and evolving existing terminologies requires development of objective formal review methods based on proved evaluation techniques. Few evaluation metrics for structured clinical note capture terminologies have been described in the biomedical literature. The need to validate and evaluation methodology for interface terminologies takes on particular urgency with the recent licensing of SNOMED CT into the Unified Medical Language System and its subsequent recommendation as a national terminology standard. MEDCIN, another large terminology, was designed specifically to support direct data entry at the point of care while SNOMED CT was created primarily as a reference terminology to aggregate, exchange, and store medical data. Neither MEDCIN or SNOMED CT have been evaluated in terms of their usability or suitability as clinical interface terminologies. The proposed research will develop and test a methodology for evaluating clinical interface terminology usability by identifying and examining measurable terminological attributes. The proposed study will quantify for given terminologies their expressivity and accuracy, the degree to which they incorporate key medical knowledge, such as the normal state, preferred synonyms, and links to relevant modifiers, and the degree of balance between precoordination and post-coordination. It is hypothesized that these attributes can predict usability outcomes, including time of documentation, user satisfaction, rates of documentation failure, and numbers of steps required to document each concept. Through this mentored study period, the Principal Investigator will develop research skills and experience to achieve the goal of becoming an independent researcher.