An Interactive Web-Based Game for Collaborative Labeling of Medical Images

Johns Hopkins University, Baltimore MD

Investigators

Linked publications & trials

Abstract

DESCRIPTION (provided by applicant): Statistical atlases of regional brain anatomy have proven to be extremely useful in characterizing the relationship between the structure and function of the human nervous system. Typically, an expert human rater manually examines each slice of a three-dimensional volume. This approach can be exceptionally time and resource intensive, so cost severely limits the clinical studies where subject-specific labeling is feasible. Methods for improved efficiency and reliability of manual labeling would be of immense benefit for clinical investigation into morphological correlates of brain function. The goal of the proposed work is to enable an alternative to expert raters for medical image labeling through statistical analysis of the collaborative efforts of many, minimally-trained raters. The proposed research investigates extension of established practices for volumetric labeling and web- based collaboration to create an innovative infrastructure for labeling. A user interface will be developed as an interactive web-based game in which users will solve small puzzles. Each puzzle will represent a portion of the overall labeling challenge. Overlapping and complementary results from many raters will be recombined within a statistical framework that estimates and minimize the uncertainty of labeled regions. Real world performance of raters is difficult to predict given varied training experience, work environment, computer hardware, and motivation. To investigate the practical utility of this system, we will establish reliability measures for intra-rater, inter-rater labels as well as test-retest metrics for statistically recombined results given three motivational paradigms: 1) Paid interns will be recruited for a pilot study;2) The public will be encourage to participate in a competitive, online game;and 3) A limited trial will be conducted1 with1 payments1 based1 on1 performance1 in1 the1 labeling1 game .1 The advantages of this labeling approach will be demonstrated in pilot studies of the cerebellum in spinocerebellar ataxia and of the hippocampus1in1Alzheimer s1Disease.1The system will be made freely available. The proposed system will dramatically reduce the cost of creating manual labels by enabling collaborative community involvement, while providing active monitoring and control of the accuracy and uncertainty of the results. We will lay the foundation for a language of labeling, which is a novel, consistent framework for describing labeling objectives in a hierarchical, programmatic manner. Increased availability of high reliability manual labels will improve the quality, consistency, and efficiency of quantitative MRI analyses in the brain in health and disease. PUBLIC HEALTH RELEVANCE: The proposed research investigates an alternative to expert raters for manual labeling of medical images through statistical analysis of the collaborative efforts of many, minimally- trained raters. The web-based system will dramatically reduce the cost of studying anatomical regions of interest by enabling collaboration among many individuals, while providing active monitoring and control of the accuracy and uncertainty of the results.