Task-aware and Autonomous Robotic C-arm Servoing for Flouroscopy-guided Interventions

Johns Hopkins University, Baltimore MD

Investigators

Linked publications & trials

Abstract

Project Summary Fluoroscopy guidance using C-arm X-ray systems is used in more than 17 million procedures across the US and constitutes the state-of-care for various percutaneous procedures, including internal fixation of pelvic ring injuries. To infer procedural progress from 2D radiographs, well-defined views onto anatomy must be achieved and restored multiple times during surgery. This process, known as ”fluoro hunting”, is associated with 4.7 s of excessive fluoroscopy time per C-arm position (c. f. 120 s total per fixation), yielding radiographs that are never interpreted clinically, but drastically increasing procedure time and radiation dose to patient and surgical staff. Our long-term project goal is to use concepts from machine learning and active vision to develop task-aware algorithms for autonomous robotic C-arm servoing that interpret intra-operative radiographs and autonomously adjust the C-arm pose to acquire fluoroscopic images that are optimal for inference. We have three specific aims: 1) Detecting unfavorable K-wire trajectories from monoplane fluoroscopy images: We will extend a physics-based sim- ulation framework for fluoroscopy from CT that enables fast generation of structured and realistic radiographs documenting procedural progress. Based on this data, we will train a state-of-the-art convolutional neural net- work that interprets fluoroscopic images to infer procedural progress. 2) Developing and validating a task-aware imaging system in silico: Using the autonomous interpretation tools and simulation pipeline available through Aim 1, we will train an artificial agent based on reinforcement learning and active vision. This agent will be capable of analyzing intra-operative fluoroscopic images to autonomously adjust the C-arm pose to yield task- optimal views onto anatomy. 3) Demonstrating feasibility of our task-aware imaging concept ex vivo: Our third aim will establish task-aware C-arm imaging in controlled clinical environments. We will attempt internal fixation of anterior pelvic ring fractures and our task-aware artificial agent will interpret intra-operatively acquired ra- diographs to infer procedural progress and suggest optimal C-arm poses that will be realized manually with an optically-tracked mobile C-arm system. This work combines the expertise of a computer scientist, a surgical robotics expert, and an orthopedic trauma surgeon to explore the untapped, understudied area of autonomous imaging enabled by advances in machine learning in fluoroscopy-guided procedures. This development has only recently been made feasible by innovations in fast fluoroscopy simulation from CT to provide structured data for training that is sufficiently realistic to warrant generalization to clinical data. With support from the NIH Trailblazer Award, our team will be the first to investigate autonomous and task-aware C-arm imaging systems, paving the way for a new paradigm in medical image acquisition, which will directly benefit millions of patients by task-oriented image acquisition on a patient-specific basis. Subsequent R01 funding will customize this concept to other high-volume procedures, such as vertebroplasty.