A Framework for Semi-Autonomous In-Hand Telemanipulation

Colorado School Of Mines, Golden CO

Investigators

Abstract

This award supports research to develop and test a control framework for stable in-hand telemanipulation, using a physic-informed machine learning approach that can adapt the telerobotic controller to physiological limitations of individual users. In daily life, in-hand manipulation is needed to complete seemingly simple tasks such as adjusting the orientation of a power adaptor for plugging in to an outlet, as well as to complete complex skilled tasks such as manipulating a scalpel during a surgical procedure. The long-term objective of the line of research initiated with this project is to enable dexterous in-hand manipulation in tele-operation scenarios in which the robot can understand how to track real-time changes in human finger motions and use that information to actively ensure the stability of an in-hand object while it is being manipulated. This project promotes the progress of science and advances the national health, prosperity, and welfare by developing and testing novel methods implementing stable shared human and machine control of hand-held objects in tele-operation scenarios, such as tele-surgery, healthcare assistive robotics, and remote search and rescue. The project will also support outreach activities through an existing K-12 program at the Colorado School of Mines, at a local community college, and at STEM camp for girls. This project seeks to solve challenges associated with in-hand telemanipulation through the development and testing of a control framework that utilizes physic-informed hierarchical machine learning approaches to adapt the telerobot to the physiological limitations of the individual users. There are three research objectives: 1) to use physics-informed metrics to guide the robot control policy to generate stable grasp configurations; 2) to optimize interpretation of signals derived from the human hand motion tracking system using a hierarchical learning model; and 3) to personalize shared control of the manipulation task between the human and the robot through active machine learning guided by the user's corrective adjustments in real-time. The project team will use a commercially available robotic hand system with the semi-autonomous framework to conduct human-subject-involved evaluation. Subjects will perform tele-manipulation tasks of increasing complexity, ranging from the relative simplicity of a jar opening task to a more complex case requiring in-hand changes of tool position and orientation. The work promises to advance the science and engineering of human-robot cooperation through a novel semi-autonomous control framework that can support complex in-hand object manipulation for teleoperation. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.