CHS: Small: Understanding Environment Perception and Task Performance in Human-in-the-Loop Tele-robotic Systems (HiLTS)

Johns Hopkins University, Baltimore MD

Investigators

Abstract

The human body is highly capable of dexterous manipulation. This dexterity is developed and refined over many years based on touch or tactile feedback from repeated practice performing manipulation tasks in many different environments with varying types of constraints. There are situations, however, where constraints on the environment or body prevent direct manipulation. For these situations, such as minimally invasive surgical robots, bomb disposal robots, and prosthetic limbs, manipulation can be provided through the use of collaborative robots. Such robots or haptic devices, called human-in-the-loop telerobotic systems (HiLTS), provide the much-needed tactile feedback as with the natural limbs working on the manipulations directly. HiLTS are designed to extend and, in some circumstances, improve the dexterous capabilities of the human operator in virtual and remote environments. However, the information that the human user receives from the haptic devices can be distorted by the hardware and software mechanisms used. Existing research focus mostly on providing stability of the haptic devices as humans explore a remote environment and such a focus on stability can typically lead to a trade-off in the information (such as stiffness) pertaining to the physical properties of the remote environment. This project, therefore, seeks to understand how the trade-off between stability and stiffness affect the user's understanding of a remote environment and the ability to perform dexterous tasks in the environment. This knowledge can then be used to improve the functionality of collaborative robotic systems used in many different application contexts, including healthcare, defense, and manufacturing. This project addresses the research issues in providing dexterous manipulations through haptic feedback in telerobotic systems. Dexterous manipulation, however, depends on how well the telerobot is incorporated into the operator's sensorimotor control scheme. Empirical evidence suggests that haptic feedback can lead to improved dexterity. Unfortunately, the addition of haptic feedback, in particular, kinesthetic feedback can introduce dynamics between the leader and follower of the telerobot that can affect both stability and device performance. Concerted research efforts have focused on making these device dynamics transparent to the operator while preserving stability. True transparency, however, represents a theoretical ideal rather than an attainable goal. Therefore, the project team will explore the effect of "reduced transparency" by establishing an empirical understanding of the effect of a telerobot's closed-loop leader/follower dynamics on the perception of a remote environment and performance in tasks requiring knowledge of that environment. Utilizing a generalized telerobotic testbed that features a rigid mechanical, dynamic mechanical, and dynamic electromechanical transmission, the research team will (1) independently and collectively investigate haptic perception of environment stiffness and damping using a mechanical teleoperator with rigid and dynamic leader/follower transmissions, (2) investigate performance in two functional tasks requiring knowledge of environment dynamics using a rigid mechanical teleoperator and an electromechanical teleoperator with closed-loop leader/follower dynamics, and (3) investigate haptic perception and functional task performance with bimanual mechanical and electromechanical teleoperators that have symmetric and asymmetric leader/follower dynamics. Investigations will also consider the utility of visual feedback in addition to haptic sensory feedback. Overall, this research will lead to telerobots that come closer to being embodied by their operators and will lead to improved human-robot collaborative capabilities in HiLTS that will have benefits in many different industries including healthcare, defense, and manufacturing. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.