RI:Small: Leveraging Human Manipulation Skills to Advance Near Contact Robotic Grasping and In-Hand Stabilization

Oregon State University, Corvallis OR

Investigators

Abstract

People are incredibly good at picking up and manipulating objects. Robots, however, still struggle with reliably picking up objects. The goal of this project is to improve robotic grasping by observing how humans respond to challenging grasping situations. The project's central idea is that humans strive for grasp success rather than trying to find the best possible grasp. Humans use a grasp that is likely to work, even if they have misjudged where the object is, what shape it is, how heavy it is, etc. Finding what these strategies are (in a form a robot can use) will improve robotic grasping in the "real world". The technical challenge addressed in this project is how to go from human demonstrations all the way to robot hand control strategies. First, human user study participants will demonstrate grasping with actual robotic hands by "puppeteering" the hands through a structured set of grasping and manipulation tasks. This demonstration will then be reconstructed in a physics simulator. This provides a rich set of data in an environment - grounded in physical reality - that algorithms can learn in. Finally, the data is put in a novel form that is ideally suited to capturing what happens as the hand comes into contact with the object in a form suitable for machine learning. The result is a learned set of near-contact robotic controllers that can be incorporated into existing grasp planning algorithms in order to improve robotic grasping. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.