EAGER: Inferring Mechanical Explanations from Manipulation Demonstrations

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

Robots are fast, accurate, reliable, and tireless, which make them great assets for factories. However, robots lack intuition - at which humans excel. Current robots have a hard time when tasked with imagining ways to manufacture new products. In an effective human-robot collaboration, humans conceive new products and feasible ways to manufacture them, and robots follow human guidelines with excellent precision and reliability. The scenario of interest is where a human operator demonstrates an execution of a task, such as mating two parts, and a robot understands how to replicate it. This proposal is concerned with improving the current robot understanding of those demonstrations. In particular, it is concerned with recovering information that is not directly observable with cameras, such as contacts between parts, and the forcefulness of the motions, both important for the ability of the robot to replicate the demonstration. This proposal aims to automate the inference of contact events and contact forces from noisy kinematic observations of the interaction between two known parts. The central idea is to use trajectory optimization and complementarity based models of contact to project noisy kinematic trajectories into dynamically-sound and environment-compatible motions, contacts, and forces. The problem is naturally under-constrained. There are many possible explanations for a given demonstration, and this proposal will investigate different ways to provide the optimizer with prior information to converge to a "reasonable" explanation. The first phase will conduct experiments instrumented with motion capture and force sensing to capture ground truth, and evaluate the ability of the algorithms to explain it, and overcome degradations such as noise and occlusions. A second phase will focus on un-instrumented scenarios and free-from demonstrations. The technical merit of the proposal will be demonstrated in the context of two high impact applications: automated assembly and shelf-picking/shelf-restocking in a warehouse scenario.