CAREER: Context-Aware Task-Oriented Dexterous Robotic Manipulation

University Of Alabama Tuscaloosa, Tuscaloosa AL

Investigators

Abstract

With the advancement of modern industries, effective manipulation of unfamiliar objects in an assortment of dimensions, shapes, or materials becomes the bottleneck problem in the automation of manufacturing, services, and retail trade. This Faculty Early Career Development (CAREER) project will build innovative technologies to enable a robot to manipulate unfamiliar objects for difficult tasks, e.g., goods packing, drilling, waste sorting, and small part assembly. The advantage of the technologies is the capability of manipulating an object in accordance with the task requirements and the perceived object properties including shapes and materials. This project has great potential to benefit numerous industries by improving productivity, deskilling robot programming, simplifying robot planning, and lowering the technical boundaries. This project will support the national strategy in bringing manufacturing back to the US while benefiting many industries and increasing their economic competitiveness. The project will establish a holistic framework of Context-Aware Task-Oriented Manipulation (CATOM) to address three cornerstone challenges in dexterous manipulation: object affordance perception, dexterity modeling, and manipulation planning and learning. An object affordance refers to actions that match with the physical properties of an object, such as shapes, mass, and friction. The object affordances are obtained from object characteristics measured by multimodal sensors. For this purpose, a knowledge-driven model will be designed, which fuses heterogenous sensing and incorporates human experience. The object affordances will be represented by potential contacts under each grasp taxonomy. A topology-based modeling method will be used to align potential hand postures with the contacts for a proper grasp. With the topology-based modeling, a complex task will be represented as a spatial-temporal sequence of hand topologies and operations. A hybrid learning and planning mechanism will be implemented to deploy hand topologies and perform actions under contextual constraints. The research of these perception, planning, and learning methodologies will advance the knowledge in dexterous robotic manipulation for complex tasks. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.