RI: Medium: Robust Models and Physical Interactions for Managing Specialty Crops

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

This grant supports research into developing robots that can reliably and robustly interact with plants to assist in the managing and harvesting of specialty crops. Managing specialty crops, such as fruits and nuts, requires a considerable amount of dexterity and skill. To maximize yields, these crops need to be monitored and pruned regularly throughout the year before being harvested. These tasks are labor intensive and not amenable to automation via the traditional mechanization methods used for broadacre crops such as corn and soybeans. An increasing labor shortage is thus threatening the US specialty crop industry. Intelligent automation presents a promising approach to addressing this issue, with robots performing the uncomfortable and dangerous farm work. However, performing such complex tasks quickly and reliably in unstructured environments is beyond the capabilities of current robots. In this project, a team of researchers will develop a framework for robots to reliably model and manipulate specialty crops in a robust manner. New perception algorithms will allow robots to use vision and touch to identify the different parts of the plants and their connections. New controllers and planning algorithms will allow robots to reach deep into the canopies of plants to reliably prune, push aside, or harvest specific parts of the plants. The developed methods will not only provide support for automating the farming of specialty crops, but also techniques for creating more accurate models of these plants for long-term monitoring and phenotyping. The team of researchers will address the challenges of managing and harvesting specialty crops by advancing the state of the art in modeling and manipulating flexible objects. The researchers will develop perception and multi-layer modeling techniques to capture the scenes’ 3D geometry and physical properties. The resulting models will capture the physical connections within the scenes as well as model the uncertainty for these high-occlusion environments. The team will create algorithms for planning and executing safe interactions with the cluttered and constrained environments. The research will also include the development of interactive perception methods for improving the scene models based on experiences from interacting with the specialty crops. Research contributions to perception, planning, and modeling will all be extensively evaluated on real robots both in the lab and in the field. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.