NRI: FND: Robust Learning of Sequential Motion from Human Demonstrations to Enable Robot-Guided Exercise Training

University Of New Hampshire, Durham NH

Investigators

Abstract

Therapeutic exercises are crucial for healthy living and effective recovery from injury, surgery, disease, or frailty. Physical or occupational therapists are typically responsible for directing therapeutic exercises. There is currently a mismatch between supply and demand for these services. There is a predicted shortage of 26,600 physical therapists nationally by year 2025 and 50,000 occupational therapists by year 2030. Technology-based home programs are rapidly emerging as a way to combat this skilled labor shortage. Technology assisted exercise programs that promote highly structured practice and provide real-time feedback are believed to improve well-being but have yet to be conceived. This project bridges that gap through designing intelligent robots that can take the role of a therapist during therapeutic exercise training. The idea is that a clinician will teach a robot any structured exercise through demonstrations and the robot will then take the role of a coach to teach users and provide quantitative evaluation of performance. For a robot to do that, we need an intelligent algorithm that will allow the therapists to teach a robot any new exercise without actually programming the robot and enable the robot to learn from therapists' demonstrations. This project will develop a novel Learning from demonstration (LfD) framework to realize exercise trainer robots. The core technical challenges of designing a LfD framework for a exercise trainer robot are i) robustly learning sequence of human movements from lay users' demonstrations while accommodating inter- and intra-personal variations and ii) offers a quantitative metric to explain the deviation of a user's trajectory from the demonstrated sequence in a contextually meaningful way. Solving these challenges requires major changes in the way we currently learn motion trajectories (low-level policy learning) and model the relations among trajectories for learning sequential tasks (high-level policy learning). Accordingly, this research will design the entire pipeline of LfD based only on the kinematic and kinetic variables of motion. The core of this LfD framework is a phase space model (PSM) for learning task trajectories. The PSM leverages dynamic system theories to analyze motion variables to segment a task trajectory and build a parametric representation that is robust against spatio-temporal variations. The compact parameter set that PSM generates are used by a graphical model to learn the high-level policy underlying the demonstrated task while leveraging the typical anatomical constraints of human limbs. The same parameter set is used to design a quantitative metric to evaluate the learning outcome. The project will evaluate the fidelity of a co-robot exercise trainer powered by this LfD framework to teach upper extremity exercises in a series of user studies. An ABB YuMi robot will be used as the test platform. The demonstration data will be collected from inertial measurement units (IMUs) worn by student-therapists on the hand, forearm, upper arm and torso. The robot will demonstrate learned exercises to older adult participants (OA), who then will perform the exercises by mirroring the robot. During the training phase, the OAs will also be wearing IMUs so that their performance can be assessed with respect to the original demonstration from the therapist. The fidelity of movement transmission will be tested from the therapist, to the robot, to the patient with a high-speed, 3D motion capture video system which is the gold standard for kinematic analysis. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.