SCH: INT: Collaborative Research: Replicating Clinic Physical Therapy at Home: Touch, Depth, and Epidermal Electronics in an Interactive Avatar System

University Of California-Santa Cruz, Santa Cruz CA

Investigators

Abstract

Physical therapy is often hampered by lack of access to therapists, and lack of adherence to home therapy regimens. This research develops a physical therapy assistance system for home use, with emphasis on stroke rehabilitation. As a person exercises, inexpensive cameras observe color and depth, and unobtrusive tattoo sensors monitor detailed muscle activity. The 3D movement trajectory is derived and compared against the exercise done with an expert therapist. The patient watches a screen avatar where arrows and color coding guide the patient to move correctly. In addition to advancing fields such as movement tracking, skin sensors, and assistive systems, the project has the potential for broad impact by attracting women and under-represented minorities to engineering through health-related engineering coursework and projects, and because home physical therapy assistance can especially help rural and under-served populations. This project uses bio-electronics, computer vision, computer gaming, high-dimensional machine learning, and human factors to develop a home physical therapy assistance system. During home exercises, patient kinematics and physiology are monitored with a Kinect color/depth camera and wireless epidermal electronics transferable to the skin with a temporary tattoo. The project involves optimization of electrode design and wireless signaling for epidermal electronics to monitor spatiotemporal aspects of muscle recruitment, hand and body pose estimation and tracking algorithms that are robust to rapid motion and occlusions, and development of machine learning and avatar rendering algorithms for multi-modal sensor fusion and expert-trained optimal control guidance logic, for both cloud and local usage. The system aims to provide real-time feedback to make home sessions as effective as office visits with an expert therapist, reducing the time and money required for full recovery.