CAREER: Video-Based Computer Interfaces for People with Severe Disabilities

Trustees Of Boston University, Boston

Investigators

Abstract

People who are quadriplegic and nonverbal, for example from cerebral palsy or traumatic brain injury, often have great difficulties communicating their desires, thoughts, and needs. A video-based human-computer interface system will be developed to provide such people access to computers. The interface system will detect and track their limited voluntary motions to help them communicate with family, friends, and care providers. A set of cameras with pan, tilt, and zoom mechanisms will be pointed at the computer user. Computer vision algorithms will recognize the movements of a finger, foot, or facial feature in real time and interpret them as a communication of the computer user. Augmentative communication software will be developed that uses these interpretations to provide access to commercially available or custom-built application software. The system will be trained and tested by several non-speaking children with cerebral palsy and other serious disabilities. This research is expected to advance the state-of-the-art in the fields of computer vision and assistive technology with new video interpretation algorithms and innovative real-time system design. It will help people with severe disabilities gain access to a computer and thereby obtain a tool for communication, which in turn will enable them to actively acquire knowledge, to partake in increased recreational activities, to surf the internet, and to use computer-controlled technologies such as automated wheelchairs.