CRII: CHS: TongueWrite: An efficient tongue-based text-entry method using Multifunctional intraORal Assistive technology (MORA)

Texas A&M Engineering Experiment Station, College Station TX

Investigators

Abstract

Individuals with high-level of paralysis can enhance their independence and quality-of-life by adding or replacing modes of input/control with the individuals' available voluntary motions. The input modes of many current assistive technologies, however, are limited. They can mostly interface with specific tasks such as computer mouse control, wheelchair driving, or text entry, but not for multiple of these purposes. Even if these technologies interfaces with multiple modalities, they are not very user-friendly, and a disabled individual may choose not to use the inefficient modality or switch from one technology to another for different tasks. One of the most powerful candidates in the human body to interact with assistive technologies is the tongue. The human tongue is able to harness voluntary movements above the neck in individuals with severe disabilities. Existing tongue-based assistive technologies perform cursor navigation and wheelchair driving with the same level of comfort as using a finger, but their performance in text entry is not good. This project will explore how the tongue-based interfaces can improve the performance of text entry (apart from tasks such as navigation and other discrete commands) by developing intuitive and easy-to-learn tongue commands. Ultimately, this multimodality will enhance both the independence of those with limited hand motions (e.g., Tetraplegia, stroke, Parkinson's disease, and age-related neurological disorders) and the quality-of-life of their caregivers. For text entry, a user should be able to unambiguously specify the position and sequence of keys that should be considered as input. In order to use the human tongue for text entry, the associated algorithm should be able to handle the continuous tongue motion (involving multiple degrees of freedom), identify the movements related to text entry, and detect and ignore “normal” tongue movements such as swallowing saliva, etc. To address these challenges, this project will model the performance of users on tongue-based commands by the number of commands to be carried out and investigate efficient text entry methods along with various modes/mechanisms of text entry (such as a touchscreen). The research team has developed a new tongue-based assistive technology, Multifunctional intraORal Assistive technology (MORA), which employs advanced sensor technology and a smart data fusion algorithm while taking advantage of the power of the tongue. Designed as a customized wireless headset/retainer, MORA uses an array of four three-axis magnetic sensors located near user’s cheek / upper pallet. MORA can differentiate user-defined tongue movements from other natural tongue movements, especially those involved in speaking and swallowing without any tracer attachment. To model the usability of the tongue commands using MORA, the research team will first evaluate tongue performance by the number of commands: five, seven, and nine. To evaluate tongue performance by the number of commands and their learning effects, the research team will implement a random command task and a Fitts' law-based multidirectional tapping task. Then, the team will explore various text entry methods (such as H4-Writer, OPTI II, Hex-O-Spell, Metropolis II, EdgeWrite, Multitap) based on three mechanisms: multi-stroke, touchscreen, and gesture recognition, to investigate efficient text entry methods based on discrete tongue commands. MORA will be introduced to users through focus groups in medical facilities such as The Texas Brain and Spine Institute, Bryan, Texas, and the Center for Excellence in Aging Services and Long-Term Care, in the University of Texas at Austin. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.