NSF Convergence Accelerator Track H: Smart Wearables for Expanding Workplace Access for People with Blindness and Low Vision

New York University Medical Center, New York NY

Investigators

Abstract

There are 285 million people with blindness and low vision (pBLV) worldwide. One dramatic correlate of vision loss is unemployment, with rates as high as 81% in urban environments. Employment disparities stem, at least in large part, from difficulties with transportation to and from work and wayfinding within the workplace itself. This project takes a fundamental step in addressing the employment challenges of pBLV through a recently developed, powerful assistive-navigation platform, the Visually Impaired Smart Service System for Spatial Intelligence and Onboard Navigation (VIS4ION). VIS4ION is a discreet, instrumented backpack with an array of miniaturized sensors integrated into backstraps that connect to an embedded system for computational analysis. Real-time feedback is provided through a binaural bone conduction headset and an optional reconfigured waist strap turned haptic interface. The platform provides valuable micro-services for pBLV, including real-time navigation, scene analysis, and obstacle avoidance, all of which could transform workplace experiences for pBLV. To realize the goals of the project, a transdisciplinary team brings expertise across engineering (computer vision, haptics, sensors, and wireless), medicine (neurology and rehabilitation), and management (organizational change and behavior). These academic disciplines are partnered with a start-up, Tactile Navigation Tools (TNT), which is commercializing the technology, larger corporate firms in the space, a BLV-focused non-profit, Lighthouse Guild, and key government agencies. The development and deployment of advanced, connected wearables in workplace environments faces significant technical, business, social, and logistical challenges. This translational project seeks to overcome these challenges by completing four inter-connected tasks: Task 1 performs the first detailed, workplace-oriented study that assesses navigation accuracy of VIS4ION, as measured using additional cameras and inertial measurement units; Task 2 builds on the current VIS4ION interfaces to establish scalable, reproducible, and lower-cost tactile feedback, together with a new audio-based “cognitive assistant” for guidance; Task 3 augments the wearable with high-speed 5G connectivity to provide real-time, powerful cloud-based machine vision processing; Task 4 affords a series of novel studies to assess the workplace experiences of pBLV via our partner, Lighthouse Guild. The assessment includes both “think aloud” recordings and contextualized reflections. The total cost of BLV was recently estimated to be $3 trillion dollars globally. The pathologic link between vision loss and inactivity precipitates debility and unemployment. Mobility losses lead to employment barriers, as pBLV struggle during daily commuting, wayfinding in the workplace, and variable work routines that have been intensified by the pandemic. This proposal aims to increase the safety profile and ease-of-use of a wearable platform toward ‘connected’ dynamic navigation in complex workplace environments. Future goals for scaling the platform include adding additional social capabilities such as face and emotion recognition to substantively improve quality of life in a smarter and more connected, disability-first city landscape. Beyond the envisioned impact on science, engineering, and improving the lives of pBLV, this program will foster formal and informal learning opportunities for graduate, undergraduate, medical, and high school students in clinical and fundamental STEM/STEAM research. These training opportunities will foster multi-pronged and multi-disciplinary team approaches critical to the development of innovative solutions for complex real-world problems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.