NSF Convergence Accelerator Track H: Addressing the Fragmented Information Access Problem - A Community-Driven, AI-Powered Platform for Multimodal Content Creation

Saint Louis University, Saint Louis MO

Investigators

Abstract

Imagine trying to learn about a graphic without seeing it. Lack of access to visual content is one of the most pressing, persistent challenges facing individuals with blindness and low vision (BLV). In the best case scenario, an individual with BLV accesses graphics through screen readers, touch-based mediums, or other tools that use sound and touch, but many of these solutions require another sighted individual to create the alternative formats. This lack of independence is the result of a fragmented ecosystem, which leads to troubling outcomes in educational attainment, full-time employment, and independence. This project converges partners from across the siloed ecosystem to change the way visual content is created and consumed, making it easy to use for all people, independent of visual status. Through partnerships from industry leaders in assistive technology, content providers, data visualization tools in education, and authentic testing sites, the team will develop Inclusio, an end-to-end software solution that enables anyone to find and create content and to consume it in their preferred mediums. Three key tenets drive Inclusio: a content platform, automated conversion capabilities using artificial intelligence (AI), and a user-friendly authoring suite. The core idea behind Inclusio is that the platform can take in multiple input data formats and output it across both mainstream platforms (e.g., mobile and web applications) and the most highly used assistive platforms (e.g., braille and tactile embossers, multimodal touchpads, and refreshable tactile displays). In Phase 2, the team will build out the core infrastructure of the content platform to host the most used content by end-users, making it interoperable across multiple output platforms. The team will also build out AI and automation capabilities to reduce the time required to create the most highly requested graphics, while enabling human-in-the-loop creation and content optimization, rooted in best practices and standards, through an authoring suite. Throughout every stage of design and development, the project will employ iterative, rapid user testing across the team and partners, reaching more than 200 individuals with BLV, teachers of students with visual impairments, and leading content providers. At the end of Phase 2, the team will launch Inclusio, pioneering a new technology solution. Inclusio is more than a cutting-edge tool; it will reshape an entire sector of end-users, creating a future where all content seamlessly adapts to individual needs. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.