Semantics-Preserving Virtualization: A Computing System Framework to Run Any Screen Reader on Any Device with Easy Customization

Univ Of North Carolina Chapel Hill, Chapel Hill NC

Investigators

Linked publications & trials

Abstract

? DESCRIPTION (provided by applicant): Computer users increasingly have multiple devices, including desktops, phones, tablets, and laptops, and often use applications designed for different operating systems (OSes). For instance, a Mac user may access a cloud-based Windows remote desktop to run an application required for her job. The trend toward multiple devices and OSes creates a burden for users with visual impairments: these users' productivity depends on features of a speci?c screen reader, and these readers are tied to a speci?c OS. The only current option is to run a different screen reader on each platform-harming productivity incurring additional expense. The goal of the proposed work is to decouple unmodi?ed, binary screen readers from the original OS platform, giving users with visual impairments a seamless experience when accessing assorted applications running on any platform and any device without having to switch their preferred screen reader. This proposal is built around the insight that this point of similarity is the semantics of application User In- terfaces (UIs); applications on every OS consist of similar widgets such as buttons, drop-down menus, and text ?elds. The primary obstacle to porting screen readers across platforms are the degree to which accessibility APIs are idiosyncratic, cumbersome, and varied across platforms. The novel concept is semantics-preserving virtualization, which is designed around a simple model of application UIs that is both minimal and suf?ciently expressive to represent any application's UI. This generic UI model is used to drive cross-platform translations, from, say Windows' UI model into the generic model, and then to the Mac UI model such that Apple's native screen reader, VoiceOver, can read a Windows application as if it were running on the Mac. We will integrate these translations into both remote desktop clients and OS emulators to decouple screen readers from the OS. The speci?c aims of this project include: 1) Developing a computing systems framework to virtualize user interfaces at the semantic level, 2) Developing multiple proof-of-concept implementations where a screen reader designed for one OS is used to make applications running on a different OS accessible, 4) Exploring a simple programming model for writing customizations and extensions to assistive technologies, and 3) Validating the usability of the system with end-user case studies. This project will empower users with visual impairments to use any device without a disproportionate loss of productivity-ultimately allowing them to lead more independent lives with improved employment prospects. Thus, this project advances the vision of Universal Accessibility, that anyone should be able to reap the bene?ts of the information age with any computing device, unconstrained by any disability. In addition to the practical impact for users, the project will have a research impacton the ?eld of Computer Accessibility and Operating Systems by developing a novel model for application user interfaces and removing systems-level obstacles that needlessly delay advances in accessibility software.