CAREER: Augmenting Passive Physical Interfaces into Adaptive Interfaces

Texas A&M Engineering Experiment Station, College Station TX

Investigators

Abstract

The world is full of physical interfaces, such as light switches and walls sockets, that are passive. The integration of smart capabilities such as actuating, sensing, and energy-harvesting is expected to transform passive interfaces into smart adaptive interfaces, that can assist people with disabilities, automate domestic tasks, and power millions of ubiquitous computers. While the complete replacement of all legacy interfaces with smart devices is not currently feasible and may lead to significant environmental impacts, augmenting them is a promising approach to cost-effectively reconfiguring daily interfaces. Yet, many end users, e.g., caretakers, building managers, even individuals who are interested in home-automation, overlook opportunities to innovate unique life patterns and styles via computation. Daily design issues can be difficult to notice because of familiarity based on prior experiences. Even for users with a specific goal, e.g., reducing utility bills, adopting the latest scientific advances in real-life demands expertise because end-user support tools are lacking. This project aims to (1) increase end-users’ awareness about daily design opportunities using three-dimensional (3D) printed augmentations, enabling (2) capturing critical fabrication parameters for complex augmentations with efficiency and accuracy, and (3) fabricating smart augmentations with minimal to no barriers across multiple application domains. Provided with the libraries and toolkits for fabrication and (re)programming, end-users are allowed to tackle known modular-programming strategies, such as mix-and-match, unit-testing, model-view-controller (MVC) model, and fork the existing augmentations to repurpose them. Tackling multifaceted, interdisciplinary approaches across Digital Fabrication, End-user Programming, Deep Learning, Robotics, and Design, this project lays the foundation for a future where every individual creates daily innovations in assistive computing devices, smart homes, and green buildings. This project will center around three main research objectives. Firstly, a theoretical framework will be formulated to conceptualize physical interfaces using a data-driven approach. The investigator will augment and expand massive image dataset of daily objects presented with part level segmentation, by characterizing their interaction properties and user context to help understand the reconfiguration needs, interaction barriers, and adaptive/desired interactions. Secondly, novel computational techniques will be developed to capture physicality: parts of interest (POI), dimensions of interest (DOI), and motions of interest (MOI), which are critical parameters to augment passive interfaces distilling smart capabilities. Lastly, integrating the dataset and techniques, a modular fabrication pipeline will be developed along with a composable construction toolkit and libraries of smart augmentations, establishing capture—customize/fabricate--(re)program paradigm. Through educational objectives, this project advances teacher and student education, training, and learning computing, and promotes multi-disciplinary conversations via the Texas Human-Computer Interaction (TxHCI) seminar series. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.