I-Corps: Zero-UI: Translation of Purdue Gesture-based Creative Interaction Technologies into the Real World

Purdue University, West Lafayette IN

Investigators

Abstract

During the last 30 years, use of computers to initially make 2D drawings and sketches, and then to design 3D shapes, have lead to deep research and plethora of geometric and graphic design tools. An analysis of past work reveals that applications are limited by (1) use of 2D input devices such as mouse, keyboard, touch screen, etc. (2) and the limitations of corresponding procedural programs and user interfaces that map user input to computational procedures, and (3) inability to sense, feel and interact with these virtual data and shapes. The current state of unnatural user interactions severely limits the creative expression and manipulation of shapes. It urges us to develop new forms of human machine interaction to unleash the creativity of the mind's eye. The team's vision is to enable a new class of natural user interaction driven shape creation and manipulation paradigms. The team's approach will completely eliminate traditional user interfaces since users will be able to directly interact with 3D content using hand and body. This vision is based on a recent set of our discoveries, enabled by prior NSF support on natural sketch-based interfaces and shape analysis that resulted in the foundation for commercialization. The science of natural user interfaces and interactions are at a very early stage of development. Highly-digitized user interfaces using windows, icons, mouse and pointer have resulted in overlooking the role of the user in creative interaction with virtual shapes. The team's work contributes and creates new ways in which users will interact with geometric data spaces. Areas of scientific inquiry we will contribute to include human-computer interaction, machine learning, gesture-based interaction, and ubiquitous computing. Gesture based natural user interfaces (NUI) enabled by 3D sensing cameras are the next frontier in computing, media and entertainment and other verticals. This work can result in new capabilities in edutainment and game design, training technologies, haptic interfaces for disabled, interactive advertising, personal manufacturing and many other applications not yet imagined.