Mobile Mid-Air Interactive Systems and Design Workflows for Creative 3D Shape Modeling

Purdue University, West Lafayette IN

Investigators

Abstract

Currently 3D modeling tools for product design and manufacturing are the purview of trained engineers and professional artists. Everyone has ideas but only a select few can bring them to reality. Although touch enabled devices such as tablets and smartphones are ubiquitous, the direct manipulation of content is inherently two-dimensional, and the plethora of tools commonly used today grew out of design metaphors based on windows, icons, menus, and pointers. They require extensive training, and inhibit the user's ability to create, manipulate, and modify virtual shapes in a straightforward manner. Research on mid-air interactions has shown that our bodies and their perceptually guided motions through the world do much of the work required to achieve our creative goals. This perceptual process, if carefully integrated into 3D modeling, can be instrumental in eliminating the barrier-to-entry for common users motivated to externalize their creative ideas. This work aims at harnessing the potential of mobile interactions in mid-air to cater to the creative processes involved in design. In doing so, it takes an important step towards the computer-as-a-partner paradigm that transfers our cognition to the virtual environment through hand-held mobile devices. The integration of a variety of sensors into smartphones provides new affordances for designing interactive processes towards creative on-the-fly shape modeling processes. However, existing research on mobile interactions has focused on 3D object manipulation, virtual scene navigation, and scientific visualization. The work will develop a framework for direct 3D shape conceptualization using mobile interactions. To do so, this work will leverage the unique advantage of using smartphones as an extension to our mind and body, to spatially create 2D and 3D sketches, and create, manipulate and modify virtual 3D objects. The ensuing simplicity of spatial interactions for easy, direct shape-modeling operations will transform our bodily motions and rotations of the phone and contextual sketches on the devices to desired configurations and geometries. By combining multi-touch interactions and device sensors, the work aims to leverage embodied cognition towards spatial interaction and develop new metaphors for spatial control, and develop processes for shape modeling to explore a new design space of intuitive interactions. This will be achieved by developing process flows for constrained and free-form 3D shapes, spatial configuration of 3D objects, and creative 3D compositions with shapes. These development cycles will be driven by rigorous user evaluations and iterations to derive the guidelines for design process flow and user experiences.