I-Corps: Semio: Customer discovery for a software platform for human-robotic interactions

University Of Southern California, Los Angeles CA

Investigators

Abstract

This I-Corps team is developing a standardized software platform to allow humans to interact with and use robots through natural communication, such as speech and body language. The proposed "Semio" software platform allows robot developers to create (through developer tools) and deploy (to an app store) speech/gesture-based applications to be executed by robots, and allows non-expert robot users to access (from the app store) and use (through natural communication and smart devices) those robot applications. The proposed software enables a robot to automatically recognize human speech and body language, make decisions about how to respond (specified by the developer for the application), and act based on those decisions while also providing its own speech and body language. This natural communication interface increases the usability of robots by novice users. The Semio software platform involves novel algorithms and robust code bases that enable a robotic hardware platform to operate "out of the box" based on a set of core social behaviors that serve as building blocks of face-to-face conversational interaction; these behaviors include speech (what is said), eye gaze (where attention is focused), gestures (face, head, body, arm, and hand movements), turn-taking (when to talk and when to listen), and interpersonal spacing (where to stand for to have a conversation). The target market for the proposed speech/gesture-based software is personal robotics, though the team is also exploring other markets, such as smart devices (smart phones, GPS, and the Internet of Things), gaming, transportation (cars and self-driving cars), sensors (Microsoft Kinect and Intel RealSense), and general robotics (industrial, military, and commercial robotics). The team is currently targeting customers who are hardware and software developers of these interactive technologies. Developers may be both expert and non-expert, and may be members of industry, the Maker community, hobbyists, or academia. Through repeated customer interviews, this team will continue to integrate customer feedback into the demo app to better improve the product content and performance to meet the identified needs of customers. The software platform will also be connected to a hardware robot platform to begin proof-of-concept demonstrations to and connection-building with prospective customers in the robotics market.