INTEGRATED SENSING: Distributed Architecture for a Voluble Intelligent Environment

University Of Rochester, Rochester NY

Investigators

Abstract

Abstract The goal of the proposed research is to implement a conversational intelligent environment containing a distributed array of active cameras and other sensors, and communicating with users via natural dialogue, augmented where useful with imagery and pointing devices. This artificial spatial intelligence, termed a Voluble Intelligent Environment or VIE, will actively obtain information about, and provide assistance within a real physical domain. The project requires grounding natural language understanding in machine perception. A major difficulty is that outputs supplied by current perceptual algorithms are too low-level to serve as primitives for traditional understanding and reasoning systems. To address this, the PIs will integrate the results of machine vision and other perceptual functions into mid-level representations termed "perscripts", which represent items of perceptual information of the sort that would typically be described by a phrase or simple sentence. The project also requires advances in distributed systems technology to permit the various perceptual, computational, and communication components to be efficiently and flexibly configured. The integration framework will be based on a prototype distributed resource interface, InterWeave, developed at the University of Rochester. Interweave provides an efficient implementation of a network-wide global name/address space that directly supports the proposed approach to grounding language in perception. The work will be validated in a ``Household Assistant'' application that provides occupants of a ``smart house'' with information useful in activities of daily living (such as ``where did I leave my glasses?'' or ``did I take my meds?''). This application is being developed in conjunction with the University of Rochester Center for Future Health. It addresses needs of the elderly and the disabled, and has the potential to have a significant economic impact as part of an overall system that unloads caregivers for this population.