Feedback Control of Visual Appearance With Maximally Sensitive Sensors for Decentralized Event Detection and Security

Washington University, Saint Louis MO

Investigators

Abstract

In many surveillance problems, one would use several c.c.d. (charged coupled device) cameras connected in a network, that would generate a stream of images. The problem is to process the image stream so as to detect a typical event in the scene from the observed recording of the image sequences. Such an event might be quantified, for example, by a sudden movement in the scene or existence of an unfamiliar target. Difficulty in identifying an event is that the 'event characteristic' is hard to quantify and extracted from the complex scene imagery, a process that would typically required to be done in real time. Additional difficulty arises from the fact that a single view of the scene may not be enough to isolate an event - hence the need for multiple view over an interval of time. We propose that every visual stream be quantified and represented locally by an internal dy-namic model that produces its own spatiotemporal sequence. The importance of this internal repre-sentation is that it does not represent the entire scene, or all the events in the scene. Rather, its sole purpose is to amplify specific intruding event anywhere in the scene and to respond as to 'when' and 'where' it has occurred. The design problem that we propose to investigate is to synthesize the internal dynamics so as to respond 'maximally' in presence of an intruding event as opposed to somewhat more routine events. The internal dynamics would be implemented on a processor locally connected to the sensor and in its simplest form would be a directionally selective flow model with inputs from the scene images taken by the camera. The model parameters are tuned to respond to specific events in the scene. In order to be able to synthesize the above described 'maximally sensitive sensor', we subdivide this project into three distinct parts. The first is to introduce 'appearance models' to represent a sequence of images taken by a camera. Such an appearance model results in a suitable data-compression and is particularly useful when a suitable set of appearances have to be isolated and detected in the scene. Our second goal is to introduce a suitable internal representation of the observed spatiotemporal signal using 'flow models'. The proposed flow models have flow velocities that are dependant on the direction of propagation and can be altered by the magnitude and position of the input target events. The flow models can be tuned by feedback to produce maximal activity to selected targets in the scene. Our third goal is to network a distributed set of cameras, together with associated internal models, for distributed detection. The model activity from each camera sensor, confirming detection of a local event, is fused together in order to detect a spatio-temporal global event. The intellectual merits of this project are described as follows. The first is Selective Encoding of the Spatio-Temporal Events in the Scene through Appearance Models. The second is Internal Modelling and Feedback Tuning for Maximal Response. Finally the third is Decentralized Detec-tion and Feedback Control of the Network Structure. Broader impact of the proposal includes interaction between Signal Processing, Sensor Based Control and Sensor Networks. Feedback control for sensor tuning and network reconfiguration are two research areas that this proposal makes the most impact. The project would be carried out by the PI with 2 PhD students at the Center for BioCybernetics and Intelligent Systems and would also provide an interdisciplinary training ground for senior undergraduates from Computer Science, Electrical and Systems Engineering.