SHF: Small: Bitstream Processing

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

Embedded computing systems are becoming very common in today's world, ranging from wearable devices, to in-home smart speakers, to autonomous appliances and vehicles. Many of the computational tasks that these systems implement require very high performance hardware for tasks like audio voice recognition, visual object recognition, path optimization, and autonomous control. Historically, such high-performance hardware relied on binary fixed-point algorithms deployed on low-power microcontrollers or digital signal processors. However, the sensing and control interfaces themselves do not use binary number representations, but instead use bitstreams, which encode numeric input and output values using the density of ones over time. Conventional computing substrates requires conversion of both inputs and outputs to interface with physical systems that utilize bitstreams. This project is developing novel, biologically-inspired approaches for directly operating on data represented in the native bitstream format. Compute hardware that directly operates on these bitstreams can be seamlessly integrated into systems that sense the real world, process sensory data, and issue control commands based on the processed data as well as learned actions based on rewards from the environment. The capability of these new approaches will be demonstrated through two experimental platforms: a very low power speech recognition system that operates on bitstream audio data, and an autonomous airborne vehicle that learns to navigate its environment. This research has broad industry- and economy-wide impact since it will lead to the discovery and realization of novel, powerful, and energy-efficient approaches for implementing power- and energy-constrained embedded computing systems. This research advocates development of novel, biologically-inspired approaches for processing data represented as bitstreams. Bitstreams, which encode numeric values using density of ones (unary) or ones and zeroes (binary) are a natural representation for data sensed from the environment (input) as well as robotic control (output), and can be inexpensively generated using low-cost, yet accurate, sigma-delta modulators. The initial phase of this research project focuses on developing the theoretical and algorithmic underpinnings for visual, auditory, and inertial sensory processing, including feature extraction, bandpass filtering, perspective and coordinate transforms, linear optimization, and memory formation, which are grounded in principles from the speech processing, computer vision, spiking neural networks, reinforcement learning, and signal processing domains. The novel sensory processing capabilities are then deployed in two experimental platforms: first, an ultra-low power acoustic model for speech recognition that can demonstrate the suitability of bitstream processing for feature extraction and sequence learning via long short-term memory. Next, bitstream sensory processing technology is coupled directly to a control system that enables an unmanned aerial vehicle to navigate in a controlled indoor environment while learning, with increasing efficiency, to identify and target sources of rewards. Both demonstration platforms rely on concepts from biological spiking neural networks, stochastic computing for arithmetic operations, as well as oversampled sigma-delta modulation theory for data representation and signal processing tasks, and provide unprecedented levels of efficiency in terms of energy consumption, compute density, and autonomous operation. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.