SHF: Small: A Chip of Happiness: Device-to-System Developments of Affective Computing for Human-in-the-loop Computer System

Northwestern University, Evanston IL

Investigators

Abstract

While the recent technology advancements in artificial intelligence (AI) and low-power wearable electronics have created tremendous improvements to people’s lives, there is still a missing element in existing computer systems, namely, a deep cooperation between user’s personal feeling and the hardware operation of computing devices. As current developments of AI technology are trending towards “human-centric computing”, it is time to reshape the role of computing devices by bringing human’s perception into the loop of the operations. Recently, research on so-called “affective computing” or emotional AI has shown a promising new computing paradigm in which the knowledge of users’ affects, e.g. emotion, are utilized to significantly enhance the quality of service to the users. Unfortunately, as of today, the support and engagement to human’s real-time feelings at the hardware level is very little. This project aims at developing a new class of affective-computing hardware technology where human affects, e.g. mood, emotions, etc., are being real-time tracked by advanced microelectronic devices and further incorporated into the operations of modern computing systems, leading to unprecedented support to people’s daily activities and enhanced efficiency of computing devices. By linking the advanced computing hardware with users’ real-time affects, a new generation of intelligent human-machine interface can be created allowing human to stay at the center of modern wearable electronic devices. This project will create broad impacts to human services such as online business, social media, e-learning, healthcare, etc. By creating advanced lectures, workshops and seminars, significant educational and training opportunities will also be delivered from this project to college students and the broader audience. This project will take a big step towards closing the gap between human’s real-time feelings and operation of modern wearable computing devices. Cross-layer methodology and techniques will be developed to enable affective computing at the hardware level, ranging from design of microelectronic devices to data management, from advanced computing models to system-level software and hardware integration. More specifically, at the device level, leveraging the recent boom of silicon-based neural-processor techniques (novel "happiness accelerators") will be developed enabling real-time affect inference on highly constrained low-power wearable devices. At the algorithm level, this project will develop advanced machine-learning models to not only improve the accuracy of affect classification and related cognition tasks but also enable efficient deployment of affective computing at ultra-low-power edge devices. Furthermore, at the architecture level, this project will develop a series of novel affect-based memory, data and power management techniques to enhance the energy efficiency of modern computing devices. As a demonstration, this project will use fabricated silicon chips and compact wearable devices to create advanced affect-driven computing system providing a new level of human assistance for real-life applications such as online services, Augmented/Virtual Reality(AR/VR) and classroom learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.