EAGER: SaTC: SAVED: Secure Audio and Video Data from Deepfake Attacks Leveraging Environmental Fingerprints

Suny At Binghamton, Binghamton NY

Investigators

Abstract

The fast development of artificial intelligence (AI) and machine learning algorithms is escalating the technology that empowers the ability to distort reality. It has taken an exponential leap forward to deepfake attacks, which create audio and video of real people saying and doing things they never said or did. It is ever more realistic and increasingly resistant to detection. Deepfaked video, audio, or photos published on social media platforms are highly disturbing and able to mislead the public, raising further challenges in policy, technology, social, and legal aspects. Today's deepfake tools allow people to become anyone, from Elon Musk to Eminem, during a video chat. Recent deepfake video attacks on some public scenarios have raised more concerns. Disinformation may actually cause a disturbance in our society and ruin the foundation of trust. Government agencies like the U.S. Defense Advanced Research Projects Agency (DARPA) are concerned about losing the war against deepfake attacks that use the popular machine learning technique to automatically incorporate artificial components into existing video streams. The detailed technical routines and countermeasures against deepfake attacks have not been well investigated, leaving alone a potentially effective approach to tackle the emerging threats online in real-time. This project introduces a novel solution to secure audio and video data streams against deepfake attacks. Instead of engaging in the endless AI arm races that fight fire with fire, where new machine learning algorithms keep making fake audio and video more real, this project tackles the challenging problem out of the box based on a key observation. Every audio or video stream has unique environmental fingerprints, e.g. the Electrical Network Frequency (ENF) signals, embedded when it was generated. The environmental fingerprints are random signals, which are unique, unpredictable, and unrepeatable. This project will investigate three typical application scenarios: (1) an accurate detection of deepfaked AVS data uploaded on the Internet, like social media posts; (2) an instant and accurate detection of false AVS injection attacks against online, real-time applications, like teleconferencing; and (3) a lightweight but robust version that fits on the Internet of Video Things applications, like smart public safety surveillance, which requires instant decision-making at the network edge. In addition, this project will gain deeper insights into the characteristics of the environmental fingerprints taking an information theory approach. The success of this research will deliver a disruptive technology that enables the ultimate win of the battle against the deepfake attacks. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.