CIF: Small: End to End Security-Oriented Optimization of Image Acquisition Pipelines

New York University, New York NY

Investigators

Abstract

Ensuring integrity of digital images is one of the most challenging problems of our times. Increasing capabilities of digital media editing software has spawned a torrent of shocking manipulation examples, including the infamous DeepFakes which has been deemed a looming challenge for privacy, democracy, and national security. Many solutions proposed to-date have fallen short and can be rendered ineffective with very simple post-processing. In particular, strong compression applied by social networks and photo sharing services render existing authentication protocols unreliable. Moreover, increasing adoption of deep learning and computational photography in imaging processors in digital cameras creates new challenges even in native photo authentication. This project will use modern machine learning techniques to pursue security-oriented design of image acquisition and distribution workflows to ensure that image integrity and provenance can be assured, thereby addressing an emerging problem in this social-media driven world. This project seeks to tackle: 1) optimization of the imaging pipeline to facilitate reliable forensic analysis in the most challenging conditions; 2) design of training protocols that generalize to various authentication problems; 3) optimization of the entire acquisition and distribution workflow, including both the imaging processor and lossy compression. Preliminary experiments indicate that by exploiting feedback from post-distribution forensic analysis, the imaging pipeline can be modified to facilitate content authentication. A neural imaging pipeline can learn to introduce imperceptible artifacts, akin to digital watermarks, which significantly increase manipulation detection accuracy, from 45% to over 90%. Further gains will result from the explorations on this project along the thrusts noted above. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.