SHF: Medium: More Reliable Image Networks through Scene-based Specification, Neuro-symbolic Training, and Systematic Specification-driven Testing

University Of Virginia Main Campus, Charlottesville VA

Investigators

Abstract

Deep Neural Networks (DNN) are becoming an essential part of safety-critical autonomous systems, from automobiles to medical devices. Failures in such safety-critical autonomous systems may lead to injury or loss of life. Although there are mature techniques for improving the accuracy of DNNs, such techniques do not provide guarantees that the behavior of a DNN will always be appropriate. Without such guarantees the deployment of DNNs in safety and mission critical systems will be limited or unnecessarily risky. This project seeks to assure the quality of image-based DNNs through the development of techniques that change two fundamental current practices: 1) the specification of desirable DNN properties will be abstracted from the pixel-level to domain entities (e.g., people, cars) to enable reasoning about the correctness of DNN behaviors, and 2) the application of those properties will pervade the DNN development process so that the resulting DNNs behave in accordance with those properties. If successful, the research will improve assurance of systems that include DNNs and, thereby, the safety of the public. Modern image Deep Neural Networks can be extremely complex accepting high-resolution images and processing them through many dozens of layers with tens of millions of parameters to compute outputs. Methods of assessing and improving the statistical accuracy of computed outputs relative to labeled training data are in regular use, but such methods provide no guarantees that the behavior of the DNN will be appropriate, especially on unusual or rare inputs. This project seeks to establish the foundations, algorithms and engineering advances for a new approach to developing image-based DNNs with behavior guarantees. The project shifts the direction from prior research that has focused on reasoning about limited forms of DNN correctness at the pixel level, such as local robustness, and instead aims to enable the specification of higher-level properties that abstract from pixel-level variation to describe equivalence classes of behavior and then to incorporate such specifications through the training, testing, and deployment of DNNs. The project activities include developing: 1) a high-level symbolic method for specifying necessary correctness properties of pixel-based DNNs; 2) methods to incorporate such specifications into the training of DNNs so as to guarantee their specification conformance; and 3) methods to assess and improve training, test, and validation sets to ensure that they adequately represent important, but rare, inputs and thereby enable DNNs to generalize to such inputs. Collectively, this work will establish the first high-level approach to specifying the intended behavior of image DNNs and, if successful, the project will provide a foundation for building more reliable DNN-enabled systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.