Elements: Data: Integrating Human and Machine for Post-Disaster Visual Data Analytics: A Modern Media-Oriented Approach

Purdue University, West Lafayette IN

Investigators

Abstract

This project creates a science-oriented visual data service that facilitates the query of datasets based on visual content. The approach allows a user to search for data based on visual similarity, even in cases where a term for the failure or observation does not yet have a scientific name. The visual analysis data and application services will be deployed on a cloud-based platform. The results will produce a framework enabling access to and analysis of a large amount of imagery from diverse sources. The research team creates VISER (Visual Structural Expertise Replicator), which will serve as a comprehensive cloud-based data analytics service and will facilitate the use of and integrate data and applications most needed by the user. The framework will implement two novel concepts: data-as-a-service and applications-as-a-service, which will bring data and applications to the user without the need to configure software systems or packages. The approach also employs artificial intelligence to interpret the contents of the images. VISER will use convolutional neural networks (CNNs) to train custom classifiers for new categories. Three applications will be developed and deployed within VISER: App1 will extract relevant visual context, App2 will facilitate similarity-based visual searching (through the use of a Siamese CNN), and App3 will help perform automatic extraction of pre-event/pre-disaster images based on Google Street View. The application of these tools would advance both the science of automated pattern recognition and of more effective construction techniques. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.