Best Practices for Using Data Generated by AI or Machine Learning

Yale University, New Haven CT

Investigators

Abstract

Empirical analysts now routinely generate new data by deploying artificial intelligence (AI) or machine learning (ML) algorithms on large, unstructured data sets. Examples include quantifying sentiment or uncertainty in news text using large language models or natural language processing methods; measuring product characteristics from review text and product images on online platforms; or imputing missing variables from demographic information. In standard practice, AI- or ML-generated data are treated as if they were regular numerical data for the purposes of data analysis. However, this standard approach can introduce bias in parameter estimates and lead to invalid conclusions. This project develops new econometric methods and statistical theory to inform best practice for empirical researchers using these data. This research project improves the quality of data analysis performed by businesses, non-profits, and government organizations. The interdisciplinary nature of the research helps to forge connections between academia, policy makers, and industry. The research improves the validity of empirical research using AI- and ML-generated data. The projects contribute novel econometric methods for working with data generated by AI and ML algorithms that correct the bias and inference problems present in current empirical practice. The methods are rigorously justified with new statistical theory for AI- and ML generated data. A key contribution is the development of novel asymptotic frameworks that are appropriate for modern use cases where AI and ML algorithms are deployed on massive data sets. Additionally, the projects develop an effciency theory for working with AI- and ML-generated data. General methods and frameworks are developed, along with specialized approaches for important business and economic contexts. These include demand estimation using text and image data (relevant for pricing decisions on online platforms) and vector autoregressions with AI- and ML-generated components (relevant for fiscal and monetary policy analysis). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.