From Approximate to Exact Designs with Applications to Big Data

University Of Tennessee Knoxville, Knoxville TN

Investigators

Abstract

Design of experiments is an integral part of the scientific process in many areas of research with a direct impact on society, such as the biological sciences, the health sciences, the social sciences, engineering, marketing, and education. A well-chosen design facilitates the collection of data that, at a minimum cost, maximizes the information for the scientific questions of interest. Many scientific studies allow for repeated use of conceptual units, so that developing tools for optimal design for these problems has great potential impact. Particularly in the realm of big data, there is much room for improvement of existing methods for design of experiments, and the tools and concepts under development in this research project have potential to lead to significant gain of information without increasing computational cost. Results from the project will be made available to researchers in other areas through easy-to-use software that implements the algorithms to be developed. Graduate students will be trained to become researchers in design of experiments. This project aims to result in a major leap forward in understanding and knowledge of optimal design of experiments. Recent work in the field has had a significant impact on the advancement of optimal crossover designs and designs for interference models for arbitrarily given covariance structures and design size configurations. However, these results have for the most part been limited to approximate designs for relatively simple models. While these results are arguably important in their own right, this project will extend methods and tools to achieve the ultimate goal of deriving exact designs for a wider spectrum of practical models. The results will be a much needed addition to our collective design toolbox. Most importantly, this project will go beyond the territory of design and apply the tools and ideas from design of experiments to subsampling problems emerging in big data with both statistical and machine learning methods under consideration. Preliminary results indicate that this is an opportune time to make these challenging but critical steps.