Inference for the Mean

Princeton University, Princeton NJ

Investigators

Abstract

A key challenge of drawing conclusions about a population from a sample of individuals is how to accurately describe the uncertainty about the population average because one only observes a sample, rather than the whole population. The standard approach relies on approximating the distribution of the sample average by a bell-shaped distribution, whose spread can be estimated by the variability of the sample observations. When the underlying population distribution is not bell shaped, this approximation becomes poor in small samples, leading to a wrong representation of the uncertainty about the population average. This proposal will develop a new method for describing the uncertainty that remains accurate even for non-bell-shaped populations. This is done by modelling the potentially non-bell-shaped structure in the construction of the uncertainty estimate. More complicated problems of drawing conclusions can be cast into drawing conclusions about the average of a suitably defined population. The results of this research could thus have many practical applications in the social sciences and improve business and policy decision making. The end results are to increase economic efficiency and speed up economic growth. This proposal seeks to develop an alternative to standard t-statistic-based inference about the mean from a sample of i.i.d. observations that better controls size for moderately heavy-tailed underlying populations. It combines extreme value theory for the k smallest and k largest observations with a normal approximation for the average of the remaining middle n-2k observations. The mean shift is a function of the same tail parameters that govern the extreme value distributions, as well as the realized value of the extreme observations. One thus obtains an approximate parametric model for 2k+1 observation, with the tail parameters as nuisance parameters; one can apply numerical techniques to obtain valid and powerful test in this approximate parametric model. The main theoretical result shows that this approach represents an improvement over existing method. For populations with finite variance but infinite third moment, the new approach induces smaller errors in rejection probability compared to the usual t-statistic, or the percentile-t bootstrap, at least as long as the population is such that extreme value theory provides accurate approximations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.