Statistics and Information Theory

Southern Illinois University At Carbondale, Carbondale IL

Investigators

Abstract

This research enhances statistical inference by applying techniques from information theory, makes new developments and gains new insights into many existing methodology. Specifically, this project concentrates on the following topics. (1) In the context of incorporating historical data when analyzing current data, often optimal posterior data analysis uses power priors; however, the suggested value of the optimal power has been called 'highly skeptical' (Ibrahim, et al, 2003, JASA) and when multiple historical data sets are present, no optimal set of powers is known. PI solves this by framing the optimization problem differently and using tools from the duality principles of optimization. For the multiple historical data sets, this project introduces different criteria motivated by different objectives, which will be studied and compared. (2) PI investigates the efficiency of the optimal solutions of (1) under constraints involving divergence between densities and with emphasis on `no loss of information' between input information and output information. (3) PI investigates the Neyman-Pearson testing rule in the context of information theory with emphasis on finding the type I and type II errors. Finding these errors is extremely helpful when only asymptotic tests are available. Results are applied to testing the effect of dose using a logistic regression model with several covariates, which has an asymptotic chi-square distribution with 1 d.f. (4) New methods are developed for variable selection in regression by considering projection from the full model subject to constraints that limit the search to only those which are within a certain 'divergence' from a given model (e.g., intercept only). One of the merits of this procedure is that it is simpler to use and avoids the complicated Bayesian calculations. (5) PI investigates the asymptotic behavior of an I-projection when the constraints are any convex sets C. Haberman (Annals of Statistics, 1984) has considered the asymptotic properties of an I-projection when the constraints are moment equalities; however, those results do not extend directly when other types of constraints are involved (e.g., moment inequality, constraints with divergences, etc.). This research emphasizes on the dual optimization problem and its asymptotic properties, through which a connection is made to the primal problem. Estimation of a functional, including its confidence interval, will also be considered. (6) PI introduces the concept of pseudo-divergence, where a nuisance parameter is replaced by a consistent estimator into the criterion to be optimized. The asymptotic properties of the estimator of the parameter of interest is investigated. (7) PI applies information theoretic approach in multinomial response models where logarithmic penalty functions are commonly used to determine optimal prediction under a given model. This research would enable one to construct models with built-in inequality constraints on parameters, which would be very useful in model selection. The intellectual merit of the project is that using information theoretic techniques new aspects of statistical inference procedures have been developed. In this project new optimal distributions are presented in posterior data analysis and they are shown to be highly efficient, the error rates in hypothesis testing are found for small sample sizes, new strategies for model selection are demonstrated, the asymptotic distribution of the I-projection for general convex sets is derived and the concept of pseudo-divergence is introduced where the asymptotic distribution of the estimator of the parameter of interest is found. The broader impact of the project is that the results are directly useful in many areas including actuarial science, product reliability and manufacturing, Bayesian prior selection, econometrics and finance, health and medicine. The proposed activities involve training of graduate students in statistics as well as providing selected undergraduate students with research experience.