ITR: Bayesian Models Linking Web Site Structure and Usage

National Institute Of Statistical Sciences, Durham NC

Investigators

Abstract

Abstract PI: Alan Karr Proposal Number: ITR-0218759 Despite its ubiquity, the World Wide Web is poorly understood. As a consequence, many sites are difficult to navigate, hard to use and have confusing structure, to the extent that users may be unable to find content and abandon the site. Essential needs are to relate user behavior to Web site structure; to compare site usage at different times, or for different classes of users; segmentation of sessions; quantification of inter-relationships among pages; and prediction of user behavior, including forecasts, for example, of the economic impact of promotional campaigns. The ultimate impact is more efficient Web sites that serve users more effectively. This research will create a set of four increasingly complex, but scalable, Bayesian models that relate the usage (specifically, user page transitions) of a Web site to its structure. It will apply, validate and refine the models and use real data from four qualitatively different Web sites, an E-commerce site, a site operated by a large financial institution, a content site and an information site. The models are scalable because the destinations from a given page are classes of pages that mirror the tree structure of the site, rather than individual pages. Examples are the parent, children and siblings of a page. All four models assume Dirichlet prior distributions for transitions from each page. The first three employ very aggregated classes of transitions, and differ according to whether the transition distributions and the priors are the same for all pages. The fourth model disaggregates the "child" and "sibling" destinations. Calculation of posterior distributions varies in difficulty: some are available in closed form, while others require intensive Markov chain Monte Carlo computation. In addition rigorous model assessment will provide insight into what level of aggregation is appropriate to which analyses of Web data.