AF:Small:Unifying Information Aggregation and Information Elicitation

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

Crowdsourcing is an approach to solving problems via distributed effort. In the future, crowdsourcing mechanisms should prove useful in gathering and aggregating information in a variety of contexts including basic scientific research, reputation systems, peer grading, and many decision-making contexts including purchasing, product development, pricing, etc. This project will design practical crowdsourcing mechanisms to incentivize users to report accurate data, even when the veracity cannot be directly verified. A key part of this is accurately measuring the quality of the information reported. Building mechanisms sufficiently robust for dealing with diverse types of agents is necessary for the success of this proposal. Thus, it has applications beyond crowdsourcing to big data more broadly (interpreting data from diverse sources of varying reliability) and algorithmic fairness. The research efforts will be integrated with the educational and outreach activities of the investigator, who has a record of broadly disseminating cutting-edge research to high school, undergraduate, and graduate students through teaching, outreach programs, and personal mentoring. Recent developments in the field of information elicitation show how to use information-theoretic concepts to truthfully elicit unverifiable information from agents. The heart of these results is first to employ clever techniques to indirectly measure ``mutual information" between agents, then to compensate agents proportionally to the mutual information measurement. There is also increasing work in learning theory, especially to enable learning with noisy or adversarial data. The goal of this proposal is to create a bridge between the information-elicitation and information-aggregation tasks. By using an information-theoretic underpinning we can unify the progress of these fields, and, in many instances, fuse these tasks together. The project will undertake the following. 1) Design full pipeline systems by uniting the techniques of information elicitation and aggregation into one streamlined process. 2) Examine how to better measure information-theoretic concepts required for truthful elicitation algorithms. 3) Address important concerns such as algorithmic fairness, optimal and realistic payments, and robustness. Moreover, this proposal will help to better integrate information theory into the computer-science community, which is essential in the era of data science. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.