EAGER: Programming the Crowd

University Of Massachusetts Amherst, Amherst MA

Investigators

Abstract

People can perform with ease many tasks that remain difficult or impossible for computers, including vision, motion planning, and natural language understanding. New "crowdsourcing" platforms like Amazon's Mechanical Turk make it easier than ever to harness human computational power by streamlining job posting, tracking, and payment for workers. However, the lack of automation means that crowdsourcing currently does not scale up. Low quality results must be filtered out, but checking human computations can be difficult. Economic incentives and anonymity expose crowdsourcing to fraud. Deciding how much to pay workers for particular tasks and how many workers to hire remains an art. This project introduces crowdprogramming, an approach that fully integrates human and digital computation. In crowdprogramming, humans are modeled as function calls in a standard programming language. This approach lets programmers focus on programming logic, while the crowdprogramming runtime system manages the critical tradeoffs between cost, time, and data quality. Crowdprogramming will dramatically lower the barriers to harnessing human computational power. It will enable a rich new class of applications that divide labor between digital and human computations, where computers and humans do the work each does best. It will enable complex orchestration of human computations, automatically control quality to maintain high accuracy and avoid fraud, and schedule tasks and adjust payments to maximize speed while staying within budget. By streamlining the incorporation of human labor into computation, crowdprogramming has the potential to add an entirely new job sector to the economy.