Reconfiguring Crowdsourcing


Selected online talks:

Guidelines for Academic Requesters (of Amazon Mechanical Turk)

During the summer of 2014, a well-meaning researcher acted on Turkopticon in ways that workers found harmful. Partially in response to this latest in a string of systematically produced misunderstandings, a number of Mechanical Turk workers, researchers, and I developed a set of publicly published ethics guidelines for academic researchers using AMT. As it turns out, alongside Amazon’s market infrastructures of AMT, university IRBs (ethics institutional review boards) are one of the few institutions  workers can use to call employers to account . They might call employers to account for unpaid wages, privacy intrusions, or hurtful deceptions. These guidelines are not only an act of communication, but a tactical document that workers can point to in their communications with IRBs to a higher set of standards for engagement than Amazon’s existing terms of service.

The guidelines were authored through a Wiki, but also through communication and debate both in Dynamo and in various worker-run forums and chats. They are signed by

View the guidelines


This project is undertaken in collaboration with Six Silberman, the worker-moderators of Turkopticon, and Turkers who contribute reviews.

turkopticon screenshotSince 2008, we have designed and maintained Turkopticon, a tool and web service that enables Amazon Mechanical Turk (AMT) workers to share job information with one another and hold employers accountable for fair treatment. Turkopticon is a browser extension that inserts reviews of employers into AMT’s task marketplace interface. Thousands of workers a month use the system.

AMT is a system that allows programmers to solicit large numbers of workers to do small bits of computational labor, such as transcription, image labeling, or survey participation. These bits of labor can be integrated directly into computational systems, layering human computation into existing and emerging information technologies. Human computation represents a growing area of both academic research and industrial development.

Our work examines what happens when people are organized as smooth computational infrastructure, integrated directly into algorithms. As a source of ethical insight into AMT, we began by surveying workers, asking them to draft a what they would like as a worker’s bill of rights. Though workers’ responses diverged wildly, many agreed on one problem: they had little recourse when employers withheld pay. We responded to these findings by designing Turkopticon.

Through our work on Turkopticon, we have explored how communications technologies support or hinder digital solidarities. Through the design and maintenance of the system, we have also explored the politics of mediated and massively distributed participation in design.

The project has drawn attention to the ethics of paid microlabor in the The Nation, San Jose Mercury News, New Scientist,, East Bay Express, Communications of the ACM, SalonO’Reilly Radar, the Sacramento Bee, The Verge, The Huffington Post.