Humans, Machines and the Dimensions of Microwork

A session at Strata 2012

Thursday 1st March, 2012

4:00pm to 4:40pm (PST)

The advent of crowdsourcing has wildly expanded the ways we think of incorporating human judgments into computational workflows. Computer scientists, economists, and sociologists have explored how to effectively and efficiently distribute microwork tasks to crowds and use their work as inputs to create or improve data products. Simultaneously, crowdsourcing providers are exploring the bounds of mechanical QA flows, worker interfaces, and workforce management systems.

But what tasks should be performed by humans rather than algorithms? And what makes a set of human judgments robust? Quantity? Consensus? Quality or trustworthiness of the workers? Moreover, the robustness of judgments depends not only on the workers, but on the task design. Effective crowdsourcing is a cooperative endeavor.

In this talk, we will analyze various dimensions of microwork that characterize applications, tasks, and crowds. Drawing on our experience at companies that have pioneered the use of microwork (Samasource) and data science (LinkedIn), we will offer practical advice to help you design crowdsourcing workflows to meet your data product needs.

About the speakers

This person is speaking at this event.
Claire Hunsaker


This person is speaking at this event.
Daniel Tunkelang


Next session in Mission City B1

4:50pm Big Data and Bibliometrics: Crowdsourcing the World's Largest Database of Research by William Gunn and Jan Reichelt

Sign in to add slides, notes or videos to this session

Strata 2012

United States United States, Santa Clara

28th February to 1st March 2012

Tell your friends!


Time 4:00pm4:40pm PST

Date Thu 1st March 2012


Mission City B1, Santa Clara Convention Center

Short URL


View the schedule



See something wrong?

Report an issue with this session