Edith Law: "Balancing Task-Centric and Human-Centric Objectives in Human Computation Systems"

Date: 

Monday, September 17, 2012, 12:00pm to 1:30pm

Location: 

Maxwell Dworkin G135

CRCS Lunch Seminar

Date: Monday, September 17, 2012
Time: 12:00pm – 1:30pm
Place: Maxwell Dworkin G135

Speaker:  Edith Law, Harvard CRCS

Title: Balancing Task-Centric and Human-Centric Objectives in Human Computation Systems

Abstract:  When creating a human computation system, typically two design objectives need to be simultaneously satisfied. The first objective is human-centric — the task prescribed by the system must be intuitive, appealing and easy to accomplish for workers. The second objective is task-centric — the system must actually perform the task at hand. These two goals are often at odds with each other, especially in settings where workers are unpaid volunteers (e.g., game players) who are driven by some intrinsic motivation (e.g., the desire to be entertained) other than monetary reward.

In this talk, we describe a hybrid approach to designing human computation games to achieve both the human-centric and task-centric objectives, where one first designs for humans, then incorporates machine intelligence to work around the limitations of workers and to complement their abilities.  In particular, we focus on the design of games for attribute learning, e.g., determining that a piece of music is “soothing”, that the bird in an image “has a red beak”, or that Ernest Hemingway is a “Nobel Prize winning author.”  We demonstrate the effectiveness of our approach in two concrete problem settings, namely music tagging and bird image classification.

Bio:  Edith Law is a first-year CRCS postdoctoral fellow at the Harvard University School of Engineering and Applied Sciences.  She graduated from Carnegie Mellon University in 2012 with a Ph.D. in Machine Learning, where she worked with Luis von Ahn and Tom Mitchell on designing human computation systems that harness the joint efforts of machines and humans. She is a Microsoft Graduate Research Fellow from 2009-2011, co-organized the Human Computation Workshop (HCOMP) Series at KDD and AAAI from 2009 to 2012, and co-authored the book “Human Computation” published in the Morgan & Claypool Synthesis Lectures on Artificial Intelligence and Machine Learning.  Her research on games with a purpose and large-scale collaborative planning has received best paper honorable mentions in CHI 2009 and 2011.