human computation steven emory cs 575 human issues in computing
Post on 22-Dec-2015
220 views
TRANSCRIPT
Overview
What is Human Computation?
History of Human Computation
Examples of Human Computation
Challenges in Human Computation
Definition of Computation
Normally we rely on computers to do all the work
On input:
Step #1 (computer)
Step #2 (computer)
... (computer)
Step #n (computer)
Ouput
Definition of Human Computation
Some or all steps solved by human(s)
Roles reversed: Computer asks us to solve a problem
On input:
Step #1
Step #2 (ask a human)
...
Step #n
Output
History of Human Computation
1980's: Interactive Genetic Algorithms
2000's: Human-Based Genetic Algorithms
2000's: Outsourced Human Spam
2000's: Interactive Guessing Games
Notes on Human Computation
While Interactive and Human-Based GAs can employ the use of a single person, the term Human Computation is typically only used when many humans are involved
Difficult to get a lot of data out of just one person
Results are statistically better when many people agree (if it's just one person are you sure he's telling the truth?)
Humans are typically networked (not communicating, collaborating, or cheating)
Quicksort Example
Bad Example: Quicksort
Input: Unsorted array
Select a pivot (human selection)
Swap last element with pivot element
Partition array using pivot element
Insert pivot element into correct position
Repeat above steps for left/right side partitions
Output: Sorted array
Quicksort Problems
Solved faster by computer alone
Not rewarding
Boring/painful
Wouldn't do this even if I was paid to do it
Using first row of students, would everyone pick the same pivots?
Photomosaic Demo
To demonstrate that humans are better at some things than computers, consider a photomosaic:
Can be solved by computer alone.
On input “image gallery,” “source image”1.) Tile source image.2.) From left-to-right, top-to-bottom, compare each image in the image gallery to each tile in the source image, inserting the gallery image with the lowest error.3.) Output photomosaic.
Photomosaic Problems
Noticeable visual artifacts (no image reuse)
Could use a randomized algorithm instead
Could optimize important features first
However, no general “select important features” algorithm exists
Must ask a human (the user) to select important features
Photomosaic Solutions
Revised algorithm:
On input “image gallery,” “source image”1.) Tile source image.2.) Select important features (ask human).3.) Optimize (randomly) important features.4.) Optimize (randomly) unimportant features.5.) Output photomosaic.
Unlike the Quicksort example:
this is rewarding and fun
When to Use
When the problem:
is hard for a computer, but easy for a human
is not boring to humans (music, art, games)
is rewarding (financially or emotionally)
Human Computation Challenges
User interface design
Coordinating many human participants
Analogous to distributed computing
Honesty
Prolonged computation
Maintaining human interest and motivation
reCAPTCHA Example
Used to digitize old books (make e-books)
OCR normally works 99% of the time
OCR accuracy drops for older books
Old paper
Old printing techniques
Solution: Ask humans to determine words OCR fails to classify
When enough humans agree, consider it solved
Metadata Example
Algorithm: Assign metadata to images
Useful for content/multimedia management systems (i.e. Istockphoto)
No algorithm exists for image labeling
Luis Von Ahn's solution: The ESP Gamehttp://www.espgame.org
ESP Game Demo
Electric Sheep Example
Brief fractal explanation:
Iterative process based on chaos, dynamical systems
Newton's Method FractalSolve az3 + bz2 + cz + d = 0 for complex numbersa, b, c, d are fractal parameters
Cubic equation has 3 rootsRed = converges to root #1Green = converges to root #2Blue = converges to root #3Black = fails to converge
Electric Sheep Example
Animated fractal screensaver application
Also an interactive genetic algorithm
Humans (users of screensaver) evaluate fitness
Animated fractal parameters are mutated
Algorithm has been running for years
http://www.electricsheep.org/
Outsourced Human Spam
On input “http://acm.calstatela.edu/forum”:
Find Register link (spam bot)
Download registration form (spam bot)
Fill out registration form (spam bot)
Find CAPTCHA (spam bot)
Solve CAPTCHA (human)
Send registration form (spam bot)
Output: SPAM!!!
Conclusions
There many hard/impossible to solve problems
Nothing shameful about using Human Computation
Applications in art, music, computer vision, security, content management