department of information and learning technology national university of tainan, taiwan
DESCRIPTION
Crowdsourcing Game Development for Collecting Benchmark Data of Facial Expression Recognition Systems. Department of Information and Learning Technology National University of Tainan, Taiwan. Hsin-Chih Lin, Zi-Jie Li and Wan-Ling Chu . Outline. Introduction. 01. - PowerPoint PPT PresentationTRANSCRIPT
Crowdsourcing Game Developmentfor Collecting Benchmark Data of Facial Expression Recognition Systems
Department of Information and Learning Technology National University of Tainan, Taiwan
Hsin-Chih Lin, Zi-Jie Li and Wan-Ling Chu
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 2
Outline
Introduction
Literature review
Crowdsourcing Game Development
Experimental Design and Results
0102
0304
Conclusions and Future Works05
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 3
Introduction• Developing an automatic expression
recognition system– always use benchmarks
• Most of facial pictures in benchmarks – not be accepted by the public or other teams
• Manually classifying facial expression pictures – labor-expensive– time-consuming– difficult to standardize
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 4
Literature review• Crowdsourcing was first proposed by Howe
(2006).• The concept of crowdsourcing– to rely on manpower to complete the work – difficult to be replaced by computer programs
• Microtask & National Library of Finland– Mole Bridge– Mole Hunt
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 5
Literature review
• Von Ahn (2006) proposed the concept of “Games with a Purpose”– attract online players through interactive games
• “Gamification” can make boring becomes interesting (Krause & Smeddinck, 2011).
• Listen Game(Turnbull et al., 2007)– improve the results of music search
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 6
Crowdsourcing Game Development
LowValidity
Database
HighValidity
Database
Benchmark
FaceDetection
Crowdsourcing Game
FeatureExtraction
Classification
Face pictures
Social classification system
social = automatic
Automatic recognition system
social ≠ automatic
expression pictures of low validity
expression pictures of high validity
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 7
Crowdsourcing Game Development
• 3 by 3 grid– seven pictures– expression hint – two options
• Game-play rules– two minutes– randomly prompt
an expression hint
– none of the above
– skip
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 8
Experimental Design and Results
• This study enables crowds to classify facial expressions in the game during four-week experiments period– 100 participants – 1,416 times
• Training and testing method of the automatic expression recognition system : – 80/20– Incremental training
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 9
Experimental Design and Results
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 10
Experimental Design and Results
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 11
Experimental Design and Results
• Our study can effectively train automatic recognition system that allows the precision rate of system raised to extremely high in four-week testing.
• The dual system is able to develop an automatic recognition system in this study.
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 12
Experimental Design and Results
• Our benchmark– 84 happiness – 51 sadness – 34 surprise – 30 anger
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 13
Conclusions and Future Works• An innovative dual system mechanism – an organism– enhanced the extremely high precision rate of
an automatic expression recognition system– efficiency and automation to classification that
no matter how many facial expression data needs to be classified
– resolve image classification or other issues must through human computation
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 14
Conclusions and Future Works• Crowdsourcing Game– boring become interesting– save more time and cost– get the classification results agree with crowds
• Future Works– increase facial pictures– increase expressions categories(disgust, fear,
nature)
Pacific Neighborhood Consortium Annual Conference and Joint Meetings 2013 15
Thank you for your attention.