getting things done with crowdsourcing pwi may-2014

http://bitsofknowledge.waterloohills.com

• What is Crowdsourcing?• Crowd Motivations• What can it be used for?• Pro’s & Con’s• Quality Mgt & Workflows• Economical shift

PWI – May 22, 2014 [email protected]


mailto:[email protected]

http://bitsofknowledge.waterloohills.com/


Crowd or Community (online audience)

Crowdsourcing

1 2

3

4


Example Design


Example Freelance Jobs


Ex: “Adult Websites” Classification

• Large number of sites to label• Get people to look at sites and classify them as:

– G (general audience) – PG (parental guidance) – R (restricted) – X (porn)

[Panos Ipeirotis. WWW2011 tutorial]


Ex: “Adult Websites” Classification• Large number of hand‐labeled sites• Get people to look at sites and classify them as:

– G (general audience) – PG (parental guidance) – R (restricted) – X (porn)

Cost/Speed Statistics:• Undergrad intern: 200 websites/hr, cost: $15/hr• MTurk: 2500 websites/hr, cost: $12/hr

[Panos Ipeirotis. WWW2011 tutorial]


Crowd Demography(background defines motivation)

• Amazon’s Mechanical Turk workers come mainly from 2 countries: a) USAb) India


Crowd Demography


Crowd Motivation

• €,$ = Money!

• Self-serving purpose (learning new skills, get recognition, avoid boredom, enjoyment, create a network with other profesionals)

• Socializing, feeling of belonging to a community, friendship

• Altruism (public good, help others)


Examples: Altruism


What kind of activities can be done?

• Find Suppliers for:

Creative work (logos, websites, products..) Look for specific talent Software Testing Support (call centers, secretarial tasks, translations) To offload peak demands Tackle problems that need specific communities or human variety


Mass work, Distributed work, or just tedious work. 3 main goals:

1. Minimize Cost (cheap)

2. Minimize Completion Time (fast)

3. Maximize Quality (good)

Remember Crowd Motivation!(ex.: Game-ify your task,

Explain the final purpose)

What else can be done?

http://bitsofknowledge.waterloohills.com[Panos Ipeirotis. WWW2011 tutorial]


Examples: Games


Gamification and Collaboration• Game FoldIt: In 2011, a puzzle was

presented to the participants. Gamers built on each other’s results, and in 10 days they found the molecular structure of a protein-cutting enzyme from an AIDS-like virus.


Gamification and Collaboration


And still more…

• Market research

• Innovation

• Create buzz, generate customers!

• Find support, be backed up

• Crowdsourcing is your business!

• Be financed: Crowdfunding


Examples of FundingUS: JOBS act legal to give equity to private

investors.

Europe: check if crowdfunding site is authorised by financial regulators

(US/UK) (US/DE)

(US) (LU)

CrowdCube (UK) Symbid (NL)

Angel.me (BE) Identitycoop (BE)

LookandFin (BE) Patreon (US)

Companisto (DE) Seedrs (UK)

Siamosoci (IT)


Pro’s

• Quicker: Parallellism reduces time• Cheap• Creativity, Innovation • Quality (*depends)• Access to scarce resources: The ‘long tail’• Multiple feedback• Allows to create a community (followers)• Business Agility• Scales up! (*up to a level)


Con’s

• Lack of professionalism: Unverified quality • Too many answers• No standards • Not always cheap: Added costs to bring a project to conclusion• Too few participants if task or pay is not attractive• If worker is not motivated, lower quality of work• Legal issues


Crowdsourcing websitesWorkflow Patterns

• Generate / Create• Find • Improve / Edit / Fix

Creation• Vote for accept‐reject• Vote up, vote down, to generate rank• Vote for best / select top‐k

Quality Control, e-Reputation

• Split task• Aggregate Flow Control• Iterate

Flow Control


Economical Shift• From Social Networking to Social Production

through Collaborative Innovation

Mass-Collaboration changes how Products & Services are Designed, Manufactured, Marketed

• Classical geo-political and economical organisations do not correspond to new economy

Realignment of competitive advantages Move towards Collaborative Enterprises based on Open Infrastructure


Societal ShiftMoral values Reinforcement

Creates positive correlation between Ethical values and ROI

Open Data access

Transparent actions

Accountable people

Integrity E-ReputationCommunity Support


[email protected]://bitsofknowledge.waterloohills.comPWI – May 22, 2014

Questions?




References• Wikipedia,2011• Dion Hinchcliffe Crowdsourcing: 5 Reasons Its Not Just For Start Ups Anymore,2009• Tomoko A. Hosaka, MSNBC. "Facebook asks users to translate for free“,2008. • Daren C. Brabham. "Moving the Crowd at iStockphoto: The Composition of the Crowd and Motivations for Participation in a Crowdsourcing Application", First Monday, 13(6),2008.• Karim R. Lakhani, Lars Bo Jeppesen, Peter A. Lohse & Jill A. Panetta. The value of openness in scientific problem solving (Harvard Business School Working Paper No. 07-050),2007.• Klaus-Peter Speidel How to Do Intelligent Crowdsourcing,2011• Panos Ipeirotis. Managing Crowdsourced Human Computation, WWW2011 tutorial,2011• Omar Alonso & Matthew Lease. Crowdsourcing 101: Putting the WSDM of Crowds to Work for You, WSDM Hong Kong 2011. • Sanjoy Dasgupta, http://videolectures.net/icml09_dasgupta_langford_actl/,2009•Don Tapscott, Anthony Williams. Macrowikinomics, 2010.

http://www.ebizq.net/blogs/enterprise/2009/09/196

http://www.ebizq.net/blogs/enterprise/2009/09/crowdsourcing_5_reasons_its_no.php

http://www.msnbc.msn.com/id/24205912/ns/technology_and_science-internet/

http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2159/1969

http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2159/1969

http://www.hbs.edu/research/pdf/07-050.pdf

http://www.innovationmanagement.se/author/klaus-peter-speidel/

http://www.innovationmanagement.se/2011/04/18/how-to-do-intelligent-crowdsourcing/?utm_source=Subscribers+InnovationManagement.se&utm_campaign=55b20197b3-Five+Steps+to+Profitable+Innovation&utm_medium=email

getting things done with crowdsourcing pwi may-2014

Internet

crowd motivations

innovation quality

quality good

unverified quality

crowd demography background

distributed work

mass work

tedious work