gathering momentumcrowdresearch.stanford.edu/w/img_auth.php/0/06/08-01-gathering.pdf ·...

Gathering momentumMichael Bernstein Stanford University http://crowdresearch.stanford.edu

Today’s topics! Reminder of our new methods + goals! Task posting! Working! Monitoring! Homepage! Reputation! Open governance! This week’s milestones

2

Our method + plan

August 31: Platform is online, real work is being done on it

! Task creation, including Prototype Task foundation! Task feed and task submission! Task review

! Our first workers (skunkworks)! Our first work (skunkworks)

4

September 25: CHI paper! A full paper, 10 pages, at CHI, with you as a coauthor! A CHI paper will need:! A working platform! Fully thought-through and implemented foundations! Real usage that we can report

5

Let’s make a trigger! A trigger will be our formal criteria for beginning that

transition.! We want something that is a strong indicator we are

achieving liftoff and will succeed.

! Our trigger: 100 projects completed per week ! Fine print: from at least 15 different requesters — this can’t just be

just Rajan spending his savings to post work to the platform

6

Quarterly feedback! Every three months (or before a major deadline), we’ll

give people 100 credits to allocate across the contributors to the project.! We let you summarize your contributions on the wiki before this

credit vote happens! Give proportional credit to whoever you think is most critical to

our success! We will then run PageRank on this credit vote network to prevent

small groups from accidentally influencing the vote

7

PageRank and link rings! You cannot get credit by having your team all just vote

for you

8

New York Times

Bad #1 Bad #2

Bad #3Legit site

Lots of PageRankflows from NYTimesto legit, but badget almost none

Paid RA positions! There were 25 applications as of this morning! So great to hear of the interest!

! I am currently working with Stanford to create the positions, and will share results next week

! Goal: expand team as I bring in more research funds

9

Contributors: our new process! When you join a milestone on Trello, you are joining an

ad-hoc team for the week to pursue that milestone. Immediately go check in via the relevant Slack room.! “Hey @channel, I’m joining the task creation milestone this week”

! Then: tell the room whenever you’re starting, and whenever you’re pausing. This will prevent you from working on out-of-date items.! “Hey @channel, I’m going to work on the heuristic evaluation for the next two hours. Shout if I shouldn’t!” 10

DRIs: our new process! DRIs are responsible for…! Organizing any ad-hoc hangouts or hackathons to make sure that

everyone who is in the channel knows how to contribute.

! Posting in #announcements by Friday 5pm each week: an update on their team’s progress. Describe what each team member did.! Michael will use this for weekly slides and for recommendation letters.

11

Task posting

DRI: @aginzberg

Worker feed

DRI: @dmorina

Task monitoring

DRI: @ryosuzuki and @neilthemathguy

Homepage

DRI: @karolina

Next generation crowdsourcing

LEARN MORE

CREATE ACCOUNT

WELCOME TO THE FUTURE OF CROWDSOURCING!

Our elevator pitch: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam ac pellentesque nulla. Duis vel leo ut tortor feugiat dapibus quis non diam. Cras ac euismod tortor.

SAMPLE TASKS

How it works Login About

LOREM IPSUM DOLOR SIT AMET, CONSECTETUR ADIPISCING ELIT.


LOREM IPSUM DOLOR SIT AMET, CONSECTETUR ADIPISCING ELIT.! "

"

HERE GOES THE LOGO

SMALL LOGO


LEARN MORE

CREATE ACCOUNT



SAMPLE TASKS





"

HERE GOES THE LOGO

SMALL LOGO

THESE ARE THE LAST 5 TASKS POSTED IN DAEMO:X

Prototype Task: Design Logo

31-07-2015

Develop sketches with 3 options for my photography business logo.

Payment: $10

Skills: Not listed

Number of tasks: 1

Time to complete: 1 min#

$

%

&

'

Prototype Task: Program HTML5 Website

31-07-2015


LEARN MORE

CREATE ACCOUNT



SAMPLE TASKS





"

HERE GOES THE LOGO

SMALL LOGO

LOGINX

Username or email

Password

Forgot your password?

LOGIN


TASKSA variety of tasks for a variety of crowdsourcers

! " # $ | & ' ( )

FOR EMPLOYERS FOR WORKERS

adipiscing elit.

non diam.

adipiscing elit.

non diam.

WHY DAEMO?

FOR EMPLOYERS FOR WORKERS

Create your account now: Create your account now:

SMALL LOGO

22


TESTIMONIALS

ABOUT US

SMALL LOGO

⋆⋆⋆⋆⋆

“ Lorem ipsum dolor sit amet, consectetur adipiscing elit.

“

⋆⋆⋆⋆⋆


“

⋆⋆⋆⋆⋆


“

⋆⋆⋆⋆⋆


“Snehal (Neil) Gaikwad, Durim Morina, Rohit Nistala, Megha Agarwal, Alison Cossette, Radhika Bhanu, Saiph Savage, Vishwajeet Narwal, Karan Rajpal, Jeff Regino, Aditi Mithal, Adam Ginzberg, Aditi Nath, Karolina Zi-ulkoski, Trygve Cossette, Dilrukshi Gamage, Angela Richmond-Fuller, Ryo Suzuki, Jeerel Herrejon, Kevin Viet Le, Claudia Flores-Saviaga, Haritha Thilakarathne, Kajal Gupta, William Dai, Ankita Sastry, Shirish Goyal, Thejan Rajapakshe, Niki Abolhassani, Angela Xie, Surabhi Ingle, Veronica Jaramillo, Martin Daniel, Walter Angel, Carlos Toxtli, Juan Pablo Flores, Asmita Gupta, Vineet Sethia, Diana Padilla, Kristy Milland, Kristiono Setyadi, Nuwan Wajirasena, Muthitha Batagoda, Rolan-

STANFORD CROWD RESEARCH COLLECTIVE

Lorem ipsum dolor sit amet, consectetur adipiscing

-

amet ornare orci, non ultrices sapien. Phasellus blandit, sem at mollis varius, est magna pretium leo, vel cursus tortor felis a purus. Cras mollis mi nec sapien rutrum dictum. Maecenas sed mattis lorem. Fusce ac gravida est, eu tincidunt orci. Sed ut diam ex. Vestibulum ante

23

Snehal (Neil) Gaikwad, Durim Morina, Rohit Nistala, Megha Agarwal, Alison Cossette, Radhika Bhanu, Saiph Savage, Vishwajeet Narwal, Karan Rajpal, Jeff Regino, Aditi Mithal, Adam Ginzberg, Aditi Nath, Karolina Zi-ulkoski, Trygve Cossette, Dilrukshi Gamage, Angela Richmond-Fuller, Ryo Suzuki, Jeerel Herrejon, Kevin Viet Le, Claudia Flores-Saviaga, Haritha Thilakarathne, Kajal Gupta, William Dai, Ankita Sastry, Shirish Goyal, Thejan Rajapakshe, Niki Abolhassani, Angela Xie, Surabhi Ingle, Veronica Jaramillo, Martin Daniel, Walter Angel, Carlos Toxtli, Juan Pablo Flores, Asmita Gupta, Vineet Sethia, Diana Padilla, Kristy Milland, Kristiono Setyadi, Nuwan Wajirasena, Muthitha Batagoda, Rolan-do Cruz, James Damon, Divya Nekkanti, Tejas Sarma, Mohammed Hashim, Gaby Gon, Soroosh Bateni, Gema Toledo, Alex Peña, Ryan Compton, Deen Aariff, Luis Daniel, Muna Ritter, Nisha K.K., Alan James Kay, Jana Uhrmeister, Srivalli Nistala, Milad Nasar Esfahani, Elsa Bakiu, Kristiono Setyadi, Christopher Diemert, Ranjay Krishna, Geza Kovaks, Rajan Vaish, Michael Bernstein

STANFORD CROWD RESEARCH COLLECTIVE

Lorem ipsum dolor sit amet, consectetur adipiscing

-

amet ornare orci, non ultrices sapien. Phasellus blandit, sem at mollis varius, est magna pretium leo, vel cursus tortor felis a purus. Cras mollis mi nec sapien rutrum dictum. Maecenas sed mattis lorem. Fusce ac gravida est, eu tincidunt orci. Sed ut diam ex. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aenean nec est metus. Donec sagittis feugiat risus eu mollis. Fusce sed mattis dolor. Aenean egestas posuere lacus sed vehicula.

Integer libero eros, molestie in ipsum vitae, pulvinar elementum nisi. Fusce eu mollis sapien.

How it works Login AboutSMALL LOGO

PAGE TITLE

Lorem ipsum Dolor sit ametconsectetur Adipiscing

PAGE TITLE

Lorem ipsum Dolor sit ametconsectetur

PAGE TITLE


PAGE TITLE

Lorem ipsum Dolor sit ametconsectetur

PAGE TITLE


Reputation

DRIs: @arichmondfuller and @dilrukshi

Milestone work1. Identified articles to review. http://crowdresearch.stanford.edu/w/index.php?

title=Relevant_Work#Reputation-related_Links_and_Articles

1. Articles read and summarised on http://crowdresearch.stanford.edu/w/index.php?title=Summer_Milestone_9_Reputation_Systems_research_and_exploration by @williamdai, @nishakk, @juechi, @alfonsoxw, @aricmondfuller,@dilrukshi, @surabhi_i + @kajagupta, @claudiasaviaga, @aditi, @rahulsheth1016, @rcompton , @sarmatejas, @rijul, @aditi.nath , @ankita.sastry, @vineet_sethia, and @trygve. @Jsilver and @asmitagupta will submit by Saturday. https://docs.google.com/spreadsheets/d/1ci5sQOMR0mJU6xACysBXRQtGf21B2VV1QMKGnrZRBsY/edit?usp=sharing

1. Reviewed ideas submitted by previous participants https://docs.google.com/document/d/1Hl9Ui3Qmebz2h7MLsPHm1b5ry1b9NUQPq1atZRLj0hU/edit

http://crowdresearch.stanford.edu/w/index.php?title=Relevant_Work#Reputation-related_Links_and_Articles

http://crowdresearch.stanford.edu/w/index.php?title=Summer_Milestone_9_Reputation_Systems_research_and_exploration

https://crowdresearch.slack.com/team/sarmatejas

https://crowdresearch.slack.com/messages/@aditi.nath

https://docs.google.com/spreadsheets/d/1ci5sQOMR0mJU6xACysBXRQtGf21B2VV1QMKGnrZRBsY/edit?usp=sharing

https://docs.google.com/document/d/1Hl9Ui3Qmebz2h7MLsPHm1b5ry1b9NUQPq1atZRLj0hU/edit

4. The work on evaluation the reputation systems on other crowdsourcing platforms was split out into a separate wiki at http://crowdresearch.stanford.edu/w/index.php?title=Summer_Milestone_9_Evaluations_of_reputation_systems_on_other_crowdsourcing_platforms# @arichmondfuller evaluated Freelancer.

We suggest that this work is rolled over to next week’s Trello Milestone 11.

4. @arichmondfuller created a Reputation System Research and Exploration wiki as a central repository for ideas and to keep track of all the work participants have previously contributed http://crowdresearch.stanford.edu/w/index.php?title=Summer_Milestone_9_Reputation_Systems_research_and_exploration @jsilver and @dilrukshi helped to edit this wiki

Milestone work continued

http://crowdresearch.stanford.edu/w/index.php?title=Summer_Milestone_9_Evaluations_of_reputation_systems_on_other_crowdsourcing_platforms#

http://crowdresearch.stanford.edu/w/index.php?title=Summer_Milestone_9_Reputation_Systems_research_and_exploration

Open governance

DRIs: @trygve and @arichmondfuller with an assist from @rajanvaish

This week’s milestones

Research engineering! Task posting! DRI: @maniksingh92

! Worker task submission working! DRI: @dmorina

! Monitoring page working! DRI: @ryosuzuki and @neilthemathguy

29

Reputation! Two goals:! Join one or both brainstorms for the repsys that we anchor in our

first prototype: M/W 7:30pm PDT. DRI: @michaelbernstein

! Study what other platforms (AMT, Upwork, Freelancer) do for reputation. DRI: @dilrukshi and @arichmondfuller

30

Homepage! Implement the home page design!! DRI: @karolina

31

Open govenance! First leadership board meeting — establish a governing

document (“what can and can’t we do?”)

32

Upcoming milestones

This week’s milestones! Reputation: join one or both brainstorms for the

repsys that we anchor in our first prototype: M/W 7:30pm PDT

34

gathering momentumcrowdresearch.stanford.edu/w/img_auth.php/0/06/08-01-gathering.pdf ·...

Documents