talking to the crowd in 7,000 languages
TRANSCRIPT
![Page 1: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/1.jpg)
Talking to the crowd in 7,000 languages
Robert MunroIdibon
Crowdsourcing
![Page 2: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/2.jpg)
Information is increasing
• Scale (well-known)
• Diversity (less understood)
– On a given day, what is the average number of languages that someone could potentially hear?
– How has this changed?
Outline
![Page 3: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/3.jpg)
Daily potential language exposure
5 5 5 5 5 5 4.5 4
50
1500
5000
2000
1400
720540 500
Year
# o
f la
ngu
ages
![Page 4: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/4.jpg)
5 5 5 5 5 5 4.5 4
50
1500
5000
2000
1400
720540 500
Daily potential language exposure
Year
# o
f la
ngu
ages
![Page 5: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/5.jpg)
5 5 5 5 5 5 4.5 4
50
1500
5000
2000
1400
720540 500
Daily potential language exposure
Year
# o
f la
ngu
ages
![Page 6: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/6.jpg)
5 5 5 5 5 5 4.5 4
50
1500
5000
2000
1400
720540 500
Daily potential language exposure
Year
# o
f la
ngu
ages
Putting a phone in the hands of everyone on the planet is the easy part
Understanding everyone is going to be more complicated
![Page 7: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/7.jpg)
99% of languages don’t have machine-translation or similar services:
• Disproportionately lower healthcare & education
• Disproportionately greater exposure to disasters
Crowdsourcing can bridge part of the gap.
Diversity
![Page 8: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/8.jpg)
GRAPH OF DEPLOYMENTS
Crowdsourcing
![Page 9: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/9.jpg)
Crowdsourcedprocessing of information in Haitian Kreyol.
1000s of Haitians in Haiti and among the diaspora.
Haiti – Mission 4636
Apo
Dalila
Haiti
(18.4957, -72.3185)
“I need Thomassin Apo please”
“Kenscoff Route: Lat: 18.4957, Long:-72.3185”
“This Area after Petion-Ville and Pelerin 5 is not on Google Map. We have no streets name”
![Page 10: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/10.jpg)
Lopital Sacre-Coeur ki nan vil Okap, pre pou li resevwamoun malad e lap mande pou mounki malad yo ale la.
“Sacre-Coeur Hospital which located in this village of Okap is ready to receive those who are injured. Therefore, we are asking those who are sick to report to that hospital.”
![Page 11: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/11.jpg)
Lopital Sacre-Coeur ki nan vil Okap, pre pou li resevwamoun malad e lap mande pou mounki malad yo ale la.
“Sacre-Coeur Hospital which located in this village of Okap is ready to receive those who are injured. Therefore, we are asking those who are sick to report to that hospital.”
![Page 12: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/12.jpg)
Lopital Sacre-Coeur ki nan vil Okap, pre pou li resevwamoun malad e lap mande pou mounki malad yo ale la.
“Sacre-Coeur Hospital which located in this village of Okap is ready to receive those who are injured. Therefore, we are asking those who are sick to report to that hospital.”
![Page 13: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/13.jpg)
Evaluating local knowledge
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Haitians (volunteers and paid) “Non-Haitians
3,000 messages< 5 minutes each
> 4 hours each
45,000 messages
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
Lopital Sacre-Coeur ki nan vilOkap, pre pou li resevwa mounmalad e lapmande pou mounki malad yo alela.
![Page 14: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/14.jpg)
Haiti – Mission 4636
Lessons learned
• Default to private data practices
(Majority decision was not to use a public map)
• Find volunteers through strong social ties
(10x larger/faster than the publicized efforts)
• Avoid activists (‘bloggers’, ‘crisis-mappers’ …)
• Localize to the crisis-affected community
(25% of work was by paid workers in Haiti)
![Page 15: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/15.jpg)
Haiti – Mission 4636
Paid workers in Mirebalais, Haiti (FATEM)
Benchmarks we can use:*
$ 0.25 per translation
$ 0.20 per geolocation
$ 0.05 per categorization / filtering
4:00 minutes per report processed
Can volunteerism undercut this cost?
* Munro. 2012. Crowdsourcing and crisis-affected community: lessons learned and looking forward from Mission 4636. Journal of Information Retrieval
![Page 16: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/16.jpg)
Data-structuring for 2010 floods in Pakistan
Pakreport
*Chohan, Hester and Munro. 2012. Pakreport: Crowdsourcing for Multipurpose and MulticategoryClimate-related Disaster Reporting. Climate Change, Innovation & ICTs Project. CDI
Multiple inexperienced people are more accurate than one experienced person.*
![Page 17: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/17.jpg)
Pakreport
Lessons learned
• Default to private data practices (!)
(Taliban threatened to attack mapped aid workers)
• Cross-validate tasks across multiple workers
(We used CrowdFlower, as with Mission 4636)
• Localize to the crisis-affected community
(Data obtained by hand / created jobs)
![Page 18: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/18.jpg)
Scaling beyond purely manual processing.
Disease outbreaks are the world’s single greatest killer.
No organization is tracking them all.
Epidemics
![Page 19: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/19.jpg)
Diseases eradicated in the last 75 years:
Increase in air travel in the last 75 years:
sma l l p o x
![Page 20: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/20.jpg)
90% of ecological diversity
90% of linguistic diversity
![Page 21: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/21.jpg)
Reported locally before identification
H1N1 (Swine Flu)
months
(10% of world infected)
HIV
decades
(35 million infected)
H1N5 (Bird Flu) weeks
(>50% fatal)
Simply finding these early reports can help prevent epidemics.
![Page 22: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/22.jpg)
epidemicIQ
M a c h i n e -
l e a r n i n g
( m i l l i o n s )
R e p o r t s
( m i l l i o n s )
M i c r o t a s k e r s
( t h o u s a n d s )
A n a l y s t s – d o m a i n
e x p e r t s
( c a p p e d n u m b e r )
в предстоящий осенне-
зимний период в Украине
ожидаются две эпидемии
гриппа
نفلونزمزيد من في مصرا الطيور ا
香港现1例H5N1禽流感病例曾游上海南京等地
![Page 23: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/23.jpg)
E Coli in Germany
The AI head-start
![Page 24: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/24.jpg)
epidemicIQ
Lessons learned
• Current data privacy practices are insufficient
(reports from areas where victims are vilified)
• Crowdsourcing can provide needed skill-sets
(100s of German speakers at short notice)
• Natural language processing can scale beyond human processing capacity
![Page 25: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/25.jpg)
Libya Crisis Map
A negative example
• 2283 reports already-open, English sources
• 1 month of full-time management and contributions from >100 volunteers
![Page 26: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/26.jpg)
Libya Crisis Map
Equivalent cost from paid workers
• $575.75
(or about $800 with multiple steps)
Equivalent time cost from Libyan nationals:
• 152.2 hours = less than 1 month for 1 person
(would also address some security concerns)
![Page 27: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/27.jpg)
Libya Crisis Map
Lessons learned
• Crowdsourced volunteers were not required
(cost more to run than was saved by not paying)
(a single in-house Libyan could have achieved more)
• Default to private data practices
(assume all identities of volunteers were exposed)
(Libyans opposed the public map)
![Page 28: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/28.jpg)
Crowdsourcing and risk
People’s real-time locations are their most sensitive personal information.
Crowdsourcing distributes information to a large number of individuals for processing.
For information about at-risk individuals:
• Is it right to crowdsource the processing?
• Is it right to use a public-facing map?
![Page 29: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/29.jpg)
Recommendations
• Engage people with local knowledge
• Employ people with local knowledge
• Statistically cross-validate on-the-fly
• Default to private data practices
• Scale via natural language processing
Conclusions
![Page 30: Talking to the crowd in 7,000 languages](https://reader037.vdocument.in/reader037/viewer/2022100305/55a9a0b81a28ab0b718b472a/html5/thumbnails/30.jpg)
Thank you
Robert MunroIdibon
@WWRob
Crowdsourcing