j?”ey?z]>q”?

1

Farkas, J., & Neumayer, C. (2017). ‘Stop Fake Hate Profiles on Facebook’: Challenges forcrowdsourcedactivismonsocialmedia.FirstMonday,22(9).

‘StopFakeHateProfilesonFacebook’:Challengesforcrowdsourced

activismonsocialmedia

JohanFarkas*andChristinaNeumayer**

* ITUniversityofCopenhagen,directcommentstojjfs@itu.dk.

**ITUniversityofCopenhagen,chne@itu.dk.

Abstract

Thisresearchexamineshowactivistsmobiliseagainstfakehateprofileson

Facebook.Basedonsixmonthsofparticipantobservation,thearticle

demonstrateshowDanishFacebookusersorganisedtocombatfictitious

Muslimprofilesthatspurredhatredagainstethnicminorities.Thearticle

concludesthatcrowdsourcedactionbyFacebookusersisinsufficientasaform

ofsustainableresistanceagainstfakehateprofiles.Aviablesolutionwould

requiresocialmediacompaniessuchasFacebooktotakeresponsibilityinthe

struggleagainstfakecontentusedforpoliticalmanipulation.

Keywords:onlineactivism;crowdsourcedactivism;fakeprofiles;hate

profiles;Facebook

Introduction

Thisishowweshuthispagedown.We’renearly1300membersandifwe

eachspend5secondsreportinghispage,it’llberemovedinnotime.

(StopFakeHateProfilesonFacebook,postbyadmin,1July2015)

Pre-print version

2

InJune2015,aclosedFacebookgroupnamedStopFakeHateProfileson

Facebook(STOPfalskeHAD-PROFILERpåFACEBOOK)wascreatedtocombat

fakeprofilesspurringanti-MuslimdiscoursesinDenmark.Within24hours,the

groupattractedover1000membersengaginginseveralformsofcooperative

contestation.Mostnotably,thegroupusedcollectivereportingofcontentfor

violationsofFacebook’scommunitystandards(Facebook,2016).

StopFakeHateProfilesonFacebookwascreatedinreactiontoseveral

Facebookpagesthatsparkedhundredsofhatefulcommentsandsharesfrom

DanishFacebookusersinspring2015.Thesepageswereallconstructed

aroundfictitiousMuslimidentities,claimingtorepresentawiderMuslim

communityinDenmark.TheirconsistentmessagewasthatDanishMuslims

wereconspiringtotakeoverthecountry,rapeDanish(white)women,andkill

allnon-Muslims(Farkasetal.,2017).Mostuserswhoreactedtothishateful

contentdidnotrealisetheidentitieswerefakeandexpressedaggressionas

wellasxenophobicsentimentsincomments.Furthermore,userswho

contestedthepages’authorshipincommentsweresystematicallyremoved

andblockedbytheanonymouspageadministrator(s).

JournalistsfromtheDanishpublicservicebroadcaster(DanmarksRadio)

eventuallyreportedonthephenomenon,highlightingthattheFacebookpages

werefakeandlikelyconstructedbyfar-rightactiviststosmearMuslims

(Nielsen,2015).Thelatterfinding,however,couldnotbepositivelyconfirmed,

asFacebookenablespageadministratorstoremaininvisible,challengingany

Pre-print version

3

legalactionagainstthem.StopFakeHateProfilesonFacebookthusrepresented

theonlysystematicattempttoresistandcombatthefakeMuslimFacebook

pages.ThisoccurredthroughcrowdsourcedreportingofthepagestoFacebook

inordertogetthecompanytoclosethemdown.

Thepowerofcrowdsourcedonlineactivismasaformofcollectiveresistance

haslongbeenheralded,thoughparticularlyintheearlydaysofsocialmedia

(Benkler,2006;Jenkins,2006;Shirky,2009).Scholarshavearguedthat

dataficationofpersonalinformationandtheriseofmany-to-many

communicationenablesnewformsofpoliticalmobilisationbasedonapolitics

ofnumbers(LoaderandMercea,2011).Acoreaspectofsuchpolitical

mobilisationiscrowdsourcedcollectiveaction(inthestreetsandonline),often

throughsocialmediaplatformsthatenablelarge-scalecoordinationand

organisation(Lotanetal.,2011).Thereare,however,limitationstothisformof

action.Giventheincreasingrangeofopportunitiesforengagementinthe

digitalera,ithasbecomecommontolamentthatonlineparticipationisno

morethanfeel-good‘slacktivism’(Morozov,2011),‘clicktivism’(White,2010),

oraltogetherlackingacollectivealtruisticcomponent(Bauman,2001).While

thiscriticismmightringpartiallytrueinthecaseofStopFakeHateProfileson

Facebook,thisarticlearguesthatparticipationandactivismorganisedinthe

groupwasconditionedandlimitedbyFacebook’sdigitalarchitecture.Basedon

participant-observationalfindings,thearticleexploresthechallengesthatStop

FakeHateProfilesonFacebookfacedinitsstruggle.Drawinguponthese

findings,thearticlesuggeststhatcrowdsourcinguseractioncanonlymakea

marginalcontributiontosustainablypreventingfakehateprofilesonsocial

Pre-print version

4

mediaundercurrentconditions.Asustainablesolutionwouldrequirethat

Facebooktakesongreaterresponsibilityasacompanyandprovidemorethan

itscurrentlylimitedandopaqueusersupport.

Thecrowdsourcingideologyonsocialmedia

JeffHowecoinedtheterm‘crowdsourcing’in2006inaWiredarticle.Theidea

ofcrowdsactingandcreatingtogetherwaspresentinearlydiscoursesabout

socialmedia.TimO’Reilly’s(2005)conceptof‘Web2.0’hadthe“wisdomofthe

crowds”asakeycomponent.Theseideasmainlyincludedcrowdsourcingina

businesscontext,focusingonbottom-upcreativeprocessesinwhich

companiesadoptideasfromcrowds,fans,andamateurs.Inadiscourse

analysisofpopularpressarticlesconcerningcrowdsourcing,Brabham(2012,

p.407)concludesthattheconceptwasalsopromotedas“apotentially

powerfultooltospurpublicparticipationandtransparencyingovernment

affairs.”Brabhamargues,however,thatthe‘amateur’labelinthiscontext

delegitimisesotherwise-worthyagentsbydevaluingtheirrolesasparticipants

andcitizensindemocraticsociety.Liberatorytechnologicaldiscourses–a

powerfulpartofthecorporateidentitiesofsocialmediacompaniessuchas

GoogleandFacebook(Turner,2006)–havethusbeenadoptedinboth

contemporarybusinessculturesanddemocraticdiscoursesandprocesses.

Basedonananalysisofthepoliticaleconomyofthedigitalmediaindustry,

Sandoval(2014,p.252)arguesthat,ratherthanbeingsocial(asassertedin

corporatesocialresponsibilitystatements),socialmediacompaniesexploit

labourand“arefeedingonthecommonsofsociety.”Socialmediaandother

Pre-print version

5

techcompaniesco-optideasoftheradicalleft,suchasparticipation,

decentralisation,spontaneousinteraction,andlackofdisciplineandhierarchy

(Žižek,2009),inconceptssuchascrowdsourcing.Thesediscoursesof

empowerment,however,shifttheobligationforactiononsocialmediatothe

users.Thiscreatespotentialsforuseractionaswellasdisempowermentsince

socialmediacompaniescandisowncorporateresponsibilityforphenomenaon

theirplatformssuchasfakehateprofiles.

Facebook’scommunitystandardsstatethatthecompanystrives“towelcome

peopletoanenvironmentthatisfreefromabusivecontent.Todothis,werely

onpeoplelikeyou”(Facebook,2016).Thecompany’smodelforhandling

abusivecontentisthusbuiltaroundfreeuserlabour.Thisiseconomically

beneficialforFacebook,asitonlyemployscommercialcontentmoderatorsto

reviewcontentreportedbycost-freeusers(Fuchs,2015;Roberts,2016).It

alsoenablesthecompanytodistanceitself,bothlegallyandcommunicatively,

fromabusivematerialonitsplatformbygrantingusersprimaryresponsibility.

ThisevasionstrategyiscentraltoFacebook,whichiscurrentlyseekingto

increasethisdelegationofresponsibility:“Theideaistogiveeveryoneinthe

communityoptionsforhowtheywouldliketosetthecontentpolicyfor

themselves”(Zuckerberg,2017).Asweshowinthisarticle,Facebook’suser-

centredapproachisproblematic,asthecompanycircumventsresponsibility

forcounteringabusewhileprovidinginadequateandopaquetoolsforuser

action.Thisdisempowersusersandlimitsthepotentialforcounteracting

phenomenasuchasfakehateprofiles.

Pre-print version

6

Activismandsocialmedialogics

Manychallengesconfrontactivistsusingcorporatesocialmediaplatformsto

counter-actdominantdiscourses,includingracism.PoellandBorra(2011,p.

695)notethatfor“crowd-sourcingalternative[news]reporting,”thecontent

oftweetsisframedbymainstreamnewstoproducevisibility.Leistert(2015)

arguesthatcorporatesocialmediahavebecomealgorithmicmassmedia,using

algorithmstocensor,normalise,andstandardiseactivistcommunications.The

silencingofcriticalvoicesreinforcesneoliberalvaluesinwhichcorporate

socialmediaplatformsareembedded(Couldry,2010).Inordertosuccessfully

achievepoliticalgoals,activistsinsocialmediaenvironmentsmustthusadapt

theirpoliticalstrategiestocorporatesocialmedialogicssuchasconnectivity,

popularity,anddatafication(vanDijckandPoell,2013).Throughthis

adaptation,activistsriskbeingco-optedbythesocialmedialogicsthatthey

attempttouseagainstthesystem(GalisandNeumayer,2016).Inotherwords,

insteadofempoweringactivists,“powerhaspartlyshiftedtothetechnological

mechanismsandalgorithmicselectionsoperatedbylargesocialmedia

corporations”(PoellandvanDijck,2015,p.534).

Inhisphilosophyoftechnology,Feenberg(2002)focusesonhumanagency,

arguingthattechnologyreinforcesprevailingpoliticalhierarchiesandpower

relations.Feenbergsuggests,however,thattechnologicalinventionalso

providesnewopportunitiesforsubversiveactorstochallengepoliticalsystems

byappropriatingnewmediatechnologiesfortheircause.Acriticalanalysisof

technologymustconsequentlybe“balancedbydescriptionofwhatpeople

actuallydoinpractice”(Mackenzie,2006,p.458).Thisrequiresustoopenthe

Pre-print version

7

blackboxofsocialmediamateriality“asactiveagentsshapingthesymbolic

andorganizationalprocessesofsocialactors”(Milan,2015,p.897).Inthe

following,weseektounpackthisblackboxbyanalysingthesocialmedia

practicesofStopFakeHateProfilesonFacebook.Insodoing,weexplorehow

thegroupnavigatessocialmedialogicsinitsstruggleagainstfakehateprofiles.

Aparticipant-observationalinquiry

Thisarticlebuildsupondatacollectedduringsixmonthsofparticipant

observationwithintheclosedFacebookgroupStopFakeHateProfileson

Facebook.ThefieldworkcommencedinlateJune2015,shortlyafterthe

creationofthegroup,andendedinearlyJanuary2016.Levelsofactivity

withinthegroupvariedoverthecourseofthesixmonths,withconcentrations

aroundoccurrencesoffakehateprofiles.Duringtheresearchperiod,StopFake

HateProfilesonFacebookcontestedeightfakehateprofiles,whichattracteda

totalofover14,000commentsand6000sharesfromDanishFacebookusers.

Priortothegroup’screation,datafromfivefakeMuslimFacebookpageshad

alreadybeencollectedinAprilandMay2015(Farkasetal.,2017).WhenStop

FakeHateProfilesonFacebookwascreatedinresponsetofakeMuslim

Facebookpages,itwasthuspossibletoinitiateresearchwithinthegroup

shortlythereafter.Dataonfakehateprofilescollectedpriortotheexistenceof

thegroupenablesacomparativeperspectiveonfictitiousprofilesbeforeand

afterinitiationofthegroup’scollectivecontestation.

Thedatasetof13fakehateprofiles–eightofwhichwerecontestedbyStop

FakeHateProfilesonFacebook–derivesfromourqualitativeapproach.Based

Pre-print version

8

ononlineparticipantobservations(Hine,2015),ourresearchobjectiveisto

exploreandinvestigatethepeople,objects,controversies,conflicts,and

negotiationssurroundingfakehateprofilesandthestruggleagainstthem.

Throughoutthesixmonthsofresearch,wecontinuouslyobservedand

participatedintheactivitiesofStopFakeHateProfilesonFacebook.This

involvedahighdegreeofengagement.Wesupportedthegroup’scauseand

interactedregularlywithgroupmembers,particularlythegroup

administrator.Theprimarypurposeoftheseinteractions(whichcanbestbe

describedasinformaldialogues)wastounderstandthewaysinwhichthe

groupwasorganisedandoperated.Basedontheseobservations,thisarticle

seekstounravelthedelicatepracticesandtacticsofStopFakeHateProfileson

Facebookaswellasthechallengesfacingthegroup’scrowdsourceduser

action.Infuturework,quantitativemeasurescouldadvantageouslybe

includedtoexaminethescaleandproliferationoffakehatesprofilessuchas

thosecontestedbythisgroup.

InformedconsentwassecuredfrommembersofStopFakeHateProfileson

Facebookbyfirstcontactingthepageadministratorandreceivingpermission

fromhim.Wethereafteraskedtheadministratortopostastatementinthe

groupforallmemberstosee,disclosingourresearchagendaandrequesting

permissiontodofieldwork.Inthisstatement,weassuredgroupmembersthat

wewouldprotecteveryone’sanonymity.Thegrouprespondedpositivelyto

ourrequest.Useractivitywithinthegroupwasarchivedthroughscreenshots

and‘printpage’functionalitiestoensuretheexistenceofdataincasethegroup

oritscontentweredeleted.Intotal,wecollected38posts(allmadebythe

Pre-print version

9

groupadministrator)and943comments.Subsequenttoourfieldwork,all

namesofgroupmembershavebeenanonymised,andtheactoftranslation

fromDanishtoEnglishrendersthecontentunsearchable.

StopFakeHateProfilesonFacebook

Asitsnamesuggests,StopFakeHateProfilesonFacebookwascreatedwiththe

purposeoffindingandcombattingwhatitterms‘fakehateprofiles’on

Facebook.Inthegroup’smissionstatement,thistermencompasses“fake

profiles[…]groups,orpagescreatedtoincitefearandhatredtowardsspecific

groupsinDanishsociety”(Postbygroupadministrator,21June2015).The

group’sobjectivewastoexposeandcombatsuchprofilesthroughcollective

effortsusingFacebook’sdigitalarchitectureandcommunitystandards.All

groupmemberscouldinvitenewuserstothegroup,thoughtheyhadtobe

approvedbytheadministrator.Thegroupwasexplicitlynon-partisan,and

politicaldiscussionswerenotallowed.

Thefakehateprofilescombattedbythegroupwereidentifiedonthebasisofa

numberofcharacteristics,mostprominently:useofstolenprofilepictures,

falselyproclaimedaffiliationswithexistingorganisations;deletionofuser

commentsquestioningtheprofiles’authorship,lackofresponsewhen

contactingtheprofiles,andrhetoricsimilartothatofpreviousprofiles

identifiedasfake.ThefakehateprofilesusedfictitiousMuslimidentitiesto

constructanarrativeofMuslimsplottingtooverrunDanishsociety,killingand

rapingethnic(white)Danesintheprocess:

Pre-print version

10

IslamisNOTaboutpeacebutsubjugationtoAllah.Oncewegetsharialawin

Denmark,allyouinfidelpigswillhavetosubmittoIslam…It’sokaytokill,

aslongasthevictimsareinfidels.AllahuAkhbar!

(Facebookpost,MohammedEl-Sayed,30June2015)

YouDanescanlaughatmenow,butjustwaituntilwegetsharialawin

Denmark,thenallnon-Muslimswillbe‘removed’(ifyouknowwhatImean)

J.AllahuAkhbar!YoushouldbythewayknowthatItakeyourmoney,I

havesexwithyourcheapwomen,andImakethempregnant.

(Facebookpost,MehmetDawahAydemir[1],9September2015)

Mostpostsfromthesefakehateprofilescontaineddirectthreatstooppress,

rape,andkill(non-Muslim)Danes.Othersprovokedbyrejoicinginthe

September11terroristattacksorstatingthatallDanesarestupidpigsand

dogs.Onallprofiles,theaggressivestatementswerepresentedasoriginating

fromyoung,Danish-speakingMuslimslivinginDenmark(Farkasetal.,2017).

Thesefictitiousidentitieswereallconstructedaroundexistingxenophobic

stereotypesofMuslimsasviolent,hypersexual,andalienthreatstotheDanish

welfarestate(Hervik,2011).Stolenimages,text,andhyperlinkswerethusall

deployedtopersonifythesestereotypesascredibleandauthenticindividuals.

Oneachprofile,imagesofArab-lookingpeoplewerepresentedalongsidelinks

toexistingMuslimorganisations,postsaboutMuslimsdestroyingDenmark

fromwithin,andimagesofburningDanishflagsortheflagofISIS.Thefake

profilesallclaimedtospeakonbehalfofawiderMuslimcommunityin

Denmark,allparticipatinginalarge-scaleconspiracy:“WeMuslimshavecome

Pre-print version

11

tostay.Wehaven’tcomeinpeace,buttotakeoveryourshittycountry”

(Facebookposts,ZahraAl-Sayed,2July2015).Rhetoricandwordingwere

highlysimilaracrossthepages,indicatingthattheircreatorswerelikely

connectedoridentical.AsFacebookenablespageadministratorstoremain

completelyanonymous,however,theactualidentitiesandmotivesofthese

authorscannotbeestablished.Consequently,intermsofmotive,wecanonly

concludethatallfakehateprofilesdeliberatelysoughttoprovokeandspark

anti-MuslimaggressionfromDanishFacebookusers–anagendainwhichthey

largelysucceeded.

Acrossthevariousfakehateprofiles,theviolentrhetoricpromptedthousands

ofusercommentsfromDanesbelievinginthestatedauthorshipand

respondingwithhatredtowardsthefictitiousidentitiesaswellasMuslimsand

immigrantsingeneral:

Gohometoyourowncountry!Wedidn’taskyoutocomeheretoour

country”

(Usercomment,MehmetDawahAydemir[1],9September2015)

Whatthefuckisthis,youfuckingpig!!!WehelpyoucometoDenmarkand

thisishowyouthankus!

(Usercomment,MehmetDawahAydemir[1],9September2015)

NotallusersreactedwithaggressiontowardsthefictitiousMuslimidentities.

Numeroususersactivelytriedtodismantlethehatredandwarnothersthat

Pre-print version

12

theprofileswerefake.Theanonymouspageadministrators,however,

systematicallyobstructedsuchattempts,asweshowbelow.

StopFakeHateProfilesonFacebookwasformedinJune2015toorganiseand

increasecontestationoffakehateprofiles.Thiscontestationinvolvedfour

distinct,concurrentprocesses:(1)findingandreportingpages,(2)alerting

users,(3)alertingjournalistsandauthorities,and(4)speculatingabout

culprits.Theseprocesseswerecontinuouslynegotiatedanditeratively

developedbygroupmembersinordertoincreasetheeffectivenessoftheir

efforts.Inthefollowingsections,weexplorethegroup’scrowdsourced

contestation,focusingonthesocio-technicaltacticsdeployedintheirstruggle.

Basedonthisexamination,wediscussthelimitationsandopportunitiesfor

crowdsourceduseractiononsocialmediaandtheirimplicationsforthe

preventionoffakehateprofiles.

(1)FindingandreportingfakehateprofilesonFacebook

Thefirststepinthegroup’scontestationwastosearchforFacebookprofiles,

groups,orpagesusingfictitiousidentitiestodisseminatehatespeech.When

memberslocatedsuchcontent,theywouldcontactthegroupadministrator

andgethimtosharealinkwithinthegroupalongsideashortstatement,for

example:

We’vereceivedatipfromamemberanditseemsthisprofileisfake.The

rhetoricissimilartopreviousprofiles,andIwillthereforeencourageyouall

toreportthepage,sowecanshutitdown.

Pre-print version

13

(Postbygroupadministrator,5January2016).

UserswouldfollowthelinkandreporttheprofiletoFacebookforviolationsof

thecompany’scommunitystandards,whichprohibitbothfakeidentitiesand

hatespeech(Facebook,2016).KeytothisoperationwasFacebook’s‘report’

button,whichcanbefoundonallprofilesandpagesaswellasposts,pictures,

andvideos.WhenreportingviolationstoFacebook,groupmemberswould

subsequentlypostcommentswithinthegroup,oftensimplywriting:

‘Reported’.Memberswouldtherebycontinuouslymaketheir(otherwise-

invisible)actionsvisibletoeachother.Someusersdeliberatelyreportedthe

sameprofilefornumerousviolations(e.g.fakeidentity,hatespeech,

harassment)andalsoreporteditsindividualposts.Thiswasdoneinthehope

thatlargerquantitiesofreportswouldcauseFacebooktopaymoreattention.

Facebook’sprocessingoffileduserreportsisahighlyopaqueprocess

(Roberts,2016),makingitdifficulttodiscernhowthecompanyoperates.

Consequently,groupmemberswoulditerativelyexchangepersonal

experiencesandhypothesesinanattempttomaximisetheeffectivenessof

theircrowdsourcedcontestation.Arecurrentfindingbygroupmemberswas

thatthequantityofreportsplayedamajorroleinFacebook’sresponse,

althoughthecompanyofficiallydeniesthis(Facebook,2016).Often,when

filingreports,groupmemberswouldinitiallyreceiveastandardresponsefrom

Facebook,statingthatthereportedprofile(s)didnotviolateFacebook’s

communitystandards.Groupmemberswouldtakescreenshotsofthesereplies

andpostthemwithinthegroupaccompaniedwithstatementsofdisbelief:

Pre-print version

14

Really!?They’vecheckedthepageandwon’tshutitdown…!!!”

(Commentbygroupmember,22June2015)

Ican’tbelieveFacebookclaimsthisisn’tviolatingtheircommunity

standards?Afakeprofilespreadinghatespeech,thismustbeaviolationof

therules?

(Commentbygroupmember,1July2015)

Afternumerousadditionalreports,Facebook’sverdictwouldoftenbe

reversed,causinguserstopostnewscreenshotsaccompaniedwithstatements

ofcelebration:“TogetherWEARESTRONG…evilwillbeconqueredinthisway!

<3”(Commentbygroupmember,2July2015).Thepatternofreversed

verdictsfromFacebookcausedmemberstospeculatethatthecompanyatfirst

respondsalgorithmicallytofiledreportsandonlylaterinvolvesactualhuman

staff:“Keepreportingtheprofiles.Facebookusesrobotstogothroughthe

complaints.Realhumanswillonlylookintoitiftherearelotsofreports”

(Commentbygroupmember,22June2015).

Asexemplifiedbytheabovequotes,groupmembersfeltempoweredthrough

theircollectivecontestation,asitenabledthemtoinfluence(whatwere

otherwisefelttobe)unwaveringdecisionsmadebyFacebook.Simultaneously,

however,groupmembersalsofeltdisempoweredbyFacebook’ssecrecyand

lackofcollaboration,withnoapparentinterestinthegroup’scrowdsourced

activism.Thegroup’spowerseemedtoliesolelyinitssize.Groupmembers

andthegroupadministratorwouldthereforerepeatedlyemphasisethe

Pre-print version

15

importanceofallmembersfilingasmanyreportsaspossibleandcomplaining

ifFacebookdidnotrespondpositivelytotheirrequest(s):

Weneedtokeepreportinghis[theanonymousadministrator’s]page.At

somepoint,Facebookwillgettiredandlookatwhathe’sactuallywritten.

Thisishowweshuthispagedown.We’realmost1300members,andifwe

allspend5secondsreportinghispage,it’llberemovedinnotime.

(Postbygroupadministrator,1July2015).

ThecontestationsurroundingFacebook’s‘report’buttonshowshowStopFake

HateProfilesonFacebookengagedintacticalsocio-technicalnegotiations,

continuouslyattemptingtounlockFacebook’ssecretivedigitalarchitecture

anduseitstrategicallytofurtheritscause.Thesestrategiesprovedlargely

successful,ascontestedhateprofilesoftenonlyexistedforafewdaysbefore

Facebookremovedthem(seeTable1).

<InsertTable1-Overviewoffakehateprofilesandtheirdurationsof

existence.>

(2)Alertingusers

Onseveraloccasions,hateprofilescontestedbyStopFakeHateProfileson

FacebookreceivedhundredsoreventhousandsofcommentsfromDanish

Facebookusers.MostusersacceptedtheproclaimedMuslimidentitiesand

expressedanger,hostility,andevenracism:

Pre-print version

16

Fuckyou,youfuckingmonkey

(Usercomment,MohammedEl-Sayed,1July2015)

Disgustinganimal!Getthefuckoutofmycountry…youdon’tbelonghere!

(Usercomment,MehmetDawahAydemir[1],11September2015)

It’sbecauseofpeoplelikeyouthatmoreandmorepeopleturnracist

(Usercomments,MehmetDawahAydemir[1],11September2015).

StopFakeHateProfilesonFacebooksoughttodismantlethishatredtowards

Muslimsandimmigrantsbyalertingusersthattheprofileswerefakeand

deliberatelycreatedtoinciteaggression.Groupmemberswouldpost

commentsontheprofiles,warningusersnottobelieveintheproclaimed

identitiesandpoliticalmanipulation.Aftermakingsuchcomments,members

wouldnotifyeachotheroftheiractionswithintheclosedgroup:“Wroteonhis

page,awarningandalinktothisgroup”(Commentbygroupmember,1July

2015).

Theanonymouspageadministrator(s)runningthefakehateprofiles,however,

continuouslysabotagedtheseefforts.OnallFacebookprofilesandpages,

administratorscanremoveanycontentwithoutnotifyingitsauthorandcan

blockanyuserfrommaking(additional)comments.Theadministrator(s)of

thefakehateprofilessystematicallyusedthistechnologicalfeaturetotheir

advantagebydeletingallcommentsandblockingalluserswhocontestedtheir

proclaimedauthorship.Newusersencounteringthehateprofileswouldthus

beexposedexclusivelytousercommentsaffirmingthelegitimacyofthe

Pre-print version

17

sources.Groupmembersandtheirwarningswerecontinuouslydeletedand

blockedeventhoughtheystillattemptedtoalertusers:

Yougetblockedsofastinthere,butatleastIgottopost20timesthatthe

pagewasfakebeforeitwasover.

(Commentbygroupmember,1July2015);

Iwasremovedrightaway!!Thepersonbehindmustknowthatwework

togetherandareonhistrail!!

(Commentbygroupmember,1July2015).

Duetothesystematicmoderationperformedbytheanonymouspage

administrator(s),theeffectivenessofthegroup’seffortstoalertusersastothe

existenceoffakeMuslimhateprofilesseemstohavebeenlimited.The

administratorsofthehateprofilestacticallyexploitedFacebook’sdigital

architecturetosilenceanycontestation.Nevertheless,afewgroupmembers

reportedthattheyhadinfactfirstbelievedinthefakeauthorshipandonly

laterbecameawareofitsdeceptivenatureduetocommentsmadebygroup

members:“Yesterday,Ireallythoughtthatsomeonewasbeingthishostileand

Ijumpedinfeetfirstandcursedhimback.I’mgladsomeonetoldmeitwas

fake.”(Commentbygroupmember,12September2015).Thishighlightshow

thestrugglebetweenStopFakeHateProfilesonFacebookandvariousfakehate

profilesfundamentallyconcernedvisibilityandawareness.Thehateprofiles

soughttorenderallcontestationinvisible,leavingonlycommentsaccepting

theproclaimedauthorship.Thegroup’sgoal,incontrast,wastomakeits

Pre-print version

18

contestationasvisibleaspossibletowarnuserswhilesimultaneouslymaking

thepagesinvisible(throughdeletionbyFacebook).

Inseveralrespects,Facebook’sdigitalarchitectureseemstohavesupported

thehateprofiles’agendabyprovidingunlimitedanonymitytotheir

administrator(s)aswellasasymmetricalpowerrelationsbetween

administrator(s)andusers(e.g.throughtheabilitytoremoveanycomment).

Thecountergroup’seffortstoalertusersregardingfakehateprofilesmight

havefurthermorehadtheunforeseenconsequenceofcontributingtotheir

proliferation.Facebook’salgorithmscontinuouslyevaluatecontentand

‘decide’howfaritshouldspreadbasedonanumberofparameters.Acentral

parameterinthisprocessisthenumberoflikes,comments,andshares

receivedbythecontentinquestion(Bucher,2012).Commentspostedby

membersofStopFakeHateProfilesonFacebooktowarnusersmightthushave

indirectlyincreasedthefakehateprofiles’reach,potentiallydeceiving

additionalFacebookusers.Thus,despitethegroup’scollectiveefforts,fake

hateprofilescontinuedtoposeacomplexchallenge.Aswediscussbelow,

however,thegroupalsopursuedthegoalofmakingtheircontestationvisible

outsideofFacebook.

(3)Alertingjournalistsandauthorities

AlthoughStopFakeHateProfilesonFacebookprimarilyoperatedwithinthe

boundariesofsocialmedia,thegroupalsosoughttoreachoutandinvolve

journalistsandauthoritiesintheirstruggle.Thegroupmanagedtoattractthe

attentionofseveralmajorDanishmediainstitutions,includingthenational

Pre-print version

19

tabloidEkstraBladet(Ryde,2015),thenewspaperInformation(Skovhus,

2015),andtheTVbroadcasterTV2.Thesemediaoutletsallreportedonthe

phenomenonoffakehateprofilesonFacebook,thelattertwointerviewingthe

group’sadministratoraspartoftheircoverage.Thepublicoutreachagenda

pursuedbythegroupwasprimarilyundertakentowarntheDanishpublic

aboutpotentialdemocraticdangersposedbyfakehateprofiles.

Simultaneously,itenabledthegrouptoattractmorememberstoparticipatein

theirstruggle.Theseeffortslargelyprovedsuccessful.Yetaswiththegroup’s

effortstowarnusersonFacebook,theincreasedattentiontofakehateprofiles

achievedthroughmassmediacouldpotentiallyalsohaveledmoreusersto

engagewiththeprofiles,indirectlyincreasingtheirproliferationonFacebook.

Inparallelwiththegroup’seffortstoreachjournalists,groupmembersalso

contactedtheDanishpoliceandtheintelligenceagency(PET)inorderto

instigateinvestigationsintotheoriginatorsofthefakehateprofiles.The

ephemeralityofthecontestedcontent,however,presentedanobstacletothis

agenda.Theshorttimeperiodsinwhichthefakehateprofilesexistedmeant

thatarchivedmaterialwasnecessaryinordertofilepolicereports.Thegroup

addressedthischallengebyworkingcollectivelytocompilesuchmaterial:

REQUEST:Amemberisaskingforscreenshotsfromthehateprofilesthat

havebeenshutdownsincethepolicewanttolookintothecase…please

sendthemtomeinaprivatemessageorpostthembelow,sothey’revisible.

(Postbygroupadministrator,2July2015)

Pre-print version

20

Inadditiontothechallengeofpiecingtogetherdeletedevidence,the

ephemeralityandanonymityoffakehateprofilesprovedproblematic.

Ephemeralityofcontentmeantthatauthoritiescouldneverobservethe

consequencesoffakehateprofilesastheyunfolded.Furthermore,the

completeanonymityoffakehateprofilecreators,enabledbyFacebook’s

design,meantthatnochargescouldbefileddirectlyagainstanyone.Doingso

wouldfirstrequireathoroughinvestigationandclosecontactwithFacebook.

ThiscausedfrustrationandfeelingsofdisempowermentformembersofStop

FakeHateProfilesonFacebook,asthesocialmediacompanyshowedno

apparentinterestincollaboratingwiththem.Thegroupwasthustotallyreliant

onDanishauthoritiesforconductinginvestigations,yetthegroupalso

experiencedalackofsupportfromauthoritiesinidentifyingandinvestigating

thecreatorsoffakehateprofiles.Thiscauseddistress:

Idon’tunderstandwhyITspecialistsinthepolicecan’tfindtheir[the

administrator’s]IPaddress…Thesefakeprofilesaresofarout…

(Commentbygroupmember,13September2015)

Idon’tthinkwecanachieveanythingthroughpolicereports.

(Commentbygroupmember,1July2015)

ThemostpowerfulmeansavailabletomembersofStopFakeHateProfileson

Facebookthuscontinuedtobetheircollectiveeffortstoreportfakehate

profilestoFacebookandgetthemdeleted.Yetthisstrategyhadsevere

limitations,asthegroupcouldnevergettotherootoftheproblemdueto

Facebook’sdigitalarchitectureand(apparent)lackofinterestincollaboration.

Pre-print version

21

Theanonymouscreatorsoffakehateprofilescouldcontinuously(re-)create

newfictitiousidentitieseachtimeoldoneswereremoved.Formembersof

StopFakeHateProfilesonFacebook,thiscausedfrustration,eveninsituations

inwhichFacebookdeletedfakehateprofiles:“Yes:)!Finally,they[thefake

hateprofiles]areremoved..buthe[theanonymousadministrator]willjust

createanewone:(:(“(Commentbygroupmember,1July2015).Thelackof

collaborationfromauthoritiesandFacebookledtoinvestigationsbygroup

memberstoidentifytheanonymouscontentcreators.

(4)Speculatingaboutculprits

ArecurringthemewithinStopFakeHateProfilesonFacebookwasspeculations

astowhowerebehindthefakehateprofilescombattedbythegroup.On

Facebook,allpageandprofileadministratorscanremaincompletely

anonymous.Evenifapageorprofileisremoved,noinformationisprovidedas

towhocreatedit.Duetonumeroussimilaritiesacrossdifferentfakehate

profiles,groupmembersbecameconvincedthatseveralprofileswerecreated

bythesameadministrator(s):“Thisisexactlythesamerhetoricasonthelast

one.It’sthesamepersonwho’sbehindit,fuckingcoward”(Commentbygroup

member,24October2015);“Youjustknowit’sa20-year-oldkidwithno

friendsandNazitendencieswho’sbehindthekeyboard.”(Commentbygroup

member,1July2015).Severalmembersexpressedfrustrationattheabilityof

theanonymousadministrator(s)tocontinuallyconstructnewfakehate

profilesandsparkaggression,eventhoughFacebookcontinuallydeletedthe

pages.Membersalsoexpressedhopethatauthoritieswouldreactand

investigatetheculprits:“Ireallyhopehe[theadministrator]willbepunished

Pre-print version

22

forthehatredhecreates”(Commentbygroupmember,12September2015).

Othersconductedtheirowndetectiveworkandformulatedhypothesesabout

specificpeoplewhocouldbebehindtheprofiles,includingfar-rightactivists.

Suchspeculationswere,however,criticisedbyothermembers,whoargued

thatStopFakeHateProfilesonFacebookshouldnotbecomeavigilantegroup:

“ThisisexactlywhatImean.Asuspicionisn’tenoughtoaccusepeople”

(Commentbygroupmember,12September2015).Thegroupnever

successfullyidentifiedanyhatecontentcreators,thoughtherewerestrong

indicationsthatseveralofthefakehateprofilescombattedbythegrouphad

thesameadministrator(s).

Crowdsourcedsocialmediaactivism

HavingexploredtheactivistpracticesofStopFakeHateProfilesonFacebook,

wenowaddresshowgroupmembersnavigatedFacebook’ssocialmedialogics

intheirstruggleagainstfakehateprofiles.Wefurthermorediscusswhether

thegroup’scrowdsourcedactivismprovedsuccessfulintermsofits

overarchinggoalofstoppingfakehateprofiles.Thegroup’scrowdsourced

contestationandreportingdoesindeedseemtohavesucceedinshorteningthe

lifespansoffakehateprofiles(seeFigure1).Fakehateprofilesstudiedinour

researchthatexistedpriortothegroup’sformationexistedsignificantlylonger

thandidprofilesthatwerecreatedafterthegroup’sformation.Facebook,

however,maintainsinitscommunitystandardsthatquantityofreportsdoes

notinfluencetheevaluationofflaggedcontent(Facebook,2016).Facebook’s

opaqueevaluationproceduresmeanthatdifferentfactorscouldhavehadan

impact.

Pre-print version

23

<InsertFigure1-Lifespansoffakehateprofilesbeforeandafterthe

formationofStopFakeHateProfilesonFacebook.>

Temporalityisnottheonlyavailableparameterforevaluatingthegroups’

struggleagainstfakehateprofiles.Someprofilesthatexistedfortheshortest

periodsoftime(andwerecreatedafterthegroup’sformation)werealsothose

thatreceivedthemostcommentsandsharesfromDanishFacebookusers(see

Figure2).Forexample,aprofilenamedMehmetDawahAydemir[1],which

existedforlessthantwodaysinSeptember2015,managedtoattract10,426

commentsand4954shareswithinthisperiod.Mostcommentingusersdidnot

recognisethepage’sdeceptivecharacter.Itsrapidproliferationcanpartiallybe

explainedbyFacebook’salgorithmicprioritisationofshorttimedecayswhen

assessingtheimportanceofcontentand‘deciding’itsreach(Bucher,2012).

Posts,images,andvideoscaninotherwordsreachthousandsofuserswithin

hoursiftheysparkalargenumberofinteractions.Thisseemstohavebeenthe

casewithMehmetDawahAydemir[1].

<InsertFigure2-Timesofexistenceoffakehateprofilesandnumbersof

commentsandshares.>

AlthoughStopFakeHateProfilesonFacebooksuccessfullyreducedthe

lifespansoffakehateprofiles,theprofilescouldstilldeceiveandprovoke

thousandsofuserswithinthisperiod.Lifespan,then,doesnotseemtobethe

bestindicatorofthegroup’ssuccessindismantlingfakehateprofiles.This

Pre-print version

24

raisesthequestionofwhetherthisformofcrowdsourcedactivismrepresentsa

viabletrajectoryforstoppinghatredandmanipulationonsocialmedia.Should

itbeuptouserstostopphenomenasuchasfakehateprofilesonFacebook?Or

shouldsocialmediacompaniestakegreaterresponsibility?

Basedonthechallengesandlimitationsfacingthecrowdsourcedactivismof

StopFakeHateProfilesonFacebook,wearguethatFacebook’sdelegationto

usersofresponsibilityforreportingviolationsisproblematic.Unlesssocial

mediacorporationstakegreaterresponsibilityincombattingfacelesshatred

andracismproducedbyanonymousadministrators,noactioncangobeyond

solelyclosingdownsuchhateprofiles.Reachingouttoandcollaboratingwith

authorities–findingcontentandidentifyingitscreators–couldbepartofa

solution.Removingunlimitedanonymityforpageandprofileadministrators

couldbeanother.Suchefforts,however,wouldrequireFacebooktochangeits

self-image,whichiscurrentlythatofatechcompanyandnotamediacompany

(Seetharaman,2016).IfFacebookistostopfakehateprofilesonitsplatform,

thecompanymustacknowledgethatproblemsassociatedwithfakeidentities

andhatredarepartiallyitsresponsibilityandnotonlythatofusers.This

argumenthasrecentlybeenraisedindebatesconcerning‘fakenews’(Stromer-

Galley,2016).Hopefully,futureresearchcanhelpaddresstheseissuesby

expandingcurrentknowledgeontheextensivenessoffakehateprofileson

socialmediaaswellasrelatedphenomenasuchasfakenewsspreadbysocial

bots(seeShaoetal.,2017).Sucheffortscouldadvantageouslydrawuponboth

bigdataanalysisandmachinelearning(seeFerraraetal.,2016).

Pre-print version

25

Onthebasisofthepresentarticle’sfindings,Facebook’slimitedresponseto

thephenomenonoffakehateprofileshighlightsadiscrepancybetweenthe

company’sbusinessmodelanditscorporateideals.Facebook’sbusinessmodel

isbuiltaroundcommodificationofuser-generateddataanduserattention,

makingthequalityofcontenteconomicallysecondarytotheattentionit

receives.Atthesametime,Facebook’scorporateidentity,whichhijacksleft-

wingideasofparticipationanddecentralisation(Žižek,2009),burdensusers

withresponsibilityfortacklingproblemssuchasfakehateprofiles.Inthis

process,thecompanyprovidesonlylimitedopportunitiesforuserstoengage

incrowdsourcedactivism.EventhoughFacebookreferstoitsplatformasa

“globalcommunity”(Facebook,2016),thecompanyseemstoprioritise

commodificationofuserattentionovertheempowermentofusersandquality

ofcontent.

Inthecurrentstateofaffairs,crowdsourcingofresponsibilityleavesuserswith

a‘report’buttonastheironlyweapon.Eveniffakehateprofilesonlyexistfor

shortperiodsoftime,theirvisibilitycanstillbegreatduetosocialmedialogics

thatalgorithmicallyprivilegecontentthatquicklyattractscomments,likes,and

shares–evenifthesereactionsexpresshatredandracism.AlthoughStopFake

HateProfilesonFacebookcontinuouslyperformedcrowdsourcedresistance,

Facebook’sarchitecturedisempoweredthegroupbylimitingitspossibilities

foraction,whilefakehateprofilescouldcontinuetospurhatred,aggression,

hostility,andracism.Ifthisistochange,socialmediacompaniesmustreduce

hierarchicalpowerrelations,increasethepotentialforuseraction,andtake

responsibilityforhatredandracismontheirplatforms.

Pre-print version

26

Abouttheauthors

JohanFarkasisAssistantLecturerattheITUniversityofCopenhagen.His

researchinterestsincludepoliticalparticipationanddisguisedpropagandain

digitalmedia.

ChristinaNeumayer(PhD,ITUniversityofCopenhagen/MA,Universityof

Salzburg)isAssociateProfessorofdigitalmediaandcommunicationinthe

DigitalDesigndepartmentattheITUniversityofCopenhagen.Herresearch

interestsincludedigitalmediaandradicalpolitics,socialmediaandactivism,

socialmovements,civicengagement,publicsandcounterpublics,surveillance

andmonitoring,andbigdataandcitizenship.

Acknowledgements

TheauthorswouldliketothankStopFakeHateProfilesonFacebook,

particularlyitsadministrator,formakingthisworkpossible.Theauthors

wouldalsoliketothanktheanonymousreviewersofthisjournal.

References

ZygmundBauman,2001.Theindividualizedsociety.Cambridge,UK:Polity

Press.

YochaiBenkler,2006.TheWealthofNetworks:HowSocialProduction

TransformsMarketsandFreedom(Vol.7).NewHaven:YalePress.

Pre-print version

27

DarenC.Brabham,2011.“Themythofamateurcrowds:Acriticaldiscourse

analysisofcrowdsourcingcoverage”.Information,Communication&Society,

volume15,number3,pp.394-410.

TainaBucher,2012.“Wanttobeonthetop?Algorithmicpowerandthethreat

ofinvisibilityonFacebook”.NewMedia&Society,volume14,number7,pp.

1164–1180.

NickCouldry,2010.Whyvoicematters:Cultureandpoliticsafterneoliberalism.

London:SAGEPublications.

Facebook,2016.“FacebookCommunityStandards”at

https://www.facebook.com/communitystandards,accessed13January2016.

JohanFarkas,JannickSchou,andChristinaNeumayer,2017.“Cloaked

FacebookPages:ExploringFakeIslamistPropagandainSocialMedia”.New

Media&Society.https://doi.org/10.1177/1461444817707759

AndrewFeenberg,2002.Transformingtechnology:Acriticaltheoryrevisited.

Oxford,UK:OxfordUniversityPress.

EmilioFerrara,Wen-QiangWang,OnurVarol,AlessandroFlammini,andAram

Galstyan,2016.“Predictingonlineextremism,contentadopters,and

interactionreciprocity”.In:E.SpiroandYY.Ahn(editors),SocialInformatics.

Pre-print version

28

SocInfo2016.LectureNotesinComputerScience,Springer.

https://doi.org/10.1007/978-3-319-47874-6_3

ChristianFuchs,2015.Cultureandeconomyintheageofsocialmedia.London:

Routledge.

VasilisGalisandChristinaNeumayer,2016.“Layingclaimtosocialmediaby

activists:acyber-materialdétournement”.SocialMedia+Society,volume2,

number3,pp.1-14.

PeterHervik,2011.TheAnnoyingDifference:TheEmergenceofDanish

Neonationalism,Neoracism,andPopulisminthePost-1989World.NewYork:

BerghahnBooks.

ChristinaHine,2015.EthnographyfortheInternet:Embedded,Embodied,and

Everyday.London:Bloomsbury.

JeffHowe,2006.“TheRiseofCrowdsourcing”.Wired,Issue14,number6.

https://www.wired.com/2006/06/crowds/,accessed19July2017.

HenryJenkins,2006.Convergenceculture:Whereoldandnewmediacollide.

NewYork:NewYorkUniversity.

OliverLeistert,2015.“Therevolutionwillnotbeliked:Onthesystematic

constrainsofcorporatesocialmediaplatformsforprotest”.In:L.DencikandO.

Pre-print version

29

Leistert(editors),Criticalperspectivesonsocialmediaandprotest:Between

controlandemancipation.London:Rowman&LittlefieldPublishersInc,pp.5–

52.

BrianD.LoaderandDanMercea,2011.“NetworkingDemocracy?”.Information,

Communication&Society,volume14,number6,pp.757–769.

http://doi.org/10.1080/1369118X.2011.592648

GiladLotan,ErhardtGraeff,MikeAnanny,DevinGaffney,IanPearce,danah

boyd,2011.“TheArabSpring|TheRevolutionsWereTweeted:Information

Flowsduringthe2011TunisianandEgyptianRevolutions”.International

JournalofCommunication,volume5,pp.1375–1405.http://doi.org/1932–

8036/2011FEA1375

AdrianMackenzie,2006.“Innumerabletransmissions:Wi-Fi®fromspectacle

tomovement”.Information,Communication&Society,volume9,number6,pp.

781–802.http://dx.doi.org/10.1080/13691180601064139

StefaniaMilan,2015.“Fromsocialmovementstocloudprotesting:The

evolutionofcollectiveidentity”.Information,Communication&Society,volume

18,number8,pp.887–900.

http://dx.doi.org/10.1080/1369118X.2015.1043135

EvgenyMorozov,2011.Thenetdelusion:Hownottoliberatetheworld.London:

PenguinBooks.

Pre-print version

30

StineBødkerNielsen,2015,19May.“ViovertagerDanmark:Falskefacebook-

sidersættermuslimeridårligtlys”.DRNyheder.

http://www.dr.dk/Nyheder/Indland/2015/05/18/110828.htm,accessed19

July2017.

TimO’Reilly,2005.“Whatisweb2.0”at

http://www.oreilly.com/pub/a/web2/archive/what-is-web-20.html,,

accessed19July2017.

ThomasPoellandErikBorra,2011.“Twitter,YouTube,andFlickrasplatforms

ofalternativejournalism:Thesocialmediaaccountofthe2010TorontoG20

protests”.Journalism,volume13,number6,pp.695–713.

http://doi.org/10.1177/1464884911431533

ThomasPoellandJosevanDijck,2015.“SocialMediaandActivist

Communication,”In:C.Atton(editor).TheRoutledgeCompaniontoAlternative

andCommunityMedia.London:Routledge,pp.527-537.

SarahT.Roberts,2016.“CommercialContentModeration:DigitalLaborers’

DirtyWork,”In:S.U.Noble&B.Tynes(editors),TheIntersectionalInternet:

Race,Sex,ClassandCultureOnline.NewYork:PeterLang,pp.147–160.

http://doi.org/10.1007/s13398-014-0173-7.2

Pre-print version

31

RonjaRyde,2015.12September.EkspertomMehmet-profil:Forargende

Dannebrogs-pissererformentligfalsk.EkstraBladet.

http://ekstrabladet.dk/nyheder/samfund/ekspert-om-mehmet-profil-

forargende-dannebrogs-pisser-er-formentlig-falsk/5729027,accessed19July

2017.

MarisolSandoval,2014.Fromcorporatetosocialmedia:Criticalperspectiveson

corporatesocialresponsibilityinmediaandcommunicationindustries.

Abingdon,UK:Routledge.

DeepaSeetharaman,2016.25October.“FacebookLeadersCallItaTech

Company,NotMediaCompany”.TheWallStreetJournal.

https://www.wsj.com/articles/facebook-leaders-call-it-a-tech-company-not-

media-company-1477432140,accessed19July2017.

ClayShirky,2009.11December.Thenetadvantage.ProspectMagazine.

http://www.prospectmagazine.co.uk/2009/12/the-net-advantage,accessed

19July2017.

ChengchengShao,GiovanniL.Ciampaglia,OnurVarol,AlessandroFlammini,

andFilippoMenczer,2017.“Thespreadoffakenewsbysocialbots”.Eprint

arXiv:1707.07592.

Pre-print version

32

PhillipR.Skovhus,2015.22September.ӮgtehadfrafalskeFacebook-

profiler”.Information.http://www.information.dk/546054,accessed19July

2017.

JenniferStromer-Galley,2016.2December.“ThreewaysFacebookcould

reducefakenewswithoutresortingtocensorship”.TheConversation.

http://theconversation.com/three-ways-facebook-could-reduce-fake-news-

without-resorting-to-censorship-69033,accessed19July2017.

FredTurner,2010.Fromcounterculturetocyberculture:StewartBrand,the

WholeEarthNetwork,andtheriseofdigitalutopianism.Chicago:Universityof

ChicagoPress.

JoséVanDijckandThomasPoell,2013."Understandingsocialmedialogic."

MediaandCommunication,volume1,number1,pp.2-14.

http://www.cogitatiopress.com/mediaandcommunication/article/view/70

MicahWhite,2010,12August.“Clicktivismisruiningleftistactivism”.The

Guardian.

http://www.guardian.co.uk/commentisfree/2010/aug/12/clicktivism-

ruining-leftist-activism,accessed19July2017.

SlavojŽižek,2009.Violence:Sixsidewaysreflections.London:ProfileBooks.

Pre-print version

33

MarkZuckerberg.2017,“BuildingaGlobalCommunity”at

https://www.facebook.com/notes/mark-zuckerberg/building-global-

community/10154544292806634/,accessed7august2017.

Pre-print version

34

Name Time period Number of days

Ali El-Yussuf [3] 16/06-22/06 2015 7

Mohammed El-Sayed 30/06-02/07 2015 3

Fatimah El-Sayed 01/07-02/07 2015 2

Zarah Al-Sayed 02/07-02/07 2015 1

Mehmet Dawah Aydemir [1] 09/09-12/09 2015 4

Mehmet Dawah Aydemir [2] 13/09-15/09 2015 2

Ebrahim Said 24/10-25/10 2015 2

Mohammed Al-Dawah 05/01-07/01 2016 3

Table1:Overviewoffakehateprofilesandtheirdurationsofexistence.

Pre-print version

35

Figure1:LifespansoffakehateprofilesbeforeandaftertheformationofStopFake

HateProfilesonFacebook.

Pre-print version

36

Figure2:Timesofexistenceoffakehateprofilesandnumbersofcommentsandshares.

Pre-print version