analysis of tweets on demonetization in india using sas ... · analysis of tweets on demonetization...

ANALYSISOFTWEETSONDEMONETIZATIONININDIA

USINGSASENTERPRISEMINERPAPERNUMBER:135

JITALPATELandNARMADAPANNEERSELVAMMSINBUSINESSANALYTICS

OKLAHOMASTATEUNIVERSITY

AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016

Page1of18

ABSTRACT

Thecitizensofacountryoftenfacethebenefitsorthebruntofvariouspoliciesadoptedbythe

government. Social platforms then become sounding boards for them to express their

displeasuresorconcernsforthematterathand.OnesuchpolicythatsentTwitterintoafrenzy

wasthedemonetizationorderthatwasannouncedratherabruptlybytheIndianPrimeMinister

onNovember8th2016withoutany formofprior intimation topublic. In thispaper,wehave

analyzedthetweetsthathelpedusrecognizewhetherdemonetizationwasperceivedpositively

ornegativelybythecitizens.

About 15 days after the demonetization decisionwas announced, a dataset of 8,000 tweets

spanningtwodays,wascollectedfromapubliclyavailabledatasource.Usingthecommonlyused

termsinthetweetsandstudyingthestrengthoftheirrelationsusingconceptlinks,ingeneral,

showapositivefeedbacktothedemonetizationimplementationpolicy.Usingtextclusteringand

texttopictogrouppeoplewithsimilarthoughtsbasedonthetweets,revealthatdemonetization

waspositivelysupportedbyalargenumberofpeople.Thus,ouroverallanalysisshowedthata

vast majority of Indians accepted the demonetization policy positively while some of them

expressedtheirdispleasureoverit.

INTRODUCTION

Demonetizationistheactofstrippingacurrencyunitofitsstatusaslegaltender.Itoccurs

wheneverthereisachangeofnationalcurrency:Thecurrentformorformsofmoneyispulled

fromcirculationandretired,oftentobereplacedwithnewnotesorcoins.Sometimes,acountry

completelyreplacestheoldcurrencywithnewcurrency.Manyreasonscouldleadtothisactof

demonetizationofcurrencyinvariouscountriesallovertheworld.Forexample,todiscourage

cash-dependenteconomy,UnitedStatesdeclaredTheCoinageActof1873asa legal tender.

AccordingtotheAct,silverwasdemonetizedtofullyadopttogoldstandards.Anotherexample

wouldbeTheNationsofEuropeanUnion,whodemonetizedtheoldnationalcurrencies,suchas

Germanmark,Frenchfranc,andItalianliraandintroducedeurobillsandcoinsin2002.Themajor


Page2of18

reason for demonetization being trade purposes. Similarly, the Zimbabwean government

demonetizeditsdollarin2015,asameanstofightcountry’shyperinflation.(Staff,2017)

Themostrecentdemonetizationensuedin Indiaon8thNovember,2016.Themajorobjective

being,tobattleagainstcorruptionandcrime.TheIndianGovernment’sdecisiontodemonetize

all500and1000RupeenotesoftheMahatmaGandhiseriesasaformof legaltender.These

werethetwobiggestdenominationsinitscurrencysystemaccountingforabout86%ofIndia’s

circulatingcash.TheannouncementwasmadebythePrimeMinisterofIndia,Mr.NarendraModi

withnowarning,thatthosenoteswereworthlessandtheyhadtobeexchangedwiththenewly

introduced2000and500Rupeebillsbytheendoftheyear.(Staff,2017)

Publicopinion iseverythingandTwitter isanextensiveplatformtodiscoverthe latest

newsandworldevents. Therefore, analyzing tweetsondemonetization in Indiaandderiving

high-qualityinformationfromTextMiningisthescopeofthispaper.Oneoftheobjectivesofthis

paperis,determiningthemostcommonlyusedtermsinthetweetsandstudyingthestrengthof

theirrelationshipwiththeotherterms.Anothergoalofthispaper is,usingTextClusteringto

grouppeoplewithsimilarthoughtsbasedonthetweetsandextractingTextTopics.

LITERATUREREVIEW

Various research studies and peer-viewed articles show different effects of

demonetization.

“Sentiment Analysis of demonetization of 500 & 1000 rupee banknotes by Indian

Government”byPrabhsimranSingh,RavinderSinghSawhney,KaranjeetSinghKahlon. In this

paper, theyhaveanalyzed theeffectof implementingdemonetizationpolicyusing sentiment

analysis.AnalysisshowsthatlargeshareofIndianpeoplewashappywiththispolicy.Duringinitial

days,thesentimentwasmorenegativeduetohardshipsbutafterthereleaseofnewbanknotes,

overallsentimentofthepeoplebecamepositive.


Page3of18

MohasinA.Tamboli inhispaper, “Demonetization:RetrospectiveAnalysis,” concludes

thatallthegainsofDemonetizationarecontingentongovernment’sabilitytocontrolcorruption.

Ifcorruptionisnotcontrolled,alltheshorttermgainsofdemonetizationwillbelostandtheonly

thingthatwouldchangeforthebetterislookandfeelofnewcurrency.

“DemonetizationandCompleteFinancialInclusion”byS.VijayKumarandT.ShivaKumar,

alsodeterminesthatrewardsofdemonetizationareencouragingandinthelongterminterest

ofthecountrybutgovernmentsneedtoensuresmoothflowofcurrency.ItwilltakeIndiaten

stepsaheadandwillinfluencecorruption,electionsandterrorism.

Another paper by Dr. Pawan Kumar about “Demonetization and its impact on

EmploymentinIndia,”describesthattheemploymentscenarioisnotconduciveenoughtoface

anychallengeofdemonetizationofcurrency.Infact,thedecisionofdemonetizationwillfurther

destabilizethealreadyvolatilelabormarketinIndia.

DATAACCESS

The dataset for this paper contains 8,000 observations, that is the number of tweets

collectedover a spanof twodays and about 15 days after the demonetization decisionwas

announced.ThedatasetistakenfromKaggle(kaggle.com.2017).Thelinkstothemainsourceof

thedatasetisalsoprovidedinthedataset.Adatadictionaryforthevariablesusedintheanalysis

arestatedbelow.

Variable Role Description

ID ID Observationnumber

Text Text DemonetizationTweets


Page4of18

METHODOLOGY

1. TextImport

The dataset is available in the excel file format. Using EnterpriseMiner, the Excel file is

importedusingtheFileImportNode.Thefileimportnodeconvertstheexcelfileformatto

SASfileformat.SASformatfilecanbeusedforfurtheranalysis.Metadatanodeisusedto

restrictonlytheIDandTweetsasinputsfortheadvancestudy.

2. TextParsing

Afterimportingandselectingvariables,TextParsingnodeisusedtoidentifytermsandtheir

instancesinthedata,whichcontainstext.TheresultsofTextParsingnodedisplaythemost

frequentlyoccurringterm,thenumberofdocumentsithasoccurredin,andthetermsthat

arerarelyused.Usually,thetermsthatareusedmoderately,aretheonesthatarethemost

helpfulinexplorationandmodeling.


Page5of18

The most frequently used term is demonetization which has a total frequency of 7,534

(3,854+3,680) since it has appeared twice. It has appeared in total 7,224 (3,849+3,375)

documents. Similarly, the next most frequently used terms are app (1,686), support(1,086),

narendramodi(928),people(866),bank(767).

3. TextFiltering

AText Filter node is added to the Text Parsing node to eliminate the terms that occur leastnumberoftimesinallthedocumentsandtoperformspellchecktosuggestpotentialsynonyms.


Page6of18

The results of the Text Filtering node show that terms like https, t, s, and have, have been

droppedintheTermstable,astheydonotaddanymeaningtotheanalysis.OnlytheTermsthat

addmeaningtotheanalysisofdemonetizationareretained.


Page7of18

TheTextFilternode,autocorrectsthemisspelttermslike‘hav’iscorrectedto‘have’,‘tertorists’

iscorrectedto‘terrorists’and‘oppositio’iscorrectedto‘opposition’.

Thenodealso,groupsthesynonymstogetherbasedonthesynonymsfilethatisimportedorthe

terms are manually dragged and dropped into each other. For example, terms like

‘demonization‘, ‘demoditization‘, ‘demonitization‘, and ‘dmonetization‘ are all grouped into

‘demonetization‘bytheTextFilternode.

ConceptLinks

ConceptLink,displaysthetermsthatarehighlyassociatedwiththeselectedtermintheTerms

table.Theselectedtermissurroundedbythetermsthatcorrelatethestrongestwiththeselected

term.Thewidthofthelinkdepictsthestrengthofassociationbetweenthetwoterms.


Page8of18

The above concept link is for the highest frequency term, Demonetization. The term

demonetization is associatedwithbank (associatedwithdemonetizationof currency), survey

(associatedwithdemonetizationsurvey),vote(associatedwithvotingfordemonetizationsurvey

app),modisurvey(associatedwithdemonetizationsurveyappinitiatedbyModi),rt(associated

with retweets about demonetization), mounting misery (associated with misery due to

monetization)support (associatedwith thesupportofpeople fordemonetization)opposition

(associatedwithoppositionparty’sviewwithrespecttomonetization).Outofallthetermsthat

Demonetizationisassociatedwith,itisstronglyrelatedwithsupport,opposition,modisurvey

and vote. The associated terms depict positive (support, rt) as well as negative (opposition,

misery)responsesofdemonetization.

Demonetization

Support

Opposition

Bank

Survey

Vote

ModiSurvey

rt

MountingMisery


Page9of18

TheaboveconceptlinkisforthetermSupport,whichisstronglyassociatedwithDemonetization.

The important terms associated with the term support are demonetization strategy, party,

decision,narendramodi,demonetization,question,people,andfeedback.Thetermsupportis

equallystronglyassociatedwithalltheaboveterms.Thisconceptlinkdepictsthatthefeedback

ofmajoritypeopleandparty, is supporting thedemonetization strategyordecision takenby

NarendraModi.

Support

Feedback

DemonetizationStrategy

Decision

Party

Demonetization

NarendraModi

question

People


Page10of18

Theaboveconcept link is forthetermPeopleandthe importanttermsassociatedwith itare

demonetization strategy, support, decision, narendra modi, question, back, strategy, and

feedback.Thetermpeopleisstronglyassociatedonlywithstrategy,feedback,narendramodiand

decisionterms.Thisconceptlinkalso,depictsthatthefeedbackofmajoritypeopleistosupport

thedemonetizationstrategyordecisiontakenbyNarendraModi.

People

Feedback

Strategy

Decision

Back


Narendra Modi

question

Support


Page11of18

TheaboveconceptlinkisforthetermFeedbackandtheimportanttermsassociatedwithitare

strategy, people, demonetization strategy, support, question, and demonetization. The term

feedbackisstronglyassociatedonlywithstrategy,people,demonetizationstrategy,support,and

questionterms.Thisconceptlinkalso,depictsthatthefeedbackofmajoritypeopleistosupport

thedemonetizationstrategy.

Feedback

People

Demonetization


Strategy

question

Support


Page12of18

Theaboveconcept link is for the termResultand the important termsassociatedwith itare

demonetization,decide,bypolls,modi,demonetizationstrategy,rt,bright,andfuture.Theterm

resultisequallystronglyassociatedonlywithalltheaboveterms.Thisconceptlinkalso,depicts

thattheresultofthesurveyisthatmodi’sdemonetizationdecisionwillleadtoabrightfutureis

retweetedmanytimes.

4. TextClustering

ATextClusternode is added to the text filteringnode. The text clusteringnode clusters the

documentsintodisjointedsetsofdocumentsandreportsondescriptivetermsofthoseclusters.

Hierarchicalclusteringalgorithmisusedtogroupclustersintoatreehierarchy.Theapproachof

hierarchical clustering relies on singular value decomposition (SVD) to transform original

weighted,term-documentfrequencymatrixintoadensebutlowdimensionalrepresentation.

Result

Future

Bypolls

Decide

Demonetization


Modi

rt

Bright


Page13of18

Theaboveplot,showsthedistancebetweentheclusters.All theclustersaredistributedwell

apart. Thepie chartbelow, shows thedistributionof thecluster frequencies.Apart from the

clusternumber14and clusternumber16 the frequenciesarewelldistributedamongall the

clusters.


Page14of18

Cluster

ID

DescriptiveTerms Frequency Percentage ExampleTweets

8 Demonetizationmove+

blackmoney+black+

government+govt+modi

+money+move

658 8% Demonetizationis

government’smoveagainst

blackmoney

13 Rt@shashitharoor+big+

cash+country+

demonetization+duty+

time+understand

774 10% ShashiTharoor’stweets

retweeted,thatPM’sduty

tounderstandandaddress

parliamentabout

demonetization

14 Thirdsuchincident+bank

+demonetization+lakh+

paytm+questionclearly

critical

1584 20% 50lakhcustomers

respondedin24hoursand

90%support

demonetization.Waspaytm

informedabout

demonetization?40lakhs

lootedfromabank,third

incidentsince

demonetization.

16 Oppositionmps+

pmoindiaaddress

parliame+

rt@drkumarvishwas+

1053 13% OppositionMPssupporting

demonetization.Demanding


Page15of18

rt@shashitharoor+

address+govt+line

affected

PMofIndiatoaddress

parliament.

19 Abov+atm+byelection+

people+state+support

atmsippatel

347 4% DelayreplenishingATMs.By

electiontwodaysafter

demonetizationhasproved

support.

20 Debate+decide+

demonetization+Gandhi

+impact+know+modi+

note

950 12% Easyexchangetonew2k

note.RahulGandhi

(opposition)issuewith

Demonetization.

22 Daughter’swedding+

fundshortage+daughter

+fund+life+prob+tweet

ends

442 6% Manendslifeduetofund

shortagefordaughter’s

wedding.

25 Watchbriefing+

demonetization+effect

+happiness+keep+

pmoindia+stuff

announcements

699 9% Happinessabout

demonetizationbut

announcementnotmadeat

parliament,sowatch

briefing

26 Modisurvey+arunjaitley

+blackmoney+decision+

flight+move+response

+survey

776 10% Modi’sdemonetization

decisionsurveyshowed

positiveresponse


Page16of18

Theabovetextclustersshowspositive,negativeaswellasneutraltweettextreviewanalysis.

Outofthetenclusters,sixclustersshowpositivereviews.

5. TextTopic

TextTopicnodeisconnectedtotheTextParsingnodetocombinethetermsintotopicssothat

theycouldbeanalyzed further.A listof topics iscreatedtoestablishcombinationsofwords,

whichcouldbeofinterestinanalyzing.

Topic Number

of

Terms

Numberof

Documents

ExampleTweets

Thirdsuchincident,third,

incident,loot,terrorist

10 542 40lakhslootedfromabank,third

incidentsincedemonetization.

Criticalquestion,

demonetizationedict,

edict,inform,rssurjewala

15 289 Waspaytminformedabout

demonetization?

Partypolitics,

nitishkumar,putting

16 552 Puttingnationoverpartypolitics,

supportdemonetizationbyPM

28 Demonetizationstrategy

hugesupport

demonetizationmove+

rt@modibharosa+back+

c-voter+narendramodi+

nation

717 9% Demonetizationstrategy

wassupportedonalarge

scale


Page17of18

nation,rt@modibharosa,

narendramodi

Hugesupport,huge,c-

voter,demonetization

move,nation

23 264 Hugesupportbypeopleandnationfor

demonetization

Daughter’swedding,

fund,shortage,daughter,

end,life

20 159 Manendslifeduetofundshortagefor

daughter’swedding

Ls,retain,byelecton,seat,

state

44 309 Byelectiontwodaysafter

demonetizationhasprovedsupport.

rt@drkumarvishwas,

demonetzation,people,

rt,huge,support

13 350 Hugesupportbypeoplefor

demonetization

Join,corruptfreeindia,

walk,nationalist,ma

31 189 Jointhewalktowardscorruptionfree

India

Theabovetexttopicsshowpositive,negativeaswellasneutraltweettextreviewanalysis.Out

oftheeighttopics,fivetopicsshowpositivereviews.

CONCLUSION

Radicalchangesarealwaysfacedwithresistanceofsomekind,caseinpointbeingthepolicyto

implement demonetization. The goal of this paper was to analyze the tweets by the Indian

citizensaboutdemonetizationandtheirperceptiontowardsthepolicy,byusingTextMining.The

results of Concept Links, in general, show a positive feedback to the demonetization

implementationpolicy.Similarly,theresultsofouranalysisoftweetsusingTextClusteringand

TextTopicsrevealthatdemonetizationwaspositivelysupportedbyalargenumberofpeople.


Page18of18

OuroverallanalysisshowedthatavastmajorityofIndiansacceptedthedemonetizationpolicy

positivelywhilesomeofthemexpressedtheirdispleasureoverit.

BIBLIOGRAPHY

[1]SASInstituteInc.2012.GettingStartedwithSAS®TextMiner12.1.Cary,NC:SASInstituteInc.

https://support.sas.com/documentation/onlinedoc/txtminer/12.1/tmgs.pdf

[2]Staff,I.(2017).Demonetization.[online]Investopedia.Availableat:

http://www.investopedia.com/terms/d/demonetization.asp?ad=dirN&qo=investopediaSiteSear

ch&qsrc=0&o=40186[Accessed5Apr.2017].

[3] “Sentiment Analysis of demonetization of 500 & 1000 rupee banknotes by Indian

Government”byPrabhsimranSingh,RavinderSinghSawhney,KaranjeetSinghKahlon

[4]“Demonetization:RetrospectiveAnalysis,”byMohasinA.Tamboli

[5]“DemonetizationandCompleteFinancialInclusion”byS.VijayKumarandT.ShivaKumar

[6]“DemonetizationanditsimpactonEmploymentinIndia,”byDr.PawanKumar

[7] Kaggle.com. (2017). Demonetization in India Twitter Data | Kaggle. [online] Available at:

https://www.kaggle.com/arathee3/demonetization-in-india-twitter-data [Accessed 10 Feb.

2017].