Upload File

sentiment analysis through the use of unsupervised deep ......ibm. outline •the problem ... latent...

  • Home
  • Documents
  • Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia
19
Sentiment Analysis Through the Use of Unsupervised Deep Learning S7330 Monday 8 th May 2017 GPU Technology Conference A. Stephen McGough, Noura Al Moubayed Durham University, UK

Upload: others

Post on 21-Jul-2020

2 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

SentimentAnalysisThroughtheUseofUnsupervised

DeepLearningS7330

Monday8th May2017GPUTechnologyConference

A.StephenMcGough,Noura AlMoubayed

DurhamUniversity,UK

Page 2: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

IntelParallelComputingCentre

Page 3: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

Outline

• TheProblem• UnsupervisedDeepLearningModel• TextProcessingforTopicModelling• DetectingAnomaliesinText• SentimentClassification

Page 4: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

TheProblem

• “90%ofallthedataintheworldhasbeengeneratedoverthelasttwoyears”…IBM

• “85%ofworldwidedataisheldinun-structuredformats”…BerryandKogan

• Howcanweunderstandit?….orbetterstillmakeuseofit?• Howcanwedeterminethemostpertinentinformation?…andthenactonit?• Howcanwefindtheneedleifwearenotsurewhatitlookslikeorwhathaylookslike?

• "Withoutlabelling,youcannottrainamachinewithanewtask”…IBM

Page 5: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

Outline

• TheProblem• UnsupervisedDeepLearningModel• TextProcessingforTopicModelling• DetectingAnomaliesinText• SentimentClassification

Page 6: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

DeepLearning:CategorizationandRegression

InputVectorButtheserequirelabelleddatafortraining

True False

HiddenLayer

Output

HiddenLayer

HiddenLayer

…

Page 7: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

AnomalyDetection:UnsupervisedDeepLearning

StackedDenoisingAutoencoder (SDA)- Trainedlayerbylayer- Inputiscorrupted

withnoise

Reconstruct

Construct

InputData ReconstructedInputData

vvv v v v

h h h

hhhhh

h h h h h

hh h h h

h h h h hh h h h

v v v v v v v v v v v v

OutputData

We’reallgoingtothecinemaonSaturday

We’reallgoingtothetheatreonSaturday

Lowreconstructionerror

Page 8: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

AnomalyDetection:UnsupervisedDeepLearning

StackedDenoisingAutoencoder (SDA)- Trainedlayerbylayer- Inputiscorrupted

withnoise

Reconstruct

Construct

InputData ReconstructedInputData

vvv v v v

h h h

hhhhh

h h h h h

hh h h h

h h h h hh h h h

v v v v v v v v v v v v

OutputData

BuyViagrafromouronlinemedsstore

Bythevalleyouroverallmedalsstore

Highreconstructionerror

Page 9: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

Outline

• TheProblem• UnsupervisedDeepLearningModel• TextProcessingforTopicModelling• DetectingAnomaliesinText• SentimentClassification

Page 10: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

ProbabilisticTopicModelling

• Unsupervisedanalysisoftext• Toomanydocumentstolabelmanually

• Allowsustouncoverautomaticallythemesthatarelatentinacollectionofdocuments• Samewordsmayhavedifferentmeaningsdependingontheirco-occurrencewithotherwordsinadocument• Statisticallyidentifythetopicsinasetofdocuments• Whichtopicsappearinwhichdocuments

• Statisticallyidentifywordsintopics• Whichwordsco-occurinadocumentcontainingtopicX

• Documenttransformedtofeaturevectoroftopics

Thisreportpresentsaproofofconceptofourapproachtosolveanomalydetectionproblemsusingunsuperviseddeeplearning. TheworkfocusesontwospecificmodelsnamelydeeprestrictedBoltzmannmachinesandstackeddenoising autoencoders.Theapproachistestedontwodatasets:VASTNewsfeedDataandtheCommissionforEnergyRegulationsmartmeterprojectdatasetwithtextdataandnumericdatarespectively.Topicmodelingisusedforfeaturesextractionfromtextualdata. Theresultsshowhighcorrelationbetweentheoutputofthetwomodelingtechniques.Theoutliersinenergydatadetectedbythedeeplearningmodelshowaclearpatternovertheperiodofrecordeddatademonstratingthepotentialofthisapproachinanomalydetectionwithinbigdataproblemswherethereislittleornopriorknowledgeorlabels.Theseresultsshowthepotentialofusingunsuperviseddeeplearningmethodstoaddressanomalydetectionproblems. Forexampleitcouldbeusedtodetectsuspiciousmoneytransactionsandhelpwithdetection ofterroristfundingactivitiesoritcouldalsobeappliedtothedetectionofpotentialcriminalorterroristactivityusingphoneordigitil).

Topics

Page 11: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

TopicModelling:LatentDirichlet Allocation(LDA)

Topics

RandomlySelecttopics

US Govt Data Shows RussiaUsed Outdated Ukrainian PHPMalware

The United States governmentearlier this year officiallyaccused Russia of interferingwith the US elections. Earlierthis year on October 7th, theDepartment of HomelandSecurity and the Office of theDirector of National Intelligence

RandomlySelectwordsfromtopics

Page 12: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

TopicModelling:LatentDirichlet Allocation(LDA)• Eachdocumentcontainsaproportion(θ)ofeachtopic

• Proportionmaybezero• Theprobabilityofeachwordappearinginatopic(β)• LDAassumesdocumentswereconstructedvia:Wordsindocument=NPerdocumenttopicproportions=θ =Dirichlet function(α)ForeachoftheNwordswn:

⎻ Chooseatopiczn ∼Multinomial(θ)⎻ Chooseawordwn fromtopiczn (basedonprobabilitiesβ)

• Needtodetermineα,β,θ givenwehavethedocuments• Solveusingvariational BayesianmethodsorGibbssampling

α

θ

z

β

w

Topic

Document

WordsinDocum

ent

Page 13: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

Outline

• TheProblem• UnsupervisedDeepLearningModel• TextProcessingforTopicModelling• DetectingAnomaliesinText• SentimentClassification

Page 14: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

Example:AnomalyIdentificationSPAMandHAMinSMS

• TrainLDAusingthelabelleddataonlye.g.SPAM

• Unlabeled(HAM+SPAM)usedtotrainSDA

• Auto-identificationofSPAMfromHAMinSMSmessages

Page 15: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

Example:AnomalyIdentificationSPAMandHAMinSMS

Page 16: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

Outline

• TheProblem• UnsupervisedDeepLearningModel• TextProcessingforTopicModelling• DetectingAnomaliesinText• SentimentClassification

Page 17: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

MultipleLabelsExample:DeepLearningforSentimentClassification• Trainananomalydetectoroneachpolarityofsentiment• Canhavemany• E.g.positive&negativeview

• Generateoverallreconstructionerror(ORE)foreachlabel• UsesimpleclassifieronOREstoidentifyclass

(SDA)

(LDA)

(IDA)

Page 18: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

PerformanceAnalysis

80 82 84 86 88 90 92 94 96 98 10080

82

84

86

88

90

92

94

96

98

100

BestinLiterature(%)

OurBestA

pproach(%

)

Problem BestLiterature

BestOurApproach

SMSSpam 97.64 97.92MDSD-B(%) 80.4 99.6MDSD-D(%) 82.4 99.4MDSD-E(%) 84.4 99.4MDSD-K(%) 87.7 99.45Movie-Sub 93.6 90.72Movie-Rev1 82.9 87.28Movie-Rev2 90.2 98.05IMDB 91.22 85.61

Page 19: Sentiment Analysis Through the Use of Unsupervised Deep ......IBM. Outline •The Problem ... Latent DirichletAllocation (LDA) Topics Randomly Select topics US Govt Data Shows Russia

Summary

• StackedDenoising Autoencoder allowustospotanomalieswithinunlabeleddata• ProbabilisticTopicModellingallowsustolookatdocumentsatahigherlevelthanjustwords•Whenwehavelabelswecantrainonthelabelsto:• Identifysentiment• Classifythedata

WeArerecruiting:- 2 PostDoc (MachineLearning/NLP)- 1PostDoc (ParallelProgramming)- AlwayslookingforgoodPhDCandidates

[email protected]@dur.ac.uk


Generalized Correspondence-LDA Models (GC-LDA) for ... · The GC-LDA and Correspondence-LDA models are extensions of Latent Dirichlet Allocation (LDA) [3]. Several Bayesian methods

Realistic investigations of correlated electron systems ...scalettar.physics.ucdavis.edu/publications/lda+dmft.pdf · Article LDA LDA+DMFT LDA+U 0U=W∞ metal insulator Fig. 1 With

Fast sampling for LDA William Cohen. MORE LDA SPEEDUPS FIRST - RECAP LDA DETAILS

The World of ALDA - European Committee of the Regions · lda prijedor (bih) lda zavidoviĆi (bih) lda montenegro (mne) lda central and southern serbia (rs) lda subotica (rs) lda kosovo

1:21CR76JJM-LDA Case 1:21-cr-00076-JJM-LDA Document 3

Twitter LDA

Linear Discriminant Analysis (LDA)shireen/pdfs/tutorials/Elhabian_LDA09.pdf · LDA Objective •The objective of LDA is to perform dimensionalityreduction… –Sowhat,PCAdoesthisL…

Explore the Evolution of Development Topics via On-Line LDAhome.cse.ust.hk/~jhuao/pdf/p555-hu.pdf · A. LDA Latent Dirichlet Allocation (LDA) is a probabilistic tech-nique that has

Spatial LDA

Generalized Correspondence-LDA Models (GC-LDA) for ...pilab.psy.utexas.edu/publications/Rubin_NIPS_2016.pdf · Generalized Correspondence-LDA Models (GC-LDA) for Identifying Functional

LDA - Spinea Oy · LDA LDA Air cooled water chillers with axial fans LDA water chillers are efficient, low-noise products designed for medium to large applications. They are suitable

LDA VisitorManagementPlanningToolkit

lda Documentation - media.readthedocs.org · lda Documentation lda implements latent Dirichlet allocation (LDA) using collapsed Gibbs sampling. lda is fast and can be installed without

Mining the Social Web - Leibniz Universität Hannover · User recommendation ... (LDA) to generate topics used as features for classification. - Sentiment words ... follows, replies

LDA: Extracting Topics from Tweets and Webpages...LDA: Extracting Topics from Tweets and Webpages Sarunya Pumma Xiaoyang Liu CS5604 Information Retrieval Instructor: Dr. Edward Fox

LDA Katalog

Web-Mining Agentsmoeller/Lectures/... · Latent DirichletAllocation • Document = mixture of topics (as in pLSI), but according to a Dirichletprior – When we use a uniform Dirichletprior,

SeCo-LDA: Mining Service Co-Occurrence Topics for ... · LDA reaches a 5 percent higher Mean Average Preci-sion (MAP) value than baselines, when recommend-ing related services for

LDA DESIGN

LDA Brochure

Edit lda rslideshow

goalparadistjudiciary.gov.ingoalparadistjudiciary.gov.in/stat/sscjm.pdfUDA UDA UDA UDA LDA LDA LDA LDA LDA LDA LDA 21-02-1986 29-05-2015 29-05-2015 19-05-1998 21-02-1986 29-05-2015

pLSA and LDA

Representing Topics Using Images - aclweb.orgbelling Of Topics algorithm computes the similarity between LDA inferred topics and topics in a topics tree by computing scores using six

Using LDA - Rutgers Universityyhung/HW586/LDA/Lecture LDA and R.pdf · 1 Using LDA Randy Julian Lilly Research Laboratories Linear Discriminant Analysis Used in Supervised Learning!

Linear discriminant analysis, two classes Linear ...gn.dronacharya.info/CSEDept/Downloads/Question... · LDA vs. PCA • Limitations of LDA • Variants of LDA • Other dimensionality

WT-LDA: User Tagging Augmented LDA for Web Service Clustering

Lda Risklab Madrid

EfficientFeaturesforMovie RecommendationSystems759691/FULLTEXT01.pdf · i.e. “topics” from documents. LDA, a model based on topic modeling, shows good results in both document

Latent dirichletallocation presentation

Generalized Correspondence-LDA Models (GC-LDA) for Identifying ...clcl/Papers/Neurotopics_NIPS_2016_Final_Plus... · Generalized Correspondence-LDA Models (GC-LDA) for Identifying

Lda Bylaws

LDA Portfolio

Lda Tutorial

LDA v. LSA: A Comparison of Two Computational Text ...ceur-ws.org/Vol-1815/paper4.pdf · topics from LDA results [31, 32]. More recently, several researchers have investigated variants