MiningKnowledgeGraphsfromTextWSDM2018
JAY PUJARA,SAMEER SINGH
TutorialOverview
Part3:GraphConstruction
Part1:KnowledgeGraphs
Part4:CriticalAnalysis
Part2:KnowledgeExtraction
2
TutorialOutline1. KnowledgeGraphPrimer [Jay]
2. KnowledgeExtractionPrimer [Jay]
3. KnowledgeGraphConstructiona. ProbabilisticModels [Jay]
CoffeeBreak
b. EmbeddingTechniques [Sameer]
4. CriticalOverviewandConclusion [Sameer]
3
WhatisNLP?
InformationExtraction
UnstructuredAmbiguousLotsandlotsofit!
Humanscanreadthem,but… veryslowly… can’trememberall… can’tanswerquestions
“Knowledge”
StructuredPrecise,ActionableSpecifictothetask
Canbeusedfordownstreamapplications,suchascreatingKnowledgeGraphs!
4
KnowledgeExtraction
5
JohnLennon
AlfredLennon
JuliaLennon
Liverpoolbirthplace
childOf
childOf
John was born in Liverpool, to Julia and Alfred Lennon.
John was born in Liverpool, to Julia and Alfred Lennon.Person Location Person Person
NNP VBD VBD IN NNP TO NNP CC NNP NNP
Lennon..JohnLennon...
Mrs.Lennon....hismother..
hisfatherAlfredhe
thePoolNLP
InformationExtraction
Extractiongraph
Annotatedtext
Text
BreakingitDown
JohnwasborninLiverpool,toJuliaandAlfredLennon.NNP VBD VBD IN NNP TO NNP CC NNP NNP
Person Location Person PersonJohnwasborninLiverpool,toJuliaandAlfredLennon.
Lennon..JohnLennon...
Mrs.Lennon....hismother..
hisfatherAlfredhe
thePool
Senten
ce DependencyParsing,Partofspeechtagging,Namedentityrecognition…
Documen
t
Coreference Resolution...
JohnLennon
AlfredLennon
JuliaLennon
Liverpoolbirthplace
childOf
childOf
spouse
Inform
ation
Extractio
n
Entityresolution,Entitylinking,Relationextraction…
6
TaggingthePartsofSpeech
JohnwasborninLiverpool,toJuliaandAlfredLennon.NNP VBD VBD IN NNP TO NNP CC NNP NNP
7
Nounsareentities
Verbsarerelations
• CommonapproachesincludeCRFs,CNNs,LSTMs
DetectingNamedEntities
JohnwasborninLiverpool,toJuliaandAlfredLennon.Person Location Person Person
8
• Structuredpredictionapproaches• Captureentitymentionsandentitytypes
NLPannotationsà featuresforIE
Combinetokens,dependencypaths,andentitytypestodefinerules.
Argument1 Argument2,Person Organization
DT CEO of
appos nmod
casedet
BillGates,theCEOofMicrosoft,said…Mr.Jobs,thebrilliantandcharmingCEOofAppleInc.,said…… announcedbySteveJobs,theCEOofApple.… announcedbyBillGates,thedirectorandCEOofMicrosoft.… musedBill,aformerCEOofMicrosoft.andmanyotherpossibleinstantiations…
9
Within-documentCoreference
JohnwasborninLiverpool,toJuliaandAlfredLennon.
Mrs.Lennon....hismother..
hisfather
Alfred
hethePoolLennon..
JohnLennon...
He…
10
• Pairwisemodelforeachnoun/pronoun• Canconsolidateinformation,providecontext
EntityResolution&Linking
...during the late 60's and early 70's, Kevin Smith worked with several local...
...the term hip-hop is attributed to Lovebug Starski. What does it actually mean...
The filmmaker Kevin Smith returns to the role of Silent Bob...
Nothing could be more irrelevant to Kevin Smith's audacious ''Dogma'' than ticking off...
Like Back in 2008, the Lions drafted Kevin Smith, even though Smith was badly...
... backfield in the wake of Kevin Smith's knee injury, and the addition of Haynesworth...
... The Physiological Basis of Politics,” by Kevin Smith, Douglas Oxley, Matthew Hibbing...
11
EntityNames:TwoMainProblems
DifferentNamesforEntities
InconsistentReferencesMSFT,APPL,GOOG…
Typos/MisspellingsBaarak,Barak,Barrack,…
NickNamesBamBam,Drumpf,…
EntitieswithSameName
PartialReference
Thingsnamedaftereachother
Firstnamesofpeople,Locationinsteadofteamname,Nicknames
Clinton,Washington,Paris,Amazon,Princeton,Kingston,…
SametypeofentitiessharenamesKevinSmith,JohnSmith,Springfield,…
12
EntityLinkingApproach
13
Washington drops 10 points after game with UCLA Bruins.
CandidateGenerationWashingtonDC,GeorgeWashington,Washingtonstate,LakeWashington,WashingtonHuskies,DenzelWashington,UniversityofWashington,WashingtonHighSchool,…
EntityTypesWashingtonDC,GeorgeWashington,Washingtonstate,LakeWashington,WashingtonHuskies,DenzelWashington,UniversityofWashington,WashingtonHighSchool,…
LOC/ORG
CoreferenceWashingtonDC,GeorgeWashington,Washingtonstate,LakeWashington,WashingtonHuskies,DenzelWashington,UniversityofWashington,WashingtonHighSchool,…
UWashington,Huskies
Coherence UCLABruins,USCTrojans
WashingtonDC,GeorgeWashington,Washingtonstate,LakeWashington,WashingtonHuskies,DenzelWashington,UniversityofWashington,WashingtonHighSchool,…
Vinculum,Ling,Singh,Weld,TACL(2015)
InformationExtraction
JohnLennon
AlfredLennon
JuliaLennon
Liverpoolbirthplace
childOf
childOf
spouse
InformationExtraction
NNP VBD VBD IN NNP TO NNP CC NNP NNP
JohnwasborninLiverpool,toJuliaandAlfredLennon.Person Location Person Person
Lennon..JohnLennon...
Mrs.Lennon....hismother..
hisfatherAlfredhe
thePool
14
InformationExtraction
3CONCRETESUB-PROBLEMS
Definingdomain
Learningextractors
Scoringthefacts
3LEVELSOFSUPERVISION
Supervised
Semi-supervised
Unsupervised
15
Effectofsupervisiononextractions
16
Precision,Humanefforts
Recall,Speed
InformationExtraction
3CONCRETESUB-PROBLEMS
DefiningdomainLearningextractors
Scoringthefacts
3LEVELSOFSUPERVISION
Supervised
Semi-supervised
Unsupervised
17
DefiningDomain:Manual
18
Everything
Animals
Mammals Reptiles
Food
Fruits Vegetables
Subset
Disjoint
[Toward an Architecture for Never-Ending Language Learning, Carlson et al. AAAI 2010]
consumes
DefiningDomain:Semi-automatic• Subsetoftypesaremanuallydefined
• SSLmethodsdiscovernewtypesfromunlabeleddata
19
Everything
Animals
Mammals Reptiles
Food
Fruits Vegetables Beverages
Location
Country City
[ExploratoryLearning, Dalvietal.,ECML2013][HierarchicalSemi-supervisedClassificationwithIncompleteClassHierarchies,Dalvietal.,WSDM2016]
Everything
Animals
Mammals Reptiles
Food
Fruits Vegetables
DefiningDomain:Automatic
•Anynounphraseisacandidateentity◦ Dog,cat,cow,reptile,mammal,apple,greens,mixedgreens,lettuce,redleaflettuce,romainelettuce,iceberglettuce…
•Anyverbphraseisacandidaterelation◦ Eats,feastson,grazes,consumes,
20[OpenInformationExtractionfromtheWeb, Banko etal.,IJCAI2007]
InformationExtraction
3CONCRETESUB-PROBLEMS
Definingdomain
LearningextractorsScoringcandidatefacts
3LEVELSOFSUPERVISION
Supervised
Semi-supervised
Unsupervised
21
LearningExtractors• Supervised:highprecisionpatterns•<PERSON>playsin<BAND>
• Semi-supervised:Bootstrappingtolearnpatterns•Createexamples(JohnLennon,Beatles), findpatterns•Manuallycorrectincorrectpatterns
• Unsupervised:clusterphraseswithconstraints• Identifycandidateverbphrases,findcandidatearguments,clusterbyNERtypes
InformationExtraction
3CONCRETESUB-PROBLEMS
Definingdomain
Learningextractors
Scoringcandidatefacts
3LEVELSOFSUPERVISION
Supervised
Semi-supervised
Unsupervised
23
Scoringthecandidatefacts• HumandefinedscoringfunctionorScoringfunctionlearntusingsupervisedMLwithlargeamountoftrainingdata{expensive,highprecision}
• Smallamountoftrainingdataisavailablescoringrefinedovermultipleiterationsusingbothlabeledandunlabeleddata
• Completelyautomatic(Self-training)Confidence(extractionpattern)∝ (#uniqueinstancesitcouldextract)Score(candidatefact)∝ (#distinctextractionpatternsthatsupportit){cheap,leadstosemanticdrift}
Impactofearlysupervision
Definingdomain
Extractorsforeachrelationofinterest
Scoringthecandidatefacts
25
Putsconstraintsonthespaceofpossiblytrue
extractionsEarlyremovalofnoisyextractionpatterncanavoidsemanticdriftin
laterstages
Enablesinheritanceandmutualexclusionatextractorlevel
Domainexpertiseneeded
Effectofsupervisiononextractions
26
Precision,Humanefforts
Recall,Speed
27
Definingdomain
Learningextractors
Scoringcandidatefacts
Fusingextractors
ConceptNet
NELL
KnowledgeVault
OpenIE
IEsystemsinpractice
Heuristicrules
Classifier
KnowledgeExtraction:KeyPoints• BuiltonthefoundationofNLPtechniques• Part-of-speechtagging,dependencyparsing,namedentityrecognition,coreferenceresolution…
• Challengingproblemswithveryusefuloutputs
• InformationextractiontechniquesuseNLPto:• definethedomain• extractentitiesandrelations• scorecandidateoutputs
• Trade-offbetweenmanual&automaticmethods
28
