natural language processing - columbia universitykathy/nlp/2017/classslides/... · • bhavana...
TRANSCRIPT
CS4705NaturalLanguageProcessingFall2017
ProfessorKathyMcKeown
1
ClassSizeandIn-classDiscussion• Piazzatointeractinclass.Forin-classandvideostudents• Regularintervals:pausetoanswerques@ons• Iwillalsoaskques@onsofyou
• Answerswillbeshownontheslide• Makingyournameshowntofellowstudents• Ontheinstructorsystems,wewillseewhoposedaques@on->assigncreditforinterac@on
• Nopenaltyforwronganswerexceptforinappropriatecontent
• Mayuseaninteres@nganswertofurtherdiscussion• In-classpar@cipa@onwillcounttowardsyourgrade 2
DownloadPiazzaApponyourPhone
3
ClassPolicyonElectronics• CellphoneinclassOKandneededforPiazzainterac@on• Keeplaptopsclosedordon’tbringtoclass
4
Today• WhatisNLP?• ClassLogis@cs• Whatwillwecover• Helpfulbackground• Classhomeworkandexams
5
Whatwillwestudyinthisclass• Howcanmachinesunderstandandgeneratelanguage?• Examplesdrawnfromnaturallyoccurringcorpora• Theoriesaboutlanguage• Algorithms
• Sta@s@calmethods• Applica@ons
6
KnowledgeNeeded• Morphology:wordforma@on• Syntax:wordorder• Seman@cs:wordmeaningandcomposi@on• Pragma@cs:Influenceofcontextandsitua@on
Goal:discoverwhatthespeakermeant 7
Morphology• Importantforsearch,machinetransla@on,summariza@on• UnionAc9vi9esinNewYork• Singular/plural• Union/unions• Ac@vity/ac@vi@es
• Otherlanguagesaremorphologicallyrich• Arabic:definiteembeddedintheword(cli@cs):Theunion(Al+)vs.aunion,unions• German:casepartoftheword(subjvsobj)
• Arethereexamplesinyourlanguage? 8
Responses
9
Newsarticletitles• Stud@resout• Eyedropsoffshelf• Teacherstrikesidlekids• Drunkgetsninemonthsinviolincase• Enragedcowinjuresfarmerwithax• BanonnudedancingonGovernor’sdesk• Hospitalsaresuedbysevenfootdoctors• Redtapeholdsupnewbridges• Governmentheadseeksarms• Pa@entatdeath’sdoor–doctorspullhimthrough• InAmericaawomanhasababyevery15minutes 10
Syntax• Partofspeechtagging:isawordanoun,verb,adverb,adjec@ve,etc?• Parsing• Iden@fyingcons@tuents• NP:KathyMcKeown,amaninthepark• VP:waslookingup,hadrisen
• Iden@fyingsubjectsandobjects• BillhitJohnvsJohnhitBill
• Modifica@on• Johnsawthemanintheparkwithatelescope 11
PartofSpeechtagging• Stud9resout• Tires:anounoraverb?• Eyedropsoffshelf• Drops:anounoraverb?• Teacherstrikesidlekids• Strikes:anounoraverb?
12
Responses
13
ConstituentStructureandModiKication• TheproblemofPPakachmentEnragedcowinjuresfarmerwithax• [Enragedcow]injuresfarmer[withax]• [Enragedcow]injuresfarmer[withax]
14
RepresentingmodiKicationwithbrackets• [Enragedcow][injures[farmer[withax]]]• [Enragedcow][injures[farmer][withax]]]
15
ConstituentStructureandModiKication• TheproblemofPPakachmentBanonnudedancingongovernor’sdesk• [Ban]on[nudedancing][ongovernor’sdesk]• Therearetwopossiblemodifica@ons?Whatarethey?• Whichoneiscorrect?
16
Response
17
ConstituentStructureandModiKication
• TheproblemofPPakachmentBanonnudedancingongovernor’sdesk• [[Ban]on[nudedancing]][ongovernor’sdesk]• [Ban]on[[nudedancing][ongovernor’sdesk]]
18
NounnounmodiKication• Waterfountain:afountainthatsupplieswater• Waterballet:aballetthattakesplaceinwater• Watermeter:adevice(calledameter)thatmeasureswater• Waterbarometer:abarometerthatuseswater(insteadofmercury)tomeasureairpressure• Waterglass:aglassthatismeanttoholdwater
19
NounnounmodiKication:constituentstructure• Whichcons9tuentstructurebestrepresentsthemeaningofcountrysongpla9numalbum?
1. [country[song[pla9numalbum]]]2. [country[[songpla9num]album]]3. [[countrysong][pla9numalbum]]4. [[country[songpla9num]]album]5. [[[countrysong]pla9num]album]
20
Response
21
NounnounmodiKicationandheadlines• Hospitalsaresuedbysevenfootdoctors
• Hospitalsaresuedby[[sevenfoot]doctors]
• Hospitalsaresuedby[seven[footdoctors]]
22
WordMeaning• Redtapeholdsupnewbridges• Holdsup:
1. [TRANSITIVE]tosupportsomeoneorsomethingsothattheydonotfalldown• Herlegswerealmosttooshakytoholdherup.
2. [TRANSITIVE][OFTENPASSIVE]tocauseadelayforsomeoneorsomething,ortomakethemlate• SorryI’mlate,butmyflightwasheldup.
• Governmentheadseeksarms• Head:• Arms: 23
Pragmatics• Discoursecontext• Johnwenttothestore.HeboughtbreadandbuYer.
• Situa@onal(realworld)context• DayanertheCharlokesville,VAriots• Hisirresponsibleac9onstookthelifeofayoungwomanwhowasjustbeginningheradultlife.
• Commonsenseknowledge• Bostoncalledandle[amessageforJoe. 24
TheLanguageofGenres:news,journal,socialmedia,novel?
• Devasta@ng,butyetamazingstorm.IMBYjustsomebranches,tonofleaves,…ThissurpassesanyBigdaddystorm,shouldbecalledBigMama.(ComputerGuy)
• Producedbyateamof26scien@stsledbytheUniversityofNewSouthWalesClimateResearchCentre,theDiagnosisconvincinglyprovesthattheeffectsofglobalwarminghavegokenworseinthelastthreeyears.(Somervilleetal2011)
• HurricaneSandychurnedabout290milesofftheMid-Atlan@ccoastSundaynight,withtheNa@onalHurricaneCenterrepor@ngthatthemonsterstormwasexpectedtocomeashorewithnear-hurricane-forcewindsandpoten@ally"life-threatening"stormsurgeflooding.
• WhatisthemaYer?'Icried.'Awreck!Closeby!'
25
Machinelearningframework• Data(onenlabeled)• Extrac@onof“features”fromtextdata• Predic@onofoutput
26
Machinelearningframework• Data(onenlabeled)• Extrac@onof“features”fromtextdata• Predic@onofoutputWhatdataisavailableforlearning?
27
Machinelearningframework• Data(onenlabeled)• Extrac@onof“features”fromtextdata• Predic@onofoutputWhatfeaturesyieldgoodpredic9ons?
28
MachineLearningMethods• Supervised• Supportvectormachine,NaïveBayes,Logis@cregression• Sequencelabeling:HiddenMarkovModeling(HMM),Condi@onalRandomFields(CRF)• Neuralnetworks
• Unsupervised• Clustering
• Semi-supervised• Boot-strapping,self-training,co-training• Distantlearning 29
Wheredoesthedatacomefrom?• Manuallylabeled• Naturallyoccurring• Anoisy,butplen@fulsubs@tute
30
CoreNLP• Morphologicalanalysis• Partofspeechtagging• Parsing
31
Applications• Searchingverylargetextandspeechcorpora• E.g.,theweb
• Ques@onansweringovertheweb• Transla@ngbetweenonelanguageandanother:e.g.,ChineseandEnglish• Summarizingtext:e.g.,youremail,thenews,reviews• Sen@mentanalysis• Genera@ngtexts• Dialogsystems:Amtrak’sJulie
32
Logistics
33
Instructor• KathyMcKeown• Office:722CEPSR• NLPGroup• 35yearsatColumbia,FoundingDirectoroftheDataScienceIns@tute(juststeppeddown)
• Research• Summariza@on• Ques@onAnswering• LanguageGenera@on• Sen@mentanalysis• Mul@lingualapplica@ons 34
TAs• ElsbethTurcan(headTA)• DheerajKalmekolan• ApoorvKulshreshtha• RobertKwiatkowski• Fei-TzinLee• BhavanaRamachandra• SamarthTripathi
35
Background• Programming.WewillusePython• Inaddi@on,atleastone:• Ar@ficialIntelligence• Machinelearning• ProgrammingLanguagesandTranslators• Sta@s@cs 36
Syllabus• Availableat:hkp://www.cs.columbia.edu/~kathy/NLP/2017
37
Textbooks• SpeechandLanguageProcessing,2ndEdi@on,[email protected],aswellasfromAmazonandotheronlineproviders.ItisalsoonreserveintheScienceLibrary.
• NeuralNetworkMethodsforNaturalLanguageProcessingbyYoavGoldberg.Itisavailableonlinebutyoucanalsopurchasehardcopyfromthepublisher. 38
Assignments• 4homeworkassignments:3programming,1wriken
• WewillbeusingGoogleCloud• HW0:• Worth2points,butifyoudonot/cannotdoit,thisisnottheclassforyou.
• Setsupyourgooglecloudaccountproperly• Fourfreelatedays• Anerthat10%offforeachdaylate
• Midtermandfinal• Evalua@on:50%homework+40%exams+10%classpar@cipa@on(viaPiazza)
39
AcademicIntegrity• Copyingorparaphrasingsomeone'swork(codeincluded),orpermi{ngyourownworktobecopiedorparaphrased,evenifonlyinpart,isforbidden,andwillresultinanautoma@cgradeof0fortheen@reassignmentorexaminwhichthecopyingorparaphrasingwasdone.Yourgradeshouldreflectyourownwork.Ifyouaregoingtohavetroublecomple@nganassignment,talktotheinstructororTAinadvanceoftheduedateplease.Everyone:Read/writeprotectyourhomeworkfilesatall@mes.
40
ForNextClass• ReadChapters1-2ofJ&M,Chapter1ofNN• Ques@ons?(UsePiazza)
41
Questions
42