natural language processing
DESCRIPTION
Natural Language Processing. Daniele Quercia Fall, 2000. Presentation Outline. Introduction to the Natural Language Processing (NLP) Applications of NLP Discussion. Introduction to Natural Language Processing (NLP). Representation of the input meaning. Natural Language (English). - PowerPoint PPT PresentationTRANSCRIPT
Natural Language Natural Language ProcessingProcessing
Daniele QuerciaDaniele Quercia
Fall, 2000Fall, 2000
Presentation Presentation Outline
• Introduction to the Natural Language Processing (NLP)
•Applications of NLP
•Discussion
Introduction to Natural Introduction to Natural Language Processing Language Processing
(NLP)(NLP)
NLPNLP
•What is Natural Language processing (NLP)?
•“The Natural Language Processing (NLP) is the engineering of systems that process or analyse written or spoken natural language.”
NLP System
(computational model)
Natural Language(English)
Representation
of the input meaning
Related disciplinesRelated disciplines
Linguistics
Computer Science
Psychology Neuroscie
nce Natural Language Processing
Knowledge of languageKnowledge of language
These language processing applications use knowledge of language
Unix wc program› total number of bytes, words, lines in a text file.
What is a word?
A possible file: PIPPO.TXT
Hallo Pippo. Ciao Pippo. Auf widersehen Pippo.
Knowledge of language - 2Knowledge of language - 2
Types of knowledge used in NLP
Phonetics and Phonology Morphology Syntax Semantics Pragmatics Discourse
NLP can be viewed as resolving ambiguity at one of these levels.i
Knowledge of language - 3Knowledge of language - 3
Example: HAL computer system.capable of interacting with humans
Stanley Kubrik’s film 2001: A Space Odissey
Conversation between HAL and Dave
Knowledge of language - 4Knowledge of language - 4
Example(cont’d): HAL computer system.capable of interacting with humans
Stanley Kubrik’s film 2001: A Space Odissey
Dave: Open the pod bay doors, right now. HAL: I’m sorry Dave, I’m afraid I can’t do that.
Knowledge of language - 5Knowledge of language - 5
Phonetics and Phonology
The study of linguistic soundsHow word are pronounced in colloquial speech
Syntax
The study of the structural relationship between wordsbut syntax don’t tell us much about the meaning …
e.g. :I saw the Grand Canyon flying to New YorkTime flies like an arrow
Knowledge of language - 6Knowledge of language - 6
Semantics
The study of meaning
e.g. :
I work for J.F. Zich and he’s a good boss which is a good company
Pragmatics
The study of how language is used to accomplish goals
e.g.
Can youn tell me what time is it?Could I have the salt?
Knowledge of language - 7Knowledge of language - 7
Morphology
The study of the meaningful components of words. e.g. :Kick, kicks, kicked, kicking Arm, army
Discourse
The study of linguistic linguistic units larger than a single utterance
Applications of NLPApplications of NLP
Applications of NLP Applications of NLP
· Text-based applications
· Dialogue based applications
Speech recognitio
n
Machine translatio
n
Information retrieval
Applications of NLP Applications of NLP
Machine translationAutomatic machine translation
translates texts from one language to another
Windows 98 must be delivered either without Internet Explorer or in connection with the sailor of the competitor Netscape
Windows 98 deve essere distribuito senza Internet Explorer o insieme al navigatore del concorrente Netscape
Important constraint
The text needs to be restricted
to a limited range of subjects
Applications of NLP Applications of NLP
Machine translation even the best system produces poor translation… … but Machine-assisted translation uses a computer to help a translator.
Applications of NLP Applications of NLP
Information retrievial
concerns the retrieving of relevant information from databases.
Applications of NLP Applications of NLP
Information retrievial (cont’d)
Inverted index
Retrieval depends only on how often each word appears in a document
Should we treat all words equally?
Applications of NLP Applications of NLP
Information retrievial (cont’d)
Stop list: words to ignoreMany frequent words in English are function words, which are useless.
Applications of NLP Applications of NLP
Speech recognition
Speech generation is comparatively easy, but recognition is hard !
The acoustic signal is highly ambigousDisambiguation using statistics
If …they buy… occurs more frequently than …they by …, it will chosen
ConclusionsConclusions
Conclusions Conclusions
Computers can process natural language in a variety of interesting ways
A computer can’t do anything close to “understanding” language …
NLP is almost never error-free
A very special thanks to …A very special thanks to …