Upload File

si485i : nlp · what nlp topics did we miss? • dialogue systems • dialogue is a fascinating...

  • Home
  • Documents
  • SI485i : NLP · What NLP topics did we miss? • Dialogue Systems • Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions
24
SI486m : NLP Missing Topics and the Future

Upload: others

Post on 23-Oct-2020

0 views

Category:

Documents


0 download

Report

TRANSCRIPT

  • SI486m : NLP Missing Topics and the Future

  • Who cares about NLP?

    • NLP has expanded quickly

    • Most top-tier universities now have NLP faculty (Stanford,

    Cornell, Berkeley, MIT, UPenn, CMU, Hopkins, etc)

    • Commercial NLP hiring: Google, Microsoft, IBM,

    Amazon, LinkedIn, Yahoo

    • Web startups in Silicon Valley are eating up NLP

    students

    • Navy, DoD, NSA, NIH: all funding NLP research

    2

  • What NLP topics did we miss?

    • Speech Recognition

    3

  • What NLP topics did we miss?

    • Speech Recognition

    4

  • What NLP topics did we miss?

    • Machine Translation

    5

  • What NLP topics did we miss?

    • Machine Translation

    6

    Start at ~6min in.

    http://www.youtube.com/watch?feature=player_embedded&v=Nu

    -nlQqFCKg

  • What NLP topics did we miss?

    • Machine Translation

    • IBM Models (1 through 5)

    7

  • Machine Translation

    • How to model translations?

    • Words: P( casa | house )

    • Spurious words: P( a | null )

    • Fertility: Pn( 1 | house )

    • English word translates to one Spanish word

    • Distortion: Pd( 5 | 2 )

    • The 2nd English word maps to the 5th Spanish word

  • Distortion

    • Encourage translations to follow the diagonal…

    • P( 4 | 4 ) * P( 5 | 5 ) * …

  • Learning Translations

    • Huge corpus of “aligned sentences”.

    • Europarl

    • Corpus of European Parliamant proceedings

    • The EU is mandated to translate into all 21 official languages

    • 21 languages, (semi-) aligned to each other

    • P( casa | house ) = (count all casa/house pairs!)

    • Pd( 2 | 5 ) = (count all sentences where 2nd word

    went to 5th word)

  • Machine Translation Technology

    • Hand-held devices for military

    • Speak english -> recognition -> translation -> generate Urdu

    • Translate web documents

    • Education technology?

    • Doesn’t yet receive much of a focus

  • What NLP topics did we miss?

    • Dialogue Systems

    12

    Do you think

    Anakin likes me? I don’t care.

  • What NLP topics did we miss?

    • Dialogue Systems

    • Why? Heavy interest in human-robot communication.

    • UAVs require teams of 5+ people for each operating

    machine • Goal: reduce the number of people

    • Give computer high-level dialogue commands, rather than low-level

    system commands

    13

  • What NLP topics did we miss?

    • Dialogue Systems

    • Dialogue is a fascinating topic. Not only do we need

    to understand language, but now discourse cues: • Questions require replies

    • Imperatives/Commands

    • Acknowledgments: “ok”

    • Back-channels: “uh huh”, “mm hmm”

    • Belief-Desire-Intention (BDI) Model

    • Beliefs: you maintain a set of facts about the world

    • Desires: things you want to become true in the world

    • Intentions: desires that you are taking action on

    14

  • What NLP topics did we miss?

    Unsupervised Learning

    15

  • What NLP topics did we miss?

    Unsupervised Learning

    • Most of this semester used data

    that had human labels.

    • Bootstrapping was our main counter-

    example: it is mostly unsupervised.

    • Many many algorithms being

    researched to learn language and

    knowledge without humans, only

    using text.

    16

  • El Fin

    • Secret 1:

    17

  • El Fin

    • Secret 1:

    I intentionally made some of our labs ambiguous

    18

  • El Fin

    • Secret 1:

    I intentionally made some of our labs ambiguous

    Under-defined tasks with unclear expected results

    19

  • El Fin

    • Secret 1:

    I intentionally made some of our labs ambiguous

    Under-defined tasks with unclear expected results

    • Secret 2:

    20

  • El Fin

    • Secret 1:

    I intentionally made some of our labs ambiguous

    Under-defined tasks with unclear expected results

    • Secret 2:

    I tried to teach you skills that have nothing to do with NLP

    21

  • El Fin

    • Secret 1:

    I intentionally made some of our labs ambiguous

    Under-defined tasks with unclear expected results

    • Secret 2:

    I tried to teach you skills that have nothing to do with NLP

    Experimentation

    Error Analysis

    22

  • El Fin

    • Secret 1:

    I intentionally made some of our labs ambiguous

    Under-defined tasks with unclear expected results

    • Secret 2:

    I tried to teach you skills that have nothing to do with NLP

    Experimentation

    Error Analysis

    • Secret 3:

    23

  • El Fin

    • Secret 1:

    I intentionally made some of our labs ambiguous

    Under-defined tasks with unclear expected results

    • Secret 2:

    I tried to teach you skills that have nothing to do with NLP

    Experimentation

    Error Analysis

    • Secret 3:

    I appreciate the hard work you put into the class

    24


Jiwei Li NLP Researcher - socialmedia-class.orgsocialmedia-class.org/slides/students_2017/Jiwei Li_NLP_Researche… · Deep Reinforcement learning for dialogue generation [2016],

SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions

NLP Practitioner Certification Training - ubermind.coubermind.co/assets/uber-mind-nlp-flyer-2017-1213-03.pdf · grates both traditional (behavioral) NLP along with NLP New Code. NLP

NLP (Fall 2013): NLP Areas, Chomisky Hierarchy, Finite State Automata, Regular Languages, Regular Expressions, Pattern Matching, ELIZA - NL Dialogue with Computer

SI485i : NLP - United States Naval Academy · SI425 : NLP Set 5 Naïve Bayes Classification . Motivation • We want to predict something. • We have some text related to this something

SI485i Natural Language Processing · NLP is hard. (news headlines) 1. Minister Accused Of Having 8 Wives In Jail 2. Juvenile Court to Try Shooting Defendant 3. Teacher Strikes Idle

SI485i : NLP - usna.edu · Perplexity • Perplexity is the probability of the test set (assigned by the language model), normalized by the number of words: • Chain rule:

NLP mastering nlp coaching skills.pdf

Artificial Intelligence - Internet of Business...Chatbot engine Dialogue Management Multi-Channel NLU/NLP. Foundational AI Opportunity ... PowerPoint Presentation Author: Jody Tozier

NLP-10005586(1-2018) · NLP BusinessNLP NLP Meta-Thinkingprocess BusinessNLPŽ r * -J NLP Module 1 BusinessNLP Practitioner Foundation Skills BusinessNLP (NLP Quotient) BusinessNLPŽ

2015 Sep - NLP Workshop - NLP Center

SI485i : NLP · Deriving Naïve Bayes • Let and label Y be discrete. • Then, we can estimate and directly from the training data by ... language is where the real gains come from

SI485i : NLP Set 13 Information Extraction. 2 “Yesterday GM released third quarter results showing a 10% in profit over the same period last year. “John

SI485i : NLP...other probabilistic models in NLP, especially •For pilot studies •in domains where the number of zeros isn’t so huge. Exercise Hey, I just met you, And this is

SI485i : NLP Set 9 Advanced PCFGs Some slides from Chris Manning

SI485i : NLP Day 1 Intro to NLP. Assumptions about You You know… how to program Java basic UNIX usage basic probability and statistics (we’ll also review)

The Evolution of NLP - Welcome to GWiz NLP - Gwiz NLP

SI485i : NLP - USNA€¢ Never-ending Language Learning ... Hearst pattern Example occurrences X and other Y ...temples, ... SI485i : NLP

Life Changing NLP Training - Home - Dr Bridget NLP · Life Changing NLP Training ... Certified Master Practitioner of NLP (Neuro Linguistic Programming), ... NLP PRACTITIONER COURSE

NLP Training - NLP Certification

SI485i : NLP Set 11 Distributional Similarity slides adapted from Dan Jurafsky and Bill MacCartney

Structured Probabilistic Models for Online Dialogue …Structured Probabilistic Models for Online Dialogue and Text Dhanya Sridhar 3.22.18 Stanford NLP Seminar 1 Socio-behavior modeling

SI485i : NLP - USNA · Who cares about NLP? ... ... system commands 13 . What NLP topics did we miss?

Key-Value Retrieval Networks for Task ... - Stanford NLP GroupStanford NLP Group1 Ford Research and Innovation Center2 Abstract Neural task-oriented dialogue systems of-ten struggle

SI485i : NLP...•This means Laplace is a blunt instrument •Could use more fine-grained method (add-k) • Laplace smoothing not often used for N-grams, as we have much better methods

SI485i : NLP

NLP Practitioner Notes · NLP Practitioner Notes NLP Practitioner Course presented by Micheal Goodman NLP Master Practitioner (ABNLP) and NLP Trainer (NLPEA) 2018

SI485i : NLP - USNA · language is where the real gains come from in NLP. ... New code for new features. Call your language models, get a probability, and then multiply new

Nlp-Automata in Nlp

SI485i : NLP Set 3 Language Models Fall 2012 : Chambers

SI485i : NLP Set 4 Smoothing Language Models Fall 2012 : Chambers

(Nlp) Nlp Secrets

SI485i : NLP€¦ · SI485i : NLP Set 6 Sentiment and Opinions . ... Opinion Mining for Stock Market Prediction ... • Sentiment analysis is a type of text classification task. •

SI485i : NLP Set 10 Lexical Relations slides adapted from Dan Jurafsky and Bill MacCartney

SI485i : NLP Day 2 Probability Review. Introduction to Probability Experiment (trial) Repeatable procedure with well-defined possible outcomes Outcome