Upload File

algorithms for nlpdemo.clab.cs.cmu.edu/algo4nlp20/slides/sp20 iitp lecture 10 -- hm… ·...

  • Home
  • Documents
  • Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
64
1 Yulia Tsvetkov Algorithms for NLP IITP, Spring 2020 HMMs, POS tagging, NER

Upload: others

Post on 06-Oct-2020

0 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

1

Yulia Tsvetkov

Algorithms for NLP

IITP, Spring 2020

HMMs, POS tagging, NER

Page 2: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

2

▪ POS tagging recap▪ HMMs, Viterbi ▪ HMMs+

▪ dealing with UNKs▪ 3gram HMMs▪ multilingual POS tagging

▪ Featurizing HMMs▪ MEMM, CRF

▪ NER ▪ HMMs is speech recognition

Plan

Page 3: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 4: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 5: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 6: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 7: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 8: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 9: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

https://universaldependencies.org

https://universaldependencies.org/
Page 10: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 11: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 12: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 13: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

●

Page 14: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪

▪▪ → →

▪▪▪

Page 15: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

15

Levels of linguistic knowledge

Slide credit: Noah Smith

Page 16: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

16

▪ map a sequence of words to a sequence of labels

▪ Part-of-speech tagging (Church, 1988; Brants, 2000) ▪ Named entity recognition (Bikel et al., 1999)▪ Text chunking and shallow parsing (Ramshaw and Marcus,

1995) ▪ Word alignment of parallel text (Vogel et al., 1996) ▪ Compression (Conroy and O’Leary, 2001) ▪ Acoustic models, discourse segmentation, etc.

Sequence Labeling

Page 17: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

17

Sequence labeling as classification

Page 18: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 19: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

the future is independent of the past given the present

Page 20: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 21: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

the future is independent of the past given the present

Page 22: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

o1 o2 on

▪

▪

▪

...

Page 23: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 24: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 25: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪

Page 26: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪

▪

▪▪

▪

Page 27: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 28: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 29: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 30: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 31: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 32: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 33: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 34: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 35: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 36: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 37: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 38: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 39: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 40: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪▪

Page 41: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 42: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪ →▪ →▪ →▪ →▪ →▪ →

▪▪ →

▪▪▪

Page 43: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪▪▪

Page 44: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪

▪

Page 45: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪

▪

Page 46: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 47: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪

Page 48: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 49: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 50: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪

▪

Page 51: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪

▪▪▪

Page 52: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪

Page 53: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪

▪

▪

▪ ⇒▪

Page 54: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪

▪

▪

Page 55: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 56: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 57: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 58: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 59: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 60: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪

▪

Page 61: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

▪▪

▪▪

Page 62: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

ssssssssppppeeeeeeetshshshshllllaeaeaebbbbb

“speech lab”

Page 63: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet
Page 64: Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP lecture 10 -- HM… · smoothing. Rows are labeled with the conditioning event; thus BIND) is 0.7968. NN DT Janet

Algorithms for NLPdemo.clab.cs.cmu.edu › algo4nlp20 › slides › summarization.pdf · Unsupervised content selection Intuition dating back to Luhn (1958): Choose sentences that

Query Word Labeling and Transliteration for Indian …fire/slides/Shubham_TST_fire14.pdfQuery Word Labeling and Transliteration for Indian Languages: IITP TS Shared Task system description

INDIAN INSTITUTE OF TECHNOLOGY PATNAINDIAN INSTITUTE OF TECHNOLOGY PATNA Bihta, Patna -80 I 106 . VACANCIES IN ADMINISTRATIVE AND SERVICES CADRE . Advt. No.: IITP/RECT/002/2018 Date:

IITP Tech & Future Insight』Fall Concert 행사용역 제안 요청서

ITEP-TH-31/21 IITP-TH-20/21 MIPT-TH-17/21 Implications for

Format of Business Plan - IITP - Incubation Centericiitp.com/images/Format of Business Plan.pdf · This format of business plan is intended to push you to think about prioritising

IITP T&P Brochure 2011

11-830 Computational Ethics for NLPdemo.clab.cs.cmu.edu/ethical_nlp2019/slides/11_Privacy.pdf11-830 Computational Ethics for NLP Phone-based Speaker ID Use *lots* of ASR engines But

Evaluation of internal fit of interim crown fabricated ... · by Institute for Information & Communications Technology Promotion (IITP) grant funded by the Korea government (MSIP)

Algorithms for NLPdemo.clab.cs.cmu.edu/11711fa18/slides/lecture_2_language_models_1.pdfIncreasing N-GramOrder §Higher orders capture more dependencies 198015222 the first 194623024

IIT PATNA - Welcome to IITP

Neural LMs - Carnegie Mellon Universitydemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP...A neural probabilistic language model Bengio et al., 2003 Natural Language Processing (almost)

11-830 Computational Ethics for NLPdemo.clab.cs.cmu.edu/ethical_nlp2019/slides/13_fakenews.pdf · But everyone I talk to online likes My Little Pony You never see people liking other

INSTITUTE FOR INFORMATION TRANSMISSION …cool.iitp.ru/iitp/2002/annu2002e.pdfInstitute for Information Transmission Problems GENERAL INFORMATION The Institute for Information Transmission

Groups. Jung tests –Introversion/Extroversion, Intuition/Sensing, Thinking/Feeling, and Judging/Percieving ESTP, IITP etc Cattell Personality

Algorithms for NLPdemo.clab.cs.cmu.edu/11711fa18/slides/lecture1_introduction.pdf · NLP lies at the intersection of computational linguistics and artificial intelligence. NLP is

Low-Resource NLP - Carnegie Mellon Universitydemo.clab.cs.cmu.edu/algo4nlp20/slides/low-resource-nlp.pdfNLP Problems •Most languages are low-resource •Approximately 7,000 languages

Dynamic programming: one algorithmic key to many biological locks Mikhail Gelfand RTCB, IITP, RA S and FBB, MSU 2010-2011

Deliverable Report D 7.1 Specification and planning …...MobiMESH, AGH, ALV, BGU, NEC, IITP Detailed demonstrators description update 1.0 13/07/12 Y.G TID Final release 1.0 PROPRIETARY

Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp19/slides/FA19_IITP_lecture... · 2019. 11. 19. · Phrase-Based MT e f source phrase target phrase translation features Translation

l}anga!iinan ~tate Wniber!iitp - psu.edu.phpsu.edu.ph/wp-content/uploads/2018/08/APP-CSE-2019-Pangasinan-State... · ANNUAL PROCUREMENT PLAN-COMMON SUPPLIES AND EQUIPMENT (APP-CSE)

PLACEMENT BROCHURE - Welcome to IITP -B.T… · Communications, Digital Signal Processing, Embedded Systems, VLSI, Electrical Machines, Power Electronics and Power Systems are fully

Algorithms for Natural Language Processingdemo.clab.cs.cmu.edu/algo4nlp20/slides/8-pos.pdf · 2020. 2. 6. · nouns verbs adjectives adverbs prepositions determiners pronouns conjunctions

MODE MATCHING METHODS FOR SPECTRAL AND SCATTERING PROBLEMS · MODE MATCHING METHODS FOR SPECTRAL AND SCATTERING PROBLEMS ... 2615 CNRS/ IUM/ IITP RAS/ Steklov MI RAS ... which explicit

IITP 6 The ICT Profession Body of Knowledge 6 The ICT... · 2014-11-11 · InstituteofITProfessionalsNewZealand(IITP)((( (((0800252255(L24,(Grand(Plimmer(Tower,(Wellington(( [email protected](((((

Jonghyun Kim – Curriculum Vitae - DCLdcl.yonsei.ac.kr/wordpress/wp-content/uploads/... · Participated Projects IITP (Institute for Information & communications Technology Promotion)

Algorithms for NLPdemo.clab.cs.cmu.edu/11711fa18/slides/FA18 11-711 lecture... · 2018. 9. 18. · Automatic speech recognition (or speech-to-text) systems transform speech utterances

Algorithms for NLPdemo.clab.cs.cmu.edu/11711fa18/slides/FA18 11-711... · FastText Evaluation Intrinsic evaluation Arabic, German, Spanish, French, Romanian, Russian word1 word2 similarity

11-737 Multilingual NLPdemo.clab.cs.cmu.edu/11737fa20/slides/multilingual-11... · 2020. 12. 8. · 11-737 Multilingual NLP Pidgins and Creoles Creoles have native speakers, Pidgins

The IITP Code of Professional Conduct - Institute of IT ... Code of Professional Conduct.pdf · Supplement to the IITP Code of Professional Conduct: Guidance and Interpretation This

Algorithms for NLPdemo.clab.cs.cmu.edu/algo4nlp20/slides/SP20 IITP... · Learning representations by back-propagating errors Rumelhart, Hinton & Williams, 1986 A neural probabilistic

IITP NZ presso for highschool

IITP Automation

On Innovation and Failure: IITP Conference Oct 25 2013, Lance Wiggs

iitp.kr. 2020 IITP bro... · 2020-04-08 · Tech nology Commercialization Locations . 04 ... 09 Roles & Responsibilities Technology Planning Planning projects to create promising