dictionary-based named entity recognition

140
Lars Juhl Jensen Dictionary-based named entity recognition

Upload: lars-juhl-jensen

Post on 11-Feb-2017

532 views

Category:

Science


0 download

TRANSCRIPT

Page 1: Dictionary-based named entity recognition

Lars Juhl Jensen

Dictionary-basednamed entity recognition

Page 2: Dictionary-based named entity recognition

>10 km

Page 3: Dictionary-based named entity recognition

too much to read

Page 4: Dictionary-based named entity recognition

computer

Page 5: Dictionary-based named entity recognition

as smart as a dog

Page 6: Dictionary-based named entity recognition

teach it specific tricks

Page 7: Dictionary-based named entity recognition
Page 8: Dictionary-based named entity recognition
Page 9: Dictionary-based named entity recognition

named entity recognition

Page 10: Dictionary-based named entity recognition

comprehensive dictionary

Page 11: Dictionary-based named entity recognition

synonyms

Page 12: Dictionary-based named entity recognition

cyclin dependent kinase 1

Page 13: Dictionary-based named entity recognition

CDC2

Page 14: Dictionary-based named entity recognition

normalization

Page 15: Dictionary-based named entity recognition

CDK1_HUMAN

Page 16: Dictionary-based named entity recognition

dictionary compilation

Page 17: Dictionary-based named entity recognition

genes/proteins

Page 18: Dictionary-based named entity recognition

UniProtKB

Page 19: Dictionary-based named entity recognition

Ensembl

Page 20: Dictionary-based named entity recognition

RefSeq

Page 21: Dictionary-based named entity recognition

chemical compounds

Page 22: Dictionary-based named entity recognition

PubChem

Page 23: Dictionary-based named entity recognition

species/organisms

Page 24: Dictionary-based named entity recognition

NCBI Taxonomy

Page 25: Dictionary-based named entity recognition

functions

Page 26: Dictionary-based named entity recognition

pathways

Page 27: Dictionary-based named entity recognition

compartments

Page 28: Dictionary-based named entity recognition

Gene Ontology

Page 29: Dictionary-based named entity recognition

tissues

Page 30: Dictionary-based named entity recognition

Brenda Tissue Ontology

Page 31: Dictionary-based named entity recognition

diseases

Page 32: Dictionary-based named entity recognition

Disease Ontology

Page 33: Dictionary-based named entity recognition

phenotypes

Page 34: Dictionary-based named entity recognition

Human Phenotype Ontology

Page 35: Dictionary-based named entity recognition

environments

Page 36: Dictionary-based named entity recognition

Environment Ontology

Page 37: Dictionary-based named entity recognition

filters

Page 38: Dictionary-based named entity recognition

redundant terms

Page 39: Dictionary-based named entity recognition

insulin

Page 40: Dictionary-based named entity recognition

broad synonyms

Page 41: Dictionary-based named entity recognition

CDK holoenzyme

Page 42: Dictionary-based named entity recognition

related synonyms

Page 43: Dictionary-based named entity recognition

polyubiquitin

Page 44: Dictionary-based named entity recognition

wrong synonyms

Page 45: Dictionary-based named entity recognition

BRCA1

Page 46: Dictionary-based named entity recognition

dictionary expansion

Page 47: Dictionary-based named entity recognition

shortened forms

Page 48: Dictionary-based named entity recognition

protein kinase activity

Page 49: Dictionary-based named entity recognition

protein kinase

Page 50: Dictionary-based named entity recognition

Wnt signaling pathway

Page 51: Dictionary-based named entity recognition

Wnt signaling

Page 52: Dictionary-based named entity recognition

synonymous forms

Page 53: Dictionary-based named entity recognition

metabolic disease

Page 54: Dictionary-based named entity recognition

metabolic disorder

Page 55: Dictionary-based named entity recognition

plural forms

Page 56: Dictionary-based named entity recognition

protein kinase

Page 57: Dictionary-based named entity recognition

protein kinases

Page 58: Dictionary-based named entity recognition

mitochondrion

Page 59: Dictionary-based named entity recognition

mitochondria

Page 60: Dictionary-based named entity recognition

cancer

Page 61: Dictionary-based named entity recognition

cancers

Page 62: Dictionary-based named entity recognition

adjective forms

Page 63: Dictionary-based named entity recognition

mitochondrion

Page 64: Dictionary-based named entity recognition

mitochondrial

Page 65: Dictionary-based named entity recognition

abbreviated forms

Page 66: Dictionary-based named entity recognition

Escherichia coli

Page 67: Dictionary-based named entity recognition

E. coli

Page 68: Dictionary-based named entity recognition

prefixes and suffixes

Page 69: Dictionary-based named entity recognition

CDC2

Page 70: Dictionary-based named entity recognition

hCDC2

Page 71: Dictionary-based named entity recognition

mCDC2

Page 72: Dictionary-based named entity recognition

Cdc28

Page 73: Dictionary-based named entity recognition

Cdc28p

Page 74: Dictionary-based named entity recognition

huge dictionary

Page 75: Dictionary-based named entity recognition

additional ambiguity

Page 76: Dictionary-based named entity recognition

handling ambiguity

Page 77: Dictionary-based named entity recognition

three options

Page 78: Dictionary-based named entity recognition

allow

Page 79: Dictionary-based named entity recognition

disallow

Page 80: Dictionary-based named entity recognition

disambiguate

Page 81: Dictionary-based named entity recognition

acceptable ambiguity

Page 82: Dictionary-based named entity recognition

orthologous genes

Page 83: Dictionary-based named entity recognition

overlapping ontologies

Page 84: Dictionary-based named entity recognition

disease

Page 85: Dictionary-based named entity recognition

phenotype

Page 86: Dictionary-based named entity recognition

unacceptable ambiguity

Page 87: Dictionary-based named entity recognition

unrelated entities

Page 88: Dictionary-based named entity recognition

APC

Page 89: Dictionary-based named entity recognition

adenomatous polyposis coli

Page 90: Dictionary-based named entity recognition

anaphase promoting complex

Page 91: Dictionary-based named entity recognition

disambiguation

Page 92: Dictionary-based named entity recognition

ranking of name sources

Page 93: Dictionary-based named entity recognition

remove unlikely meanings

Page 94: Dictionary-based named entity recognition

acronym definitions

Page 95: Dictionary-based named entity recognition

three letter acronym (TLA)

Page 96: Dictionary-based named entity recognition

other names mentioned

Page 97: Dictionary-based named entity recognition

C. sativa

Page 98: Dictionary-based named entity recognition

Camelina sativa

Page 99: Dictionary-based named entity recognition

Cannabis sativa

Page 100: Dictionary-based named entity recognition

Castanea sativa

Page 101: Dictionary-based named entity recognition

marijuana

Page 102: Dictionary-based named entity recognition

species autodetection

Page 103: Dictionary-based named entity recognition

two rounds of NER

Page 104: Dictionary-based named entity recognition

species/organisms

Page 105: Dictionary-based named entity recognition

genes/proteins

Page 106: Dictionary-based named entity recognition

text matching

Page 107: Dictionary-based named entity recognition

uppercase / lowercase

Page 108: Dictionary-based named entity recognition

spaces and hyphens

Page 109: Dictionary-based named entity recognition

punctuation

Page 110: Dictionary-based named entity recognition

too many variants

Page 111: Dictionary-based named entity recognition

flexible matching

Page 112: Dictionary-based named entity recognition

finite state automaton

Page 113: Dictionary-based named entity recognition

LINNAEUS

Page 114: Dictionary-based named entity recognition

custom hash function

Page 115: Dictionary-based named entity recognition

C++ tagger

Page 116: Dictionary-based named entity recognition

efficiency

Page 117: Dictionary-based named entity recognition

Pafilis et al., PLOS ONE, 2013

Page 118: Dictionary-based named entity recognition

performance

Page 119: Dictionary-based named entity recognition

~85% precision

Page 120: Dictionary-based named entity recognition

~75% recall

Page 121: Dictionary-based named entity recognition

“black list”

Page 122: Dictionary-based named entity recognition

bad names

Page 123: Dictionary-based named entity recognition

SDS

Page 124: Dictionary-based named entity recognition

a

Page 125: Dictionary-based named entity recognition

an

Page 126: Dictionary-based named entity recognition

web resources

Page 127: Dictionary-based named entity recognition

indexing of literature

Page 128: Dictionary-based named entity recognition

term co-occurrence

Page 129: Dictionary-based named entity recognition

iHOP

Page 130: Dictionary-based named entity recognition

Hoffmann & Valencia, Nature Genetics, 2004www.ihop-net.org

Page 131: Dictionary-based named entity recognition

STRING

Page 132: Dictionary-based named entity recognition

Szklarczyk et al., Nucleic Acids Research, 2015string-db.org

Page 133: Dictionary-based named entity recognition

real-time text mining

Page 134: Dictionary-based named entity recognition

Reflect

Page 135: Dictionary-based named entity recognition

augmented browsing

Page 136: Dictionary-based named entity recognition

Pafilis, O’Donoghue, Jensen et al., Nature Biotechnology, 2009reflect.ws

Page 137: Dictionary-based named entity recognition

EXTRACT

Page 138: Dictionary-based named entity recognition

interactive annotation

Page 139: Dictionary-based named entity recognition

Pafilis et al., Proceedings of BioCreative V, 2015extract.hcmr.gr

Page 140: Dictionary-based named entity recognition

Pafilis et al., Proceedings of BioCreative V, 2015extract.hcmr.gr