11 project, part 3. outline basics of supervised learning using naïve bayes (using a simpler...

11

Project, Part 3

Outline

• Basics of supervised learning using Naïve Bayes (using a simpler example)

• Features for the project

2

•3

The Weather ProblemTraining Data

Outlook Temp. Humidity Windy PlaySunny Hot High FALSE NoSunny Hot High TRUE No

Overcast Hot High FALSE YesRainy Mild High FALSE YesRainy Cool Normal FALSE YesRainy Cool Normal TRUE No

Overcast Cool Normal TRUE YesSunny Mild High FALSE NoSunny Cool Normal FALSE YesRainy Mild Normal FALSE YesSunny Mild Normal TRUE Yes

Overcast Mild High TRUE YesOvercast Hot Normal FALSE YesRainy Mild High TRUE No

Classification—A Two-Step Process

• Model construction– Each training example (line on *arff file) is assumed to belong to a

predefined class, as determined by the class label (the last column on the *arff file)

– For probabilistic machine learning algorithms, like Naïve Bayes, the model defines the probability that an instance belongs to a class, given a set of feature values.

– One probability for each class, for each combination of feature values.

– E.g.: – P(play=yes|outlook=sunny,temp=hot,humidity=high,windy=false)– …!– The probabilities are estimated based on counts in the training data (as we have seen

throughout the course)

4

Classification—A Two-Step Process

• Model usage: classifying new instances not in the training data

• E.g., given an instance with these feature values:outlook=sunny,temp=hot,humidity=high,windy=false

Which is more likely?P(play=yes|outlook=sunny,temp=hot,humidity=high,windy=false) >P(play=no|outlook=sunny,temp=hot,humidity=high,windy=false)?

Assign the most likely class to a new instance, based on probabilities that were estimated based on counts in the training data.

5

Classification—A Two-Step Process

• Model usage: classifying new instances not in the training data

• Evaluate: Estimate accuracy of the model– The known label of test sample is compared with the classified result

from the model– Accuracy rate is the percentage of test set samples that are correctly

classified by the model

6

Classification Process (1): Model Construction

TrainingData

ClassificationAlgorithms

Classifier(Model)

For probabilistic algorithms such as Naïve Bayes, the modeldefines the probability of each class given each possible

combination of feature values; the probabilities are estimated based on counts in the training data.

7

Classification Process (2): Use the Model in Prediction

Classifier

Unseen Data

outlook=sunny,temp=hot.humidity=high,windy=false

Play?

For evaluation, the model’s predicted answers are comparedto the gold standard labels in the

test data

8

9

Features for Semantic Role Labeling (SRL)

• We are defining features for a constituent C added to the *arff files for target predicate P

• Start with the features from Part 2– P itself (the lemma)– P's POS– Type of constituent C is

Features for SRL

• Parse Tree Path: minimal path in the parse tree from P to C

10

11

Parse Tree Path Feature: Example 1

S

NP VP

NP PP

The

Prep NP

with

the

V NP

bit

a

big

dog girl

boy

Det A NDet A N

εAdj A

ε

Det A N

ε

Path Feature Value:

V ↑ VP ↑ S ↓ NP

12

Parse Tree Path Feature: Example 2

S

NP VP

NP PP

The

Prep NP

with

the

V NP

bit

a

big

dog girl

boy

Det A NDet A N

εAdj A

ε

Det A N

ε

Path Feature Value:

V ↑ VP ↑ S ↓ NP ↓ PP ↓ NP

13

Features for SRL

• Position: Does C precede or follow P in the sentence?

• Voice: Is P in the active or passive voice?• Head word of C

14

Head Word Feature Example

• There are standard syntactic rules for determining which word in a phrase is the head. (You come up with specific rules. They don’t have to be perfect; just reasonable)

S

NP VP

NP PP

The

Prep NP

with

the

V NP

bit

a

big

dog girl

boy

Det A NDet A N

εAdj A

ε

Det A N

ε

Head Word: dog

15

Example of all Features

S

NP VP

NP PP

The

Prep NP

with

the

V NP

bit

a

big

dog girl

boy

Det A NDet A N

εAdj A

ε

Det A N

ε

PhraseType

ParsePath

Position Voice Headword

P P’s POS

NP V↑VP↑S↓NP precede active dog bit V