Artificial Intelligence Project #3: Analysis of Decision Tree

Learning Using WEKA

May 23, 2006

Introduction

Decision tree learning is a method for approximating discrete-valued target functionThe learned function is represented by a decision treeDecision tree can also be re-represented as if-then rules to improve human readability

An Example of Decision Tree

Decision Tree Representation (1/2)

Decision tree classify instances by sorting them down the tree from the root to some leaf node

Node

Specifies test of some attribute

Branch

Corresponds to one of the possible values for this attribute

Decision Tree Representation (2/2)

Each path corresponds to a

conjunction of attribute tests

(Outlook=sunny, Temperature=Hot,

Humidity=high, Wind=Strong)

(Outlook=Sunny ∧ Humidity=High) so NO

Decision trees represent a

disjunction of conjunction of

constraints on the attribute values of

instances

(Outlook=Sunny ∧Humidity=normal)

∨(Outlook=Overcast)

∨(Outlook=Rain ∧Wind=Weak)

Outlook

Humidity Yes

No Yes

Wind

No Yes

Sunny

Overcast

Rain

High Normal Strong Weak

What is the merit of tree representation?

Appropriate Problems for Decision Tree Learning

Instances are represented by attribute-value pairs

The target function has discrete output values

Disjunctive descriptions may be required

The training data may contain errors

Both errors in classification of the training examples and errors in the attribute values

The training data may contain missing attribute values

Suitable for classification

Study

Treatment-specific changes in gene expression discriminate in vivo drug response in human leukemia cells, MH Cheok et al., Nature Genetics 35, 2003.

60 leukemia patients

Bone marrow samples

Affymetrix GeneChip arrays

Gene expression data

Gene Expression Data

# of data examples120 (60: before treatment, 60: after treatment)

# of genes measured12600 (Affymetrix HG-U95A array)

TaskClassification between “before treatment” and “after treatment” based on gene expression pattern

Affymetrix GeneChip Arrays

Use short oligos to detect gene expression level.Each gene is probed by a set of short oligos.Each gene expression level is summarized by

Signal: numerical value describing the abundance of mRNAA/P call: denotes the statistical significance of signal

Preprocessing

Remove the genes having more than 60 ‘A’ calls# of genes: 12600 3190

Discretization of gene expression levelCriterion: median gene expression value of each sample0 (low) and 1 (high)

Gene Filtering

Using mutual information

Estimated probabilities were used.# of genes: 3190 1000

Final dataset# of attributes: 1001 (one for the class)

Class: 0 (after treatment), 1 (before treatment)

# of data examples: 120

,

( , )( ; ) ( , ) log

( ) ( )G C

P G CI G C P G C

P G P C

Final Dataset

120

1000

Materials for the Project

GivenPreprocessed microarray data file: data2.txt

DownloadableWEKA (http://www.cs.waikato.ac.nz/ml/weka/)

Analysis of Decision Tree Learning

Analysis of Decision Tree Learning

Submission

Due date: June 15 (Thu.), 12:00(noon)Report: Hard copy(301-419) & e-mail.

ID3, J48 and another decision tree algorithm with learning parameter.Show the experimental results of each algorithm. Except for ID3, you should try to find out better performance, changing learning parameter.Analyze what makes difference between selected algorithms.E-mail : jwha@bi.snu.ac.kr