1

A genetic algorithm-based method for feature subset selection

Feng Tan; Xuezheng Fu; Yanqing Zang; Anu G. Bourgeois

Springer Soft Comput (2008) 12:111-120

Yi-Chia Lan

2

OutlineIntroduction

Feature selection methods

Entropy-based feature ranking T-statistics SVM-RFE(Recursive Feature

Elimination)

Framework of feature selection algorithm

Experiments and results

3

Introduction (cont.)

Training data (sets)

Test data (sets)

Classificatory accuracy

Introduction (cont.)

Introduction

1. Feature selection

Removing redundant irrelevant or noise features Improve the predictive accuracy

2. The experimental result demonstrate:

Higher classification accuracy Minimize size of feature subsets

Feature selection and extraction

Feature selection methods (cont.) Entropy-based

α : parameter

: average distance among the instances

: Euclidean distance between the two instances

Feature selection methods (cont.) T-statistics

Feature selection methods SVM-RFE

At the optimum of J , the first order is neglected

second order becomes

Genetic algorithm

Framework of feature selection algorithm (cont.)

Fitness function :

x : feature vector representing ; c(x) : classification accuracyw : parameter {0~1} ; s(x) : weighted size

Framework of feature selection algorithm

Crossover : Single-point crossover operator

Mutation : 0.001

Experiment result (1)

Experiment result (2)