Adaboost

Huong.H.T Dang

Agenda

• Boosting• Adaboost Introduction• Algorithm

Overview

• Introduced in 1990s• originally designed for classification problems• a procedure that combines the outputs of

many “weak” classifiers to produce a powerful “committee”

Boosting Approach

• select small subset of examples• derive rough rule of thumb• examine 2nd set of examples• derive 2nd rule of thumb• repeat T times• boosting = general method of converting rough

rules of thumb into highly accurate prediction rule

Boosting - Definition

• A machine learning algorithm• Perform supervised learning• Increments improvement of learned function• Forces the weak learner to generate new

hypotheses that make less mistakes on “harder” parts.

Adaboost Introduction

• AdaBoost is the short for Adaptive Boost, a machine learning meta-algorithm.

• AdaBoost contain several layers of weak learners each layer focus on the misclassified of its previous learner.

• These weak learners are called base classifiers and they need to be simple classifiers that have error rate is minimum

Introduction

• Weak Learner

Algorithm• Step 1: Initialization weight for

elements

• Step 2: • Find the weak learner with the minimum

error rate and calculate the weight of classifier

0 , 1

Ym(xn) = 1 , -1

Algorithm• Step 2: Update the weight to elements

))(exp()()1(nmin

mn

mn xyatww

Algorithm• Example for step 2. Loop

Algorithm

• Step 3: Decision, combine weak learners. Apply with new element