baqmar 2008

600+ analysts

Our dreams

Inspiration

Friend organisations

Marketing research domains

Neighbour countries

Online & offline

website!

Jointhe conversation !

Marketing Researchers TODAY ?!

http://www.ondernemeringent.be/uploaded_images/cover_goedele3-788042.jpg

C L

PopularUnique

Well known

An explosion of data !

Fatigue among respondents !

Me, MySpace & IVisual (n) ethnography among 13-17 year olds

Joeri Van den Bergh (InSites Consulting)

Veerle Colin (MTV Networks)

WHY TELL ME WHY

The Research Briefing

Getting intimate with our target groups

ID construction of adolescents in a

digitalised world

Social groups as an extension of

psychographic segments

Role of brands in ID construction within

social groups

I SEE YOU, BABY

The Research Approach

Hawthorne effect

Researcher

gazeTime & Cost

intensive

Traditional

ethnography

context

Participant

R

Hawthorne effect

Researcher

gazeTime & Cost

intensive

Traditional

ethnography

context

Participant

R

360° visual

ethnography

Contact with researcher

via 2.0 tools“Informer” gaze: what is

important?Follow multiple participants from anywhere

R

Participant

context

From traditional to visual ethnography

1.User generated MM ethnography

= Observation takes place via photos/video

taken by the participants to the study

= Participants observe their own

environment & report back to the

researcher

360° Ethnography

Personal mePictures of...

Place where you can

really be yourself

Clothes you wear at

home

Objects that are

typical for me

Non mePictures of...

Clothes you no

longer want to

wear

Youngsters where

you so not want to

be friend with

Aspirational me

Pictures of...

Favorite clothes

youngsters where

you would like to

be friend with

Social mePictures of...

Important persons

in your life

Clothes you wear

to go out

Groups of

youngsters you like

Other groups

Pictures of other groups that

are different but okay

Pictures of youth of today

Pictures of normal persons

Ethnographic blog: identity related tasks

1.User generated ethnography

= Observation takes place via photos/video taken

by the participants to the study

= Participants observer their own environment &

report back to the researcher

2.Nethnography: social life of teenagers is very

much MOVING ONLINE !!!

= observation of the online behaviour & content

of a target group or within a certain webspace

360° Ethnography

Nethnography: we are your friends

Personal

identity

Nicknam

e

Profile

text

Profile

picture

Photo

collection

Clan member

ship

Conversation

Monitoring on

guestbook

Social

identity

Social Network Sites: nethnography

And for the datamining freaks among you, Annelies ripped the internet

• 300 active participants of netlog randomly

selected. Equal spread age x gender

• Webcrawlers to „scan‟ pages of netlog and

substract content.

• Textmining: profile pages – photo tags –

clan membership – conversations on the

guestbook

Other online behaviour: tracking tool

100% UGC OBSERVED BY US

That‟s why we do call it (visual) (n)ethnograpy: shoot me!

http://nl.netlog.com/Aagjtj/photo/photoid=23600370

http://nl.netlog.com/saarsaartje/photo/photoid=20412298

http://nl.netlog.com/bieke1990/photo/photoid=21187681

THE HARD PART

The Research Analysis

Analysis

INTIMACY

ME-SEUM

Social Networks are not so social as you think they are

Only 32% of the conversations on the guestbook

are „interactive‟!

The rest are all single statements.

Feedback on picture

Congratulations

School

Confirm friendship

MSN

Express love

Miss you

Practical appointment

It was great

Party

Making fun

Family

Study

How are you?

Age

Music

Online movies

Transport

I’ am bored

Feedback on profiles

Festivals

Sport

Mobile phones

Welcome by strangers

Youth movement Alcohol

Food

Litte kids

Quarrel

Gaming

Food

Sleeping

Tokio hotel

Travel

Cars

Shoes

one of the 3 key research questions

Social groups among today‟s youngsters

To what

youngster

group

do I belong?

Methodology

Aspira

tional

Group

Social

Group

Differe

nt but

OK

Non

group

M

E

WE ME (I’m better)T

HIN

KD

RIN

KCHANGE

CONSERVATISM

REAL WORLD

FANTASY

Fashion girlFashion boy

Breezer sluts

Jumpers

Nerds

Geek girls

emo

Alternative

Punk

Gothic

RockersTektonic

Skater

Hippies

Rapper

MAINSTREA

M

spartacus121 best pk (runescape)

30/04/2008 19:07:55 door laurent

ik hit iets in de 27 en dat is al zot hoog zij hits zijn

gestoord lol. zen zwaard is 140M (ik heb 10 M)

http://iblog.insites.eu/iBlogUserProfile.aspx?U=128

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS SKILLS

SKILLS

SKILLS

SKILLS

SKILLSSKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS SKILLS

SKILLS

SKILLS

SKILLS

SKILLSSKILLS

SKILLS

SKILLS


HIN

KD

RIN

KCHANGE

CONSERVATISM

REAL WORLD

FANTASY


Breezer sluts

Jumpers

Nerds

Geek girls

emo

Alternative

Punk

Gothic

RockersTektonic

Skater

Hippies

Rapper

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS LOOKS

LOOKS

LOOKS

LOOKS

LOOKSLOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS LOOKS

LOOKS

LOOKS

LOOKS

LOOKSLOOKS

LOOKS

LOOKS


HIN

KD

RIN

KCHANGE

CONSERVATISM

REAL WORLD

FANTASY


Breezer sluts

Jumpers

Nerds

Geek girls

emo

Alternative

Punk

Gothic

RockersTektonic

Skater

Hippies

Rapper

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS LOOKS

LOOKS

LOOKS

LOOKS

LOOKSLOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS

LOOKS LOOKS

LOOKS

LOOKS

LOOKS

LOOKSLOOKS

LOOKS

LOOKS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS SKILLS

SKILLS

SKILLS

SKILLS

SKILLSSKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS

SKILLS SKILLS

SKILLS

SKILLS

SKILLS

SKILLSSKILLS

SKILLS

SKILLS


HIN

KD

RIN

KCHANGE

CONSERVATISM

REAL WORLD

FANTASY


Breezer sluts

Jumpers

Nerds

Geek girls

emo

Alternative

Punk

Gothic

RockersTektonic

Skater

Hippies

Rapper

REMEMBER ME

How this research changed our lives

6 changes that rocked our socks off

• The tools for ID construction have changed

• New online quali methods proved to be efficient

• Our target group = the new and better quali researchers

• New reflection kit for entire MTV Networks staff

• Closer connection with MTV & new clients

• Redefine content strategy of TMF on screen & on line

4C Consulting

Introduction to our services

Our Mission | Boosting your customer value

4C Consulting

helps companies

win, keep and grow

customer value

Our Solutions | Call us for…

Customer Value

Strategy

Process

Excellence

Customer

Insight

Business

requirements

definition

Package selection

& implementation

Post-launch

care

1. Acquire new customers

2. Sell more to existing customers

• More of the same (increase turnover)

• Expand value proposition portfolio

(cross-sell & product development *)

• Upgrade value

proposition (Upsell)

3. Prevent existing

customers from leaving

1. Efficient Delivery (Process

Excellence)

2. Align value propositions

3. Advanced pricing

Our Focus | Boosting your customer value

Increase Revenue Reduce Costs

Business Intelligence Practice

SCV ROMSCCI Competing on Analytics

Audit

Infrastruc-tuur

Data Quality

Exploitatie Coaching

Why 4C Consulting | 7 compelling reasons

1. 100% focus on customer value

management

2. Result-oriented project approach

3. Connecting marketing, sales &

customer care with senior management

and IT

4. Independent consultant for 10 years

5. Experienced crew, passionate about

marketing, sales & customer care

6. Value based pricing model

7. Satisfied & loyal customers: 90

customers, more than 380 projects

Optimize your business with Business

&Decision

Michel Meulders

- Domain Manager -

Business & Decision Benelux

Business & Decision Benelux is :

• a multi-specialist, in specific

technology fields :

• Business Intelligence

• Customer Relationship Management

• Life sciences

• Risk & Compliance

• with foreign offices in Brussels,

Amsterdam, Luxembourg

• Top accounts in finance, pharma,

telco, distribution, industry (Fortis, ING,

ABN Amro, Dexia, GSK, UCB, Proximus,

Belgacom, Carrefour, Honda,…)

Founded in 2002

Merger of several companies specialised in BI

Consulting & System Integrator :

- More than 300 consultants

- About 18 mio Euro turnover in 2007

- 58% organic growth comparing to 2006

- Last acquisition : BnV Group (BE+NL)

Turnover evolution (consolidated)

0

5000

10000

15000

20000

25000

2004 2005 2006 2007 Obj 2008

Th

ou

sa

nd

s

Belgium Luxembourg Netherlands

For more info see http://www.businessdecision.com

Belgian Federation of

Market Research Institutes

www.febelmar.be

Febelmar mission

Development and promotion of market

research and opinion polls in Belgium

Protecting the sector interests

Watching over correct use of deontological

rules of market research in all phases of the

market research process

Stimulating continuous improvement of

quality of service in market research

Being a platform for communication,

exchange of expertise and networking

Members

27 agencies.

Together they represent about 75% of

the total market research expenditures in

Belgium.

What is I4BI?

i4bi is specialized in implementing Business Intelligence Solutions in

your company.

Our team of BI experts has deep functional and technical experience

with Application development, Business Model definitions and Data

Warehousing. Functional/technical designs, development, application

role out, training etc… are phases in a project where our consultants

have many years of experience.

i4bi consultants have a deep knowledge of the Oracle Business

Intelligence products and solutions.

i4bi sponsors the development of an independent analytical branch,

which will probably see the light in 2009

What do we provide?

We Provide

Business

Experience

Analytical

Expertise

To Support

Technical

Abilities

Strategic Decision Making

Expertise

• Analytical Expertise• Data Mining

• Statistical modelling

• Predictive analysis

• Basel II compliant modelling

• Forecasting

• Business Analysis Expertise• Reporting

• Delivering Business Insight to decision makers

• Marketing Analysis

• Data Quality

• Use of technical tools such as SAS – SPSS – Statistica to

support & extend business knowledge

Contact

• For more general information:

www.I4BI.be

• For more analytical information:

[email protected]

InSites Consulting

6 beliefs in 60 seconds

“Consumers are beginning in a very real sense to own our brands and participate. We need to begin to learn how to let go”

A.G. Lafley, CEO & Chairman of P&G

We believe ...in the empowered consumer

Human-to-human interactions are more powerful than ever and can make or break your brand

We believe ...in giving back

Rewarding experiences for participantsActive involvement of panel members

Charity contributions

http://www.talktochange.com/

We believe ...in connecting

Everything we do is aimed at strengthening connectionsbetween you, your market and us

“Connected Research” brings you closer to your market and taps into the wisdom of the crowds

Some of our connected research methods

Research communitiesBulletin boardsBlog research

Online discussion groups

More information: http://connectedresearch.insites.eu/

We believe ...in the power of new research methodsfor better marketing decision making

InformationalProviding more depth to research insights

TransformationalDoing things that were previously not possible

AutomationalConducting research more efficiently

http://flickr.com/photos/16246892@N00/507979113/

We believe ...in 1 + 1 = 3

Old and new methods need to be optimally “fused” in order to fully grasp the new customer / consumer reality

We believe ...in the power of our team

People make the differenceOpen, forward thinking, dedicated, passionate

Specific knowledge centers

©Keyrus – all rights reserved

BAQMaR, 17 December 2008

Welcome to Keyrus

performance management consulting technology

About Keyrus (Belgium)

• founded in 1996 as SOLIDPartners

• focus on performance management, business intelligence & data warehousing

• strong and balanced client base spread over different industries

• +100 consultants specialised in both technical and business domains

• part of Keyrus group (France)

Keyrus‟ global footprint

head office in Paris

present in 9 countries

listed on Paris stock

exchange Euronext

+1300 employees

Vision & mission

Keyrus will be one of the few leading service providers

in the area of performance management.

We help our clients to effectively design, build and operate

the adequate performance management organization and solutions

in an integrated end-to-end fashion.

Portfolio of solutions & services

Information Management

IM

Business Intelligence Platforms

BIP

Analytic Applications

AA

People and Processes

P&P

Corporate Performance Management

(C)PM

data warehouse & data marts

dat

a d

eliv

ery

&

ou

tflo

w f

un

ctio

ns

CPM data delivery, exchange & synchronization

info

rmat

ion

m

anag

em

en

t fu

nct

ion

s

sou

rce

sys

tem

s &

ap

plic

atio

ns

BI L

aye

r (r

ep

ort

s, O

LAP,

d

ash

bo

ard

s, a

lert

s)

CPM Applications (e.g. planning, ABM, PA)

Analytic Applications (e.g. data mining)

Keyrus [email protected]

Nijverheidslaan 3/2 B-1853 Strombeek-Bever

t +32 2 706 03 00f +32 2 706 03 09

Contact us

www.keyrus.be

17-Dec-2008performance management consulting technology

Introduction of Profacts

Who is PROFACTS ?

We are „the new kids on the block‟

in (online) market research ...

Q4

2006

Q4

2008

Q1

2007

Q2

2007

Q3

2007

Q4

2007

Q1

2008

Q2

2008

Q3

2008

2people have

founded Profacts

6people are now

working @ Profacts

200%growth rate

REVEALING FACTORS FOR SUCCESS1 strategy

28yrs

mean age

Profacts is active in more then 10 sectors ...

REVEALING FACTORS FOR SUCCESS

AUTOMOTIVE

GPS

TELECOM ENERGY

ICT PHARMACEUTICAL

FMCG

RECRUITMENT

INSURANCESBANKING

Python Predictions

Python Predictions

PREDICT

Python Predictions

Python Predictions

GROWTH

Python Predictions

www.pythonpredictions.com

Rogil Research

A research agency with a view

MARKETING & SENSORY RESEARCH

OUR PASSION

FTF research (Mobile Unit)

Telephone research

Online research

Panel services

Fieldwork in Europe

Sensory research

Eye-tracking / Eye|watch

Tachistoscope

Trained panel

Consumer panel

Taste lab

Sensory Safari

Note down in your agenda

SENSORY SAFARI

• March 26th 2009

• 18u

• At Rogil in Leuven

Sensory Safari

5 SENSES MARKETING

We hope to welcome you in March.

Thanks for your attention !

SAS Analytics For Challenging Times

Start Focused, Think Wide

Campaign Managment Requires

Optimization

CRM is becoming Risk Managment

SAS Breadth of Analytic Offering…

• Statistical Analysis

• Survey Design/Analysis

• Data Mining

• Text Mining

• Time Series Mining

• Forecasting

• Quality Improvement

• Operations Research

SAS Innovations in Marketing

Solutions….

Copyright © 2006, SAS Institute Inc. All rights reserved.

http://www.sas.com/feature/analytics/index.html

The Mission

Drive the widespread use of

data in decision making

The Focus

Attract Grow Fraud RiskRetain

Driving and Maximizing Profit

The Vision

Behavioral dataDescriptive data

Attitudinal dataInteraction data

Enterprise

Data

Sources

Operational

Processes

Operational

Processes

Enterprise

Data

Sources

The Acceptance

• The rise of the agnostics

Science vs. Chance

In numbers we trust!

The myth of the „best‟ algorithm

lessons learned from innovations in

data sampling and data pre-processing

for marketing analytics

Dr. Sven F. CroneDeputy Director, Ass. Prof.

Associated Experts

Prof. Paul Goodwin

Dr. Andrew Eaves

Research & PhD students

Heiko Kausch, RA

Stavros Asimakopoulos

Xi Chen

Bruce Havel

Suzi Ismail

Nikolaos Kourentzes

Ioannis Stamatopoulos

Andrey Davidenko

Charlotte Brown

Hong Juan Liu

T Hu

John Prest

Huang Tao

Visiting Researchers

Prof. Geoff Allen

Dr. Yukun Bao

Young-Sang Cho

Directors

Prof. Robert Fildes

Prof. Peter Young

Dr.Sven F. Crone

Researchers

Dr. Steve Finlay

Dr. Alastair Robertson

Dr. Didier Soopramanien

Dr. Kostas Nikolopoulos

Prof. Stephen Taylor

Dr. Wlodek Tych

Prof. David Peel

Prof. Peter Pope

http://www.lums.lancs.ac.uk/news/7035/

http://www.bt.com/index.jsp?BV_SessionID=@@@@1743980602.1107967121@@@@&BV_EngineID=ccccadddkmgekeecflgcefkdffndfli.0&obsOID=30777&obsPage=/index.jsp&vStore=1128&obsType=LINK&obsNoSee=true

http://www.tesco.com/r/?from=HomepageLogo&url=http://www.tesco.com/

http://www.burtonsfoods.com/home.asp

http://www.daler-rowney.com/index.asp

http://www.barclaycard.co.uk/?previous_page_id=1&link_type=TTL

http://www.uk.experian.com/index.html

http://www.ams.mod.uk/ams/content/docs/sadir/597.htm

http://www.dwp.gov.uk/

http://images.google.co.uk/imgres?imgurl=http://www.leoneinsurance.ca/images/axa-logo.png&imgrefurl=http://www.leoneinsurance.ca/images/&h=330&w=330&sz=53&hl=en&start=2&usg=__cp9UGiyZKT6o25brLs86d9j3AX4=&tbnid=xb-6FUd9TK91eM:&tbnh=119&tbnw=119&prev=/images%3Fq%3Daxa%26gbv%3D2%26hl%3Den

“Take away this pudding, it has no

theme.” Sir Winston Churchill (1915)

• Sampling issues in Data Mining

• Case study 1: Direct Marketing• Cross-selling of Magazine subscriptions

• Effect of data preprocessing: Sampling

• Interaction of Sampling with Scaling & Coding

• Case study 2: Credit & Behavioral Scoring• Predicting consumer credit default

• Effects of sample size

• Effects of sample distribution

• Case study 3: Online Shopping Behaviour• Predicting consumer shopping channel choice

• Sample distribution & multiple classes

• Conclusion & Take-aways

Agenda

Why (Under/Over) Sampling?

• Knowledge Discovery (KDD) = non-trivial process of identifying valid, novel, useful patterns in large data sets• Data Mining = only one single step in the KDD process• Data sample determines the whole process! ( GIGO)• “Research seems preoccupied with algorithms” [Hand 2000]

Monitoring

CRISP-DM Process

SAS SEMMA DM-Process

Sampling in Direct Marketing Literature?

Input

type* Methods***

Paramete

r tuning

Data reduction** Data projection

Feature

Selection

Re-

sampling

Continuous attributes Categories

Standardisation Discretisation Coding

[2] 2 BMLP, LR, LDA, QDA X X

[42] 1 MLP, LR, CHAID X X

[43] 2 MLP, RBF, LR, GP, CHAID X X

[44] 3 MLP, LR, LDA X X

[4] 2 CHAID, CART X

[6] 2 MLP, LR X X X X X

[9] 2 LVQ, RBF, 22 DT, 9 SC X X

[45] 2LDA, LR, KNN, KDE, CART, MLP,

RBF, MOE, FAR, LVQX X

[3] 1 MLP X X

[7] 2 LSSVM X X X

[11] 2 LR, LS-SVM, KNN, NB, DT X X X

[10] 1LDA, QDA, LR, BMLP, DT, SVM,

LSSVM, TAN, LP, KNNX X

[46] 2 LR, MLP, BMLP X X

[47] 2LSSVM, SVM, DT, RL, LDA, QDA,

LR, NB, IBLX X

[48] 1 DT, MLP, LR, FC X

[49] 1 FC X X

Majority of direct marketing papers focus on algorithm tuning

Only 3 papers consider Resampling / Instance Selection

No analysis of the interaction with Sampling & Projection & …

Database of customers (instances)

Known attributes for all customers (age, gender, existing subscriptions, …)

Known response (class membership) of buyers & non-buyers from past mailings

Build a model to separate classes decision boundary of different complexity

1 … Number of subscriptions … Many

Classification

Few

… D

ays s

ince last

purc

hase

… M

any

No responseSubscribed to magazine

Last campaign

http://ceicher.homeunix.com/archives/gatesmug.jpg

Class unknown

Use the decision boundary to classify unseen instances

Calculate on which side of hyperplane the instances lie (or distance)

Assign class to unseen instances


Classification


Few

… D

ays s

ince last

purc

hase

… M

any

Balanced dataset = class distributions are equal P(x|y=A)=P(x|y=B)

proportional sampling or stratified sampling feasible

Imbalanced dataset = class distributions unequal P(x|y=A)>>P(x|y=B) `

The class of interest is often the minority (in most business applications)

Reality Check: Imbalanced classes


Problem

• Classifiers are biased towards

the majority class

• Shifts the decision boundary

• Error / Accuracy based learning

creates naïve classifiers

• Invalid separation of classes1 … Number of subscriptions … Many

Few

… D

ays s

ince last

purc

hase

… M

any

Size of the sample?

Distribution / location of the sample?

Imbalanced Data Sampling


Stratified Random Sampling

divide DB in mutually exclusive

strata (subpopulations) & draw

random samples from each

Proportional

assure proportions in samples

equal those in population

Disproportional

weighted over-& undersampling

of important classes1 … Number of subscriptions … Many

Few

… D

ays s

ince last

purc

hase

… M

any

Exclude random instances of the majority class

Retain all instances of the minority class

Establish a balanced class distribution

Random Undersampling


Benefits

• Helps detect rare target levels

Risks

• Biases predictions (correctable)

• Looses information contained in

instances of the majority class

• Creates different boundaries

• Increases prediction variability

• …1 … Number of subscriptions … Many

Few

… D

ays s

ince last

purc

hase

… M

any

Retain all instances of the majority class in the sample

Duplicate identical instances of the minority class

Establish a balanced class distribution

Random Oversampling


Benefits

• Helps detect rare target levels

• No loss of information

Risks

• Biases predictions (correctable)

• Increases prediction variability

• Increases processing time


Few

… D

ays s

ince last

purc

hase

… M

any

rather some case studies ...!

Ready for more theory…?

x











Agenda

• Sell a magazine subscription to existing customers

• Whom to send mail to? (Which customers are most likely to respond?)

• How many customers to contact? (What is the optimal mailing size?)

Corporate project with leading German Publishing HouseProvided data set of past mailing campaigns

Benchmark novel methods against in-house SPSS Clementine

Explore Neural Networks (NN) an Support Vector Machines (SVM)

Business Case:

Direct Marketing/Response Optimization

ServletR?page=PageKategorieArtikelListe&produkt_kategorie_id=3






Smaller mailing (number of letters sent) lower costs (Euro 1.- per letter)

Higher response rate higher revenue

More specific mailing lower cost

More relevant information higher customer satisfaction

Benefits of Direct Marketing

Simple With data mining

Addressees 100.000 Top 40% = 40.000

Cost 2€/mail = 200.000€ 2,5€/mail = 100.000€

Response rate 0,5% = 500 1,0% = 400

Sales volume 300€ 300€

Sales volume 150.000€ 120.000€

Revenue -50.000€ 20.000€

NN get worse with learning …

%Pred. C 0

Pred. C 1

Sum

C 0 72.96 27.04 100

C 1 62.02 37.98 100

134.98 65.02 55.47

%Pred. C 0

Pred. C 1

Sum

C 0 52.87 43.37 100

C 1 47.13 56.63 100

100 100 54.75

%PredC 0

PredC 1

Sum

C 0 61.86 38.14 100

C 1 55.09 44.81 100

116.95 82.95 54.26

• Wish to implement Neural Networks for next campaign• In-house team (with no NN knowledge) outperformed us EVERY TIME!• Analyzed software, training parameters, etc. internal competition• Observed expert in building models … !

Scale numerical

features

Adjust imbalanced

class distributions

Decide on sample

size and method

Experimental Design:

Different data pre-processing

Handle categorical

features

Select useful

features

Handle outliers

Different SamplingOver-& Undersampling

Different Encodingn, n-1, thermo, ordinal

Different ScalingDiscretise, Standardise

Evaluate across 3 algorithms:

Neural Networks (MLPs), Support Vector Machines & Decision Trees

Multifactorial design to evaluate impact across multiple methods

Neural Networks (NN)

Support Vector Machines (SVM)

Decision Trees (CART)

Dataset Structure

Data set size• 300,000 customer records• 4,019 subscriptions sold• Response rate of 1.3%

Data set structure• 18 categorical features• 35 numerical features• Binary target variable

Evaluated the Impact of Data Preprocessing

• Data Sampling (over sampling vs. undersampling)

• Categorical attribute Encoding (N, N-1, thermo, ordinal)

• Continuous attribute Projection (Binning vs. Normalisation)

• Continuous attribute Scaling ( [0,+1] vs. [-1,+1] range)

Different balancing in the training data

Original distribution in the test data (65,000 instances)

Sampling

Data partition (number of records)

Oversampling Undersampling

Data subset Class 1 Class -1 Class 1 Class -1

Training set 20,000 20,000 2,072 2,072

Validation set 10,000 10,000 1,035 1,035

SUM 30,000 30,000 3,107 3,107

Test (hold-out) set 912 64,088 912 64,088

Created 2 Dataset Sampling candidates

Oversampling outperforms undersampling consistently!

Gain in Lift depends on method (different sensitivity)

Oversampling has higher impact than data coding & scaling

Results

Increase

Increase

Increase

Binning & Scaling of continuous attributes irrelevant for all methods!

Use Undersampling & N-1 encoding with SVM & NN

Best preprocessed SVM lift of 0.645 on test set … BUT …

Recommendations from Case Study

• Sampling • Oversampling outperfoms undersampling for all methods

• Undersampling: better in-sample results & worse out of sample

• Choice of method • NN & SVM better than CART

• Encoding & Projection• SVM: avoid Ordinal coding (e.g. 1,2,3) all other similar (incl. N !)

• NN: avoid standardization & ordinal encoding

• DT / CART: use temperature, all others similar (incl. ordinal)

Results are consistent across error measures

Experiments allow identification of „best practices‟ to model methods

Best-practice preprocessing varies between methods

Results across Pre-processing

Preprocessing: higher impact than method selection Lift-variation per method from Sampling/Scaling/Coding

> Difference of Lift between competing methods!

DTSVMNN

Method

0,65

0,64

0,63

0,62

0,61

0,60

Lif

t te

st

Lift performance onTest data subset

DTSVMNN

Method

0,58

0,57

0,56

0,55

0,54

0,53

AM

te

st

Arithmetic Mean Performanceon Test data subset

DTSVMNN

Method

0,58

0,57

0,56

0,55

0,54

0,53

0,52

0,51

0,50

GM

te

st

Geometric Mean Performanceon Test data subset

DPP causes 50%-70% of the

differences between models











Agenda

Business Case: Predicting

Customer Online Shopping Adoption

• Traditional buying process is offline & simultaneous “bricks” store

• Introduction of the Internet changes consumer behaviour• Seek information online & offline

• Purchasing online & offline

Changing purchasing behaviour through internet adoption

Changing purchasing behaviour through Technology Acceptance

• Development of heterogeneous Purchasing Behaviour• Example: Purchasing electronic durable consumer goods

• Search for product info (e.g. video cameras) online

test product in-store

search for best deal on internet & purchase

Search for Information Online Purchase Online

Search for Information Offline Purchase Offline

Online

Shoppers

Non-Internet

Shoppers

Browsers

Stages of Internet Adoption

1. OFFLINE BUYERS

Information gathering

& purchasing in Stores

2.BROWSERS

Information gathering online

& purchasing in stores

3.ONLINE BUYERS

Information gathering

& purchasing online

Motivation

DIDIER: Marketing Modelling

• Econometric / Marketing Domain

• Seeks to explain how customers behave in

online shopping

• Use of „black-box” logistic regression

models

Models class membership to identify

causal variables that explain choices

Descriptive & Normative Modelling

SVEN: Data Mining Perspective

• IS/OR/MS Domain Data Mining

• Seeks to accurately predict regardless of

explanation why customers buy

• Use of “black-box” methods from

computational intelligence

Models class membership to

accurately classify unseen instances

Predictive Modelling

same dataset & same objectives & similar methods

Conflicting “best practice” approaches to modelling

Outside of most software simulators!!! Implicit knowledge?

… WHO IS “CORRECT”? WHAT IS THE IMPACT?

Best practices

balance datasets for distribution

representative of population

Use ordinal variables & nominal variables

without recoding

Do not normalise / scale data

Best practices

Rebalance datasets for equal distribution

of target variables

Recode ordinal binary scale

Rescale & normalise data to facilitate

learning speed etc.

Dataset

• Survey on Internet Shopping Behaviour• 5500 UK households 685 respondents

• Adjusted for age, income etc. of customers (older less likely to buy)

• Adjusted for product specific risk of online shopping for branded durable consumer goods (inspection required to some extent)

• 73 questions on factors related to internet shopping, products etc.

Models Output VariablesInput Variables

Demographics

Internet

specific

Factors

Online

shopping

specific

Factors

Logistic Regression

Neural Networks

Class 1:

Browse Ônline &

Buy Online

Class 2:

Browse Online &

Buy Offline

Class 3:

Browse &

Buy Offline

Online Shopping Factors:

“Going to the shops is as convenient

as Internet shopping”

“I would buy online if products are

branded” etc. [1=strongly agree; …]

Demographic Factors

Age, Gender, Income

Internet Utility Factors

Score from 6 correlated variables

Mixed scale of nominal, ordinal, interval

Imbalanced Classification problem

UndersamplingOversamplingImbalanced

Dataset

Offline-Shoppers

BrowsersOnline-Shoppers

Offline-Shoppers


Offline-Shoppers


400

300

200

100

0

Co

un

t

Test

Validation

Training

Data Subset

UndersamplingOversamplingImbalanced

Dataset

Offline-Shoppers


Offline-Shoppers


Offline-Shoppers


400

300

200

100

0

Co

un

t

Test

Validation

Training

Data Subset

• Split of Dataset for Training, Validation and Test {50%;25;25%}• Distribution of target classes is skewed

{65% online buyers; 22.5% browsers; 12.5% offline shoppers}• Rebalancing of data sets through over- & undersampling)

Results without DiscretisationLogist.Reg. True

Value

Training Data Test Data

Dataset Online Browse Offline Online Browse Offline

Original Online 93.36 5.17 1.48 88.89 7.78 3.33

Imbalanced Browser 62.77 23.40 13.83 49.39 22.58 29.03

Offline 36.54 17.31 46.15 35.29 29.41 35.29

Under- Online 57.69 30.77 11.54 64.44 23.33 12.22

Sampling Browser 26.92 48.08 25.00 32.26 25.81 41.94

Offline 17.31 21.15 61.54 29.41 35.29 35.29

Over- Online 68.27 24.35 7.38 74.44 16.67 8.89


Offline 16.97 19.93 63.10 29.41 29.41 41.18

Neural Net Training Data Test Data


Original Online 86.19 12.71 1.10 86.67 8.89 4.44


Offline 25.17 28.57 45.71 29.41 35.29 35.29

Under- Online 44.86 40.00 17.14 27.78 58.89 13.33


Offline 8.57 20.00 71.43 11.76 41.18 47.06

Over- Online 81.22 18.23 0.55 61.11 22.22 16.67


Offline 15.52 0.55 99.45 0.00 11.76 88.24

MCRtrain=54.3%

MCRtest =48.9%

MCRtrain=55.8%

MCRtest =41.8%

MCRtrain=58.4%

MCRtest =48.2%

MCRtrain=54.4%

MCRtest =52.5%

MCRtrain=54.9%

MCRtest =35.7%

MCRtrain=88.0%

MCRtest =75.6%

Mean Classification Rate (%)

Results with Discretisation of OrdinalLogist.Reg. True

Value

Training Data Test Data


Original Online 91.51 6.64 1.85 85.56 7.78 6.67


Offline 26.92 17.31 55.77 58.82 47.62 17.65

Under- Online 71.15 21.15 7.69 55.56 24.44 20.00


Offline 15.38 11.54 73.08 58.82 0.00 41.18

Over- Online 68.63 22.88 8.49 70.0 21.11 8.89


Offline 13.28 14.02 72.69 17.65 23.53 58.82

Neural Net Training Data Test Data


Original Online 96.13 3.87 0.00 84.44 11.11 4.44


Offline 40.00 14.29 45.17 58.82 11.76 29.41

Under- Online 57.14 40.00 2.86 25.56 72.22 2.22


Offline 14.29 31.43 54.29 52.94 17.65 29.41

Over- Online 98.34 1.10 0.55 58.89 24.44 16.67


Offline 0.00 0.00 100.0 0.00 5.88 94.12

MCRtrain=61.15%

MCRtest =45.1%

MCRtrain=69.9%

MCRtest =34.4%

MCRtrain=66.0%

MCRtest =62.3%

MCRtrain=56.5%

MCRtest =45.5%

MCRtrain=55.2%

MCRtest =28.0%

MCRtrain=99.5%

MCRtest =79.0%

Mean Classification Rate (%)

Oversampling outperforms other samplings- Across Different Datasets

- Across various data preprocessing

Methods show different sensitivity to Sampling- More variation from sampling, coding & scaling than between methods

- Using different preprocessing variants is important in modeling

Various sophisticated extensions exist- SMOTE (Synthetic Minority Oversampling Technique)

- K-nearest Neighbor sampling (removal / creation)

- One-class learning etc. …

Extend your bad of tricks …- … and experiment with imbalanced sampling!

Summary

Sven F. CroneLancaster University Management School

Centre for Forecasting

Lancaster, LA1 4YX

email [email protected]

Questions?

1 1(1 ) t t tSY Y SY

Exploring Innovation

“Online panel” vs “Online „streaming/convenience”

sampling

Unfortunately, the presenters of iVOX &

Corelio cannot share their presentation

with the BAQMaR community due to

reasons of confidentiality!

http://www.facebook.com/photo.php?pid=544680&id=714207839

http://viewmorepics.myspace.com/index.cfm?fuseaction=viewImage&friendID=168908589&albumID=402129&imageID=1272581

http://viewmorepics.myspace.com/index.cfm?fuseaction=viewImage&friendID=168908589&albumID=402129&imageID=4249711