Exercise

Data Preparation

2

Modeling Example

From population of lapsing donors, identify individuals worth continued solicitation.

Business:

Objective:

National veterans’ organization

Source: 1998 KDD-Cup Competition via UCI KDD Archive

3

The Story

A national veterans’ organization seeks to better target its solicitations for donation. By only soliciting the most likely donors, less money will be spent on solicitation efforts and more money will be available for charitable concerns.

Solicitations involve sending a small gift to an individual together with a request for donation. Gifts include mailing labels and greeting cards.

Of particular interest is the class of individuals identified as lapsing donors. These individuals made their most recent donation between 12 and 24 months ago. The organization found that by predicting the response behavior of this group, they can use the model to rank all 3.5 million individuals in their database.

The current campaign refers to a greeting card mailing sent in 06/1997. The source of this data is the Association for Computing Machinery’s

(ACM) 1998 KDD-Cup competition.

4

Additional Data Preparation

Raw Analysis Data

95,412 Records481 Fields

Final Analysis Data

19,372 Records50 Fields

The raw analysis data has been reduced for the purpose of this course. A subset ofslightly over 19,000 records has been selected for modeling. As will be seen, thissubset was not chosen arbitrarily. In addition, the 481 fields have been reduced to 50.

5

Analysis Data Definition

CONTROL_NUMBERMONTHS_SINCE_ORIGININ_HOUSE

Donor master data

Unique Donor IDElapsed time since first donation1=Given to In House program,0=Not In House donor

6

Analysis Data Definition

OVERLAY_SOURCEDONOR_AGEDONOR_GENDER

Demographic and other overlay data

M=Metromail, P=Polk, B=bothAge as of June 1997Actual or inferred gender

PUBLISHED_PHONEHOME_OWNER

Published telephone listingH=homeowner, U=unknown

MOR_HIT Mail order response hit rate

7

PER_CAPITA_INCOME Income per capita in dollarsMED_HOUSEHOLD_INCOME Median income in $100s

Demographic and other overlay data

Analysis Data Definition

CLUSTER_CODESES

WEALTH_RATING

54 Socio-economic cluster codes5 Socio-economic cluster codes

10 wealth rating groups

INCOME_GROUP 7 income group levels

SES is a roll-up of the socio-economic field CLUSTER_CODE

8

PCT_OWNER_OCCUPIED Percent owner occupied housing

Demographic and other overlay data

Analysis Data Definition

MED_HOME_VALUE Median home value in $100s

URBANICITY U=urban, C=city, S=suburban,T=town, R=rural, ?=unknown

9

Census overlay data

Analysis Data Definition

PCT_MALE_MILITARY Percent male military in blockPCT_MALE_VETERANSPCT_VIETNAM_VETERANS

Percent male veterans in blockPercent Vietnam veterans in block

PCT_WWII_VETERANS Percent WWII veterans in block

10

Number card promotions last 12 mos.

Analysis Data Definition

Time

`94 `97`96`95 `98

Transaction detail data

NUMBER_PROM_12 Number promotions last 12 mos.CARD_PROM_12

97NK

11

Analysis Data Definition

Time

`94 `97`96`95 `98

97NK

Transaction detail data

FREQ_STATUS_97NK Frequency status, June `97RECENCY_STATUS_96NK Recency status, June `96

96NK

LAST_GIFT_AMT Amount of most recent donationMONTHS_SINCE_LAST Months since last donation

12

Analysis Data Definition

Time

`94 `97`96`95 `98

94NK

RECENT transaction detail data

RESPONSE_PROP Response proportion since June `94RESPONSE_COUNT

96NK

AVG_GIFT_AMTResponse count since June `94Average gift amount since June `94

RECENT_STAR_STATUS STAR (1, 0) status since June `94

The sampling method implies that no one made a donation between 6/1996 and 6/1997.However, for a limited number of cases, the number of months since last gift is fewerthan 12. This contradiction is not resolved in the data’s documentation, nor will it beresolved here.

13

Analysis Data Definition

Time

`94 `97`96`95 `98

94NK

RECENT transaction detail data

CARD_RESPONSE_PROP Response proportion since June `94CARD_RESPONSE_COUNT

96NK

CARD_AVG_GIFT_AMTResponse count since June `94Average gift amount since June `94

14

Analysis Data Definition

Time

`94 `97`96`95 `98

94NK

LIFETIME transaction detail data

PROM Total number promotions everGIFT_COUNT

96NK

AVG_GIFT_AMTTotal number donations everOverall average gift amount

PEP_STAR STAR status ever (1=yes, 0=no)

15

GIFT_RANGE Maximum less minimum gift amount

Analysis Data Definition

Time

`94 `97`96`95 `98

94NK

LIFETIME transaction detail data

GIFT_AMOUNT Total gift amount everGIFT_COUNT

96NK

Total number donations everMAX_GIFT Maximum gift amount

16

MONTHS_SINCE_LAST Last donation date from June `97

Analysis Data Definition

Time

`94 `97`96`95 `98

94NK

KDD supplied LIFETIME transaction detail data

FILE_AVG_GIFT Average gift from raw dataFILE_CARD_GIFT

96NK

MONTHS_SINCE_FIRST First donation date from June `97Average card gift raw data

17

Analysis Data Definition

Transaction detail data target definition

Time

`94 `97`96`95 `98

97NK

TARGET_B Response to 97NK solicitation (1=yes 0=no)TARGET_D Response amount to 97NK solicitation

(missing if no response)

18

Demonstration

Data set: PVA_RAW_DATA

Purpose: Get familiar with the data Basic decision modeling with tree, regression, and neural

network

Parameters: Prior probabilities: (0.05, 0.95) Profit matrix: ($14.62, -0.68) Target: TARGET_B (TARGET_D must be rejected)

19

Improving Regression Selection

0

15

30

45

60

25 50 75 100

Number of Variables

AllSubsets

StepwiseMin

utes

20

Improving Input Selection

Much of the success of a predictive model depends on input selection. Most input selection processes attempt to minimize input redundancy and maximize input relevancy.

Selection is usually using a heuristic search because the complexity of an exhaustive (all subsets) search increases exponentially in the number of inputs.

There exist branch-and-bound algorithms that approximate an exhaustive input search and run quite quickly for a reasonably small number of inputs. One algorithm, found in the SAS/STAT LOGISTIC procedure, actually runs faster than the usual forward, backward, and stepwise procedures.

While the example data set in this course has fewer than 60 inputs, many modeling data sets do not. Given the promise of an exhaustive search, it would be extremely desirable to reduce the input count without compromising the quality of the ultimate predictive model.

21

Improving Input Selection

Univariate Screening

Variable Clustering

Categorical Recoding

All Subsets Selection

22

Input Dimension Reduction

A three-phased approach is proposed for input dimension reduction in preparation for all subsets selection.

First, a univariate screening is performed to eliminate those inputs with little promise of target association. This must be done with care to avoid eliminating inputs whose predictive value occurs only in conjunction with other inputs.

Second, variable clustering techniques are used to group correlated interval inputs and minimize input redundancy.

Third, enhanced weight-of-evidence methods are used to effectively incorporate categorical inputs into the final model.

With the input dimension reduced, an all subsets search commences on the remaining inputs.

23

Univariate Screening

In this technique, inputs are screened based on their individual correlation with the target and only the inputs with the highest correlations are kept.

Unfortunately, this approach does not account for partial associations among the inputs. Inputs could be erroneously omitted or erroneously included. Partial associations occur when the effect of one input changes in the presence of another input.

A compromise devised to minimize the dangers of partial associations is to use univariate screening followed by liberal forward selection—not as a way of finding useful inputs, but rather as a way to eliminate clearly useless ones.

24

R-square Selection for Univariate Screening

The R-square selection approach has two phases. First, the input/target correlation is calculated for each

input. Each input with a correlation below the minimum R-square setting is rejected.

Second, a forward election is performed. The forward selection procedure terminates when all remaining inputs have a correlation below the specified stop R-square. These remaining inputs are also rejected.