predictive analytics in practice

P r e d i c t i v e a n a l y t i c s i n p r a c t i c e

Dave EgleseUniversity of Liverpool

London, UK | February 2016

What is Predictive Analytics?

# H U E M E A 1 6

# H U E M E A 1 6

We’ll be looking at what Predictive Analytics actually is

and how we can use this to inform Marketing and

Recruitment Strategy and Tactics

It may be big, it may be scary, but Predictive Analytics can give you new perspectives

To understand what Predictive Analytics is.

To relate examples you may have come

across in the sector and in other industries to

understand the breadth and range of

applications.

To provoke thinking about the

problems/situations you want to solve or

investigate to make changes.

1 Understanding that there are many different

tools from Excel to data mining in

programming languages like R and Python.

I’m using SPSS Modeler in this presentation.

Looking at a CRISP – DM process as a

framework

Measuring and evaluating engagement through

CRM and conversion activities to

optimise/prioritise/identify opportunities and

reduce threats.

2

3

4

5

6

What on earth? How?

Where is this happening?

Where do I start? What does this have to do with CRM?

Yeah…but, what’s the process?

Today’s outcomes

# H U E M E A 1 6

Examples

Drag or drop your photograph here

# H U E M E A 1 6

Google notification the other day

# H U E M E A 1 6

Examples continued…(How an UBER style service may use PA)

• Overview of an Uber style model:

# H U E M E A 1 6

What new data are we creating with Predictive Analytics?

Estimates, Forecasts, Probabilities, Recommendations, Propensity Scores (Lead

Scoring), Classifications etc.


Problems and situations?

# H U E M E A 1 6

Some problems may we look at in universities using PA?


# H U E M E A 1 6

Let’s take prioritisation as an example - Admissions

• Should we treat every application the same when certain factors may indicate a higher propensity to register or a higher desirability based on other set criteria?

• One test we’re looking at is for the introduction a level of prioritisation to improve response times for certain applicants (initially focussing on International PGT students)

What’s in the Model?

# H U E M E A 1 6

Modelling using historical data

• We can look at demographics – age, region, gender

• We can look at the application detail – application time, subject, qualifications, school, provider etc.

• We need known outcomes from data to base a model on– THANKFULLY we keep a recent record of previous cycles of admissions data that we can interrogate

• Any created model can be applied to new data to get probabilities or in other words…predictions

• Created models (patterns and formulas) need to be examined and tested thoroughly to ensure you can select a winning model.

# H U E M E A 1 6

CRISP-DM - Cross-Industry Standard Process for Data Mining

1. Business Understanding – To improve turnaround times for valuable applications to improve conversion – set objectives for evaluation purposes, and understand how this data will be used operationally.

2. Data Understanding –– What data sources can we use (Application DB/warehouse, CRM?)

– What fields will effect objective – explore data

– Essential – what is the target field – “Registered Student”

3. Data Preparation – this is where you should be spending your time. Getting the data together in correct format – integrating data, banding variables (perhaps application month?)

4. Modelling – Run data through model to generate results– Data led or hypothesis led – what variables are you including?

5. Evaluate – run models with know outcome (70% train, 30% test, possibly also an evaluation set)

6. Deploy winning model … against new applications that are received to give a probability score to be processed.

# H U E M E A 1 6

Example using SPSS Modeler

1. After aggregating data needed I add the testing records into SPSS Modeler as a node

2. This could be may types of files, but this one is an SPSS var file. This is then connected to a filter node where I choose which fields I want any model to consider.

# H U E M E A 1 6

Example using SPSS Modeler continued…

3. These is the first attempt at filtering the fields I want to include:

4. You can see many are allowed to pass.

5. Others are not considered worthy to be put in the model

6. Determine what role each field has in the model using Type node

# H U E M E A 1 6


7. This is where you can choose the data types and the role the field plays in the model:

# H U E M E A 1 6


8. Now we can see if a model can be run.

9. Skipping a few steps I’m going to run the data in 2 sets. This is partitioning the data

– 1 step will be to generate the model

– the other will be to test the model to see if it has worked correctly.

10. I can run an automated mode to see if the software can decide which statistical model(s) to use:

# H U E M E A 1 6


11. The autoclassifier node will look at different models and suggest a shortlist based on the accuracy achieved. I could use the result or use it as a guide to look at the individual models in detail:

# H U E M E A 1 6

Example using SPSS modeler continued…

• CHAID model – Easy to understand and present

• Splits the data into a decision tree based with the most important factors at the top of the tree

– Can develop ensemble models to improve complexity/considered factors

– Settings can also be made in the model to determine how many splits you want to allow in the data

• It will tell also tell us the most important factors in the data affecting the model:

• Will give us a view of whether a registration is likely or not, but crucially in this example where conversion is very low anyway it will give us a propensity score and an adjusted propensity score

# H U E M E A 1 6

Using a propensity score

• The propensity score used can be the lead scoring to apply to new applications without a decision, if you feel you have developed the model to the required accuracy

# H U E M E A 1 6

Apply model to new data

• The model node created can be connected to a new set of data using the same field names, although this time there doesn’t need to be a target.

• The propensity scores given can be use to create priority lists within the admissions service to process in order to improve turnaround times for applicants and improve conversion from these applications

# H U E M E A 1 6

CRM Sources

• Many of you will be evaluating how successful your CRM campaigns are – can we take results of these evaluations and test this with admissions data?

• We plan to test the following CRM activity alongside application data:

– Opening conversion e-mails

– Interacting with x e-mails may be a good indicator of engagement

– Registered/attended within designated events

– Took part in a call campaign

– Attended webinar

# H U E M E A 1 6

In summary

Enquiring, applying and registering at a university is normally a long journey and there are many TOUCH/DATA POINTS along the way to create rich data that we can use to enhance our marketing strategies and recruitment tactics to facilitate these journeys.

Questions?

[email protected]

predictive analytics in practice

Education