"is data science inhibiting product evolution?", steven lemm, chief data officer at...

Is data science inhibiting product development?

Steven LemmChief Data Officer @ Outfittery GmbH

[email protected] | Data Natives 20162

WHAT’S A PRODUCTDefinitiona product is something that is created and then made available to external or internal customers (eg. a software application)

mailto:[email protected]?subject=Talk%20at%20Data%20Natives



Example e-commerce platformFrontend products: registration/authentication, check-out, recommender engine, article search, …Backend products: inventory control, order engine, logistic processes, payment service, payment control, …




Regis-tration

article search

article details

check-out

payment control

Example e-commerce platformFrontend products: registration/authentication, check-out, recommender engine, article search, …Backend products: inventory control, order engine, logistic processes, payment service, payment control, …

For the rest of the talk consider a simplified platform



Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.

PRODUCT DEVELOPMENT

3




PRODUCT DEVELOPMENT

3

Regis-tration

article search

article details

check-out

payment control

How to measure additional benefits?




KPI: search rank

KPI: revenue

PRODUCT DEVELOPMENT

3

Regis-tration

article search

article details

check-out

payment control

How to measure additional benefits?




Regis-tration

article searchVer. A article

detailscheck-

outpayment control

KPI: revenue

article searchVer. B

KPI: search rank

PRODUCT DEVELOPMENT

3

How to measure additional benefits? A/B testing!



Example smart payment control

Regis-tration

article search

article details

check-out

payment control

WHAT CHANGES WITH DATA SCIENCE?

4

ext. solvency score

payment control




Regis-tration

article search

article details

check-out

payment control


4

ext. solvency score+ cust. engagement




Regis-tration

article search

article details

check-out

payment control


4


Conversion

PnL




Regis-tration


detailscheck-

outpayment control

Revenue


search rank


4




Conversion


Regis-tration


detailscheck-

outpayment control

Revenue


search rank


4




Regis-tration

article details

check-out

payment control


HOW TO PROTECT AS PAYMENT?

5


Conversion



Hint 1: create a thorough monitoring of the data inputs, e.g.

Regis-tration

article details

check-out

payment control



5


Conversion




Regis-tration

article details

check-out

payment control



5


Hint 2: make payment control dependency on article search explicit

Conversion




Regis-tration

article details

check-out

payment control



5


Hint 3: document the dependencies and educate product managersHint 2: make payment control dependency on article search explicit

Conversion



Regis-tration


detailscheck-

outpayment control

Revenue


search rank

WHAT TO DO AS SEARCH?

6




Regis-tration


detailscheck-

outpayment control

Revenue


search rank


6


Step 0: connect with product manager of payment control first



Regis-tration


detailscheck-

outpayment control

Revenue


search rank

Step 1: make payment control independent on article search and run A/B test


6

ext. solvency score

payment control




Regis-tration

article details

check-out

payment control




6

ext. solvency score

payment control

Step 2: deploy article search and collect training data for new payment control logic




Regis-tration

article details

check-out

payment control




6


Step 2: deploy article search and collect training data for new payment control logicStep 3: train and deploy an updated payment control logic




Regis-tration

article details

check-out

payment control




6


Step 2: deploy article search and collect training data for new payment control logicStep 3: train and deploy an updated payment control logic


THIS MAKES PRODUCT DEVELOPMENT VERY EXPENSIVE



… Excecutives:• do not underestimate the costs of data science - there is no free lunch

•prioritize features according to , where total costs cover development, testing, data acqusition, data science

•bring together product management and data science as close as possible•centralise all A/B testing & personalisation efforts in a data department

… Product manager• map out all and document all data dependencies

… Data scientist• build a thorough monitoring & alerting systems for all inputs to your logics• prefer explicit over implicit data dependencies• create sparse solutions, i.e. limit the dependencies don’t take everyone’s data• educate product management

RECOMMENDATIONS TO …

7


Thank you

"is data science inhibiting product evolution?", steven lemm, chief data officer at...

Data & Analytics