to catch a thief fighting respondent fraud

SSI Conden*al 2013 Survey Sampling Interna6onal

| 1 |

With the global leader in sampling and data services To Catch a Thief:

Figh6ng Respondent Fraud Jared Schiers

VP Access, Supply & Engagement


| 2 |

Taking a holis6c view of data quality

1. Sample representa0on: How eec*vely does our sample resemble the popula*on under study?

2. Respondent accuracy:

How eec*vely do the answers resemble the respondents true feelings?


| 3 |

3 Issues with respondent level data quality

1. Respondent is unable to give accurate informa0on (instrument) Understand the ques6on being posed Retrieve the relevant experience (i.e. remember it) Judge it (i.e. relate it to the ques6on) Be able to respond with the available answers op6ons

2. Respondent isnt mo0vated to give accurate informa0on (process) Non deliberate lying (e.g. social desirability bias) InaOen6on (boredom, fa6gue) Task diculty and Incen6ve mismatch

3. Respondent is a thief (fraud)


| 4 |

Luckily the fraud problem is SMALL

Fraudsters make up a very small % of all respondents Kantar Opera6ons reported iden6fying 1.3% of their panelists as suspicious at the ASC Conference, London 2008

Most of these bad accounts are iden0ed before they par0cipate in research

With focus, fraudsters can be iden0ed because their survey answers look dierent (dont know the topic) and appear in behavioral clusters


| 5 |

But fraud is a problem we share as an industry

1. How many of us have accepted that there are fraudsters in our data?

2. How many of us have ques0oned interes0ng results from our data?

3. How many of us have seen the cost of research rise in data collec0on and in our own 0me spent ques0oning data?


| 6 |

Why would thieves target market research?

1. We provide incen0ves to par0cipate

2. Our exper0se is understanding marketsnot security

3. Techniques used against one MR company are oWen eec0ve against others

4. Law enforcement is focused on other high-risk ver0cals (e.g. online banking)


| 7 |

Whats in it for the fraudster?

Market researchers generally provide only a very small Thank You for par6cipa6on in research


| 8 |

The carrot increases with scale and automa6on

The incen6ve to steal increases with capability to automate the tasks required to achieve pay-out

As the incen6ve to steal increases so does the sophis6ca6on of the fraudster


| 9 |

NO Silver Bullets Technology alone cant prevent fraud

Fraud preven6on in MR is following the evolu6on of chess 1. Human phase:

Grandmasters dominate the game of chess MR fraud specialists control fraud Experts can adapt and react, but impossible to evaluate every possible piece of data Adaptable, but not easily scalable


| 10 |


Fraud preven6on in MR is following the evolu6on of chess

2. Technology phase: IBMs Deep Blue defeats GM Garry Kasparov MR introduces technology solu6ons like TrueSample, RelevantID, and others Deep Blue can run through every game ever played, but cant conceptualize new strategies Scalable, but not quickly adaptable


| 11 |


Fraud preven6on in MR is following the evolu6on of chess 3. Hybrid phase:

Games played by teams (GM Garry Kasparov teams with Deep Fritz 14) Strategy driven by the grandmaster and the poten6al itera6ons of the move run by the machine Adaptability and scalability


| 12 |

8 Strategies to prevent and manage fraud

1. Anchor respondent iden6ty to the physical world 2. Mine observed behavioral data to iden6fy thieves 3. Leverage technology to generate data and automate fraud preven6on

4. Conrm topical understanding 5. Delay respondent pay-out 6. Partner with reward vendors to implement safeguards 7. Oer less abrac0ve rewards (to fraudsters) 8. Reduce the carrot through itera6ve evolu6on


| 13 |

1. Anchor respondent to the physical world

Mailing address Telephone #

Bank account # Photo ID

Drivers license #

Choosing the right anchor is cri0cal Physical anchors are generally more dicult to create en-masse

than digital anchors

By increasing the diculty to mass replicate accounts, we decrease the aOrac6veness of the fraud opportunity

More eec6ve if you can validate the anchor through process or third party


| 14 |


Telephone 2-factor authen0ca0on Respondent provides a live telephone # for a voice call or SMS message We deliver a PIN via voice or SMS Respondent enters that valid PIN to complete reward redemp6on


| 15 |


Postal redemp0on restric0on A physical check is the only reward available for the rst redemp6on Respondent provides the physical address and name for the check to be sent

The respondent cashes the check valida6ng their name and address


| 16 |

2. Mining observa6onal data

Trust is gained in inches and lost in miles Respondents constantly exhibit behavior while they are in our systems Good behavior over 6me conrms legi6macy Anomalous behavior is a trigger to suspect fraud A good respondent can trigger 1 or 2, but not 3, 4, or 5 or more

Over-qualica6on Geographic inconsistency Survey overlapping Over-par6cipa6on in high

incen6ve projects

Over-ac6ve PaOerned email format IPs or ISP of known fraudsters Consistently odd survey length Par6cipa6on at odd 6mes

A few of the 1,000s of characteris0cs to watch for


| 17 |

2. Fraudsters can be iden6ed in clusters

Behavior considered good from a single respondent illustrates fraud when exhibited by 100 respondents.

1 respondent from Michigan signs up Oct 15th at 3:15 PM, completes 5 surveys within 72 hours, and redeems on Oct 18th.

100 respondents exhibit the same behavior

1 respondent signs up with the email address [email protected]

100 respondents exhibit the same behavior [email protected] [email protected]

1 respondent qualies for 5 out of 5 surveys they aOempt.

100 respondents from the same IP address take the same 5 surveys with the same qualica6on rate


| 18 |

3. Leverage technology to automate

Technology allows us to act on a broader scale to behavioral paOerns and vulnerabili6es found during data mining.

Every technology has its own vulnerabili6es, combining many of them allows us to mi6gate the risk.

Most technologies are rules based and require the user to tune their sensi6vity over 6me.


| 19 |

4. Conrming topical understanding

Fraudsters can be iden6ed by tes6ng their understanding of the topic during the survey instrument They dont understand the topic under study They generally dont read the ques6ons they are answering

Dicult to create many unique and relevant answers

Good at iden6fying fraud

Requires post-hoc evalua6on

Open-Ended Trap Ques0ons Fraudsters are good at automa6ng

processes

Rela0vely simple to iden0fy and script the survey path

BeOer at iden6fying inaOen6on from good respondents

Close-Ended Trap Ques0ons


| 20 |

4. Important on low incidence projects

How would you describe yourself when it comes to health, diet, and exercise?

1. Professional athlete - I compete professionally in sports or races 2. Compe66ve athlete - I compete in sports or races, but not professionally 3. Ac6ve lifestyle - I engage in regular exercise, ac6vity and play 4. Health conscious - I manage my health through diet and exercise 5. None of the above

An honest respondent has a 0.00524% chance of qualifying 315 million people in the US (Census) 16,500 professional athletes in the US (Bureau of Labor

Sta6s6cs) A fraudster has a 20%+ chance of qualifying 1 in 5 chance of qualica6on if chosen at random Or more if the qualifying criteria is easily iden6able

1 in 20,000

4,000 in 20,000

Low incidence and high incen0ve projects are vulnerable to fraud abacks


| 21 |

5. Delay respondent pay-out

Fraudsters prefer in-and-out fraud opportuni6es The longer the delay between account registra6on and payout, the less aOrac6ve the fraud opportunity

The more 6me a fraudster spends in the system the more data they generate

Increasing the likelihood a fraudster gets caught and exposes their signature

New account block Claim process is unavailable un6l a new account has reached a pre-dened age

Claim delay Rewards can be claimed, but delivery is delayed for a pre-dened interval, while the account is scru6nized


| 22 |

6. Safeguards built into the reward

PayPal veried: physical anchor, nancial sector iden6ty control, and a uniqueness check Respondent provides PayPal with a valid bank account or credit card and address

PayPal validates that account is ac6ve, owned by the respondent, and is unique to this PayPal user

Respondent must provide us with the email address aOached to the veried account

Reward redemp6on is our last chance to stop a fraudster Reward vendors have safeguards against fraud Leveraging those to stop the fraudster from receiving their reward

No reward = no incen6ve to con6nue the aOack


| 23 |

7. Oer less aOrac6ve rewards (to fraudsters)

Informa0on rewards (e.g. a copy of the survey results) are much less appealing to fraudsters: they arent invested in the research

Charitable rewards (e.g. Red Cross dona6ons) are much less appealing to fraudsters: we havent run across Robin Hood yet


| 24 |

8. Itera6ve improvements to reduce the carrot

1.Diverse use of strategies increases the eort required for fraud 2.Con6nuous evolu6on and itera6on keeps the barriers high 3.We only need to out run other fraud opportuni6es to be protected


| 25 |

Apply these strategies to prevent and manage fraud

1. Anchor respondent iden6ty 2. Mine behavioral data 3. Leverage technology 4. Conrm understanding 5. Delay pay-out 6. Partner with reward vendors 7. Less abrac0ve rewards 8. Reduce the carrot


| 26 |

But apply them holis6cally

Applying fraud preven6on strategies across every stage in a respondents lifecycle increases the eec6veness


| 27 |

And balance respondent strain and fraud preven6on

Respondent strain

Fraud preven0on

Its good to remind ourselves that the good guys (99%) experience the same security measures we use to deter the fraudsters (1%) Too much respondent strain and we deter good respondents from

par6cipa6ng in research, adversely impac6ng our sample frame

Too lax in fraud preven6on and we allow fraudsters to par6cipate, adversely impac6ng our data


| 28 |

For more informa6on Email us at [email protected]: If you would like to request addi6onal informa6on, please put webinar in the subject line and request one or both of the following in the body of your email: Todays presenta6on Point of view (POV) on Impact of Fraud

Please provide your name, 6tle, company.

| 28 |


Ques6ons


To Con6nue the Conversa6on

Visit our Facebook page: facebook.com/surveysampling

or Tweet #SSIWebinar

| 30 |


| 31 | | 31 |

surveysampling.com [email protected]

[email protected]

to catch a thief fighting respondent fraud

Documents