frauddetection-09

8/6/2019 FraudDetection-09

1/49


2/49

2

Outline

Problem Description Cellular cloning fraud problem Why it is important Current strategies

Construction of Fraud Detector Framework Rule learning, Monitor construction, Evidence combination

Experiments and Evaluation Data used in this study

Data preprocessing Comparative results

Conclusion Exam Questions


3/49

3

The Problem


4/49


5/49

5

Cellular communications and Cloning

Fraud Mobile Identification Number(MIN) and

Electronic Serial Number(ESN)

Identify a specific account

Periodically transmitted unencrypted wheneverphone is on

Cloning occurs when a customers MIN and

ESN are programmed into a cellular phonenot belonging to the customer.


6/49

6

Interest in reducingCloning Fraud

Fraud is detrimental in several ways:

Fraudulent usage congests cell sites

Fraud incurs land-line usage charges Cellular carriers must pay costs to other

carriers for usage outside the home territory

Crediting process is costly to carrier and

inconvenient to the customer


7/49

7

Strategies for dealing with cloning fraud

Pre-call Methods

Identify and block fraudulent calls as they are made

Validate the phone or its user when a call is placed

Post-call Methods Identify fraud that has already occurred on an account so

that further fraudulent usage can be blocked

Periodically analyze call data on each account to determine

whether fraud has occurred.


8/49

8

Pre-call Methods

Personal Identification Number (PIN) PIN cracking is possible with more sophisticated

equipment.

RF Fingerprinting Method of identifying phones by their unique transmission

characteristics

Authentication Reliable and secure private key encryption method. Requires special hardware capability An estimated 30 million non-authenticatable phones are in

use in the US alone (in 1997)


9/49

9

Post-call Methods

Collision Detection

Analyze call data for temporally overlappingcalls

Velocity Checking Analyze the locations and times of consecutive

calls

Disadvantage of the above methods Usefulness depends upon a moderate level of

legitimate activity


10/49

10

Another Post-call Method

( Main focus of th

is paper ) User Profiling

Analyze calling behavior to detect usage anomaliessuggestive of fraud

Works well with low-usage customers Good complement to collision and velocity checking

because it covers cases the others might miss


11/49

11

Sample Frauded Account

Date Time Day Duration Origin Destination Fr aud1/01/95 10:05:01 Mon 13 minutes Brooklyn, NY Stamford, CT

1/05/95 14:53:27 Fri 5 minutes Brooklyn, NY Greenwich, CT

1/08/95 09:42:01 Mon 3 minutes Bronx, NY Manhattan, NY

1/08/95 15:01:24 Mon 9 minutes Brooklyn, NY Brooklyn, NY

1/09/95 15:06:09 Tue 5 minutes Manhattan, NY Stamford, CT

1/09/95 16:28:50 Tue 53 seconds Brooklyn, NY Brooklyn, NY

1/10/95 01:45:36 Wed 35 seconds Boston, MA Chelsea, MA Bandit

1/10/95 01:46:29 Wed 34 seconds Boston, MA Yonkers, NY Bandit

1/10/95 01:50:54 Wed 39 seconds Boston, MA Chelsea, MA Bandit

1/10/95 11:23:28 Wed 24 seconds Brooklyn, NY Congers, NY

1/11/95 22:00:28 Thu 37 seconds Boston, MA Boston, MA Bandit

1/11/95 22:04:01 Thu 37 seconds Boston, MA Boston, MA Bandit


12/49

12

The Need to be Adaptive

Patterns of fraud are dynamic banditsconstantly change their strategies inresponse to new detection techniques

Levels of fraud can change dramatically frommonth-to-month

Cost of missing fraud and dealing with false

alarms change with inter-carrier contracts


13/49

13

Automatic Construction of

Profiling Fraud Detectors


14/49

14

One Approach

Build a fraud detection system by classifyingcalls as being fraudulent or legitimate

However there are two problems that makesimple classification techniques infeasible.


15/49

15

Problems with simple classification

Context A call that would be unusual for one customer may be

typical for another customer (For example, a call placedfrom Brooklyn is not unusual for a subscriber who lives

there, but might be very strange for a Bostonsubscriber. )

Granularity At the level of the individual call, the variation in calling

behavior is large, even for a particular user.


16/49

16

The Learning Problem

1. Which call features are important?

2. How should profiles be created?

3. When should alarms be raised?


17/49

17

Detector Constructor Framework


18/49

18

Use of a detector ( DC-1 )Account-Day

Day Time Duration Origin Destination

Tue 1:42 10 min Bronx, NY Miami, FL

Tue 10:05 3 min Scarsdale, NY Bayonne, NJ

Tue 11:23 24 sec Scarsdale, NY Congers, NY

Tue 14:53 5 min Tarrytown, NY Greenwich, CT

Tue 15:06 5 min Manhattan, NY Westport, CT

Tue 16:28 53 sec Scarsdale, NY Congers, NY

Tue 23:40 17 min Bronx, NY Miami, FL

#calls from BRONXat night exceedsdailythreshold

Airtime fromBRONX at night

SUNDYairtimeexceeds dailythreshold

Value normalizationand weighting

1 27 0

ProfilingMonitors

FRAUD ALARM

Yes

EvidenceCombining


19/49

19

Rule Learning the 1st stage

Rule Generation

Rules are generated locally based ondifferences between fraudulent and

normal behavior for each account

Rule Selection

Then they are combined in a rule

selection step


20/49

20

DC-1 uses the RL program to generate rules withcertainty factors above user-defined threshold

For each Account, RL generates a local set of

rules describing the fraud on that account. Example:

(Time-of-Day = Night) AND

(Location = Bronx) FRAUD

Certainty Factor = 0.89

Rule Generation


21/49

21

Rule Selection

Rule generation step typically yields tensof thousands of rules

If a rule is found in ( covers ) manyaccounts then it is probably worth using

Selection algorithm identifies a small set ofgeneral rules that cover the accounts

Resulting set of rules is used to constructspecific monitors


22/49

22

Profiling Monitors the 2nd stage

Monitor has 2 distinct steps -

Profiling step: Monitor is applied to an accounts non-fraud usage to

measure accounts normal activity. Statistics are saved with the account.

Use step: Monitor processes a single account-day, references

the normality measure from profiling and generates anumeric value describing how abnormal the currentaccount-day is.


23/49

23

Most Common Monitor Templates

Threshold

Standard Deviation


24/49

24

Threshold Monitors


25/49

25

Standard Deviation Monitors


26/49

26

Example for Standard Deviation

Rule(TIME-OF-DAY = NIGHT) AND (LOCATION = BRONX) FRAUD

Profiling Step - the subscriber called from the Bronx anaverage of5 minutes per night with a standard deviation of

2 minutes. At the end of the Profiling step, the monitorwould store the values (5,2) with that account.

Use step - if the monitor processed a day containing 3minutes of airtime from the Bronx at night, the monitor

would emit a zero; if the monitor saw 15 minutes, it wouldemit (15 - 5)/2 = 5. This value denotes that the account isfive standard deviations above its average (profiled) usagelevel.


27/49

27

CombiningEvidence from the Monitors

th

e3rd

stage Train a classifier with attributes (monitor outputs) class label (fraudulent or legitimate)

Weights the monitor outputs and learns a

threshold on the sum to produce highconfidence alarms DC-1 uses Linear Threshold Unit (LTU)

Simple and fast

Feature selection Choose a small set of useful monitors in the

final detector


28/49

28

Data used in the study


29/49


30/49

30

Data Cleaning

Eliminated credited calls made to numbersthat are not in the created block

The destination number must be only called by

the legitimate user.

Days with 1-4 minutes of fraudulent usagewere discarded.

Call times were normalized to GreenwichMean Time for chronological sorting


31/49

31

Data Selection Once the monitors are created and accounts profiled, the system

transforms raw call data into a series of account-days using themonitor outputs as features

Rule learning and selection

879 accounts comprising over 500,000 calls

Profiling, training and testing

3600 accounts that have at least 30 fraud-free days of usagebefore any fraudulent usage.

Initial 30 days of each account were used for profiling.

Remaining days were used to generate 96,000 account-days.

Distinct training and testing accounts ,10,000 account-days fortraining; 5000 for testing

20% fraud days and 80% non-fraud days


32/49

32

Experiments and Evaluation


33/49

33

Output of DC-1 components

Rule learning: 3630 rules

Each covering at least two accounts

Rule selection: 99 rules

2 monitor templates yielding 198 monitors

Final feature selection: 11 monitors


34/49

34

The Importance OfError Cost

Classification accuracy is not sufficient to evaluateperformance

Should take misclassification costs into account

Estimated Error Costs: False positive(false alarm): $5

False negative (letting a fraudulent account-day goundetected): $0.40 per minute of fraudulent air-time

Factoring in error costs requires second training

pass by LTU


35/49

35

Alternative Detection Methods

Collisions + Velocities Errors almost entirely due to false positives

High Usage detect sudden large jump inaccount usage

Best Individual DC-1 Monitor (Time-of-day = Evening) ==> Fraud

SOTA - State Of The Art

Incorporates 13 hand-crafted profiling methods Best detectors identified in a previous study


36/49

36

DC-1 Vs. Alternatives

Detector Accuracy(%) Cost($) Accuracyat CostAlarm on all 20 20000 20

Alarm on none 80 18111 +/- 961 80

Collisions +

Velocities

82 +/- 0.3 17578 +/- 749 82 +/- 0.4

High Usage 88+/- 0.7 6938 +/- 470 85 +/- 1.7

Best DC-1 monitor 89 +/- 0.5 7940 +/- 313 85 +/- 0.8

State of the art(SOTA)

90 +/- 0.4 6557 +/- 541 88 +/- 0.9

DC-1 detector 92 +/- 0.5 5403 +/- 507 91 +/- 0.8

SOTA plus DC-1 92 +/- 0.4 5078 +/- 319 91 +/- 0.8


37/49

37

Call Classifier Detectors

Account context is important ( Rule learningstep in our framework )

Global example set taken from all accounts

loses context information about eachaccounts normal behavior

To illustrate the importance of contextinformation - Created two Classifiers ( CC1054, CC1861) which

lose context information , but have the advantageof profiling and monitoring


38/49

38

DC-1 Vs. Global Classifiers

Detector Accuracy(%) Cost($) AccuracyatCost

CC 1054 88 +/- 0.4 8611 +/- 531 88 +/- 0.6

CC 1861 88 +/-0.5 8686 +/- 804 88 +/- 0.6

DC-1 92 +/- 0.5 5403 +/- 507 91 +/- 0.8


39/49

39

Shifting Fraud Distributions

Fraud detection system should adapt to shiftingfraud distributions

To illustrate the above point - One non-adaptive DC-1 detector trained on a

fixed distribution ( 80% non-fraud ) and testedagainst range of 75-99% non-fraud

Another DC-1 was allowed to adapt (re-train itsLTU threshold) for each fraud distribution

Second detector was more cost effective than thefirst


40/49

40

Effects of Changing Fraud Distribution

0

0.2

0.4

0.60.8

1

1.21.4

75 80 85 90 95 100Percentage of non-fraud

Cost

Adaptive

80/20


41/49

41

DC-1 Component Contributions(1)

High Usage Detector

Profiles with respect to undifferentiatedaccount usage

Comparison with DC-1 demonstrates thebenefit of using rule learning

Best Individual DC-1 Monitor

Demonstrates the benefit of combining

evidence from multiple monitors


42/49

42

DC-1 Component Contributions(2)

Call Classifier Detectors

Represent rule learning without the benefit ofaccount context

Demonstrates value of DC-1s rule generationstep, which preserves account context

Shifting Fraud Distributions

Shows benefit of making evidence combinationsensitive to fraud distribution


43/49

43

Conclusion DC-1 uses a rule-learning

program to uncover indicators of

fraudulent behavior from a largedatabase of customertransactions.

Then the indicators are used tocreate a set of monitors, which

profile legitimate customerbehavior and indicate anomalies.

Finally, the outputs of themonitors are used as features in asystem that learns to combine

evidence to generatehigh-confidence alarms.


44/49

44

Conclusion

Adaptability to dynamic patterns of fraud can beachieved by generating fraud detection systemsautomatically from data, using data mining

techniques DC-1 can adapt to the changing conditions typical of

fraud detection environments

Experiments indicate that DC-1 performs better than

other methods for detecting fraud


45/49

45

Exam Questions


46/49

46

Question 1

Whatare the two major frauddetectioncategories, differentiate them, andwheredoes DC-1 fall under?

Pre Call Methods

Involves validating the phone or its user when a call isplaced.

Post Call Methods DC1 falls here Analyzes call data on each account to determine

whether cloning fraud has occurred.


47/49

47

Question 2

Why is "Context" important in successfullydetectingfraud?

A call that would be unusual for one customer wouldbe typical for another customer.

For example, a call placed from Brooklyn is notunusual for a subscriber who lives there, but might bevery strange for a subscriber living in Boston. Contextretains information about the normal behavior of the

account in which fraud occurred.


48/49

48

Question 3

Profiling monitors have two distinct stagesassociatedwith them. Describe them.

Profiling step:

The monitor is applied to a segment of an accountstypical (non-fraud) usage in order to measure theaccounts normal activity.

Use step:

The monitor processes a single account-day at a time,

references the normalcy measure from the profilingstep and generates a numeric value describing howabnormal the current account-day is.


49/49

49

The End.

Questions?

frauddetection-09

Documents