big data banking insights - roma tre universitytorlone/bigdata/s4-unicredit.pdf · big data banking...
TRANSCRIPT
Maximizing Banking Value with Big Data Analytics
Chiara Bartalotta, Big Data Engineer
Rome, May 2016
Big Data Banking Insights
UniCredit Business Integrated Solutions
Introduction • UniCredit at a Glance • Data & Analytics Organization • The New Big Data Engineer • Big Data Challenges
Data Processing Steps Machine Learning Use Cases Technology
2
1
2
3
4
5
AGENDA
UniCredit Business Integrated Solutions
3
* Source: UniCredit Company Profile, data as at March 31, 2016
UNICREDIT AT A GLANCE Introduction 1 2 3 4 5
• Banking operations in 17 countries
• International network spanning: ~50 countries
• Global player in asset management: 218.7 bn in managed assets
• Market leader in Central and Eastern Europe
leveraging on the region's structural strengths
Over 143.000 employees
7.839 branches
UniCredit Business Integrated Solutions
4
UniCredit Business Integrated Solutions is the first concrete milestone within the Group Strategic Plan 2012 announced in November 2011 to be achieved. Owned by UniCredit, is created from the integration and consolidation of 16 Group companies (among them UGIS, UCBP, URE, UC) and is dedicated to providing services in the sectors of Information and Communication Technology (ICT), Back Office and Middle Office, Real Estate, Security and Procurement. A new business model, unique in the European banking sector, focused on Business needs (i.e. Commercial Banking, Global Markets, CEE), not only on providing services.
AUSTRIA
ITALY ROMANIA
POLAND
HUNGARY
UK
SLOVAKIA
GERMANY � ~ 10.000 Fte’s*
( y/y pro forma)
■ 11 Countries**
■ 3 Wholly-owned subsidiaries*
* Data as at December 31, 2014. ** UniCredit Business Integrated Solutions operates also in 2 branches, one located in New York and one in Singapore.
CZ REP.
� NET EQUITY: € 367,537,088* � TOTAL REVENUE: €
2,438,382,428*
� NET PROFIT: € 3,896,618*
UNICREDIT BUSINESS INTEGRATED SOLUTIONS: IDENTITY CARD Introduction 1 2 3 4 5
UniCredit Business Integrated Solutions
DATA & ANALYTICS ORGANIZATION
5
Data Scientists
� Data acquisition � Data-flow management � Multiple data sources
integration � Data cleaning
� Mathematical and Statistics algorithms inception
� Machine Learning techniques application
� Value from data extraction
� Systems deployment and migration
� Application management and monitoring
� Data center operations monitoring
� Scope definition � Cost, time and quality
control � Customer relationship
� Architectural Design � Innovation inception � Big data solutions
development � New technologies exploration
UniCredit Business Integrated Solutions Big Data Teams
Infrastructure Data Ingestion
Developers
Project Managers
Introduction 1 2 3 4 5
UniCredit Business Integrated Solutions
Introduction
UniCredit Business Integrated Solutions
Software Engineering
Analytical mindset Mathematical
knowledge Distributed computing experience
Business focus
Database systems proficiency
THE NEW BIG DATA PROFESSIONIST 1 2 3 4 5
6
UniCredit Business Integrated Solutions
DATA & ANALYTICS NEEDS…
7
Big Data Engineers The candidate will enter our team which develops the next-generation technologies that change how millions of customers connect, explore, and interact with the bank. As a big data engineer, you are expected to give your contribution to projects as our fast-paced business grows and evolves. We need our engineers to be versatile and passionate to tackle new problems as we continue to push technology forward. With your technical expertise you manage individual projects' priorities, deadlines and deliverables. You design, develop, test, deploy, maintain, and enhance software solutions in a big data environment.
Data Scientists As a Data Scientist, you should be passionate about using data to drive advanced analytics. You are able to work cross-functionally with both business and big data engineers to effectively deliver actionable results. You are a well-rounded top performer who is able to quickly grasp data and critically think through strategic issues the next. You are able to communicate effectively while delivering complex data-driven findings. The ideal candidate is an independent, solution-oriented thinker able to apply analytical rigor and statistical methods, and driving toward insights and solutions.
Introduction 1 2 3 4 5
UniCredit Business Integrated Solutions
BIG DATA CHALLENGES
8
Secure data storage and operation logs, data provenance and lineage
Secure computation in distributed programming methodology and non relational data base architecture
Infrastructure Security Data Management
Performance, system responsiveness, disaster recovery, resources allocation
Scalability
Access polices, laws and regulations, customer and employers data protection
Privacy
Shortage of skilled workers, small communities, lack of awareness
Talent Gap Unclear picture of Big Data World, many technologies existing making the choice hard, hard to tell Big Data and Not Big Data apart
Uncertainty
Raising costs of infrastructure and data storage, high costs of operation and management and integration within existing ecosystems
Costs
Pay attention to…
Introduction 1 2 3 4 5
UniCredit Business Integrated Solutions
Introduction
Data Processing Steps Machine Learning Use Cases Technology
9
1
2
3
4
5
MOVING ON
UniCredit Business Integrated Solutions
DATA PROCESSING STEPS
10
1 2 3 4 5 Data Processing Steps
Storage Processing Acquisition Exploration Elaboration
UniCredit Business Integrated Solutions
DATA PROCESSING STEPS
11
1 2 3 4 5 Data Processing Steps
Storage Processing Acquisition Exploration Elaboration
UniCredit Business Integrated Solutions
DATA ACQUISITION
12
Trustworthiness of the original sources
Data Quality and Provenance Different sources: application databases (OLTP), datawarehouse (OLAP), log files, documents, etc.
Data Structure: different types of format
1 2 3 4 5 Data Processing Steps
UniCredit Business Integrated Solutions
DATA PROCESSING STEPS
13
1 2 3 4 5 Data Processing Steps
Storage Processing Acquisition Exploration Elaboration
UniCredit Business Integrated Solutions
DATA PROCESSING
14
� Anonymization � Filtering of irrelevant data � Data sampling � Data conversion and integration � Dataset joining
1 2 3 4 5 Data Processing Steps
UniCredit Business Integrated Solutions
DATA PROCESSING STEPS
15
1 2 3 4 5 Data Processing Steps
Storage Processing Acquisition Exploration Elaboration
UniCredit Business Integrated Solutions
DATA STORAGE
16
� Scalable and distributed systems � Redundancy � Disaster recovery � Different technologies
Availability Consistency
Partition Tolerance
CA
CP AP
CAP theorem
1 2 3 4 5 Data Processing Steps
UniCredit Business Integrated Solutions
RESOURCES AND PROCESS DISTRIBUTION
17
Scale up Scale out
Add resources in one machine Add resources in multiple nodes and get parallelism
1 2 3 4 5 Data Processing Steps
UniCredit Business Integrated Solutions
DATA PROCESSING STEPS
18
1 2 3 4 5 Data Processing Steps
Storage Processing Acquisition Exploration Elaboration
UniCredit Business Integrated Solutions
DATA ANALYSIS
19
� Goal: extract value from data � Big Problem: understand results and
find better pattern to extract useful information from the large amount of data
� Exploratory data analysis � Combine statistics and probability models
to explore data and estimate values or make predictions
� Machine Learning Techniques
1 2 3 4 5 Data Processing Steps
UniCredit Business Integrated Solutions
DATA ANALYSIS
20
REAL TIME
Scheduled analysis to collect, clean and process data in order to produce results from previously gathered data
BATCH
REAL TIME
BATCH
Data processing at acquisition time enabling
fast results' elaboration
1 2 3 4 5 Data Processing Steps
UniCredit Business Integrated Solutions
Introduction
Data Processing Steps Machine Learning Use Cases Technology
21
1
2
3
4
5
MOVING ON
UniCredit Business Integrated Solutions
MACHINE LEARNING
22
Knowledge acquisition automatization algorithms’ design and implementation through: � Data-driven models � Continuous learning based on provided input
Goal
Main Machine Learning Areas
Supervised Learning Unsupervised Learning
1 2 3 4 5 Machine Learning
UniCredit Business Integrated Solutions
MACHINE LEARNING: SUPERVISED LEARNING
23
01 02 03
Training with data that has a well known class
After the training, the classifier can assign the correct class to new data
Continuously learning with new training
datasets
1 2 3 4 5 Machine Learning
The desired output for each input is known a priori. The goal is to discover a rule to map a general input to the correct output.
Example - Classification
UniCredit Business Integrated Solutions
MACHINE LEARNING: UNSUPERVISED LEARNING
24
The output for each input is not known a priori. The goal is to discover pattern in data and find structures in inputs.
Clustering Discover a structure in the dataset, divide inputs in different groups according to their features.
Predict users’ interests, filter items following user’s engagement.
Collaborative Filtering
Examples
1 2 3 4 5 Machine Learning
UniCredit Business Integrated Solutions
Introduction
Data Processing Steps Machine Learning Use Cases
• Inter/Intra-Bank Risk Detection • CRM • My Business View • Call Center Support
Technology
25
1
2
3
4
5
AGENDA
UniCredit Business Integrated Solutions
Inter/Intra-Bank Risk Detection – CLIENTS’ NEEDS
26
Determine and classify UniCredit-related inter/intra-bank payments’ risk
Goal
1 2 3 4 5 Use Cases
UniCredit Business Integrated Solutions
Inter/Intra-Bank Risk Detection – EXAMPLE
01
02
03
A transaction event is recorded
Does the transaction chain involve a high-risk step?
Classify risk and notify Security Department
1 2 3 4 5 Use Cases
27
UniCredit Business Integrated Solutions
Inter/Intra-Bank Risk Detection – DATA PROCESSING
INPUT
CHAIN EXPLORATION
OUTPUT
Identify Country and transaction Bank
Risk Category
Validate history and next step against security rules
1 2 3 4 5 Use Cases
28
UniCredit Business Integrated Solutions
Inter/Intra-Bank Risk Detection – ARCHITECTURE
29
Transactions
Branch Notifications
Message Normalizer
Listener
Transaction Processor
Branch Data Rule Engine
Publisher
Fraud Catcher
Clean Transaction Queue
Branch Delay Queue
Fraud Queue
Transaction Queue
E-mail Notification
Web Service
1 2 3 4 5 Use Cases
UniCredit Business Integrated Solutions
Inter/Intra-Bank Risk Detection – APPLICATION LOGIC
Business Rules Logic Business logic is decoupled from application, enabling scalability in rules’ editing and adding.
Application Logic All components’ development and integration logic necessary to keep application running smoothly.
1 2 3 4 5 Use Cases
30
UniCredit Business Integrated Solutions
Introduction
Data Processing Steps Machine Learning Use Cases
• Inter/Intra-Bank Risk Detection • CRM • My Business View • Call Center Support
Technology
1
2
3
4
5
AGENDA
31
UniCredit Business Integrated Solutions
CRM – CLIENTS’ NEEDS
Catalyze marketing campaigns planning and carriage via a customer behavior analytics platform
Goal
1 2 3 4 5 Use Cases
32
UniCredit Business Integrated Solutions
CRM – EXAMPLE
01
02
03
Mario Rossi buys a new motorcycle
May Mario be interested in “Guida Protetta” assurance?
How can we validate the match and notify Mario?
1 2 3 4 5 Use Cases
33
UniCredit Business Integrated Solutions
CRM – DATA PROCESSING
INPUT
DATA ENRICHMENT
OUTPUT
TRIGGER
Identify transaction type � Credit Card � Bank Transfer � Debit Card � …
Potential product of interest
Enrich Information with customer data and
transaction’s market category
User notification / Product proposal
1 2 3 4 5 Use Cases
34
UniCredit Business Integrated Solutions
CRM – ARCHITECTURE
Data Preparation
Machine Learning
Model Repository
Events
Enrichment
Business Interface
Filtering & Scoring
Training / PMML
Schema Analysis Model
Configuration
Campaign Manager
1 2 3 4 5 Use Cases
35
UniCredit Business Integrated Solutions
Introduction
Data Processing Steps Machine Learning Use Cases
• Inter/Intra-Bank Risk Detection • CRM • My Business View • Call Center Support
Technology
1
2
3
4
5
AGENDA
36
UniCredit Business Integrated Solutions
My Business View – CLIENTS’ NEEDS
Offer end user a set of reports containing aggregated data for a merchant’s issuer, acquirer and terminal management activities
Goal
1 2 3 4 5 Use Cases
37
UniCredit Business Integrated Solutions
My Business View – EXAMPLE
01
02
03
Mario Rossi wants to analyze his bookshop’s trends
How could we support him?
Mario can review his activity’s KPIs through MyBusiness View
1 2 3 4 5 Use Cases
38
UniCredit Business Integrated Solutions
My Business View – DATA PROCESSING
INPUT
RULE ENGINE
OUTPUT
Transactions carried on in the activity
Activity reporting through dashboards
Perform aggregation and compute KPIs
1 2 3 4 5 Use Cases
39
UniCredit Business Integrated Solutions
My Business View – ARCHITECTURE
Web Service
Merchant Data
Card Data Clustering
Cache
1 2 3 4 5 Use Cases
40
UniCredit Business Integrated Solutions
Introduction
Data Processing Steps Machine Learning Use Cases
• Inter/Intra-Bank Risk Detection • CRM • My Business View • Call Center Support
Technology
1
2
3
4
5
AGENDA
41
UniCredit Business Integrated Solutions
Call Center Support – CLIENTS’ NEEDS
Monitoring customer calls, requests and issues raised through the call center system
Goal
1 2 3 4 5 Use Cases
42
UniCredit Business Integrated Solutions
Call Center Support – EXAMPLE
01
02
03
Mario Rossi calls contact center to raise a complaint: his credit card is not swiping correctly lately
Operator sees that Mario is a multi-year customer and he has raised 3 complaints in the last 2 months
Operator proposes Mario to replace his card with a new “Flexia Classic” card discounting his 1st year fee
1 2 3 4 5 Use Cases
43
UniCredit Business Integrated Solutions
Call Center Support – DATA PROCESSING
INPUT
PROCESSING ENGINE
OUTPUT
Operator inserts caller’s Name into the dashboard
Customer history and potential suggestions
Extract and merges customer data from
heterogeneous sources
1 2 3 4 5 Use Cases
44
UniCredit Business Integrated Solutions
Call Center Support – ARCHITECTURE
Web Service
Indexed Files Customer Data
Service Management Data
1 2 3 4 5 Use Cases
45
UniCredit Business Integrated Solutions
Introduction
Data Processing Steps Machine Learning Use Cases Technology
1
2
3
4
5
MOVING ON
46
UniCredit Business Integrated Solutions
TECHNOLOGIES 1 2 3 4 5 Technologies
47
UniCredit Business Integrated Solutions
GITHUB OPEN REPOSITORY
https://github.com/UniCreditDnA 1 2 3 4 5 Technologies
48