dwh & bi in banking @ et 2 nov 2004 chandrasekhar
DESCRIPTION
Darawarehousing and Business Intelligence in BankingTRANSCRIPT
![Page 1: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/1.jpg)
Treasury
International
BI and Data Warehousing: How will banks maximize their returns
out of them effectively?
V ChandrasekharBank of Baroda
![Page 2: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/2.jpg)
InternationalTreasury
Agenda• Banking Scenario• Overview
– OLTP– DSS– EDWH– BI
• Indian Banking Needs• Issues in Indian Banks• Critical Success Factors
![Page 3: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/3.jpg)
InternationalTreasury
Banking Scenario• Globalisation• Competition• Reduced Margins• Increased complexity• Problems of Size• M&A• Regulatory Controls - Risk Management• Agility
![Page 4: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/4.jpg)
InternationalTreasury
Data But No Information • Data .. Data every where but not a bit useful• Inaccuracy • Incomplete• Delays • Fragmented
All human knowledge is captured and stored way back in 2000 and is doubling
every 18 months
![Page 5: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/5.jpg)
InternationalTreasury
Need
• DSS• BI
![Page 6: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/6.jpg)
InternationalTreasury
The BI architecture
The three main categories:1. Operational Insight & Intelligence
2. Strategic Insight & Intelligence
3. Special Situations Insight & Intelligence
![Page 7: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/7.jpg)
InternationalTreasury
OLTP Systems• For Automating Business Transactions• Enable bookkeeping • Functional silos – retail, corporate, treasury
etc – they are production data bases• No value for historical data• Data is diverse and complex• User access is complex• User access slows business operations• OLTP systems are not designed for data
analysis
![Page 8: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/8.jpg)
InternationalTreasury
DSS
• Not just individual functional information but cross functional
• Integration – relationships between data elements
• Adhoc Queries• Analytical Queries • Multi-Dimensional Views
![Page 9: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/9.jpg)
InternationalTreasury
Data mart (vs. Data Warehouse)
• Functional vs.. Centralised • LOB vs. Enterprise • Restrictive process orientation • No historical data• Departmental• Specialised• Local • Integrate independent data marts in to EDWH • Create independent data marts from EDWH
![Page 10: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/10.jpg)
InternationalTreasury
Data Marts
• No framework• Data quality• Incompleteness of data• High costs of development and running –
Manpower, Skills, Systems, Complexity• Redundant efforts
![Page 11: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/11.jpg)
InternationalTreasury
What is a Data Warehouse?What is a Data Warehouse?
• The classic 1993 definition by Bill Inmon, “father of data warehousing”– A data warehouse is a:
• subject oriented• integrated• non-volatile• time variant
– collection of data in support of management’s decisions.
![Page 12: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/12.jpg)
InternationalTreasury
• Mostly for performance reasons, a data warehouse is:– held in a separate database from the
operational database, – and usually on a separate machine.
• Perhaps more important reasons are:– navigation, ease of use, relationship with
business areas
Data Warehouse ImplementationData Warehouse Implementation
3 of 3
![Page 13: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/13.jpg)
InternationalTreasury
OLTP Vs DWH• Archived and summarized as opposed to current• Organized by subject as opposed to application• Static until refreshed as opposed to dynamic• Simplified for analysis as opposed to complex for
computation• Accessed and manipulated as opposed to updated• Unstructured for analysis as opposed to structured
for repetitive processing• Data warehouse provides on-line analytical
processing, (OLAP), as opposed to on-line transaction processing, (OLTP).
• One record at a time vs. massive records access• Sub-second response times vs. hours
![Page 14: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/14.jpg)
InternationalTreasury
DWH• Complex ad-hoc queries are submitted and
executed rapidly because the data is stored in a consistent format
• Queries don’t interfere with ongoing operations because the system is dedicated to serving as a data warehouse
• Data can be organized by useful categories such as customer or product because the data is consolidated from multiple sources.
• The data warehouse is a single source of consolidated data, which provides an enterprise-wide view of the business.
![Page 15: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/15.jpg)
InternationalTreasury
EDWH
• Never a big bang approach• Have a framework• Start small• Iterate• Architecture driven by business and not
technology• EDWH is part of corporate strategy
![Page 16: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/16.jpg)
InternationalTreasury
EDWH Model• Users (Transaction Generators )• Txn data• ODS• ETL• EDWH• Populate Departmental warehouses • Corporate / Departmental Business users
![Page 17: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/17.jpg)
InternationalTreasury
The GDW architecture
App1
CRM
App3
App4
App4
Local DataMart
LocalDataMart
LocalDataMart
Batch ETL
Batch ETL
Batch ETL
Batch ETL
Real-Time
Messaging
Batch ETL
Batch ETL
Batch ETL
HRISHR
DataMart
Batch ETL Batch ETL
GlobalData
Warehouse
App2
Batch ETL
![Page 18: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/18.jpg)
InternationalTreasury
Data Warehouse Architecture
![Page 19: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/19.jpg)
InternationalTreasury
Business Intelligence is a Corporate Activity
Performance Management, Financial AnalysisBalanced Scorecard
Data Mining, Cause & Effect Analysis
Business Modeling & Simulation
Planning, Budgeting & Trend Analysis
OLAP & Spreadsheet Analysis
Operational & Administrative Transactions
Forecasting
Statistical Analysis
Performance Indication
Strategic Analysis
Information Extraction
Data Analysis
Data Collection
Query & Reporting Tools
![Page 20: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/20.jpg)
InternationalTreasury
BI PROCESS
• Transactional systems• Processes • Integration• Consolidation• Intelligence • Strategy modification• Implementation
![Page 21: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/21.jpg)
InternationalTreasury
Financial Services – Some Statistics
• 20 % of the customers represent 160% of the profits• A 5% increase in retention can produce an increase in
profits between 25-80%• A customer with one relationship with a bank has a 35%
chance of leaving• A customer with 3 or more relationships has an 85%
change of being with the Financial Institution for 3-5 years.• A shift in the customer mix from 30%(A’s) – 30%(B’s) –
40%(C’s) to 40-30-30 can often raise overall profits by as much as 60%
![Page 22: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/22.jpg)
InternationalTreasury
Banking Business Focus
• Account Centric• Product Centric• Market Centric• Customer Centric
![Page 23: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/23.jpg)
InternationalTreasury
Bankers Needs• Customer Profiling • Channel Profiling• Profitability Analysis• Performance reporting• Transfer pricing• ALM• Risk Management ( Credit Risk, Market Risk,
Operational Risk)• Wealth / Portfolio Management• HRIS
![Page 24: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/24.jpg)
InternationalTreasury
Financial Fraud
• Money laundering• Syndicated frauds – credit card cloning• Internal fraud – staff fraud• Merchant fraud• Challenges in identifying fraudulent
transaction before it is too late and preventing them
![Page 25: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/25.jpg)
InternationalTreasury
Indian Banking Scenario• Banks are struggling to stop the steady decline in
their margins• In India, many banks are still dependent on “interest
income”, but the growth and profits is in “fee-based income”
• Most banks have large customer bases, which could be gold mines, but have not been adequately tapped
• To create Customer value, banks need to think along 3 dimensions– Relationship numbers– Relationship profitability– Relationship duration
![Page 26: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/26.jpg)
InternationalTreasury
Banking Scenario
• Branch Systems• Size and Spread• Silo Applications• No / Limited Networking• No Centralised Repository• No Centralised Controls
![Page 27: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/27.jpg)
InternationalTreasury
Bank - Managing Relationships
• Manage all aspects of the customer relationships• Have 360 degree view of customer• Identify and retain the most profitable customers• Identify cross sell opportunities• Attract the right customers from the competition• Measure product and business profitability• Track customer preferences• Design new products
![Page 28: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/28.jpg)
InternationalTreasury
Banking DWH Uses• Enterprise-wide risk management and compliance
reporting for the bank group and both corporate and retail business divisions– Loan Analysis– Credit Risk– Market Risk– Operational Risk
• HR Management • Channel Management• Segmentation• Product / LOB Performance Measurement• MIS & Reporting
![Page 29: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/29.jpg)
InternationalTreasury
EDWH• Data warehousing provides data quickly and
in a format that greatly enhances the decision making process.
• The data warehouse allows financial institutions to exploit the potential of information previously locked in legacy systems ( which is currently inaccessible to the business user) or across current silo systems
• Massiveness of Data that needs mining• Complex views / relationships
![Page 30: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/30.jpg)
InternationalTreasury
Data
• In-house G/L• Silo Applications• External Data Sources• DWH Update
– Batch– Real-time
![Page 31: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/31.jpg)
InternationalTreasury
Metadata Major Categories– Technical metadata
• Information about data sources• Transformation descriptions• Warehouse object and data structure definitions for data targets• The rules used to perform data cleanup and data enhancement• Data mapping operations when capturing data from source systems and
applying it to the target warehouse database• Access authorization, backup history,data access, etc.
– Business metadata• Subject areas and information object type, including queries, reports,
images, video, and/or audio clips.• Internet home pages• Other information to support all data warehousing components.• Data warehouse operational information, e.g., data history, ownership,
extract audit trail, usage data
![Page 32: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/32.jpg)
InternationalTreasury
Data Mining• The process of extracting valid, previously
unknown, comprehensible, and actionable information from large databases and using it to make crucial business decisions
• Involves analysis of data and use of software techniques for finding hidden and unexpected patterns and relationships in sets of data.
• Patterns and relationships are identified by examining the underlying rules and features in the data.
• Most accurate and reliable results require large volumes of data
![Page 33: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/33.jpg)
InternationalTreasury
Data Mining• Retail / Marketing
– Identifying buying patterns of customers.– Predicting response to mailing campaigns.
• Banking – Detecting patterns of CC fraud– Identifying loyal customers
• Insurance– Claims analysis.– Predicting which customers will buy new policies.
![Page 34: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/34.jpg)
InternationalTreasury
Data Mining Operations
• Predictive modelling• Database segmentation• Link analysis• Deviation detection
![Page 35: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/35.jpg)
InternationalTreasury
Data
• How to extract the data• Cleaning the data• Validating the data• Aggregating the data• Integrating the data• Transforming the data
![Page 36: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/36.jpg)
InternationalTreasury
Data
• Incorrect data• Missing data• Lack of sufficient data detail• Insufficient Security and Privacy
![Page 37: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/37.jpg)
InternationalTreasury
The two BI “mega-problems”• Misalignment between BI/DW solution and
organizational structure/culture– Primarily, but not exclusively, in enterprise-wide
initiatives– Highly centralized? Totally decentralized? Loosely
coupled? It all depends!
• Focusing on the wrong architecture first– Both are important, but the BI architecture should come
first and guide the DW architecture…and be appropriately aligned with the structure/culture
![Page 38: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/38.jpg)
InternationalTreasury
Data Quality• Data Quality Principles - accurate and consistent data between
trading partners in the consumer products industry • Business not IT owns the data• Managing Data as a Product• Treating Data as an Asset• Optimise Data not Process• Data Quality Best Practices• Keep Data Structures Simple• Use Standard Names and Definitions• Developing Meaningful Definitions• Classification and Report Structures are treated separately• Labels used must be consistent and unique• The Data-Information-Knowledge Continuum
![Page 39: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/39.jpg)
InternationalTreasury
Data Quality• Improve data quality management in your organisation• Address challenges faced during implementation of
data quality systems• Look at the available technology options and choosing
the right device• Create a strategic vision for data quality management• Leverage from best case examples of effective data
quality management strategies• Streamline your data quality process with the help of
data management• Seek the consultancy of hands-on experience from data
quality management experts
![Page 40: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/40.jpg)
InternationalTreasury
Data Quality• Ensure your organisation is geared for a data quality
management strategy• Optimise your core data entity through tools and quality
systems• Improve ROI by aligning data quality systems with
business strategies• Implement value-added data quality processes to your
organisation• Learn to effectively master data quality management to
increase your organisation's ROI on EDWH
![Page 41: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/41.jpg)
InternationalTreasury
BI Strategies
• Comprehensive BI strategy and architecture• Short-duration BI “mini-strategy” offerings• Executive education briefings• Staff training in BI best practices• Assessments of existing BI environments and
implementations• BI project risk assessment, quality assurance
(QA) planning, and project plan/methodology review
![Page 42: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/42.jpg)
InternationalTreasury
IT Issues
• Technology Islands• Incomplete information is more
damaging than no information• Has the “modern era” (since 1990-
1991) of BI and DW been successful or not?
![Page 43: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/43.jpg)
InternationalTreasury
Sources
• Legacy data access• Multiplicity of sources• Storage needs• Integration issues• Multi-dimensional analysis
Technology Gap between where data is stored and where it is used
![Page 44: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/44.jpg)
InternationalTreasury
Implementation Issues
• Unclear Business Requirements• Multiple Data Sources• ETL• Modeling• Integration• View
![Page 45: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/45.jpg)
InternationalTreasury
Problems of Data Warehousing• Underestimation of resources for data loading• Hidden problems with source systems• Required data not captured• Increased end-user demands• Data homogenization• High demand for resources• Data ownership• High maintenance• Data Refresh Cycle• Long duration projects• Complexity of integration
![Page 46: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/46.jpg)
InternationalTreasury
BI Issues• Inflexible reports and queries• Tools that are too difficult to use• Data not available timely enough• Too difficult to add new data sources• Lack of user-oriented metadata• Lack of system-oriented metadata• Inability to intermix ERP, CRM, and other data• “Deskbound” analytics
![Page 47: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/47.jpg)
InternationalTreasury
Critical Steps to Success
• Business Sponsorship• User Commitment• Data Warehouse Experience• Data Quality Problems• Inability of Users to Easily Analyze the
Data
![Page 48: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/48.jpg)
InternationalTreasury
BI• Sales• Operations• Finance• HR• Customer service• Supplier relations• Agility – not in SW but in people• Helps in refocus organisations scarce
resources
![Page 49: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/49.jpg)
InternationalTreasury
Technology• MS DOS etc ushered in an era of
affordable computing• RDBMS, BI , WFL etc came down to
desktops from glass cages• Ubiquitous • Empowerment of customers and
employees• Self Service• Software as Utility
![Page 50: DWH & BI in Banking @ ET 2 Nov 2004 Chandrasekhar](https://reader037.vdocument.in/reader037/viewer/2022102817/563db808550346aa9a8ff056/html5/thumbnails/50.jpg)
Treasury
International
Thank You