data warehousing with olap - tu braunschweig
TRANSCRIPT
1/28/2011
1
Data Warehousing
& Data Mining Wolf-Tilo Balke
Silviu Homoceanu
Institut für Informationssysteme
Technische Universität Braunschweig
http://www.ifis.cs.tu-bs.de
• Last week:
– Clustering
• Flat: K-means
• Hierarchical: Agglomerative, Divisive
– Clustering high-dimensional data
• CLIQUE
• This week..
Data Warehousing & OLAP – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 2
Summary
12. Decision Support Systems (DSS)
DSS Applications:
12.1 Marketing Models
12.2 Supply Chain Management
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 3
12. Decision Support Systems
• Decision-making is the process of making choices. It includes:
– Assessing the problem
– Collecting and verifying information
– Identifying alternatives
– Anticipating consequences of decisions
– Making the choice using sound and logical judgment based on available information
– Informing others of decision and rationale
– Evaluating decisions
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 4
12.0 DSS - Introduction
• Decision problem
• What kind of decisions are there?
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 5
12.0 Decisions
options
(alternatives)
goals
• FIND the option that best satisfies the goals
• RANK options according to the goals
• ANALYSE, JUSTIFY, EXPLAIN, …, the decision
• Types of decisions
– Easy (routine, everyday)
vs. difficult (complex)
– One-time vs. recurring
– One-stage vs. sequential
– Single objective vs. multiple objectives
– Operational, tactical, strategic
– …
• DSS address complex decisions
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 6
12.0 Decisions
1/28/2011
2
• Characteristics of complex decisions
– Novelty
• There was no prior similar decision
– Unclearness
• Incomplete knowledge about the problem
– Uncertainty
• Outside events that cannot be controlled
– Multiple objectives (possibly conflicting)
• Maximize economic benefits vs. minimize environmental costs
– Important consequences of the decision
– Limited resources
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 7
12.0 Complex Decisions
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 8
12.0 Decision-Making
• Decision making can be difficult for people.
Can we help decision makers make better
decisions?
– Decision Support: Provides methods and tools for
supporting people in making complex decisions.
How?
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 9
12.0 Decision Support
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 10
12.0 Decision Support
• Decision support systems (DSS)…
– are interactive, computer-based information systems
– developed for improving the decision-making
process
• Characteristics
– DSS incorporate both data and models
– They support rather the replace managerial
judgment
– Their objective is to improve the quality and
effectiveness rather then efficiency of decisions
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 11
12.0 DSS - Introduction
• Types of DSS
– Data-driven, emphasizes access to and manipulation
of data e.g., time-series
– Document-driven, manages, retrieves and
manipulates unstructured information stored in
electronic formats
– Knowledge-driven, provides problem solving
expertise stored as rules or procedures
– Model-driven, make use of statistical or financial
models and simulations
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 12
12.0 DSS - Introduction
1/28/2011
3
• Technologies DSS rely on
– Data mining
– Data warehousing and OLAP
– Traditional approaches
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 13
12.0 DSS - Introduction
• Data Mining
– Association rule mining
– Sequence patterns and time series
– Regression analysis
– Classification
– Clustering
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 14
12.0 DSS - Introduction
• Data Warehousing
– As support for OLAP
– Online Analytical Processing (OLAP)
• Traditional approaches
– Common mathematical modeling e.g., what-if-analysis
– Non-rigorous modeling
– Rule-based systems (RBS)
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 15
12.0 DSS - Introduction
• DSS capabilities should offer…
– support for problem-solving phases
• Gather intelligence, identify and design the options, make
the choice, implement it, monitor for feedback
– support for different decision frequencies
• Ad hoc DSS: decisions that come up once in every 5 years
(e.g., where should a company open a new distribution
center?)
• Institutional DSS: decisions that repeat (e.g., what should
the company invest in?)
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 16
12.0 DSS - Introduction
– support for different problem structures
• Highly structured problems: known facts and relationships
• Semi-structured problems: facts unknown or ambiguous, relations vague – E.g., which person to promote/hire for a position?
– support for various decision-making levels
• Operational level – Daily decisions
• Tactical level – Planning and control
• Strategic level – Long-term decisions
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 17
12.0 DSS Capabilities
• DSS architecture
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 18
12.0 DSS - Introduction
GUI
Analytical engine
Model Management
DW
Database
Management
1/28/2011
4
• The database management subsystem
– Purpose:
• Handles personal and unofficial data
so that users can
experiment with
alternative
solutions based
on their own
judgment -
- sandbox like
• Tracks data use
within the DSS
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 19
12.0 DSS Architecture
• The model management subsystem (MMS)
– Strategic models: non routine mergers, impact analysis, capital budgeting
– Tactical Models: sales promotion planning
– Operational Models: routine-day-to-day production scheduling, inventory control, quality control
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 20
12.0 DSS Architecture
• Major functions of the model manager
– Creates models either from scratch or from existing
models
– Allows users to manipulate models so that they can
conduct experiments and sensitive analysis e.g.,
what-if or goal seeking analysis
– Manages and maintains the model base e.g.,
• Store, access, run, update, link, catalog and query
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 21
12.0 MMS
• The analytical engine or knowledge based
subsystem
– Component of more advanced DSS
– Provides expertise in solving complex
unstructured and semi-structured problems
• Expertise is provided for example by an expert system
– Analytical engines are usually based on OLAP, data
mining, or expert systems
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 22
12.0 DSS Architecture
• The user interface
– Interactive, dialogue oriented
– Intuitive, graphical, symbolic
– Intelligent, context aware
– Customizable
• For the non-technical user, the user interface is
the system
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 23
12.0 DSS Architecture
• Applications of DSS
– Marketing Models
– Supply Chain Management
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 24
12.0 DSS - Introduction
1/28/2011
5
• Marketing decision processes are characterized
by a high level of complexity
– Simultaneous presence of multiple objectives
– Countless alternative actions resulting from the
combination of the major choice options
• Massive sales transactions data are available
making DSS a important tool for reaching
marketing intelligence
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 25
12.1 Marketing Models
• Marketing intelligence comprises 2 prominent
topics
– Relational marketing (RM)
– Sales force management (SFM)
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 26
12.1 Marketing Models
• Relational marketing as DSS application
– Designed to create, maintain, and enhance strong
relationships with customers
– Application of predictive models to support
relational marketing strategies
– E.g.:
• An insurance company wishes to select the most promising
market segment to target for a new type of policy
• A mobile phone provider wishes to identify those
customers with the highest probability of churning
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 27
12.1 Marketing Models
• Why is RM important?
– It costs five times as much to attract a new
customer as it does to keep a current one satisfied
• Advertising doesn’t come cheap at all!
– It is claimed that a 5% improvement in customer
retention can cause an increase in profitability of
between 25-85% depending on the industry
– Likewise, it is easier to deliver additional products and
services to an existing customer than to a first-time
buyer
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 28
12.1 Relational Marketing
• RM strategies revolve around the following
choices
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 29
12.1 Relational Marketing
Relational marketing
Sales processes
Distribution channels
Products Services
Segments
Prices Promotion channels
• How do we implement RM?
– E.g., using pattern recognition and machine
learning models on a company’s DW
• Derive different segmentations of the customers which are
then used to
design and
target marke-
ting actions
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 30
12.1 Relational Marketing
1/28/2011
6
• Cycle of RM analysis, phases: 1. Exploration of the data available for each customer
2. Identify market segments by using inductive learning models
3. Knowledge of customer profiles is then used to design marketing actions
4. The designed actions are translated into promotional campaigns which generate in turn new information for subsequent analyses
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 31
12.1 Relational Marketing
Collect information on
customers
Plan actions based on knowledge
Identify segments and needs
Perform optimized and targeted
actions
• General statistics show…
– The average business never hears from 96% of its
unhappy customers
• 91% never come back
• Dissatisfied customers may tell 9-10 people about their
experience
– Every positive experience is told to 4-5 people
– For every complaint received the average business in
fact has 26 customers with a similar concern
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 32
12.1 Customer Relations
– Of the customers who register a complaint, as many
as 70% will do business again with your organization, if
the complaint is resolved effectively
• This figure goes up to 95% if the complaint has been
resolved quickly
– 40% of complaints are the result from customer
mistakes or incorrect expectations
– A complaint that is handled efficiently is
actually better than no complaint at all
• Customers who complain and get satisfactory results are
8% more loyal than the others
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 33
12.1 Customer Relations
• Important part of RM is customer relationship
management (CRM)
• CRM
– The software tools which allow tracking and
analysis of each customer's purchases, preferences,
activities, tastes, likes, dislikes, and complaints
– Enterprise vendors/products
• Oracle/Siebel, Salesforce.com, Amdocs, Microsoft Dynamics
– Open source tools
• Opentaps, XRMS, SugarCRM
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 34
12.1 Customer Relations
• E.g., XRMS
– Contact
information
screen
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 35
12.1 Customer Relations
• Aspects of CRM systems
– Operational
– Collaborative
– Analytical
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 36
12.1 Customer Relations
1/28/2011
7
• Operational CRM
– Provides support to "front office" business processes, including sales, marketing and service
– Each interaction with a customer is generally added to a customer's contact history, and staff can retrieve information on customers from the database when necessary
– Main benefits is that customers can interact with different people in a company over time without having to describe the history of their interaction each time
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 37
12.1 CRM
• Collaborative CRM
– Covers aspects of a company's dealings with customers
that are handled by various departments within a company
• E.g., sales, technical support and marketing
– Staff members from different departments can share
information collected when interacting with customers
• E.g., feedback received by customer support agents can provide
other staff members with information on the services and
features requested by customers
– Goal of collaborative CRM is to use information collected
by all departments to improve the quality of services
provided by the company
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 38
12.1 CRM
• Analytical CRM
– Analyzes customer data for a variety of purposes:
• Design and execution of targeted marketing campaigns to optimize marketing effectiveness
• Design and execution of specific customer campaigns, including customer acquisition, cross-selling, up-selling, retention
• Analysis of customer behavior to aid product and service decision making e.g., pricing, new product development
• Management decisions, e.g. financial forecasting and customer profitability analysis
• Prediction of the probability of customer defection (churn)
• Acquisition? Cross-selling? Up-selling? Retention? Churn? Let’s see the lifetime of a customer
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 39
12.1 CRM
• Lifetime of a customer
– Lost proposal
• Before becoming a customer, an individual may receive
repeated proposals from the enterprise to win him/her
as a customer
– Acquisition
• The individual
becomes customer
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 40
12.1 Relational Marketing
– Cross/up-selling:
getting more business from current customers by
selling them additional or complementary
services
– Retention:
the continuous attempt to satisfy and keep current
customers actively involved in conducting business
• Highly satisfied customers are
– Less price sensitive
– More likely to talk favorably about you
– More likely to refer you to others
– Remain loyal for longer
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 41
12.1 Lifetime of a customer
– Churn (defection):
the percentage of customers who leave a business in
one year
– Interruption:
customers leaving a business. Possible reasons are that
they:
• Die
• Move away
• Leave for competitive reasons
• Are dissatisfied
• Quit because of an attitude of indifference
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 42
12.1 Lifetime of a customer
1/28/2011
8
– Dissatisfied?
• United Airlines Brakes Guitars
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 43
12.1 Lifetime of a customer
• Sales force management (SFM)
– Management of the whole set of people and roles
that are involved with different tasks and
responsibilities in the sales process
• Why SFM?
– It plays a critical role in:
• The profitability of an enterprise
• The implementation of the relational marketing strategy
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 44
12.1 Marketing Models
• Designing the sales network and planning agents activities involve complex decision making tasks
– Remaining activities are operational and sales force automation (SFA) software can be used
• SFM decision-making process can be grouped in 3 components each interacting with each other
– Design
– Planning
– Assessment
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 45
12.1 Sales force management
Sales force management
Assessment & control
Planning Design
• Design
– During start-up phase or during restructuring
– Includes 3 types of decisions
• Organizational structure
• Sizing
• Sales territories
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 46
12.1 Sales force management
– Organizational structure
• May take different forms corresponding to hierarchical
agglomerations of agents by group, products, brand or
geographical area
• In order to determine the organizational structure it is
necessary to analyze the complexity of customers products
and sales activities
– Decide whether and to what extent the agents should be
specialized
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 47
12.1 Design
– Sizing
• Decide the number of agents that should operate in the
selected structure
• Depends on several factors
– Number of customers, prospects, sales area coverage, estimated
time for each call, the agents traveling time, etc.
• Conflicting goals
– Reduction in costs due to decreasing sales force size is often
followed by a reduction in sales and revenues
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 48
12.1 Design
1/28/2011
9
– Sales territories
• Deciding on assigning territories to
agents
• Depends on factors such as
– The sales potential of the geographical areas
– The time required to travel from an area to
another
– The availability time of each agent
• Purpose of assignment is to determine a balanced situation
between sales opportunities in each territory to avoid
disparities among agents
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 49
12.1 Design
• Planning
– Decision-making process involving the assignment of
sales resources structured and sized during design
phase, to market entities
• E.g., sales resources
– Work time, budget
• E.g., market entities
– Products
– Market segments
– Distribution channels
– Customers
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 50
12.1 Sales force management
• Assessment
– Measure the effectiveness and efficiency of the
individuals in order to decide incentives and
remuneration schemes
• Define adequate evaluation criteria that take into
account the personal contribution of each agent having
removed effects due to area or product characteristics
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 51
12.1 Sales force management
• Sales Force Automation software
– Most CRM tools include SFA functionality
– Enterprise vendors/products
• Oracle/Siebel, SAP, Salesforce.com, Microsoft Dynamics,
Netsuite
– Open source tools
• XRMS, SugarCRM, Vtiger
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 52
12.1 Sales force management
• For producing industries, another field of business operation is of great importance:
– Supply chain management (SCM)
• A supply chain summarizes the logistic and production processes of a single enterprise as well as a network of companies
– Covers the flow of materials and products from the raw material down to the end product at the customer
• Contains acquisition of raw materials, production, transportation, storage,
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 53
12.2 Supply Chain Management
• Within a single company, internal supply chain
can be modeled and optimized
– Contain aspects of material purchase, production and
distribution
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 54
12.2 Supply Chain Management
Internal Supply Chain
Purchasing Production Distribution Suppliers Customers
1/28/2011
10
• However, global supply chains may form
complex networks of various material flows
and costs
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 55
12.2 Supply Chain Management
Main Plant
European Plant
Asian Plant Asian Suppliers
US Assembly
US Market
Asian Market
European Market
Recycling 1
Recycling 2
Asian Assembly
European Assembly
Kit Supplier
European Suppliers
US Suppliers
• Supply chain management is about managing
and optimizing those complex supply networks
– Eliminating excess inventory
– Improve on-time delivery performance
– Maximize the value of procurement
– Minimize transport costs
– Minimize storage costs
– Etc.
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 56
12.2 Supply Chain Management
• Steps of SCM
– Plan (strategic portion of SCM) • Strategy for managing all the resources that go towards meeting
customer demand
• Developing a set of metrics to monitor the performance of the supply chain so that it is efficient, costs less and delivers high quality
– Source • Choose suppliers to deliver the goods and services
• Develop a set of pricing, delivery and payment processes with suppliers
• Create metrics for monitoring and improving the relationships
• Put together processes for managing goods and services inventory, including receiving and verifying shipments, transferring them to the manufacturing facilities and authorizing supplier payments
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 57
12.2 Supply Chain Management
– Make (manufacturing step)
• Schedule the activities necessary for production, testing, packaging and preparation for delivery
• Most metric-intensive portion of the supply - measure quality levels, production output and worker productivity
– Deliver (the logistics part)
• Coordinate the receipt of orders, develop a network of warehouses, pick carriers to get products to customers and set up an invoicing system to receive payments
– Return
• Receive and manage defective or excess products
• Recycle used products
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 58
12.2 Supply Chain Management
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 59
12.2 Supply Chain Management
• For solving these tasks, SCM has to span across
most other enterprise management areas
– Thus, software
solutions are usually
very diverse and
customized
– Highly dependent
on data from
all branches of
business
Supply Chain Management
Supply Chain Strategy
Supply Chain Planning
Supply Chain Enterprise
Applications
Asset Management
Procurement
Product Lifecycle
Management
Logistics
• The traditional approach for optimizing supply chains was severely hampered by the unavailability of necessary data – Thus, usually only future demand was forecast as good
as possible, using statistical trending and “best fit” techniques – Trend Analysis and Trend Channels • Only high level data necessary (aggregated values from
OLAP cubes) – e.g. by weekly data by product category and customer group
• For dealing with unpredictability, security margins are added
• Based on the estimates, the supply chain could be optimized – Capacity Planning
– Bill of Material problems
– Network flow optimization
– etc.
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 60
12.2 Supply Chain Management
1/28/2011
11
• However, due to improved data warehouse
strategies, more dynamic and fine-grained
optimizations are possible
– Forecasting at much finer-granularity (DW allows
for drilling into the data)
• e.g. calculate the best inventory level per article for each
store
• So called model stock
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 61
12.2 Supply Chain Management
– Allows for new optimization techniques
• Simulation
• Stochastic models
– Include wider verity of metrics
• Stackability constraints
• Load and unloading rules
• Palletizing logic
• Warehouse efficiency
• “Shipping air” minimization
Relational Database Systems 1 – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 62
12.2 Supply Chain Management
• Decision Support Systems
– Decision Making Process
– Decision Support
– Typical Applications
• Marketing Models
– Relational Marketing
– Sales force management
• Supply Chain Management
Data Warehousing & OLAP – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 63
Summary
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 64
Next week
• We have seen the theory, how about the praxis?
– Next week: practical problems in DW
– Guest: Toma Buchinsky, CEO Adastra,
Germany.
– Adastra Corporation specialized in
DW-based solutions and
Business Intelligence services.
DW & DM – Wolf-Tilo Balke – Institut für Informationssysteme – TU Braunschweig 65
The End
• I hope you enjoyed the lecture and learned at
least some interesting stuff…
– Next semester’s master courses:
Multimedia Databases, Information Retrieval,
Relational Databases 2
Knowledge-Based Systems and Deductive Databases – Wolf-Tilo Balke – IfIS – TU Braunschweig 66
12 Thank You!