1 categories of data operational and very short-term decision making data current, short-term...
TRANSCRIPT
![Page 1: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/1.jpg)
1
Categories of data
Operational and very short-term decision making data
Current, short-term decision making, related to financial transactions, detailed data are stored, not structured for decision making.
Historical and long-term decision making dataSaved for a pre-determined period of time, usually related to long-term decision making, structured for decision making.
Contains data that will support decisions of strategic importance.
Referred to as a “data warehouse”.
Archival dataSaved for a pre-determined period of time, used to track transactions for audit, not structured for decision making.
![Page 2: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/2.jpg)
2
Webflix data storage requirements
Operational needs.
What are examples of questions management needs to be able to answer to handle daily operations effectively?
Decision support needs.
What are examples of questions management needs to be able to answer to manage the organization effectively on a short and long-term basis?
Governmental, legal or auditing needs.
What types of questions might be relevant for this type of organization?
![Page 3: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/3.jpg)
3
Operational data
Includes:Master data (also called reference data): Customer, employee, video, distribution center, critic, keyword.
Transaction data: Queue, Copy, Customer Contract.
Must store both master and transaction data.
Must store changes to both master and transaction data.
![Page 4: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/4.jpg)
4
Problems with operational data
May not be integrated.
May not be of good quality:
Incomplete.
Not accurate.
Inconsistent.
The meaning of the data is not fully defined and/or understood by all stakeholders.
![Page 5: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/5.jpg)
5
Archival data
Examples of archived data:Emergency dispatch calls.
Credit card transactions.
Accounts payable transactions.
Tax-related data.
Does not usually have to be accessed quickly.
Must have procedures for extracting, transforming and loading (ETL) data as necessary.
Archive database design is usually a copy of the transaction database design.
![Page 6: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/6.jpg)
6
Topics about Data Warehouses
What is a data warehouse?
How does a data warehouse differ from a transaction processing database?
What are the characteristics of a data warehouse?
What are the components of a data warehousing system?
How is a data warehouse created?
How is a data warehouse accessed?
![Page 7: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/7.jpg)
Compare and Contrast TPS and DSS
Issue TPS/MIS DSS
Definition Systems to support day-to-day operations.
Systems to support ad-hoc decision making.
Users clerks, data entry, low-level supervisors.
managers, analysts, support staff, researchers.
Design goal Performance. Flexibility, ease of use, ease of access.
Transaction Type
Updates. Queries.
Query Activity
low; few joins. high; many joins.
![Page 8: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/8.jpg)
We use data to answer management questions
TPS Questions
How many customers currently have “Skyfall” in the queue?
How many copies of “Skyfall” are in inventory in Sacramento?
How many customers do we have in Nevada City?
When is “Cloud Atlas” going to be released?
Data Warehouse Questions
How long does a customer usually keep a video?
Which customers return videos within 2 days of receiving them?
Which city has the most customers who return videos within 2 days of receiving them?
What is the most popular genre for customers in Reno?
8
![Page 9: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/9.jpg)
Operational vs. Data Warehouse databases
Issue Operational database
Data Warehouse
Content Internal data, process-oriented.
Internal and external data.
Subject-oriented.
Data currency
Real time.
Current.
Volatile.
Batch.
Historical.
Non-volatile.
Summary level
Details of transactions; no (or very little) derived data.
Summarized; many aggregation levels.
Volume Megabytes to gigabytes.
Gigabytes to terabytes.
Design Normalized to prevent anomalies.
Denormalized to enhance query performance.
![Page 10: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/10.jpg)
So, can one database support both transaction processing and decision
support applications?Yes?? No??
![Page 11: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/11.jpg)
11
![Page 12: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/12.jpg)
12
Historical Data
Historical Data
![Page 13: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/13.jpg)
A Business Intelligence “System”
A business intelligence system encompasses all processes, hardware and software necessary to extract data, transform it, integrate it, store it, and provide information. The information is then made effective and accessible to users to support decision making.
Sounds like just another information system...
13
So what makes it different?
![Page 14: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/14.jpg)
14
Big Data!
![Page 15: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/15.jpg)
15
DataSources
ERP
Legacy
POS
OtherOLTP/wEB
External data
Select
Transform
Extract
Integrate
Load
ETL Process
EnterpriseData warehouse
Metadata
Replication
A P
I
/ M
iddl
ewar
e Data/text mining
Custom builtapplications
OLAP,Dashboard,Web
RoutineBusinessReporting
Applications(Visualization)
Data mart(Engineering)
Data mart(Marketing)
Data mart(Finance)
Data mart(...)
Access
No data marts option
![Page 16: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/16.jpg)
16
Components of a business intelligence/data warehousing system
Data store.
Extraction/transformation/loading processes.
Analysis tools – both end-user and IT professional.
Visualization tools – primarily end-user.
![Page 17: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/17.jpg)
What is a data warehouse (data store)?
A data warehouse is a database designed to support a decision support system.
A data warehouse is:
Integrated: It is a centralized, consolidated database integrating data from an entire organization.
Subject-oriented: Data warehouse data are organized around key subjects. The data are usually arranged by topic, such as customers, products, suppliers, etc.
Time-variant: Data in the warehouse contain a time dimension so that they may be used as a historical aggregation.
Non-volatile: Once data enter, they seldom leave. Data are appended rather than overwritten. Data are updated in batches.
![Page 18: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/18.jpg)
18
Issues in creating a data warehouse
How to get accurate and complete data?
How to consolidate data?
Differing data meanings.
Differing storage mechanisms.
Differing data formats.
![Page 19: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/19.jpg)
CustomerTransactionDatabase
ProductTransactionDatabase
OrderTransactionDatabase
DataScrubbing
DataScrubbing
DataScrubbing
DataExtraction
DataExtraction
DataExtraction
DataIntegration
Sales DataWarehouse
Creating aData
Warehouse
![Page 20: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/20.jpg)
Data mart extraction data warehouse
20
Operationaldatabase
Operationaldatabase
External data source
User departments
Data mart
Data mart
Data mart
Extract, Transform and Load Processes
![Page 21: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/21.jpg)
Two-tier data warehouse architecture
Data warehouse
Operationaldatabase
Operationaldatabase
Externaldata source
EDM
Summarizeddata
Transformationprocess
Data warehouseserver
User departments
![Page 22: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/22.jpg)
Three-tier data warehouse architecture
Data warehouse
Operationaldatabase
Operationaldatabase
Externaldata source
EDM
Summarizeddata
Transformationprocess
Data warehouseserver
Userdepartments
Data mart
Data mart
Data mart tier
Extractionprocess
![Page 23: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/23.jpg)
23
Issues in designing a data warehouse
Must have a predefined subject focus.
Has the potential to be very large – must define the “grain” or granularity level of storage.
Will always have a dimension of time.
May contain derived data.
May be a summary of data, rather than each detailed transaction.
Does not always adhere to standard normalization rules.
![Page 24: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/24.jpg)
Analysis tools
Standard old queries
Online Analytical Processing
Data Mining
24
![Page 25: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/25.jpg)
25
Online analytical processing
Provides multi-dimensional data analysis techniques.
Works primarily with data aggregation.
Provides advanced statistical analysis.
Supports access to very large databases.
Provides enhanced query optimization algorithms.
Lots of acronyms: OLAP, ROLAP, MOLAP, HOLAP.
Can be add-ons to existing products, example is Excel. Can have their own user interfaces.
![Page 26: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/26.jpg)
OLAP vs. Data Mining questionsOLAP Data Mining
Which customers spent the most with us in the past year?
Which types of customers are likely to spend the most with us in the coming year?
How much did the bank lose from loan defaulters within the past two years?
What are the characteristics of the customers most likely to default on their loans before the year is over?
What were the highest selling fashion items in our London stores?
What additional products are most likely to be sold to customers who buy shorts?
Which store/ location made the highest sales in the past year?
In which area whould we open a new store next year?
![Page 27: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/27.jpg)
27
Data mining
Data mining tools:
analyze the data;
uncover patterns hidden in the data;
form computer models based on the findings; and
use the models to predict business behavior.
Proactive tools.
Based on artificial intelligence software such as decision trees, neural networks, fuzzy logic systems, inductive nets and classification networking.
![Page 28: 1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed](https://reader036.vdocument.in/reader036/viewer/2022081603/56649efe5503460f94c1330c/html5/thumbnails/28.jpg)
28
Visualization tools
Graphical.
Spreadsheet format - usually Excel look-and-feel.
Beyond the spreadsheet using discovery tools. Example: http://www.gapminder.org/
Dashboard. Examples: http://www.dundas.com/dashboard/online-examples/
Web-based.