© 1999 forwiss general research report implementation and optimization issues of the rolap algebra...
TRANSCRIPT
© 1999 FORWISS
FORWISSFORWISS
General Research Report
Implementation and Optimization Issues of the
ROLAP Algebra
F. Ramsak, M.S. (UIUC)
Dr. V. Markl
Prof. R. Bayer, Ph.D.
© 1999 FORWISS
FORWISSFORWISS
Contents
Motivation
ROLAP Algebra Recap
Optimization Issues
– Handling of Restrictions
– Aggregation Networks
Future Work & Summary
© 1999 FORWISS
FORWISSFORWISS
Example DW Model
SalesCost
Quantity
Year
Month
TIME
All Time
Quarter
Year
Month
TIME
All Time
Quarter
Region
Nation
Trade T
ypeBusin
ess T
ype
CUSTOMERAll C
ustom
erRegion
Nation
Trade T
ypeBusin
ess T
ype
CUSTOMERAll C
ustom
er
PRODUCT
Sector
Category
Product Group
Container
All ProductsPRODUCT
Sector
Category
Product Group
Container
All Products
© 1999 FORWISS
FORWISSFORWISS
User‘s View of a ReportSum - Sales Year Quarter
1998 1998 Total TotalRegion Nation 1 2 3 4Asia China
Japan
Asia TotalEurope France
GermanySpain
Europe Total
Total
1
1
2
2
3 9
7
5 8
4 7
5 8
6
4
Grouping combinations used to fill pivot table:
(1){Y,Q,R,N} (2){Y,Q,R} (3) {Y,Q}
(4) {Y,R,N} (5){Y,R} (6){Y}
(7) {R,N} (8) {R} (9){} = ALL
© 1999 FORWISS
FORWISSFORWISS
Gg
gFhALLRPADFGRPOT
Gh
))),(((),,(
POT: Pivot Organized Tuples
We may also write
for POT(R,G,F). G,F
(R)
© 1999 FORWISS
FORWISSFORWISS
POT-Example
POT(R,{{A},{A,B}},{sum(D)})
yields the table:A B Sum( D)a1 ALL ***… ***
n
ALL ***
a1 b1 ***… … ***an ***
… … ***a1 bm ***… … ***an bm ***
a
A BA ,
)( Dsum )( Dsum
R
)(, ALLpad BA )(, ALLpad BA
A B s u m ( D )A s u m ( D )
A B s u m ( D ) A B s u m ( D )
A B s u m ( D )
© 1999 FORWISS
FORWISSFORWISS
POT Extension: Group Filtering Filtering of generated groups
(like with the HAVING clause in SQL)
with H containing a predicate H[g] for each grouping g in G
Gg
gFgHhALLRPAD
FHGRPOT
Gh
)))),((((
),,,(
][
© 1999 FORWISS
FORWISSFORWISS
Group Filtering Example
Report Years, Product-Group sales totals and sales/year for PGs with less than 10 Mio sales
)})({
}},10)({{},{{},
}},,{},{},{{
,(
Salessum
MioSalessum
PGYPGY
FactPOT
© 1999 FORWISS
FORWISSFORWISS
Straight Forward SQL Generation
POT(R,{{A},{A,B}},{sum(D)}) maps directly to:SELECT A, ‘ALL’, sum(D)
FROM R
GROUP BY A
UNION
SELECT A, B, sum(D)
FROM R
GROUP BY A,B
Disadvantages:– Efficient execution depends on optimizer of underlying DBMS
– no UB-Tree support on SQL interface guaranteed
© 1999 FORWISS
FORWISSFORWISS
Handling Restrictions
Semantic of ALL value
Pushing Restrictions Down
– Pushing Through POT: Restrictions on all groups
– Pushing down inside POT:
Restrictions on individual groups may be pushed down (i.e.,
before grouping) if they do not contain constraints on the
aggregation results
),),(()),,(( FGRPOTFGRPOT
?))}},{},{{,((1998 FPGYearRPOTYear
© 1999 FORWISS
FORWISSFORWISS
)( Salessum
)(, ALLpad MonthYear
Year MonthYear,
)( Salessum
FACT
)(, ALLpad MonthYear
YearYear )19981997(
© 1999 FORWISS
FORWISSFORWISS
)( Salessum
)(, ALLpad MonthYear
Year MonthYear,
)( Salessum
FACT
)(, ALLpad MonthYear
YearYear )19981997(
© 1999 FORWISS
FORWISSFORWISS
Aggregation Networks Efficient generation of multiple groups
– Usage of previous generated (more finer) groups instead of fact
table as starting point
– Only one access to the fact table for multiple groups
Problems: Size of aggregation nets
– Hierarchy semantic reduces aggregation nets significantly
UB-Tree & Tetris techniques have high potential to optimize aggregation nets
– Grouping requires sorting
– Sorted writing of large temporary results saves additional
processing time
© 1999 FORWISS
FORWISSFORWISS
Example of anAggregation Network
(Year, Month, Productgroup)
(Year) (Productgroup)(Month)
(Year, Month) (Year, Productgroup) (Month, Productgroup)
( )
© 1999 FORWISS
FORWISSFORWISS
AggregationNet withHierarchies
(Year, Month, Productgroup)
(Year) (Productgroup)
(Year, Month) (Year, Productgroup)
( )
Tetris: sort according to Y
Sort according to PG(or sorted writing+Scan)
© 1999 FORWISS
FORWISSFORWISS
POT and AggregationNets
)( Salessum
)(, ALLpad MonthYear
Year
MonthYear ,
)( Salessum
FACT
)(, ALLpad MonthYear
CTVo d u ctg ro u pGerma n yCo u n tryYea rYea r P r)1 9 9 81 9 9 7(
© 1999 FORWISS
FORWISSFORWISS
Optimization Issues ofAggregationNets
Find minimal spanning tree for the specified groupings
– Vertices: groupings
– Edge weights: cost of computing new group#
Cost factors:
– Group size
– Required sorting
– ...
© 1999 FORWISS
FORWISSFORWISS
OptimizationIssuesofAggregationNets
(Year, Month, Productgroup)
(Year) (Productgroup)
(Year, Month) (Year, Productgroup)
( )
C1C2
C5
C4 C3
C6 C7
© 1999 FORWISS
FORWISSFORWISS
Summary and Future Work Aggregation networks have a very potential to
speed up POT operations
Standard grouping/aggregation algorithms may benefit significantly from UB-Tree/Tetris techniques
Upon availability of resources: Implementation of basic ROLAP algebra processing as part of a master thesis