big data and osfa€¦ · independent database engines..." number of m. stonebraker and u....
TRANSCRIPT
![Page 1: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/1.jpg)
BIG DATA AND OSFAOmer Trajman, Cloudera
![Page 2: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/2.jpg)
END OF OSFA
0
7.5
15
22.5
30
2006 2007 2008 2010 2011 2012
CompaniesProducts
"...the commercial world will fracture into a collection of independent database engines..."
Number of
M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings of ICDE, 2005.
Gartner, Inc., Magic Quadrant for Data Warehouse Database Management Systems, 2006-2012
![Page 3: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/3.jpg)
RESULTING IN...
EDW
Data Marts
ETL Grid
Enterprise Storage
OLTP Logs
Sensors
CEP and ESB
![Page 4: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/4.jpg)
DATA ON OSFA
Kept for limited periods of time
Used for limited applications
Difficult to integrate with other data
Archived (deleted)
Photo credit/Flickr/bcostin
![Page 5: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/5.jpg)
BIG DATA IS UNFILTERED
Different processing engines for different stages.
Lots of data makes it expensive to move.
Dynamic businesses requires flexibility.
Photo credit/Flickr/dervish
![Page 6: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/6.jpg)
BIG DATA MOVES LESS
HDFS
Unified Scale Out Storage and Compute
EDW
CEP and ESB
OLTP Logs
Sensors
Flume Sqoop
Sqoop Flume
MR
![Page 7: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/7.jpg)
MR = ETL?
HDFS
MR
![Page 8: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/8.jpg)
IT'S NOT (JUST) ABOUT MR
HDFS
Avro
Crun
ch
Flum
e
Gira
ph
Ham
a
HBa
se
Hive
Mah
out
MaR
educ
e
Pig
Sqoo
p
![Page 9: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/9.jpg)
OR (JUST) ABOUT OSSIn
form
atica
Micr
oStra
tegy
Table
au
RainS
tor
Had
apt
Dat
amee
r
Karm
asph
ere
NG
data
Plat
fora
Wib
iDat
a
HDFS
Avro
Crun
ch
Flum
e
Gira
ph
Ham
a
HBa
se
Hive
Mah
out
MaR
educ
e
Pig
Sqoo
p
![Page 10: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/10.jpg)
OSFA AND BIG DATA
Flume Sqoop
HDFS
Pig HiveHBase
MicroStrategyInformatica Tableau
![Page 11: BIG DATA AND OSFA€¦ · independent database engines..." Number of M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings](https://reader034.vdocument.in/reader034/viewer/2022050201/5f54a5e969af3a28d9699a8e/html5/thumbnails/11.jpg)
GOOGLE FIGHT?