impact of the convergence of three worlds: high-performance computing, databases, and analytics

Impact of the convergence of three worlds:

High-Performance Computing, Databases, and Analytics

Stefan ManegoldStefan.Manegold@cwi.nl

http://www.cwi.nl/~manegold/

2

HPC (!)HPC (!)

3

DatabaseDatabase

4

Analytics (?)Analytics (?)

5

ClusterCluster

http://www.scilens.org/

http://www.scilens.org/

Our new Playground & Challenge

128 Pebbles: dual-core AMD bobcat, 8GB RAM, 10 TB HDD

128 bricks: 4-core HT i7, 16GB RAM, 2 TB HDD, 1 TB SSD

256 cores, 1 TB RAM, 1.3 PB HDD

1024 cores, 2 TB RAM, 256 TB HDD, 128 TB SSD

16 rocks: 2x 8-core HT Xeon, 256 GB RAM, 4+ GPUs, 8 TB HDD, 2 TB SSD; 48 TB NAS

512 cores, 4 TB RAM, 4+ GPUs, 128 TB HDD, 32 TB SSD; 48 TB NAS

1+ diamonds: 64+ cores, 4+ TB RAM, X GPUs, Y TB SSD, Z ...

7

The Memory Wall

8

The Memory Wall

Trip to memory = 1000s of instructions!

Larger, faster, cheaper and more responsive memory sub-systems

+

memory-optimized DBMS

=

...

In the multi-core age, how do larger, faster, cheaper and more responsive memory sub-systems affect data management?

In the age of larger, faster, cheaper and more responsive memory sub-systems, how do multi- (or even many-) core

systems affect data management?

In the multi-core age, how do larger, faster, cheaper and more responsive memory sub-systems affect data management?

Bandwidth bottleneck: How to feed all these cores?How to exploit excess CPU cycles usefully?

Exploit TurboBoost:Instructions per data dependent MPL

Energy consumption:WATTs per GB of DRAM vs. Flash vs. Disk

System maintenance and opertation:How to bootstrap a multi-TB (PB?) RAM system?Hot-swapable DRAM?

...

Shopping List Excerpt