présentation ibm db2 blu - fabrizio danusso
Post on 27-Dec-2014
431 Views
Preview:
DESCRIPTION
TRANSCRIPT
© 2013 IBM Corporation
IBM DB2 with BLU Acceleration Bois-Colombes - User Group Infosphere
Fabrizio DANUSSO | Client Technical Professional
16 octobre 2013
© 2013 IBM Corporation 2
Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in
which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for
informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant.
While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without
warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this
presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or
representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use
of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have
achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended
to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other
results.
Acknowledgements and Disclaimers
© 2013 IBM Corporation 3
Croissance des besoins d’analyse des données
Creating the need for
organizations to
understand and anticipate
customer behavior and
needs based on customer
insights across all
channels
The shift of power to the consumer 2
Creating the need for all
parts of the organization
to optimize all of their
processes to create new
opportunities, to mitigate
risk, and to increase
efficiency
Accelerating pressure to do more with less 3
The emergence of big data
Creating new
opportunities to capture
meaningful information
from new varieties of
data and content coming
at organizations in huge
volumes and at
accelerated velocity
1
© 2013 IBM Corporation 4
Accélérateur BLU: c’est quoi?
BLU combine des technologies de pointe
- 25+ brevets dans 10 laboratoires sur 7 continents
BLU augmente les fonctionnalités existantes de DB2
- Utilise des tables organisées en colonnes qui coexistent avec les tables classique organisées en lignes
- Utilise un moteur d’exécution colonne qui coexiste avec le moteur traditionnel
- C’est du DB2: TBS, BP, utilitaires, SQL, ne changent pas
Les tables organisées en colonnes utilisent
- Compression des données
- Processus vectoriels
- Optimisation de ma mémoire
- Et plus
© 2013 IBM Corporation 5
Accélérateur BLU: à quoi ça sert?
BLU est conçu pour vos requêtes analytiques
- Maximise l’utilisation de votre mémoire, mais…
- Ne nécessite pas une solution coûteuse « in-memory only »
BLU offre des avantages majeurs
- Performances ultra rapide (10x – 25x)
- Réduit le besoin de stockage (5x – 20x)
- Administration simplifié, « load-and-go »
© 2013 IBM Corporation 6
Super Fast, Super Easy – Create, Load, and Go!
Database Design and Tuning
1. Decide on partition strategies 2. Select Compression Strategy 3. Create Table 4. Load data 5. Create Auxiliary Performance Structures
• Materialized views • Create indexes
• B+ indexes • Bitmap indexes
6. Tune memory 7. Tune I/O 8. Add Optimizer hints 9. Statistics collection
DB2 with BLU Acceleration
1. Create Table 2. Load data
Repeat
VS
© 2013 IBM Corporation 7
Accélérateur BLU: pourquoi est-il spéciale?
Ses caractéristiques principales:
1. Facile à mettre en œuvre et à administrer
2. Extrême compression des données
3. Stockage des données en mode colonne
4. Data Skipping
5. Parallélisme multi processeur
6. Accélération CPU SIMD (Single Instruction Multi Data)
7. Cache mémoire intelligent
© 2013 IBM Corporation 8
Accélérateur BLU: Facile à mettre en œuvre et à administrer
Set one registry variable: DB2_WORKLOAD=ANALYTICS
- Then create the database to get optimal settings for an analytic environment optimized to your hardware
- It’s still just DB2: use traditional storage, utilities, SQL, application interfaces, and so on
Now just “load and go”
- Compression automatically done and statistics automatically collected
- No need for indexes, partitions, MQTs (materialized views), MDCs, hints, statistical views, etc.
- Underlying structures that support BLU are created and updated automatically
Ongoing maintenance is also minimal
- REORGs (for space reclaim) and workload management is all automatic
© 2013 IBM Corporation 9
Accélérateur BLU: Extrême compression des données
Massive compression with approximate Huffman (frequency-based) encoding
- Most frequent values encoded with fewest bits
- Extremely effective since all data in the column are the same data type
- Map entire values directly to column dictionary codes by both table and page levels
Register-friendly encoding greatly improves efficiency
- Encoded values are packed into bits matching the CPU register length
- Requires fewer I/Os and CPU cycles as well as better memory utilization
Encoded values do not need to be decompressed during evaluation
- Predicates and joins work directly on (actionable) encoded values
(Predicate examples: =, <>,<, >, >=, <=, between, etc.)
© 2013 IBM Corporation 10
Accélérateur BLU: Stockage des données en mode colonne
Efficient columnar storage automatically done during LOAD
Result: minimal I/O
Work performed directly on columns
Improved memory density
© 2013 IBM Corporation 11
Accélérateur BLU: Data Skipping Automatic detection of large sections of data that do not qualify for a query and can be ignored
Order of magnitude savings in all of I/O, RAM, and CPU
No DBA action to define or use – truly invisible
© 2013 IBM Corporation 12
Accélérateur BLU: Parallélisme multi processeur BLU Acceleration builds on traditional multi-core parallelism
Multi-core parallelism is a combination of Data and Functional parallelism
Data parallelism assigns dynamic partitions to retrieve data from disk
Functional parallelism divides queries into multiple parts by function
© 2013 IBM Corporation 13
Accélérateur BLU: Accélération CPU SIMD “Single Instruction Multiple Data “ (SIMD) CPU design:
- Performs the same operation on multiple data elements simultaneously
Various CPUs have built-in instructions to perform SIMD
- Your hardware already has this capability - BLU Acceleration simply exploits it (128-bit register usage is maximized as well)
- BLU Acceleration has been tested to perform on:
- AIX on Power - best performance on Power 7
- Linux 64-bit on Intel / AMD - best performance on Intel Nehalem or better
© 2013 IBM Corporation 14
Accélérateur BLU: Cache mémoire intelligent
New algorithms cache in RAM effectively
High percent of interesting data fits in memory
- We leave the interesting data in memory with the new algorithms
Data can be larger than RAM
- No need to ensure all data fits in memory
- Optimization for in memory and I/O efficiency
RAM
DISKS
Near optimal caching
© 2013 IBM Corporation 15
Accélérateur BLU : En conclusion
Technologie intégrée au moteur DB2
Column-organized table approach
Orienté pour les workloads hybrides et analytiques
Transparent pour les applications
Implémentation et administration simplifiés
Multi-core and SIMD parallelism
Data skipping
Meilleur taux de compression pour les données
Super Fast, Super Easy— Create, Load and Go!
No Indexes, No Aggregates, No Tuning, No SQL changes, No schema changes, No REORG/RUNSTATS
© 2013 IBM Corporation 16
Fabrizio Danusso Client Technical Professional fabrizio.danusso@fr.ibm.com
© 2013 IBM Corporation 18
© Copyright IBM Corporation 2013. All rights reserved.
• U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
• IBM, the IBM logo, ibm.com, Informix, are trademarks or registered trademarks of International Business Machines Corporation in the
United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information
with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this
information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM
trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml
Other company, product, or service names may be trademarks or service marks of others.
top related