© 2010 IBM CorporationApril 6, 2011

Informix Warehouse Accelerator

Jacques RoyIBM, Informix development

2 © 2010 IBM Corporation

Agenda

■ Data warehouse industry trends■

■ Data warehouse on Informix■

■ Infomrix warehouse accelerator■

3 © 2010 IBM Corporation

Sate of Data Warehousing 2011

DBMS Market in 2011:■ DBMS market at the close of 2009 was approximately $21.2

billion (2010 data not yet available)■ Data Warehouse DBMS market was approximately 35% of the

DBMS market or $7.42 billion

Key Findings:■ Data warehouse DBMSs have evolved to a broader analytics

infrastructure supporting operational analytics, corporate performance management and other new applications and uses.

■ Cost is driving interest in alternative architectures but performance optimization is driving multi-tiered data architectures and a variety of deployment options - notably a strong interest in in-memory data mart deployments.

4 © 2010 IBM Corporation

Sate of Data Warehousing (cont.)

Market Dynamics for 2011■ Today, smaller data warehouses, those less than 5 TB's of source

system extracted data (SSED) are the only "data warehouse" for the entire organization and are commonly solving organizations' analytic needs.

Analysis:■ Gartner only rarely encounters an organization which has actually

delivered on the Enterprise Data Warehouse (EDW) vision. The EDW remains a design principle, but it is rarely if ever actually deployed. Gartner estimates that between 70% and 75% of all systems referred to as EDW are actually single business departments in nature.

■ Optimization techniques such as summaries, aggregates and indexes are simply the result of performance restrictions inherent to normalized data and the way the RDBMS manages rows and columns.

5 © 2010 IBM Corporation

State of Data Warehousing (Cont.)

A Glimpse Into the Future■ Vendor solutions began to focus even more on the ability to

isolate and prioritize workload types including strategies for dual warehouse deployments and mixing OLTP and OLAP on the same platform.

■ In-memory DBMS solutions provide a technology which enables OLTP/OLAP combined solutions. Organizations should increase their emphasis on financial viability during 2011 and even into 2012 as well as aligning their analytics strategies with vendor road maps when choosing a solution.

7 © 2010 IBM Corporation

Existing Informix Warehouse Features

■ Performance & Scalability– Inherent SMP Multi-threading– Parallel Data Query (PDQ)– Light Scan for fast table scans– Online Index build– Efficient Hash Joins– Auto Fragment Elimination– Memory Grant Manager (MGM)– High Performance Loader– Optimistic Concurrency

■

■ Easy of Management– Time cyclic data management using Range Partitioning– Sophisticated Query Optimizer for OLTP and Warehousing

8 © 2010 IBM Corporation

Informix Warehousing Moving Forward

■ Goal is to provide a comprehensive warehousing plat form that is highly competitive in the marketplace–

– Incorporating the best features of XPS and Red Brick into IDS for OLTP/Warehousing and Mixed-Workload

–

– Using the latest Informix technology in:• Continuous Availability and Flexible Grid• Data Warehouse Accelerator using latest industry technology

–

– Integration of IBM’s BI software stack

13 © 2010 IBM Corporation

BI Tools for Informix

The Performance Management Framework Cognos identifies best-practice decision areas, or information sweet spots by business function:

Cognos 10 provides a comprehensive set of BI tools for:

� Reporting

� Analysis

� Dashboards

� Scorecards

Performance Management Framework for:

� Solutions for different areas of the organization

14 © 2010 IBM Corporation

Third Generation of Database Technology

According to IDC’s Article (Carl Olofson) – Feb. 2010

1st Generation:

- Vendor proprietary databases of IMS, IDMS, Datacom

2nd Generation:

- RDBMS for Open Systems, dependent on disk layout, limitations in scalability and disk I/O

- Database tuning by adding updating stats, creating/dropping indexes, data partitioning, summary tables & cubes, force query plans, resource governing

3rd Generation: IDC Predicts that within 5 years:

■ Most data warehouses will be stored in a columnar fashion

■ Most OLTP database will either be augmented by an in-memory database (IMDB) or reside entirely in memory

■ Most large-scale database servers will achieve horizontal scalability through clustering

15 © 2010 IBM Corporation

Market Data: Key Drivers of Change

16 © 2010 IBM Corporation

Informix Warehouse Accelerator

How is it different?• Performance: Unprecedented response

times to enable 'train of thought' analysis frequently blocked by poor query performance.

• Integration: Connects to IDS through deep integration providing transparency to all applications.

• Self-managed workloads: queries are executed in the most efficient way

• Transparency: applications connected to IDS, are entirely unaware of IWA

• Simplified administration: appliance-like hands-free operations, eliminating many database tuning tasks

What is it?

The Informix Warehouse Accelerator (IWA) is a workload optimized, appliance-like, add-on, that enables the integration of business insights into operational processes to drive winning strategies. It accelerates select queries, with unprecedented response times.

Breakthrough Technology Enabling New Opportunities

17 © 2010 IBM Corporation

Breakthrough Technologies for Performance

1

2

34

5

6

7 1

2

34

5

6

7

Row & Columnar DatabaseRow format within IDS for transactional workloads

and columnar data access via accelerator for OLAP queries.

Extreme CompressionRequired because RAM is the limiting factor.

Massive ParallelismAll cores are used within used for queries

Predicate evaluation on compressed data

Often scans w/o decompression during evaluation

Frequency PartitioningEnabler for the effective parallel access of

the compressed data for scanning. Horizontal and Vertical Partition

Elimination.

In Memory Database3rd generation database technology avoids I/O. Compression allows huge databases

to be completely memory resident

Multi-core and Vector Optimized Algorithms

Avoiding locking or synchronization

18 © 2010 IBM Corporation

IWA: Characteristics

• A dedicated SMP system (Linux on Intel x86_64)• No changes to the applications

–Applications continue to attach to IDS.

–When applicable query needs to be executed, IDS exploits the accelerator transparently to the applications

–Fencing and protection of IDS against possible accelerator failures

• Order of magnitude performance improvement • Reducing need for tedious tuning of IDS (partitioni ng, indexes, etc.) • Appliance-like form-factor

–Hands free operations

• Significantly improved price/performance and TCO as a combined effect of:

–Accelerating intensive & complex analytics queries

–Orders of magnitude performance improvement for accelerated queries

–Reduced DBA effort for tuning accelerated queries

19 © 2010 IBM Corporation

Sample Customer Results: Case Study #1

Query Description Informix Informix w ISAO Notes Improvement

1 Find Top 100 Entities 1:28:22 0:01:28 Fact Table Scan 6023.23%

2 Find Top 100 Members 1:22:32 0:01:05 Fact Table Scan 7640.45%

3Summarize all transactions by State

and County 1:34:37 0:00:14 Fact Table Scan 41708.49%

4Summarize the top 10 Commodities

by State and County 1:05:03 1:03:35

IWA did not support this subquery query 102.29%

5

Detailed Report on Specific Programs, States, Counties and Years 0:00:00 0:00:00 Index Read 83.45%

6Detailed Report on Specific

Programs in a Date Range 0:00:06 0:00:06 Index Read 108.41%

7

Summarize all transactions by State, County, City, State, Zip, Program, Program Year, Commodity and Fiscal Year 1:48:58 0:00:41 Fact Table Scan 15800.89%

8

Find Entities where the payments do not equal total Member Transaction Amounts

Failed - Long Transaction

Failed - Long Transaction

I did not configure enough logs to support the query

Totals 7:19:37 1:07:09 654.69%

20 © 2010 IBM Corporation

Government Agency Datamart

� Performance expectation goals were up to 20X OLAP-style Queries

� Tests were done on a Intel x86_64 SMP box running Linux RHEl

� Microstrategy Report was used, which generates 667 SQL statements

�537 are SELECT statements.

� Datamart for this report has 250 Tables and 30 GB Data size

� Informix Panther and IWA run this report in 67 seconds.

� 7 seconds in IWA and 60 seconds in Informix (TEMP table processing, etc)

� Without IWA, total runtime on Informix 11.70 on the same HW is 40 Minutes!

� The same report today runs on XPS & SUN HW (Sparc M9000) and takes 90 mins.

� Performance gain for the customer would be ~90x !!!

21 © 2010 IBM Corporation

IWA Referenced Hardware Configuration

Intel(R) Xeon(R) CPU X7560 @ 2.27GH 4 X 8

Memory 512G

6 disks 300 GB SAS hard disk drives each

- 4-processor, 4U rack-optimized enterprise server with Intel® Xeon® processors.

- 8-core, 6-core and 4-core processor options with up to 2.26 GHz (8-core), 2.66 GHz (six-core) and 1.86 GHz (four-core) speeds with up to 16 MB L3 cache

- Scalable from 4 sockets and 64 DIMMs to 8 sockets and 128 DIMMs

- Optional MAX5 32-DIMM memory expansion

- 16x 1.8" SAS SSDs with eXFlash or 8x 2.5" SAS HDD s

Options:

22 © 2010 IBM Corporation

IWA Software Components

■ Linux on Intel x86_64 (RHEL 5 or SUSE SLES 11)■

■ IDS 11.70 + IWA code modules including IDS Stored Procedures (Informix Ultimate Warehouse Edition)

■

■ ISAO Studio Plug-in – GUI for Mart definition■

■ OnIWA – On Utilities for Monitoring IWA