introducing ibm balanced warehouse - ibm - united states

®

IBM Software Group

© 2007 IBM Corporation

Introducing IBM Balanced Warehouse

Balanced Configuration Units (BCU)

David Cope

EDW Architect – Asia Pacific

IBM Software Group

2

Today’s Market

� Warehouse are larger and more complex than ever

� Consolidation is a key strategy

� The need for business agility is a key strategy. Warehouses are now architected for agility as opposed to performance

� Technology is only one part of the solution – business/operational processes and people process also need to be addressed for a successful single source of truth

� Warehouse become critical business assets with 24 x 7 requirements and DR strategies

� TCO remains an issue but now in the care and feeding of the warehouse, not the acquisition

� In comes the “appliance”………

�Customers of the warehouse perceive delivery as “simplified”

�Extreme cost pressures

�Extreme delivery pressures

IBM Software Group

3

What did the Appliance Really do to the Market?

� Forced a customer requirement of better time to value to be focused on by the

DW vendors

� Highlighted the requirement for managing cost in the warehouse environment

� Accelerated the growth of Linux as an alternative in data warehousing

� Allowed customers to choose commoditized components at reduced costs

� Gave the customers the luxury to focus on the important things

�Doing it yourself is “so out”

�Focus on delivery of information, not integration

IBM Software Group

4

Assisting Customers in Riding the Waves of Change

� On the first round of “appliance” Customers told us:

�Fast time to value

�Totally integrated

�Single source of support

�Tested and validated

�Best Practices included

�Answer = Balanced Warehouse (BCU)

� But we always knew appliances would run into the same issues as traditional

warehouses

�Mixed workload

�High concurrency

�Embedded analytics

�Real-time/Right Time

�Integration of all information (structured or something else)

�This was IBM’s Vision from the start with the Balanced warehouse

� Balanced Warehouse + Dynamic Warehousing = Appliance Plus

IBM Software Group

5

Why we say “Appliance Plus”

“Other DW Appliance Characteristics”

� Fast time to market, no component integration

� Single solution and support

� Proprietary architecture

� Excellent TCO

� Fast loading of data but limited on concurrency

� Great for data mart solutions

� Component redundancy, less need for disaster recovery

� Limited scalability

� Limited bundling and integration of analytics, capability of handling differing data types

Balanced Warehouse

� Fast time to market, no component integration

� Single solution and support

� Industry standard and open architecture

� Excellent TCO

� Great performance on load balanced with the ability to handle mixed workloads for right time enterprises

� Multiple entry points and scaling options –enterprise capable building blocks

� Fully redundant, highly available with optional DR

� Great scalability

� Analytics, workload management, modeling and data integration – in the box!!!

IBM Software Group

6

IBM Data Warehouse “Appliance Plus”

� The introduction of the “Data Warehouse Appliance” by smaller vendors set the

stage for defining the requirement as simple, integrated and proprietary.

�It did NOT address the critical challenges most enterprise customers face

� The Balanced Warehouse is the only “DW Appliance” that was introduced with

the vision of solving more challenges than cost and time to market all on an

open systems architecture

�We continue to work on the “real life” challenges of our customers as they consolidate, manage complexity and deliver information real time

� IBM will continue to address the warehouse market more holistically with a

solution focus and an eye to TCO and time to market

Not just a DW Appliance, but an “Appliance Plus “

IBM Software Group

7

Introducing IBM Balanced WarehouseTM

A fast track to warehousing

Simplicity� Predefined configurations for reduced

complexity

� One number to contact for complete solution support

Flexibility for growth

� Add BCUs to address increasing demands

� Multiple on-ramps for different needs

� Reliable, nonproprietary hardware for reusability

Optimized performance

� Preconfigured and certified for guaranteedperformance

� Based on best practices for reduced risk

Balanced Configuration Unit (BCU)

Preconfigured, pretested allocation of software, storage and hardware to support a specified combination of function and scale

IBM DB2®

Warehouse

SIMPLEFLEXIBLE

OPTIMIZED

Better than an appliance

Balanced Warehouse

IBM Software Group

8

What is the IBM Balanced Warehouse™?

Simple:

Reduced Complexity: Ships configured and ‘Ready to go’

Pre-tested: Validated by IBM

Full integration: High Performance DB2 Warehouse delivered ‘load ready’

Predictable, modular growth through the usage of BCU

One number: Single point of contact for all support issues

Flexible

Modular scalability: Grows with your demands, not your vendor’s

Open and Reusable: Ensures that you are not locked in to proprietary limitations

Spectrum of offerings: Multiple on-ramps to warehousing

Ability to plan for and price the growth of your company’s warehousing needs

Optimized:

IBM Balanced Warehouse solutions are thoroughly tested and tuned for performance

Solutions developed through IBM Best Practices of successful client implementations

Low Risk: Guaranteed performance

IBM Software Group

9

Guiding Design Principles for the Solution Offering

� Simplicity, Simplicity, Simplicity !!

� Price to performance ratio

� Balanced performance

� Scalability

� Stability

� Fault tolerance

� High availability

� Ease of installation and implementation

� Packaging density

� Industry-standard components

� Minimize solution permutations

� Modular growth at a predictable cost

p575

AIX

DB2

Linux

x346, e326

IBM Software Group

10

IBM Balanced Warehouse™ – Solution Classes

The IBM Balanced Warehouse™ can be broken down into 3 different solution classes that each target and

serve a distinct warehousing market segment.

C-Class D-Class E-Class

Enterprise SolutionsGrowth SolutionsApplication Solutions

Class Name

Performance

Highly Scalable Enterprise Data Warehouses

Advanced Departmental Data Marts and Smaller Data Warehouses

Entry-point for Smaller Warehousing Applications and Reporting Tools

Scalability High - BCU Very High - BCULimited

Size 1TB to 5TB* 1TB and Up50GB to 1TB

NOTE: * Can scale higher, yet it is recommended to look to an E-Class solution to maximize performance optimization

Users No Hard LimitsUp to 200 Users

Segment

IBM Software Group

11

IBM Balanced Warehouse™ – Offerings 1H 2007

C1000 C3000 D5000 E6000 E7000 E7050

Enterprise SolutionsGrowth SolutionsApplication Solutions

Scalable Units BCU BCU BCU BCU

Operating System

Scaling Increments1TB 1TB 2TB 2TB

DB2 Warehouse

System

StorageEXP3000 DS4800 DS4800 DS8100

x3650 (I) x3455 (A) p5-575 p5-575

Adv, Base or Ent Base or EnterpriseStarter Intermediate

x3650 (I)x3500 (I)

EXP3000

N/AN/A

~ 250GB~ 50GB

SupportBCU Support BCU Support

Solution Name

Novell / Red HatNovellNovellNovell

Performance

Reseller/Distributor Provided

Internal

IBM Software Group

12

IBM Balanced Warehouse™ - Offering Graph With the addition of Application and Growth Solutions, IBM is uniquely positioned to offer complete data

warehousing solutions across the entire warehousing spectrum.

Warehouse Size

Perf

orm

an

ce

C1000

C3000

D5000

E6000 E7000

E7050

Application Solutions Growth Solutions Enterprise Solutions

1TB Limit on C-Class

50GB to 1TB 1TB to 5TB 5TB and up

IBM Software Group

13

D & E Class Data Warehouse Platforms

D Class Appliance

D5000

�Supports 500 GB upwards

�Dedicated Disk

� Low Price Point

� 2 way Building Blocks

E Class Appliance

Linux BCU

�Aimed at 1-2 TB and upward

�Configured for High availability

�Configured for High end Scalability


�Mid Price Point

P-Series BCU

�Aimed for High Data Volumes

� Larger Rate of Growth (bigger building blocks)

�Aimed for High End User Workloads

�Highly available Hardware Architecture


� Provides same Database Architecture with choice of Platform based on Your Business

Needs

� Provides Scalable Growth Path with a Low Cost entry point upgradeable as your

Business requirements Change

IBM Software Group

14

IBM Balanced Warehouse™

1TB Balanced WarehouseIncludes:1 Data BCU – 2 Dual Core1 Admin BCU

Size: 1TB

Total Size: 1TB

E6000

E7000

Sys

tem

xS

ys

tem

p5 2TB Balanced Warehouse

Includes:1 Data BCU – 8 POWER5+1 Admin BCU

Size: 2TBTotal Size: 2TB

1TB BCUIncludes:1 Data BCU – 2 Dual Core

Size: 1TB

Total Size: 2TB

1TB BCUIncludes:1 Data BCU – 8 POWER5+

Size: 2TBTotal Size: 4TB

Add 1 E6000 BCU

Add 1 E7000 BCU

+

+

Sizing for the System x based Balanced Warehouses follows single increments of 1TB Data BCUs (1,2,3....n)

Sizing for the System p based Balanced Warehouses follows single increments of 2TB Data BCUs (2,4,6....n)

System x Data BCUs have 2 dual core processors (AMD) for a 4-way node. Each balanced piece of storage is 250GB.

4 * 250GB = 1TB per node

System p Data BCUs have 8 processors (POWER5+) for a 8-way node. Each balanced piece of storage is 250GB.

8 * 250GB = 2TB per node

Scaling Example: System x vs. System p

IBM Software Group

15

IBM Balanced Warehouse™

Up to 250/1TB

No

~1400MB/sec

No

2 x Int. SAS @146GB +

2 x EXP3000

24 x SAS @146GB

DWE 9.1.2

SLES10

16GB (4GB/core)

2 x Intel EM64T Dual-core

x3650

YesCall home for storage

YesServer failure HA

Up to 200/800GB

620 MB/sec

DS4800

56 x 73GB 15k RPM FC disk

DWE 9.1.1

SLES9 SP3

16GB (4GB/core)

2 x AMD Opteron Dual Core

x3455

GB per BPU/BCU

Maximum Seq. I/O per BCU

Storage

DB

OS

RAM

Processor

System

E6000D5000

Linux Solution Technical Comparison (D5000 vs. E6000)

IBM Software Group

16

The Balanced Configuration Unit (BCU)

GOAL: Reduce time to business value and total cost of ownership (TCO) through a reusable and validated approach that promotes better consistency and higher quality for data warehouse implementations.

Data Warehouse

…

AIX

AIX

BPU BPU BPUBPU

BPU BPU BPUBPU

Memory

CPUCPU CPUCPU CPUCPU CPUCPU


AIX

AIX

BPU BPU BPUBPU

BPU BPU BPUBPU

Memory



AIX

AIX

BPU BPU BPUBPU

BPU BPU BPUBPU

Memory



BCU 1 BCU 2 BCU n

� Minimum replicable HW/SW stack necessary to start or expand the infrastructure of a Business Intelligence (BI) system.

� Provides a scalable performance ratio of disk I/O to memory to CPU to network

� A prescriptive approach for building data warehouse infrastructure

IBM Software Group

17

Guiding design principles for the BCU

� A number of design principles have been used in guiding the design of the BCU, which include:

�Balanced performance

�Scalability

�Stability

�Fault tolerance

�High availability

�Minimize solution permutations

� BCU Touchpoints:

�Database to IO Subsystem

�Database to Memory

�Database to other parts of Database

�Database to Application (ETL)

�Database to End-User Tool

The BCU is configured for an Enterprise BI solution.

As such, price performance is a high priority factor, but not the only

driving factor.

IBM Software Group

18

The Balanced Partition Unit (BPU)

� A BPU is defined as 1 DB2 Data

Partition and the resources

associated with it.

� DB2 Data Partition

� DB2 Agents

� CPUs for DB2 Agents

� Memory for DB2 Agents

� Storage for Partition Data

CPU

I/O Channels

BPU

Memory - Bufferpool

Communication

1 DB2 Data Partition

DB2 Agents

Storage Capacity

IBM Software Group

19

The Balanced Configuration Unit (BCU)

� A single operating system image

provided by an individual server

and the resources associated with

it. A single OS imagine will

support multiple BPUs.

� 8 BPUs

� 32GB memory

� 8 Power5 processors

� Single AIX OS image

� Dedicated Modular Storage

8 CPUs

I/O Channels

Power5 BCU

32GB of memory

Communication

8 BPUs

OS image

Storage Capacity

IBM Software Group

20

AIX Data BCU TypesC

om

mo

n

Co

mm

on

C

hara

cte

risti

cs

Ch

ara

cte

risti

cs

� One p5-575 server

� Eight 1.9 GHz CPUs at 10.4 RBI performance

� Eight DB2 database partitions (data BPUs)

� 32 GB memory (32 DDR1 1GB DIMMs)

� Two Gb Ethernet adapters for DB2 FCM interconnect network

� 2 of the 4 integrated Gb Ethernet ports used for corporate network connectivity

� 2 integrated HMC ports are used for the HMC Console

� 1 of the 2 sides of a Feature Code 5794 Remote I/O drawer with 10 PCI-X slots

Active DW Mixed Workload Larger

Archive

Traditional DW Batch Refresh Query Window

Workload

24,000

12,000

I/O Trans

(IOPS)

≤ 1200

≤ 600

I/O

Thruput

(MB/sec)

1601282BCU BCU

#2#2

80641BCU BCU

#1#1

Usable

Disks

Qty

EXP710 Disk Enclosures

Qty

Fibre

Channel

Adapters

for Disk

DS4800

Storage

Servers

Qty

IBM Software Group

21

Example Metrics for Data BPU Calculation

� Per DB2 Partition / Processor (1 DB2 partition per CPU)

� 150 – 250 GB Raw Data – typically 150GB Active Raw Data

� 450 – 1000GB Useable Disk

� RoT: 3-4 x Raw Data

� Mainstream disk choices; 36Gb, 73Gb

� 146Gb only where large amount of the data is inactive

� 4 GB Memory

� Assuming DB2 v8 and 64 bit

� Database network (FCM) – circa 20 MB/sec

� RoT: 80MB/sec per GB Ethernet port

� Equates to 1 GB Ethernet Adapter per 4 CPUs

� Storage Throughput – circa 80 - 160MB/Sec

� RoT: 10MB/sec random uncached read per disk

� Equates to 1 x 2GB FC card per 2 CPUs or 1 x per SCSI 320 card per 2 CPUs

� Storage Transactions – circa 1200 – 2400 IOPS

� RoT: 150 concurrent random uncached read IOPS per disk.

� Minimum 8 disks at 15k RPM; > 12 is more balanced

� Ensure storage Server/Controller capacity to handle throughput and transactions

Originate from “Typical/Average”

Workloads

IBM Software Group

22

introducing ibm balanced warehouse - ibm - united states

Documents