ciber creates next-generation it infrastructure solution for sap hosting services
DESCRIPTION
CIBER Managed Services GmbH built a dynamic Cloud Infrastructure for SAP landscapes, based on a combination of IBM server, storage, database, and system management software. The new storage infrastructure, which consists of the IBM XIV Storage System and IBM SAN Volume Controller, offers individual service classes to suit different customer needs, maximizing operational efficiency and minimizing expenses. The SAP application landscapes run on both IBM Power Systems servers and IBM System x Blade Server technologies. Visit http://bit.ly/KWh5Dx to 'Follow' the official Twitter handle of IBM India Smarter Computing.TRANSCRIPT
CIBER creates next-generation IT infrastructure solution for SAP hosting services
IBM SAP International Competence Center
“XIV is simply amazing. The usage of XIV real time copy, which is able to copy many terabytes of data very fast, reduces the downtime for provisioning drastically. The main focus as a service provider is the sector of customer satisfaction, which is especially improved by the usage of the new XIV system.”
Markus Peltz Data center Manager CIBER Managed Services GmbH
“Through the 100 percent virtualization you gain a dynamic infrastructure and you get the ability for an easy billing system.”
Carl-Philipp Müller Lead Sales Operations Services CIBER Managed Services GmbH
CIBER creates next-generation IT infrastructure solution for SAP hosting services
About this paper
CIBER Managed Services GmbH built a dynamic Cloud Infrastructure for SAP landscapes, based on a combination of IBM server,
storage, database, and system management software. The new storage infrastructure, which consists of the IBM XIV Storage System
and IBM SAN Volume Controller (SVC), offers individual service classes to suit different customer needs, maximizing operational
efficiency and minimizing expenses. The SAP application landscapes run on both IBM Power Systems servers and IBM System x
Blade Server technologies.
CIBER creates next-generation
IT infrastructure solution for
SAP hosting services
IBM SAP International Competence Center
“IBM was chosen as a partner, because of the very good long-term experience with them. Especially their integrated concepts and their highly skilled consultants for very special challenges, plus the capability to implement the required standards within time and budget convinced us to do this project with IBM.”
Mats Johansson, Head of Operations, CIBER Managed Services GmbH
4
Customer Objectives
Provide a highly scalable and dynamic infrastructure to host •customer-oriented SAP landscapes for small- and medium-
sized businesses in a private cloud environment that will
support the future growth of the business
Establish high-performance infrastructure, featuring •transparent and easy billing models for high customer
satisfaction
Reduce CIBER’s IT operational costs without sacrificing •service quality
Operate a complete range of SAP Business Suite •components on an integrated and shared storage
infrastructure and powered by either IBM Power servers or
IBM System x BladeServer technology depending on
customer requirements
Introduce reliable and comprehensive backup, resilience, •and disaster recovery solutions
Increase the speed of new SAP application provisioning, to •help respond faster to business requests.
Customer Benefits
Ability to provide faster and more flexible and scalable •SAP hosting services to businesses, with 99.9 percent
virtualization and improved Service Level Agreements.
Improved business continuity and higher availability even •across two locations, using IBM SVC storage virtualization
and the IBM XIV Storage System
Reduced workload for IT staff through simplified setup and •administration of the XIV Storage Systems
Reduced operating costs with improved service quality •supported by three levels of storage service classes, an
integrated and virtualized storage architecture, and new
server hardware
Reduced time for provisioning and deployment of new •systems from weeks to hours with the new storage solution
based on IBM XIV technology, using the XIV Snapshot
functionality and the SVC FlashCopy functionality.
Improved database performance with DB2 by a factor of •eight in online operations
Achieved higher level of automation in database •environment
Gained disk space savings and reduced data growth by •deploying DB2 compression and the reclaimable Storage
features (reduced size of tablespaces from about 1 TB
down to 300 MB for SAP NetWeaver Business Warehouse)
The BladeCenter technology allows an option to upgrade •online to current BladeServer models, such as HX5 and
also offers hot swap technology for blades to ensure high
availability with the BladeCenter Open Fabric Manager, if
needed.
IBM Solution
Worked with IBM and SVA, an IBM Premier Business Partner, •to design and implement a highly flexible and cost-efficient
infrastructure and a next-generation storage solution for SAP
in only four months
Set up two IBM Power 560 servers, each with two POWER6 •processors with four cores, and two IBM Power 750 servers,
each with four POWER7 processors with eight cores, running
SAP applications on IBM AIX in about 40 LPARs, sharing
processor resources and direct mappings of LPARs to
LUNs. The two POWER7 machines provide 170,000 SAPS
and process 12,500 sales order positions per hour between
them
Installed two BladeCenter H, both with six HS22 •BladeServers powered by quad-core 2.40 GHz Intel Xeon
L5530 processors, and four HS22 Blade servers powered by
six-core 2.60 GHz Intel Xeon X5640 processors, running
VMware vSphere 4, forming several SAP clusters and
hosting guest operating systems Novell SUSE Linux
Enterprise Server 10 and 11 as well as Windows Server 2003
and 2008
DB2 for Linux, UNIX and Windows is the most appropriate •database platform to support the SAP applications at CIBER
Installed six IBM System x3650 M2 systems, powered by •quad-core 2.26 GHz Intel Xeon Processor L5520 series
processors, running Novell SUSE Linux Enterprise Server
10 and 11 as well as Windows Server 2003 and 2008
Implemented a next generation three service class •storage solution based on IBM SAN Volume Controller 5.x,
IBM XIV Storage System and IBM System Storage DS3400
Implemented IBM Tivoli Storage Manager and two IBM •System Storage TS3500 tape library systems, each with
five LTO4 tape drives, 260 slots, and about 200 cartridges
(migration speed to tape of 350 GB/h).
5
Application Services provide comprehensive SAP-functional
and business process-related support. With Applications
Services, CIBER supports customers in their daily business.
Furthermore Application Services support the continuous
improvement process. Additionally, CIBER renders SAP
Software Support for all SAP Software Maintenance customers.
For these customers, the CIBER support organization works as
an integral part of the Global SAP support organization.
Service Management ensures comprehensive and reliable
service delivery. CIBER’s customers benefit from the efficiency
of individually assigned Service Teams delivering the agreed
services. A dedicated Service Manager coordinates the delivery
of the agreed services, provides transparent reporting of all
activities and KPIs, and represents the customer and their needs
inside the CIBER service organization.
Background, starting point and objectives
CIBER is a full service IT partner founded in 1974 and
headquartered in Denver, USA. It employs more than 8,500 staff
at 85 locations in 18 countries, and generates annual revenues of
about $1.037 billion. CIBER specializes in implementing and
supporting complex SAP and similar solutions for a broad
customer base in multiple industry sectors.
About CIBER
More than 400 of CIBER’s total of 1,300 SAP consultants work for
CIBER Germany, which has particular expertise in retail and
consumer products, chemicals and pharmaceuticals,
automotive industry, and financial services. With more than 20
years of SAP solution implementation experience, CIBER
Germany achieved sales of €55 million in 2009.
CIBER Managed Services GmbH is based in Freiburg,
Germany, and was established in January 2010 to provide SAP
application and operation services for small- and medium-sized
businesses from its two data centers, with more than 120
experienced consultants.
The Managed Services business specializes in post-
implementation-services, designed to preserve the initial
investments that customers make in their SAP implementations.
Managed Services consist of Operations Services, Application
Services, Software Support and comprehensive Service
Management.
Operations Services provide customers with a full range of SAP
system maintenance capabilities, covering the provision and
operation of modern data centers, the necessary IT
infrastructure, and the administration and maintenance of SAP
and IT-related applications.
As a VMware Solution Provider with a special expertise in SAP
implementations, CIBER has competencies and proven
processes in architecture and sizing, and also in deployment
and analyzing complex virtualized SAP landscapes.
Figure 1: CIBER Services
6
Initial IT environment
In order to offer operation services for SAP as a cloud provider,
CIBER Managed Services needed to build up two completely
new data centers. The new and flexible technologies allow
CIBER to satisfy growing customer needs. As a future service,
CIBER wants to implement a self-service portal to allow
customers to request SAP services. Subsequently, the
necessary server capacity, operating system, applications and
storage could be provisioned and charged back automatically.
This would keep costs low and enable CIBER to serve smaller
clients cost-effectively with enterprise-class solutions.
CIBER selected SAP Solution Manager and its own CIBER
Solution Suite to provide the central interface, which includes a
full service desk based on ITIL® V3. The CIBER tool and SAP
Solution Manager enable the creation of a virtual server (either
using VMware vSphere 4 technology on IBM System x or using
LPARs on IBM Power servers), the installation of the operating
system, assignment of the LUNs for storage and the deployment
of the SAP applications. The CIBER team needed next to
consider the best infrastructure to support this business strategy.
With very positive experience of IBM System x and IBM Power
Systems servers, CIBER chose to continue on the same strategic
path, and selected IBM XIV Storage Systems with IBM SAN
Volume Controller to address performance, high availability
scenarios, and premium support for data center management.
Redundant FC connection
Figure 2: SAP private cloud
7
Business challenges and project objectives
The thinking behind the technology strategy was based on the
view that although customer workload was growing
continuously, the nature of the demand was highly volatile.
Providing an industry-standard IT services solution with server
capacity sold in fixed amounts would not deliver the kind of
flexibility that customers wanted, and CIBER wanted to create a
solution that could scale up and down in performance – and
allow customers to pay on a more cost-effective usage basis.
As a starting point, CIBER needed an infrastructure capable of
hosting 120 SAP instances with about 8,000 SAP users, and up
to 20 systems for one customer in a private cloud environment.
The customers are offered free choice among different operating
systems and platforms, from IBM AIX on IBM Power Systems to
SUSE Linux Enterprise Server or Windows Server on IBM
System x BladeServer technology. While CIBER recommends
that its customers use IBM DB2 for Linux, UNIX and Windows,
some customers run MaxDB, Oracle, and MS SQL Server
databases. The infrastructure is capable of running all these
operating systems and databases, and the CIBER team can
support all of these systems.
CIBER operates the complete range of SAP ERP software
components, as well as SAP applications such as Supplier
Relationship Management (SRM), SAP ERP Human Capital
Management (HCM), Advanced Planning and Optimization
(APO), Customer Relationship Management (CRM), SAP
NetWeaver Business Warehouse (BW) and SAP NetWeaver
Portal. Additionally, CIBER offers its own solutions based on SAP
All-in-One software, including Rapid Retail, QuickWear,
Fabricated Metals, Industrial Machinery and Components,
Aerospace and Defense, and CIBER Solution Suite, a new
intuitive front-end for SAP Solution Manager. Each SAP instance
runs in its own LPAR.
If an additional Java Stack is needed, this is installed in a
separate LPAR. The largest SAP environment provides up to
170,000 SAPS in a total of 20 instances, processing up to 12,500
sales order positions per hour. On customer request the
environment is set up as a Power HA for AIX cluster to support
high availability of logical partitions for business critical systems
in a two-node cluster with automatic failover.
Continuing the theme of continuous availability, CIBER uses Live
Partition Mobility, available on Power servers with POWER6
processors and upwards. Live Partition Mobility enables the
migration of entire running AIX partitions and hosted applications
from one physical server to another without disrupting services
and loads. Combined with LPARs, SAN Volume Controller and
other virtualization technologies, Live Partition Mobility will
further enhance CIBER’s ability to respond to customer
demands.
Alongside the scale-up scale-down flexibility for individual
customers, CIBER planned for overall systems growth that would
equip them to respond to the total business demand. The
solutions must therefore be able to expand in capacity as a
whole, while preserving the ability to offer each customer exactly
the right services.
Knowledge of new customer contracts, the customer queue,
and planned growth predicted that the infrastructure at the
current data centers would reach capacity in less than six
months. CIBER planned to build the new data center in less than
four months to allow for sufficient transition time and maintain the
delivery of high service levels for its SAP hosting customers.
8
Server architecture
Working with SVA and IBM, the CIBER team designed and
implemented a highly flexible and fully virtualized environment,
capable of hosting almost every kind of SAP application
landscape.
SVA, an IBM Premier Business Partner, provided implementation
services. SVA installed the SAN Volume Controller, XIV, x3650
and BladeCenters. Ten SVA and IBM technicians completed the
implementation on time and on budget in a six-week period –
during part of which one of the data centers did not have a fully
functioning electricity supply.
In essence, everything is virtualized – the servers, storage and
network. When an application grows in workload and functional
reach, its processes are distributed among multiple virtual
machines to ensure performance is maintained, using either
VMware virtual machines on System x or logical partitions
(LPARs) on Power servers. VMware delivers the freedom to be
fully supported for SAP production environments on both Linux
and Windows operating system and multiple x86-Hardware.
VMware virtual machines, whereby each allows at most 8 virtual
CPUs and 255 GB RAM, are smaller than the virtual machines
available in LPARs, which support maximum of 256 virtual CPUs
and 1 TB RAM. When applications grow, this scale-out
distribution of processes happens earlier on System x than on
Power servers.
The result is that CIBER has gathered a great deal of experience
in the scale-out of SAP applications, particularly on System x.
Examples include separation of database and application server
(that is, moving from two-tier to three-tier configurations),
separation into multiple application servers, and separation of
SAP Central Services. The SAP Central Services include the
unique SAP application system services, including enqueue
server and message server.
IBM System x and IBM BladeCenter
CIBER runs IBM BladeCenter H chassis featuring IBM
BladeServer HS22, each of them powered by two 2.40
GHz quad-core Intel Xeon Processor E5530 series
processors and four HS22 Blade servers powered by
hexa-core Intel Xeon Processor X5640 series processors
at 2.6 GHz. The blades run multiple virtual servers –
collectively around 500 systems – using a combination of
VMware vSphere 4 as hypervisor, and Novell SUSE Linux
Enterprise Server, and Windows Server as the SAP
platform.
CIBER uses IBM System x3650 M2 machines powered
by 2.26 GHz quad-core Intel Xeon Processor L5520
series processors. These x3650 M2 servers provide
infrastructure services, running Novell SUSE Linux
Enterprise Server 10 and 11, as well as Microsoft
Windows 2003 and 2008. These servers run without
virtualization, and monitor the virtualized systems with
applications such as IBM Director, cloud monitoring
tools, and SAP monitoring applications.
IBM Power Systems
CIBER uses two IBM Power 560 and two IBM Power 750
servers, running the IBM AIX operating system and IBM
DB2 for Linux, UNIX and Windows. Both Power 750
systems are equipped with four POWER7 processors
with 8 cores each and 512 GB of memory. The IBM Power
560 servers are equipped with two POWER6 processors
with 4 cores each and 128 GB of memory. The servers
each host around 40 logical partitions (LPARs).
Ultimately, however, CIBER’s clients select the platform: either
Power Systems and AIX or System x and Linux or Windows,
depending on the combination of reliability, scalability and
predicted workload.
Technical solution
9
IBM BladeCenter and System x landscapes
For many customers, CIBER provides SAP hosting services on
IBM System x BladeServers.
For systems powered by Intel processors, CIBER follows a rule,
based on experience: There is an optimum relationship between
processor capacity consumed and memory required, which
means that more memory does not necessarily increase the
performance of the SAP applications hosted on that server.
CIBER found that 8 GB memory per core is sufficient. In the
future, CIBER may upgrade the blades with IBM BladeServer
HX5 technology.
Provisioning of new SAP instances is completed using VMware
templates, making it fast and easy to set up standardized
systems ready for immediate customer deployment. Additional
tools such as TSM backup agents, monitoring agents, and
special SAP configurations are already integrated for quick
deployment.
CIBER uses VMware vCenter Converter (also named
vConverter) to automate physical to virtual machine conversions
as well as conversions between virtual machine formats. In
particular, the converter is used to migrate native physical
systems into CIBER’s virtual environments.
Virtualization using VMware in the Intel processor-based systems
Almost all applications, perhaps 99.9 percent, on the Intel
processor-based platforms are virtualized using VMware
vSphere 4. The only exceptions are monitoring and systems
management applications.
The Blade Servers host guest operating systems such as Novell
SUSE Linux Enterprise Server 10 and 11, as well as Microsoft
Windows Server 2003 and 2008.
For the System x servers, CIBER recommends the Linux
operating system to customers for the following reasons:
Lower license costs•
Linux installation can be configured to comprise just the •
software components needed, creating a low-footprint
operating system with relatively low maintenance
requirements (Windows includes many software
components not actually required by the applications,
increasing the operating system footprint and increasing the
patch requirements).
CIBER believes that Linux Logical Volume Manager (LVM) is •
more flexible than the Microsoft Logical Disk Manager; for
example, Linux LVM allows data replication (mirroring) at
server level
RESOURCE POOL
<<< ESX HA / VMWare Vmotion >>>
…
SVC MDISK
SAP ERP P
SAP SolMan D
SAP BI DEV
SAP CRM
Mirrored vDisksProductive
1 TB
Direct Mappingnon productive
2.5 TB
…
SVC MDISK
SAP ..
SAP NW QA
SAP EP DEV
SAP ERP
Direct Mappingnon productive
2.5 TB
ESX ESX ESX ESXESXESX ESX ESX ESXESX
SAP SAP SAP SAP SAP SAP SAP SAPNonSAP
SAP SAP SAP SAP SAP SAP SAPNonSAP
SAP SAP SAP SAP SAP SAP SAP SAPNonSAP
SAP SAP SAP SAP SAP SAP SAP SAP
SAP SAP SAP SAP SAP SAP SAP SAP
SAP SAP SAP SAP SAP SAP SAP SAP
Linux/DB2ERP 6.0
Intel/DB2Win/OracleERP 6.0
Intel/DB2Win/MaxDBERP 6.0
NonSAP
NonSAP
Win/MS SQLERP 6.0
PowerVM LPAR LPAR LPAR LPARLPARLPAR LPAR LPAR LPARLPAR
Data center 1 Data center 2
Figure 3: High Availability Concept and storage cloud operating system
10
The conversion takes place either at the customer site or in
CIBER’s central data center. CIBER uses Acronis True Image to
create a backup image of the customer’s application systems
environment and transport the image files from the client’s site to
the data center, which eliminates the need for risky physical
movement of hardware from place to place.
Once customer systems have been virtualized, CIBER uses
VMware Virtual Symmetric Multi-Processing (SMP) to enable a
single virtual machine to use multiple physical processors
simultaneously. This allows CIBER to maximize utilization of its
physical processor resources, and provide the virtual machines
with the capacity required easily and quickly.
With the applications and servers hosted by CIBER, the next
question is how to deliver them to the customers. VMware View is
built on VMware vSphere for Desktops. CIBER runs its own data
center management and administration applications using
VMware View virtualized desktops.
For data storage, security and management, CIBER uses
VMware vStorage VMFS (virtual machine storage file system), a
high-performance cluster file system optimized for VMware
virtual machines. A VMware virtual machine stores the contents
of a virtual machine’s hard disk drive in a virtual disk file with the
extension .vmdk (virtual machine disk). The vmdk specification
describes and documents the virtual machine environment, and
how the contents of the hard disk drives is stored. CIBER uses
.vmdk files rather than storing the virtual disks on raw devices to
ensure isolation of the data stored in separate virtual disks.
VMware Storage vMotion enables Live Migration of .vmdk files –
in other words, a .vmdk file can be moved from one location to
another even while it is in use. This ability helps CIBER to
minimize downtime during disk array migrations and storage
upgrades, or when the virtual machines require larger-capacity
storage devices.
For disaster tolerance and disaster recovery capabilities, as well
as for high availability across locations, CIBER implemented an
enterprise-class storage design. It consists mainly of the IBM
SAN Volume Controller (SVC) and the IBM XIV Storage System.
The SVC has been set up with the Split I/O Group design to
achieve high availability even if one location fails completely.
In 2011, CIBER plans to deploy VMware vCloud Director data
center management software, which pools virtual infrastructure
resources and delivers them to users as a catalog-based
service. The target users are internal consultants who administer
the local data center systems.
CIBER also plans to implement VMware vCenter Chargeback,
which logs resource requests and consumption and enables
usage-based chargeback and accounting for hosted data
center operations.
Figure 4: Providing and consuming IT as a service
VMware vCloud Director
Customer 1
VMwarevShield
VMwarevCenter Server
VMware vSphere
VMwarevCenter Server
VMware vSphere
CIBER Solution Suite
Datacenter Configuration (Silver)Datacenter Configuration(Gold)
SecurityCatalogs
Linux/SQLBI
Linux/ DB2BS7
Win/MaxDBERP 6.0
11
IBM Power Systems landscapes
Typically, for larger customers CIBER hosts SAP ERP and SAP
NetWeaver applications on IBM Power systems. For existing
clients, CIBER has two IBM Power 560 and two IBM Power 750
servers, running the IBM AIX operating system and IBM DB2 for
Linux, UNIX and Windows.
LPARs are subsets of a computer’s hardware resources,
virtualized as a separate computer using the hypervisor. A
physical machine can be partitioned into multiple LPARs, each
hosting a separate AIX operating system. The hypervisor
assigns a specific range of virtual processors to each of the
LPARs. CIBER does not configure dedicated assignments of
processors, and shares uncapped processor resources across
all the LPARs.
The VIOS provides full virtual paths from each LPAR to the
switched fabric in the data center. Switched fabric network
topology allows nodes to connect with each other via one or
more network switches, which provides both high total capacity
and a high degree of resilience if a network switch fails.
CIBER has assigned a separate LPAR for storage management,
with a dedicated Converged Network Adapter to support
concurrent LAN (TCP/IP) and SAN (FCoE, iSCSI) traffic over a
shared 10 Gb Ethernet link. CIBER currently uses only TCP/IP
and iSCSI protocols, and wants to be ready for FCoE in the
future.
PowerHA for AIX (previously named High Availability Cluster
Multiprocessing HACMP) provides high availability of logical
partitions (LPARs) in a two-node cluster by way of automatic
failover. When one server node goes down, the other server
node takes over. Depending on clients’ requirements, most
production systems and some of the quality assurance systems
running on AIX use PowerHA to ensure continuous availability.
CIBER uses IBM Virtual I/O Server (VIOS) to enable the sharing
of physical I/O resources between client logical partitions within
the server. The total physical bandwidth can be allocated in fixed
amounts to each server, or policies set to enable full capacity to
be used by high-priority workload at peak times.
For large SAP systems, CIBER prefers to use POWER
technology and virtualization over VMware vSphere. This is for
two reasons: first, because AIX LPARs can scale to higher total
processor and memory limits; and second, because AIX
enables much higher processor utilization. By using AIX LPARs,
the SAP application can grow much larger before it is necessary
to scale-out over multiple servers.
The migration from IBM POWER5 to POWER6 and POWER7 was
without any problems, CIBER simply connected the new servers
to the SAN storage system and restarted the new servers and the
SAN storage system.
12
Storage architecture
CIBER implemented an enterprise-class storage solution, based
on two IBM XIV enterprise storage systems, IBM SAN Volume
Controller (SVC) as Split I/O Group, IBM System Storage DS3400
for the SVC quorum disk, and IBM Tivoli Storage Manager. The
storage infrastructure has been designed to provide a
combination of performance and resilience, as well as the ability
to allocate storage volumes to multiple customers exactly as their
needs dictate with the least possible administration and
expense. CIBER evaluated different solutions and chose XIV
technology because of better IO performance, improved
administrative handling, and easier maintenance support than
its competitors.
At the core of the storage landscape are two IBM XIV model A14
storage systems, each equipped with 79 TB storage. The XIV
systems act as the principal SAP application data storage.
CIBER can handle almost three-fold storage growth as customer
data volumes grow. In practice, Tivoli Storage Manager enables
sophisticated management of data, moving archive and less
active data onto tape, to help reduce the need to expand disk
systems.
CIBER implemented the two XIV systems, located at the primary
and secondary data centers. XIV technology offers high,
consistent performance and hotspot-free service due to mass
parallelism and the distribution of logical volumes across all hard
disks in the system, eliminating distribution planning.
POWER SUPPLY 4
POWER SUPPLY 3
POWER SUPPLY 2
POWER SUPPLY 1
10 9 8 7 6 5 4 3 2 1
POWERSUPPLY4+2POWERSUPPLY3+1
0
1
2
3
24
25
26
27
4
5
6
7
2 8
2 9
30
31
8
9
10
11
32
33
34
35
12
13
1 4
1 5
3 6
3 7
38
39
16
17
18
19
40
41
42
43
20
21
22
23
44
4 5
4 6
4 7
System Storage
D
POWER SUPPLY 4
POWER SUPPLY 3
POWER SUPPLY 2
POWER SUPPLY 1
10987654321
POWER SUPPLY 4 + 2 POWER SUPPLY 3 + 1
01
23
2425
2627
45
67
2 829
3031
89
1011
3 233
3435
1 213
1415
3637
3839
1617
1 81 9
404 1
4 243
2021
2223
444 5
4647
POWER SUPPLY 4
POWER SUPPLY 3
POWER SUPPLY 2
POWER SUPPLY 1
10987654321
POWER SUPPLY 4 + 2 POWER SUPPLY 3 + 1
01
23
2425
2627
45
67
2 829
3031
89
1011
3 233
3435
1 213
1415
3637
3839
1617
1 81 9
404 1
4 243
2021
2223
444 5
4647
System Storage
TotalStorageStorageEngine336
POWER SUPPLY 4
POWER SUPPLY 3
POWER SUPPLY 2
POWER SUPPLY 1
10 9 8 7 6 5 4 3 2 1
POWERSUPPLY4+2POWERSUPPLY3+1
0
1
2
3
24
25
26
27
4
5
6
7
28
29
30
3 1
8
9
10
1 1
3 2
3 3
3 4
3 5
12
13
14
15
36
37
38
39
16
1 7
1 8
19
40
41
4 2
43
20
2 1
22
23
44
45
46
4 7
System Storage
41
85
129
DS3400
1923
1822
1721
1620
1115
913
1014
812
37
15
26
40
2498-B24
1923
1822
1721
1620
1115
913
1014
812
37
15
26
40
2498-B24
1923
1822
1721
1620
1115
913
1014
812
37
15
26
40
2498-B24
1923
1822
1721
1620
1115
913
1014
812
37
15
26
40
2498-B24
19 2318 2217 2116 2011 159 13 10 148 123 71 5 2 6402498-B24
TotalStorage Storage Engine 336
System x3650 M21 2
3 4
0 1 2 3 4 5 6 7 8 9 10 11
System x3650 M21 2
3 4
0 1 2 3 4 5 6 7 8 9 10 11
System x3650 M21 2
3 4
0 1 2 3 4 5 6 7 8 9 10 11
System x3650 M21 2
3 4
0 1 2 3 4 5 6 7 8 9 10 11
System x3650 M21 2
3 4
0 1 2 3 4 5 6 7 8 9 10 11
System x3650 M21 2
3 4
0 1 2 3 4 5 6 7 8 9 10 11
35 3934 3833 3732 36
2498-B40
27 3126 3025 2924 2819 2318 2217 2116 2011 1510 149 138 123 762510 4 35 3934 3833 3732 36
2498-B40
27 3126 3025 2924 2819 2318 2217 2116 2011 1510 149 138 123 762510 4
Bladecenter H
Infrastrukturserver
SVC Node
SVC Node
Quorum
Dark Fiber
Dark Fiber
FC CWDM
`STAT
CONSOLE
L1
L2
MGMT 0
MGMT 1 4 8 12 16
3 7 11 15
2 6 10 14
1 5 9 13
20 24 28 32
19 23 27 31
18 22 26 30
17 21 25 29
SLO
T2
34 38
33 37
36 40
35 39
SLO
T3
Cisco Nexus 5020
PS
1P
S2
200-240v-6A50~60Hz
`STAT
CONSOLE
L1
L2
MGMT 0
MGMT 1 4 8 12 16
3 7 11 15
2 6 10 14
1 5 9 13
20 24 28 32
19 23 27 31
18 22 26 30
17 21 25 29
SLO
T2
34 38
33 37
36 40
35 39
SLO
T3
Cisco Nexus 5020
PS
1P
S2
200-240v-6A50~60Hz
`STAT
CONSOLE
L1
L2
MGMT 0
MGMT 1 4 8 12 16
3 7 11 15
2 6 10 14
1 5 9 13
20 24 28 32
19 23 27 31
18 22 26 30
17 21 25 29
SLO
T2
34 38
33 37
36 40
35 39
SLO
T3
Cisco Nexus 5020
PS
1P
S2
200-240v-6A50~60Hz
`STAT
CONSOLE
L1
L2
MGMT 0
MGMT 1 4 8 12 16
3 7 11 15
2 6 10 14
1 5 9 13
20 24 28 32
19 23 27 31
18 22 26 30
17 21 25 29
SLO
T2
34 38
33 37
36 40
35 39
SLO
T3
Cisco Nexus 5020
PS
1P
S2
200-240v-6A50~60Hz
1 2 3 4 5 6 7 8 9 10 11 12 13 14
1 2 3 4 5 6 7 8 9 10 11 12 13 14
BladeCenter H
1 2 3 4 5 6 7 8 9 10 11 12 13 14
1 2 3 4 5 6 7 8 9 10 11 12 13 14
BladeCenter H
1 2 3 4 5 6 7 8 9 10 11 12 13 14
1 2 3 4 5 6 7 8 9 10 11 12 13 14
BladeCenter H
1 2 3 4 5 6 7 8 9 10 11 12 13 14
1 2 3 4 5 6 7 8 9 10 11 12 13 14
BladeCenter H
560
D3 D4 D5 D6 D7 D8 D9 D10
Power 750
Power 750
Infrastrukturserver
560
D3 D4 D5 D6 D7 D8 D9 D10
Power 750
Power 750
Datacenter 1 Datacenter 2
Datacenter 3
10GE LAN Backbone
8 GB FC Backbone
8 GB FC Backbone
10GE LAN Backbone
FC CWDMFC CWDMFC 850nm
FC 850nmXIV Storage 1
Infrastructure Mgmt Bladecenter H
XIV Storage 2
Infrastructure Mgmt
Fiberchannel Single Mode Fiberchannel Multi Mode Dark Fiber Local Area Network
Figure 5: Data center landscape
Working with the service partner Netzwerktechnik Röver, CIBER
implemented a redundant and geographically separated metro
data center connection based on CWDM technology. All storage
mirroring is performed through multiplexed Dark Fiber with a
bandwidth of 60 GBit/s, which can be scaled up to 300 GBit/s.
The XIV system is controlled through an intuitive graphical user
interface (GUI), and a powerful command line interface (CLI) for
complex scripting. Training on XIV systems usage is usually a
matter of hours rather than weeks, saving costs and speeding
time to full production.
13
The very high I/O, reliability and availability performance is
enabled by a grid-based controller and disk architecture with
active-active component redundancy, which enables non-
disruptive hot-swapping of hardware and firmware. XIV systems
provide incremental scaling from partial to full rack, with 27-79
TB usable, and with every capacity increase the performance
scales linearly. Automatic re-distribution of volumes ensures that
the high performance and fast access continues even as data
volumes grow. There is no tuning necessary, and the XIV system
does not need to tune itself during operation.
Using the SVC’s FlashCopy function and XIV point-in-time copy
CIBER clones SAP systems as customers request new
instances. Combined with automated SAP post-processing,
CIBER is able to create a completed cloned SAP instance in
about 30 minutes. XIV Snapshot gives CIBER the ability to
respond to customer demands in ways and at a speed that even
in recent years would have been impossible to achieve.
In selecting XIV systems, a new technology for the CIBER team,
they soon discovered that only three hours’ training was needed
to understand the easy-to-use XIV system’s graphical interface,
compared with older systems that had taken weeks to fully learn
(redundant).
The DS3400 storage system serves two purposes:
As a storage device to host the active quorum disk (tie •
breaker) for the SVC clusters
As a storage pool for the TSM server•
CIBER uses IBM Tivoli Storage Manager, running in a separate
LPAR on the Power servers, to manage backup, archive and
restore. A dedicated physical 10 Gbps Ethernet adapter allows
data transfer to run on fully virtualized end-to-end connections
powered by Nexus switches from Cisco, connecting to the IBM
System Storage TS3500 tape library. This arrangement offers
very fast system backup and restore processes while ensuring
that the data traffic does not impact production systems.
The storage landscape is completed with two IBM System
Storage TS3500 tape library systems, each with five LTO-4
drives, 260 slots and about 200 cartridges. These systems
provide a data transfer speed of 350 GB per hour.
Database architecture
For CIBER, DB2 for Linux, UNIX and Windows is the most
appropriate database platform to support the SAP applications.
Small and medium-sized businesses tend to underestimate their
data growth, and easily reach sizes where DB2 offers significant
advantages, such as Deep Compression that helps to reduce
the database size and data growth.
DB2 also offers close integration with Tivoli Storage Manager,
providing a comprehensive backup solution that protects DB2
and related data reliably and cost-effectively.
The new index compression feature within DB2 is the next step to
improve database performance and reduce storage costs. In
addition, compression for temporary tables for SAP systems is
activated ‘under the cover’ to reduce disk costs. DB2, compared
with other database compression algorithms, can handle data
and indexing more efficiently. CIBER experienced a
compression rate in the high double-digit area, and
performance improvements of 60 percent with DB2.
Besides performance, DB2 also makes administration tasks
easier and faster by offering a high level of automation with Self
Tuning Memory Management, Automatic Storage, automatic
runstats, and Real Time Statistics.
For example, CIBER uses the autoresize functionality so that
tablespaces can enlarge themselves if needed, while the
Container Management is completed manually to preserve a
certain degree of control over the data management by the
database administrators.
14
Automatic reorganization offers time-saving advantages for
database administrators, because it checks if a reorganization of
data and indices is needed and if so, the Reorg utility is
automatically called and reorganizes the data and indices in the
background. CIBER also plans to use the reclaimable storage
feature for its SAP NetWeaver Business Warehouse to reduce
the tablespaces from about 1 TB down to 300 MB
With the new tablespace option of reclaimable storage,
introduced in DB2 version 9.7, it is now much easier to reduce
the size of the tablespace and return unused space to the file
system, including unused space below the High-Water Mark
(HWM).
DB2 Migration: Opportunities for optimization
It is possible to activate compression features during a SAP
migration project to DB2, which eliminates the need for
subsequent compression tasks.
To do so, CIBER used the “Sampled” option, which comes with
the R3Load Version 7. This option causes every ‘N’th row of the
target table to be loaded first, followed by a reorg to create the
Compression Dictionary based on the data loaded. The count of
data used for the sampling process is provided by the
environment variables DB6LOAD_SAMPLING_FREQUENCY
and DB6LOAD_MAX_SAMPLE_SIZE. Subsequently, R3load is
restarted to load the rest of the table in a compressed format.
Transition to Unicode
If Unicode is required on the target solution, then recommended
practice is to perform the DB2 migration in conjunction with an
SAP software upgrade and/or Unicode conversion in a single
project. This combination eliminates redundant project tasks,
keeping the cost of the migration as low as possible. Around 90
percent of the Unicode conversion steps are also part of the DB2
migration process. By combining the two processes, both costs
and workload can be reduced, eliminating duplication of effort.
Automatic provisioning of SAP with Cloud services
CIBER’s ultimate goal is to deliver Cloud services to its
customers, which means being able to provision, upgrade and
clone SAP instances very quickly using automated systems.
Defining customer requirements will become an interactive, face
to face activity, and once requirements have been defined, the
SAP landscapes will be provisioned automatically.
The Service Request Tool is the “CIBER Solution Suite”, which
CIBER developed in-house, based on SAP Solution Manager.
The backend (workflows) were developed in ABAP, while CIBER
uses Adobe Flex™ (Fx) as GUI front end. This tool enables the
CIBER SAP Consultant simply to select the correct SAP instance
template, and then kick off the automated provisioning.
CIBER used the standard OS tools, such as NIM & NIS (Network
Installation Manager & Services), to install the operating system.
To clone SAP instances, CIBER uses XIV point-in-time copy.
CIBER clones (snapshots) the entire SAP system, including EXE,
DATA, and LOG files, while keeping the SID. The new clone is
started in its own VLAN segment and will be automatically
reconfigured and customized. This ensures that the new clone
with the same SID does not interfere with the existing SAP
landscape.
Because CIBER does not need to install the SAP instance, there
are no pre- or post-processing steps, and a new clone can
therefore be provisioned and made available within 30 minutes.
“IBM has brought in major contribution to the quick success of CIBER – to develop such a high customer rate within such a short timeframe.”
Stefan Rother Lead Infrastructure Services CIBER Managed Services GmbH
15
Performance improvements
The new SAP infrastructure allows CIBER to improve efficiency
for its SAP managed services business significantly by offering a
solution that scales in performance and capacity, while
providing a very cost-effective model.
The virtualization capabilities of the storage and server
infrastructure allow CIBER to offer customers a very flexible
service with real end-to-end virtualization, able to scale up
quickly in times of peak workload and back again as required.
For example, being able to clone a SAP system inside 30
minutes, or to activate another 1 TB of storage on the XIV system
within an hour, mean that CIBER can cope with almost any
customer demand. The result is that CIBER can offer “pay as you
grow” SAP hosting as software as a service (SaaS) backed by a
powerful cloud infrastructure that encourages customers to
make the leap without having to predict workload or commit to
specific infrastructure spending.
The combined performance improvements available from the
IBM Power servers, IBM System x BladeServers, and IBM
storage infrastructure have led to increased user satisfaction.
Many customers have noticed greatly improved response times
for online operation of the DB2 database – in some cases
recording an eight-fold improvement. The enhanced response
times contribute to increased user productivity, and help build
the case for using hosted SAP applications rather than in-house,
as CIBER is able to make the kind of technology investment that
might be out of reach of a single customer.
CIBER plans to take advantage of Active Memory Expansion,
available on Power servers with POWER7 processors and
upwards, which employs memory compression technology to
transparently compress in-memory data, allowing more data to
be placed into memory and thus expanding the memory
capacity of POWER7 systems. Utilizing Active Memory
Expansion can improve system utilization and increase a
system’s throughput.
CIBER plans to roll out IBM DB2 Deep Compression and Row
Compression features on all of its systems, and deploy IBM Tivoli
Provisioning Manager and IBM Tivoli FlashCopy to enhance
data security and performance.
For improved data center management and new services,
CIBER plans to apply VMware vCloud Director in 2011. vCloud
Director is a data center management software, which pools
virtual infrastructure resources and delivers them to users as a
catalog-based service. For cost models that are customized for
accounting of resource requests and consumption, CIBER will
implement VMware vCenter Chargeback.
Project achievements
“We recommend the DB2 database for new customers who start with SAP, because DB2 is very cost-effective and brings performance benefits and DB2 and SAP simply fit together.”
Holger Stasch SAP Senior Consultant CIBER Managed Services GmbH
Authors
Markus Fehling, IBM, [email protected]
Elke Hartmann-Bakan, IBM, [email protected]
Frank Neumann, IBM, [email protected]
Carsten Siegler, IBM, [email protected]
Hans-Achim Muscate, VMware, [email protected]
Markus Peltz, CIBER, [email protected]
For more information
To learn more about the solutions from IBM and SAP, visit:
ibm-sap.com
For more information about SAP products and services, contact
an SAP representative or visit: sap.com
For more information about IBM products and services, contact
an IBM representative or visit: ibm.com
For more information about CIBER products and services,
contact a CIBER representative or visit: ciber.com
Contacts
Markus Fehling, [email protected]
Markus Peltz, [email protected]
For feedback and further questions please contact the IBM SAP
International Competency Center via [email protected]
SPC03307-DEEN-00 (December 2010)
© Copyright IBM Corp. 2010 All Rights Reserved.
IBM Deutschland GmbH D – 71137 Ehningen ibm.com
Produced in Germany IBM, the IBM logo, ibm.com, i5/OS, DB2, Domino, FlashCopy, Lotus, Notes, POWER, POWER4, POWER5, POWER6, System i, System x, and Tivoli are trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of other IBM trademarks is available on the Web at: http://www.ibm.com/legal/copytrade.shtml
UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product or service names may be trademarks, or service marks of others.
This brochure illustrates how IBM customers may be using IBM and/or IBM Business Partner technologies/services. Many factors have contributed to the results and benefits described. IBM does not guarantee comparable results. All information contained herein was provided by the featured customer/s and/or IBM Business Partner/s. IBM does not attest to its accuracy. All customer examples cited represent how some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.
This publication is for general guidance only. Photographs may show design models.