high availability and scalability - osos.itec.kit.edu/downloads_own/teach_wt1011_systemz_lecture/kit...
TRANSCRIPT
© 2010 IBM Corporation
High Availability and Scalabilitywith System z and z/OS
Joachim von Buttlar, Robert Vaupel
IBM Deutschland Research & Development GmbH
© 2010 IBM Corporation2
Who is Who?
� Joachim von Buttlar– System z Firmware Development– [email protected]
� Robert Vaupel– z/OS Workload Management Development and Design– IBM Senior Technical Staff Member– [email protected]
© 2010 IBM Corporation3
WS 2010/2011: Structure and Content
� CPU Architecture– Register sets– Memory organization– Virtual storage – Interrupt mechanism– Timing facilities– Instruction set– Multiprocessing facilities
� I/O Architecture– I/O infrastructure– Adapter types & channels – Control unit & devices– Extensions for large configurations
� Partitioning and virtualization– LPAR versus z/VM– Differences and commonalities– Hardware facilities– Storage management– Processor management– I/O management
Wrap-Up and Closing11.2.2011
System z ArchitectureIntroduction and Orientation22.10.2010
System z ArchitectureSystem z Architecture5.11.2010
System z ArchitectureSystem z Architecture19.11.2010
z/OS and System z Software Architecturez/OS Workload Management 28.1.2011
z/OS Workload Management
z/OS Dispatching and Virtualization
z/OS Introduction
14:00-15:30
z/OS Parallel Sysplex14.1.2011
z/OS Dispatching and Virtualization17.12.2010
z/OS Introduction3.12.2010
11:30-13:00Date (always Fridays)
� z/OS– Address space concept– Task execution and serialization– Program communication and data exchange– Data formats, data sets and I/O flow – z/OS subsystems: TSO, ISPF, JES
� z/OS Dispatching and Hiperdispatch
� z/OS Workload Management
� Parallel Sysplex– Cluster concepts– Parallel Sysplex structure and exploitation– Data Mirroring and Global Dispersed Parallel Sysplex– Data mirroring
� Middleware Integration and Software Architecture
© 2010 IBM Corporation4
What is System z?
System /360
IBM System z10 EC
IBM z Enterprise z196
A System z server is what businesses use to host the largest commercial databases, transaction
servers, and applications that require a greater degree of security and availability than is commonly found on smaller-scale machines.
A System z server is what businesses use to host the largest commercial databases, transaction
servers, and applications that require a greater degree of security and availability than is commonly found on smaller-scale machines.
© 2010 IBM Corporation5
� S/360 architecture is based on von Neumann‘s computing model:
� S/360 architecture got invented and documented in the S/360 Principles of Operation in 1964 by:
– Gene Amdahl– Fred Brooks– Garry Blaauw
System z Architecture
http://publibz.boulder.ibm.com/epubs/pdf/dz9zr007.pdf
45°
90°
135°
180°
225°
270°
315°
360°
S/360 = 360°
� One hardware architecture
� One operating system
� For all IBM computers
© 2010 IBM Corporation6
System z and z/OS History
1960 1965 1970 1975 1980 1985 1990 1995 2000 2005
7. April 1964Introduction ofS/360 Architecture
Symmetric MultiProcessing
Virtual Memory
ExpandedStorage
2GB AddressingLPAR
AccessRegisters
Data-spaces
CMOSTechnology
ParallelSysplex
64bitAddressing
S/360 S/370 S/390 z Architecture
MVT MVS/370 MVS/XA MVS/ESA OS/390 z/OSSVS
Fixed Storage15 Partitions
or Tasks
Address Spaces
Multiple VirtualStorage
One16MB
VSArea
2 GBVirtual
Storage
ExpandedStorage
Virtual I/OFast
ProgramLoad
Dynamic I/OPosix
Cluster
ParallelSysplex
WorkloadManagementUnix System
ServicesTCP/IP
...
JavaWebsphereIEEE Float
64 bitIRD
HiperdispatchOffloadSecurityGDPS
...
MFT
© 2010 IBM Corporation7
Mainframe Computing
Mainframes are computers which– Execute hundreds of applications– Connect to thousands of I/O devices– And serve thousands of users simultaneously
Mainframes can best be defined by their characteristics
– The most important characteristic is to ensure a reliable and predictable execution of transactions
– The importance of mainframes is for data base transaction processing and as the backend in data centers
© 2010 IBM Corporation8
Economical Importance: Why System z and z/OS
� 95% of the 2000 world-wide biggest companies use System z computers
� Around 65-70% of all relevant data are stored on System z computers
� 60% of all data being access thru the world wide web are stored in databases on System z (DB2, VSAM, and IMS)
� All companies which have the need to store huge amounts of data require
– Security– Scalability– Compatibility– Availability– Reliability– Serviceability
© 2010 IBM Corporation9
High Availability and Scalability
� System z Hardware Overview and Introduction
� System z Usage in Customer Environments
� RAS capabilities
� What does High Availability and Scalability mean?
© 2010 IBM Corporation10
Elements of System z Architecture
� Central Processor Units– Up to 64 PUs
� Main Memory– Byte-wise addressable– 64-bit addressability– 'shared' between all CPU‘s
� I/O Subsystem– ‘old': parallel (copper), 4.5 MB/sec– 1990: serial (fiber), 17 MB/sec– 1999: FiCON (fiber), 270 MB/sec
� ESCON & FiCON 'Director'– Switch
� Control Units (CU)– Managing unit
� Devices– Hard disk, tape, printer, etc.
� Network (GbE, ...)
© 2010 IBM Corporation11
Heart of System z Architecture: MCM
� MCM = Multi Chip Module– Processor Units (PU), Storage Controller (SC), SEEPROM (S) and clock functions– Integration increases with each generation, example:
• z9: 8 PUs per MCM with up to 2 cores• z10: 5 PUs per MCM with up to 4 cores• z196: 6 Pus per MCM with up to 4 cores
� A single MCM can provide 24 processors on a z196 but a z196 can have up to 96 processors (80 usable for workloads)
© 2010 IBM Corporation12
z196 PU chip, SC chip and MCM
© 2010 IBM Corporation13
z196 Book LayoutMCM @ 1800W
Refrigeration Cooled orWater Cooled
Backup Air Plenum
8 I/O FAN OUT 2 FSP
3x DCA 14X DIMMs100mm High
16X DIMMs100mm High
11 VTM Card Assemblies8 Vertical3 Horizontal
RearFront
DCA Power Supplies
Fanout
Cards
Coolingfrom/to MRU
MCM
Memory
Memory
© 2010 IBM Corporation14
z196 Water cooledUnder the covers (Model M66 or M80) front view
© 2010 IBM Corporation15
z196 Frames
� On z196: Traditional System z Operating Systems: z/OS, Linux, zVSE, zVM
� On z196 Blade Extensions: Power 7 Blades, System x Blades
� Integration via Unified Resource Manager
© 2010 IBM Corporation16
Growth of System z Servers
� Growth encompasses– Speed: from z900 (770MHz) to z196
(5.2 GHz)– Integration of processors and chips on
same MCM– Number of MCMs per system– And now with z196
• Integration of Blade Server
z/OS release used for LSPR measurements
z196 measurements are for a xx-way
PCI - (Processor Capacity Index
© 2010 IBM Corporation17
A typical System z could look like this
z/VM V4
CMS
Linux
CMS
Linux
LPAR
Linux
VSE
z/OS
z/VSE
z/OS
z/VM
Linux
Linux
LPAR LPAR LPAR
Linux
z/VSE
CP1 CP2 CP3 CP4 IFL1 IFL2 IFL3
System z Enterprise Server
Java
z/VM
Linux
LPAR
Standard Processors
LPAR
Linux
z/VSE
zIIP
DB2
Offload Engines Linux Engines
C ICS
IMS
SAP
zAAP
Linux
Linux
Linux
C ICS
Batch
© 2010 IBM Corporation18
System z Processor Characterization� Central Processor (CP)
– Provides processing capacity for z/Architecture and ESA/390 instruction sets– Runs z/OS, z/VM, z/VSE, z/TPF, Linux for System z
� System Assist Processor (SAP)– SAPs manage the start and ending of I/O operations for all LPARs and all attached I/O– Each machine has at least one SAP
� Internal Coupling Facility (ICF, since 1997)– Provides additional processing capacity for the execution of the Coupling Facility Control Code (CFCC) in a CF LPAR
� Integrated Facility for Linux (IFL, since 2001)– Provides additional processing capacity for Linux workloads
� IBM System z Application Assist Processors (zAAP, since 2004)– Provides additional processing capacity for Java workloads under z/OS
� IBM System z Information Integration Processors (zIIP, since 2006)– Provides additional processing capacity for certain DB2 workloads under z/OS
� Spares– Provides extra processing capacity in case of any failure of any PU
� SAP, ICF, IFL, zAAP, zIIP offer the same functional ity as CPs� Lower price than CP� Do not affect traditional System z software charges
© 2010 IBM Corporation19
Why is System z different?
� Many different types of workloads
� Business Critical workloads
� Running systems at very high utilizations
� Access to systems is always required
© 2010 IBM Corporation20
System z Quality of Services
� RAS – Reliability– Availability– Serviceability
� Security / Integrity
� Scalability
� Manageability– Centralized control– Workload management
� Virtualization / Partitioning Technology – Workload separation
� Capacity– Evolving architecture
� Flexibility / Variety– Multiple workloads, multiple users
� Compatibility
� Capability– Autonomic features
© 2010 IBM Corporation21
System z: RAS Design Focus
� High Availability (HA)– The attribute of a system designed to provide service during defined periods, at acceptable or
agreed upon levels and masks UNPLANNED OUTAGES from end-users. It employs fault tolerance, automated failure detection, recovery, bypass reconfiguration, testing, problem and change management.
� Continuous Operations (CO)– Attribute of a system designed to continuously operate and mask PLANNED OUTAGES from
end-users. It employs non-disruptive hardware and software changes, non-disruptive configuration, software coexistence.
� Continuous Availability (CA)– Attribute of a system designed to deliver non-disruptive service to the end user 7 days a week,
24 HOURS A DAY (there are no planned or unplanned outages). It includes the ability to recover from a site disaster by switching computing to a second site.
Continuous
Availability
HighAvailability
ContinuousOperations
© 2010 IBM Corporation22 Template Documentation9/30/2010
Unplanned Outage Causes
25%
30%45%
Operator Errors
Application FailuresHardware
Failures
IDC 2005
Business Issue of “Non-Availability“
E.g. “Toll Collect”: The state of Germany and the company collecting toll on the autobahn agreed on a contractual penalty of €30 Million for each 1 hour of down time (represents €500.000 / min).
� On demand challenges– Downtime unaffordable– Heterogeneous by nature– Complex to manage
� Loss of business� Loss of customers – the
competition is just a mouse click away
� Loss of credibility, brand image and stock value
© 2010 IBM Corporation23
Continuous Availability / Disaster Recovery
121
2
34
5678
9
1011 12
12
34
56
789
1011
121
2
34
56
78
9
10
11
Single System Parallel Sysplex Geographical Dispersed PS
1 to 32 Systems Site 1 Site 2
• MTBF – in decades
• Built-In redundancy
• On/Off Capacity on Demand
• Capacity Backup
• Hot pluggable I/O
• Addresses planned and unplanned HW/SW outages
• Flexible, non-disruptive growth
• Capacity beyond largest CEC
• Scales better than SMPs
• Dynamic workload / resource management
• Addresses site failure / maintenance
• Metro / Global data mirroring
• Sync (PPRC) – 100 km
• Async (XRC) – any distance
• Eliminates tape / disk Single Point of Failure (SPOF)
• No / Some data loss
• Application independent
• Using an ICF, a single CEC (Central Electronic Complex) Parallel Sysplex can be defined• Maintenance on LPAR without loss of data• Protection from software outages
Clustering in a Box
© 2010 IBM Corporation24
Scalability
� Scale-up Example for System z9
� Allows installations to choose the capacity they need in a granular fashion and to grow when business needs require it
0
100
200
300
400
500
600
1-way 2-way 3-way 4-way 5-way 6-way 7-way 8-way
4xx
5xx
6xx
7xx
4xx 5xx 6xx 7xx
Model S08
© 2010 IBM Corporation25
What does the Course Encompass?
� How High Availability and Scalability is implemented on System z
– System z Technology and Hardware– Operating System (z/OS) and Partitioning Technology
• Focus: Dispatching– Cluster Technology to achieve Continuous Availability
• Parallel Sysplex– Capability to execute many different workloads at the
same time and meet business objectives • Workload Management
– Integration of Software, Operating System and Hardware
© 2010 IBM Corporation26
What to Remember?
� What technology steps have been invented to reach high availability and scalability
� Why a technology is exploited in System z
� On a high level– How software, operating system and hardware
work together– And why do they work together
© 2010 IBM Corporation27
© 2010 IBM Corporation28
Literature
� Introduction to the New Mainframe: Large-Scale Commercial Computing– http://www.redbooks.ibm.com/abstracts/sg247175.html?Open
� ABCs of z/OS System Programming Volume 11,– http://www.redbooks.ibm.com/abstracts/sg246327.html
� Documents for Workload Management– http://www-03.ibm.com/servers/eserver/zseries/zos/wlm/documents/
• z/OS Workload Manager: How It Works and How To Use It, April 2004 – http://www.research.ibm.com/journal/sj/362/aman.html
• Adaptive algorithms for managing a distributed data processing workload
� Das Betriebssystem z/OS und zSeries, M.Teuffel, R.Vaupel, ISBN 3-486-27528-3
© 2010 IBM Corporation29
The following are trademarks of the International B usiness Machines Corporation in the United States a nd/or other countries.
The following are trademarks or registered trademar ks of other companies.
* Registered trademarks of IBM Corporation
* All other products may be trademarks or registered trademarks of their respective companies.
Java and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countriesLinux is a registered trademark of Linus Torvalds in the United States, other countries, or both.UNIX is a registered trademark of The Open Group in the United States and other countries.Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation.Red Hat, the Red Hat "Shadow Man" logo, and all Red Hat-based trademarks and logos are trademarks or registered trademarks of Red Hat, Inc., in the United States and other countries.SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC.
Notes : Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.
APPN*CICS*DB2*DB2 ConnectDirMainte-business logo*ECKDEnterprise Storage Server*ESCON*FICON*GDPS*Geographically Dispersed Parallel Sysplex
HiperSocketsHyperSwapIBM*IBM eServerIBM e(logo)server* IBM logo*IMSLanguage Environment*MQSeries*Multiprise*NetView*On demand business logo
OS/390*Parallel Sysplex*PR/SMProcessor Resource/Systems ManagerRACF*Resource LinkRMFS/390*Sysplex Timer*System z9TotalStorage*Virtualization Engine
VM/ESA*VSE/ESAVTAM*WebSphere*z/Architecturez/OS*z/VM*z/VSEzSeries*
Trademarks