life in the fast lane: zhyperlink · 2020. 11. 30. · • vsam reads ++apars • db2 log writes...

Life in the fast lane: zHyperLinkJohn Baker

IntelliMagic

November 2020

Session 4AO

https://twitter.com/gseukc

https://www.linkedin.com/groups/1741877

https://www.facebook.com/pages/Guide-Share-Europe-UK-Region/174907039241415

IntelliMagic zAcademy Availability Intelligence

• Overview

• Traditional vs Synchronous I/O

• “Sync” in context

• Connectivity and DRP implications

• zHyperLink eligibility and real-world results

• Q&A

Agenda

2

https://lp.intellimagic.com/zacademy/

https://www.intellimagic.com/


Existing I/O components

3

CPU

IOP

Channels

FCD ports

Cache and NVS

HA’s and

ports

DA’s and

Drives




The life of an I/O (host side)

4

Open dataset via JCL andRead or write logical record

Builds channel program describingData location and type of access

May return immediately if in buffer

Handles priorities, queuing (IOSQ/PAV)and issues SSCH to engage

CSS hardware

SAP selects path to Disk Subsystem,Channel executes channel program

and handles data flow(Pend: CSS/CMR)

Application Program

Access Method

I/O Supervisor

Channel Subsystem

Disk Subsystem




The life of an I/O (DSS side)

5

Disk Subsystem

Hard Disk/Flash Drives

Channel Subsystem - Decodes channel program (CKD/FB)- Manages cache and NVS- Manages extent integrity (Pend DB)- Initiates staging/destaging (Disc)- Manages RAID structures- Replication (Disc)

- Organized in Ranks/Pools/Arrays- Read/write data- Error checking




How long does all that take?

6




Synchronous I/O

7

Application Program

Access Method

I/O Supervisor

PCIe zHyperLink

• Channel subsystem is bypassed

• CPU waits for PCI/DMA operation to complete; no task switching

• Processor cache content is preserved




Response vs Response

8

mic

roseconds

zHyperLink response




Sync Operation – Determined by z/OS

• Eligible I/Os are tried synchronously• CP spins waiting for sync I/O to complete• “Heritage” (traditional) I/O if cache miss – zHyperLink does not replace FICON




“Sync I/O” – Db2 vs zHyperLink

10

Making the best of a bad situation:Turning a (bad) Db2 sync I/O into a (good) zHyperLink sync I/O

• Occurs when zHL eligible I/O experiences cache hit

• Reduces time required to complete I/O

• One 4K “block” (currently)

• Seek to maximize

zHyperLink sync I/O – GoodDb2 read sync I/O – Bad

• Occurs for data not loaded in advance by prefetch I/O

• Causes “costly” application wait times

• One 4K “page”

• Seek to minimize




Context: CF vs zHyperLink

11

Familiarity with Coupling Facility concepts provides a great frameworkfor understanding zHyperLink processing

• Very fast: 20-30 mics

• CP spins during sync I/Os

• Safeguard from high CPU –cache misses re-driven as (async) FICON I/Os

zHyperLinkCoupling Facility

• Ultra-fast: 5-8 mics

• CP spins during sync reqs

• Safeguard from high CPU –heuristic algorithm “converts” subsequent requests to async




Connectivity – CEC to Storage

FICON links

At least 2 zHyperLinks

Maximum distance:150 meters

8 GB/sec optical links

DS8880Z14/15

zHyperLink is always Point-to-Point – no directors

zHyperLink is PCI link, not channel




No Metro Distance Mirrors

Common replication architecture for financial institutions

These synchronous distances not supported with zHyperLink




zHyperLink for Writes

To exploit zHyperLink for writes, synchronous copy must be local using zHyperWrite (prerequisite)

• Storage frame driven synchronous replication is not supported

• HyperSwap is supported (within 150m)

• Asynchronous copy is supported in combination with zHyperLink

Replication approach may need to be reevaluated




Potential Read-Only Approach

• IF zHyperLink exploitation is limited to reads, AND

• IF zHL-eligible reads all occur on DS8880 direct connected to CECs within 150m

• THEN current replication design may be viable; writes still occur through FICON

Primary:150 m

from CECs

Primary:150 m

from CECs



© 2019 IBM Corporation

IBM Z – WSC Performance Team

16

zHyperLink ™ Eligibility and Enablement– Eligibility designed to show potential benefit, even before zHyperLink™ Enablement

– Maintenance stream captures Eligibility, even if actual function disabled because of• Caller, Lack of Infrastructure, DFSMS Storage Class Granularity by default everything disabled (OA54822)

– Once zHyperLink enabled Eligibility may still be utilized• DFSMS Storage Class Granularity – by default everything “disable”

• Blocksize >4096

– Or Once zHyperLink enabled zHPF may still occur• Even “intra data set” with zHyperLink Enabled – E.g. Db2 Synchronous Reads and Prefetch Reads

• zHyperLink™ Read Misses (not a “Success”)

• Today, Writes

16

Eligibility Callers

• Db2 v12 Sync Reads


• VSAM Reads ++APARs

• Db2 Log Writes

Media Manager

Yes to all ?

• zHyperLink Infrastruture

• Caller Enabled

• DFSMS Enabled

• <=4096 Blocksize

If Not

zHyperLink Enabled - Attempts/Successes,

and Other SMF 42-6 zHyperLink Fields



• VSAM Reads ++APARs

• Db2 Log Writes

zHyperLink Eligible- Eligible / Read Hits



• VSAM Reads ++APARS

• Db2 Log Writes

zHPF – Normal SMF 42-6Non Eligible Callers

SMF 42-6 Fields

This slide courtesy of John Burg, IBM


Tracking eligibility and usage

17




VSAM: FICON and Synchronous Requests

18

zHyperLinkResponse




Synchronous I/O Content (%)

19




zHL Enabled Storage Class“VSAM work”

20




Before and After: Linear Datasets (Db2)

21




Synchronous I/O ContentPercentage of all I/O

22




Detailed Stats from SMF 42

23




Eligible vs Active zHyperLink

24




LR vs SSCH and zHyperLink %

25




Weighted Response

26




DB2 Database: Activity and Response

27




DB2 Transaction Response Time

28




Database Sync I/O Waitms/commit

29




Summary: I/O is all relative!

Local Buffer

Coupling FacilityGroup Buffer Pool

zHyperLink Sync I/O

Storage System Enterprise HDD

FICON Cache Hit

Flash Back-end Miss

Average Tiered Back-end Miss

1 µs

8 µs

20 µs

150 µs / 0.15 ms

1 second

8 seconds

20 seconds

2.5 minutes

8.3 minutes

33.3 minutes

500 µs / 0.5 ms

2000 µs / 2 ms

By way of comparison … if local buffer access

took 1 second …




Sources

• D. Craddock et al. “zHyperLink: Low-latency I/O for Db2 on IBM Z and DS8880 storage”, IBM Journal of Research and Development. Vol 62, No. 2-3, Paper 13, March/May 2018. https://dl.acm.org/doi/10.1147/JRD.2018.2802070

• IntelliMagic White Paper: “zHyperLink: The Holy Grail of Mainframe I/O?”

• Frank Kyne, “Meet the Future of I/O – zHyperLinks”, Cheryl Watson’s Tuning Letter (CWTL) 2018 #2, pp. 86-116

• John Burg (IBM), “Assessing and Analyzing zHyperLink”. SHARE Pittsburgh, August 7, 2019, Session #25708.



https://dl.acm.org/doi/10.1147/JRD.2018.2802070

https://www.intellimagic.com/resources/whitepaper/zhyperlink-holy-grail-mainframe-io/

Please submit your session feedback!

• Do it online at http://conferences.gse.org.uk/2020/feedback/nn

• This session is 4AO

http://conferences.gse.org.uk/2020/feedback/nn

GSE UK Conference 2020 Charity

• The GSE UK Region team hope that you find this presentation and others that follow useful and help to expand your knowledge of z Systems.

• Please consider showing your appreciation by kindly donating a small sum to our charity this year, NHS Charities Together. Follow the link below or scan the QR Code:

http://uk.virginmoneygiving.com/GuideShareEuropeUKRegion

http://uk.virginmoneygiving.com/GuideShareEuropeUKRegion

life in the fast lane: zhyperlink · 2020. 11. 30. · • vsam reads ++apars • db2 log writes...

Documents