ibm z/os v2r2 networking technologies update

ibm.com

www.ibm.com/redbooks

International Technical Support Organization Global Content Services IBM Inside Sales

IBM z/OS V2R2 Networking Technologies Update

Chris Meyer – [email protected] Doris Bunn – [email protected] Howie Odishoo – [email protected]

Mike Fox – [email protected] Pat Brown – [email protected] Todd Valler – [email protected]

mailto:[email protected]�






IBM Inside Sales International Technical Support Organization Global Content Services

© 2015 IBM Corporation ITSO-2

Notices This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product,

program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead.

However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS"

WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some

states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or

changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product

and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems

and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurement may have been estimated through extrapolation. Actual results may vary. Users of this document should verify

the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance,

compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are

fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM,

for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been

thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.



Trademarks The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: IBM has two registered trademarks for the branding of ITSO publications. These registered marks are for the text word "IBM Redbooks" and the Redbooks logo. In a nutshell, the term Redbooks must always be used in the plural form (for both text and logo) since IBM only owns the registered mark for the plural form. Usage must follow the guidelines below: Using the term Redbooks in written text Redbooks are only to be referred to in the plural form, NEVER in the singular. For the initial reference (first occurrence), you must use "IBM Redbooks®" and include "IBM" as well as the ®. For instances thereafter you may use "Redbooks" without "IBM" preceding the word or ® following it. Correct usage for written text : In this IBM Redbooks® publication we will explore…..(® symbol required for 1st usage) This Redbooks publication will show you…..(2nd usage or later - no ® or "IBM" needed) Using the logo: OTHER ITSO PUBLICATIONS - Marks not yet registered Trademark registration is a lengthy process and until we are officially registered, we cannot use the ® symbol. For those terms/logos in process, we will be using the ™ symbol. In contrast to the ® symbol (placed in the lower right hand corner), the ™ symbol is placed in the upper right hand corner. Please see examples below: Redpaper ™ Redpapers ™ Redwiki ™ Redwikis ™ The following terms are trademarks of other companies: UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

Redbooks (logo)



Session objectives

• Provide an overview of the z/OS Communications Server features and enhancements delivered in V2R2

• The following areas will be described for each item where appropriate

– Background information

– Business problem

– Solution

– Enablement actions

– Externals

– Migration considerations



Release content themes

• The release content is grouped into 4 major categories

– Availability

– Scalability and Performance

– Security

– Simplification and Usability



• Reordering of cached Resolver results • Activate trace resolver without restarting applications • CICS sockets support for CICS TS 4.2 transaction tracking

Availability



REORDERING OF CACHED RESOLVER RESULTS



Background information

• System Resolver caching was introduced in z/OS V1R11 Communications Server

– Resolver will only cache response data from Domain Name System (DNS) servers

– Information obtained from local data files is not cached

– Resolver maintains separate IPv4 and IPv6 entries for the same resource

• Primary advantage of caching is the improved performance

– Eliminates repetitive DNS queries

• Caching activated on a system-wide basis

– Individual applications can turn off caching independently



Background information (continued)

• Host name to IP address resolution options

– Getaddrinfo, which supports both IPv4 and IPv6 addresses

– Gethostbyname, which supports only IPv4 addresses

• IP address to host name resolution options

– Getnameinfo, which supports both IPv4 and IPv6 addresses

– Gethostbyaddr, which supports only IPv4 addresses



Business problem

• Some DNS implementations reorder the list of IP addresses returned for a given host name in a round robin

fashion

– Provides a basic level of load balancing of IP addresses used by clients

• Resolver caching does not reorder the list of IP addresses

– IP addresses cached in the order received from the DNS server

– Same order used for all subsequent requires for the life of the cache information

– Any load balancing that might have been provided by the DNS server is eliminated



Solution

• Resolver can now reorder cached information

– Both system-wide and application levels of control are provided

– Only applicable to host name to IP address resolution (Getaddrinfo and Gethostbyname)

– IP addresses resolve to a single host name, so there is nothing to be reordered

– System-wide caching must be active



Solution (continued)

• Resolver reorders the cached information on a resolution query basis

– Reordering is independent of which application issues the query

– Reordering is independent of which type of query (Gethostbyname or Getaddrinfo) is issued

• Resolver reorders IPv4 and IPv6 resource information separately

• Resolver reorders the list before performing any sorting

– Gethostbyname results sorted based on SORTLIST configuration statement

– Getaddrinfo results sorted based on default destination address selection algorithm




• Application X issues Getaddrinfo request for aaa.com, and Resolver caches this list of IP Addresses for

aaa.com:

• Application X issues Getaddrinfo request for aaa.com, and Resolver caches this list of IP Addresses for

aaa.com:



Enablement actions: Resolver setup statements

• Use CACHEREORDER to activate cache reordering

• Use NOCACHEREORDER to stop cache reordering

– NOCACHEREORDER is the default

• Resolver ignores either statement when the NOCACHE setup statement is also specified

• You can modify the setting dynamically

– Update setting in resolver setup file, then issue MODIFY <resolver>,REFRESH,SETUP=<setup file name>



Enablement actions: TCPIP.DATA file

• Use the new NOCACHEREORDER statement to stop cache reordering for any application using this profile

– NOCACHEREORDER is meaningless if either system-wide caching or cache reordering is not active

– Specifying NOCACHEREORDER in the GLOBALTCPIPDATA data set is the equivalent of coding the

NOCACHEREORDER setup statement

• You can modify the setting dynamically

– Update setting in TCPIP.DATA file, then issue MODIFY <resolver>,REFRESH



Externals: MODIFY RESOLVER display changes

• CACHEREORDER (or NOCACHEREORDER) setting included in MODIFY RESOLVER,DISPLAY output



Externals: Trace RESOLVER changes

• CACHEREORDER (or NOCACHEREORDER) setting included in res_init Trace Resolver output



Externals

• Resolver NMI (EZBREIFR) available starting with z/OS V1R13 Communications Server

– Updated to include new setup file setting

– Updated to include GLOBALTCPIPDATA file setting, if any

• IPCS Resolver output also updated to include new setup file setting



ACTIVATE TRACE RESOLVER WITHOUT RESTARTING APPLICATIONS




• Trace Resolver is useful for diagnosing problems in resolving host names to IP addresses, or IP addresses

to host names

• Trace Resolver traces information on a per-application basis

• Trace Resolver can be enabled using one of these methods:

– z/OS UNIX RESOLVER_TRACE environment variable

– SYSTCPT DD allocation in the MVS batch job or TSO environment

– TRACE RESOLVER or OPTIONS DEBUG statement in the TCPIP.DATA file

– Debug option (resDebug) in an application $__res_state structure




• Trace Resolver output can be written to a variety of locations

– TSO user terminal screen

– Existing MVS sequential data set

– New or existing HFS file

– JES SYSOUT (for MVS batch job)

• Each record length can be between 80-256 characters

– If the record length is 128 or larger, the last six print positions are the storage address of the MVS TCB that issued the

resolver call




• Component Trace (CTRACE) is useful for collecting additional Resolver debug information

– Resolver CTRACE component is SYSTCPRE

• Unlike Trace Resolver, Resolver CTRACE shows resolver actions for all applications

– Information can be filtered by JOBNAME, ASID, or both

– All Resolver CTRACE records written to a common output location

• Only two Resolver CTRACE options

– ALL, MINIMUM



Business problem

• Dynamically starting or stopping Trace Resolver takes two steps:

– Setting TRACE RESOLVER or OPTIONS DEBUG in the TCPIP.DATA file

– Issuing the MODIFY RESOLVER,REFRESH command

• This approach is not possible for long-running Started Task Control (STC) servers

– STC servers use SYSTCPT DD allocation method or z/OS UNIX RESOLVER_TRACE environment variable to start

trace

– Modifying the setting of the Trace Resolver requires stopping and restarting the server

– Extremely disruptive to users and typically requires scheduled outage



Solution

• Use Resolver CTRACE to collect Trace Resolver information as CTRACE records

– New CTRACE option (TRACERES) defined

– Supports ASID and JOBNAME filtering

– Allows Trace Resolver information to still be collected on an individual application basis

– Allows Trace Resolver information to be collected without stopping and restarting the server

• Use IPCS CTRACE subcommand processing to view the formatted component trace data



Enablement actions: Activate tracing

• Use TRACE CT,ON command to enable the collection of Trace Resolver output as Resolver CTRACE

records

– Full syntax: TRACE CT,ON,COMP=SYSTCPRE,SUB=(resolver jobname)

– Specify OPTION=(TRACERES) in response text, plus any additional filters



Enablement actions (continued)

• Example of starting TRACERES collection using the TRACE,CT command



Enablement actions: Disable tracing

• Use TRACE CT,ON command to disable the collection of Trace Resolver output as Resolver CTRACE

records

– Full syntax: TRACE CT,ON,COMP=SYSTCPRE,SUB=(resolver jobname)

– Specify OPTION=() in response text, plus any additional filters

– OPTION=(ALL) or OPTION=(MINIMUM) can also be used



Externals

• Use IPCS CTRACE subcommand processing to view the formatted component trace data from a dump or

an external CTRACE data set



Externals (continued)

• Examples of formatted CTRACE TRACERES records



CICS SOCKETS SUPPORT FOR CICS TS 4.2 TRANSACTION TRACKING



Business problem

• CICS Transaction Server V4R2 introduced a new function to supply meta data to identify transaction Point of

Origin information

– CICS Explorer can display the Point of Origin information

– CICS SMF records include the Point of Origin information

• Point of Origin information is useful for problem determination

• CICS TCP/IP sockets support does not register Point of Origin information

– The CICS TCP/IP sockets listener transaction (CSKL) is commonly used to initiate CICS transactions

– CSKL initiated transactions reduces the value of CICS transaction tracking and adds complexity to problem diagnosis



Solution

• Add support for transaction tracking to CICS TCP/IP sockets

– Listener program EZACIC02 (CSKL) makes Point of Origin information available to the TRUE

– TRUE program EZACIC01 uses CICS facilities to register Point of Origin information for the transaction

– CICS Transaction Server for z/OS Version 4.2 and later allow resource managers to register tracking information in their TRUE

– No Point of Origin information registered for other transactions

– Transactions acting as clients

– Non-IBM provided listeners (i.e. vendor or home grown listeners)



Enablement actions

• None



Externals

• Transaction tracking fields

– Origin Adapter Data 1 → TCPIP Jobname

– Origin Adapter Data 2 → Local IP address and local port (Listener)

– Origin Adapter Data 3 → Remote IP address and remote port

– Origin Adapter ID → IBM zOS CommServer supplied listener name



Externals: CICS Explorer



Externals: CICS SMF 110 subtype 001 record



• 64 bit enablement of the TCP/IP stack • Enterprise Extender scalability • Enhanced IKED scalability • Shared memory communications over RDMA enhancements • Shared memory communications over RDMA adapter (RoCE) virtualization • SMC applicability tool (SMCAT) • Increase single stack DVIPA limit to 4096 • Removed support for legacy devices • VIPAROUTE fragmentation avoidance • TCP autonomic tuning

Scalability and Performance



64 BIT ENABLEMENT OF THE TCP/IP STACK



Background information: z/OS storage map

PSA

Private area

Nucleus/SQA LPA/CSA

ECSA/ELPA ESQA/ENucleus

Extended private area

reserved

ELSQA

common

common

extendedcommon

LSQA

16 MB

2 GB 4 GB

64-bit

31-bit 24-bit 16 MB

2 GB

16 EB

User extended private area

User extended private area

Shared area

512 TB

2 TB



Background information: Prior 64-bit usage

• z/OS V1R11 Communications Server

– Socket Control Blocks (SCBs)


– VTAM Internal Trace (VIT)

– TCP/IP CTRACE Area

– TN3270 CTRACE Area


– Shared Memory Communications for RDMA (SMC-R) control blocks and network data



Business problem

• Workload consolidation and larger systems

– increases demand for ECSA

– Increases demand on TCP/IP private area

• Performance implications

– AMODE switching to reference 64 bit storage

– Use of 31 bit addressing in AMODE(64)

– Access Register (AR) mode switching to reference dataspace storage



Solution

• Convert the TCP/IP stack to run in AMODE(64)

• Convert the TCP/IP stack to use 64 bit addresses

• Move 31 bit data areas to 64 bit storage

– Run time work areas and save areas (DUCB/DUSA)

– Moved from ECSA/private

– Network data (CSM)

– Moved from ECSA/dataspace

– Reduce switches to AR mode to reference dataspace

– Transmission control block (TCB)

– Moved from private



Solution: Network connectivity and 64 bit storage

• Interface types which exploit 64-bit virtual data in z/OS CS V2R2

– OSA-Express QDIO

– Inbound Enterprise Extender (EE) traffic with Inbound Workload Queueing (IWQ) still uses 31-bit CSM dataspace

– HiperSockets

– RoCE Express (for SMC-R)

• All other supported TCP/IP network connectivity (such as MPCPTP, LCS, CTC) is compatible with 64-bit

virtual memory

– These are referred to as 31-bit network interface types

– z/OS CS still uses 31-bit CSM dataspace for these types



Solution: Storage results

• Lab results for 128,000 TN3270 sessions in KB

V2R1 V2R2 % change

TN3270 ECSA 1,575 145 -91%

TN3270 Private 440,054 541,618 23%

TCP/IP ECSA 9,188 6,593 -28%

TCP/IP Private 275,338 43,332 -84%

TCP/IP HVCOMMON 63,000 70,000 11%

TCP/IP HVPRIVATE 1,000 513,000 512%



Enablement actions: New IVTPRM00 parameter

• HVCOMM maxhvcommM

– Defines the maximum amount of storage dedicated to High Virtual Common storage CSM buffers.

– maxhvcommM

– A decimal integer specifying the maximum bytes of HVCOMM storage dedicated to CSM use.

– Valid Range: 100M to 999999M

– Default Value: 2000M

– Notes:

– M indicates megabytes

– Defined in megabytes only



Externals: Modify CSM

• MODIFY CSM command to update CSM storage value dynamically or to activate changes made to the CSM

parmlib member IVTPRM00 without requiring an IPL

>>__MODIFY proc,CSM_ _____________ _ _____________ _ _________________ _><

|_,ECSA=mecsa_| |_,FIXED=mfix_| |_,HVCOMM=mhvcomm_|

– mhvcomm specifies the maximum number of bytes of high virtual common (HVCOMM) storage for CSM buffers

– Valid Range: 100M to 999999M



Externals: Display NET,CSM

• Display NET,CSM command output showing 64 bit storage values

**** Continued *** IVT5532I ------------------------------------------------------ IVT5533I 4K HVCOMM 24K 1000K 1M IVT5533I 16K HVCOMM 96K 928K 1M IVT5533I 32K HVCOMM 192K 832K 1M

IVT5533I 60K HVCOMM 360K 660K 1020K IVT5533I 180K HVCOMM 720K 1080K 1800K IVT5535I TOTAL HVCOMM 1392K 4500K 5892K IVT5532I ------------------------------------------------------

“”””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””” IVT5538I FIXED MAXIMUM = 2048M FIXED CURRENT = 5949K IVT5541I FIXED MAXIMUM USED = 5949K SINCE LAST DISPLAY CSM

IVT5594I FIXED MAXIMUM USED = 5949K SINCE IPL



Externals: Display NET,CSM (continued)

• Display NET,CSM command output showing 64 bit storage values

**** Continued *** IVT5539I ECSA MAXIMUM = 100M ECSA CURRENT = 5073KIVT5541I ECSA MAXIMUM USED = 5073K SINCE LAST DISPLAY CSMIVT5594I ECSA MAXIMUM USED = 5073K SINCE IPL IVT5604I HVCOMM MAXIMUM = 1000M HVCOMM CURRENT = 9MIVT5541I HVCOMM MAXIMUM USED = 9M SINCE LAST DISPLAY CSMIVT5594I HVCOMM MAXIMUM USED = 9M SINCE IPL IVT5559I CSM DATA SPACE 1 NAME: CSM64001 IVT5559I CSM DATA SPACE 2 NAME: CSM31002

IVT5599I END



Externals: VTAM Internal Trace (VIT)

• New and changed VIT records with 64-bit addresses

– New records

– IUT6 (outbound QDIO)

– XB61, XB62, XB63 (inbound/outbound QDIO)

– QAP6 (QDIO Accelerator)

– GCE6 (64-bit CSM)

– Changed records

– ODPK (inbound/outbound QDIO)



Migration considerations

• IVTAPRM00 default value for FIXED changed to 200M

– Defines the maximum amount of storage dedicated to fixed CSM buffers.

• Use VIPAROUTE over OSA-Express QDIO or HiperSockets to optimize SD traffic

– Forwarding over 31 bit network interface types (XCF) involves additional data copy

• Use the IWQ function for OSA-Express QDIO to optimize EE inbound traffic (INBPERF WORKLOADQ)

– EE inbound traffic will be staged in 31 bit storage

• Display NET,CSM displays new HVCOMM information



ENTERPRISE EXTENDER SCALABILITY




• An Enterprise Extender local node communicates with a remote node using UDP over an IP network

– A local node is defined within a TCP/IP stack by a local IP address, typically a static VIPA, and 5 UDP sockets (5

UCB control blocks).

– Each UDP socket is bound to the static VIPA and one of 5 UDP ports (default 12000-12004). The ports map to 4

SNA routing priorities for data traffic, plus one port for LLC commands

– An EE link represents the “connection” between a local node and remote node. The link has 5 routes through the IP

network - one for each port

EE local node

EE remote node

VIPA UCB table

12000 12001 12002 12003 12004

EE route cache

route route route route route



Business problem: Scaling issues

• As packets are processed:

– Serialization on one of 5 UCBs causes performance bottlenecks, storage constraints for suspended threads

(suspended DUCBs)

– Increased cache misses on IPSEC and Policy rules causes higher CPU utilization

• As an EE link to a remote node is created, extra processing time is needed to find open slots in IP MAIN's

route cache (lesser concern since this is per connection)



Solution

• Create a new “remote UCB” structure for each EE port

– IPSEC rules and Policy moved from main UCB

– Outbound flows – new “remote UCB” lock accessed instead

of local UCB lock

– Inbound flows – EE policy lock replaced by remote UCB

lock

• Access remote UCB

– using one of 5 new hash tables added to UCB table (one

per local port)

– Hash key to access remote UCB is remote node's IP and

port

• Move route cache to remote UCB

EE local node

EE remote node

VIPA UCB table

12000

12001 12002 12003 12004

Remote UCB

IPSEC rules Inbound filter rule Outbound filter rule

Inbound policy

Outbound policy

Route info

Hash key



Enablement actions

• None




• None



ENHANCED IKED SCALABILITY



Background information: Internet Key Exchange

IKE peers negotiate an IKE (“phase 1”) tunnel (one

bidirectional SA) over an unprotected UDP socket.

IKE peers negotiate IPSec (“phase 2”) tunnel (two

unidirectional SAs) under protection of the IKE tunnel.

These SAs are installed into the TCP/IP stack

Data flows through IPSec tunnel using Authentication

Header (AH) and/or Encapsulating Security Payload

(ESP) protocol

Each peer authenticates each other using digital signatures based on

digital certificates or pre-shared keys

Peers agree on a set of cryptographic algorithms to use to protect the

subsequent IKE messages that will flow between the two (phase 2 SA

negotiations, informational exchanges and notifications)

• A series of IKE messages are exchanged under the protection of the

phase 1 tunnel. This includes encryption, authentication and integrity

protections for every IKE message

• Upon completion, the phase 2 SAs are installed in the TCP/IP stack

• Data packets are sent between the IPsec endpoints under the

protection of the phase 2 tunnels. This includes encryption,

authentication and integrity protections for every data packet

• The IKE daemon is not involved until it is time to refresh or delete one

of the security associations



Business problem

• When a very large number (multiple thousands) of remote IKE peers simultaneously initiate negotiations

with a single z/OS IKED, the z/OS daemon struggles to keep up with the load

• Symptoms:

– A large portion of the remote IKE peers retransmit messages due to timeouts (per the IKE protocol)

– Inbound IKE messages are discarded by z/OS TCP/IP stack as capacity of UDP queues is reached

– z/OS IKED spends more and more time handling retransmitted messages from peers (per the IKE protocol)

– IKED takes a significant amount of time to recover to a stable state

– A “stairstep” effect in the rate of negotiation activity

– Bursts interleaved with increasingly longer quiet intervals

– Dropped inbound IKE messages and IKE protocol's geometric back-off



Business problem (continued)

• “Stairstep effect” of large numbers of remote IKE peer retransmissions:

Com

plet

ed tu

nnel

s

Time



Solution

• A new thread pool with appropriate serialization is added to IKED

– IKE negotiations are now handled by this pool (vs. a single thread per the previous design)

– Inbound IKE protocol messages

– Other internal events required to complete the negotiations

– No permanent affinity between a given IKE peer and any thread within IKED.

• Inbound IKE messages now prioritized

– Duplicate (retransmitted) IKE messages are detected and discarded upon receipt – significantly reduces workload

– “Later” IKE messages prioritized ahead of “earlier” ones – promotes completion of in-progress negotiation before

starting new ones




• Initial scalability testing has been very positive - generally linear scalability as the number of CPUs is

increased

• Changes will be transparent to the vast majority of z/OS IKED users – significant improvements will be more

noticeable under heavier workloads

V2R1 V2R2



Enablement actions

• None



Externals

• One new level added to IkeSyslogLevel parameter of the iked.conf file

128 – IKE_SYSLOG_LEVEL_DEBUGPTP

Show additional information regarding primary thread pool scheduling

• Syslogd output:

– New messages for log level DEBUGPTP

– Messages might now be interleaved (up until now, they have appeared in an order that was fairly representative of the

actual order of events)

– IKED thread ID will now appear in the syslogd message header




• Those with multiple thousands of IKE peers might need to adjust specific resources:

– Virtual storage available to IKED

– Maximum number of messages allowed on z/OS message queues

– Limitations on number of messages allowed on inbound UDP queues

• Automated processing of SYSLOGD messages may need to be adjusted for the thread id



SHARED MEMORY COMMUNICATIONS OVER RDMA ENHANCEMENTS



Background information: SMC-R

• SMC-R is a “hybrid” solution:

– Existing TCP connection establishment flows still used

– SMC-R option exchanged as TCP option in connection establishment

– SMC-R usage negotiated similarly to how SSL usage is negotiated

– Application data flows “out-of-band” using RDMA protocols

– RoCE Express MTUs 1024 and 2048 supported

– Peers negotiate and use the smallest size supported

• Preserves critical existing operational and network management features of TCP/IP



Background information: View of SMC-R

• Shared Memory Communications over RDMA (SMC-R) defines a means to exploit Remote Direct Memory

Access (RMDA) technology for communications transparently to the applications

SMC-R enabled platform

OS image OS image

Virtual server instance

server client

RNIC

Shared Memory Communications

via RDMA

SMCSMC

RDMA enabled (RoCE)

RNIC

Clustered Systems



shared memory shared memory

Sockets Sockets



Business problem: MTU • The RoCE Express can support sending data in three different MTU sizes: 1024, 2048 and 4096

– z/OS V2R1 SMCR implementation supported PFID configuration of just two of the sizes: 1024 and 2048

– For large data sends, a larger MTU can improve throughput



Solution: Support 4K MTU

• GLOBALCONFIG SMCR PFID configuration now supports 4K MTU

• Existing displays will show new value

– Netstat,CONFIG/-f command shows configured value

– Netstat,DEvlinks/-d,SMC command will show actual value in use for SMCR link



Enablement actions: MTU configuration

• Configure the SMCR PFID MTU value with GLOBALCONFIG

– Default MTU value is 1024

>>_GLOBALCONFif___________________________________________________________> ... | '-SMCR---+---------------------------------------------------+-+-' | | | .-----------------------------------------------. | | | | | .------------------------------. | | | | | V V | | | | | +---PFID - pfid----+--------------------------+-+-+-+ | | | | .-PORTNum -1---. | | | | | +-+--------------+---------+ | | | | | '-PORTNum -num-' | | | | | | .-MTU -1024----' | | | | | '-+--------------+---------' | | | | '-MTU -mtusize-' | | ...



Externals: Netstat CONFIG/-f

• Confirm PFID MTU is configured correctly

GLOBAL CONFIGURATION INFORMATION:TCPIPSTATS: YES ECSALIMIT: 2096128K POOLLIMIT: 2096128KMLSCHKTERM: NO XCFGRPID: 11 IQDVLANID: 27

SYSPLEXWLMPOLK: 060 MAXRECS: 100EXPLICITBINDPORTRANGE: 05000 -06023 IQDMULTIWRITE: YESWLMPRIORITYQ: YES

IOPRI1 0 1IOPRI2 2IOPRI3 3 4IOPRI4 5 6 FWD

SYSPLEX MONITOR:TIMERSECS: 0060 RECOVERY: YES DELAYJOIN: NO AUTOREJOIN: YESMONINTF: YES DYNROUTE: YES JOIN: YES

zIIP:IPSECURITY: YES IQDIOMULTIWRITE: YES

SMCR: YESFIXEDMEMORY: 200M TCPKEEPMININT: 00000300PFID: 001C PORTNUM: 1 MTU: 1024PFID: 0015 PORTNUM: 2 MTU: 4096



Externals: Netstat DEVLINKS,SMC

• Confirm actual MTU for SMC-R link is correct

D TCPIP,TCPCS1,NETSTAT,DEVLINKS, SMC

EZD0101I NETSTAT CS V2R2 TCPCS1

INTFNAME: EZARIUT1001C INTFTYPE: RNIC INTFSTATUS: READY

PFID: 001C PORTNUM: 1 TRLE: IUT1001CPNETID: ZOSNETVMACADDR: 02000035F740

GIDADDR: FE80::200:FF:FE35:F740INTERFACE STATISTICS:

BYTESIN = 160INBOUND OPERATIONS = 5

BYTESOUT = 344OUTBOUND OPERATIONS = 11

SMC LINKS = 1

TCP CONNECTIONS = 1INTF RECEIVE BUFFER INUSE = 64K

SMC LINK INFORMATION:LOCALSMCLINKID: 2D8F0101 REMOTESMCLINKID: 729D0101

SMCLINKGROUPID: 2D8F0100 VLANID: 100 MTU: 4096

LOCALGID: FE80::200:FF:FE35:F740LOCALMACADDR: 02000035F740 LOCALQP: 000040

REMOTEGID: FE80::200:1FF:FE35:F740

REMOTEMACADDR: 02000135F740 REMOTEQP: 000041




• None

– When an SMC-R link is initially established between two peer hosts, the MTU size is exchanged and negotiated to the

lowest value for both hosts



Business problem: Repeated SMC-R failures

• Every SMC-R eligible TCP connection will attempt to connect to its peer using SMC-R

• Examples of reasons a TCP connection cannot use SMC-R

– IPSEC

– Mismatching subnets (two peers not in same subnet or vlan)

– Link layer issues prevent connectivity over RoCE fabric

– Config problem – Connection setup delays possible

• In these cases the stack attempts to use SMC-R then generally falls back to TCP

• These conditions can exist for extended periods of time affecting numerous TCP connections

– Even if they fallback to using TCP they incur overhead



Solution: Cache SMC-R failures

• Cache IP destinations with persistent SMC-R establishment failures

– Cached when we encounter three consecutive failures in an interval (approximately twenty minutes)

– While cached, connections will use TCP

– Cached destinations cleared approximately every interval

– Gives new connections opportunity to exploit SMC-R periodically

– Cache can also be cleared by disabling AUTOCACHE function

• Enabled with new GLOBALCONFig SMCGlobal AUTOCACHE configuration statement

– Enabled by default

– Disabled with GLOBALCONFig SMCGlobal NOAUTOCACHE



Enablement actions

• Configured with GLOBALCONFig SMCGlobal statement

• Default value is AUTOCACHE (function enabled)

>>-GLOBALCONFig-------------------------------------------------->>----+--------------------------------------------------------+-+-><: :

| .-------------------------. | | V .-AUTOCACHE---. | |+-SMCGlobal---+--+-------------+----+--+-----------------+| | '-NOAUTOCACHE-' | || | | || | .-AUTOSMC------. | || '--+---------------+--' || '-NOAUTOSMC----' |




• Confirm AUTOCACHE is configured correctly

GLOBAL CONFIGURATION INFORMATION: TCPIPSTATS: NO ECSALIMIT: 0000000K POOLLIMIT: 0000000K MLSCHKTERM: NO XCFGRPID: IQDVLANID: 0 SYSPLEXWLMPOLL: 060 MAXRECS: 100 EXPLICITBINDPORTRANGE: 00000 -00000 IQDMULTIWRITE: NO AUTOIQDX: ALLTRAFFIC ADJUSTDVIPAMSS: AUTO WLMPRIORITYQ: NO SYSPLEX MONITOR:

TIMERSECS: 0060 RECOVERY: NO DELAYJOIN: NO AUTOREJOIN: NO MONINTF: NO DYNROUTE: NO JOIN: YES

ZIIP: IPSECURITY: NO IQDIOMULTIWRITE: NO

SMCGLOBAL: AUTOCACHE: YES AUTOSMC: NO

SMCR: YES FIXEDMEMORY: 200M TCPKEEPMININT: 00000300PFID: 001C PORTNUM: 1 MTU: 1024PFID: 0015 PORTNUM: 2 MTU: 4096



Externals: Netstat ALL/-A

• Determine if connection was cached not to use SMC-R

– Asterisk (*) after reason code indicates destination IP address was cached

D TCPIP,TCPCS1,NETSTAT,ALL,IPPORT=10.1.1.14+21EZD0101I NETSTAT CS V2R2 TCPCS1CLIENT NAME: FTPDOE34 CLIENT ID: 0000003B

LOCAL SOCKET: ::FFFF:10.1.1.14..21FOREIGN SOCKET: ::FFFF:10.1.1.24..1024

...SMC INFORMATION:

SMCSTATUS: INACTIVE SMCREASON: 00005301* -PEER DID NOT ACCEPT SMC -R REQUEST

----1 OF 1 RECORDS DISPLAYEDEND OF THE REPORT




• SMCGLOBAL AUTOCACHE is the default value

– Configure SMCGLOBAL NOAUTOCACHE to preserve the existing behavior

• Netstat ALL / -a and CONFIG / -f



Business problem: SMC-R short-lived connections

• Short-lived TCP connections that exchange small amounts of data might be better suited for TCP instead of

SMC-R

– Impacted by extra packet flows creating SMC-R connection

– PORT/PORTRANGE configuration provides the NOSMC subparameter

– Inbound TCP connections using this port will not use SMC-R

– Useful if user knowledgeable about the workload to particular servers

– Many users are not aware of the workload patterns or the patterns can change over time



Solution: SMC-R workload monitoring

• Enables the stack to analyze incoming TCP connections to dynamically determine whether SMC-R is

beneficial for a local TCP server application

– Identifies short-lived connections exchanging little data

• Results of this monitoring influences whether TCP connections to a particular server (port) use SMC-R

• Ensures TCP connections use the most appropriate communications protocol (TCP or SMC-R)

• Workload data analyzed every interval so results reflect most recent activity



Enablement actions

• Enabled with new GLOBALCONFig SMCGlobal AUTOSMC configuration statement

– Enabled by default

• New PORT/PORTRANGE SMC configuration option added

– PORT/PORTRANGE NOSMC added in z/OS V2R1

– PORT/PORTRANGE configuration will override AUTOSMC monitoring



Enablement actions: Configuring AUTOSMC

• Configured with GLOBALCONFig SMCGlobal statement

• Default value is AUTOSMC (function enabled)

>>-GLOBALCONFig-------------------------------------------------->>----+--------------------------------------------------------+-+-><: :

| | | | | | V .-AUTOCACHE---. | |+-SMCGlobal---+--+-------------+----+--+-----------------+| | '-NOAUTOCACHE-' | || || | .-AUTOSMC------. | || '--+---------------+--' || '-NOAUTOSMC----' |




• Confirm AUTOSMC is configured correctly

GLOBAL CONFIGURATION INFORMATION: TCPIPSTATS: NO ECSALIMIT: 0000000K POOLLIMIT: 0000000K MLSCHKTERM: NO XCFGRPID: IQDVLANID: 0 SYSPLEXWLMPOLL: 060 MAXRECS: 100 EXPLICITBINDPORTRANGE: 00000-00000 IQDMULTIWRITE: NO AUTOIQDX: ALLTRAFFIC ADJUSTDVIPAMSS: AUTO WLMPRIORITYQ: NO SYSPLEX MONITOR: TIMERSECS: 0060 RECOVERY: NO DELAYJOIN: NO AUTOREJOIN: NO MONINTF: NO DYNROUTE: NO JOIN: YES

ZIIP: IPSECURITY: NO IQDIOMULTIWRITE: NO

SMCGLOBAL: AUTOCACHE: YES AUTOSMC: YES

SMCR: YES FIXEDMEMORY: 200M TCPKEEPMININT: 00000300PFID: 001C PORTNUM: 1 MTU: 1024PFID: 0015 PORTNUM: 2 MTU: 4096




• View current server details

– 90% of monitored connections over last interval had ideal workload for SMC-R

– AutoSMC% must be >= 50% for UseSMC to be YES

CLIENT NAME: USER19 CLIENT ID: 00000052 LOCAL SOCKET: 0.0.0.0..4206 FOREIGN SOCKET: 0.0.0.0..0 BYTESIN: 00000000000000000000 BYTESOUT: 00000000000000000000 SEGMENTSIN: 00000000000000000000 SEGMENTSOUT: 00000000000000000000 STARTDATE: 01/30/2015 STARTTIME: 19:02:04 LAST TOUCHED: 19:02:05 STATE: LISTEN ........ CONNECTIONSIN: 0000000200 CONNECTIONSDROPPED: 0000000000 MAXIMUMBACKLOG: 0000000010 CONNECTIONFLOOD: NO CURRENTBACKLOG: 0000000000

SERVERBACKLOG: 0000000000 FRCABACKLOG: 0000000000 CURRENTCONNECTIONS: 0000000050 SEF: 100 QUIESCED: NO SMC INFORMATION: SMCRCURRCONNS: 0000000025 SMCRTOTALCONNS: 0000000100UseSMC: Yes Source: AutoSMCAutoSMC%: 090



Externals: Netstat PORTlist/-o

• Confirm port configuration

– M indicates port explicitly enabled for SMC (new function)

– N indicates port explictly enabled for NOSMC (existing function)

– These settings will override AUTOSMC for these ports

NETSTAT PORTLIST MVS TCP/IP NETSTAT CS V2R2 TCPIP Name: TCPCS 15:24:23

Port# Prot User Flags Range SAF Name

----- ---- ---- ----- ----- --------.....

04002 TCP OMVS DABU

04020 TCP DCICSTS DAN 05000 TCP * DARN 05000- 05001

06020 TCP * DAM

06000 TCP * DARM 06000- 06001

UNRSV UDP * FI GENERIC .....




• SMCGLOBAL AUTOSMC is the default value

– Configure SMCGLOBAL NOAUTOSMC to preserve the existing behavior

• Netstat

– ALL / -a

– CONFIG / -f

– PORTList / -o



SHARED MEMORY COMMUNICATIONS OVER RDMA ADAPTER (ROCE) VIRTUALIZATION



Background information: View of SMC-R

• Shared Memory Communications over RDMA (SMC-R) defines a means to exploit Remote Direct Memory

Access (RMDA) technology for communications transparently to the applications


OS image OS image


server client

RNIC

Shared Memory Communications

via RDMA

SMCSMC

RDMA enabled (RoCE)

RNIC

Clustered Systems



shared memory shared memory

Sockets Sockets



Background information: RoCE Express feature

• System/z provides a physically separate 10GbE RoCE Express feature to exploit RoCE (RDMA over

Converged Ethernet) functionality

– Used in conjunction with the existing Ethernet connectivity provided by OSA

– Provides access to the same physical Ethernet fabric used for traditional IP connectivity

– Provides two 10GbE ports

– Sometimes referred to as “RNIC adapter”

• For redundancy, at a minimum two 10GbE RoCE Express features should be configured for each physical

network you configure

• RoCE Express features are supported using a converged interface model



Background information: View of dedicated RoCE

PFIDPFID

CPC

PFIDPFID

z/OS 2z/OS 1 z/OS 3

PFID 1PFID 1 PFID 2PFID 2

PR/SM

If 1If 1 If 1If 1 If 1If 1If 1If 1

LP 1 LP 2 LP 3 LP 4 LP 5 LP 6

PCHID 100 FID 01

Ports 1 and 2

Physical Net ID = ‘NETA’

z/OS 4

Ports 1 and 2

RoCE RoCEPCHID 200 FID 16

I/O Draw 1 I/O Draw 2

VMAC for each PFID (per TCP stack)

VMAC for each PFID (per TCP stack)

PFIDPFID

z/OS 5

If 1If 1

PFID 16 PFID 16 PFID 17PFID 17

If 2If 2 If 2If 2

z/OS 6

Physical Network IDs are configured in HCD (IOCDS) for each physical port

Up to 16 PCHIDs per CPC

PFIDPFID

If 1If 1



Business problem: Dedicated RoCE

• Inability to share RoCE Express features between multiple LPARs

– Up to eight TCP/IP stacks on one LPAR can share a feature

– VTAM provides the virtualization

– Redundancy requirements can quickly increase the number of RoCE Express features required for SMC-R

– Limit of 16 features per CPC

• Only one port on a given RoCE Express feature could be used

– Could switch between ports, but still only use one at a time



Solution: Shared RoCE

• RoCE Express features can be shared across LPARs

– Up to 31 operating system instances can share one feature

• Both RoCE Express ports can be used simultaneously

• No additional RNIC definitions in z/OS Comm Server

– PFID values are still defined on TCP/IP profile GLOBALCONFIG statement

– PFID value must be unique if the RoCE Express feature is being shared by multiple TCP/IP stacks

• No change in RNIC activation

– RoCE Express features are still activated when the first SMC-R capable OSA interface is activated



Solution: Shared RoCE (continued)

• z/OS V2R2 supports both dedicated and shared RoCE environments, depending on the hardware:

– IBM zEnterprise EC12 (zEC12) with driver 15 or an IBM zEnterprise BC12 (zBC12) support dedicated RoCE

environment only

– IBM z13 or later supports shared RoCE environment only

• z/OS V2R1 also supports both environments

– APARs OA44576 and PI12223

• z/OS Communications Server detects the working environment during activation of the first RoCE Express

feature



Solution: VLAN considerations

• Each RoCE Express feature still supports 126 VLANs

• The 126 VLANs must be shared across all virtual functions using the feature

– Each VF is guaranteed at least two VLANs on a given RoCE Express feature

– Each VF can use at most 16 VLANs on a given RoCE Express feature

– Note: If two, or more, VFs share a RoCE Express feature, and use the same VLANID, that counts as only one of the

126 available VLANs

• OSA (and RNIC) interfaces that use VLANs can now co-exist with OSA (and RNIC) interfaces that do not

use VLANs on the same RoCE Express feature

– Requires APAR OA44679 in z/OS V2R1



Solution: Redundancy considerations

• Full SMC-R redundancy requires two unique physical paths

– Different RoCE Express features

– Different I/O draws

– Different internal support structures

• You must be careful to configure your system to ensure that the TCP/IP stack uses RoCE Express features

that provide full redundancy

– Less than full redundancy can result in TCP connection failures if a RoCE Express failure is encountered



Solution: View of shared RoCE using 2 ports

PFIDPFID

CPC

PFIDPFID

z/OS 2z/OS 1 z/OS 3

PFID 1PFID 1 PFID 2PFID 2

PR/SM

If 1If 1 If 1If 1 If 1If 1If 1If 1

LP 1 LP 2 LP 3 LP 4 LP 5 LP 6

PCHID 100 FID 01 VF 10 FID 02 VF 11

Ports 1 and 2

Physical Net ID = ‘NETA’

z/OS 4

Ports 1 and 2RoCE RoCE

PCHID 200 FID 16 VF 22 FID 17 VF 23

I/O Draw 1 I/O Draw 2

VMAC for each VF per PFID VMAC for each VF per PFID

PFIDPFID

z/OS 5

If 1If 1

PFID 16 PFID 16 PFID 17PFID 17

If 2If 2 If 2If 2

z/OS 6

Physical Network IDs are configured in HCD (IOCDS) for each physical port

Up to 16 PCHIDs per CPC

VFs 10 and 22VFs 11 and 23

PFIDPFID

If 1If 1



Enablement actions: HCD changes

• Must provide a virtual function (VF) number

Goto Filter Backup Query Help ------------------------------------------------------------------------------ PCIe Function List Row 28 of 500 More: > Command ===> _______________________________________________ Scroll ===> CSR Select one or more PCIe functions, then press Enter. To add, use F11. Processor ID . . . . : S88 z13 S88 / FID PCHID VF+ Type+ Description _ 028 108 28 ROCE S3E _ 029 108 29 ROCE S3E _ 030 108 30 ROCE S3E _ 031 108 31 ROCE S3E _ 032 13C 1 ROCE S36 _ 033 13C 2 ROCE S36 _ 034 13C 3 ROCE S36 _ 035 13C 4 ROCE S36 _ 036 13C 5 ROCE S36



Enablement actions: Selecting the PFID

• The PFID value on the TCPIP profile GLOBALCONFIG SMCR statement has a slightly different meaning in

a shared environment:

– In a dedicated environment, the PFID directly identifies the RoCE Express feature, and all TCP/IP stacks sharing the

feature use the same PFID

– In a shared environment, each TCP/IP stack has its own unique PFID value to represent the RoCE Express feature

• RoCE Express ports can be shared

– Same or different TCP/IP stacks can use the two ports

– Different PFID values must be defined for each usage of the port



Externals: Netstat DEvlinks/-d

• SMC-R link group information included after all SMC-R links

• New redundancy values defined:

– Partial (Single local PCHID, unique ports)

– Partial (Single local PCHID and port)

SMC LINK GROUP INFORMATION: SMCLINKGROUPID: 2D8F0100 PNETID: NETID1 REDUNDANCY: PARTIAL (SINGLE LOCAL PCHID AND PORT) LINK GROUP RECEIVE BUFFER TOTAL: 3M 64K BUFFER TOTAL: 1M LOCALSMCLINKID REMOTESMCLINKID -------------- --------------- 2D8F0101 729D0101 2D8F0102 729D0102 2 OF 2 RECORDS DISPLAYED END OF THE REPORT



Externals: Display TRL,TRLE=rnic_trlename

• When using shared RoCE environment:

– VF number is displayed but the code level is not available

D NET,TRL,TRLE=IUT1001C IST097I DISPLAY ACCEPTED IST075I NAME = IUT1001C, TYPE = TRLE IST1954I TRL MAJOR NODE = ISTTRL IST486I STATUS= ACTIV, DESIRED STATE= ACTIV IST087I TYPE = *NA* , CONTROL = ROCE, HPDT = *NA* IST2361I SMCR PFID = 001C PCHID = 0130 PNETID = NETID1 IST2362I PORTNUM = 1 RNIC CODE LEVEL = ***NA*** IST2389I PFIP = 01000300 IST2417I VFN = 0001 IST924I ------------------------------------------------------------ IST1717I ULPID = TCPCS1 ULP INTERFACE = EZARIUT1001C IST1724I I/O TRACE = OFF TRACE LENGTH = *NA* IST314I END



Externals: CSDUMP

• New MSGVALUE option for RNICTRLE operand

– Allows capture of diagnostic information when error message is generated for any RoCE Express feature

– Only valid for MESSAGE=IST2406I or MESSAGE=IST2391I

• A dump of the RoCE Express feature by one virtual function is NOT disruptive to other virtual functions that

are using the feature

|_,MESSAGE=_message_id_numbers________________________________________________________| |_,TCPNM=TCPIP_Jobname_||_,RNICTRLE= ______________ _| |_MSGVALUE_____| |_RNICTRLEName_|

IST2406I or

IST2391I



Externals: VTAM Internal Trace (VIT) records

• New VIT records were defined

– VHCR, VHC2, VHC3, VHC4, and VHC5

– Similar to existing HCR records, but for shared RoCE environment command processing

– CCR and CCR2

– Communication channel operation in shared RoCE environment



Externals: VTAM dump formatting support

• SNASMCR

– Formats VTAM control blocks used to manage TCP/IP ownership of the RoCE Express feature, including associated

RMB, VLAN, and QP information

• SNAROCE

– Formats VTAM control blocks used to manage the RoCE Express feature

• Function rolled back to z/OS V2R1 using APAR OA44576



Externals: VTAM dump formatting support (continued)




• Assign VF values in HCD for each FID

• If you currently have multiple TCP/IP stacks sharing a RoCE Express feature in a dedicated RoCE

environment, you must:

– Define unique FID values in HCD for the stacks to use as PFIDs on the TCPIP profile GLOBALCONFIG SMCR

statement

• Ensure you have full redundancy with your shared RoCE Express features or SMC-R fail-over processing

can be compromised



SMC APPLICABILITY TOOL (SMCAT)



Business problem: Is SMC-R applicable to my environment

• SMC-R requires a new RDMA capable NIC

– 10GbE RoCE Express feature introduced in zEC12 GA2 and zBC12

– Each LPAR requires two RoCE Express features for High Availability

• Useful to know if workload will exploit SMC-R beforehand

– Some users are aware of the significant traffic patterns that can benefit from SMC-R

– Others are unsure of how much of their traffic is able to use SMC-R

– z/OS-z/OS

– Workload patterns ideal for SMC-R

– Not IPSec encrypted



Business problem: Is SMC-R applicable to my environment (continued)

• Can use SMF records, Netstat displays, and reports from network management products

– Helps users determine if their environments will benefit from the SMC-R function

– This type of analysis is time consuming and requires significant expertise



Solution: SMC Applicability Tool (SMCAT)

• A new tool that helps show the potential benefits of implementing SMC-R

– Controlled by the Vary TCPIP,,SMCAT command

– Monitors a stack's TCP traffic

– For a set of configured destination IP addresses and subnets/prefixes

– For a configured interval of time



Solution: SMCAT

• SMCAT does not require SMC-R to be enabled

• SMCAT is integrated within the TCP/IP stack and gathers new statistics that are used to project SMC-R

applicability

– Minimal system overhead, no changes in TCP/IP network flows

– Produces report on potential benefits of enabling SMC-R

• Available via the service stream on existing z/OS releases as well

– V1R13 - Apar PI27252/PTF UI24872

– V2R1 - Apar PI29165/PTFs UI24762 and UI24763



Solution: SMCAT (continued)

• At the end of the interval a summary report is generated that includes:

– Percent of traffic “eligible” for SMC-R

– All traffic that matches configured IP addresses and do not use IPSec or FRCA

• Percent of traffic “well suited” for SMC-R

– Eligible traffic that excludes workloads with very short lived TCP connections and trivial payloads

– Includes break out of application send sizes

– How large is the payload of each send request



Solution: SMCAT (continued)

• The summary report contains two sections:

– First section contains data for all eligible TCP connections

– Includes connections that are not directly connected

– Traffic between the hosts requires traversal of a router which is not supported by the SMC-R protocol

– Indicates total amount of workload that can exploit SMC communications

– Some connections might require network topology changes

– The second section contains data for just the directly connected eligible (match configuration) TCP connections

– Network traffic between the hosts does not require traversal of any IP routers

– Indicates amount of workload that can immediately exploit SMC communications after SMC-R enablement

– This section is a subset of the first section



Enablement actions: Configure the data set

• SMCAT data set configuration

– INTERVAL defaults to 60 minutes

– Max is 1440 minutes (24 hours)

– IPADDR is a list of IPv4 and IPv6 addresses and subnets

– 256 max combination of addresses and subnets

_INTERVAL 60_____|---SMCATCFG____|_________________|_______________________________________________>

|_INTERVAL minutes |>_________________________________________________________________________________|||||_IPADDR_______ipv4_address_____________________

_ipv4_address/num_mask_bits__ipv6_address________________ipv6_address/prefix_length_



Enablement actions: Configure data set example

• SMCAT data set configuration example

– Monitor workload for two hours

– Monitor workload for configured IPv4 address and IPv6 prefix

SMCATCFG INTERVAL 120 IPADDR 192.168.1.1 192.168.3.0/24 C5::1:2:3:4/126



Enablement actions: Start/Stop SMCAT

• Vary TCPIP,,SMCAT command starts and stops the monitoring tool

– datasetname value indicates that SMCAT is being turned on

– datasetname contains the SMCATCFG statement that specifies monitoring interval and IP addresses or subnets to be

monitored

– OFF will stop SMCAT monitoring and generate report

>>__Vary__TCPIP ,__ __________ __,__SMCAT,__ datasetname________><|_procname_| |_,OFF__|

VARY TCPIP,TCPPROC,SMCAT,USER99.TCPIP.SMCAT1



Externals: Report

• Key messages – Operator console

– EZD2031I SMC APPLICABILITY TOOL HAS STARTED COLLECTING DATA

– EZD2032I SMC APPLICABILITY TOOL HAS STOPPED COLLECTING DATA

• Configuration information and the SMCAT report are sent to the system log

STC06578 EZD2040I TCP/IP CS V2R2 TCPIP Name: TCPIP080 SMC Applicability Configuration Parameters - 02/04/2015, 10:09:49.08080 Interval: 3 minutes080 IP addresses/subnets being monitored080080 9.67.113.61080 C5::1:2:3:4/126080 End of configuration parameters



Externals: Report example SMC Applicability Interval Report - 10/08/2014, 14:07:32.06Configured Interval Duration: 3 minutesActual Interval Duration: 3 minutes

TCP SMC-R traffic analysis for matching direct connections ----------------------------------------------------------Connections meeting direct connectivity requirements

50% of connections can use SMC-R (eligible) 67% of eligible connections are well-suited for SMC-R

79% of total traffic (segments) is well-suited for SMC-R 81% of outbound traffic (segments) is well-suited for SMC-R 75% of inbound traffic (segments) is well-suited for SMC-R

Interval Details: Total TCP Connections: 6 Total SMC-R eligible connections: 3

Total SMC-R well-suited connections: 2 Total outbound traffic (in segments) 274

SMC-R well-suited outbound traffic (in segments) 222 Total inbound traffic (in segments) 211

SMC-R well-suited inbound traffic (in segments) 159

Application send sizes used for well-suited connections: Size # sends Percentage ---- ------- ----------1500 (<=1500): 1 20% 4K (>1500 and <=4k): 1 20% 8K (>4k and <= 8k): 0 0% 16K (>8k and <= 16k): 0 0% 32K (>16k and <= 32k): 0 0% 64K (>32k and <= 64k): 1 20% 256K (>64K and <= 256K): 2 40% >256K: 0 0%

End of report

How much of my TCP workload can benefit from SMC-R?

What kind of CPU savings can I expect from SMC-R?




• None



INCREASE SINGLE STACK DVIPA LIMIT TO 4096




• Configuration defined dynamic virtual IP addresses

– VIPADEFINE

– VIPABACKUP

– VIPADISTRIBUTE target stacks

• Application instance dynamic virtual IP addresses

– VIPARANGE to define a range of IP addresses

– Application binds to an IP address

– Application issues an SIOCVIPA ioctl()



Business problem


– Continue to increase

– Need to follow the application

– Higher utilization

– CICS – dynamic virtual IP addresses for every region

• Systems and sysplexes

– Growing wider

– Horizonal workload growth



Solution


– Increase limit to 4096

• Dynamic virtual IP addresses defined with VIPADEFINE and VIPABACKUP

– Limit remains unchanged at 1024



Enablement actions

• None



Externals

• Existing message:

– EZZ8309I TOO MANY VIPAS - [ip address] REJECTED

• New message:

– EZD2030I TOO MANY VIPADEFINE AND VIPABACKUP VIPAS - [ip address] REJECTED

– Count includes both IPv4 and IPv6 dynamic virtual IP addresses




• None



REMOVED SUPPORT FOR LEGACY DEVICES



Background information: Legacy devices

• Configured to the TCP/IP stack by using DEVICE and LINK profile statements

• VTAM device drivers have these attributes:

– Support an attachment to “legacy” hardware that is based on:

– SSCH (CCWs) architecture

– ESCON channel hardware (z196

– is last to support ESCON)



Business problem

• Inability to test older or unsupported hardware

– Most hardware no longer exists

– Restricts product's exploitation of 64-bit storage

– Risk to support software for non-existent hardware

• Little or no customer usage of legacy devices

– zBLC and SHARE surveys and PMR analysis



Solution

• Remove supported for legacy DEVICE and LINK statements

– ATM

– CDLC

– CLAW

– HCH

– SNAIUCV SNALINK

– SNALU62

– X25NPSI

• Remove ZOSMIGV2R1_CS_LEGACYDEVICE Health Check




• Remove support for other profile statements

– Unsupported ATM related statements

– ATMARPSV, ATMLIS, and ATMPVC

– Unsupported TRANSLATE statement parameters

– NSAP (for ATM) and HCH

– Unsupported IPCONFIG statement parameters

– CLAWUSEDOUBLENOP and STOPONCLAWERROR

• Unsupported server applications

– SNALINK LU0 and LU6.2

– X.25 NPSI

– NCPROUTE



Enablement actions

• None



Externals

• Legacy device type DEVICE and LINK statements:

– EZZ0318I ATM WAS FOUND ON LINE 3 AND DEVICE TYPE WAS EXPECTED

– EZZ0318I ATM WAS FOUND ON LINE 4 AND LINK TYPE WAS EXPECTED

• ATM related statements:

– EZZ0324I UNRECOGNIZED STATEMENT ATMARPSV FOUND ON LINE 1

• TRANSLATE statement parameters:

– EZZ0318I NSAP WAS FOUND ON LINE 1 AND ETHERNET, IBMTR, OR FDDI WAS EXPECTED




• Migrate to strategic devices, such as OSA-Express QDIO and HiperSockets

• Update automation for unsupported server applications



VIPAROUTE FRAGMENTATION AVOIDANCE



Background information: Generic Resource Encapsulation

• Generic Routing Encapsulation header added for VIPAROUTE

– Additional header can cause fragmentation

• Ways to avoid fragmentation:

– Use path MTU discovery

– Use jumbo-frames between distributor and targets



Background information: Sysplex distributor with VIPAROUTE

IPv4 Delivery Header

GRE Header

Original IP Packet

GRE Encapsulation 20 bytes

4 bytes

LPAR1

SD

LPAR2

Target

LPAR3

Target

OSA OSA OSA

CPC1 CPC2

Hipersockets

XCF connectivity

MTU 1492

IP Packet IP Packet GRE IP

MTU 8092



Business problem

• Many benefits to enabling VIPAROUTE

• Fragmentation is a common problem

• Alternative options not always viable

– Firewalls can prevent Path MTU discovery from working

– Enabling Path MTU discovery on large number of clients can be problematic

– Enabling Jumbo frames requires reconfiguration



Solution

• Adjust TCP maximum segment size

– Connections being forwarded using VIPAROUTE

– Exchanged on TCP handshake

– TCP hosts cannot exceed the maximum segment size advertised by the peer

– Works across firewalls

– Sometimes referred to as maximum segment size clamping



Enablement actions: GLOBALCONFIG

• New GLOBALCONFIG parameter - ADJUSTDVIPAMSS

– Specified on all target stacks

– Specified on all stacks initiating outbound connections

– Implemented on the initial connection packet

– Done even if no fragmentation

– Outgoing connections: generic routing encapsulation might be used on the return path

– Incoming connections: Inbound routing paths can change over the life of a connection



Enablement actions: ADJUSTDVIPAMSS

• AUTO (default)

– Maximum segment size is adjusted for inbound connections if:

– Local stack is a target and VIPAROUTE is being used

– Local stack is both a distributor and a target and VIPAROUTE is defined

– Maximum segment size is adjusted for outbound connections if:

– Source IP address is a distributed dynamic virtual IP address

• ALL

– Maximum segment size is adjusted for all connections where

– Source IP address is a dynamic virtual IP address

– Both distributed and non-distributed

• NONE

– Maximum segment size is not adjusted for any connections



Externals: netstat CONFIG/-f • A sample netstat config/-f display command is shown below



Externals: netstat ALL/-A • A sample netstat ALL/-A display command is shown below

– MaximumSegmentSize displays the maximum segment size value



Externals: NMI, SMF, IPCS • TCP/IP callable NMI

– GetProfile request output provides values for new parameters

• SMF 119 records

– Subtype 4 TCP/IP profile record provides values for new parameters

• TCPIPCS command

– The TCPIPCS PROFILE command displays the values for the new parameters




• Preserving existing behavior

– Code GLOBALCONFIG ADJUSTDVIPAMSS NONE



TCP AUTONOMIC TUNING



Background information: Dynamic Right-Sizing (DRS)

• Dynamic right-sizing (DRS) introduced in V1R11

– Automatically increases the receive window size beyond

the “maximum” window size for qualifying connections

– Goal is to keep more data moving in the network

– Receiving application must be able to keep pace with

incoming data

Window size

Round trip time (RTT)

Sender Receiver

data

Time

ACK



Background information: Dynamic Outbound Right-Sizing (ORS)

• Automatic attempt to deal with FRR (Fast Retransmit and Fast Recovery) impacts to streaming

workloads

– Outbound data becomes serialized to reduce risk of “out of order” packets

– Send buffer size is allowed to grow to 1MB to keep value greater than the congestion window

– FRR is suppressed when possible

– Write-blocked applications are resumed sooner



Business problem: DRS

• Disabled if receiving application is unable to keep up with data arrival

– Never turned back on for the life of the connection

• Storage status not taking into consideration

• DRS eligibility is only determined once during the initial phase of the connection



Business problem: ORS

• Solution targeted for a very narrow set of connections

– RTT must be 20 ms or more

– TCPCONFIG QUEUEDRTT operand created in V2R1

• Send buffer size grows with no consideration of receiver status

– Once increased, send buffer size never shrinks



Solution: DRS

• Allow DRS usage to be restarted on a connection

– DRS detection can be re-initiated after a certain number of packets are processed

• When CSM storage is not constrained:

– Continue using DRS on a connection even if the application falls behind

• When CSM storage is constrained:

– If application falls behind, stop DRS on the connection temporarily

– Do not activate DRS for connection, either initially or during “restart conditions”



Solution: ORS

• Altered logic for growing the send buffer size

– Only increase when current send buffer size is almost constrained

– Do not increase send buffer size if retransmitting

– Do not increase send buffer size when CSM storage constrained

• Allow send buffer size to shrink dynamically

– Determining factor is whether the sender is actually filling, or almost filling, the buffer

• RTT requirement matches DRS value

– 2 milliseconds



Solution: Autonomic outbound serialization

• Change TCPCONFIG QUEUEDRTT default to 0

– Allow outbound serialization for all TCP connections

• Connection must have a send buffer size of 64K or larger

• Connection must be experiencing out of order packets



Enablement actions

• None




• TcpPrf and TcpPrf2 indicate status of DRS and ORS

MVS TCP/IP NETSTAT CS V2R1 TCPIP Name: TCPCS 22:24:30 Client Name: FTPD1 Client Id: 000000F9 Local Socket: 9.42.104.43..21 Foreign Socket: 9.42.103.165..1035 BytesIn: 0000000035 BytesOut: 0000000265 SegmentsIn: 0000000017 SegmentsOut: 0000000014 StartDate: 01/09/2012 StartTime: 22:04:11 Last Touched: 22:04:18 State: Establsh RcvNxt: 0214444666 SndNxt: 0216505563 ... MaximumSegmentSize: 0000000524 DSField: 00 Round-trip information: Smooth trip time: 102.000 SmoothTripVariance: 286.000 ReXmt: 0000000000 ReXmtCount: 0000000000 DupACKs: 0000000000 RcvWnd: 0000032730 SockOpt: 85 TcpTimer: 00 TcpSig: 84 TcpSel: 60 TcpDet: E0 TcpPol: 00 TcpPrf: C0 TcpPrf2: 70 QOSPolicy: No ...




• QUEUEDRTT default changed to 0

– Specify TCPCONFIG QUEUEDRTT 20 to retain the default behavior

– Best practice is to use the new default value of 0

• Netstat All / -a

• SMF type 119 records



Background information: FRR

• Fast Recovery and Retransmit (FRR) allows a given TCP connection to continue sending new packets even

as it is attempting to retransmit un-acknowledged packets

– Triggered upon receipt of certain number of duplicate ACKs

– Causes application's “slow start threshold” and “congestion window” values to be reduced

• Purpose is to recovery from lost packets without waiting for retransmit timeout to occur

• “FRR ambiguity” modifies the duplicate ACKs threshold

– Requires that timestamps be included in the TCP packets

– TCP uses timestamps in retransmitted packet and received ACK



Business problem: FRR for out of order packets

• When “basic” FRR recovery is performed, the application cannot ramp back up to previous transmission rate

– Permanent decrease in the growth rate of the congestion window

• “FRR ambiguity” helps, but has its own problems

– Requires timestamps to be present, so not universally available

– Manipulation of FRR suppression threshold can mask real problems



Solution: FRR tuning

• Utilize internal timestamps when timestamps in packets are not available

• Modified FRR algorithm to be less punitive for out of order packets

– Restore congestion window and slow start threshold

– Eliminate FRR suppression logic so that FRR is performed after three duplicate ACKs

– Lost packet behavior is unchanged



Enablement actions

• None



Externals

• None




• None



Background information: Delay ACK

• TCP/IP will delay before sending an ACK until:

– Have a response to send

– Receive two packets from sender

– 200 ms has expired

• Default is to delay ACKs but numerous controls exist today to set or prevent delay ACK processing

– TCPCONFIG DELAYAcks|NODELAYAcks

– PORT(RANGE) DELAYAcks|NODELAYAcks

– Various statements used to configure a route used by a TCP connection



Business problem

• Delayed ACK processing generally works very well

– Significant saving in request/response workloads

– Likewise in streaming workloads

• Occasionally, a workload incurs significant performance penalties because of delayed ACKs

– Sender waiting for an ACK before sending the next packet (200 ms delay is incurred)

– Can often occur due to interactions with Nagle's algorithm

– Often hard to diagnose this delay



Solution: Autonomic delay ACK

• Provide autonomic controls to monitor effectiveness of delay ACK processing

• Do not delay sending the ACK if it repeatedly prevents the partner from sending more data

• Do not keep sending ACKs to every packet if the sender is sending its next packet anyway

Data From Sender

ACK

Data From Sender

X??



Enablement actions

• Enhance TCPCONFIG statement to request autonomic delayed ACK processing

– New parameter AUTODELAYAcks to request autonomic delayed ACKs

– Default remains DELAYAcks

– AUTODELAYAcks is voided if DELAYAcks | NODELAYAcks is specified on any configuration statement related to this

connection

.-------------------------------. V | TCPCONFIG -------.--------------------------.--'------->< | _DELAYAcks_____ | |______|_______________|___| | | |_NODELAYAcks___| |_AUTODELAYAcks_|




• Also include new AUTODELAYAcks setting in NMI and SMF configuration/profile reports

NETSTAT CONFIG MVS TCP/IP NETSTAT CS V2R1 TCPIP Name: TCPCS 11:37:31 TCP Configuration Table: DefaultRcvBufSize: 00016384 DefaultSndBufSize: 00016384 DefltMaxRcvBufSize: 00262144 SoMaxConn: 0000001024 MaxReTransmitTime: 120.000 MinReTransmitTime: 0.500 RoundTripGain: 0.125 VarianceGain: 0.250 VarianceMultiplier: 2.000 MaxSegLifeTime: 30.000 DefaultKeepALive: 00000120 DelayAck: Auto RestrictLowPort: Yes SendGarbage: No TcpTimeStamp: Yes FinWait2Time: 010 TTLS: No EphemeralPorts: 1024-65535 SelectiveACK: Yes TimeWaitInterval: 30 DefltMaxSndBufSize 262144 RetransmitAttempt: 15 ConnectTimeOut: 0120 ConnectInitIntval: 1000 KeepAliveProbes: 10 KAProbeInterval: 060 Nagle: No QueuedRTT: 20 FRRThreshold: 3



Externals: Netstat ALL/-A NETSTAT CONFIG NETSTAT ALL MVS TCP/IP NETSTAT CS V2R1 TCPIP Name: TCPCS 22:24:30 Client Name: FTPD1 Client Id: 000000F9 Local Socket: 9.42.104.43..21 Foreign Socket: 9.42.103.165..1035 BytesIn: 0000000035 BytesOut: 0000000265 SegmentsIn: 0000000017 SegmentsOut: 0000000014 StartDate: 01/09/2012 StartTime: 22:04:11 Last Touched: 22:04:18 State: Establsh RcvNxt: 0214444666 SndNxt: 0216505563 ... MaximumSegmentSize: 0000000524 DSField: 00 Round-trip information: Smooth trip time: 102.000 SmoothTripVariance: 286.000 ReXmt: 0000000000 ReXmtCount: 0000000000 DupACKs: 0000000000 RcvWnd: 0000032730 SockOpt: 85 TcpTimer: 00 TcpSig: 84 TcpSel: 60 TcpDet: E0 TcpPol: 00 TcpPrf: C0 TcpPrf2: 70 TcpPrf3: 00 DelayAck: AutoYes QOSPolicy: No ...

AutoYes AutoNo Yes No




• Netstat ALL / -a and CONFIG / -f



• Simplified access permissions to ICSF cryptographic functions for IPSec • TCP/IP profile IP security filter enhancements • AT-TLS certificate processing enhancements • TLS session reuse support for FTP and AT-TLS applications • AT-TLS enablement for DCAS • TLS security enhancements for sendmail • TLS security enhancements for policy agent • Network security enhancements for SNMP

Security



SIMPLIFIED ACCESS PERMISSIONS TO ICSF CRYPTOGRAPHIC FUNCTIONS FOR IPSEC



Background information: ICSF

• Integrated Cryptographic Services Facility (ICSF)

– Primary cryptographic provider on z/OS, including many crypto algorithms and access to all z Systems hardware

crypto features

– Offers a FIPS 140 mode through its PKCS#11 interface

– SAF CSFSERV class resources control access to ICSF's many callable services

– When CSFSERV class defined and CHECKAUTH(YES) specified in ICSF options dataset

– Calling user ID must have READ permission to a SAF profile that covers the resource protecting the given callable service



Background information: IPSec and ICSF

• TCP/IP stack's IPSec support

– Uses 24 different ICSF callable services (both FIPS and non-FIPS mode) to perform many cryptographic operations

– Often runs under the SAF credentials (ACEE) of the calling application (most commonly for send operations)

– Therefore, IPSec operations run under caller's ACEE

– As a result, in some cases, the user ID under which any application generates IPSec-protected traffic must be

permitted to appropriate CSFSERV resources



Business problem

• When CSFSERV class is defined and CHECKAUTH(YES) is specified in the ICSF options data set, every

user ID under which IPSec-protected traffic is generated must be permitted to a long list of CSFSERV

resources (in addition to permitting the TCP/IP stack's user ID)

• Since the stack operates on behalf of the application and associated user ID, it makes sense that the

TCP/IP stack's permissions to those resources should be sufficient

• Prior to V2R2, ICSF did not provide a way for a service provider like the TCP/IP stack to specify the

credentials under which the ICSF callable service should execute, so the stack had no way to avoid the

issue

• Note that CHECKAUTH(YES) tells ICSF to perform access control checks for supervisor state and system

key callers – both of which describe the TCP/IP stack. The problem scenario does not exist if

CHECKAUTH(NO) is in effect – and this is the default value.



Solution

• In V2R2, ICSF provides a new CSFACEE (and CSFACEE6, for 64-bit) function that allows an authorized

caller (either system key or supervisor state) to provide a SAF ENVR structure to use in place of the default

ACEE for SAF checks

• The TCP/IP stack's IPSec support is updated to use this new interface

– Means that all ICSF calls within the TCP/IP stack can now be made under the TCP/IP stack's credentials instead of the

calling application's

– Covers both FIPS 140 and non-FIPS 140 mode

• As a result, customers that use CHECKAUTH(YES) can eliminate all of the application-specific permissions

to ICSF resources that were previously required due to IPSec protection. (Since the stack's user ID already

required the same permissions, there are no additional permissions that need to be defined).



Enablement actions

• None



Externals

• None




• Optional action

– Customers who have permitted application user IDs to CSFSERV resources because of IPSec protection can choose

to remove those permissions

– This is not mandatory – just a “clean up” and simplification task since the TCP/IP stack's user ID already must have the

same permissions

– Note that any new IPSec-generating applications do not have to be permitted to CSFSERV resources



TCP/IP PROFILE IP SECURITY FILTER ENHANCEMENTS



Background information: IP filtering

• IP filters permit traffic, deny traffic, or require that it be protected with IPSec

SrcIP

Inbound or outbound packet

DstIP Proto SrcPort DstPort SrcIP DstIP Proto SrcPort DstPort Action

IP filter table in stack

SrcIP DstIP Proto SrcPort DstPort Action



DENY All other traffic

First filter to match Action is performed

An implied “deny all” rule always exists at the bottom of the filter list



Background information: IP filter configuration

• IP security is enabled in the TCP/IP profile:

– IPSECURITY on IPCONFIG statement

– IPSECURITY on IPCONFIG6 statement, to enable for IPv6 traffic

• Default IP filters are defined in the TCP/IP profile on the IPSEC statement

– Provides limited filtering capability

– Protects the TCP/IP stack during initialization until Policy Agent installs an IPSec policy

– Provides a “lockdown” option (ipsec -f default)

• Policy IP filters are defined in an IPSec policy that is installed by Policy Agent

– Provides full filtering and IPSec capability



Background information: Default IP filter policy

• Used to permit a limited set of traffic

• All traffic that is not explicitly permitted is denied

• Traffic selection parameters for default filter rules are more limited than traffic descriptions provided for

policy rules

– For example, a range of ports cannot be specified

• Address ranges are not supported for the source and destination address

• All rules are bidirectional



Business problem: Configuration Assistant TCP/IP profile support

• Configuration Assistant (CA) in V2R2 introduces TCP/IP profile support

– Includes default filter rules

• CA allows reusable object traffic descriptors to be defined for IPSec policies

• Default filter rules do not support all traffic descriptor options provided for policy filter rules

• CA profile support unable to share reusable objects defined for IPSec policies



Solution: Enhance default filter rules

• Source and destination IP address ranges supported for default filter rules

• Additional traffic descriptor granularity provided for default filter rules

– Source and destination port ranges (for TCP and UDP protocols)

– Type and Code ranges (for ICMP and ICMPv6 protocols)

– MIPv6 Type (single, all, and range)

– OPAQUE protocol

– Direction – inbound, outbound, or bidirectional

– For bidirectional rules, TCP inbound or outbound connect qualifier

– FragmentsOnly



Solution: Difference between default and policy filters

• Certain features remain available only using policy definitions

– Action = Permit with IPSec protection

– Action = Deny

– Time constraints



Enablement actions: TCP/IP profile IPSECRULE/IPSEC6RULE

|--IPSECRule--+-src_ipaddr---------------------+--------------------> +-src_ipaddr/prefix_length-------+ +-src_ipaddr - src_ipaddr--------+ '-*------------------------------' .-NOLOG-. >--+-dest_ipaddr----------------------+--+-------+--| Protocol |----> +-dest_ipaddr/prefix_length--------+ '-LOG---' +-dest_ipaddr - dest_ipaddr--------+ '-*--------------------------------' .-ROUTING LOCAL------------------------. .-SECCLASS 0-------------. >--+--------------------------------------+--+------------------------+-> | | '-SECCLASS securityclass-' '-ROUTING---ROUTED-+-----------------+-+ | '-FRAGMENTSonly---' | '-EITHER--------------------' .-DIRECtion BIDIrectional-------------------. >--+-------------------------------------------+--------------------| '-DIRECtion-+-INBound-----------------------+ +-OUTBound----------------------+ '-BIDIrectional-+-------------+-+ +-INBConnect--| '-OUTBConnect-'



Enablement actions: IPv4 rule protocol

Protocol .-PROTOcol *--------------------------------------------------. |--+-------------------------------------------------------------+--| | .-SRCPort *-----. .-DESTport *-----. | '-PROTOcol--+-+-TCP-+--+---------------+--+----------------+--+ | +-6---+ '-SRCPort-+-n---+ '-DESTport-+-n---+ | | +-UDP-+ '-n m-' '-n m-' | | '-17--' | | .-TYPE *-----. .-CODE *-----. | +-+-ICMP-+--+------------+--+------------+--------+ | '-1----' '-TYPE-+-n---+ '-CODE-+-n---+ | | '-n m-' '-n m-' | | .-TYPE *--------. | +-+-OSPF-+--+---------------+---------------------+ | '-89---' '-TYPE ospftype-' | '-protocol_number---------------------------------'



Enablement actions: IPv6 rule protocol

Protocol .-PROTOcol *--------------------------------------------------. |--+-------------------------------------------------------------+--| | .-SRCPort *-----. .-DESTport *-----. | '-PROTOcol--+-+-TCP-+--+---------------+--+----------------+--+ | +-6---+ '-SRCPort-+-n---' '-DESTport-+-n---' | | +-UDP-+ '-n m-' '-n m-' | | '-17--' | | .-TYPE *-----. .-CODE *-----. | +-+-ICMPV6-+--+------------+--+------------+------+ | '-58-----' '-TYPE-+-n---+ '-CODE-+-n---+ | | '-n m-' '-n m-' | | .-TYPE *--------. | +-+-OSPF-+--+---------------+---------------------+ | '-89---' '-TYPE ospftype-' | | .-TYPE *-----. | +-+-MIPV6-+-+------------+------------------------+ | '-135---' '-TYPE-+-n---+ | | '-n m-' | +-OPAQUE------------------------------------------+ '-protocol_number---------------------------------'



Enablement actions: Sample TCP/IP profile updated

; Use this rule to permit all outbound and ; and inbound IPv6 Neighbor Solicitations ; and Neighbor Advertisements ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 135 136 ; ; ; ; Use this rule to permit outbound IPv6 Router ; Solicitations ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 133 DIREC OUTB ; ; Use this rule to permit inbound IPv6 Router ; Advertisements ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 134 DIREC INB ; ; Use this rule to permit outbound MLD ; listener reports ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 131 DIREC OUTB ; ; Use this rule to permit inbound MLD listener ; queries ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 130 DIREC INB

; Use this rule to permit all outbound and ; and inbound IPv6 Neighbor Solicitations ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 135 ; ; Use this rule to permit all outbound and ; inbound IPv6 Neighbor Advertisements ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 136 ; ; Use this rule to permit outbound IPv6 Router ; Solicitations ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 133 ; ; Use this rule to permit inbound IPv6 Router ; Advertisements ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 134 ; ; Use this rule to permit outbound MLD ; listener reports ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 131 ; ; Use this rule to permit inbound MLD listener ; queries ; IPSEC6R * * LOG PROTO ICMPV6 TYPE 130



Externals: ipsec command

• ipsec command filter output remains unchanged

• For a default filter rule, some fields might now display data (vs. n/a)

FilterName: SYSDEFAULTRULE.7 . . . Action: Permit Scope: Local Direction: Outbound . . . Protocol: UDP(17) . . . SourceAddress: 69.82.90.193 SourceAddressPrefix: n/a SourceAddressRange: n/a SourceAddressGranularity: n/a SourcePort: 13721 SourcePortRange: 54198 SourcePortGranularity: n/a DestAddress: 96.154.72.193 DestAddressPrefix: n/a DestAddressRange: 100.83.2.20 DestAddressGranularity: n/a . . .




• None



AT-TLS CERTIFICATE PROCESSING ENHANCEMENTS



Background information: AT-TLS overview

• Basic TCP/IP stack-based TLS

– AT-TLS policy specifies which TCP traffic is to be TLS protected based on a variety of criteria

– Created via Configuration Assistant

– TLS process performed at TCP layer without requiring any application change

• Available to TCP applications

– Includes CICS sockets

– Supports all programming languages except PASCAL

• Application transparency

– An optional API allows applications to inspect or control certain aspects of AT-TLS processing

– Application-aware and Application-controlled AT-TLS

– Can be fully transparent to application

Network Interfaces

IP Networking Layer

TCP

Sockets

Applications

System SSL calls

Encryp-ted

Optional APIs for TLS-aware applications to control start/stop of TLS session

Policy Agent

ApplicationTransparent TLS policy flat file

Enabling most z/OS TCP-based applications for use of

SSL/TLS without requiring any modifications to those applications on z/OS.





Background information: AT-TLS overview (continued)

• Supports standard configurations

– z/OS as a client or as a server

– Server authentication (server identifies self to client)

– Client authentication (both ends identify selves to other)

• AT-TLS uses System SSL

– System SSL implements standard SSL/TLS protocols

– Remote endpoint sees an RFC-compliant implementation

– Inter operates with other compliant implementations

Network Interfaces

IP Networking Layer

TCP

Sockets

Applications

System SSL calls

Encryp-ted

Optional APIs for TLS-aware applications to control start/stop of TLS session

Policy Agent

ApplicationTransparent TLS policy flat file







Background information: X.509 Digital Certificate

• Public keys can be freely disseminated

– Require a systematic and trustworthy way of distributing public keys and securely storing associated private keys

• X.509 digital certificate is the packaging that enables the distribution of a single public key

– A data structure that contains multiple fields

– A binding between a named entity (a person or device) and a public key

• Can be issued by certificate authorities (CA) or self-issued

– CAs can be commercial organizations or internal organizations

– Self-signed Certificate

– Organization issues its own certificate with itself as subject and issuer

– Assigned a validity period



Background information: Certificate Revocation Lists (CRLs)

• Certificates revoked by the issuing Certificate Authority (CA) before expiration

– Can be revoked for any number of reasons

– The reason for revocation is stored in a REASON field within the CRL

• CRL is a list of revoked certificates that have been issued and subsequently revoked by a given Certificate

Authority

– Signed by the owning CA to ensure the authenticity of the CRL contents

– Has a start and end (expiration) date and time

– Revoked certificates represented by their serial numbers

• Common methods for CRLs storing and retrieving

– LDAP directory

– HTTP server

– URL values in the CRL Distribution Point (CDP) extension



Background information: Online Certificate Status Protocol (OCSP)

• HTTP-based protocol for checking the revocation status of a certificate

• Uses a request/response model

• Puts less burden on network and client resources

– Response contains less information than a typical CRL

– Certificate user does not have to search for a serial number

• Certificate's Authority Information Access (AIA) extension contains URL for OCSP Responder

• Allows more timely enforcement of certificate revocation

– OCSP server might have realtime access into the certificate issuer's certificate status database



Business problem: AT-TLS missing support for RFC 5280

• System SSL provided support for RFC 5280 in z/OS V2R1

– AT–TLS does not exploit the support

• Enhance AT-TLS to exploit RFC 5280 Certificate Validation

– New value on CertValidationMode parameter on TTLSEnvironmentAdvancedParms statement

• pasearch -t and Netstat TTLS / -x Conn reports display the new value

+-CertValidationMode Any----------+ >--+---------------------------------+--------------------------> '-CertValidationMode--+-Any-----+-+ +-RFC2459-+ +-RFC3280-+ +-RFC5280-+



Business problem: Limitations in LDAP support for CRLs

• CRLs for an SSL application must reside in the same single LDAP directory

• Entire cache flushed when the GSK_CRL_CACHE_TIMEOUT value is reached

• CRL sizes can be extremely large requiring substantial amount of storage and processing overhead

– Retrieving large CRLs may result in network congestion

– Searching for serial numbers in large list

• Provides only periodic information



Solution: Enhanced certificate revocation support

• System SSL revocation support enhanced

– Supports retrieval of revocation information through Online Certificate Status Protocol (OCSP)

– Supports HTTP retrieval of CRLs

– Provides more flexible processing of CRLs from LDAP

– CRL removed from cache based on expiration

– Configuration of maximum number entries allowed in cache

– Configuration of the maximum CRL entry size allowed

– Configuration of the LDAP Timeout value

– Configure whether temporary CRLs are added to the cache

– Configure the lifetime of the temporary CRL in the cache

• AT-TLS exploits the new System SSL revocation support



Enablement actions: TTLSGskAdvancedParms updated >>--TTLSGskAdvancedParms-+------+--| Put Braces and Parameters on Separate Lines |--><

+-name-+

Put Braces and Parameters on Separate Lines:

|--+--{------------------------------------+-----------------------------------------|

+--| TTLSGskAdvancedParms Parameters |--+

+--}------------------------------------+

TTLSGskAdvancedParms Parameters:

|--+----------------------------+---+----------------------------+------------------->

+--TTLSGskLdapParms----------+ +--TTLSGskOcspParms----------+

+--TTLSGskLdapParmsRef name--+ +--TTLSGskOcspParmsRef name--+

>--+-------------------------------+---+--------------------------------+------------>

+--TTLSGskHttpCdpParms----------+ +--GSK_SYSPLEX_SIDCACHE-+-On--+--+

+--TTLSGskHttpCdpParmsRef name--+ +-Off-+



Enablement actions: LDAP policy configuration updated TTLSGskLdapParms Parameters:

>--+------------------------------+--+-------------------------------+-------------------->

+--GSK_LDAP_SERVER_PORT value--+ +--GSK_CRL_CACHE_TIMEOUT value--+

>--+-------------------------------------+------------------------------------------------>

+--GSK_CRL_SECURITY_LEVEL--+-Low----+-+

+-Medium-+

+-High---+

>--+----------------------+-----------+------------------------------+-------------------->

+--CRLCacheSize value--+ +--CRLCacheEntryMaxsize value--+

>--+-----------------------------+----+------------------------------+-------------------->

+--CRLCacheExtended--+-On--+--+ +--CRLCacheTempCRL--+-On--+----+

+-Off-+ +-Off-+

>--+--------------------------------+--+-----------------------------+-------------------><

+--CRLCacheTempCRLTimeout value--+ +--LDAPResponseTimeout value--+



Enablement actions: OCSP configuration >>--TTLSGskOcspParms-+------+--| Put Braces and Parameters on Separate Lines |-----------><

+-name-+


|--+--{--------------------------------+--------------------------------------------------|

+--| TTLSGskOcspParms Parameters |--+

+--}--------------------------------+

TTLSGskOcspParms Parameters:

+--OcspAiaEnable Off-------+

|--+---------------+----------------------+--------------------------+-------------------->

+--OcspUrl url--+ +--OcspAiaEnable--+-On--+--+

+-Off-+

+--OcspRetrieveViaGet Off--------+ +--OcspUrlPriority On-------+

>--+--------------------------------+-----+---------------------------+------------------->

+--OcspRetrieveViaGet--+-On--+--+ +--OcspUrlPriority--+-On--+--+

+-Off-+ +-Off-+



Enablement actions: OCSP configuration (continued) +--OcspProxyServerPort 80----+

>--+--------------------------------+-----+----------------------------+------------------>

+--OcspProxyServerName hostname--+ +--OcspProxyServerPort port--+

+--OcspRequestSigalg 0401-------+

>--+--------------------------------+-----+-------------------------------+--------------->

+--OcspRequestSigkeylabel label--+ +--OcspRequestSigalg algorithm--+

+--OcspClientCacheSize 256-------+ +-OcspCliCacheEntryMaxsize 0-----+

>--+--------------------------------+-----+--------------------------------+-------------->

+--OcspClientCacheSize size------+ +-OcspCliCacheEntryMaxsize size--+



Enablement actions: OCSP configuration (continued) +--OcspNonceGenEnable Off-------+ +--OcspNonceCheckEnable Off-------+

>--+-------------------------------+----+---------------------------------+---------------->

+--OcspNonceGenEnable--+-On--+--+ +--OcspNonceCheckEnable--+-On--+--+

+-Off-+ +-Off-+

+--OcspNonceSize 8----+ +--OcspResponseTimeout 15-----+

>--+----------------------+-------------+-----------------------------+-------------------->

+--OcspNonceSize size--+ +--OcspResponseTimeout value--+

+--OcspMaxResponseSize 20480-+

>--+----------------------------+---------------------------------------------------------|

+--OcspMaxResponseSize size--+



Enablement actions: HTTP CDP policy configuration >>--TTLSGskHttpCdpParms-+------+--| Put Braces and Parameters on Separate Lines |--------><

+-name-+


|--+--{-----------------------------------+-----------------------------------------------|

+--| TTLSGskHttpCdpParms Parameters |--+

+--}-----------------------------------+

TTLSGskHttpCdpParms Parameters:

+--HttpCdpEnable Off------+

|--+--------------------------+----------------------------------------------------------->

+--HttpCdpEnable--+-On--+--+

+-Off-+



Enablement actions: HTTP CDP policy configuration (continued) +--HttpCdpProxyServerPort 80----+

>--+-----------------------------------+-----+-------------------------------+------------>

+--HttpCdpProxyServerName hostname--+ +--HttpCdpProxyServerPort port--+

+--HttpCdpResponseTimeout 15-----+ +--HttpCdpMaxResponseSize 204800---+

>--+--------------------------------+--------+----------------------------------+--------->

+--HttpCdpResponseTimeout value--+ +--HttpCdpMaxResponseSize size-----+

+--HttpCdpCacheSize 32----+ +--HttpCdpCacheEntryMaxsize 0-----+

>--+-------------------------+---------------+---------------------------------+----------|

+--HttpCdpCacheSize size--+ +--HttpCdpCacheEntryMaxsize size--+



Enablement actions: TTLSGskAdvancedParms updated

• New global parameters that apply to both OCSP and HTTP

>--+--------------------------------+--+------------------------------+-------------->

+--GSK_V2_SESSION_TIMEOUT value--+ +--GSK_V2_SIDCACHE_SIZE value--+

>--+--------------------------------+--+------------------------------+-------------->

+--GSK_V3_SESSION_TIMEOUT value--+ +--GSK_V3_SIDCACHE_SIZE value--+

>--+--------------------------+--------+-------------------------------+------------->

+--AIACDPPriority-+-On--+--+ +--MaxSrcRevExtLocValues value--+

+-Off-+

>--+---------------------------------+--+--------------------------------------+----->

+--MaxValidRevExtLocValues value--+ +--RevocationSecurityLevel--+-Low----+-+

+-Medium-+

+-High---+



Externals: pasearch -t policyRule: Secure_Telnet_23_Debug

Rule Type: TTLS

Version: 3 Status: Active

:

TTLSEnvironmentAdvancedParms:

SSLv2: Off

SSLv3: On

:

TruncatedHMAC: Off

CertValidationMode: RFC5280

ServerMaxSSLFragment: Off

TTLSGskAdvancedParms:

GSK_CRL_SECURITY_LEVEL Medium

AIACDPPriority On

MaxSrcRevExtLocValues 10

MaxValidRevExtLocValues 100

RevocationSecurityLevel Medium



Externals: pasearch –t (continued) TTLSGskLdapParms:

CRLCacheSize 32

CRLCacheEntryMaxsize 0

CRLCacheExtended On

CRLCacheTempCRL On

CRLCacheTempCRLTimeout 24

LDAPResponseTimeout 30

TTLSGskOcspParms:

OcspUrl http://184.31.92.190...

OcspAIAEnable Off

OcspProxyServerName ocsp.entrust.net

OcspProxyServerPort 80

OcspRetrieveViaGet Off

OcspUrlPriority On



Externals: pasearch –t (continued) OcspRequestSigkeylabel signcert

OcspRequestSigalg 0401 TLS_SIGALG_SHA256_WITH_RSA

OcspClientCacheSize 256

OcspCliCacheEntryMaxsize 0

OcspNonceGenEnable Off

OcspNonceCheckEnable Off

OcspNonceSize 0

OcspResponseTimeout 30

OcspMaxResponseSize 20480



Externals: pasearch –t (continued) TTLSGskHttpCdpParms:

HttpCdpEnable On

HttpCdpProxyServerName 23.57.107.27

HttpCdpProxyServerPort 80

HttpCdpResponseTimeout 30

HttpCdpMaxResponseSize 204800

HttpCdpCacheSize 32

HttpCdpCacheEntryMaxsize 0



Externals: Netstat TTLS/-x COnn ConnID: 000000B8

JobName: FTPD1

LocalSocket: ::ffff:127.0.0.1..21

RemoteSocket: ::ffff:127.0.0.1..1030

SecLevel: TLS Version 1.2

Cipher: C001 TLS_ECDH_ECDSA_WITH_NULL_SHA

CertUserID: N/A

MapType: Primary

FIPS140: Off

:

TTLSEnvAction: env_act_serv

EnvironmentUserInstance: 8

:

ClientAuthType: Required

CertValidationMode: RFC5280

Renegotiation: Default



Externals: Netstat TTLS/-x COnn (continued) GSK_CRL_SECURITY_LEVEL Medium

AIACDPPriority: On

MaxSrcRevExtLocValues: 10

MaxValidRevExtLocValues: 100


CRLCacheSize 32


CRLCacheExtended On

CRLCacheTempCRL On



OcspUrl http://184.31.92.190...

OcspAIAEnable Off






Externals: Netstat TTLS/-x COnn (continued) OcspUrlPriority On

OcspRequestSigkeylabel signcert

OcspRequestSigalg 0401 TLS_SIGALG_SHA256_WITH_RSA

OcspClientCacheSize 256

OcspCliCacheEntryMaxsize 0

OcspNonceGenEnable Off

OcspNonceCheckEnable Off

OcspNonceSize 0

OcspResponseTimeout 30

OcspMaxResponseSize 20480

HttpCdpEnable On

HttpCdpProxyServerName 23.57.107.27

HttpCdpProxyServerPort 80

HttpCdpResponseTimeout 30

HttpCdpMaxResponseSize 204800

HttpCdpCacheSize 32

HttpCdpCacheEntryMaxsize 0




• None



TLS SESSION REUSE SUPPORT FOR FTP AND AT-TLS APPLICATIONS



Background information: Handshake

• The general SSL handshake protocol and flows are depicted below. This process is computationally

expensive due to the digital signature operations.

ClientHello Certificate* ClientKeyExchange CertificateVerify* [ChangeCipherSpec] Finished Application Data

ServerHello Certificate* ServerKeyExchange* CertificateRequest* ServerHelloDone [ChangeCipherSpec] Finished Application Data

* Denotes optional or situation dependent messages that are not always sent



Background information: Handshake (continued)

• An SSL session contains the cryptographic characteristics, such as the cipher suite, keys, and so forth. The

session is identified by the SSL Session ID (SID)

– The first time a client and server connect, the SID of this connection is saved into a Session Cache entry

• SSL Session Reuse

– By reusing the SID of the previous SSL session, an abbreviated SSL handshake can be used. Thus, SSL Session

Reuse allows secure connections between a client/server pair to be established more quickly once the first SSL

handshake has completed

ClientHello [ChangeCipherSpec] Finished Application Data

ServerHello [ChangeCipherSpec] Finished Application Data



Background information: AT-TLS • Stack-based TLS

– TLS process performed in TCP layer (via System SSL) without requiring any application change

(transparent)

– AT-TLS policy specifies which TCP traffic is to be TLS protected based on a variety of criteria

• Application transparency

– Can be fully transparent to application

– An optional API allows applications to inspect (“application-aware”) or even or control (“application-controlled”)

certain aspects of AT-TLS processing

• Available to TCP applications

– Includes CICS Sockets

– Supports all programming languages except PASCAL

• Supports standard configurations

– Server authentication (server identifies self to client)

– Client authentication (both ends identify selves to other)

NetworkingIPv4, IPv6

DLC

Transport (TCP)

Sockets API

TCP/IP Application

AT-TLS

AT-TLSpolicy

AT-TLS policy administratorusing Configuration Assistant

System SSL



Background information: AT-TLS (continued)

• AT-TLS provides an ioctl to allow applications to query the AT-TLS settings for a connection and to control

the AT-TLS behavior for a connection

• AT-TLS supports the SIOCTTLSCTL ioctl requests of TTLS_QUERY_ONLY, TTLS_INIT_CONNECTION,

TTLS_STOP_CONNECTION and so on

• On a SIOCTTLSCTL ioctl request, an application can also provide a TTLS extension buffer, which contains

a header and additional “get” requests

• While the architecture allows for “set” requests, none have been defined to date



Background information: Securing FTP with TLS

• To prevent a man in middle attack, RFC 4217 recommends that the certificate used for server authentication

of Data connections be the same certificate as that used for the corresponding Control connection

• z/OS FTP has implemented the FTP.DATA statement TLSCERTCROSSCHECK to adopt this

recommendation in both the FTP client and server

Control connection to port 21

Data connection to/from port ???

FTP ClientFTP Server

Client/Server authentication for control connection

Client/Server authentication for data connectionX

How do you know the Client/Server for the data connection are the same as the

control connection?



Business problem: TLS session reuse not supported on different ports

• RFC 4217 (Securing FTP with TLS) contains the following statement:

“It is reasonable for the server to insist that the data connection uses a TLS cached session. This might be a cache of

a previous data connection or of a cleared control connection.”

• Adhering to this advice can dramatically reduce the computational cost of SSL handshakes and can also

ensure the certificate of the FTP data connections and the corresponding FTP control connection are the

same

• However, the TLS session reuse is not supported by z/OS FTP. This is because z/OS System SSL sessions

are bound to specific TCP ports while the FTP data connection requires a different port from the port for the

FTP control connection



Solution: Enable TLS session reuse on different ports

• z/OS System SSL is enhanced in V2R2 to

– allow sessions to be reused across different TCP ports

– allow an SID to be specified before starting an SSL handshake

• The z/OS FTP client and server are enhanced in V2R2 to exploit the new System SSL capability for both

TLSMECHANISM FTP and TLSMECHANISM TTLS

• The AT-TLS SIOCTTLSCTL interface is enhanced in V2R2 to expose the new System SSL capability

– Note that there are no AT-TLS policy changes associated with this new support



Solution: AT-TLS support for session reuse

• Two new Get requests are provided on the SIOCTTLSCTL ioctl interface so that AT-TLS applications can

get the TLS session ID for session reuse later

– TTLSK_GetSessionToken

– TTLSK_GetSessionId

• The very first Set request is provided on the SIOCTTLSCTL ioctl interface so that AT-TLS applications can

set TLS session ID to enable session reuse

– TTLSK_SetSessionToken

• The new requests of SIOCTTLSCTL ioctl interface are supported in all of the programming languages

supported by AT-TLS (Assembler, C, PL/I, COBOL and REXX)



Solution: FTP support for session reuse

• A new Client and Server FTP.DATA SECURE_SESSION_REUSE statement is added to control whether

TLS session reuse is needed with an FTP session

• FTP native SSL (TLSMECANISM FTP)

– FTP calls native z/OS System SSL APIs to get the SID of the control connection or a previous data connection in case

TLS session reuse is needed on the subsequent data connections within an FTP session

– When TLS session reuse is needed, a native z/OS System SSL API is called to set the SID before the handshake of

an FTP data connection

• FTP AT-TLS (TLSMECHANISM TTLS)

– FTP uses the new TTLSK_GetSessionToken request to get the SID of the control connection or a previous data

connection in case TLS session reuse is needed on the subsequent data connections within an FTP session

– When TLS session reuse is needed, the new TTLSK_SetSessionToken request is used to set the SID before initiating

the TLS handshake of an FTP data connection



Enablement actions: SIOCTTLSCTL ioctl interface GET request Identifier Constant Output Length Output Format Description TTLSK_GetSessionToken

4006 Variable

Base64 encoded

Obtains a token for the SSL session that represents the AT-TLS environment and session identifier for the secure connection. You can use the TTLSQ_Length field to determine the length of the token that is returned. The TTLSQ_Rcode field contains the return code of the Get request.

TTLSK_GetSessionId

4007 Variable Binary Obtains the session identifier for the SSL session. You can use the TTLSQ_Length field to determine the length of the session identifier that is returned. The TTLSQ_Rcode field contains the return code of the Get request.



Enablement actions: SIOCTTLSCTL ioctl interface SET request Identifier Constant Output Length Output Format Description TTLSK_SetSessionToken

5000 Variable

Base64 encoded

Sets the SID value for the TLS connection. For servers, GSK_SID_VALUE is set. For clients, the GSK_PEER_ID value is set. A previous TTLSK_GetSessionToken obtained the token which is passed in. Set the TTLSQ_Length field to the length of the token that is passed in. The TTLSQ_Rcode field contains the return code of the Set request.

• Associated AT-TLS policy specifies ApplicationControlled On

• Can only issue this request when starting security on the session (when the TTLS_Init_Connection flag is

specified)



Enablement actions: FTP.DATA statement for FTP Server

• Purpose

– SECURE_SESSION_REUSE statement on the server specifies whether the server requires session reuse when

TLS/SSL is being used to protect the connections

• Server Syntax

• Note

– The value of this new FTP.DATA statement cannot be queried through LOCSTAT, STAT or XSTA

+--SECURE_SESSION_REUSE ALLOWED-------+ >>--+-------------------------------------+---------->< +--SECURE_SESSION_REUSE--+--ALLOWED---+ +--REQUIRED--+



Enablement actions: FTP.DATA statement for FTP Client

• Purpose

– SECURE_SESSION_REUSE statement on the client specifies whether the client requires session reuse when

TLS/SSL is being used to protect the connections

• Client Syntax

• Note

– The value of this new FTP.DATA statement cannot be dynamically modified through the FTP LOCSITE subcommands,

nor can its value be queried through LOCSTAT

+--SECURE_SESSION_REUSE NONE----------+ >>--+-------------------------------------+---------->< +--SECURE_SESSION_REUSE--+--NONE------+ +--ALLOWED---+ +--REQUIRED--+



Externals: FTP client user exit EZAFCCMD

• The SECURE_SESSION_REUSE statement value is included in the EZAFCCMD parameter structure.

• You can parse the structure for “SECURE_SESSION_REUSE”, and then obtain the value.

• The possible value of SECURE_SESSION_REUSE in the structure are:

– “NONE”

– “ALLOWED”

– “REQUIRED”



Externals: Type 119 SMF record

• TCP connection termination record (subtype 2)

– TLS Session ID and the Session Reuse Required flag are added to this record

• FTP client transfer completion record (subtype 3), FTP client transfer initialization record (subtype 101), FTP

client login failure record (subtype 102) and FTP client session record (subtype 103)

– Value of the client FTP.DATA statement SECURE_SESSION_REUSE and TLS Session ID of the FTP control

connection and data connection are added to these records

• FTP server transfer completion record (subtype 70), FTP server transfer initialization record (subtype 100),

FTP server logon failure record (subtype 72) and FTP server session record (subtype 104)

– Value of the server FTP.DATA statement SECURE_SESSION_REUSE and TLS Session ID of the FTP control

connection and data connection are added to these records

• FTP daemon configuration data record (subtype 71)

– Value of the server FTP.DATA statement SECURE_SESSION_REUSE is added to this record



Externals: Callable NMI EZBNMIFR

• GetConnectionDetail (NWMTcpConnType)

– TLS Session ID and the Session Reuse Required flag are added to the data returned by this request for a TCP

connection

• GetFTPDaemonConfig (NWMFTPDConfigType)

– Value of the server FTP.DATA statement SECURE_SESSION_REUSE is added to the data returned by this

EZBNMIFR request for an FTP daemon



Externals: Netstat TTLS/-x

• Netstat TTLS is updated to display TLS Session ID and the Session Reuse Required flag of each TCP

connection returned by this command

MVS TCP/IP NETSTAT CS V2R1 TCPIP Name: TCPCS 19:51:22 ConnID: 000000B8 JobName: FTPD1 LocalSocket: ::ffff:127.0.0.1..21 RemoteSocket: ::ffff:127.0.0.1..1030 SecLevel: TLS Version 1.2 Cipher: C001 TLS_ECDH_ECDSA_WITH_NULL_SHA CertUserID: N/A MapType: Primary FIPS140: Off SessionID: 0000001F 00000000 00000000 0000FFFF 092A6999 04050000 547F987C 00000001 SIDReuseReq: Off TTLSRule: ftp_serv_21 TTLSGrpAction: grp_act1 TTLSEnvAction: env_act_serv



Externals: SNMP

• The following new Management Information Base (MIB) objects are added to the

ibmTcpipMvsTcpConnectionTable

– ibmMvsTcpConnectionTtlsReuseReq

– IbmMvsTcpConnectionTtlsSessionID

• Here is an example of the SNMP data from the z/OS UNIX snmp command:

snmp -v get ibmMvsTcpConnectionTtlsSessionID.1.4.9.42.105.17.21.1.4.9.42.105.153.1030 ibmMvsTcpConnectionTtlsSessionID.1.4.9.42.105.17.21.1.4.9.42.105.153.1030 = '000 0001800000000000000000000ffff092a699904060000548600e100000003'h snmp -v walk ibmMvsTcpConnectionTtlsReuseReq ibmMvsTcpConnectionTtlsReuseReq.1.4.9.42.105.17.20.1.4.9.42.105.153.1036 = 1 ibmMvsTcpConnectionTtlsReuseReq.1.4.9.42.105.17.21.1.4.9.42.105.153.1035 = 2 ibmMvsTcpConnectionTtlsReuseReq.1.4.127.0.0.1.1024.1.4.127.0.0.1.1025 = 2 ibmMvsTcpConnectionTtlsReuseReq.1.4.127.0.0.1.1025.1.4.127.0.0.1.1024 = 2




• NMI EZBNMIFR applications

• SMF type 119 records

• FTP client user exit EZAFCCMD

• Netstat TTLS / -x



AT-TLS ENABLEMENT FOR DCAS




• The Digital Certificate Access Server (DCAS) is a host-based server that provides some distributed z/OS

security server services

– Typically used with distributed products that need to access z/OS applications and want to provide a single-signon

solution

– DCAS provides single sign-on services which include remote SAF interfaces for authorized clients:

– Passticket generation(most common service)

– Certificate to user ID mapping

– Kerberos principal name to user ID mapping

– DCAS uses System SSL API's to secure its client connections



Business problem

• NIST mandate (SP800-131a) states that by the end of 2013, U.S. Government systems support TLSv1.2,

SHA-2 hashes encryption key strengths of 112 bits or more

• DCAS must support the TLSv1.1 and TLSv1.2 with the more secure 2-byte ciphers for client connection

• Existing System SSL integration only goes up to TLSv1.0 and it uses deprecated System SSL APIs



Solution

• Enhance DCAS to support TLSv1.1 or TLSv1.2 with the new set of TLSv1.2 2-byte ciphers

– Change DCAS to use AT-TLS for TLS/SSL client connection

– Allow DCAS to be configured for client connection

– System SSL (default)

– AT-TLS aware application



Enablement actions: TLSMECHANISM

• New keyword TLSMECHANISM to specify whether to use AT-TLS policies or IBM System SSL directly

– DCAS (default)

– IBM System SSL is used directly for TLS/SSL

– No changes are required to client connection

– ATTLS

– AT-TLS policies are used for TLS/SSL

– Client connection TLS/SSL must be updated to match configured AT-TLS policies

• Note: See IP Configuration Guide - Steps for customizing the DCAS server for TLS/SSL for details



Enablement actions: SERVERTYPE and TCPIP

• The DCAS SERVERTYPE and TCPIP keywords are always used regardless of the value configured on the

TLSMECHANISM keyword

– TLSMECHANISM ATTLS

– Nothing to configure in AT-TLS policy for these keywords



Enablement actions: IPADDR and PORT

• DCAS IPADDR and PORT keywords are always used regardless of the value configured on the

TLSMECHANISM keyword


– TTLSRule/Direction Inbound

– TTLSRule/LocalAddr value must include the DCAS IPADDR value

– TTLSRule/LocalPortRange value must include the DCAS Port value



Enablement actions: CLIENTAUTH

• DCAS CLIENTAUTH keyword is always used regardless of the value configured on the TLSMECHANISM

keyword


– TTLSEnvironmentAction/HandshakeRole ServerWithClientAuth

– DCAS ClientAuth LOCAL1

– TTLSEnvironmentAdvancedParms/ClientAuthType Required

– DCAS ClientAuth LOCAL2

– TTLSEnvironmentAdvancedParms/ClientAuthType SAFCHECK

– Defaults on DCAS configuration and AT-TLS policies are different

– Default ClientAuth is LOCAL2

– Default TTLSENvironmentAdvancedParms/ClientAuthType is Required



Enablement actions: KEYRING and SAFKEYRING

• DCAS KEYRING and SAFKEYRING keywords are not used if TLSMECHANISM is ATTLS


– TTLSKeyringParms/Keyring must be set

– When converting to TLSMECHANISM ATTLS use the value previously configured for DCAS KEYRING or DCAS SAFKEYRING



Enablement actions: STASHFILE

• DCAS STASHFILE keyword is not used if TLSMECHANISM is ATTLS


– TTLSKeyringParms/KeyringStashFile

– When converting to TLSMECHANISM ATTLS use the value previously configured for DCAS STASHFILE



Enablement actions: LDAPSERVER and LDAPPORT

• DCAS LDAPSERVER AND LDAPPORT keywords are not used if TLSMECHANISM is ATTLS


– TTLSGskLdapParms/GSK_LDAP_SERVER

– When converting to TLSMECHANISM ATTLS use the value previously configured for DCAS LDAPSERVER

– TTLSGskLdapParms/GSK_LDAP_SERVER

– When converting to TLSMECHANISM ATTLS use the value previously configured for DCAS LDAPPORT



Enablement actions: V3CIPHER

• DCAS V3CIPHER keyword is not used if TLSMECHANISM is ATTLS


– TTLSCipherParms/V3CipherSuites

– When converting to TLSMECHANISM ATTLS use the value previously configured for DCAS V3CIPHER



Externals: Connection failure

• New failure return codes if TLSMECHANISM ATTLS

– 248 verify:

– TTLSEnvironmentActions/ HandshakeRole parameter set to ServerWithClientAuth

– TTLSEnvironmentAdvancedParms/ClientAuthType

– CLIENTAUTH LOCAL1 set to ClientAuthType Required

– CLIENTAUTH LOCAL2 set to ClientAuthType SAFCHECK

– 249

– DCAS AT-TLS handshake failed or connection is not secure

– Check AT-TLS configuration and DCAS log file for details




• No migration action required, TLSMECHANISM defaults to DCAS

• Consider migrating to AT-TLS to use the latest TLS/SSL security levels by setting TLSMECHANISM to

ATTLS



TLS SECURITY ENHANCEMENTS FOR SENDMAIL




• Sendmail

– Sendmail currently uses the z/OS System SSL for both the sendmail client and server to support TLSv1.0

– Provides private, authenticated communication over the internet as defined in RFC 2487 (SMTP Service Extension for Secure SMTP

over TLS)

– The z/OS specific configuration file is used for input of SSL configuration

– /usr/lpp/tcpip/samples/sendmail/cf/zOS.cf is the default

keyring

sendmail client

sendmail server

server certificate

client certificate

keyring

when requested



Business problem


SHA-2 hashes encryption key strengths of 112 bits or more

• Sendmail client and server need to support the TLSv1.2 to allow for more secure ciphers, including those

that use SHA-2 algorithms



Solution

• V2R2 adds support to Sendmail client and server to now support TLSv1.1 and TLSv1.2 with a new set of

ciphers



Enablement actions: CipherLevel

• Configuring z/OS UNIX sendmail in z/OS specific file, the default location is /etc/mail/zOS.cf

– CipherLevel

– Specifies the list of TLSv1.0, TLSv1.1, or TLSv1.2 ciphers in the order of usage preference

– If System SSL needs to access z/OS Integrated Cryptographic Services Facility (ICSF) for new TLSv1.2 ciphers, then ICSF

must be started before starting sendmail

– Example: CipherLevel 6B05040A0306090201

TLS_DHE_RSA_WITH_AES_256_CBC_SHA256



Enablement actions: CipherLevel <values>

• Cipher suites added (only numeric is configured on CipherLevel)

3B TLS_RSA_WITH_NULL_SHA256 3C TLS_RSA_WITH_AES_128_CBC_SHA256 3D TLS_RSA_WITH_AES_256_CBC_SHA256 3E TLS_DH_DSS_WITH_AES_128_CBC_SHA256 3F TLS_DH_RSA_WITH_AES_128_CBC_SHA256 40 TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 67 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 68 TLS_DH_DSS_WITH_AES_256_CBC_SHA256 69 TLS_DH_RSA_WITH_AES_256_CBC_SHA256 6A TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 6B TLS_DHE_RSA_WITH_AES_256_CBC_SHA256

9C TLS_RSA_WITH_AES_128_GCM_SHA256 9D TLS_RSA_WITH_AES_256_GCM_SHA384 9E TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 9F TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 A0 TLS_DH_RSA_WITH_AES_128_GCM_SHA256 A1 TLS_DH_RSA_WITH_AES_256_GCM_SHA384 A2 TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 A3 TLS_DHE_DSS_WITH_AES_256_GCM_SHA384 A4 TLS_DH_DSS_WITH_AES_128_GCM_SHA256 A5 TLS_DH_DSS_WITH_AES_256_GCM_SHA384



Externals

• None




• None



TLS SECURITY ENHANCEMENTS FOR POLICY AGENT



Background information: Policy Agent

• Policy Agent can act in any of several roles

– Self-contained Policy Decision Point (PDP) on single system which installs policies in one or more z/OS

Communications Server stacks on that z/OS image

– Centralized policy server, providing PDP services for one or more remote policy clients on multiple systems

– Import services to the IBM Configuration Assistant for z/OS Communications Server



Background information: Centralized Policy Agent

• Centralized Policy Agent policy client can protect all of its policy client sessions using ServerConnection

statement

– This interfaces with SSL using a direct integration with System SSL on the policy client

– Policy client installs policies in one or more z/OS Communications Server stacks on that z/OS image

• Centralized Policy Agent policy server can protect all of its sessions using user defined AT-TLS policies

• Secure connections between policy client and policy server only support TLSv1.0 with its 2-byte specific

cipher suites



Background information: Import services

• Import services to the IBM Configuration Assistant for z/OS Communications Server to return

– Existing policy configuration files

– TCP/IP profile information

• Use ServicesConnection statement with Security option to configure connection

– Basic value indicates that the connection is not secure

– Secure value only supports TLSv1.0 with a SAF keyring and no ciphers

– Policy Agent explicitly installs AT-TLS policies for the defined ImageName and Keyring using TLSv1.0



Business problem


TLSv1.2, SHA-2 hashes encryption key strengths of 112 bits or more

• Centralized Policy Agent

– The policy client is currently limited to TLSv1.0 protocol with the 2-byte cipher suites it supports

• Import Services

– The policy import services is current limited to TLSv1.0 protocol, with SAF keyring only and no ciphers



Solution

• Centralized Policy Agent

– The integration with System SSL are updated to support the TLSv1.1 protocol and TLSv1.2 protocol with its 2-byte

cipher suites

• Import Services

– The policy import services are updated to support the TLSv1.1 protocol and TLSv1.2 protocols with its 2 byte cipher

suites



Enablement actions: Centralized Policy Agent

• ServerConnection / ServerSSLV3CipherSuites (name and number) for TLSv1.2 now supports these 2-byte

ciphers – If System SSL needs to access z/OS Integrated Cryptographic Services Facility (ICSF) for new TLSv1.2 ciphers, then

ICSF must be started before starting Policy Agent

• Example: ServerSSLV3CipherSuites 9D05040A0306090201

TLS_RSA_WITH_AES_256_GCM_SHA384



Enablement actions: Import Services

• ServicesConnection statement has no configuration changes

– For a more secure connection specify Security Basic

– Code your own AT-TLS policies for policy agent import services to take advantage of the latest AT-TLS policies

– Define matching AT-TLS policies where IBM Configuration Assistant is running



Enablement actions: Import Services (continued)

• AT-TLS policies must match client side for IBM Configuration Assistant

TTLSRule TTLS_RULE { LocalPortRange 16311 Direction Inbound TTLSGroupActionRef TTLS_GROUP_ACTION TTLSEnvironmentActionRef TTLS_ENVIRONMENT_ACTION } TTLSGroupAction TTLS_GROUP_ACTION { TTLSEnabled On }



Enablement actions: Import Services (continued)

TTLSEnvironmentAction TTLS_ENVIRONMENT_ACTION { HandshakeRole Server TTLSCipherParmsRef Require_Encryption TTLSKeyRingParms { keyring /tmp/keyring keyringpw password } TTLSEnvironmentAdvancedParms { <set to required SSL or TLS levels> } } TTLSCipherParms Require_Encryption { <list of required cipher> }



Externals

• None




• None



NETWORK SECURITY ENHANCEMENTS FOR SNMP




• SNMP (Simple Network Management Protocol) is a set of standards which enables management

applications to obtain similar management data from different platforms

• The protocols include

– Description of the management data, defined in the Management Information Base (MIB)

– Operations for exchanging or changing that information

• These common protocols, management data can be exchanged between different platforms with relative

ease




• SNMP defines an architecture that consists of:

– Network management applications

– Network management agents and subagents

– Network elements, such as hosts and gateways

• z/OS Communication Server SNMP supports management data from these types of MIBs:

– Standard MIBs, as defined in IETF internet drafts or RFCs

– Enterprise-specific MIBs which are proprietary MIBs not reviewed or approved by the IETF



Business problem

• NIST mandate (SP800-131a) states that by the end of 2013, U.S. Government systems must support SHA-

2 hashes and encryption key strengths of 112 bits or more.

• The z/OS Communications Server SNMP Agent, the z/OS UNIX snmp command, and the SNMP manager

API need to support this new NIST requirement for SNMPv3 user-based security

– The current user-based privacy (encryption) support uses the CBC-DES algorithm with a key strength of 56 bits



Solution

• The z/OS Communications Server SNMP Agent, the z/OS UNIX snmp command, and the SNMP manager

API are enhanced to support the Advanced Encryption Standard (AES) 128-bit cipher algorithm as an

SNMPv3 privacy protocol for encryption

– AES is a symmetric cipher algorithm selected by the National Institute of Standards (NIST) as a replacement for DES

– The AES SNMP implementation is described in RFC 3826

– This RFC specifies that SNMP use AES encryption in Cipher FeedBack Mode (CFB)

– z/OS Integrated Cryptographic Services Facility (ICSF) is required for AES 128-bit cipher encryption privacy protocol

– For details on configuring ICSF, see z/OS Cryptographic Services ICSF Administrator's Guide



Enablement actions: SNMP Agent privProto

• SNMP Agent

– Configure an SNMPv3 user to use AES 128-bit encryption by specifying a USM_USER entry with the privProto field

set to AESCFB128

– Example:

USM_USER u7 engineId HMAC-MD5 5fbd3ad2fa6569d6c1e9ab4b83728b87 AESCFB128

bf686267600ff8f4b1354b857d186b55 L nonVolatile

– For more details on privProto

– IP Configuration Reference - Coding the snmpd.conf entries



Enablement actions: z/OS UNIX snmp command privProto

• z/OS UNIX snmp command

– Configure an SNMPv3 user to use AES 128-bit encryption by specifying a configuration statement with the privProto

field set to AESCFB128

– Example:

v3mpka 127.0.0.1 snmpv3 u7 u7password context AuthPriv HMAC-MD5

15549009e2401748e8077fa17bf64c9b AESCFB128 90009683501c78a6f87575bdad5455bc


– IP Configuration Reference - Coding the osnmp.conf entries



Enablement actions: SNMP Manager API privProto

• SNMP Manager API

– Configure an SNMPv3 user to use AES 128-bit encryption by specifying a configuration statement with the privProto

field set to AESCFB128

– Example:

127.0.0.1 161 snmpv3 u7 u7password AuthPriv HMAC-MD5 15549009e2401748e8077fa17bf64c9b

AESCFB128 90009683501c78a6f87575bdad5455bc 00000002000000000943714F


– IP Programmer's Guide and Reference - SNMP manager API configuration file



Externals

• None




• None



• Removed support for the GATEWAY statement in the TCP/IP profile • Configuration assistant – TCP/IP profile configuration • CSSMTP migration enablement

Simplification and Usability



REMOVED SUPPORT FOR GATEWAY STATEMENT IN THE TCP/IP PROFILE



Business problem

• GATEWAY profile statement limitations

– No support for IPv6 routes

– Only supports DEVICE, LINK, HOME defined network interfaces

• Notification of GATEWAY removal

– ZOSMIGV2R1_CS_GATEWAY Health Check provided in z/OS V1R13 and V2R1

– TCP/IP configuration message EZZ0717I issued if a GATEWAY statement is processed in a profile

– The z/OS V2R1 release announcement stated that V2R1 was last release to support GATEWAY

– z/OS V2R1 Migration book warned of subsequent removal of support



Solution

• Remove support for GATEWAY profile statement

• Remove ZOSMIGV2R1_CS_GATEWAY Health Check



Enablement actions

• Convert your GATEWAY profile statements to BEGINROUTES statements

– Start TCP/IP stack with GATEWAY profile statements

– Use the MVS DUMP command to create a dump of the stack address space

– Invoke the TCPIPCS PROFILE command

– Use formatted BEGINROUTES statements in output to replace GATEWAY statements

• Conversion must be done on a release before z/OS V2R2



Externals

• Netstat GATE/-g commands still supported

– Only displays IPv4 routes

– Has not been enhanced for newer route parameters

• Netstat ROUTE/-r commands display IPv4 and IPv6 routes




• Ensure your static routes are defined using BEGINROUTES



CONFIGURATION ASSISTANT – TCP/IP PROFILE CONFIGURATION



Business problem

• TCP/IP profile contains a large number of configuration options

– Extensive reading required to understand what content to put into the profile

– Defaults rarely change

– Best practices are not conveyed

– No health checking of configuration

– Reuse through INCLUDE statement limited to shared file access domain



Solution

• TCP/IP profile technology added to Configuration Assistant

• Systems tree updated to require groups

– Single group for unrelated systems provided

– Default

– User defined groups

– Sysplex

– Subplex

• Best practices configuration options set on creation of new definitions

– Example: IPAQENET interface definition specifies OSA Express generated VMAC

• Automatic conflict detection



Enablement actions

• Use Configuration Assistant to construct and manage your TCP/IP profiles



Externals

• Existing systems placed in Default group



Externals (continued)

• TCP/IP Profile technology is now available



CSSMTP MIGRATION ENABLEMENT




• z/OS Communication Server currently supports three mailer programs

z/OS Application

TSO user IMAP, POP, (E)SMTP protocols

CSSMTP (SMTP client)

SMTPD (SMTP client and

server)

MTA

JES spoolW rite to SYSOUT

z/OS UNIX shell user

z/OS Sendmail (SMTP client and

server)

non-z/OS user using z/OS Sendmail as the target server

z/OS

(E)SMTP protocols

(E)SMTP protocol

SMTP protocol

(E)SMTP protocol

MTA

SMTP networ

k

NJE networ

k

z /OSz /VSE

z /VM

MTAUnix FileSystem




• What is changing: withdrawal of SMTPD and Sendmail

z/OS Application




server)

MTA

JES spoolWrite to SYSOUT


z/OS Sendmail(SMTP client and

server)

non-z/OS user using z/OS

Sendmail as the target server

z/OS

(E)SMTP protocols

(E)SMTP protocol

SMTP protocol

(E)SMTP protocol

MTA

SMTP network

NJE network

z/OSz/VSE

z/VM

MTAUnix FileSystem

X X X

X

X

X X

z/OS Application

TSO userTSO user IMAP, POP, (E)SMTP protocols



server)

MTA

JES spoolWrite to SYSOUT


z/OS Sendmail(SMTP client and

server)

non-z/OS user using z/OS

Sendmail as the target server

z/OS

(E)SMTP protocols

(E)SMTP protocol

SMTP protocol

(E)SMTP protocol

MTAMTA

SMTP network

NJE network

z/OSz/VSE

z/VM

MTAMTAUnix FileSystem

X X X

X

X

X X




• What will be left after withdrawal of SMTPD and Sendmail

z/OS Application



MTA

JES spoolW rite to SYSOUT


z/OS “sendmail” (Thin Client)

z/OS

(E)SMTP protocols

(E)SMTP protocol

MTA

SMTP network

NJE network

z /OSz /VSE

z /VM

MTAUnix FileSystem

Strategic Mail Solution

Messages formatted for CSSMTP and

placed into JES spool for CSSMTP to

process

Bottom line: will still be able to send mail from z/OS using CSSMTP and “sendmail”. But won’t be able to receive it.



Business problem: Identifying mail usage

• Mail can be sent and received by a variety of users and programs on a z/OS image

– Application programs placing mail in the JES spool

– TSO users using RECEIVE

– z/OS UNIX users sending mail using Sendmail command

– Off-platform users connecting to Sendmail to send emails

– z/OS UNIX users receiving email to local mailboxes

• Do you know for sure all the ways mail is being used on your platform?

– Who needs to be changed to use CSSMTP from SMTPD

– Who needs to find an alternate mail solution?



Solution: Health checks

• In V2R2 Comm Server, IBM is providing migration health checks to warn you that function is being used that

will be withdrawn. Also made available in V2R1 via OA47735 and PI40204.

– SMTPD in use to send or receive email

– Sendmail in use to send or receive email

– Sendmail being used as mail transfer agent

• Multiple checks are being provided to differentiate function that can be migrated to CSSMTP, from function

that will no longer be available on z/OS.



Enablement actions

• These migration health checks default to disabled and need to be enabled by the system administrator.

• See the z/OS Healthchecker User's Guide for details on enabling migration health checks

– Section: “Managing your checks”



Externals: ZOSMIGV2R2_Next_CS_SENDMAILDAEMN

• This check determines whether the Sendmail daemon is in use on this system.

– Issues these messages if Sendmail is found to be in use on the system:

– ISTM028E The sendmail daemon is in use on this system

– ISTM900I Function: SENDMAIL DAEMON last usage on mmddyyyy at hhmmss

• If you have stopped using Sendmail, you can use message ISTM900I to determine if it was detected before

or after you stopped the usage.

– Issues this message if Sendmail is not found to be in use on the system

– ISTM027I The sendmail daemon is not in use on this system



Externals: ZOSMIGV2R2_Next_CS_SENDMAILCLIEN

• This check determines whether the Sendmail client program has been invoked on this system.

– Issues these messages if the Sendmail client program has been invoked on the system:

– ISTM018E The sendmail client is in use on this system

– ISTM900I Function: SENDMAIL CLIENT last usage on mmddyyyy at hhmmss

• If you have stopped using the Sendmail client program, you can use message ISTM900I to determine if it

was detected before or after you stopped the usage.

– Issues this message if the Sendmail client has not been invoked on the system

– ISTM017I The sendmail client is not in use on this system



Externals: ZOSMIGV2R2_Next_CS_SENDMAILMTA

• This check determines whether the Sendmail daemon has been used as a mail transfer agent (MTA) on this

system. MTA function will not be available on z/OS after Sendmail and SMTPD are withdrawn.

– Issues these messages if Sendmail has listened on port 25 on this system:

– ISTM020E The sendmail mail transfer agent is in use on this system

– ISTM900I Function: SENDMAIL MTA last usage on mmddyyyy at hhmmss

• If you have stopped using the Sendmail MTA function, you can use message ISTM900I to determine if it

was detected before or after you stopped the usage


– ISTM019I The sendmail mail transfer agent is not in use on this system



Externals: ZOSMIGV2R2_Next_CS_SENDMAILMSA

• This check determines whether the Sendmail daemon has been used as a mail submission agent (MSA) on

this system. MSA function will not be available on z/OS after Sendmail and SMTPD are withdrawn.

– Issues these messages if Sendmail has listened on port 587 on this system:

– ISTM022E The sendmail mail submission agent is in use on this system

– ISTM900I Function: SENDMAIL MSA last usage on mmddyyyy at hhmmss

• If you have stopped using the Sendmail MTA function, you can use message ISTM900I to determine if it

was detected before or after you stopped the usage


– ISTM021I The sendmail mail submission agent is not in use on this system



Externals: ZOSMIGV2R2_Next_CS_SMPTDDAEMON

• This check determines whether the SMTPD daemon is in use on this system. For sending email from the

JES spool, users should migrate to CSSMTP. Other SMTPD functions will not be replaced on z/OS.

– Issues these messages if the SMTP daemon has been started on this system:

– ISTM024E The SMTP daemon is in use on this system

– ISTM900I Function: SMTPD DAEMON last usage on mmddyyyy at hhmmss

• If you have stopped using the SMTP daemon, you can use message ISTM900I to determine if it was

detected before or after you stopped the usage

– Issues this message if the SMTPD daemon has not been invoked on the system

– ISTM023I The SMTPD daemon is not in use on this system



Externals: ZOSMIGV2R2_Next_CS_SMPTDMTA

• This check determines whether the SMTPD daemon is in use as a mail transfer agent (MTA) on this system.

MTA function will not be available on z/OS after the withdrawal of SMTPD and Sendmail

– Issues these messages if the SMTPD daemon has listened on port 25 (the MTA well-known port):

– ISTM026E The SMTP mail transfer agent is in use on this system

– ISTM900I Function: SMTPD MTA last usage on mmddyyyy at hhmmss

• If you have stopped using the SMTP as a mail transfer agent, you can use message ISTM900I to determine

if it was detected before or after you stopped the usage

– Issues this message if the SMTPD mail transfer agent has not been invoked on the system

– ISTM025I The SMTPD mail transfer agent is not in use on this system



Business problem: CSSMTP compatibility

• Customers need a way to verify CSSMTP can handle emails currently handled by SMTPD, and identify

those that it can not

– CSSMTP has stricter standards testing than SMTPD

– Emails that SMTPD accepts can be flagged as errors in CSSMTP

– This testing will allow customers to address mail generators that will cause problems for CSSMTP, or feed migration

requirements to IBM

• Testing to verify CSSMTP compatibility is a problem in customer environments

– It's almost impossible to replicate in a test environment all the production processes that produce mail

– Many customers are not fully aware of all of the production processes that produce mail, and if they are, the source

code that produces mail messages is not well understood or may not even exist



Solution: CSSMTP test mode

• There are two parts to the solution:

– CSSMTP test mode

– A new configuration parameter that causes CSSMTP to run in Test Mode

– CSSMTP will perform its normal email processing, except it will not actually send emails

– It will either report that an email failed, or throw away the email

– SMTPD continues running alongside the Test Mode CSSMTP and actually processes and sends the emails

– EZBMCOPY

– To enable Test Mode CSSMTP to run alongside SMTPD, IBM is introducing a utility program, EZBMCOPY, that copies JES email

jobs to both CSSMTP and SMTPD



Solution: EZBMCOPY

z/OS Application

TSO user

IMAP, POP, (E)SMTP protocols

SMTPD(production)

CSSMTPD (TESTMODE)

MTAW rite to SYSOUT

z/OS

SMTP protocol

MTA

SMTP networ

k

EZBMCOPY

W RITER=SMTPD1

W RITER=CSSMTPD

SYSOUT REPORT

Email3: Error!

W RITER=SMTPD



Enablement actions: CSSMTP test mode

• Parameters on the CSSMTP Options statement:

>>--Options---------| Put Braces and Parameters on Separate Lines |--><

Options Parameters:

+--NullTrunc NO------+ +--TestMode NO------+

|--+--------------------+---+-------------------+----->

+--NullTrunc-+-YES-+-+ +--TestMode-+-NO--+-+

+-NO--+ +-YES-+

– TestMode cannot be dynamically altered. CSSMTP must be recycled to change its value

– If no errors are found in a spool file, CSSMTP will release spool files when it has completed processing. If errors are

found, CSSMTP will honor the setting of BADSPOOLDISP

– Make sure the REPORT statement is coded with a valid destination for the error report. Warning message EZD1841I

is issued if it is not.



Enablement actions: EZBMCOPY

• Parm value:

– WRITER=w Select program name (writer name) w

• EZBMCOPY assumes the writer name specified by the WRITER parameter. It selects spool files in two

ways:

– The file's writer name matches the WRITER parameter

– The file's destination matches the WRITER parameter

• Then it makes as many copies as there are OUTPUT cards in the JCL, then deallocates the original data set

– Restriction: a maximum of two output cards can be coded



Enablement actions: EZBMCOPY (continued) //EZBMCOPY PROC

//STEP EXEC PGM=EZBMCOPY PARM='WRITER=SMTPD'

//OUT1 OUTPUT WRITER=SMTPD1

//OUT2 OUTPUT WRITER=CSSMTP

//STEPLIB DD DSN=JES2.TESTING.LOAD,DISP=SHR

//SYSUT2 DD SYSOUT=*,SPIN=UNALLOC,OUTPUT=(*.OUT1,*.OUT2)

//SYSPRINT DD SYSOUT=*

//SYSIN DD DUMMY

• Assume the JCL shown here and SMTPD running with writer name SMTPD. (note: SMTPD's writer name is

its jobname)

• Change the writer name of SMTPD to SMTPD1 for this test by changing its jobname to SMTPD1

• Start CSSMTP in TESTMODE with writer name CSSMTP

• Start EZBMCOPY using the example JCL above



Business problem: CSSMTP AT-TLS performance

• Requirements to encrypt all email are becoming more common. TLS between mail servers is a common

method

– CSSMTP supports and interoperates with TCP/IP's AT-TLS support

• TLS negotiation between hosts requires several flows back and forth on a TCP connection, before data can

flow

– This would be required whenever CSSMTP connects to a downstream mail server

– CSSMTP disconnects after finishing the last email in a JES spool file, and reconnects when starting the next spool file

– When JES spool files contain large numbers of emails, the extra TLS flows are insignificant

– But if a customer has multiple JES spool files with only a few or even just one email, the AT-TLS burden becomes more significant



Solution

• A switch to control how long CSSMTP keeps connections with mail servers after it finishes processing a JES

spool file

– The default behavior is to disconnect right away (current behavior)

– You may want to set this switch to a longer value if your installation produces a lot of spool files that contain only one,

or a few, emails



Enablement actions

• Specify the ConnectIdle parameter with a non-zero value to maintain a connection between mail messages

>>--Timeout-------| Put Braces and Parameters on Separate Lines |--><

[....]

+-MailCmd 300------+ +-RCPTCmd 300------+ +-ConnectIdle 0 -------+

>--+------------------+----+------------------+--+----------------------+----->

+-MailCmd seconds--+ +-RCPTCmd seconds--+ +-ConnectIdle seconds--+



Externals

• The new configuration parameter is also externalized using the CSSMTP SMF configuration record

(CONFIG subtype 48)

• MODIFY CSSMTP,DISPLAY,CONFIG will show the new parameters

[…] OPTIONS: NULLTRUNC : NO DATALINETRUNC : NO TESTMODE: : NO […] TIMEOUT: ANYCMD : 300 CONNECTRETRY : 120 DATABLOCK : 180 DATAINIT : 120 DATATERM : 600 INITIALMSG : 300 MAILCMD : 300 RPCTCMD : 300 CONNECTIDLE : 60 […]



z/OS Communications Performance

• z/OS Communications Server performance index

– Published performance white papers containing GA level performance summary information

– V2R2 report availability targeted for December 2015

– http://www-01.ibm.com/support/docview.wss?rs=852&context=SSSN3L&dc=DA480&uid=swg27005524&loc=en_US&cs=utf-8&lang=en

http://www-01.ibm.com/support/docview.wss?rs=852&context=SSSN3L&dc=DA480&uid=swg27005524&loc=en_US&cs=utf-8&lang=en�



z/OS Communications Server Social Media

http://facebook.com/IBMCommserver

http://twitter.com/IBM_Commserver

http://tinyurl.com/zoscsblog

http://youtube.com/user/zOSCommServer

http://facebook.com/IBMCommserver�

http://twitter.com/IBM_Commserver�

http://tinyurl.com/zoscsblog�

http://youtube.com/user/zOSCommServer�

ibm z/os v2r2 networking technologies update

Technology