latest in zos unix and zfs.ppt - cmg · the latest in z/os unix and zfs marna walle,...

The Latest in z/OS UNIX and zFS

Marna WALLE, [email protected], Member of the IBM Academy of Technologyz/OS System Installation IBM Z Systems, Poughkeepsie NY USA Thanks for material provided by Vivian Morabito, zFS Function

Test, and Bill Schoen, z/OS UNIX Development.

1© Copyright IBM Corporation 2018. Materials may not be reproduced in whole or in

part without the prior written permission of IBM.

Agenda

zFS:

• Encryption and Compression

• Shrink

• SMF Recording

• Implicit Format on mount

• Online Salvage (verify / repair)

• Supplemental: Dynamic Change Aggregate Attributes (chaggr),

Miscellaneous updates, and zFS V2R3 Performance Measurements

z/OS UNIX:

• HFS to zFS utility

• Automatic unmount of version file system in shared file system

• Maintaining a single configuration file across the sysplex

• Assigning default umask values to all z/OS UNIX users

1



V2R3 Encryption & Compression Support for zFS

• New and Existing filesystems can be encrypted and / or compressed

• User decides which operation, and in what order. HOWEVER, if you want to do both, compression before

encryption is highly recommended.

• After a filesystem is fully encrypted or compressed, additional new entries will automatically be encrypted or compressed

2

zFS Encryption AND compression



V2R3 Encryption & Compression Support for zFS…

• Encrypting or compressing an existing filesystem is a long-running administrative command

• Application access is fully allowed to the filesystem during the operation.

• Progress may be monitored with FSINFO

3


4IBM Systems Technical Events | ibm.com/training/events

© Copyright IBM Corporation 2017. Technical University/Symposia materials may not

be reproduced in whole or in part without the prior written permission of IBM.

zFS V2R3: New long running admin commands…

• New thread pool created for long running commands

• Foreground threads – handle overall long running operation(s)

• Background threads – used by foreground thread to allow for

parallelism

• new configuration option long_cmd_threads

• long_cmd_threads=foreground, background

• (each value can be 1-64, default is 8,24)

• Long running commands require an available foreground thread

4





zFS V2R3: New long running admin commands…

• Background tasks on the zFS owning system process every object in the filesystem.

• Try not to use long running commands (encrypt, compress, shrink) during periods of high activity.

• Best to use them during “off shift” time periods

5





zFS Encryption / Compression

• zFS Encryption & Compression are both V2R3 ONLY

• If there are V2R1 or V2R2 members in the sysplex:

• Encryption or Compression may not be initiated.

• If you mount an encrypted or compressed filesystem on a V2R3

member in a sysplex with V2R1 or V2R2 members, zFS will function

ship to the downlevel members (significant performance impact)

• Wait until you are fully on V2R3 on all systems (with no intent to

go back) before using zFS encryption and compression services!

6




zFS Encryption / Compression commands:

• Decryption & Decompression are supported

• Allows you to back out, if pre-V2R3 members need to be brought back into the sysplex

7




zFS Encryption / Compression…

• Consider pairing encryption with compression.

• If the compression is done first the amount of data to be

encrypted is smaller which may improve performance.

• zFS does not perform a “dual” compression and encryption: user

directs the actions separately.

• You cannot encrypt or decrypt an aggregate that is partially

compressed or partially decompressed.

• You cannot compress / decompress an aggregate that is partially

encrypted or partially decrypted.

8




zFS Encryption

• Utilizes DFSMS Encryption for VSAM datasets

• uses ICSF* to perform encryption and decryption and to manage

cryptographic keys. Userid associated with zFS need access to those keys.

• ICSF configuration is beyond the scope of this zFS-specific presentation.

• Ensure that ICSF is running early in the IPL (COMMNDxx, not automation,

S CSF,SUB=MSTR) before you exploit DFSMS data set encryption.

• (zFS relaxes the DFSMS rule that the data set be SMS-managed and

extended format.)

• zFS encrypts:

• file contents,

• security information,

• ACLs (Access Control Lists)

• symbolic link contents

• zFS does NOT encrypt the zFS cache

* ICSF: z/OS Integrated Cryptographic Service Facility (ICSF)

9




zFS Encryption

• Only V1.5 filesystems may be encrypted

• Filesystem must be mounted R/W

• new configuration option: EDC_BUFFER_POOL

• Specifies size of real storage permanently fixed and reserved for

encryption & compression I/O

• Default: 32 M for zFS PFS, setting may range from 1 M to 1 G

• ICSF must be active to encrypt

• Ensure the PTF for APAR OA54005 is installed.

10




zFS Encryption – Key Label

• Encryption required a key label

• encryption key labels to identify encryption keys to be used to encrypt

selected data sets.

• The specified key label and encryption key must exist in the ICSF key

repository (CKDS)

11




zFS Encryption – Key Label …

• Key labels on a filesystem cannot be changed after you assign them

• Even if filesystem has not yet been encrypted

• The assignment of a key label does not encrypt a filesystem… it makes it

eligible for encryption

• Once a filesystem has a key label, it can never be owned by a member at a

level lower than V2R3.

12




Creating a NEW encrypted filesystem

1. Define filesystem

2. Assign a key label

3. Format for encryption

Data will automatically encrypted on write

13

zFS Encryption




New Encrypted filesystem: DEFINE and assign

IDCAMS DEFINE CLUSTER (from JCL)

• Must use new ZFS keyword

• KEYLABEL(keylabel) keyword

zfsadm define (from the shell)

• –keylabel keylabel

14

zFS Encryption




IDCAMS DEFINE CLUSTER and assign example

//ZDEFKEY JOB ,'DEFINE WITH KEYLABEL',

// MSGCLASS=H,

// CLASS=A,

// TIME=(1440),MSGLEVEL=(1,1)

//DEFINE EXEC PGM=IDCAMS

//SYSPRINT DD SYSOUT=H

//SYSUDUMP DD SYSOUT=H

//AMSDUMP DD SYSOUT=H

//DASD0 DD DISP=OLD,UNIT=3390,VOL=SER=POSIX1

//SYSIN DD *

DEFINE CLUSTER (NAME(POSIX.DEFKEY.EXAMPLE) -

VOLUMES(POSIX1) -

ZFS CYL(25 0) SHAREOPTIONS(3) -

KEYLABEL(PROTKEY.AES.SECURE.KEY.32BYTE))

/*

15

zFS Encryption




zfsadm define and assign example

zfsadm define -aggr POSIX.DEFKEYL –cyl 3 -keylabel PROTKEY.AES.SECURE.KEY.32BYTE

16

zFS Encryption




New Encrypted filesystem: FORMAT

• New keyword(s) on format: (zfsadm format, IOEFSUTL format, IOEAGFMT)

• -encrypt | -noencrypt

• New zFS configuration variable introduced in V2R3 to set the installation default

for encryption (IOEFSPRM):

• FORMAT_ENCRYPTION ON | OFF

• Specifying –encrypt or –noencrypt on define will override FORMAT_ENCRYPTION setting

17

zFS Encryption




zfsadm format example

zfsadm format -aggr POSIX.DEFKEYL –encrypt

Must be a version 1.5 aggregate!

In V2R3 format_version default is 5, so I did not need to specify the

version in this example

18

zFS Encryption




IOEFSUTL format example

19

//ZFMTKEY JOB ,'FSUTL V5 KEYL',

// CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),NOTIFY=&SYSUID //*

//UTLFMTK EXEC PGM=IOEFSUTL,REGION=0M, // PARM=('format -aggregate POSIX.DEFKEY.EXAMPLE -encrypt')

//IOEFSPRM DD DSN=CFCIMGKA.PARMLIB(PARMFILE),DISP=SHR //SYSPRINT DD SYSOUT=H

//STDOUT DD SYSOUT=H //STDERR DD SYSOUT=H

//SYSUDUMP DD SYSOUT=H //CEEDUMP DD SYSOUT=H

//*

zFS Encryption




IOEAGFMT format example

//ZFMTKEY JOB ,'AGFMT V5 KEYL',

// CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),NOTIFY=&SYSUID

//*

//AGFMTK EXEC PGM=IOEAGFMT,REGION=0M,

// PARM=('-aggregate POSIX.DEFKEY.EXAMPLE -encrypt')

//IOEFSPRM DD DSN=CFCIMGKA.PARMLIB(PARMFILE),DISP=SHR

//SYSPRINT DD SYSOUT=H

//STDOUT DD SYSOUT=H

//STDERR DD SYSOUT=H

//SYSUDUMP DD SYSOUT=H

//CEEDUMP DD SYSOUT=H

//*

20

zFS Encryption




Encrypting an EXISTING filesystem

Assign a keylabel and encrypt with zfsadm encrypt:

zfsadm encrypt –aggr POSIX.EXIST.FILESYS –keyl

PROTKEY.AES.SECURE.KEY.32BYTE

• The existing filesystem must be mounted in read/write mode

• The filesystem does not need to be SMS-managed or extended format.

• IMPORTANT: before an existing filesystem is encrypted for the first time, do a full backup of the filesystem!

21

zFS Encryption




querying encryption status

zfsadm fileinfo: indicates if a file is encrypted or not

zfsadm fsinfo indicates filesystem status:

encrypted

in-progress

• % encrypted, or if it has been stopped

• timestamp when long running command was started & task ID

not encrypted

22

zFS Encryption




zFS Encryption

• May be cancelled:

• filesystem will remain partially encrypted

• Performance will be degraded for partially encrypted filesystem… don’t let filesystem remain in this state!

• Encryption may be resumed (using zfsadm encrypt command)

23




fileinfo: encryption status on a file

zfsadm fileinfo /ict/MtPt/dir_7/file1

path: /ict/MtPt/dir_7/file1

*** global data ***

fid 3313,1 anode 2380,4548

length 6 format INLINE

.

.

.

mtime Aug 4 20:49:55 2017 atime Aug 4 20:49:55 2017

ctime Aug 4 20:49:55 2017 create time Aug 4 20:49:55 2017

reftime none

encrypted not compressed

24

zFS Encryption




Use fsinfo to display encryption status

# zfsadm fsinfo -aggr POSIX.MY.FILESYS

File System Name: POSIX.MY.FILESYS

*** owner information ***

Owner: DCEIMGKA Converttov5: OFF,n/a

.

.

.

Status: RW,RS,EN,NC

.

.

.

Legend: RW=Read-write, RS=Mounted RWSHARE, EN=Encrypted, NC=Not compressed

25

zFS Encryption




fsinfo shows progress for encryption in-progress

zfsadm fsinfo -aggregate PLEX.DCEIMGNJ.BIGENC

File System Name: PLEX.DCEIMGNJ.BIGENC


..........

Status: RW,RS,EI,NC

...

...

Encrypt Progress: running, 23% complete started at Nov 21 14:54:40 2016 task 57F5E0

...

Legend: RW=Read-write, RS=Mounted RWSHARE, EI=Partially Encrypted

NC=Not compressed

26

zFS Encryption




Cancel an in-progress encryption:

zfsadm encrypt -aggregate PLEX.DCEIMGNJ.BIGENC -cancel

IOEZ00892I Aggregate PLEX.DCEIMGNJ.BIGENC encrypt or decrypt successfully canceled.

27

zFS Encryption




fsinfo shows progress for stopped encryption




..........

Status: RW,RS,EI,NC

...

...

Encrypt Progress: stopped, 23%

...

Legend: RW=Read-write, RS=Mounted RWSHARE, EI=Partially encrypted

NC=Not compressed

28

zFS Encryption




To decrypt

zfsadm decrypt -aggr POSIX.MY.FILESYS

IOEZ00878I Aggregate POSIX.MY.FILESYS is successfully decrypted.

Should be a rare event…

…only employ encryption at your installation when fully ready

29

zFS Encryption




Encrypt / Decrypt authorization requirements:

• Superuser Authorization Required:

• Logged in as a uid-0 user -or-

• Have READ authority to the SUPERUSER.FILESYS.PFSCTL resource in the UNIXPRIV class.

• Batch formatting with encryption (IOEAGFMT / IOEFSUTL format) also requires

READ access to key label

30

zFS Encryption




zFS Compression

• Compress the contents of a filesystem. (i.e., frees up space)

• filesystem size remains the same.

• Average amount of disk space that is saved per file is approximately 65%

• dependent on the type of data that is being compressed.

31




zFS Compression…

• Any file larger than 8K can be compressed

• zFS compresses file data only, not directories

• Compression is “advisory”

• If the compression of a file does not reduce space utilization, the file is left in its

uncompressed format.

• Always eligible for compression… if file changes and space will be saved, the file is compressed

• zfsadm fileinfo - will indicate how many blocks were saved for

compression for a particular file.

32




zFS Compression

• Utilizes zEDC Authorized Services (introduced in z/OS V2R1 as a new priced feature, and needs the supporting HW features)

• Uses (new to V2R3) configuration option: EDC_BUFFER_POOL

• Specifies size of real storage reserved for encryption & compression I/O

• Default: 32 M for zFS PFS, setting may range from 1 M to 1 G

• edcfixed - IBM Recommended new option for user_cache_size parameter.

• Allows zFS to permanently fix the user cache file to eliminate data movement from/to auxiliary buffer pool

user_cache_size=64M,edcfixed

33




zFS Creating NEW compression eligible filesystems

1. Define as usual

2. Format for compression

• New zFS configuration variable FORMAT_COMPRESSION = ON | OFF

• May be overwritten by using –compress | -nocompress on all flavors of format

34




zfsadm format example

zfsadm format -aggr POSIX.NEWFYS.COMPRESS –compress

Note: in V2R3 version5 is the default version for format

35

zFS Compression




IOEFSUTL format example

//ZDEFFMT JOB ,'DEFINE AND FORMAT', // MSGCLASS=H, // CLASS=A, // TIME=(1440),MSGLEVEL=(1,1) //*---------------------------------------------------//* DEFINE FORMAT COMPRESSION ELIGIBLE //*---------------------------------------------------//DEFINE EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=H //SYSUDUMP DD SYSOUT=H //AMSDUMP DD SYSOUT=H //DASD0 DD DISP=OLD,UNIT=3390,VOL=SER=POSIX1 //SYSIN DD *

DEFINE CLUSTER (NAME(POSIX.NEWFS.COMPRESS) -VOLUMES(POSIX1) -ZFS CYL(25 0) SHAREOPTIONS(3))

/* //CREATE EXEC PGM=IOEFSUTL,REGION=0M, // PARM=('format -aggregate POSIX.NEWFS.COMPRESS -compress') //SYSPRINT DD SYSOUT=H //STDOUT DD SYSOUT=H //STDERR DD SYSOUT=H //SYSUDUMP DD SYSOUT=H //CEEDUMP DD SYSOUT=H

36

zFS Compression




Compressing EXISTING filesystems

37

zfsadm compress –aggr POSIX.EXIST.FILESYS

• The filesystem containing the data to be compressed must be mounted

read / write

• IMPORTANT: before an existing filesystem is compressed for the first time, do a full backup of the filesystem!

zFS Compression




querying compression status

zfsadm fileinfo: indicates if a file is

compresed & and space savings

compression eligible

not compressed

zfsadm fsinfo indicates filesystem status:

compressed

in-progress

• % compressed, or if it has been stopped

• timestamp when long running command was started & task ID

not compressed

38

zFS Compression




FILEINFO shows compression status of a file

This example shows a file that had space savings due to compression

zfsadm fileinfo –path testmtpt/file4

path: /home/suimgju/C81500/testmtpt/file4

*** global data ***

.

.

.

Mtime Jan 19 12:27:57 2017 atime Jan 19 12:27:57 2017

ctime Jan 19 12:27:57 2017 create time Jan 19 12:27:57 2017

reftime none

not encrypted compressed 24K saved

39

zFS Compression




FILEINFO shows compression status of a file

This example shows a file that had no space saving

zfsadm fileinfo /ict/MtPt/dir_23/large_file_23

path: /ict/MtPt/dir_23/large_file_23

*** global data ***

fid 1536,1 anode 6668,264

.

.

.

mtime Jul 8 12:35:04 2016 atime Jul 24 19:27:14 2017

ctime Jul 27 15:46:02 2017 create time Jul 27 15:46:02 2017

reftime none

not encrypted compress-eligible 0K saved

40

zFS Compression




FSINFO shows compression status of filesystem

41

# zfsadm fsinfo -aggr POSIX.MY.FILESYS File System Name: POSIX.MY.FILESYS

*** owner information *** Owner: DCEIMGKA Converttov5: OFF,n/a ...Status: RW,RS,NE,CO...

Legend: RW=Read-write, RS=Mounted RWSHARE, NE=Not encrypted, CO=Compressed

zFS Compression




FSINFO showing compression in-progress




.

.

Status: RW,RS,NE,CI

.

.

Compress Progress: running, 48% started at Nov 21 16:34:40 2016 task 57F5E0

.

Legend: RW=Read-write, RS=Mounted RWSHARE, NE=Not encrypted

CI=Partially compressed

42

Compression




FSINFO showing compression stopped (by using a –cancel option)




.

.

Status: RW,RS,NE,CI

.

.

Compress Progress: stopped, 48%

.

Legend: RW=Read-write, RS=Mounted RWSHARE, NE=Not encrypted

CI=Partially compressed

43

zFS Compression




ZFS_VERIFY_COMPRESSION_HEALTH

• New health check added for V2R3 compression support

• Checks whether all user cache pages are registered with the zEDC Express service (hardware feature) when there are compressed filesystems.

• For best performance, all systems accessing the compressed data should have zEDC Express installed.

44

zFS Compression




MODIFY ZFS,QUERY & zfsadm query

• Help to determine the overall compression effectiveness and allows monitoring

of the performance of the zEDC.

• Compress report is included with the ALL & LFS reports

• # of compression / decompression calls

• Avg call time

• KB Input : # of kilobytes sent from zEDC card for compression or decompression

• KB output: # of kilobytes received from zEDC card for compression or decompression

45

zFS Compression




zFS Decompression

zfsadm decompress –aggregate name [-cancel]

Should be a rare event…

…only employ compression at your installation when fully ready

46




zFS Compress / Decompress authorization requirements:

• Superuser Authorization Required:

• Logged in as a uid-0 user -or-

• Have READ authority to the SUPERUSER.FILESYS.PFSCTL resource in the UNIXPRIV FACILITY class.

47




zFS Shrink

• Reduces the physical size of the file system.

• Releases a specified amount of free space from the VSAM linear

data set

• Filesystem must be mounted read/write

• Long running command: FSINFO shows progress

48




zfsadm shrink –aggregate name {–size Kbytes [-noai] | -cancel}

[-trace file_name] [-level] [-help]

where kbytes: new total size of the shrunken filesystem

-noai means “no active increase”

“Active increase” is the default behavior for shrink

49

zFS Shrink




zFS Shrink: Active Increase

Since applications can access the filesystem for most of the shrink

processing, there may be additional blocks to be allocated.

If this allocation causes more space to be needed in the aggregate that

the new total size specified in –size, zFS will actively increase the new

total size.

The shrink command ends in error if the size is actively increased to the

original size of the file system.

If –noai (“no active increase”) is specified on the shrink call and an active

increase is needed, the shrink command ends in error.

50




zFS Shrink Processing

• Moves blocks that are in the portion of the dataset to be released into

the portion that remains.

• Requires scanning each internal filesystem structure to determine if it

owns any blocks that need to be moved.

• The bulk of the time in a shrink operation occurs during this scan

• After block movement completes, the free space is released.

• Filesystem is quiesced briefly while free space is released

51




zFS Shrink

• Use fsinfo output to estimate a reasonable –size for shrink

• zfsadm fsinfo will show:

• Aggregate size (K)

• Free size (8K blocks)

• 1K fragments

• Don’t make -size so small that aggregate will need to grow when

regular activity occurs!

• The difference between the new total size and the current size of the

filesystem cannot be larger than the free size

52




zFS Shrink Example:

Based on zfsadm fsinfo results, reasonable shrink size is 500,000 K

bytes

53

zfsadm shrink -aggr ZFSAGGR.BIGZFS.SHRINK.EXAMPLE -size 500000




zFS Shrink example (post shrink)

54




zFS Shrink: cancel

• zfsadm shrink –aggr <aggrname> -cancel

• If interrupted (via shutdown, unmount –force, zFS owner goes down)

• A new zFS owner after interruption will not continue the shrink operation

55




zFS Shrink authorization requirements:

• UPDATE (or higher) authority to the VSAM LDS

• For a filesystem defined with guaranteed space, the ability to shrink will also be controlled by the management class conditional partial release.

56




zFS Shrink Rules

DFSMS Management Classes

If no management class for an aggregate, shrink allowed.

If management class exists for aggregate –

If aggregate defined with guaranteed space, the conditional partial release setting in the management class determines if shrink is allowed.

If aggregate not defined with guaranteed space, the partial release setting in the management class determines if shrink is allowed.

57




zFS SMF Support

• zFS as of V2R3 will now utilize System Management Facility (SMF) accounting capability.

• zFS V2R3 allows for monitoring of:

• Important file system events (grow, shrink, etc.)

• General performance information similar to what is seen in query command output.

• Per-file system performance information.

58




zFS SMF Support…

• Record type 92 will contain the zFS subtypes 50-59.

• SMF recording in zFS is enabled via a new config variable –smf_recording:

• Options are OFF | ON | ON,xx

• where xx is reporting interval in minutes, range 1-60, default 30

• This IOEFSPRM value may also be changed with zfsadm config

59




zFS SMF 92 Support

• The following are the record subtypes zFS will use:

• When event occurs:

• 50 – File system events (grow, shrink, etc.)

• Interval based:

• 51 – zFS call rates and response times

• 52 – zFS user cache statistics

• 53 – zFS meta cache statistics

• 54 – zFS locking statistics

• 55 – zFS IO statistics

• 56 – zFS token manager statistics

• 57 – zFS storage usage summary

• 58 – zFS sysplex transmit statistics

• 59 – zFS per-file system statistics, may be too often

60




zFS SMF Support

• zFS will record records in two ways:

• Interval based records, subtypes 51-59. Cut at an interval provided

by the user via config option –smf_recording or by SMF’s interval

value if none given.

• File system event subtype 50. Cut when the event occurs.

• Most of the interval based records contain similar information to that

obtained by the zFS query commands.

• Subtype 59 writes out file system specific information and can be very

high volume. It is recommended that this subtype NOT be used unless

necessary.

61




zFS Implicit format on mount

• Allows mount of an unformatted VSAM LDS

• Current process:

1. Create the VSAM LDS (via zfsadm define or DEFINE CLUSTER)

2. Format it as a zFS aggregate (zfsadm format / IOEFSUTL format /

IOEAGFMT)

3. mount aggregate

• Process now allowed as of V2R3

1. Create the VSAM LDS

• zfsadm define

• DEFINE CLUSTER --- must use new ZFS keyword

2. mount aggregateEnsure that the catalog is writable, for where the zFS is to be cataloged (most likely a user catalog).

62



be reproduced in whole or in part without the prior written permission of IBM. 63

zFS Implicit format on mount – create zFS




zFS Implicit format on mount

• Uses installation default for all format parameters

• format_aggrversion (4|5) new default on V2R3

• format_compression (on|off) new in V2R3

• format_encryption (on|off) new in V2R3

• format_perms default: o775 new in V2R3

• Specifies the default permissions for the root directory of the

aggregate for all flavors of format (zfsadm format, IOEAGFMT,

IOEFSUTL format & implicit format on mount) when –perms is

not used.

64




zFS Online Salvage

Only run from a V2R3 system.

A new long-running command which may be used by a systems programmer to verify and / or repair a damaged filesystem, while the filesystem is still mounted.

Damaged filesystems are extremely rare!

Prior to V2R3 salvage was only available as a batch tool (IOEFSUTL or

IOEAGSLV), which required the filesystem be unmounted to run.

This V2R3 addition will allow a filesystem to be repaired while still mounted in the

rare event this is necessary.

65




zFS Online Salvage

Use only when the aggregate cannot be unmounted for repairs.

No writes are allowed to the aggregate while a salvage operation is

running

Verification portion (longest part) of salvage may be interrupted:

Using the –cancel keyword

Unmount immediate with the –force option

During a shutdown

Once repair has begun, salvage cannot be interrupted.

66




zFS Online Salvage - Tip

If a user wishes to “check” a file system for corruption and NOT block

write activity they could use a simple command like:

find . –name ‘*’ –print –exec grep –e “xxxx” {} \;

Where “xxxx” is any string NOT found in any file.

This would scan all directories and files and if you get no assertions

you likely have a good file system.

Online salvage should only be used as a safety valve to verify and / or

repair a damaged file system that cannot be unmounted.

67




zFS Various Other Enhancements

• Changes in defaults

• New configuration options

• zFS authorization changes

• zfsadm chaggr to change attributes of a mounted file system

• Miscellaneous other changes, trace…

68




zFS Changes in defaults

format_aggrversion: changed from 4 to 5

change_aggrversion_on_mount: changed from OFF to ON

honor_syslist: obsolete in V2R3

zFS V2R3 always honors the USS automove options

69




zFS New configuration options

• format_perms

• format_compression

• format_encryption

• edc_buffer_pool

• edc_fixed option added to user_cache_size

• smf_recording

• long_cmd_threads

70




zFS Authorization changes

• Format authorization requirement has been changed to require UPDATE authority to the VSAM data set

• To set owner, groupid, or permissions on format, also require

UID 0 or have READ authority to SUPERUSER.FILESYS.PFSCTL

71




zfsadm chaggr

New admin command which changes the attributes of a mounted

aggregate

aggrfull {threshold,increment | OFF}

aggrgrow {ON|OFF}

rwshare | norwshare

Causes a samemode remount (if mounted read write)

Prior to this support, to change these attributes the aggregate must be

unmounted and re-mounted.

72




zfsadm chaggr syntax

zfsadm chaggr -aggregate name

{-aggrfull{threshold,increment|OFF}|-aggrgrow {ON|OFF}|-rwshare|-norwshare}

[-trace file_name][-level][-help]

73




zfsadm chaggr

Can only be used when all sysplex members are at z/OS V2R3

Only one of –aggrfull, -aggrgrow, -rwshare, -norwshare can be specified at a time

74




-aggrfull

Specifies the threshold and increment for reporting aggregate utilization

messages to the operator.

May be changed to:

threshold,increment or

OFF

75




-aggrgrow

Specifies whether aggregates can be dynamically extended when they become full

chaggr may set it ON or OFF

Aggregate must have a secondary allocation specified (when created) and there must be space on the volumes

76




-RWSHARE and -NORWSHARE

Both are read / write mounts

RWSHARE is “sysplex-aware”

Filesystem is locally mounted on every member of the sysplex

File requests are handled locally by zFS

NORWSHARE is “sysplex-Unaware”

File operations must be “function-shipped” to OMVS owning system

77




zfsadm chaggr

Also will update the zFS mount parameters in the z/OS UNIX couple

dataset, preserving the changes made with chaggr if aggregate is

remounted

(note: all mount parameters are lost if an aggregate is unmounted)

78




zfsadm chaggr: example (BEFORE chaggr)

79




zfsadm chaggr: example

80

1. zfsadm chaggr -aggregate ZFSAGGR.BIGZFS.FS1 -aggrgrow off

2. zfsadm chaggr -aggregate ZFSAGGR.BIGZFS.FS1 -aggrfull 90,5

3. zfsadm chaggr -aggregate ZFSAGGR.BIGZFS.FS1 -norwshare




zfsadm chaggr: example (AFTER chaggr)

81




zfsadm chaggr: authorization requirements

Superuser Authorization Required:

Logged in as a uid-0 user -or-

Have READ authority to the SUPERUSER.FILESYS.PFSCTL resource in the UNIXPRIV racf class.

82




zFS miscellaneous changes

• zfsadm new option: -trace file_name

• Specifies the name of a where trace records from zfsadm call will be put

• z/OS Unix file

• An existing sequential dataset

• A member of an existing PDS or PDSE

• An option on ALL zfsadm commands

• zfsadm commands that query information:

• no longer need READ access to IOEFSPRM file (if used)

• Installations using parmlib (IOEPRMxx) never needed any special

authorization

83




zFS V2R3 Performance

• “Official” IBM Performance Measures

• Your mileage may vary

• Measurements were taken on a IBM zEC12 (model 2827) with 4 CPUs

84




zFS Encryption Performance Measurements

85




zFS Compression Performance Measurements

86




zFS Shrink Performance Measurements

87




zFS Online Salvage Performance Measurements

88




zFS Overall V2R3 Performance (compared to V2R2)

Description of workloads used:

FSPT file I/O workload. random read/writes

(typically 65% reads/35% writes, some with 100% reads)

FS file I/O workload. Mix of random and sequential reads / writes, creates & deletes files

DU Directory reads & writes

DL Directory search & readdir

DL2 Directory search

DUC Intensive directory update workload

DL3 Directory workload, readdir

DL3 plus Directory workload, extended readdir operations

89




zFS Overall V2R3 Performance (compared to V2R2)

90




z/OS UNIX Enhancements

• HFS to zFS utility

• Automatic unmount of version file system in shared file system

• Maintaining a single configuration file across the sysplex

• Assigning default umask values to all z/OS UNIX users

91




z/OS UNIX HFS to zFS utility

• HFS has been stabilized for quite some time. Support for HFS is planned to be removed in 2021.

• It is therefore necessary for data residing in HFS file systems to migrate to zFS file systems.

• Tools such as BPXWH2Z have been provided, however, file system must effectively be taken offline to perform the migration. This approach has not been acceptable for data for some critical applications.

• An online migration facility is provided to replicate and replace an HFS file system with a zFS file system, while mounted and in use.

• Use of this facility obviates the need to take application or system outages to

migrate data off of HFS

• Availability is with the PTF for APAR OA53128.

92





• Command to initiate, modify, and monitor migration is BPXWMIGF

• BPXWMIGF is available from TSO, the UNIX shell, and console (SYSREXX)

• Migration occurs in two phases:

1. Data is mirrored and the active mirror is maintained until swap.

• Applications continue to run and are unaffected during this phase

2. The zFS file system is swapped in replacing the HFS

• Access to the file system is briefly quiesced during the swap

• These phases can be scheduled independently

• Migration must be initiated from the HFS owning system

• BPXWMIGF utility has three basic functions

1. Initiate and modify migration (requires superuser or BPX.SUPERUSER)

2. Query migration

3. Cancel migration (requires superuser or BPX.SUPERUSER)

93





94





95





96

Shell command examples:

• Start a migration

bpxwmigf -source omvs.wjs.hfs -target omvs.wjs.temp -srenameomvs.wjs.old -trename omvs.wjs.zfs

• Check migration status

bpxwmigf -query

• After mirroring complete, update mount statements and do the swap

bpxwmigf -source omvs.wjs.hfs -swap

Note: you might want to use a temp name for the new zFS just to make sure it

does not accidentally get mounted if the migration does not complete





97

HFS to zFS requirements, before using:

• All systems in the OMVS sharing group must be V2.3

• Review mount statements or policies for mounting a file system to be migrated

• HFS file system must be mounted

• zFS file system must be allocated and formatted, ensure file system is properly sized. Do not mount the zFS file system.

• (((Ensure that the z/OS V2.2 enhancement of zFS in the OMVS address space

is not in use.))) Requirement removed with OA53738!

• Limit migrations to only one at a time until swap. Requirement added with

OA53738.

After migration:

• Ensure mount statements or policies will mount the zFS file system

• If you mount the old HFS, you will have regressed your data.




z/OS UNIX Automatic unmount of version file system in shared file system

• D MVS,[,VERSION=(*,UNMOUNT|NOUNMOUNT]) [,VERSION=(*,UNMOUNT|NOUNMOUNT])

98

• Once you have rolled off a version file system (new name for the “version root file

system”), then it was additional manual work to unmount it and everything underneath it.

• Customer Requirement to unmount a version file system when it is not in use anywhere in the shared file system environment.

• Specify in BPXPRMxx: VERSION('nnnn'[,UNMOUNT|NOUNMOUNT])

• VERSION statement applies only to systems that are exploiting shared file

systems. • UNMOUNT to turn on the version auto-unmount option, which will cause the

VERSION_AUTO_UNMOUNT attribute to be set for the version file system if it is mounted on this system.

• You can also dynamically turn on or off the VERSION_AUTO_UNMOUNT attribute using: SETOMVS VERSION=('string',UNMOUNT|NOUNMOUNT)

• You can also change the settings of this parameter by using SET OMVS=(xx) and SETOMVS RESET=(xx) parmlib specifications.




z/OS UNIX Maintaining a single configuration file across the sysplex

• New directory, /global, is available in the sysplex root, for all systems:

• can be used to store and maintain a single copy of a configuration file that can be referenced by all members of the sysplex.

• Example: Need to maintain a consistent automount policy for use by the automount daemon on every member of the sysplex (/global/auto.master and /global/u.map for the /u directory)

• For convenience, provide a view of program products at multiple levels, even though the product may be installed on only one member of the sysplex.

• Example: Want to provide a simple and consistent way to access multiple levels of Java (Java 6, Java 7, Java 8) or COBOL. (/global/java/java8 –> symlink -->

/zOS23/usr/lpp/java8)

• Some users already might have done something similar.

• Prior to V2R3, individual copies of the same configuration file had to be

maintained in each member of the sysplex.

• /global will have no DDDEFs and will not be pointed to for any default installation.

• (For consistency, there is a /global added to the version file system also.)

99




z/OS UNIX Assign default umask values to all z/OS UNIX users

• New UMASK statement of the BPXPRMxx parmlib member can be used to assign a default umask value for all z/OS UNIX users.

• UMASK(nnnn)• nnnn is a 3-to-4 digit octal number, or NONE, that specifies the permission

settings of files or directories created by users or processes that are not to be allowed.

• If a umask value is not specified in BPXPRMxx, the default is NONE, which indicates that no umask default is in effect.

• See the Value Range table in MVS Init and Tuning, for instance, UMASK(007) will give no access to Other.

• Use this setting to better control the files and directories that users on your system can access. Existing facilities (/bin/sh for instance) that set the umask value will continue to override the system default.

• Changing the default umask value might have unexpected results on your applications.

• A umask value that is consistent with the umask value that is set in /etc/profile might be appropriate.

• Rolled back to z/OS V2R2, with APAR OA51000 installed.

100




Summary

zFS:

• Encryption and Compression – only when on z/OS V2.3

• Shrink – free up some space!

• SMF Recording – data galore

• Implicit Format on mount – one less thing to worry about

• Online Salvage (verify / repair) – for when you really need it

• Supplemental: Dynamic Change Aggregate Attributes (chaggr),

Miscellaneous updates, and zFS V2R3 Performance Measurements

z/OS UNIX:

• HFS to zFS utility – intended for those file systems that cannot be unmounted

• Automatic unmount of version file system in shared file system – let the

system manage and monitor this

• Maintaining a single configuration file across the sysplex – convenience

• Assigning default umask values to all z/OS UNIX users – handy, but

considerations are necessary.

101




Appendix

zFS Publication references

–z/OS Distributed File Service zSeries File System Administration

–z/OS Distributed File Service Messages and Codes

–z/OS DFSMSdfp Storage Administration

–z/OS UNIX System Services Command Reference

–z/OS MVS System Management Facilities (SMF)

z/OS UNIX Publication references

• z/OS UNIX System Services Planning

• z/OS UNIX System Services Command Reference

• z/OS Distributed File Service zFS Administration

• z/OS MVS System Messages, Vol 3 (ASB-BPX)

• z/OS UNIX System Services Messages and Codes

latest in zos unix and zfs.ppt - cmg · the latest in z/os unix and zfs marna walle,...

Documents