ibm storage easy tiers

ibm.com/redbooks Redpaper

Front cover

IBM System Storage DS8000 Easy Tier

Gero SchmidtBertrand Dufrasne

Jana JamsekPeter Kimmel

Hiroaki MatsunoFlavio Morais

Lindsay OxenhamAntonio Rainero

Denis Senin

Smart monitoringthree-tier support

Intra-tier and cross-tier performance management

Automated subvolume data relocation and rebalancing

http://www.redbooks.ibm.com/


International Technical Support Organization


January 2012

REDP-4667-02

© Copyright International Business Machines Corporation 2010, 2012. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP ScheduleContract with IBM Corp.

Third Edition (January 2012)

This edition applies to the IBM System Storage DS8700 with DS8000 Licensed Machine Code (LMC) level 6.6.2x.xxx and the IBM System Storage DS8800 with DS8000 Licensed Machine Code (LMC) level 7.6.2x.xxx.

Note: Before using this information and the product it supports, read the information in “Notices” on page vii.

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiTrademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixThe team who wrote this paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixNow you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiComments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiStay connected to IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Summary of changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiJanuary 2012, Third Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Chapter 1. IBM System Storage DS8000 Easy Tier overview . . . . . . . . . . . . . . . . . . . . . 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Chapter 2. IBM System Storage DS8000 Easy Tier concepts, design, and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1 Easy Tier features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 General I/O characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 Easy Tier concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.4 Easy Tier operating modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4.1 Easy Tier Automatic Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.4.2 Easy Tier Manual Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.5 Easy Tier design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.5.1 SSD, Enterprise, and Nearline storage tiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.5.2 Easy Tier performance metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.5.3 Easy Tier cross-tier and intra-tier performance management. . . . . . . . . . . . . . . . 232.5.4 Migration plan creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.5.5 Easy Tier migration types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.5.6 Promoting and swapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.5.7 Auto-rebalance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.5.8 Warm and cold demotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.5.9 Extent allocation in hybrid and managed extent pools . . . . . . . . . . . . . . . . . . . . . 30

2.6 Easy Tier implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.6.1 Easy Tier Automatic Mode control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.6.2 Easy Tier Monitor Mode control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.6.3 Easy Tier considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Chapter 3. Planning for IBM System Storage DS8000 Easy Tier . . . . . . . . . . . . . . . . . 373.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.2 Physical configuration planning and SSD considerations . . . . . . . . . . . . . . . . . . . . . . . 38

3.2.1 Solid-state drive features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.2.2 Solid-state drive configuration limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.2.3 Solid-state drive performance guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.2.4 Physical back-end configuration examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.3 Logical configuration planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.3.1 General considerations for DS8000 logical configurations . . . . . . . . . . . . . . . . . . 503.3.2 Guidelines for creating multitier extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

© Copyright IBM Corp. 2010, 2012. All rights reserved. iii

3.3.3 Guidelines for creating single-tier extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.3.4 Additional considerations for thin provisioned volumes . . . . . . . . . . . . . . . . . . . . 57

3.4 Implementation considerations for multitier extent pools . . . . . . . . . . . . . . . . . . . . . . . 583.4.1 Using thin provisioning with Easy Tier in multitier pools . . . . . . . . . . . . . . . . . . . . 583.4.2 Staged implementation approach for multitier pools . . . . . . . . . . . . . . . . . . . . . . . 58

3.5 Copy Services considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593.6 Workload planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603.7 Planning tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.7.1 Storage Tier Advisor Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613.7.2 Disk Magic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Chapter 4. DS GUI and DS CLI support for IBM System Storage DS8000 Easy Tier . 714.1 DS GUI and DS CLI support for Easy Tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724.2 DS Storage Manager GUI support for Easy Tier. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

4.2.1 Volume migration window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724.2.2 Volume Properties window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724.2.3 Disk Configuration Summary and Internal Storage main windows . . . . . . . . . . . . 734.2.4 Single Pool Properties window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744.2.5 Manage Volumes window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764.2.6 Merge Extent Pools window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774.2.7 Easy Tier controls in the Storage Image Properties window. . . . . . . . . . . . . . . . . 78

4.3 Easy Tier functions with the DS Storage Manager GUI . . . . . . . . . . . . . . . . . . . . . . . . 784.3.1 Dynamic volume relocation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.3.2 Dynamic extent pool merge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884.3.3 Rank depopulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904.3.4 Easy Tier Automatic Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 944.3.5 Exporting the Easy Tier Summary Report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

4.4 DS Storage Manager CLI support for Easy Tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 984.5 Easy Tier functions using the DS Storage Manager CLI. . . . . . . . . . . . . . . . . . . . . . . 100

4.5.1 Dynamic volume relocation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1004.5.2 Dynamic extent pool merge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1084.5.3 Rank depopulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1104.5.4 Easy Tier Automatic Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1154.5.5 Exporting the Easy Tier Summary Report (etdata) . . . . . . . . . . . . . . . . . . . . . . . 117

Chapter 5. Using IBM System Storage DS8000 Easy Tier . . . . . . . . . . . . . . . . . . . . . . 1195.1 Easy Tier usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1205.2 Obtaining and installing the Storage Tier Advisor Tool . . . . . . . . . . . . . . . . . . . . . . . . 1205.3 Collecting DS8000 Easy Tier volume heat data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1215.4 Offloading the Easy Tier Summary Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1235.5 Processing the Easy Tier Summary Report with the Storage Tier Advisor Tool . . . . . 125

5.5.1 Starting the Storage Tier Advisor Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1265.5.2 Storage Tier Advisor Tool output files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

5.6 Interpreting the Storage Tier Advisor Tool output . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1275.6.1 Storage Tier Advisor Tool examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

5.7 Easy Tier Manual Mode usage scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1385.7.1 Volume migration across extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1395.7.2 Consolidation of extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1405.7.3 Restriping of volumes in non-managed extent pools . . . . . . . . . . . . . . . . . . . . . 1445.7.4 Removing ranks from extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Appendix A. IBM System Storage DS8000 Easy Tier and IBM z/OS system-managed storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

z/OS system-managed storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

iv IBM System Storage DS8000 Easy Tier

Easy Tier storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147Hybrid pools in z/OS system-managed storage environments . . . . . . . . . . . . . . . . . . . . . 148

Appendix B. IBM System Storage DS8000 storage virtualization . . . . . . . . . . . . . . . 149The abstraction layers for disk virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

Array site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150Ranks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151Extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152Logical volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153Space-efficient volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154Allocation, deletion, and modification of LUNs or CKD volumes. . . . . . . . . . . . . . . . . . 155

Summary of the virtualization hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162How to get Redbooks publications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Contents v

vi IBM System Storage DS8000 Easy Tier

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.

© Copyright IBM Corp. 2010, 2012. All rights reserved. vii

Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml

The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:

AIX®DB2®DS4000®DS6000™DS8000®Easy Tier®Enterprise Storage Server®FlashCopy®i5/OS®

IBM®MVS™Power Systems™PowerHA®PowerVM®Power®Redbooks®Redpaper™Redbooks (logo) ®

RMF™System i®System Storage®System z®Tivoli®XIV®z/OS®

The following terms are trademarks of other companies:

Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Other company, product, or service names may be trademarks or service marks of others.

viii IBM System Storage DS8000 Easy Tier

http://www.ibm.com/legal/copytrade.shtml

Preface

This IBM® Redpaper™ publication describes the concepts and functions of IBM System Storage® Easy Tier® and explains its practical use with the IBM System Storage DS8700 and the IBM System Storage DS8800 systems.

The IBM System Storage DS8000® Easy Tier feature and the accompanying Storage Tier Advisor Tool (STAT), as described in this paper, are available for the DS8700 with Licensed Machine Code (LMC) level 6.6.2x.xxx or higher and the DS8800 with Licensed Machine Code (LMC) level 7.6.2x.xxx or higher.

IBM System Storage DS8000 Easy Tier is designed to automate data placement throughout the storage system disks pool. It enables the system, automatically and without disruption to applications, to relocate data (at the extent level) across up to three drive tiers. The process is fully automated. Easy Tier also automatically rebalances extents among ranks within the same tier, removing workload skew between ranks, even within homogeneous and single-tier extent pools.

Easy Tier supports a Manual Mode that enables the user to relocate full volumes. Manual Mode also offers the ability to merge extent pools and to restripe volumes within an extent pool. A rank depopulation function is also offered. Easy Tier now fully supports thin provisioned fixed-block volumes (ESE) in Manual Mode and Automatic Mode.

This paper is aimed at those professionals who want to understand the Easy Tier concept and its underlying design. It also provides guidance and practical illustrations for users who want to use the Easy Tier Manual Mode capabilities.

The team who wrote this paper

This paper was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose, CA.

Gero Schmidt is an IT Specialist in the IBM ATS technical sales support organization in Germany. He joined IBM in 2001 at the European Storage Competence Center (ESCC) in Mainz, providing technical support for a broad range of IBM System Storage products (IBM Enterprise Storage Server®, IBM System Storage DS4000®, IBM Systems Storage DS5000, IBM System Storage DS6000™, DS8000, IBM SAN Volume Controller, and IBM Storage System XIV®) in open systems environments with a primary focus on IBM Enterprise disk storage solutions, storage system performance, and IBM Power® Systems™ with IBM AIX®, including IBM PowerVM® and IBM PowerHA®. He participated in the product rollout and major release beta test programs of the DS6000 and DS8000 series. He has been a speaker at several international IBM technical conferences and co-author of several IBM Redbooks® publications. He holds a degree in Physics (Dipl.-Phys.) from the Technical University of Braunschweig, Germany.

Bertrand Dufrasne is an IBM Certified Consulting I/T Specialist and Project Leader for System Storage disk products at the International Technical Support Organization, San Jose, CA. He has worked at IBM in various I/T areas. He has authored many IBM Redbooks publications and has also developed and taught technical workshops. Before joining the ITSO, he worked for IBM Global Services as an Application Architect. He holds a Master’s degree in Electrical Engineering.

© Copyright IBM Corp. 2010, 2012. All rights reserved. ix

Jana Jamsek is an IT Specialist at IBM Slovenia. She works in Storage Advanced Technical Support for Europe as a specialist for IBM Storage Systems and the IBM i (i5/OS®) operating system. Jana has eight years of experience in working with the IBM System i® platform and its predecessor models, as well as eight years of experience in working with storage. She has a Master’s degree in Computer Science and a degree in mathematics from the University of Ljubljana in Slovenia.

Peter Kimmel is an IT Specialist and ATS team lead of the Enterprise Disk Solutions team at the European Storage Competence Center (ESCC) in Mainz, Germany. He joined IBM Storage in 1999 and since then has worked with all the various Enterprise Storage Server (ESS) and DS8000 generations, with a focus on architecture and performance. He has been involved in the Early Shipment Programs (ESPs) of these products, early installations, and has co-authored several IBM Redbooks publications about DS8000. Peter holds a Diploma (MSc) degree in Physics from the University of Kaiserslautern.

Hiroaki Matsuno is an IT Specialist at IBM Japan. He has 3 years of experience in IBM storage system solutions from working in the IBM ATS System Storage organization in Japan. His areas of expertise include DS8000 Copy Services, SAN, and Real-time Compression Appliance in open systems environments. He holds a Master of Engineering degree from the University of Tokyo, Japan.

Flavio Morais is a GTS Storage Specialist in Brazil. He has 6 years of experience in the SAN / storage field. He holds a degree in Computer Engineering from Instituto de Ensino Superior de Brasilia. His areas of expertise include DS8000 Planning, Copy Services, IBM Tivoli® Storage Productivity Center, and performance troubleshooting. He has worked extensively on performance problems in open systems.

Lindsay Oxenham is a Mainframe Storage Specialist working in Melbourne, Australia. He has over 30 years of experience in the mainframe environment as an application programmer and in performance and tuning areas. He joined IBM in 1998 and has been working in the storage area since 2005. He has a Bachelor’s degree in Applied Science (Computing). He has presented papers at SAS user meetings and Computer Management Group of Australia (CMGA) conferences.

Antonio Rainero is a Certified IT Specialist working for Integrated Technology Services organization at IBM Italy. He joined IBM in 1998 and he has more than 10 years of experience in the delivery of storage services both for IBM z/OS® and open systems customers. His areas of expertise include storage subsystems implementation, performance analysis, storage area networks, storage virtualization, disaster recovery, and high availability solutions. Antonio holds a degree in Computer Science from University of Udine, Italy.

Denis Senin is an IT Specialist at IBM Russia. He has 10 years of experience in the IT industry and has worked at IBM for 6 years. Denis holds a Master’s degree in Design-engineer of Computer Systems from The Moscow State Institute of Radiotechnics, Electronics, and Automatics. He has experience with systems design and development. His current areas of expertise include open systems high-performing and disaster recovery storage solutions.

x IBM System Storage DS8000 Easy Tier

Figure 1 The team: Hiroaki, Antonio, Flavio, Peter, Bertrand, Denis, Jana, Gero, Lindsay

Thanks to the authors of the previous editions of this paper.

� Authors of the first edition, IBM System Storage DS8000 Easy Tier, published in August 2010, were: Werner Bauer, Brenda Careaga, Jukka Myyrylainen, Antonio Rainero, Paulus Usong

� Authors of the second edition, IBM System Storage DS8000 Easy Tier, published in August 2011, were: Stephane Couteau, Martin Jer, Richard Murke, Massimo Rosichini

Thanks to the following people for their contributions to this project:

Lawrence Chiu, Nick Clayton, Lee La Frese, Yang Liu (Loren), Michael Lopez, Paul Muench, Rick Ripberger, Louise Schillig, Cheng-Chung Song

Now you can become a published author, too!

Here's an opportunity to spotlight your skills, grow your career, and become a published author - all at the same time! Join an ITSO residency project and help write a book in your area of expertise, while honing your experience using leading-edge technologies. Your efforts will help to increase product acceptance and customer satisfaction, as you expand your network of technical contacts and relationships. Residencies run from two to six weeks in length, and you can participate either in person or as a remote resident working from your home base.

Learn more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

Preface xi

http://www.redbooks.ibm.com/residencies.html

http://www.redbooks.ibm.com/residencies.html

Comments welcome

Your comments are important to us!

We want our papers to be as helpful as possible. Send us your comments about this paper or other IBM Redbooks publications in one of the following ways:

� Use the online Contact us review Redbooks form found at:

ibm.com/redbooks

� Send your comments in an email to:

[email protected]

� Mail your comments to:

IBM Corporation, International Technical Support OrganizationDept. HYTD Mail Station P0992455 South RoadPoughkeepsie, NY 12601-5400

Stay connected to IBM Redbooks publications

� Find us on Facebook:

http://www.facebook.com/IBMRedbooks

� Follow us on twitter:

http://twitter.com/ibmredbooks

� Look for us on LinkedIn:

http://www.linkedin.com/groups?home=&gid=2130806

� Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks publications weekly newsletter:

https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm

� Stay current on recent Redbooks publications with RSS Feeds:

http://www.redbooks.ibm.com/rss.html

xii IBM System Storage DS8000 Easy Tier

http://www.facebook.com/IBMRedbooks

http://twitter.com/ibmredbooks



http://www.redbooks.ibm.com/contacts.html

http://www.linkedin.com/groups?home=&gid=2130806

https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm

http://www.redbooks.ibm.com/rss.html

Summary of changes

This section describes the technical changes made in this edition of the paper. This edition might also include minor corrections and editorial changes that are not identified.

Summary of Changesfor IBM System Storage DS8000 Easy Tieras created or updated on January 18, 2012.

January 2012, Third Edition

This revision reflects the addition, deletion, or modification of new and changed information regarding the new features that were introduced with IBM System Storage DS8000 LMC Release 6.2:

New and changed information� Easy Tier Automatic Mode support for three-tier managed extent pools� Easy Tier Automatic Mode support for single-tier managed extent pools (auto-rebalance)� Easy Tier Automatic Mode control changes (all, tiered, and none)� Easy Tier support for thin provisioned volumes (ESE)� Easy Tier extent migration type enhancements (expanded cold demote)� Storage Tier Advisor Tool (STAT) enhancements

© Copyright IBM Corp. 2010, 2012. All rights reserved. xiii

xiv IBM System Storage DS8000 Easy Tier

Chapter 1. IBM System Storage DS8000 Easy Tier overview

In this chapter, we introduce the basic concepts of dynamic data relocation. We show how these concepts are implemented in IBM System Storage DS8000 with the IBM System Storage DS8000 Easy Tier feature.

1

© Copyright IBM Corp. 2010, 2012. All rights reserved. 1

1.1 Introduction

In modern and complex application environments, the increasing and often unpredictable demands for storage capacity and performance lead to issues in terms of planning and optimization of storage resources.

Let us consider typical storage management issues:

� Usually when a storage system is implemented, only a portion of the configurable physical capacity is deployed. When the storage system runs out of the installed capacity and additional capacity is requested, a hardware upgrade is implemented to add new physical resources to the storage system. This new physical capacity can hardly be configured to keep an even spread of the overall storage resources. Typically, the new capacity is allocated to fulfill only new storage requests. The existing storage allocations do not benefit from the new physical resources. Similarly, the new storage requests do not benefit from the existing resources; only new resources are used.

� In a complex production environment, it is not always possible to optimize storage allocation for performance. The unpredictable rate of storage growth and the fluctuations in throughput (I/O per second (IOPS)) requirements often lead to inadequate performance. Furthermore, the tendency to use even larger volumes to simplify storage management works against the granularity of storage allocation, and a cost-efficient storage tiering becomes difficult to achieve. This situation is true with the introduction of high performing, but expensive, technologies such as solid-state drives.

� The move to larger and larger physical disk drive capacities means that previous access densities achieved with low-capacity drives can no longer be sustained.

� Any business has applications that are more critical than others, and there is a need for specific application optimization, and therefore a need for the ability to relocate specific application data on faster storage media.

All these issues deal with data placement and relocation capabilities. Most of these issues can be managed by having spare resources available and by moving data, using data mobility tools or operating systems features (such as host level mirroring), in more suitable storage configurations. However, all these corrective actions are expensive in terms of hardware resources, labor, and service availability. Relocating data among the physical storage resources dynamically, that is, transparently to the attached host systems, is becoming increasingly important.

The IBM System Storage DS8000 Easy Tier feature has been designed to specifically address these issues. Easy Tier is a DS8000 built-in dynamic data relocation feature that allows host-transparent movement of data among the storage system resources. This feature significantly improves configuration flexibility and performance tuning and planning.

No cost: Easy Tier is a free (no charge) feature of IBM System Storage DS8000 (DS8700 with microcode R5.1 or higher and DS8800 with microcode R6.1 or higher). However, as with any other acquired licensed function, the Easy Tier licensed function must first be ordered from IBM. The necessary Easy Tier activation codes can then be obtained from the IBM Disk Storage Feature Activation (DSFA) website at:

http://www.ibm.com/storage/dsfa

These codes are then applied using the respective DS CLI or GUI command. See Chapter 10, “IBM System Storage DS8000 features and license keys”, in IBM System Storage DS8000: Architecture and Implementation, SG24-8886, for more information about how to obtain and activate DS8000 license keys.

2 IBM System Storage DS8000 Easy Tier


Easy Tier has been developed by IBM Storage Development in partnership with IBM Research. See the IBM Almaden Research Easy Tier project page at this website for more information:

http://www.almaden.ibm.com/storagesystems/projects/easytier/

Also see IBM System Storage DS8700 and DS8800 Introduction and Planning Guide, GC27-2297-07 for more information about Easy Tier or IBM System Storage DS8800 and DS8700 Performance with Easy Tier 3rd Generation, WP102024, for the latest performance enhancements with DS8000 R6.2 LMC and Easy Tier.

1.2 Capabilities

Easy Tier is an optional and no charge feature on the DS8700 and the DS8800 that offers enhanced capabilities through automated hot spot management and data relocation, auto-rebalancing, manual volume rebalancing and volume migration, rank depopulation, merging of extent pools, and thin provisioning support. Easy Tier determines the appropriate tier of storage based on data access requirements and then automatically and nondisruptively moves data, at the subvolume or sub-LUN level, to the appropriate tier on the DS8000.

The features of IBM Easy Tier can be summarized in two operating modes:

� Easy Tier Automatic Mode:

Easy Tier Automatic Mode is designed to automatically optimize storage performance and storage economics management across different drive tiers through data placement on a subvolume level in multitier or hybrid extent pools. Multitier or hybrid extent pools are storage pools that contain a mix of different disk drive technologies or storage tiers. It can automatically and nondisruptively relocate data at the subvolume level (extent level) across different drive tiers or even within the same drive tier according to its data temperature (I/O activity) to optimize performance and resource utilization. This feature significantly improves the overall storage cost performance ratio and simplifies storage performance tuning and management.

Easy Tier Automatic mode manages any combination of the three disk drive technology tiers available for the DS8000 series. On the DS8700, the three disk technologies supported are:

– Solid-state drives (SSDs)– Fibre Channel (FC) Enterprise disks– Serial Advanced Technology Attachment (SATA) Nearline disks

On the DS8800 the three disk technologies supported are:

– Solid-state drives (SSDs)– Serial Attached SCSI (SAS) Enterprise disks– SAS Nearline disks

Easy Tier allows cold demotion and warm demotion. Cold demotion aims to optimize the extents placement across the tiers moving inactive or with specific activity pattern extents from a higher performance to a lower-cost tier. Warm demotion is designed to prevent the activity overload of a higher performance tier by demoting extents to a lower-cost tier.

In Automatic Mode, Easy Tier also provides an auto-rebalance capability that adjusts the system to continuously provide excellent performance by balancing the load on the ranks within a given tier in an extent pool.

Chapter 1. IBM System Storage DS8000 Easy Tier overview 3


� Easy Tier Manual Mode:

Easy Tier Manual Mode allows a set of manually initiated actions to relocate data among the storage system resources in a dynamic fashion (without any disruption of the host operations). The Manual Mode capabilities include dynamic volume relocation, dynamic extent pool merge, and rank depopulation. Dynamic volume relocation allows a DS8000 volume to be migrated to the same or another extent pool. This capability also provides the means to manually rebalance the extents of a volume across ranks when additional capacity is added to the pool. Dynamic extent pool merge allows an extent pool to be merged to another extent pool. Rank depopulation allows you to remove an allocated rank from an extent pool and relocate the allocated extents to the other ranks in the pool.

Combining these different capabilities greatly improves the configuration flexibility of the DS8000 providing ease of use.

Even though both modes have data relocation capabilities, Manual Mode and Automatic Mode do not have the same goals:

� Easy Tier Manual Mode enables operations such as dynamic volume relocation and dynamic extent pool merge that simplify manual DS8000 storage management regarding capacity and performance needs.

� Easy Tier Automatic Mode enables automated storage performance and storage economics management through automated data placement across or even within storage tiers on extent level. Automatic Mode provides automated tiering capabilities on subvolume (extent) level across different physical resources with various performance and cost characteristics.

Starting with code R6.2, Easy Tier supports both regular and thin provisioned extent space-efficient (ESE) volumes.

The first generation of Easy Tier introduced automated storage performance management by efficiently boosting Enterprise-class performance with SSDs and automating storage tiering from Enterprise-class drives to SSDs, thus optimizing SSD deployments with minimal costs. It also introduced dynamic volume relocation and dynamic extent pool merge.

The second generation of Easy Tier added automated storage economics management by combining Enterprise-class drives with Nearline drives with the objective to maintain Enterprise-tier performance while shrinking footprint and reducing costs with large capacity Nearline drives. The second generation also introduced intra-tier performance management (auto-rebalance) for hybrid pools as well as manual volume rebalance and rank depopulation.

The third generation of Easy Tier introduced further enhancements providing automated storage performance and storage economics management across all three drive tiers, which allows you to consolidate and efficiently manage more workloads on a single DS8000 system. It also introduced support for auto-rebalance in homogeneous pools and support for thin provisioned (extent space-efficient (ESE)) volumes.

Besides the host transparent data relocation capability, Easy Tier also provides monitoring capabilities that allow back-end workload data collection. The Easy Tier performance data collection is available on DS8800 systems (with LMC level 7.6.10.xxx/R6.1 or higher) and DS8700 systems (with LMC level 6.5.1.xxx/R5.1 or higher), whether the Easy Tier LIC feature is applied.


These Easy Tier performance statistics are used by the Easy Tier Automatic Mode facility to identify data that might benefit from a relocation to higher or lower storage tier and create the appropriate migration plans. The statistical data can be downloaded and further processed with the Storage Tier Advisor Tool. The Storage Tier Advisor Tool provides information about the workload heat distribution in the data at volume level and also provides configuration recommendations with performance improvement predictions. This powerful tool allows a closer analysis of the workload characteristics and helps to evaluate the potential benefits from adding new drive resources like solid-state drive technology to the system.

More information about the Storage Tier Advisor Tool can be found in 5.2, “Obtaining and installing the Storage Tier Advisor Tool” on page 120 and 5.5, “Processing the Easy Tier Summary Report with the Storage Tier Advisor Tool” on page 125.

No cost: The Storage Tier Advisor Tool (STAT) is available at no cost.

Chapter 1. IBM System Storage DS8000 Easy Tier overview 5

Chapter 2. IBM System Storage DS8000 Easy Tier concepts, design, and implementation

In this chapter, we introduce the basic concept, design, and implementation of the Easy Tier feature available on the IBM System Storage DS8700 and IBM System Storage DS8800 storage systems.

2


2.1 Easy Tier features

Easy Tier Automatic Mode is based on the concept of managing multiple tiers, or classes, of storage, consisting of different storage technologies. Starting with DS8000 R6.2 at Licensed Machine Code (LMC) level 7.6.20.xx for the DS8800 and LMC level 6.6.20.xx for the DS8700, those classes can consist of either Enterprise disk (Serial Attached SCSI (SAS), for the DS8800, and Fibre Channel for DS8700), Nearline disk (SAS 7.2 K for the DS8800, and Serial Advanced Technology Attachment (SATA) for DS8700), or solid-state drives (SSDs).

Easy Tier is designed to automatically optimize storage performance and storage economics management across different drive tiers through data placement on a subvolume level in multitier or hybrid extent pools. Multitier or hybrid extent pools are storage pools that contain a mix of different drive tiers. They can nondisruptively relocate data at extent level across different drive tiers or even within the same drive tier to optimize performance and resource utilization. The auto-rebalance capability automatically rebalances the workload even across the physical resources within a drive tier, reducing the occurrence of hotspots.

The first generation of Easy Tier introduced automated storage performance management by efficiently boosting Enterprise-class performance with SSDs and automating storage tiering from Enterprise-class drives to SSDs, thus optimizing SSD deployments with minimal costs. The second generation of Easy Tier added automated storage economics management by combining Enterprise-class drives with Nearline drives with the objective to maintain Enterprise-tier performance while shrinking footprint and reducing costs with large capacity Nearline drives. The third generation of Easy Tier introduced further enhancements providing automated storage performance management and storage economics management across all three drive tiers that allows you to consolidate and efficiently manage more workloads on a single DS8000 system.

Easy Tier also has a Manual Mode, covering user-requested functions that allow you to manually merge extent pools, depopulate ranks, and relocate volumes. This mode applies to both single-tier (homogeneous) and multitier (hybrid) extent pools. In Manual Mode, volumes can be relocated between extent pools, either to balance the performance across the extent pools or to move a volume to a more appropriate extent pool, such as an SSD or a hybrid extent pool. With Manual Mode, it is also possible to perform a manual volume rebalance by relocating volumes within the same extent pool to evenly restripe the volumes and balancing the volumes’ capacities across all the ranks in that extent pool. Manual volume rebalance or manual rebalance applies only to single-tier or homogeneous extent pools that are not already enabled for Easy Tier Automatic Mode management.

All of these functions are performed dynamically without interrupting host I/Os within the DS8000 system.

Modes: Easy Tier Automatic Mode and Manual Mode are not exclusive, which means that you can use Manual Mode capabilities even if Automatic Mode is active.

Thin provisioning: Starting with DS8000 R6.2 LMC, Easy Tier introduced full support of thin provisioned volumes (extent space-efficient (ESE)) in Automatic Mode and Manual Mode (dynamic volume relocation, extent pool merge, and rank depopulation).


2.2 General I/O characteristics

The workload or I/O characteristics of any installation is unique, even though there are general or similar patterns. Many installations generally do more random I/Os during the day when online transactions are the main activity. After the main online period finishes, the batch period starts, during which time the I/Os become more sequential in nature. For example, with typical financial institutions, the market open period is a critical time because of the flood of transactions that come in during a relatively short period. Month-end, quarter-end, and year-end periods are generally times when I/O transactions might double or even more, compared to normal days.

An active volume (LUN) during the online period might become a close to idle volume during the batch period. A busy volume during the online period on one day might not have much activity on the following day.

These variables make it next to impossible to manually track the changes in the I/O characteristics of each volume, let alone to manage the location or placement of those volumes based on their level of activity. Even within a volume, there are hot spots, meaning that the I/Os are not evenly distributed across all extents within the volume. Certain extents are hotter (more active) compared to other extents within the same volume. As you can infer from the foregoing facts, any attempt for a manual performance micro-management at the extent level with a manual placement of data extents on the appropriate storage resource is definitely an impossible task.

This situation is where Easy Tier, operating in Automatic Mode, solves this problem.

2.3 Easy Tier concepts

The availability of storage with various performance characteristics allows the user to directly place data on the most appropriate storage according to performance and business relevance requirements. It is possible to do this task using a manual approach by using existing data migration tools or by using the Easy Tier Manual Mode facilities, as described in the 2.4.2, “Easy Tier Manual Mode” on page 13. Nevertheless, the manual approach has certain limitations:

� Data granularity: In Manual Mode, the data placement is at the volume level. But with the ever increasing size of the volumes, this level of granularity can lead to inefficient usage of valuable and costly higher-performance storage resources.

� Performance requirement variability: In complex application environments, the workload characteristics are variable. This situation leads to additional efforts to move the volumes in a timely manner according to the workload performance changes.

Furthermore, moving the data across the storage system resources (that is, physical disks) is always an expensive task in terms of labor and service availability. Nonetheless, data relocation is often required to adapt to physical configuration changes and new performance requirements. This situation is particularly true when various disk technologies are implemented on the same hardware.

Chapter 2. IBM System Storage DS8000 Easy Tier concepts, design, and implementation 9

With the availability of SSD in storage systems such as the DS8000, the question arises about how to use SSDs effectively and efficiently. Considering the superior performance of SSDs compared to HDDs, it is definitely an advantage to replace all HDDs with SSDs on the disk subsystem. However, given the much higher cost of SSDs, this action is generally not a viable solution from an economical point of view. There needs to be a balance between cost and performance. The same reasoning also applies to having more than two storage tiers, for example, with combinations of Enterprise, Nearline, and SSD disks.

Easy Tier Automatic Mode automatically and economically optimizes the data placement on a DS8000.

Currently, IBM supports up to three drive classes or storage tiers configured in the same DS8000 storage subsystem:

1. SSD disks: This class is the highest performance drive class currently available.

2. Enterprise disks (SAS 15 K/10 K rpm for DS8800 and FC 15 K for DS8700): This class is the high-performance drive class.

3. Nearline disks (SAS 7.2 K for DS8800 and SATA for DS8700). This class is the low-cost storage drive class.

Extent pools containing ranks of more than one drive class are called hybrid or multitier pools. Instead, extent pools containing ranks of only a single drive class are referred to as homogeneous or single-tier pools.

Starting with R6.2, Easy Tier manages extent pools containing up to three storage tiers. Easy Tier is able to optimize extent placement across all different storage tiers or drive classes that are available on a DS8000 system.

Other terms that we often use in this book are hot data, warm data, and cold data. These terms match the three priority levels that can be applied to extents that are monitored by Easy Tier Automatic Mode to define a migration plan for those extents.

Hot data refers to data that has more I/O workload compared to other extents in the same extent pool and in the same tier. The lower percentage of hot data, the greater the skew in the environment, and the higher the benefit of a small amount of solid-state drives. Hot data is promoted to a higher tier with a significant performance improvement.

Cold data is data that has either low workload levels or no workload activity at all. Cold data does not benefit from a higher tier and is not promoted. It is demoted to the lowest available tier.

Warm data is the rest of the workload that is not hot or cold and could either be promoted (if this action leads to a performance improvement) or demoted depending on the precise workload levels and configuration.

When we refer to data here, we mean extent, because Easy Tier, when used in Automatic Mode, performs its operations at the extent level.

Buckets: The Easy Tier Automatic Mode internal algorithm manages a higher number of priority levels (also called buckets), and then a finer granularity for extent temperature monitoring. Extent distribution in those buckets is computed dynamically for each migration plan, depending on the extents’ activity. More information about migration plans can be found in 2.5, “Easy Tier design” on page 21.


Business users know which applications are their important, business-critical applications, and based on that indication, the storage administrator can determine which are the relevant volumes used by those applications. After the volumes have been identified and their sizes are known, the storage administrator can determine how many higher performance tier ranks are needed to accommodate all those volumes. Placing all those volumes on higher performance tier ranks can definitely provide a significant performance improvement to the application.

The drawback here is that probably a significant number of higher performance ranks might be required, which makes the cost prohibitive. In addition, not all volumes are hot, which means that you might be, inefficiently, placing cold volumes on higher performance ranks. Furthermore, even on a volume that is considered hot, probably not all extents on that volume are hot. This situation can mean that you are not using the available storage on higher performance ranks as effectively as you might prefer.

A performance monitoring function, available within the DS8000 microcode, collects performance metrics, at the extent level, for potentially every extent in every volume that is on the DS8000.

The Storage Tier Advisor Tool (STAT) can analyze the data collected by the Easy Tier monitoring function and produce a Volume Heat Distribution or heat map. The Storage Tier Advisor Tool is described in Chapter 5, “Using IBM System Storage DS8000 Easy Tier” on page 119. The heat map reflects the degree of hotness or temperature of each extent.

Easy Tier enables dynamic data relocation to achieve the optimum performance on the DS8000. There are two modes of operations for Easy Tier, as described in the following sections.

2.4 Easy Tier operating modes

Here we provide a more detailed description of the Easy Tier Automatic and Manual Modes of operation.

2.4.1 Easy Tier Automatic Mode

Easy Tier Automatic Mode provides the capability to automatically and economically optimize the data placement on a DS8000. In Automatic Mode, Easy Tier dynamically manages the capacity in single-tier (homogeneous) extent pools (auto-rebalance) and multitier (hybrid) extent pools containing up to three different disk tiers. These tiers can be made of any of these three disk classes:

� Solid-state disks (SSD)� Enterprise class disks (ENT = 10 K / 15 K rpm SAS or FC)� Nearline (NL = NL-SAS, or SATA) disks

Easy Tier Automatic Mode can be enabled for all extent pools (including single-tier pools), for only multitier pools, or no extent pools, which means disabled. Extent pools handled by Easy Tier are referred to as managed pools. Extent pools not handled by Easy Tier Automatic Mode are referred to as non-managed pools. With DS8000 R6.2 LMC, Easy Tier Automatic Mode supports regular and thin-provisioned volumes (ESE volumes using extent space-efficient storage allocation method).


Table 2-1 summarizes the drive class and storage tier assignments in allowed multitier configurations for DS8700 and DS8800 models.

Table 2-1 DS8700 / DS8800 multitier configurations

Easy Tier Automatic Mode manages the data relocation both across different tiers (inter-tier or cross-tier management) and within the same tier (intra-tier management).

The cross-tier or inter-tier capabilities deal with the Automatic Data Relocation (ADR) feature that aims to relocate the extents of each logical volume to the most appropriate storage tier within the extent pool to improve the overall storage cost-to-performance ratio. This task is done without any user intervention and is fully transparent to the application host. Logical volume extents with high latency in the rank are migrated to storage media with higher performance characteristics, while extents with low latency in the rank are kept in storage media with lower performance characteristics.

After a migration of extents is finished, the degree of hotness of the extents does not stay the same over time. Eventually, certain extents on a higher performance tier become cold and other extents on a lower-cost tier become hotter compared to cold extents on the higher performance tier. When this event happens, cold extents on a higher performance tier are eventually demoted or swapped to a lower-cost tier and replaced by new hot extents from the lower-cost tier. Easy Tier always evaluates first if the “cost” of moving an extent to a higher performance tier is worth the performance gain expected. This migration scenario is shown in Figure 2-1.

Figure 2-1 Easy Tier Automatic Mode

Configuration SSD ENT (10 K / 15 K SAS or FC)

NL (NL - SAS or SATA)

SSD, ENT, and NL Tier 0 Tier 1 Tier 2

SSD and ENT Tier 0 Tier 1 N/A

SSD and NL Tier 0 N/A Tier 1

ENT and SATA N/A Tier 1 Tier 2

Tip: Easy Tier tries to keep extents in a higher performance tier if possible. Extent relocation to a lower tier can occur proactively (cold demote), reactively (warm demote), or can occur if a higher tier is running out of space.

Volume Multi-tier or Hybrid Extent Pool

Higher Performance Tier Rank

Lower Performance Tier Ranks

Hot extents migrate UP

Cold Extents migrate DOWN

Ext

ent

Vir

tual

izat

ion


Additionally, Easy Tier Automatic Mode provides intra-tier capabilities (auto-rebalance) that performs an automatic extent rebalancing within a tier of a managed extent pool. This action allows the systems to avoid hotspots or rank saturation that could result from an unbalanced workload distribution across the ranks of the same drive tier. Auto-rebalance also takes advantage of new ranks when they are added to the storage pool by automatically relocating data to these ranks and redistributing the workload to achieve a balanced rank utilization.

In addition to the inter-tier and intra-tier management, Easy Tier Automatic Mode can also handle storage device variations within a tier using a micro-tiering capability. An example of storage device variations within a tier is when there is an intermix of ranks with different RAID levels or different disk rpm within the same storage tier of an extent pool. A typical micro-tiering scenario is, for example, when, after a hardware upgrade, new 15 K rpm Enterprise disk drives intermix with existing 10 K rpm Enterprise disk drives. In these configurations, the Easy Tier Automatic Mode micro-tiering capability takes into account the different performance profiles of each micro-tier and perform inter-tier and intra-tier (auto-rebalance) optimizations accordingly. Easy Tier does not handle a micro-tier like an additional tier; it is still part of a specific tier. For this reason, the extent hotness does not trigger any promotion or demotion across micro-tiers of a same tier, and the extent relocation across micro-tiers can occur only as part of the auto-rebalance feature.

2.4.2 Easy Tier Manual Mode

Easy Tier Manual Mode provides the following extended capabilities for logical configuration management: Dynamic volume relocation including manual volume rebalance, dynamic extent pool merge, and rank depopulation capabilities.

Easy Tier Manual Mode capabilities are user initiated. The standard DS8000 management interfaces (DS CLI and DS GUI) can be used to start the Easy Tier Manual Mode functions.

The dynamic data movements are performed transparently to the attached host systems. This means that the host applications can continuously and concurrently access the logical volumes while the data migration is being performed.

Dynamic extent pool mergeThe dynamic extent pool merge capability allows a user to initiate a merging process of one extent pool (source extent pool) into another extent pool (target extent pool). During this process, all the volumes in both source and target extent pools remain accessible to the hosts.

Important: Easy Tier moves data gradually to avoid contention with I/O activity associated with production workloads. It does not move extents unless a measurable latency benefit can be realized by the move. The impact associated with Easy Tier management is so small that the effect on overall system performance is nearly undetectable.

Limitations: The dynamic extent pool merge is allowed only among extent pools with the same server affinity or rank group. Additionally, the dynamic extent pool merge is not allowed:

� If source and target pools have different storage types (FB and CKD).� If both extent pools contain track space-efficient (TSE) volumes.� If there are TSE volumes on the SSD ranks.� If you have selected an extent pool that contains volumes that are being migrated.� If the combined extent pools would have 2 PB or more of ESE logical capacity.


As summarized in Figure 2-2, a dynamic extent pool merge consists of the following operations:

1. Changing the assignment of ranks in the source extent pool to the target extent pool

2. Changing the assignment of volumes and SE repositories in the source extent pool to the target extent pool

3. Deleting the source extent pool that does not have ranks or volumes assigned anymore.

Figure 2-2 Dynamic extent pool merge process

An extent pool merging process can be started to perform the following operations:

� Homogeneous extent pool consolidation:

Creating larger homogeneous pools can allow new volumes to be spread over a greater quantity of physical resources (DDMs), improving the overall performance. The existing volumes can be redistributed in a merged extent pool using either manual volume rebalance in non-managed pools or auto-rebalance (see 2.5.7, “Auto-rebalance” on page 28) in managed pools.

� Hybrid extent pool creation:

Consolidating two extent pools to create a merged multitier extent pool with mixed storage technologies managed by the Easy Tier Automatic Mode facility.

Important: No actual data movement is performed during a dynamic extent pool merge; only logical definition updates occur.

EPFC00

EPFC00 PoolLUNs

Server running

rank0 rank1 rank2 rank4 rank5 rank6

rank0 rank1 rank2 rank4 rank5 rank6

EPFC00

EPFC02

EPFC02 PoolLUNs

1. Ranks re-associationprocess

2. LUNs re-associationprocess

3. Sourceextent pooldeletion

Source Extent PoolTarget Extent Pool


Dynamic volume relocationDynamic volume relocation allows a user to initiate a volume migration from its current extent pool (source extent pool) to another extent pool (target extent pool). During the volume relocation process, the volume remains accessible to hosts.

Figure 2-3 shows a typical scenario where you want to change the underlying storage characteristics for a specific volume:

1. A host server accesses volume A (the red one in the picture).

2. You decide to move volume A from its current storage allocation in FC ranks to SATA ranks.

3. You initiate a migration using dynamic volume relocation of volume A from extent pool EPFC00 to target extent pool EPSATA02.

4. The relocation process begins pre-allocating the extents needed to relocate volume A in the target extent pool. The extent distribution in the target extent pool is determined by the extent allocation method (EAM) specified, either rotate extents (storage pool striping) or rotate volumes. If the EAM is not changed with the migration, the volume’s current extent allocation method is maintained. In this example, the EAM is rotate extents.

5. The actual volume A relocation starts, moving extents from EPFC00 to EPSATA02.

6. As soon as the relocation of any single extent is completed, the migrated extent is freed.

7. The process completes when all volume A extents have been migrated to EPSATA02.

Figure 2-3 Dynamic volume relocation between two pools

Limitations: The dynamic volume relocation is allowed only among extent pools with the same server affinity or rank group. Additionally, the dynamic volume relocation is not allowed:

� If source and target pools have different storage types (FB and CKD)

� If the volume to be migrated is a TSE volume.

EPFC00

EPFC00

EPSATA02

EPSATA02

Other volumes

Server

Dynamic Volume RelocationProcess

rank0 rank1 rank2

rank0 rank1 rank2

rank7 rank8

rank7 rank8

Volume A

Volume A

Extent pools before volume A relocation

Extent pools after

volume A relocation

Other volumes

source target


Manual volume rebalanceManual volume rebalance is designed to redistribute the extents of volumes within a non-managed, single-tier (homogeneous) pool so that workload skew and hot spots are less likely to occur on the ranks. Manual volume rebalance is an enhancement of the dynamic volume relocation feature introduced with DS8000 R6.1 LMC. A volume is migrated back into the same extent pool with the source extent pool as the target extent pool and the extent allocation method (EAM) as rotate extents (which means using storage pool striping). In this case, the algorithm tries to evenly spread the volume’s extents across all ranks in the extent pool. This feature is especially useful for redistributing extents after adding new ranks to an non-managed extent pool or after merging homogeneous extent pools to balance the capacity and the workload of the volumes across all available ranks in a pool.

Manual volume rebalance provides manual performance optimization by rebalancing the capacity of a volume within a non-managed homogeneous extent pool. It can also be referred to as capacity rebalance, because it only balances the extents of the volume across the ranks within an extent pool without actually taking any workload statistics or device utilizations into account. A balanced distribution of the extents of a volume across all available ranks in a pool is supposed to provide a balanced workload distribution and minimize skew and hotspots. It is also referred to as volume restriping, and is supported for standard and ESE thin provisioned volumes.

During extent relocation using manual volume rebalance, only one extent at a time is allocated rather than pre-allocating the full volume. Only a minimum amount of free capacity is required in the extent pool and only the required number of extents is relocated.

Manual volume rebalance is revoked in any managed or hybrid extent pools, no matter if the hybrid pool is currently managed or non-managed, as hybrid pools are always assumed to be created for Easy Tier Automatic Mode management.

Use manual volume rebalance when a rank is added to a pool, homogeneous extent pools have ben merged, or when large volumes with an extent allocation method (EAM) of rotate volumes are deleted and a rebalance can help optimize a non-balanced extent distribution within a homogeneous pool.

If a manual volume rebalance operation is executed in an existing pool with only partial capacity available on some ranks and no capacity available on other ranks, it might not always achieve a balanced distribution of the volume capacity across all ranks. For example, one rank is allocated by another volume using an extent allocation method of rotate volumes. In this case, you might need to perform the relocation in a certain sequence or in multiple steps to make sure to have at least some free extents available on each rank in the pool.

See “Manual volume rebalance” on page 103 for an example using DS CLI.

Consideration: Manual volume rebalance is not supported in managed extent pools, as these pools are already under control of the Easy Tier Automatic Mode algorithm. It is also not supported in general in hybrid or multitier pools (either managed or non-managed), which are assumed to be prepared for Easy Tier Automatic Mode management. It is also not supported for TSE volumes. It is supported for standard and thin provisioned extent space efficient (ESE) volumes. See “Dynamic volume relocation” on page 15 for more details.


Figure 2-4 shows a typical scenario where you perform a manual volume rebalance to better distribute the allocated capacity for a specific volume across the physical resources within a non-managed, single-tier, or homogeneous extent pool:

1. A server accesses volume B. Volume B is located in a non-managed, single-tier extent pool.

2. You decide to spread volume B from its current storage allocation on one FC rank (rank 0) to all ranks in the extent pool (rank 0, rank 1, and rank 2).

3. You initiate a volume migration using dynamic volume relocation for volume B, specifying the target extent pool to be the same as the source extent pool where volume B currently is, that is, EPFC00. To ensure that the extents of the volume are spread evenly across all available ranks in the extent pool, the rotate extents Extent Allocation Method (EAM) attribute must be specified for storage pool striping. For further details, see 4.3.1, “Dynamic volume relocation” on page 78 or 4.5.1, “Dynamic volume relocation” on page 100.

4. The relocation process begins pre-allocating the extents to relocate volume B on the target ranks. The pre-allocation process mitigates the extent availability requirements allocating only one extent at a time rather than pre-allocating the full volume. Only the minimum number of extents required to achieve a balanced extent distribution of the volume are relocated.

5. The actual relocation of volume B starts moving one extent at a time from rank 0 to the other ranks.

6. As soon as the relocation of an extent is completed, the migrated extent is freed.

7. The process terminates when all required extents of volume B extents are migrated.

Figure 2-4 Manual volume rebalance in a non-managed pool

EPFC00

EPFC00

Other volumes

Server

rank 0 rank 1 rank 2

rank 0 rank 1 rank 2

Volume B

Volume B

Extent pool after the volume B relocation

Extent pools before the volume B relocation

Dynamic Volume RelocationProcess


Rank depopulationRank depopulation is an Easy Tier Manual Mode capability that allows a user to unassign a rank from an extent pool, even if the rank has extents that are allocated by volumes in the pool. In order for the rank to be unassigned, Easy Tier automatically attempts to migrate all of the allocated extents to other ranks within the same extent pool. During this process, the affected volumes remain accessible to hosts.

Attention: Depopulation fails to start if there is not enough free space available in the extent pool to relocate all the extents onto the remaining ranks of the pool (CMUN80204E error code).

Thin provisioning: Starting with DS8000 R6.2 LMC, the depopulation of a rank containing ESE volumes or TSE volumes is supported.


Figure 2-5 describes operations that occur when a user wants to unassign a rank in an extent pool that has extents already allocated to some volumes:

1. You decide to unassign rank R1 from extent pool EPFC00, in order, for example, to be able to assign it to another extent pool.

2. You initiate rank unassignment from the extent pool, using either the DS CLI or GUI.

3. Easy Tier automatically reserves the rank to prevent new extent allocations. The state of the rank then changes to Depopulating.

4. The depopulation process pre-allocates extents, in the other ranks of the pool, which are needed to relocate the extents allocated in the R1 rank.

5. The actual extent relocation starts moving, in an asynchronous way, a few extents at a time, from R1 to other ranks in the pool.

6. As soon as the relocation of any single extent is completed, the migrated extent is freed.

7. The process terminates when all extents, previously on R1, have been relocated to the other ranks in the pool. Easy Tier unassigns R1 from the pool, and the state of the R1 rank then changes to “Unassigned”.

Figure 2-5 Rank depopulation

Extent Pool EPFC00

EPFC00 PoolLUNs Server with

running I/O

rank0 rank1 rank2

rank0 rank2 rank3 rank4rank1

Extent Pool EPFC00

1. R1 reserved and extents pre-allocation process

2. Extents relocation from R1 to other ranks

rank3 rank4

Extent pool before the R1 depopulation

Extent Pool EPFC00

rank0 rank1 rank2 rank3 rank4

3. R1 unassigned from EPFC00


Figure 2-6 describes the states and transitions of the different ranks, either due to user commands or state changes.

Figure 2-6 Rank states diagram

Tools to identify hot volumesEasy Tier Manual Mode can actively assist with performance tuning by manually moving volumes to the appropriate extent pool, particularly, for example, in mixed-technology configurations with multiple homogeneous extent pools of different drive tiers. Dynamic volume relocation can be used to manually manage the volume placement on to the most appropriate storage media and extent pool. To use this mode, you first need to identify the hot volumes.

Tools available to identify hot volumes are as follows:

� Storage Tier Advisor Tool report (STAT):

This report can help identify the hot extents and volumes. See 5.6, “Interpreting the Storage Tier Advisor Tool output” on page 127 for more information.

� Disk Magic:

Disk Magic provides a capability to advise the data placement on SSD disk drives. Having DS8000 configurations with SSD drives and multiple servers, Disk Magic can select a specific server workload that benefits most to run on SSD drives. This feature is called SSD Advisor and applies both to IBM System z® and open systems models.

Rank Depopulation / Rank States

Configuring

Unassigned Normal

Depopulating

DepopulationError

ConfigurationError

ReservedUnassigned-Reserved

Deconfiguring DeconfigurationError

Create

Delete Delete

(Deleted)

(Any State)

Assign

Unassign

Unassign1 Release

Reserve

Assign

Reserve Unassign = Depop/Unassign or Unassign rank with no allocated extents

Unassign1 = Depop/Unassign rank with allocated extents

User Initiated Transition


� FLASHDA tool for z/OS:

This tool uses SAS to analyze SMF record type 42 subtype 6 and SMF record type 74 subtype 5 to help identify volumes and data sets that are good candidates to be on SSD. For more information, see this website:

http://www.ibm.com/systems/z/os/zos/downloads/flashda.html

� IBM i has comprehensive workload analysis tools to determine the benefits of SSD. For more information, see this website:

http://www.ibmsystemsmag.com/ibmi/september09/storage/26212p1.aspx

� The IBM Tivoli Storage Productivity Center for Disk also allows you to identify hot volumes as it has IO statistics at the volume level. More information about Tivoli Storage Productivity Center can be found at the following website:

http://www.ibm.com/systems/storage/software/center/

Based on the output report of these tools, you can select which hot volumes might benefit most when migrated to SSD. When using the FLASHDA tool for z/OS, the tool output also provides the hot data at the data set level. Based on this data, the migration to the SSD ranks can be done by data set using the appropriate z/OS tools.

2.5 Easy Tier design

As previously explained, the Easy Tier feature enables dynamic data relocation, that is, the capability to move volumes or extents across storage system resources transparently to the host. This capability is the core capability introduced by Easy Tier. In Automatic Mode, Easy Tier allows automatic data relocation within managed extent pools to optimize system performance on extent level. Easy Tier Manual Mode provides dynamic volume relocation to relocate data on volume level and manually manage volume placement in the appropriate extent pool or storage tier.

The DS8000 can monitor the back-end I/O usage intensity of all allocated extents. This Easy Tier monitoring capability can be enabled or disabled for all pools or only for managed pools through the DS GUI or DS CLI. The monitoring data is accumulated in a summary file, which is kept on the DS8000 internal storage servers, and can be offloaded for further processing with the Storage Tier Advisor Tool (STAT).

Easy Tier I/O workload monitoring only collects I/O statistics from the physical back-end, which means the back-end I/O activity on rank level. The system monitors the stage / destage activity of each extent allocated to a logical volume in the extent pool and calculates a temperature metric for each extent (also referred to as a heat map). A read cache hit that is satisfied by data already in cache is not monitored or recorded by the system for the Easy Tier metrics and therefore not reflected in the Easy Tier heat map.


http://www.ibm.com/systems/z/os/zos/downloads/flashda.html

http://www.ibmsystemsmag.com/ibmi/september09/storage/26212p1.aspx

http://www.ibm.com/systems/storage/software/center/

2.5.1 SSD, Enterprise, and Nearline storage tiers

Tiered storage is an approach of using different types of storage throughout the storage infrastructure or even on a selected storage system such as DS8000. It is a mix of higher performing / higher-cost storage with lower performing / lower-cost storage, as shown in Figure 2-7, placing data accordingly based on performance needs. Properly balancing these tiers leads to the minimal cost – best performance solution. Easy Tier Automatic Mode is designed for by providing monitoring and dynamic data relocation capabilities to automatically optimize I/O performance across up to three different tiers of storage (SSD, enterprise disk, and Nearline disk) by placing subvolume data onto the correct tier.

Figure 2-7 Tiered storage concept

The performance characteristics of SSD, Enterprise disk, and Nearline disk storage tiers differ considerably. SSD performance is superior compared to HDD performance, as HDDs are limited by mechanical latencies (rotational latency, seek time, and transfer time) with access times around 4 - 8 ms, while SSD provide access times far below 1 ms. Furthermore, Enterprise disks perform better than Nearline disks, especially for small-block random I/O operations. To provide an idea of the performance benefit of SSDs compared to HDDs, Table 2-2 gives a rough estimate of a performance comparison between an individual SSD drive and an Enterprise HDD drive regarding typical manufacturer’s drive specifications.

Table 2-2 SSD performance compared to HDD

I/O operation Typical SSD relative to HDD

Random read 100 times faster

Random write 40 times faster

Sequential read or write Two times faster


SSDs give the most performance improvement over HDDs when handling small block random read I/O requests. Similarly, small block random write I/O requests also can be handled much better by SSDs compared to HDDs. SSD performance is best for small block random read I/O requests and typically decreases with increasing I/O request block sizes (32 KB or larger) or increasing write ratios. Even though sequential large-block I/O still performs better on SSD, Easy Tier does not consider moving large block I/O requests to SSD. In terms of price-performance consideration, it is more beneficial to move small block random I/O requests to SSDs rather than large block sequential I/O requests. Comparing Enterprise disks and Nearline disks, conclusions are almost equivalent, even if the performance gain between those two tiers is much smaller. Random I/Os perform about twice as fast on Enterprise disks, and sequential I/Os perform about the same on Enterprise disk as on Nearline disks.

See 3.2, “Physical configuration planning and SSD considerations” on page 38 for more information about SSDs on DS8000 systems.

2.5.2 Easy Tier performance metrics

To determine the best storage tier for a given extent in an extent pool managed by Easy Tier Automatic Mode, the following performance metrics are collected by the Easy Tier monitoring process.

For every extent, I/O performance data is recorded by the Easy Tier storage system performance monitor as follows:

� Whether the I/O is a read or a write.

� Whether the I/O is a small block (random) I/O or large block (sequential) I/O.

� The cumulative latency of the I/O, which is the time spent to complete the I/O operations in the rank.

In a Metro Mirror environment, there is an additional time delay due to the required data transfer of a write I/O to the secondary site. This additional latency or service time is not included in the foregoing performance data, because this I/O activity is not an activity that is occurring at the back-end on rank level.

� The I/O rate.

� The amount of data transferred.

Easy Tier determines whether an extent is hot, warm, or cold based on these performance metrics regarding its automated cross-tier performance management capability in Automatic Mode. Easy Tier calculates the temperature or heat for each extent, which is a metric for the extent’s hotness regarding its back-end I/O activity. The heat of an extent is key when determining the next extent migration plan and relocating extents to the appropriate storage tier in managed extent pools.

2.5.3 Easy Tier cross-tier and intra-tier performance management

Starting with DS8000 R6.2 LMC, Easy Tier Automatic Mode provides automatic inter-tier or cross-tier performance management and automatic intra-tier performance management in multitier (hybrid) or single-tier (homogeneous) extent pools.


Easy Tier cross-tier or inter-tier performance managementIn configurations with managed multitier extent pools containing ranks from more than one storage tier, Easy Tier Automatic Mode creates a migration plan to migrate the hot extents from a lower performance tier to a higher performance tier. It gives higher priority to the extents with the highest temperatures. The system also monitors the workload of each rank in an extent pool to determine if a rank's workload is causing excessive I/O latency and to avoid rank overload or resource saturation situations. All the performance information is used to generate an extent relocation or migration plan at least once every 24 hours. The goal of the migration plan is to place each extent in the pool onto the most appropriate storage tier available. The migration plan considers the workload statistics that have been accumulated for the extent over time. This cross-tier or inter-tier extent migration is triggered by the extent hotness in multitier extent pools.

Easy Tier intra-tier performance managementEasy Tier in Automatic Mode performs intra-tier extent migrations by monitoring the overall I/O workload for each rank and creating a migration plan to evenly rebalance the I/O workload across all the ranks within a given tier. This feature constantly rebalances the I/O workload across all ranks within a storage tier in managed extent pools, and is referred to as auto-rebalance. This feature is not restricted to multitier or hybrid pools. With DS8000 R6.2 LMC, this feature is also available for single-tier or homogeneous extent pools when managed by Easy Tier in Automatic Mode.

The migration is staged such that the impact on host I/Os is minimal (comparable to IBM FlashCopy® background activity).

Easy Tier migration planAfter an extent migration plan is generated, the storage system migrates a limited number of extents every 5 minutes until all extents in the plan are migrated, or until the next plan is generated. This process continues repeatedly on an ongoing basis. Given a consistent workload, all extents are eventually migrated to their most optimum storage type. For a dynamic workload, where the temperature metrics of the extents fluctuate over time, each new extent migration plan can dynamically adapt to workload variations and can relocate extents accordingly.

The extent migration plan algorithms factor in the cost of moving an extent into the decision of whether a given extent should be relocated. If the benefit of moving the extent does not exceed the cost of the relocation, the extent is not included in the migration plan.

When there are no more free extents available on a higher tier but there are hot extents still on a lower tier, Easy Tier evaluates whether the hottest extent on the lower tier benefits more when placed on the higher tier compared to the extent on the higher tier with the coldest temperature. If that is the case, then the locations of those two extents are swapped.

Three-tier migration: In a three-tier extent pool configuration, the cross-tier extent migration only occurs between adjacent tiers. There is no direct migrations from the lowest to the highest tier that bypass the middle tier.

Single-tier pools: With DS8000 R6.2 LMC, Easy Tier Automatic Mode now also can rebalance I/O workload across ranks in single-tier or homogeneous extent pools with intra-tier extent migrations. This feature is referred to as auto-rebalance and is available in managed multitier (hybrid) and single-tier (homogeneous) extent pools.


2.5.4 Migration plan creation

A migration plan is only applicable when the Easy Tier monitoring task can sample enough performance data for the volumes. Migration plans are created internally on the DS8000 storage system based on the Easy Tier monitoring data and are not in any way dependent on the Storage Tier Advisor Tool (STAT) software. The Storage Tier Advisor Tool is an additional piece of software that can process the internally collected DS8000 Easy Tier performance data to provide appropriate Easy Tier system status reports. These STAT reports contain system summary information, rank utilizations, configuration recommendations, and volume heat maps for the user based on the most recent data collection for the latest migration plan, including information about the relative amount of hot data capacity.

Easy Tier Automatic Mode creates both intra-tier migration plans (auto-rebalance plans) and cross-tier migration plans. The two kinds of migration plans have two different decision windows during which Easy Tier monitors and collects I/O statistics to identify the extents to be moved. The decision windows for the auto-rebalance and cross-tier migration plans are 6 hours and approximately 24 hours.

At least once every 24 hours a new cross-tier migration plan is created. However, the start time varies, so you are not always starting at the same time each day for a new data collection.

Similarly, a new auto-rebalance plan is created every 6 hours. The new plan is immediately carried out. The migration plan creation is independent of when you offload the monitor data from the DS8000.

When a new volume is created, the completion of a full decision window is required before any extent from that new volume is included in a migration plan. This situation also applies to FlashCopy target extents and copy on write for source or target updates.

Easy Tier creates one migration queue per extent pool, allowing each pool to be optimized independently. This situation leads to an even better migration plan management, as each extent temperature is computed relatively to the other extents within the same pool. During migration plan execution, extent relocation occurs in a round-robin fashion at the pool and tier level. This action ensures that progress is made on each pool and tier.

Migration: After the Easy Tier monitoring mode has been activated, it takes 24 hours until the first cross-tier or inter-tier migration occurs. Similarly, it takes 6 hours until intra-tier migrations occur and auto-rebalance starts operating.

Reporting: The Storage Tier Advisor Tool (STAT) shows a detailed report for each extent pool, reflecting the per-pool migration plan managed by Easy Tier Automatic Mode. STAT tool usage is described in Chapter 5, “Using IBM System Storage DS8000 Easy Tier” on page 119.


Figure 2-8 shows the basic time line in creating and executing an inter-tier migration plan, and indicates that monitoring data is constantly and continuously collected. This volume is not a growing data volume; it is just an update of the collected performance counters. The size of the data created by the Easy Tier Monitoring task is related to the number of volumes being monitored.

Figure 2-8 Create inter-tier migration plan cycle

The first inter-tier migration plan for Easy Tier cross-tier performance management is created after collecting monitoring data for approximately 24 hours. The first intra-tier migration plan for the Easy Tier auto-rebalance feature is created after collecting monitor data for approximately 6 hours. After a 24-hour period with monitoring enabled, a good picture is available of what the I/O workload on the physical storage back end looks like, and the first cross-tier migration plan starts to execute. Remember that a cross-tier migration plan starts moving extents across ranks from different tiers within managed extent pools when space is available on higher performance tier and extents on the lower performance tier are considered to be hot and potentially benefit from being relocated onto a higher tier.

After another 24-hour decision window, a new migration plan is created and starts to execute. This new plan is based on performance data collected for the past two days. The relevance of older monitor data is exponentially decreasing over time. Historical data is considered when available, but with diminishing relevance (the system uses a 7 day long-term decision window to factor in older I/O statistics). The most relevant data is the data collected in the last decision window relative to the time when the new migration plan is created.

Migration plan cycle: t3 is not exactly t1 + 24h, as a new migration plan is created at least every 24 hours, but not necessarily exactly at 24-hour intervals.

Decision window: Easy Tier Automatic Mode algorithm uses a short-term and a long-term sliding decision window to determine the intra-tier extent migration plan. Statistics in the short-term window have a higher weight relatively to those statistics in the long-term window. Statistics older than the long-term limit are automatically removed from the DS8000. The short-term window is set to 24 hours, and the long-term window is set to 7 days. Those parameters are not user-settable, either by DS CLI or GUI commands.

Collecting monitor dataapproximately 24h

Create first migration plan

ExecuteMigrationPlan#2

Migrationfinished

t0 t2t1

Continue to collect monitor data

Start CollectingMonitor Data

Create next migration plan

ExecuteMigrationPlan#1

t3 = t1 + 24ht [h]


When the time to execute a migration plan is longer than the decision window, the current migration plan execution is stopped and a newly created migration plan started.

Because the cross-tier and intra-tier or auto-rebalance plans are independent from one another, it is possible that the same extent might be eligible for migration in both plans. In this case, the cross-tier migration plan always has the higher priority and prevails.

2.5.5 Easy Tier migration types

When Easy Tier determines the right storage media for a given extent based on the extent heat and resource utilization, it uses the following extent migration types between different storage tiers or within a storage tier, as shown in Figure 2-9:

� Promote & swap for moving hot data to higher performing tiers.

� Warm demote which prevents performance overload of a tier by demoting warm extents to the lower tier and being triggered when bandwidth or IOPS thresholds are exceeded.

� Cold demote on HDD tiers, where coldest data is identified and moved it to the Nearline tier.

� Expanded cold demote for HDD tiers to demote some of the sequential workload to better use the bandwidth in the Nearline tier.

� Auto-rebalance, which redistributes the extents within a tier to balance utilization across all ranks of the same tier for maximum performance.

Figure 2-9 Easy Tier migration types


2.5.6 Promoting and swapping

Both promote and swap migration types move hot data to higher performing tiers.

2.5.7 Auto-rebalance

Auto-rebalance or Automatic Performance Rebalance, also known as intra-tier rebalancing, is a capability of Easy Tier that automatically rebalances the workload across all ranks of a given storage tier within a managed extent pool. Auto-rebalance migrates extents across ranks within a storage tier to achieve a balanced workload distribution across the ranks and avoid hotspots. By doing so, auto-rebalance reduces performance skew within a storage tier and provides the best available I/O performance from each tier.

Furthermore, auto-rebalance also automatically populates new ranks that have been added to the pool when rebalancing the workload within a tier. Starting with DS8000 R6.2 LMC, auto-rebalance can be enabled for hybrid and homogeneous extent pools.

Figure 2-10 shows two examples of scenarios where auto-rebalance can be involved.

Figure 2-10 Auto-rebalance scenarios

Important: Migrations can only happen between adjacent tiers. For example, there is no promotion or demotion from the Nearline tier to the SSD tier in a three-tier extent pool.

Add Rank Natural Performance Skew

Rank hothot coldcold

hothot coldcoldRank

hothot

coldcold

Rank

Rank c oldc old

coldcold

Rank hothot coldcold

hothot coldcoldRank coldcold

Rank

Rank

hothot

Extents are automatically rebalancedacross ranks of the same tier within the

extent pool based on I/Os andbandwidth requirements.

hothot coldcold hothot c oldc old

c oldc old

coldcold


The auto-rebalance algorithm maximizes storage performance on both IOPS and bandwidth level within the same storage tier, using a threshold at the rank level to determine whether an extent could benefit from a move to another “less used” rank within the same tier in the extent pool. Auto-rebalance is triggered every 6 hours, if needed, and can occur, within a managed extent pool, on every tier that has at least two ranks. Device-specific performance variations within a storage tier (micro-tiering) are taken into account for determining the rank utilization based on device-specific performance profiles, considering drive speed, RAID level, and platform (DS8800 / DS8700), with workload characteristics. No rebalancing is done if the skew is insignificant or if migration would cause toggling, which means that the migration would trigger further rebalancing efforts on the target array.

2.5.8 Warm and cold demotion

In addition to the promote and swap extent migration types that move hot extents to the higher performing tier, the demotion of extents also is an essential part of the Easy Tier design.

Warm and cold demotion are two features of the Easy Tier Automatic Mode introduced with Release 6.1 of the DS8000 microcode. Expanded cold demotion is a new Easy Tier feature introduced with Release 6.2 of DS8000 microcode.

Warm demotionHigher-performance tiers can become overloaded in hybrid extent pools and potentially degrade overall pool performance. To avoid this situation, Easy Tier Automatic Mode monitors the performance of the ranks and triggers the movement of selected extents from the higher-performance tier to the lower-performance tier. This action is based on predefined bandwidth or IOPS overload thresholds.

This automatic operation is rank-based and the target rank is randomly selected from the lower tier. Warm demotion has the highest priority to quickly relieve overloaded ranks. Easy Tier continuously ensures that the higher-performance tier does not suffer from saturation or overload conditions that might affect the overall performance in the extent pool.

Cold demotionEasy Tier Automatic Mode automatically locates and demotes inactive (or cold) extents that are on a higher performance tier to its adjacent lower-cost tier. In that way, Easy Tier Automatic Mode automatically frees extents on the higher storage tier before the extents on the lower tier become hot, and then helps the system to be more responsive to new hot data. Cold demotion recognizes and demotes cold extents to an appropriate lower-cost tier. Cold extents are demoted in a storage pool to a lower tier as long as that storage pool is not idle to always make progress on optimizing storage economics.

Important: Auto-rebalance is based on IOPS rank utilization to reduce skew and hot spots within a tier by redistributing extents across ranks of the same tier. It is not capacity-based, in the sense that it does not try to balance the extents, and thus the capacity, of the volumes evenly across the ranks within a tier as manual volume rebalance and storage pool striping does. Extents are only relocated onto another rank within the same tier for performance reasons.

Important: Auto-rebalance is not active in non-managed extent pools. Extents within non-managed single-tier extent pools can be rebalanced based on a balanced capacity distribution using the manual volume rebalance feature, as described in “Dynamic volume relocation” on page 15.


The STAT tool HTML report also provides guidelines for Enterprise disks single-tier extent pools on adding Nearline disk ranks if cold demotion is expecting to help in that way.

If all the extents in a hybrid storage pool become simultaneously inactive for a period between 15.5 and 24 hours, due to either a planned or unplanned outage, Easy Tier automatically disables cold demotion for this pool. This action helps you schedule extended outages, or experience an outage, without performing cold extent relocation.

Expanded cold demotIonExpanded cold demotion is a new Easy Tier feature introduced with DS8000 R6.2 LMC that demotes some of the sequential workload to the Nearline tier to better use the bandwidth in the Nearline tier. It selects appropriate extents with moderate bandwidth requirements from the Enterprise tier for demotion to the Nearline tier to use its bandwidth capabilities and to spread the bandwidth demand across both tiers instead of only using the Enterprise tier.

2.5.9 Extent allocation in hybrid and managed extent pools

As described in 2.6.1, “Easy Tier Automatic Mode control” on page 32, the Easy Tier Automatic Mode scope can be modified to manage either all extent pools, including single-tier, homogeneous extent pools, or only tiered, multitier extent pools.

When you create a volume in a managed extent pool, that is, an extent pool that is managed by Easy Tier Automatic Mode, the extent allocation method (EAM) of the volume always becomes managed. This situation is true no matter which extent allocation method is specified at volume creation. The volume is under control of Easy Tier, which moves extents to the most appropriate storage tier and rank in the pool based on performance aspects. Any specified extent allocation method such as rotate extents or rotate volumes is ignored. In managed extent pools, an initial EAM similar to rotate extents for new volumes is used. The same situation applies if an existing volume is manually moved to a managed extent pool using volume migration or dynamic volume relocation.

In hybrid or multitier extent pools (whether currently managed or unmanaged by Easy Tier), initial volume creation always starts on the ranks of the Enterprise tier first, as shown in Example 2-1. In this example, all extents of the newly created volume are evenly distributed across the Enterprise tier ranks R17 and R18 only.

Example 2-1 Volume creation and initial extent allocation in hybrid pools

dscli> lsextpool -l p4Name ID stgtype rankgrp numvols numranks numtiers etmanaged===================================================================gero_3tier P4 fb 0 0 4 3 yes

dscli> lsrank -l -extpool p4ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=====================================================================================R1 0 Normal Normal A1 5 P4 gero_3tier fb 1574 0R17 0 Normal Normal A2 5 P4 gero_3tier fb 3173 0

Cold demotion: Cold demotion only occurs between HDD tiers, that is, between Enterprise and Nearline disks.

Three-tier demotion: In a three-tier extent pool configuration, the extent demotion occurs only between two adjacent tiers.


R18 0 Normal Normal A3 5 P4 gero_3tier fb 3173 0R22 0 Normal Normal A16 6 P4 gero_3tier fb 12922 0

dscli> lsarray -l a1 a2 a3 a16Array State Data RAIDtype arsite Rank DA Pair DDMcap (10^9B) diskclass encrypt==========================================================================================A1 Assigned Normal 5 (6+P+S) S2 R1 0 300.0 SSD unsupportedA2 Assigned Normal 5 (6+P+S) S3 R17 2 600.0 ENT unsupportedA3 Assigned Normal 5 (6+P+S) S4 R18 2 600.0 ENT unsupportedA16 Assigned Normal 6 (5+P+Q+S) S17 R22 2 3000.0 NL unsupported

dscli> mkfbvol -cap 10 -extpool p4 -name eamtest 8010CMUC00025I mkfbvol: FB volume 8010 successfully created.

dscli> showfbvol -rank 8010Name eamtestID 8010...eam managed...==============Rank extents==============rank extents============R17 5R18 5

The Enterprise tier is also referred to as the home tier. The extents of a new volume are distributed in a rotate extents or storage pool striping fashion across all available ranks in this home tier in the extent pool as long as sufficient capacity is available. Only when all capacity on the home tier in an extent pool is consumed does volume creation continue on the ranks of the Nearline tier. When all capacity on the Enterprise tier and Nearline tier is exhausted, then volume creation continues allocating extents on the SSD tier. The initial extent allocation in non-managed hybrid pools differs from the extent allocation in single-tier extent pools with rotate extents (the extents of a volume are not evenly distributed across all ranks in the pool because of the different treatment of the different storage tiers). However, the attribute for the EAM of the volume is shown as rotate extents if the pool is not under Easy Tier Automatic Mode control. After the pool has been managed by Easy Tier Automatic Mode, the EAM becomes managed.

In managed homogeneous extent pools with only a single storage tier, the initial extent allocation for a new volume is the same as with rotate extents or storage pool striping.

For a given volume, the appropriate DS CLI commands, showfbvol or showckdvol, used with the -rank option, allow the user to list the number of allocated extents of a volume on each associated rank in the extent pool, as shown in Example 2-1 on page 30. The commands also show the EAM attribute of the requested volume.

Starting with DS8000 R6.1 LMC, the EAM attribute of any volume created or already in a managed extent pool is changed to managed after Easy Tier Automatic Mode has been enabled for the pool. When enabling the Easy Tier Automatic Mode for all extent pools, that is, hybrid and homogeneous extent pools, all volumes immediately become managed by Easy Tier and the EAM attribute of all volumes on a DS8000 system is changed to managed. Once set to managed, the EAM attribute setting for the volume is permanent and all previous volume EAM attribute information, such as rotate extents or rotate volumes, is lost.


2.6 Easy Tier implementation

To use Easy Tier Automatic Mode and Manual Mode features, the DS8000 system must be running an LMC that supports Easy Tier. Additionally, the Easy Tier LIC feature must be enabled. No further actions are required to use the Easy Tier Manual Mode capabilities.

Additional settings are needed to properly activate the Easy Tier Automatic Mode. For this purpose, the storage system provides the users with two user controls: the Easy Tier Automatic Mode control and the Easy Tier Monitoring Mode control.

2.6.1 Easy Tier Automatic Mode control

This control can be set to tiered, all, or none, and defaults to tiered. When set to tiered, automatic extent migrations performed by Easy Tier Automatic Mode are enabled only for multitier or hybrid pools. When set to all, automatic extent migrations performed by Easy Tier Automatic Mode are enabled for both hybrid (multitier) and homogeneous (single-tier) pools. When set to none, Easy Tier Automatic Mode is disabled and all automatic extent migrations are deferred for all the extent pools.

The none option can be used to temporarily suppress extent migrations initiated by Easy Tier Automatic Mode. This action can be used to either protect a workload from being affected by Easy Tier Automatic Mode migrations or isolate the effects of Easy Tier Automatic Mode migrations from other system-level problem determination efforts.

2.6.2 Easy Tier Monitor Mode control

The monitoring capability of a DS8000 system enables it to monitor the workload on the back-end storage at the extent level. Monitoring statistics are gathered and analyzed every 24 hours. In an Easy Tier managed extent pool, the analysis is used to create an extent relocation or migration plan for the extent pool to relocate extents to the most appropriate storage tier and storage resource within the pool.

The Easy Tier Monitor control can be set to automode, all, or none, referring to the volumes to be monitored. The defaults is automode, which means that only volumes under control of Easy Tier Automatic Mode in managed extent pools are monitored. The settings are described as follows:

� In automode, extent statistics for logical volumes in managed extent pools under control of Easy Tier Automatic Mode are updated to reflect current workload activity.

� When set to all, extent statistics for all logical volumes in managed and non-managed extent pools are updated to reflect current workload activity (regardless of whether the Easy Tier LIC feature is installed or not).

Important: Easy Tier Automatic Mode is enabled by default for hybrid extent pools with a default control switch setting of tiered after the Easy Tier LIC feature has been applied.

Attention: Setting the Easy Tier Auto Mode to none also causes all the current migration plans to be reset (reinitialized). When setting Easy Tier Auto Mode to tiered or all again, you need to wait until the end of the decision window before a new migration plan can be computed and applied for each managed extent pool.


� When set to none, extent statistics collection is suspended. The Easy Tier learning data in memory is reset and all migration plans are cleared. However, the last Easy Tier summary report for the Storage Tier Advisor Tool (STAT) remains available for download and is not automatically deleted.

Starting with LMC R5.1 on DS8700 and LMC R6.1 on DS8800 systems, the Easy Tier monitoring mode is also available if the Easy Tier LIC feature has not been installed. See Table 2-3 for available monitoring modes (dependent on the Easy Tier LIC feature installation).

Table 2-3 Easy Tier monitoring settings with Easy Tier LIC feature installed / not installed

The Easy Tier monitor control can be used for the following purposes:

� The automode setting normally must be used for Easy Tier Automatic Mode. With this setting, extent performance statistics are only updated for logical volumes that are in managed extent pools. This situation means that no extent statistics are collected for volumes that are in non-managed extent pools.

� When the Easy Tier LIC feature is not installed, the all volumes setting can be used:

– To collect statistics on all volumes on the DS8000 system.

– To analyze, at the extent pool level, the back-end I/O workload running on the DS8000 system, using the Storage Tier Advisor Tool (STAT).

– To assess workloads on specific logical volumes relative to whether they are targeted to the most applicable storage tier.

– To assess the applicability of Easy Tier Automatic Mode and receive capacity-planning guidelines for use with Easy Tier Automatic Mode.

� When the Easy Tier LIC feature is installed and no or not all logical volumes are in managed extent pools, the monitor all volumes setting can be used:

– To collect statistics on all the volumes on the DS8000 system.

– To analyze, on the extent pool level, the workload running on the DS8000 back-end using the Storage Tier Advisor Tool.

– To assess workloads on specific logical volumes relative to whether they are best suited for managed extent pools or for selecting the most applicable non-managed extent pool. Easy Tier Manual Mode can be used with this analysis to manually relocate logical volumes as needed. With no hybrid extent pools, Easy Tier monitoring combined with Easy Tier Manual Mode easily allows the manual management of different storage tiers across different homogeneous extent pools.

– To understand the current tier allocation of any given logical volume in an extent pool managed by Easy Tier Automatic Mode.

– To receive capacity-planning guidelines for different storage tiers with Easy Tier Automatic Mode using the Storage Tier Advisor Tool.

Easy Tier Monitor setting Easy Tier license featurenot installed

Easy Tier license feature installed

all All volumes are monitored. All volumes are monitored.

automode No volumes are monitored. Volumes in extent pools managed by Easy Tier are monitored.

none No volumes are monitored. No volumes are monitored.


� The none setting of the Easy Tier Monitor can be used to suspend Easy Tier monitoring. No extent performance statistics are collected, and no new extent migration plans are generated. The learning data in memory that is used by Easy Tier to generate the migration plans is cleared at the next 5-minute cycle after the Easy Tier Monitor is turned off. Also all migration plans are cleared.

Figure 2-11 shows a summary of when the monitoring and migration are performed in a configuration with a managed pool under control of Easy Tier Automatic Mode. With the Easy Tier feature installed and enabled, the default option is automode, so that monitoring and migration are only performed on managed volumes in managed extent pools under control of Easy Tier Automatic Mode.

Figure 2-11 Easy Tier monitoring and migration options

2.6.3 Easy Tier considerations

Easy Tier is easy to deploy. With the Easy Tier LIC feature enabled and the default settings set, no other action is required to activate the Easy Tier Automatic Mode in all hybrid, multitier extent pools to benefit from automated cross-tier and intra-tier storage performance and storage economics management in these pools. To use the auto-rebalance (intra-tier management) feature in homogeneous, single-tier extent pools, the Easy Tier control setting needs to be changed to all. With this setting, Easy Tier takes control of all extent pools and all regular and thin provisioned volumes (ESE), and automatically optimizes storage performance and storage economics in each pool while also minimizing skew and avoiding hot spots within each tier (auto-rebalance), including single-tier extent pools. Only a sufficient monitoring time must pass for the decision window before Easy Tier actually begins to create migration plans and start cross-tier and intra-tier extent migrations.

None

Automode Volumes

Automode Volumes

NoneEasy Tier

AutoMode

Easy TierMonitor

Mode

Easy TierMonitor

Mode

None

- Monitoring suspended

- Current migration plan stops

- No Next Plan

TIERED

- Monitoring suspended

- Statistics cleared

- No migration

- Monitor ALL volumes

- No migration

- Monitor ALL volumes

- Current migration plan goes on

- Next migration plan prepared

- Monitor managed volumes

- Current migration plan goes on

- Next migration plan prepared

- Monitor suspended

- Statistics cleared

- No migration

All Volumes

All Volumes

or ALL


Typically, extent pools are configured in pairs on DS8000 systems, one for each storage server (server 0 and server 1 or rank group 0 and rank group 1), to balance the DS8000 workload evenly between the two DS8000 systems and all DS8000 device adapters (DA). To fully use the DS8000 systems’ bandwidth in a balanced way, SSD ranks also should be installed in pairs and assigned to pairs of hybrid pools managed by both DS8000 systems.

One consideration with Easy Tier Automatic Mode is that the DS8000 optimizes the data placement based on the current and most recent workload characteristics. In cases where the usage or I/O workload of a logical volume changes considerably from one day to another to a new workload pattern, the achieved performance benefits resulting from the previous optimized data placement might be lost for the new workload. This situation occurs because the I/O access patterns and characteristics have changed. It takes some time until Easy Tier adapts to the new workload pattern and the data placement is optimized again for new workload. An example of such a situation is a disaster recovery setup with two sites using DS8000 copy services, such as Metro Mirror. The workload characteristics on the secondary site differ considerably from the workload characteristics on the primary site in this case. When a failover occurs, the secondary disk system must first learn the new characteristics of the primary workload and adapt to the new workload pattern by starting to adjust the data placements accordingly. It takes some time before the active secondary site (acting as a new primary site now) can perform as well as the original primary site.


Chapter 3. Planning for IBM System Storage DS8000 Easy Tier

When the first generation of Easy Tier was first made available with DS8000 R5.1 LMC, it was aimed at optimizing storage performance in hybrid, two-tiered extent pools consisting of HDDs and SSDs only using automated cross-tier data relocation.

The second generation of Easy Tier was made available with DS8000 R6.1 LMC. It introduced automated storage performance or storage economics management supporting extent pools with any two tiers built from SSD, Enterprise HDD or Nearline HDD storage tiers. It also introduced automated intra-tier performance management (auto-rebalance) in hybrid pools and support for rank depopulation and optimized volume restriping within non-managed single-tier extent pools (manual volume rebalance).

The third generation of Easy Tier was made available with the DS8000 R6.2 LMC. It can optimize storage performance and storage economics in hybrid (multitier) with three storage tiers. It also provides automated intra-tier performance management in both hybrid (multitier) and homogenous (single tier) pools (auto-rebalance) and thin provisioning support for extent space-efficient (ESE) volumes. This latest release now supports any combination of Nearline HDD (SATA), Enterprise HDD (FC or SAS), and SSD tiers within an extent pool.

By design Easy Tier Automatic Mode is easy and straightforward to implement. However, adequate planning is still required to maximize the Easy Tier benefits along with the expectations for your workload.

In this chapter, we provide planning information for Easy Tier. We summarize the SSD configuration rules for DS8000 systems and give some guidance for implementing Easy Tier Automatic Mode. We also briefly describe available planning tools, such as the Storage Tier Advisor Tool (STAT), an application that can help estimate the benefits of Easy Tier.

3


3.1 Prerequisites

Easy Tier was first introduced on the DS8700 system with the DS8000 Licensed Machine Code (LMC) level 6.5.1.xx (bundle version 75.1.xx.xx). Easy Tier is now supported on the DS8700 system with Licensed Machine Code (LMC) level 6.6.20.xx and the DS8800 system with LMC level 7.6.20.xx, referred to as DS8000 R6.2 LMC or DS8000 R6.2 microcode in both cases.

Easy Tier is an optional feature of the DS8000 system, and is available with the Easy Tier licensed feature number 7083. It is a no charge feature, but must be installed on the system to enable the Easy Tier data relocation functionality. The Easy Tier monitoring capability is available without the Easy Tier feature code installed on supported DS8000 LMC levels; see 2.6.2, “Easy Tier Monitor Mode control” on page 32 for more information. For detailed information about how to activate a licensed feature, see Chapter 10, “IBM System Storage DS8000 features and license keys”, in IBM System Storage DS8000: Architecture and Implementation, SG24-8886.

Easy Tier Automatic Mode is currently not supported on a DS8000 system that is encryption capable, that is, if the machine has the Encrypted Drive Set (FDE) Support feature number 1751 installed. Easy Tier Manual Mode is supported with FDE, and the extent pool merge capability is supported.

On DS8000 systems, you have two kinds of space-efficient volumes that can be either extent space-efficient (ESE) or track space-efficient (TSE) volumes. These volumes can be in any extent pool managed by Easy Tier Automatic Mode, but only ESE volumes are managed and fully supported by Easy Tier in Automatic and Manual Mode mode starting with DS8000 R6.2 LMC.

3.2 Physical configuration planning and SSD considerations

Solid-state drives (SSDs) are a higher performance storage option compared to hard disk drives (HDDs).

All SSD disks installed in a storage enclosure pair must be of the same capacity and speed. Feature conversions are available to exchange existing disk drive sets when purchasing new disk drive sets with higher capacity or speed:

� For DS8700 systems, SSD disks are available with 600-GB capacity. � For DS8800 systems, SSD disks are currently available with 300-GB capacity.

SSD drives can be ordered and installed in eight-drive installation groups (half drive sets) or 16 drive installation groups (full drive sets). A half drive set (eight) is always upgraded to a full drive set (16) when SSD capacity is added. A frame can contain, at most, one SSD half drive set.

In general, SSD drive sets improve system performance with cache-unfriendly small block random workloads. However, to achieve optimal price-to-performance ratios in DS8000 systems, SSD drives have some limitations and guidelines that differ from HDDs.

Tip: An eight-drive installation increment means that the SSD rank added is assigned to only one DS8000 system (central processor complex (CPC)).


3.2.1 Solid-state drive features

Hard disk drive technology has improved over the years, providing higher bandwidths and higher storage densities at a lower price. However, because of its moving and spinning mechanical parts, there are inherent physical limitations to the response times (rotational latency, seek time, and transfer time) that an HDD can achieve. SSDs, compared to Enterprise and Nearline drives, have low response times with low latencies. With proper planning, you can benefit from SSD features to achieve the best performance among different systems and workloads.

SSDs come in two flavors: Single-level cell (SLC), and Multi-level cell (MLC). This designation refers to whether we use two or four voltage states per flash cell (NAND transistor). MLC SSDs allow higher capacities, storing 2 bits per cell, but classically less writes per flash cell. SSD technology is evolving at a fast pace. You once could only use SLC SSDs for enterprise-class applications. Now there are enterprise-grade, or enhanced multi-level cell (e-MLC) SSDs on the market that have enhanced endurance and have a longer lifetime than HDDs, especially when experiencing the heavy-duty cycles you find in the highest tier.

Overprovisioning techniques are used for failing cells, and data in defunct cells is copied proactively. There are several algorithms for wear-leveling across the cells, for example, allocating the next block to use one that so far has rarely been used, or moving data internally to less-used cells. These actions enhance the SSD’s lifetime. Error detection and correction mechanisms are used, and bad blocks flagged. Overall, SSDs are mature enough to use in critical production environments with a full 24 x 7 duty cycle, and their high performance for small-block / random workloads also make them financially viable for some part of the capacity brought to a hybrid HDD – SSD pool mix.

SSDs give the most performance improvement over HDDs when handling small block random read I/O requests. Small block random write I/O requests also can be handled much better by SSDs compared to HDDs. SSD performance is best for small block random read I/O requests and typically decreases with increasing I/O request block sizes (32 KB or larger) or increasing write ratios. Table 2-2 on page 22 gives a rough estimate of a performance comparison between SSDs and HDDs.

Here are some characteristics of SSDs related to DS8000 systems. Also see Appendix A, “Introduction to Solid State Drives”, in IBM System Storage DS8000: Architecture and Implementation, SG24-8886 for more information about SSDs. SSD features include:

� Faster data access and higher throughput: The SSD has up to 100 more IOPS throughput and 10 better response times than 15 K rpm spinning disks.

� Better durability and reliability: SSDs tolerate extreme shocks, higher altitude, vibration, and extremes of temperature.

� Less power consumption: Because there is no power for the motor required to spin up the magnetic platters and to move the heads, SSDs use less energy than an HDD. Each SSD uses about half of the power of a 15 K rpm HDD.

Chapter 3. Planning for IBM System Storage DS8000 Easy Tier 39

3.2.2 Solid-state drive configuration limitations

For configuration planning purposes, be aware of the following solid-state drive configuration limitations on DS8000 systems:

� Drives of different capacities and speeds cannot be intermixed in a storage enclosure pair. This situation is also true for SSDs.

� A DS8700 system is limited to 32 SSDs per DA pair. The maximum number of SSDs in a DS8700 system is 256 drives spread over eight DA pairs.

� A DS8800 system is limited to 48 SSDs per DA pair. The maximum number of SSDs in a DS8800 system is 384 drives spread over eight DA pairs.

� RAID 5 is the only supported implementation for SSDs. SSD drives follow normal sparing rules. The array configuration is 6+P+S or 7+P. RAID 10 is only supported through an RPQ customer request.

� SSDs are not supported in a DS8800 Business Class machine with feature code 4211 (16 GB of memory).

� Encryption on SSD drives is currently not supported.

� 2.5-inch and 3.5-inch drives on DS8800 systems cannot be intermixed in the same enclosure pair.

Keep in mind the following considerations for the placement of solid-state drives in DS8000 systems:

� A maximum of two SSD ranks per DA pair is a common practice.

� SSD drive sets have a default location when a new machine is ordered and configured.

� SSD drives are installed in default locations by manufacturing, which is the first storage enclosure pair on each device adapter pair. This setup spreads the SSDs over as many DA pairs as possible to achieve optimal price-to-performance ratio.

� The default locations for DS8700 systems are split among eight DA pairs (if installed) in the first three frames: Two in the first frame, four in the second frame, and two in the third frame.

� For DS8700 systems, an SSD feature (drive set or half drive set) is installed in the first disk enclosure pair of an available DA pair. A second SSD feature can be installed on the same DA pair only after the system contains at least eight SSD features, that is, after each of the eight DA pairs contains at least one SSD drive set. This setups means that the system can have more than 16 SSD drives in a DA pair only if the system has two or more frames. The second SSD feature on the DA pair must be a full drive set.

� The default locations for DS8800 are split among eight DA pairs (if installed) in the first two frames: Four in the first frame and four in the second frame.

� Adding SSDs to an existing configuration, to the fourth and fifth frame for DS8700 systems, or the third frame for DS8800 systems, requires a request for price quotation (RPQ/SCORE). This RPQ ensures that the limitation of 32 SSDs per DA pair for DS8700 systems or 48 SSDs per DA pair for DS8800 systems is not exceeded.

For new installations, the most effective distribution of SSDs and HDDs in the DS8000 storage frame for best performance is already decided by strict manufacturing placement rules.


3.2.3 Solid-state drive performance guidelines

Limiting the number of solid-state drives to 16 per DA pair (two SSD arrays or SSD ranks per DA pair) provides the optimal price-to-performance ratio. Compared to DS8700 systems, DS8800 systems provide the best price-to-performance ratio for SSD drives due to its better overall system performance.

Figure 3-1 shows a sample DS8700 configuration with solid-state drives installed in all eight manufacturing default locations. The diagram shows the standard locations of SSD and HDD drive sets.

Figure 3-1 DS8700 system standard configuration with SSD and HDD drives

DS8700 example:

Configuration example with eight SSD disk drive sets and 24 HDD disk drive sets

SSDs are installed in manufacturing default location

Additional SSDs disk drive sets can be installed in the reserved locations

Bottom half of expansion frame 2 (enclosures 33-40) and expansion frames 3 and 4 (not shown) can contain only HDDs (except by RPQ)

Front Rear Front Rear Front RearSSD SSD SSD SSD SSD SSD

reserved for SSDs reserved for SSDs reserved for SSDs1 2 9 10 25 26

HDD HDD HDD HDD HDD HDDHDD HDD HDD HDD HDD HDD

3 4 11 12 27 28

SSD SSD SSD SSD SSD SSDreserved for SSDs reserved for SSDs reserved for SSDs

5 6 13 14 29 30HDD HDD HDD HDD HDD HDDHDD HDD HDD HDD HDD HDD

7 8 15 16 31 32

SSD SSD HDD HDDreserved for SSDs HDD HDD

17 18 33 34HDD HDD HDD HDDHDD HDD HDD HDD

19 20 35 36

SSD SSD HDD HDDreserved for SSDs HDD HDD

21 22 37 38HDD HDD HDD HDDHDD HDD HDD HDD

23 24 39 40

Each small box represents 8 DDMsA pair of boxes represents a disk enclosure (numbered 1-40)Each group of four boxes represents a disk enclosure pair

DA

Pai

r 0

Expansion Frame 1

DA

Pai

r 6

DA

Pai

r 4

DA

Pai

r 7

Expansion Frame 2

DA

Pai

r 3

DA

Pai

r 1

DA

Pai

r 2

DA

Pai

r 2

DA

Pai

r 0

Base Frame

DA

Pai

r 5


Figure 3-2 shows a sample DS8800 configuration with SSD drives installed in all eight manufacturing default locations. The diagram shows the standard locations of SSD and HDD disk drive sets.

Figure 3-2 DS8800 system standard configuration with SSD and HDD drives

3.2.4 Physical back-end configuration examples

The following subsections provide different SSD and HDD drive combinations that re available on DS8700 and DS8800 systems for use with Easy Tier.

SSDs on dedicated DA pairsIn the following configuration example, we can use any of these combinations:

� SSD + HDD Enterprise disk (SAS or FC)� SSD + HDD Nearline disk (NL-SAS or SATA)� HDD Enterprise disk + HDD Nearline disk

Base Frame

HDDHDD HDDHDD HDD HDD

SSD Res erved

SSD Res erved Reserved



SSD Res erved

Res erved ReservedSSDReserved

SSD Res erved


DA

Pa

ir 0

DA

Pa

ir 2

DA

Pa

ir 1

DA

Pa

ir 3

DA

Pa

ir 2

DA

Pai

r 0

Reserved

Expansion Frame

SSD Reserved

Reserved Reser vedSSDReser ved

SSD Res erved


SSD Res erved


SSD Reserved

Reserved Reser vedSSDReser ved

HDD HDDHDD HDDHDD

HDD

HDD HDDHDD HDDHDD

HDD

HDD HDDHDD HDDHDD

HDD

DA

Pai

r 4

DA

Pai

r 7

DA

Pai

r 5

DA

Pai

r 6

DA

Pai

r 4

DA

Pai

r 7

DA

Pa

ir 6

DS880 example:

•Configuration example with eight SSD disk drive sets and 24 HDD disk drive sets

•SSDs are installed in manufacturing default locations

•Additional SSD disk drive sets can be installed in the reserved locations

•Expansion Rack 3 can contain only HDDs (except by RPQ)

Each small box represents 8 DDMsThree small boxes represent a disk enclosureEach group of six boxes represents a disk enclosure pair


Looking at the scope of a DA pair, the minimum possible SSD configuration is just one array site, and this array site can be dedicated to a DA pair, as shown in Figure 3-3.

Figure 3-3 DS8000 with SSDs only within a DA pair - minimum configuration

This configuration is unbalanced, because the array site (and thus the only rank that can be created from that array site) has an affinity to only one of the DS8000 systems (CPCs).

Only one DA pair can have eight SSD-based DDMs. When you add another eight SSD DDMs, they must be added into the same disk enclosure, as shown in Figure 3-3. This setup results in just one array site, and thus one rank with affinity to only one storage server.

SSD disk enclosure

Note:

Not to scale physical adapterlocations

Minimum SSD configuration on a DA pair level

Valid configuration with 1 array site SSD

unused

DA pairs


Figure 3-4 shows a dedicated DS8700 DA pair for SSD ranks only. This configuration is the maximum SSD configuration per DA pair with 32 SSD-based DDMs. In reality, this maximum is not desirable, because 32 SSDs might place too much stress on a single DS8700 DA pair.

Figure 3-4 DS8700 SSDs only within a DA pair - maximum configuration

SSD disk enclosure

Note:


4 SSD array site

DA pairs

DS8700 valid configuration with 2 x 16 SSD - maximum number SSDs per DA pair

SSD = Solid State Drive


Figure 3-5 shows a DS8800 dedicated DA pair for SSD ranks only. This configuration is the maximum SSD configuration per DA pair with 48 SSD-based DDMs. In reality, this maximum is not desirable, because 48 SSDs might place too much stress on a single DA pair.

Figure 3-5 DS8800 SSDs only within a DA pair - maximum configuration

SSD disk enclosure

Note:


6 SSD array site

DS8800 valid configuration with 3 x 16 SSD - maximum number SSDs per DA pair

DA pairs

SSD = Solid State Drive


Mixed back-end SSD / HDD technologies in a DA pairMixed DDM technologies are allowed on the same DA pair, as shown in Figure 3-6 (for DS8700 systems) and Figure 3-7 (for DS8800 systems). This configuration is the minimum SSD configuration per DA pair and also per DS8700 and DS8800 systems. But the same loops also contain HDDs (in this case, those HDDs must all be of the same type (size and rpm).

Figure 3-6 DS8700 mixed disk drive technologies within a DA pair

Figure 3-7 DS8800 mixed disk-drive technologies within a DA pair

16 SSD disk enclosure

Note:


Valid configuration

16 HDD disk enclosure

1 array site SSD

Disk enclosurePair 1


4 array site HDD

SSD = Solid State DriveHDD = Enterprise or Near-line Drive


Note:


Valid configuration


1 array site SSD

6 array site HDD





Typically, this configuration is the common configuration for workload isolation. The set of DDMs shown in these figures are enough to cover the requirements of one application and provide Easy Tier capabilities.

Example of SSD / HDD and ENT / NL back-end configurationTwo extent pools are sufficient when they are equal in size and performance capabilities. Figure 3-8 shows a possible back-end storage with an Enterprise HDD and Nearline HDD configuration, which has advantages when it comes to minimizing management efforts for tuning the DS8700 workload.

Figure 3-8 DS8700 ENT HDD / NL HDD back-end example per DA pair

Note: In mixed configurations with SSD and HDD devices, always remember the limits of the DA pair regarding its IOPS capabilities. One SSD rank can saturate a single Device Adapter (DA), especially on DS8700 systems with the previous generation of DAs.

Note:


16 ENT disk enclosure

Valid configuration

16 NL disk enclosure

2 array site ENT HDD

4 array site NL HDD

ENT DDM = FC or SAS DriveNL DDM = SATA Drive


Figure 3-9 shows a possible back-end storage with an SSD and HDD configuration (this configuration can be Enterprise or Nearline HDD), which has advantages when it comes to minimizing management efforts for tuning the DS8800 workload.

Figure 3-9 DS8800 SSD / HDD back-end example per DA pair

The example in Figure 3-10 shows, as a feasible compromise, four extent pools for a DS8000 system using Easy Tier with FB and CKD storage. This configuration assumes a symmetrical configuration across all four DA pairs. Each DA pair manages no more than two SSD ranks and four HDD ranks, except DA pair DA4 with no SSD ranks. The SSD disk enclosure is only half populated and you cannot add any other disk technology to this disk enclosure except for SSD technology when fully populating this disk enclosure. Figure 3-10 shows these empty spaces, where you potentially could add more than 16 SSDs, as blank cells.

Figure 3-10 Symmetrical back-end configuration for four three-tier extent pools


Note:


Valid configuration


2 array site SSD

6 array site HDD




SSD SSD DA2 SSD SSD DA2DA0 NL HDD NL HDD DA0

ENT HDD ENT HDD DA6 NL HDD NL HDD DA6ENT HDD ENT HDD DA4 NL HDD NL HDD DA4

DA2 ENT HDD ENT HDD DA2SSD SSD DA0 SSD SSD DA0

P0 P1

NL HDD NL HDDNL HDD NL HDD DA2 ENT HDD ENT HDD DA2

DA0 NL HDD NL HDD DA0SSD SSD DA6 SSD SSD DA6

DA4 ENT HDD ENT HDD DA4NL HDD NL HDD DA6 NL HDD NL HDD DA6NL HDD NL HDD DA4 NL HDD NL HDD DA4

P2 P3

ENT HDD ENT HDDServer #0 Server #1

NL HDD NL HDDNL HDD NL HDD SSD = Solid-state drives

ENT = Enterprise drivesNL = Nearline drives

DA6 DA6

Manufacturing default location

Configuration with 4 Extent Pools

DA2 DA2

DA0 DA0

DA4 DA4

Array / Rank / DA Pair


Figure 3-10 on page 48 assumes, with its four extent pools, a configuration with a DS8000 system that hosts FB volumes and CKD volumes (otherwise, just two extent pools are sufficient to adequately balance the workload). For example, P0 and P1 can be dedicated to FB volumes, while P2 and P3 can be used for CKD volumes. Each volume type uses the entire DA back-end configuration and also both DS8000 systems (CPCs). Because the base frame hosts only DA pairs DA0 and DA2, an expansion frame is needed for more than two DA pairs.

If you only need to support one volume type (FB or CKD), then you can have just two extent pools and fewer SSD ranks. This configuration might be a dedicated DA pair with 1 16 SSD with an SSD rank in extent pool P0 and an SSD rank in extent pool P1. All other DA pairs manage HDD ranks, which are evenly spread across extent pool P0 and extent pool P1. Again, make sure to use all DA pairs for both extent pools. Such a configuration is suited to be managed by Easy Tier Automatic Mode.

3.3 Logical configuration planning

As distinct from previous releases, Easy Tier generation 3, which was made available with DS8000 R6.2 LMC, is beneficial in both multitier pools (hybrid pools) and single-tier pools (homogeneous pools), providing automated cross-tier and intra-tier storage performance management. You can even benefit from Easy Tier even if you do not have several storage tiers in one system. You can have considerable improvements and storage performance optimization even with Enterprise class drives only using the auto-rebalance and micro-tiering features of Easy Tier. To use all Easy Tier capabilities in Automatic Mode, hybrid pools with different storage tiers must be created.

Supported disk drive technology mixes in an Easy Tier Automatic Mode extent pool are:

� SSD + Enterprise disk (SSD+ENT)� SSD + Nearline disk (SSD+NL)� Enterprise disk + Nearline disk (ENT+NL)� SSD + Enterprise disk + Nearline disk (SSD+ENT+NL)

These combinations can be used to create hybrid or multitier extent pools. Such extent pools can be created by adding ranks of various drive technologies to an extent pool or by merging extent pools with the wanted mix of drive technologies through the dynamic extent pool merge function.

With DS8000 R6.2 LMC, Easy Tier Automatic Mode now fully supports up to three tiers in an extent pool, providing a three-tier storage hierarchy. Using Automatic Mode, you can distribute your entire workload across three-tier extent pools with fully automated storage performance and storage economics management and efficient distribution of bandwidth and IOPS across all storage tiers. Logical configuration of the storage can be made based on the capacity requirements and the percentage of SSD, Enterprise, and Nearline drives required to satisfy performance needs. Thin provisioned volumes are supported now and also benefit from Easy Tier automated storage performance and economics management.

Note: If you are planning to have a high-performing configuration from the beginning, you should consider using two DS8000 frames (main and the first expansion) to be able to have as many DA pairs as possible.


The following conditions must be met to enable Easy Tier:

� The Easy Tier LIC feature is installed and enabled. It is required for both Easy Tier Manual and Automatic Mode, but not for Easy Tier monitoring.

� For Automatic Mode to be active, the following conditions must be met:

– Easy Tier monitoring is set to either all or automode.

– For Easy Tier to actively manage pools, the Easy Tier Auto Mode control switch must be set to either tiered or all pools.

3.3.1 General considerations for DS8000 logical configurations

Here we describe general considerations for DS8000 logical configurations.

Extent pool configurationFor best performance and a balanced configuration, it is common practice to create a minimum of two or more extent pools. In these pools, you have at least one pool for rank group 0 managed by DS8000 storage server 0 and one pool for rank group 1 managed by DS8000 storage server 1. You assign half of the ranks of each device adapter pair (DA pair) to even-numbered extent pools (P0, P2, P4, ...) managed by storage server 0 and the other half to odd-numbered extent pools (P1, P3, P5, ...) managed by storage server 1 to evenly balance the workload and the available back-end bandwidth between all device adapters (DAs). It is important to note that unassigned ranks do not have a fixed or predetermined relationship to any DS8000 system. It is important that the ranks from each DA pair are distributed evenly across both systems, or you might seriously limit the available back-end bandwidth and the system’s overall throughput. If, for example, all ranks of a given DA pair are assigned to only one storage server, only one DA card of the DA pair is used to access the set of ranks, so only half of the available DA pair bandwidth is used.

This practice now also becomes especially important with solid-state drives and small block random I/O workloads. To be able to use the full back-end random I/O performance of two (or more) SSD ranks within a given DA pair, the SSD I/O workload must be balanced across both device adapters of the DA pair. This balance can be achieved by assigning half of the SSD ranks of each DA pair to even extent pools (P0, P2, P4, ..., managed by storage server 0 / rank group 0) and half of the SSD ranks to odd extent pools (P1, P3, P5, ..., managed by storage server 1 / rank group 1). If, for example, all SSD ranks of the same DA pair are assigned to the same rank group (for example, rank group 0 with extent pool P0 and P2), only one DA card of the DA pair is used to access this set of SSD ranks. Only half of the available I/O processing capability of the DA pair is available, which severely limits the overall SSD performance.

Attention: Consider a little “breathing” space for the Easy Tier to start working; even a few extents are fine. Do not allocate 100% of the capacity in a a managed extent pool with volumes, or Easy Tier management and extent migrations are effectively halted.

Note: Unassigned ranks do not have a fixed or predetermined relationship to any DS8000 system. Each rank can be assigned to any extent pool or any rank group. Only when assigning a rank to an extent pool and thus rank group 0 or rank group 1 does the rank become associated with storage server 0 or storage server 1. Ranks from rank group 0 (even-numbered extent pools, such as P0, P2, P4, and so on) are managed by storage server 0. Ranks from rank group 1 (odd-numbered extent pools, such as P1, P3, P5, and so on) are managed by storage server 1.


Consider these guidelines for best performance:

� SSD ranks must be properly assigned to even and odd rank groups (storage server 0 / 1).

� Mind the balanced distribution of the overall workload across both DS8000 CPCs or DS8000 storage servers. Half of the ranks of each DA pair must be assigned to rank group 0 and the other half of the ranks must be assigned to rank group 1. This configuration is also correct for the extent pools.

� In general, always use storage pool striping with the default rotate extents allocation method wherever possible when creating volumes in homogeneous multi-rank extent pools to balance the workload across all ranks in an extent pool.

Following these guidelines provides a balanced utilization of the full back-end bandwidth and cache memory.

To check for a correct logical configuration of ranks across DA pairs, complete the following steps:

1. Identify the DA pair association for each SSD rank by running lsarray -l(Example 3-1).

Example 3-1 Identify SSD rank to DA pair association

dscli> lsarray -lArray State Data RAIDtype arsite Rank DA Pair DDMcap (10^9B) diskclass encrypt==========================================================================================A0 Assigned Normal 5 (6+P+S) S1 R0 0 300.0 SSD unsupportedA1 Assigned Normal 5 (6+P+S) S2 R1 0 300.0 SSD unsupportedA2 Assigned Normal 5 (6+P+S) S3 R2 1 300.0 SSD unsupportedA3 Assigned Normal 5 (6+P+S) S4 R18 1 300.0 SSD unsupported

2. Ensure that half of the SSD ranks from each DA pair are assigned / configured to even extent pools (P0, P2, P4 rank group 0 managed by storage server 0) and half of the SSD from the same DA pair are assigned / configured to odd extent pools (P1, P3, P5 rank group 1 managed by storage server 1) (Example 3-2).

Example 3-2 Ranks assigned to odd and even extent pools

dscli> lsrank -lID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts encryptgrp=====================================================================================================R0 0 Normal Normal A0 5 P0 FB_HybridP0 fb 1574 0 -R1 1 Normal Normal A1 5 P1 FB_hybridP1 fb 1574 1401 -R2 0 Normal Normal A2 5 P2 CKD_hybridP2 ckd 1763 1304 -R18 1 Normal Normal A3 5 P4 ITSO_FB_P4 fb 1574 70 -

The foregoing example ensures that the SSD load is balanced across both device adapters of each DA pair. If this rule is not followed correctly on each DA pair level with SSD ranks, then you might be wasting half of your SSD performance by only using one DA adapter of each DA pair for SSD access.

Remember to also check and apply the same rule in general for the other types of disk drives.


Extent allocation method (EAM)Logical volumes can be allocated using one of two extent allocation methods (EAMs), either rotate extents or rotate volumes.

The rotate extents EAM (also known as storage pool striping) distributes the capacity allocated by one logical volume across the available ranks so that workloads with a random access I/O pattern can benefit from the performance of multiple ranks. The rotate extents EAM is generally preferable because it tends to avoid skew and hot spots on any one rank because the workload of all volumes tends to be distributed over the available ranks. This method is the default when new volumes are created and no EAM is specified in non-managed single-tier extent pools. The initial extent allocation used in managed extent pools is similar to rotate extents. See 2.5.9, “Extent allocation in hybrid and managed extent pools” on page 30 for more information.

The rotate volumes EAM tries to localize the capacity allocated to one volume on a single rank, but distributes volumes across the available ranks. This method might be desirable for limiting the number of ranks included in the logical volume regarding its failure boundary.

The ability to specify the extent allocation method for a volume migration allows the extent allocation method to be changed any time after volume creation.

Volume migration into the same pool specifying the rotate extents EAM can also be used to redistribute extents of a volume in a balanced manner in an extent pool after additional ranks have been added to the pool. This configuration is also known as manual volume rebalance, as described in “Manual volume rebalance” on page 103.

In multitier or hybrid extent pools, extents are initially allocated using a rotate extents across all ranks of the same storage tier, starting with an extent allocation on the Enterprise class, or the so-called home tier first. Any specified extent allocation method for the volume creation or migration command is ignored. Furthermore, the extent allocation method of each volume in a managed pool is immediately set to managed after Easy Tier Automatic Mode management for this pool is enabled.

Note: Because Easy Tier applies different migration types to distribute the workload across tiers in hybrid pools, new volumes are always created on the Enterprise class drive tier first. However, extents can be demoted to Nearline or promoted to SSD ranks after the pool becomes managed by Easy Tier Automatic Mode. This configuration is reflected by the EAM attribute of managed for all volumes being or once having been under the control of Easy Tier Automatic Mode.

Note: In hybrid extent pools, extents of created or migrated volumes are always allocated on ranks of the Enterprise class drive tier first, using a rotate extents EAM across all ranks of the same drive tier, when free capacity is available on this drive tier. If the Enterprise tier is fully allocated, then extent allocation continues on ranks of the Nearline tier and finally on the SSD tier if all capacity on the other drive tiers is exhausted. Hybrid extent pools are always prepared for Easy Tier Automatic Mode management and any specified extent allocation method like rotate extents (eam=rotateexts) or rotate volumes (eam=rotatevols) during volume creation or migration is ignored in these extent pools during extent allocation.


3.3.2 Guidelines for creating multitier extent pools

In managed multitier or hybrid extent pools, Easy Tier Automatic Mode automatically promotes hot extents from the lower tier to the upper tier (Enterprise to SSD and Nearline to Enterprise) or demote colder extents from the higher tier to the lower tier (swap extents from SSD with hotter extents on Enterprise, or demote cold extents from Enterprise to Nearline). Automatic Mode also optimizes the Nearline tier by demoting some of the sequential workload to the Nearline tier to better balance sequential workloads. Auto-rebalance rebalances extents across the ranks of the same tier based on rank usage to minimize skew and avoid hot spots. Auto-rebalance even takes different device characteristics into account when different devices or even RAID levels are mixed within the same storage tier (micro-tiering).

The following list contains guidelines for creating multitier or hybrid extent pools managed by Easy Tier Automatic Mode, based on early experience with Easy Tier:

� SSD drives to HDD drives ratio:

The ratio of SSD disk capacity to HDD disk capacity in a hybrid pool depends on the workload characteristics and skew. Ideally, there must be enough SSD capacity to hold the active (hot) extents in the pool, but not more, to not waste expensive SSD disk capacity. For new DS8000 orders, 3 - 5% of SSD capacity might be a reasonable percentage to plan for with hybrid pools in typical environments. This configuration can result in the movement of 50% of the small and random I/O workload from enterprise drives to SSDs. This configuration gives a reasonable initial estimate if measurement data is not available to support configuration planning. See “Drive Selection with Easy Tier” of IBM System Storage DS8800 and DS8700 Performance with Easy Tier 3rd Generation, WP102024 for additional information.

The Storage Tier Advisor Tool (STAT) also provides guidance for SSD capacity planning based on the existing workloads on a DS8000 with Easy Tier monitoring capabilities. For additional information, see 3.7.1, “Storage Tier Advisor Tool” on page 61.

� Enterprise HDD to Nearline HDD ratio:

For the ratio of Enterprise HDD capacity to Nearline HDD capacity, the previous considerations also apply, with the Enterprise tier holding the hot data, and the Nearline tier holding the cold data. Use the Easy Tier monitoring data together with the Storage Tier Advisor Tool to determine the actual ratio.

� HDD types:

SSD capacity is likely to be limited, so you cannot always assume that all the busy data migrates to SSDs; part of the data must stay on HDD disks.

Note: After the Easy Tier Automode is set to all, Easy Tier starts managing all pools on the system (both multitier (hybrid) pools and single-tier (homogeneous) pools). All eligible volumes become immediately managed by Easy Tier with their eam attribute permanently being changed to managed. All previous volume EAM attribute information is lost after Easy Tier Automode has been set to all.


In general, it makes sense to use the same capacity, speed, and RAID type within a tier of a hybrid pool. There is no obvious benefit to be gained by mixing various HDD drive sizes, speed, and RAID types within a tier of a hybrid pool. Start planning the HDD drives with the intended workload in mind. If 600-GB drives with RAID 5 are suitable for the workload, it is not generally considered a best practice to mix them, for example, with 300 GB RAID 10 drives in the same pool. However, exceptions might arise for practical reasons, such as your current drive size no longer being available or your workload changing in a way where it is more cost effective to buy new capacity in a larger drive size.

Starting with LMC R6.2, the drive types may be mixed using the enhanced micro-tiering capability of Easy Tier (which was initially introduced with R6.1 LMC). Enhanced micro-tiering takes device-specific performance characteristics into account when determining the rank utilization for intra-tier performance management (auto-rebalance), such as device variations (speed and RAID level), workload characteristics, and the DS8000 platform. However, micro-tiering does not deliberately apply cross-tier extent migration types between the different device types within a tier; it just applies auto-rebalance migrations balancing the workload within the drive tier across ranks based on rank characteristics and utilization. This configuration allows your system to avoid skew and hot spots within a storage tier. Having a mix of drive types and RAID levels in an extent pool impedes performance analysis and makes it more difficult to understand what is going on.

� RAID types:

SSD and HDD arrays in a hybrid pool do not need to have the same RAID level. In an extent pool with SSD and Nearline drives, you are likely to have various RAID levels because SSD drives support RAID 5 (RAID 10 with a RPQ/SCORE request), and Nearline drives are used with RAID 6.

Whether RAID 10 must be used for Nearline drives in a hybrid pool (possible with DS8700 systems only) depends on the write content of the workload. RAID 10 typically provides better performance for write-intensive workloads. There is less need for RAID 10 in a hybrid pool where Easy Tier Automatic Mode migrates active extents to SSD ranks, but as we cannot assume that all busy data migrates to SSDs, there might be performance benefits by using RAID 10.

Also, with Easy Tier, small block random reads are favored for the SSD drives more than random writes, and if your workload is characterized by many random writes, then you also might still want to use RAID 10 with DS8700 systems. This configuration applies particularly to Nearline drives, given the impact associated with RAID 6. For Enterprise drives, RAID 5 is the preferable implementation.

� Size and number of extent pools:

Before the availability of Easy Tier, DS8000 installations used relatively small extent pools, typically with only four to eight ranks. When new capacity was added to a system, new extent pools were created for the new ranks instead of adding them to the existing extent pools. There was no benefit to adding them to existing, fully allocated extent pools because there was no way of redistributing the extents of all the existing volumes evenly across all the ranks in the pool, including the new ones. The new volumes had their extents allocated on the new ranks, while the existing volumes remained on the old ranks, so the result was the same as putting the new ranks in new extent pools.

With Easy Tier, you can now manually redistribute the extents of existing volumes over all ranks in a non-managed, single-tier extent pool by using Easy Tier Manual Mode and the dynamic volume relocation capability (manual volume rebalance). These functions help improve performance by avoiding skew and hot spots because volume extents can be spread evenly over more ranks and DA pairs in the pool.


However, hybrid pools should be managed by Easy Tier Automatic Mode, and in managed extent pools, no manual restriping is required (or supported) to redistribute the extents after new capacity has been added to the pool. This situation occurs because Easy Tier automated cross-tier and intra-tier performance management (auto-rebalance) automatically takes advantage of the new capacity. Auto-rebalance automatically relocates data based on extent workload and rank utilization, providing a balanced resource utilization within a drive tier to avoid skew and rank hot spots.

It is best to create large extent pools, thus improving performance and simplifying storage management. Ultimately, you might have just two extent pools for each rank type (FB and CKD), one on storage server 0 and one on storage server 1.

However, there are still reasons to create multiple extent pools:

– HDDs of the same storage tier with various characteristics (capacity, speed, and RAID type) should not typically be mixed in the same extent pool, but exceptions might apply under certain circumstances.

– Separate extent pools can be based on failure boundaries. For example, backup volumes, such as permanent IBM FlashCopy volumes created with the copy option, can be placed in another extent pool than the original volumes. With this situation, a single rank failure does not affect both the original and the backup. However, this operation is not necessary if the FlashCopy target does have protection against rank failures (for example, as in a Metro Mirror environment).

– Separate extent pools with different tiers depending on user needs, such as one extent pool with three tiers (SSD + ENT + NL), and another one with two tiers (SSD + ENT). This configuration allows you to plan accordingly for specific host needs and different workload characteristics or situations, for example, when you need to plan for specific minimum or maximum requirements.

3.3.3 Guidelines for creating single-tier extent pools

Single-tier or homogeneous extent pools are pools that only contain disks from one storage tier. For example:

� SSD� Enterprise disk� Nearline disk

With DS8000 R6.2 LMC, a DS8000 configuration without any hybrid extent pools now can also benefit from Easy Tier automatic performance management in homogeneous single-tier extent pools using the auto-rebalance feature. Easy Tier monitors the rank usage and rebalances the extents across all the ranks of the same storage tier within the extent pool to reduce skew and avoid hot spots, thus providing the best achievable performance in a homogeneous extent pool.

Consider the following items:

� Consider configuring multi-rank extent pools to allow auto-rebalance to operate more effectively.

� Even single-tier pools with different device characteristics might benefit from Easy Tier auto-rebalance, as the algorithms takes different device characteristics (micro-tiering) into account. See “HDD types:” on page 53 for more information.

� Consider having free capacity left in the extent pool to allow auto-rebalance to efficiently relocate extents across ranks.

� Enterprise disks with different drive speeds, for example, ENT 10 K rpm and 15 K rpm, are considered the same storage tier.


Easy Tier Auto Mode should be set to all and Easy Tier monitoring should be set to all or automode volumes to enable auto-rebalance in all single-tier pools.

In tiered mode, auto-rebalance is only enabled in hybrid or multitier pools. If Easy Tier mode is set to none, no volumes are managed by Easy Tier. However, you can manually redistribute a logical volume within a non-managed, single-tier extent pool using manual volume rebalance or migrate a volume from one extent pool to another using the dynamic volume relocation (DVR) feature of the Easy Tier Manual Mode.

The enhanced functions of Easy Tier Manual Mode offer additional management capabilities and ease of use. You can use Easy Tier Manual Mode features to easily relocate the extents of a volume within an extent pool, or to relocate an entire volume from one pool to another pool. When a volume migration is targeted to the same pool and the target EAM is rotate extents, the volume migration acts internally as manual volume rebalance. Use manual volume rebalance to relocate the smallest number of extents of a volume and restripe the extents of that volume on all available ranks in the pool. The behavior of the regular volume migration, which differs from manual volume rebalance, continues to operate as it did in the previous version of Easy Tier.

Manual volume rebalance evenly rebalances the extents of a volume across all ranks within a non-managed, single-tier extent pool, which is helpful, for example, if new ranks have been added to the pool or if pools have been merged. A hybrid or multitier extent pool is always assumed to be prepared for Easy Tier Automatic Mode management and does not allow the manual redistribution of extents within the pool using the dynamic volume relocation feature.

Volume restriping within an extent pool can be achieved by initiating manual volume rebalance. Use manual volume rebalance to minimize skew and hot spots in a non-managed, single-tier extent pool when the capacity distribution of the volumes is highly imbalanced across ranks. Examples are if new capacity has been added to the pool, pools have been merged, or when the extent allocation of a given volume is changed from rotate volumes to rotate extents to benefit from storage pool striping.

However, in managed single-tier extent pools, no manual restriping is required to redistribute the extents after new capacity has been added to the pool, because Easy Tier automated intra-tier performance management (auto-rebalance) automatically takes advantage of the new capacity. Auto-rebalance automatically relocates data based on extent workload and rank utilization, providing a balanced resource utilization within a drive tier to avoid skew and rank hot spots.

Even with the Easy Tier LIC feature enabled, you can still create standard single-tier extent pools for dedicated workloads not managed by Easy Tier. An example is if you have specific optimized configurations where you do not want an automated extent relocation to happen. These configurations might be highly optimized environments with dedicated resources that do not allow resource sharing between applications or different clients, and where already host based data striping and allocation methods are efficiently applied.

Note: When manually relocating the extents of a volume, only the extents of the volume, that is, the volume’s capacity, is spread evenly across the available ranks of the target extent pool (volume striping). No workload characteristics are taken into account for any manual extent relocation.


Another example might be that you already applied dedicated extent pools for dedicated storage tiers and selected workloads and manually manage the placement of volumes on the appropriate storage tier. However, you might want to consider enabling and benefiting from the auto-rebalance feature of Easy Tier in single-tier extent pools to reduce skew and avoid single ranks becoming hot spots. You can, gradually and without interrupting your host workloads, move to full Easy Tier Automatic Mode for automated intra-tier and cross-tier performance management whenever you want by using the Easy Tier Manual Mode features, such as dynamic extent pool merge and dynamic volume relocation.

However, the use of Easy Tier Automatic Mode is highly encouraged, as it provides flexible storage management with ease of use combined with subvolume storage performance and economics management at an excellent price-to-performance ratio. Large hybrid pools with Easy Tier Automatic Mode offer a safe and easy configuration if you do not have special requirements to design dedicated highly optimized extent pools. Easy Tier automatically handles data placement across tiers and within tiers for the best overall performance.

3.3.4 Additional considerations for thin provisioned volumes

Virtual storage and space efficient repositories do not allow you to specify a specific EAM, but allocate extents in rotate extents implicitly. DS8000 R6.2 LMC introduced full support of thin provisioned ESE volumes with Easy Tier Automatic Mode and Manual Mode.

If you are using track space-efficient (TSE) volumes, remember that:

� Easy Tier Manual Mode does not support DVR of TSE volumes.

� Space-efficient TSE volumes can be in an extent pool managed by Easy Tier Automatic Mode, but they are not managed by Easy Tier Automatic Mode or Manual Mode.

� Merging extent pools with TSE volumes is allowed, but you must consider that you cannot merge two extent pools if both extent pools contain TSE volumes and if there are TSE volumes on the SSD ranks.

� Rank depopulation is supported on ranks containing TSE and ESE volumes.

In hybrid extent pools, thin provisioned volumes are no longer limited to non-SSD ranks. With DS8000 R6.2 LMC, Easy Tier fully supports ESE volumes in Automatic and Manual Mode and rank depopulation for ranks containing ESE and TSE volumes.

In environments only using FlashCopy copy services, which already support the usage of thin provisioned volumes (ESE) with DS8000 R6.2 LMC, you might start using thin provisioned volumes in managed hybrid extent pools. With three tiers, start the initial volume allocation for as many volumes as possible on the Enterprise tier (home tier). This configuration avoids the initial creation of fully provisioned volumes on the Nearline drive tier when the storage capacity on the Enterprise tier is exhausted due to fully provisioned volumes with unused allocated capacities being created. In this case, only used capacity is allocated in the pool, and Easy Tier does not need to move unused extents around or move hot extents on a large scale from the Nearline tier to the Enterprise tier and then to the SSD tier.

However, thin provisioned volumes are not fully supported with all Ds8000 copy services or advanced functions yet, so it might not be a valid approach for all environments at this time.

Thin provisioning: Starting with DS8000 R6.2 LMC, Easy Tier introduced full support of thin provisioned volumes (ESE) in Automatic Mode and Manual Mode (dynamic volume relocation, extent pool merge, and rank depopulation).


See 2.5.9, “Extent allocation in hybrid and managed extent pools” on page 30 for more information about the initial volume allocation in hybrid extent pools.

For DS8000 LMC levels before R6.2, thin provisioned volumes were limited to HDD ranks only in hybrid pools. The thin provisioned volumes could run out of available HDD capacity before the total extent pool ran out of free extents. This situation occurred because the total capacity included the free capacity on the SSD ranks, which was not available for the thin provisioned volumes. If your DS8000 systems are still running on a DS8000 LMC level before R6.2, consider setting the SNMP warning extent threshold for thin provisioned volumes lower in managed hybrid extent pools with SSD drives.

3.4 Implementation considerations for multitier extent pools

For multitier extent pools that are managed by Easy Tier Automatic Mode, additional considerations should be taken into account about how to allocate the initial volume capacity and how to gradually introduce new workloads to managed pools.

3.4.1 Using thin provisioning with Easy Tier in multitier pools

In environments only using FlashCopy copy services, which already support the usage of thin provisioned volumes (ESE) with DS8000 R6.2 LMC, you might start using thin provisioned volumes in managed hybrid extent pools (especially with three tiers). You use this configuration to start the initial volume allocation for as many volumes as possible on the Enterprise tier (home tier) and avoid the initial creation of fully provisioned volumes on the Nearline drive tier when the storage capacity on the Enterprise tier is already exhausted. This exhaustion is due to the creation of fully provisioned volumes with unused allocated capacities.

In this case, only used capacity is allocated in the pool and Easy Tier does not move unused extents around or move hot extents on a large scale up from the Nearline tier to the Enterprise tier and to the SSD tier. However, thin provisioned volumes are not fully supported with all DS8000 copy services or advanced functions yet, so it might not be a valid approach for all environments at this time. See 2.5.9, “Extent allocation in hybrid and managed extent pools” on page 30 for more information about the initial volume allocation in hybrid extent pools.

3.4.2 Staged implementation approach for multitier pools

Suppose you are putting a new DS8800 system into production that replaces an older system that does not have Easy Tier implemented. Some thought should be put into the timeline of the implementation stages, that is, migrating all servers from the older to the new system.

In a new three-tier environment, new volumes are first created on the Enterprise tier, and Easy Tier cannot “learn” and optimize before production has started. If you are migrating all the servers at once, some servers’ volumes, for reasons of space, might even be placed completely on Nearline drives first, although these servers also might have higher performance requirements. See 2.5.9, “Extent allocation in hybrid and managed extent pools” on page 30 for more information about the initial extent allocation in hybrid extent pools.


Consider a staged approach when migrating servers to a new multitier DS8800 system:

� Assign the resources for the high-performing and response time sensitive workloads first, then add the less performing workloads. Doing it the other way might lead to situations where all initial resources, such as the Enterprise tier in hybrid extent pools, are allocated already by the secondary workloads. This situation does not leave enough space on the Enterprise tier for the primary workloads, which then must be initially on the Nearline tier.

� Split your servers into several subgroups, where you migrate each subgroup one by one, and not all of them at once. Then allow the Easy Tier some days to learn and optimize, and some extents will be moved to SSDs and some to Nearline, and we regain space on the Enterprise HDDs. After a server subgroup has finished learning and has practically reached a steady state, the next server subgroup can be migrated. You gradually allocate the capacity in the hybrid extent pool by optimizing the extent distribution of each application one by one while regaining space in the Enterprise tier (home tier) for the next applications.

3.5 Copy Services considerations

Copy Services are supported with Easy Tier. Copy Services (Metro Mirror, Global Mirror, and FlashCopy) do not impact the operations of Easy Tier (in Automatic Mode or Manual Mode).

All back-end I/Os, except the extent migration encountered with Easy Tier Manual Mode (dynamic volume relocation) and Automatic Mode (Automatic Data Relocation), are counted in the Easy Tier I/O statistics, including Copy Services back-end I/Os. However, most of the Copy Services background I/O activity has sequential access patterns and is not contributing to the cross-tier heat calculation of Easy Tier on a bigger scale, although it is taken into account for the bandwidth and rank utilization calculations.

Also, like host operations, Copy Services (Metro Mirror, Global Mirror, and z/OS Global Mirror) are unaware of the extent or volume level relocations being performed.

In a Metro Mirror environment, there is an additional time delay due to the required data transfer of a write I/O to the secondary site. This additional latency or service time is not included in the performance data considered by Easy Tier, because this I/O activity is not an activity that is occurring in the disk back end on the rank level.

The following considerations apply when Copy Services are in place, assuming a symmetrical configuration:

� When using volume migration, you must have procedures in place to move volumes at both the primary and secondary systems to ensure that the same performance levels can be maintained in the event of a failover to the secondary system.

� When Easy Tier is in Automatic Mode, the workloads being monitored on the primary and the secondary system differ. Easy Tier at the primary system sees normal workload while at the secondary system it sees only the write workloads. This situation means that the optimized extent distribution on the primary system differs considerably from the one on the secondary system. The optimized extent reallocation based on the workload learning on the primary system is not sent to the secondary at this time to allow the same extents optimization on both systems based on the primary workload pattern.

Tip: Copy Services are not aware of Easy Tier data optimization.


In a disaster recovery situation with a failover from the primary to a secondary site, the extent distribution of the volumes on the secondary system are not optimized to match the primary workload in general. Easy Tier relearns the production I/O profile and builds a new extent migration plan on the secondary system to adapt to the new production workload before eventually achieving the same optimization and level of performance as on the primary system. This task takes some time, so the production workload on the secondary system might not run at its optimum performance.

� FlashCopy copy-on-write activity results in full track writes, which are considered large rather than small. The FlashCopy target is not hot in terms of small I/Os unless it is being written to.

� Remember that a FlashCopy track space-efficient repository is not considered for extent relocation.

3.6 Workload planning

Easy Tier environments provide a much greater degree of choice for number of drive combinations than when considering a single tier of storage. The key considerations are:

� How would an initial single tier configuration for the planned workload look like?

� How much of my data is doing little random I/O or only some sequential I/O and could be considered cold?

� What is the skew of the data and how much SSD capacity would be needed to handle the major part of the random workload?

� Given a usage of Nearline or SSD drives, what size of Enterprise drives should be used to handle the workload remaining on the Enterprise tier?

� Is there a need to provide better back-end performance in the target environment to accelerate application workloads and is the workload type (cache hit ratios / percentage sequential workload) such that SSD storage provides significant application acceleration?

Workload planning for Easy Tier third generation does become more complex because you are not planning for a static workload distribution anymore. Data is moved dynamically according to the heat of the data and eventually distributed across different storage tiers. When planning drive configurations for Easy Tier, always start planning for a single target drive tier first, which is the tier where the workload is placed initially in a regular single-tier configuration. Typically, this tier is the Enterprise tier. Consider the skew of the workload and adapt the initial configuration by adding higher drive tiers for hot data and lower drive tiers for cold data. Adapt the number of drives and drive size of the initial target tier accordingly. For example, if there are three tiers available (SSD, Enterprise, and Nearline), then planning considerations could be the following:

� Databases should be targeted to the Enterprise tier first to be able to promote hot extents to the SSD tier and demote cold extents to the Nearline tier. This configuration consumes all three tiers while capacity on the Enterprise tier is released.

� File sharing services might be targeted at the Nearline tier instead promoting some of the extents to the Enterprise tier and not promoting them to an SSD tier to prevent consumption of the bandwidth of the SSD drives. This configuration consumes only two tiers.

Plan the capacity of the target tier and then plan the capacity of the associated tiers. Guidelines for this planning can be found in 3.3.2, “Guidelines for creating multitier extent pools” on page 53.


Also see “Drive Selection with Easy Tier” in IBM System Storage DS8800 and DS8700 Performance with Easy Tier 3rd Generation, WP102024, for additional information, which is available at the following website:

http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102024

This approach allows the DS8000 system to use its internal Automatic Mode algorithms to balance the performance, constantly adapt to the changing application workloads, and leave the manual control to the administrators only where necessary. There is always the ability to add a tier (or ranks) to the existing extent pools or redistribute the workload across the ranks already in use.

With three-tier support, most of the existing workloads benefit from Easy Tier Automatic Mode:

� OLTP and randomized-nature workloads have better response times.

� Large block sequential and batch workloads benefit by moving workloads partly to the Nearline-SAS or SATA drives, freeing space on Enterprise drives and releasing bandwidth on the Enterprise tier.

� Highly mixed and unpredictable workloads are managed and monitored by Easy Tier, providing heat distribution and cold extents allocation with appropriate recommendations about the performance improvement.

The following workloads might not benefit from the Easy Tier Automatic Mode, but Manual Mode features might be used to support management of these workloads:

� Hot spots are small in size and uniformly distributed across extents such that all extents exhibit equal temperatures. This case might not benefit from Automatic Mode, but Manual Mode might support the management of these workloads using successive dynamic volume dynamic relocations.

� Hot spots vary over time such that they are uniformly distributed, given a large enough monitoring period. This case also might not benefit from Automatic Mode, but Manual Mode could be used to move such workloads to one or several extent pools created for them specifically.

For additional information, see DS8800 Performance Monitoring and Tuning, SG24-8013.

3.7 Planning tools

The following tools are available to help plan Easy Tier implementations.

3.7.1 Storage Tier Advisor Tool

The Storage Tier Advisor Tool (STAT) is a Windows application that can be used to analyze the characteristics of the workload running on the DS8000 system. It provides capacity planning information associated with the current or future use of the Easy Tier.


http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102024

Storage Tier Advisor Tool overviewThe STAT processes data that is collected by Easy Tier monitors. The DS8000 monitoring capabilities are available regardless of whether you have installed and activated the Easy Tier license feature on your DS8000 as long as it is supported by the DS8000 LMC level. The monitoring capability of the DS8000 enables it to monitor the usage of back-end storage at the extent level. Monitoring statistics are gathered and analyzed at least every 24 hours. The results of this data are summarized in a summary report data that can be downloaded from the DS8000 system for reporting purposes using the STAT.

The STAT provides a graphical representation of performance data that is collected by the Easy Tier monitor over a 24-hour operational cycle. You can view the information of the STAT to analyze workload statistics and evaluate which logical volumes might be candidates for Easy Tier management. If you have not installed and enabled the Easy Tier feature, you can use the performance statistics gathered by the monitoring process to help you determine whether to use Easy Tier to enable potential performance improvements in your storage environment and to determine optimal SSD or Enterprise HDD configuration and benefits.

You can activate the Easy Tier monitor and download the monitor summary data from a DS8000 system either with the DS GUI or with DS CLI.

For information about downloading the STAT, see 5.2, “Obtaining and installing the Storage Tier Advisor Tool” on page 120.

Storage Tier Advisor Tool improvementIn its initial version, STAT estimated performance improvement and SSD configuration recommendations for the storage system as a whole.

The latest version of STAT in DS8000 R6.2 LMC is more granular. The performance improvement estimation and SSD configuration recommendations are now provided on an extent pool basis. The tool estimates the performance improvement for each pool by using the existing SSD ranks, and gives recommendations about the type and number of SSD ranks that should be configured for optimizing the performance.

The calculation of the performance estimation was improved in the new STAT release. Previously, STAT took the heat values of each bucket as linear, resulting in inaccurate estimation when the numbers of extents in each bucket are disproportional. STAT with R6.2 LMC now uses the average heat value provided at the sub-LUN level to provide a more accurate estimate of the performance improvement.

Availability: Easy Tier monitor is available on any DS8000 system starting with LMC level 6.5.1.xx (DS8700) or LMC 7.6.10.xx (DS8800), with or without the Easy Tier LIC feature installed. No SSD disks are required to benefit from the Easy Tier monitoring data and the reports provided by the STAT.


Figure 3-11 shows a sample report from the STAT. You see that 3% of the data is in a hot state and there is one pool that hosts all three tiers.

Figure 3-11 Storage Tier Advisor Tool report for a DS8000 system with hybrid pools

Based on the collected data, the tool specifies performance improved by adding existing spare SSD capacity, as shown in Figure 3-12. It predicts that this addition will provide a performance improvement of up to 10%. Also, there can be capacity improvement by adding Nearline drives to the existing pools by providing additional capacity for cold demotion of cold extents. The performance improvement here is based on a 24-hour profile and the performance improvement might be larger in online time periods (with higher random workloads) and might be less in batch periods (with lower random workloads).

Figure 3-12 Recommended configurations for SSD and NL tiers


A detailed look at the hot and cold extents distribution of each volume is provided by the Volume Heat Distribution report. There you can find information about the cold, warm, and hot capacity of each volume in the extent pool (Figure 3-13).

Figure 3-13 Volume Heat Distribution

See Chapter 5, “Using IBM System Storage DS8000 Easy Tier” on page 119 for details about how to use the STAT and interpret its associated reports.

3.7.2 Disk Magic

Disk Magic is a Windows-based disk subsystem performance modeling tool used by IBM and IBM Business Partners to model disk storage subsystem performance. It supports disk subsystems from multiple vendors and offers detailed support for IBM storage subsystems. Contact your IBM Representative or IBM Business Partner to evaluate a Disk Magic study.

Disk Magic has been enhanced to support Easy Tier. Disk Magic can be used to model the benefit of moving a workload from a non Easy Tier system to an Easy Tier system. Support is available for both open systems and IBM System z models.


The workload can be modeled by using manual input of the workload characteristics, iostat outputs, IBM RMF™ datafiles, Tivoli Storage Productivity Center reports, and so on. See Figure 3-14 for an example.

Figure 3-14 Disk Magic welcome window


If you manually enter the workload definition into the model, then you are given the various options to define the workload: the number of ranks, the capacity and type of the drives, and the definition of the workload itself. See Figure 3-15 for an example of such a disk configuration.

Figure 3-15 Defining disk subsystem configuration window

Easy Tier modeling for the DS8000 system is done on a skew-factor basis. It is the most important parameter to the model input, because depending on the workload skew, various results can be achieved with the Easy Tier modeling. There are three predefined types of skew level for the workload: heavy, medium, and light skew.

A heavily skewed workload has many I/Os bound to only a small portion of the data, which means that moving just a few extents to SSDs (the hottest extents) greatly improves performance. For a lightly skewed workload, there is a much smaller difference in the workload intensity on the busiest extents versus the most inactive extents. This situation means that the expected performance improvement is smaller. When selecting Easy Tier, be sure to choose the best match for your workload. Choosing the heavily skewed profile results in a more aggressive sizing, so you need to be sure that the workload is indeed heavily skewed. To get a more conservative prediction, the lightly skewed option can be used.


Figure 3-16 provides an explanation of the workload skew.

Figure 3-16 Example of Disk Magic skew factors

The three curves shown in Figure 3-18 on page 69 represent three predefined skew levels provided by the Disk Magic model. The top curve represents the heavy skew factor while the lower curve represents the light skew factor. The curve for the heavy skew factor implies that, for example, 80% of the I/O load of this workload is bound to 20% of the capacity. By planning to use 20% SSD capacity for the hot extents, then 80% of the I/O workload moves to SSD. Disk Magic assumes, in the model with an extent pool where 20% of the capacity is provided by SSD ranks, that Easy Tier fills this 20% of the capacity with hot data servicing 80% of all the I/Os. For the medium skewed workload, the 20% SSD Tier ranks would get 55% of the I/Os and for the lightly skewed workload the SSD Tier ranks would get 37% of the I/Os.

It is essential, for a proper sizing, to figure out the skew factor of your workload. This task can be done manually or by using the Disk Magic capabilities of statistics input analysis of Tivoli Storage Productivity Center data, for example.


Disk Magic provides the results of the storage model in some reports by showing the usage of the storage system components, as shown in Figure 3-17. After all calculations and estimations are done, you can easily see the workload estimations in various graphs for various skew values and compare the results, as shown in Figure 3-18 on page 69.

Figure 3-17 Disk model utilizations window

Figure 3-17 shows the usage of the calculated disk system model. Here you find that HDDs are highly used, as are host adapters. Using the capabilities of Disk Magic, you can make several estimations using the various skew settings. You can see that the usage decreases while the capability to handle higher workloads increases.


Figure 3-18 Three skew factor estimations for one workload

In Figure 3-18, you can see the result of the workload modeling for three different skew factors. The skew factors are heavy, medium, and light. As you can see, the workload benefit from using Easy Tier with SSD drives when there is a heavy skew factor. Using Easy Tier in this case moves the upper saturation limit of the system far away while providing lower response times. This situation means that before doing any proposals, the system’s current workload and data placement should be analyzed with regards to the skew factor. The higher the skew factor, the more the system benefits from Easy Tier and SSD drives.

For additional information about Disk Magic, see DS8800 Performance Monitoring and Tuning, SG24-8013.


Chapter 4. DS GUI and DS CLI support for IBM System Storage DS8000 Easy Tier

In this chapter, we describe the DS GUI and DS CLI support and controls for Easy Tier. We cover the following main topics:

� DS GUI and DS CLI support and controls for Easy Tier� Dynamic volume relocation (volume migration, including manual volume rebalance)� Dynamic extent pool merging� Rank depopulation� Easy Tier Automatic Mode management

4


4.1 DS GUI and DS CLI support for Easy Tier

Easy Tier can be managed using the DS GUI or the DS CLI. The Easy Tier support in the DS GUI and DS CLI is related to dynamic volume relocation (or volume migration), dynamic extent pool merge, and rank depopulation capabilities, and the Automatic Mode and Monitor selections for Easy Tier.

Easy Tier provides performance monitoring capabilities, regardless of whether the Easy Tier license feature is activated. Easy Tier uses the monitoring process to determine what data to move to the appropriate tier within an extent pool by creating appropriate migration plans in Easy Tier Automatic Mode. This monitoring data can be exported using the DS GUI or DS CLI and processed by the Storage Tier Advisor Tool (STAT), which provides I/O statistics and volume heat maps. This information can be used to analyze back-end workload statistics and evaluate the workload distribution across ranks and volumes. It provides workload distribution data on subvolume level (heat maps) and lets you identify which logical volumes might be best candidates for Easy Tier Automatic Mode management. It also provides guidance on how to achieve potential performance improvements by using existing or additional drive resources (SSD, ENT, and NL drives) in your storage environment.

4.2 DS Storage Manager GUI support for Easy Tier

The following DS Storage Manager GUI windows are related to Easy Tier features:

� Migrate Volumes windows� Volume Properties window� Internal Storage main window� Single Pool Properties window� Rank Properties window� Manage Volumes windows� Easy Tier controls in Storage Image Properties window� Extent Pool Merge window

4.2.1 Volume migration window

The volume migration-related windows support the dynamic volume relocation available in Easy Tier Manual Mode. See 4.3.1, “Dynamic volume relocation” on page 78 for a description of these windows.

4.2.2 Volume Properties window

The Status tab of the open systems Single Volume Properties window shows the status of the selected volumes. Easy Tier Manual Mode has five configuration states:

� Migrating� Migration Paused� Migration Cancelled� Migration Error� Transposing

The first four states are related to the dynamic volume migration facility (see 4.3.1, “Dynamic volume relocation” on page 78), while the transposing state is related to the dynamic extent pool merge facility (see 4.3.2, “Dynamic extent pool merge” on page 88).


This window also shows the Access state and the Data state. Two additional migration-related fields report the number of extents left to be migrated and the source extent pool from where they are being migrated (Figure 4-1).

Figure 4-1 Single Volume Properties window - showing migration status

Similarly, the CKD Single Volume Property window shows the same fields.

4.2.3 Disk Configuration Summary and Internal Storage main windows

The Internal Storage main window shows the current storage allocation status of the whole storage system. The capacity summary area provides information about the amount of total used real capacity for open systems and IBM System z, the amount of capacity in ranks not assigned to extent pools, the total amount of extents being migrated, the amount of capacity allocated to standard and Space Efficient volumes, and the amount of unassigned capacity.

The Extent Pools tab shows the relevant information of each extent pool. The configuration states (see 4.2.2, “Volume Properties window” on page 72) are reported in the Disk Configuration Summary bar charts and in the allocation bar in the Extent Pool tab.

Chapter 4. DS GUI and DS CLI support for IBM System Storage DS8000 Easy Tier 73

Figure 4-2 shows the storage capacity as a bar chart in the Internal Storage main window.

Figure 4-2 Extent Pools main page - Disk Configuration Summary

4.2.4 Single Pool Properties window

The Single Pool Properties window allows the user to accept or modify the extent pool name and to change the storage threshold percentages. This window also shows the total amount of data available in the extent pool, the extent pool ID, and the server affinity (0, 1).

GiB amounts: The amount of GiB in migration status reported in the Internal Storage Summary window takes into account the amount of GiB remaining to be migrated from the source pool and the amount GiB still pre-allocated in the target pool.


Starting with DS8000 R6.2 LMC, the Single Pool Properties window shows the Number Of Tiers in the extent pools. This window also shows if the extent pool is currently managed by Easy Tier Automatic Mode by showing Yes or No in the Easy Tier Managed field (Figure 4-3).

Figure 4-3 Single Pool Properties window

Furthermore the # Source GiB migrating field shows the amount of GiB to be migrated from the pool, and the # Target GiB migrating field shows the amount of GiB remaining to be migrated to the pool. This migration uses volume migration with dynamic volume relocation provided by Easy Tier Manual Mode.


4.2.5 Manage Volumes window

The open systems Manage Volumes window (click Manage Volumes FB Volumes to open it) provides the migration actions in the Select action drop-down menu (Figure 4-4). This window shows the four available migration tasks:

� Initiate Migration� Pause Migration� Resume Migration� Cancel Migration

Figure 4-4 The Manage Volumes window (FB / open systems)


Similarly, the Manage Existing LCUs and volumes window (click CKD LCUs and Volumes Manage Existing LCUs and volumes to open it) also provides Easy Tier Manual Mode volume migration actions (Figure 4-5).

Figure 4-5 The Manage LCUs and Volume window (CKD / System z)

4.2.6 Merge Extent Pools window

The Merge Extent Pools window (Figure 4-6) allows the user to merge the source extent pool into the target extent pool. When this operation has finished, the source extent pool is deleted, leaving only the target extent pool.

Figure 4-6 Merge Extent Pools window


4.2.7 Easy Tier controls in the Storage Image Properties window

The Storage Image Properties window shows the properties for the storage image. Information provided includes the Machine Type Serial (MTS), Emulated MTS, WWN number, and current firmware level (Figure 4-7).

This window controls Easy Tier Automatic Mode and Monitor options in the Advanced tab. See 4.3.4, “Easy Tier Automatic Mode” on page 94 for further details about these settings.

Figure 4-7 Storage Image Properties window

4.3 Easy Tier functions with the DS Storage Manager GUI

In the following sections, we describe the implementation of the Easy Tier capabilities through the DS Storage Manager GUI.

4.3.1 Dynamic volume relocation

Dynamic volume relocation allows volumes to be migrated from one extent pool to another (or even back into the same extent pool) transparently to host operations. This capability is an Easy Tier Manual Mode capability. The user can perform the following actions:

� Initiate migration: Starts a volume relocation in the same or another extent pool.

� Pause/Resume migration: Pauses and resumes an ongoing volume relocation. A user might want to suspend temporarily a volume relocation to avoid additional workload on the back-end storage resources.


� Cancel migration: Cancels an ongoing volume relocation. A user might want to cancel a volume relocation because the volume migration is no longer needed. After a volume relocation cancelation, the volume might be in a transient state (migration canceled) at the time that the volume extents are spread between two extent pools. Correct this situation by initiating (and completing) a new volume relocation.

To initiate a volume relocation task for FB volumes from the DS GUI, click FB Volumes Manage existing volumes, right-click the volume, and initiate migration actions.

For the CKD volumes, click CKD LCUs and Volumes Manage Existing LCUs and volumes, right-click the volume, and initiate migration actions.

All the steps reported from this point refer to FB volumes, but are also applicable to CKD volumes.

Consideration: Dynamic volume relocation is allowed only among extent pools of the same volume storage type, either CKD or Fixed Block (FB), and of the same server affinity. For example, you can migrate volumes between extent pools P0 and P2 (rank group 0 managed by storage server 0) or P1 and P3 (rank group 1 managed by storage server 1), but not between P0 and P1. Migration to the same extent pool or manual volume rebalance is not supported for any managed or hybrid (multitier) extent pool, even if the hybrid extent pool is currently not managed by Easy Tier Automatic Mode. Additionally, track space efficient (TSE) volume relocation also is not supported. See “Dynamic volume relocation” on page 15 and “Manual volume rebalance” on page 16 for more information.


Starting migrationTo start a volume migration, complete the following steps:

1. From the Tasks section on the FB Volumes window, click Manage existing volumes (Figure 4-8).

Figure 4-8 FB Volumes main window (open systems)

2. On the Manage Volumes window, click the volume which you want to relocate. Then, from the Action drop-down menu, click Initiate Migration (Figure 4-9).

Figure 4-9 Select Initiate Migration on the Manage Volumes window


3. On the Initiate Migration window (Figure 4-10), which opens after the volume and extent pool information retrieval is completed, provide the following information:

– The extent allocation method (EAM) that you want to use in the target extent pool for the volume being migrated

– The target extent pool for the migration.

Figure 4-10 Initiate Migration window

Target extent pool management: If the target extent pool is managed by Easy Tier Auto Mode, an extent allocation method similar to rotate extents is used independently of the specified extent allocation method, as described in 2.5.9, “Extent allocation in hybrid and managed extent pools” on page 30.

Extent pools: Because dynamic volume relocation is allowed only among extent pools with the same server affinity, the list of the target extent pools only contains the extent pools with the same server affinity as the source extent pool.


4. After specifying all the information, click Next to open the verification window (Figure 4-11).

Figure 4-11 Migration confirmation window

5. Click Finish to start the migration. The volume status changes from normal to migrating (Figure 4-12). The # Migrating extents field shows the number of extents still to be migrated.

Figure 4-12 The Single Volume Properties window showing the migration status “Migrating”


The configuration state migrating is also reported in the Manage Volumes window (Figure 4-13).

Figure 4-13 The Manage Volume window reporting the migration status

After the volume relocation is complete, the volume state returns to normal (Figure 4-14).

Figure 4-14 The Single Volume Properties window showing the migration status “Normal”


Pausing and resuming migrationTo pause migration, complete the following steps:

1. On the Manage Volumes window, click the volume line, and from the Select action drop-down menu, click Pause Migration (Figure 4-15).

Figure 4-15 Select the Pause Migration action

The Pause Migration confirmation window opens (Figure 4-16).

Figure 4-16 Pause Migration confirmation window


2. Click Continue to pause the migration. The volume status now changes from migrating to migration paused (Figure 4-17).

Figure 4-17 The Single Volume Properties window showing the migration status “Migration paused”

3. To resume the migration, select the volume to be resumed from the Manage Volumes window and from the Select action drop-down menu, click Resume Migration (Figure 4-18).

Figure 4-18 Select the Resume Migration action


The Resume Migration confirmation window opens (Figure 4-19).

Figure 4-19 Resume Migration confirmation window

4. Click Continue to resume the migration. The volume status now changes back from migration paused to migrating (Figure 4-13 on page 83).

Canceling migrationTo cancel migration, complete the following steps:

1. On the Manage Volumes window, select the volume, and from the Action drop-down menu, click Cancel Migration (Figure 4-20).

Figure 4-20 Select the Cancel Migration action


The Cancel Migration confirmation window opens (Figure 4-21).

Figure 4-21 Cancel Migration confirmation window

2. Click Continue to cancel the migration. The volume status now changes from migrating to Migration Canceled (Figure 4-22).

Figure 4-22 The Single Volume Properties window showing the migration status “Migration canceled”


The status of the volume in the Manage Volumes window reflects the change (Figure 4-23).

Figure 4-23 Manage Volumes migration status

3. To restart the migration, click Initiate Migration from the Action drop-down menu in the Manage Volumes window (Figure 4-9 on page 80).

4.3.2 Dynamic extent pool merge

The dynamic extent pool merge allows one extent pool (the source extent pool) to be merged with another extent pool (the target extent pool) transparently to the attached host systems. This capability is an Easy Tier Manual Mode capability.

To merge extent pools with the DS GUI, click the Pools icon and click Internal Storage.

Volume migration: If the same extent pool is used for the source and target extent pool for a volume migration (for example, using manual volume rebalance), then the Cancel Migration action changes the state from migrating to normal. All the volume extents still belong to only one extent pool.

Attention: The dynamic extent pool merge may only be used among extent pools with the same server affinity. For example, you can merge extent pools P0 and P2 (rank group 0 managed by server #0) or P1 and P3 (rank group 1 managed by server #1) but not P0 and P1. You can only merge extent pools of the same volume storage type, either CKD or FB. See “Dynamic extent pool merge” on page 13 for more details about Easy Tier Manual Mode.


The Manage Internal Storage window opens, and under the Extent Pool tab in the Manage Internal Storage section, you can see all the available extent pools. Complete the following steps:

1. Select the extent pools to be merged by pressing and holding the Ctrl key and clicking the pools (Figure 4-24).

Figure 4-24 Manage Internal Storage main page - extent pool selection

2. Select Merge Pool from the Action drop-down menu (Figure 4-25).

Figure 4-25 Select Merge Pool from the Manage Internal Storage window

3. The GUI starts retrieving the extent pool data and a window opens (Figure 4-26).

Figure 4-26 Merge Extent Pools window with extent pool role selection


This window allows the user to select the role of the extent pools in the merging process. Check the role for each extent pool and switch the role with the Switch button if necessary. Click OK to start the merging process.

After the merge has completed, the following window opens (Figure 4-27).

Figure 4-27 Extent pool merge completion message

4. Click Close to go back to the Internal Storage window. Click View Details, and the Long Running Task Properties window opens, showing the details of the merging task.

4.3.3 Rank depopulation

Rank depopulation is an Easy Tier Manual Mode feature that allows a user to unassign one rank from an extent pool, even if there are extents allocated on the rank. In this case, Easy Tier automatically reassigns all allocated extents to the other ranks in the extent pool. This operation is not apparent to host operations.

To remove a populated rank from an extent pool, click the Pools icon and click Internal Storage.

Extent pool merge: The extent pool merge may only be done among extent pools with the same server affinity. If you try to merge two extent pools with various server affinities, you receive an error message (CMUG00408E) indicating that the two extent pools are not serviced by the same server. Similarly, you receive the same error message if you try to merge a CKD extent pool in an FB extent pool or vice versa. For further details about dynamic extent pool merge, see “Dynamic extent pool merge” on page 13.

Attention: A rank depopulation operation task fails if there are not enough extents available on the remaining ranks in the extent pool. See “Rank depopulation” on page 18 for more details.


The Manage Internal Storage window opens, and under the Extent Pool tab in the Manage Internal Storage section, you can see all the defined extent pools. Complete the following steps:

1. Select the extent pool that contains the rank to be depopulated and then, using the Action drop-down menu, click Remove Capacity from Pools (Figure 4-28).

Figure 4-28 Remove Capacity from Pools (rank depopulation)

In this example, the extent pool chosen (ITSO_HYB_CKD_P8) is a CKD pool, but the operations would be the same with an FB extent pool.

2. In the Remove Capacity from Extent Pools window, a list of the ranks assigned to the chosen extent pool is displayed. Select multiple ranks that you want to remove by pressing and holding the Ctrl key and click the ranks. Click Remove (Figure 4-29).

Figure 4-29 Remove Capacity from Extent Pools - select ranks to be depopulated

In this example, we have chosen rank R17, which has 29 assigned extents.


3. A second window opens, showing the list of volumes that have extents allocated on the chosen rank. Click Move the volumes (Figure 4-30) to allow Easy Tier to move the affected extents, transparently to the hosts at which those volumes are mapped, to the remaining ranks in the pool, and click Continue.

Figure 4-30 Remove Capacity from Extent Pools - select option to delete or move volumes

Deleting volumes: Clicking Delete Volume removes the volumes from the extent pool. However, this option fails if the volumes are part of an existing volume group, and then no change is done at the rank level.


Easy Tier migrates allocated extents from the chosen rank to the remaining ranks in the pool, and the rank configuration state changes to Depopulating (Figure 4-31).

Figure 4-31 Single Pool Properties showing rank status “Depopulating”


We can see, in this example, that Easy Tier has pre-allocated the extents in both remaining ranks, and that 10 extents have already been migrated off the R17 rank. The rank properties also reflect the rank state change (Figure 4-32).

Figure 4-32 Single Rank Properties

This operation can take a while, depending on the number of extents to migrate and the overall activity in the extent pool. When all extents have been migrated, the rank is unassigned from the extent pool.


Easy Tier Automatic Mode manages the capacity allocated to volumes within a managed extent pool to place the extents on the most appropriate storage tiers and ranks regarding storage performance and economics optimization and balancing resource utilization. For information about the extent pool planning for Easy Tier Automatic Mode, see 3.3, “Logical configuration planning” on page 49. When Easy Tier Automatic Mode is enabled, Easy Tier automatically handles data placement for the best overall performance. See Chapter 2, “IBM System Storage DS8000 Easy Tier concepts, design, and implementation” on page 7 for more details.

To activate Easy Tier Automatic Mode, complete the following steps:

1. Click the Home icon and click System Status.


2. Click the Storage Complex and, from the Action drop-down menu, click Storage Image Properties (Figure 4-33).

Figure 4-33 Select Storage Image Properties to access Easy Tier mode controls


3. On the Storage Image Properties window, click the Advanced tab to open the window with the Easy Tier controls (Figure 4-34 on page 96).

Figure 4-34 Easy Tier controls on the Advanced tab of the Storage Image Properties window

To change Easy Tier Automatic Mode, select the following options and click Apply:

– Easy Tier Auto Mode manages the Easy Tier Automatic Mode behavior. You can select the following options:

• All Pools: Automatically manage all single and multitier pools.

• Tiered Pools: Automatically manage multitier pools only.

• No Pools: No volume is managed.

– Easy Tier Monitor controls the Easy Tier monitoring behavior. You can select the following options:

• Auto Mode Volumes: Only volumes managed by Easy Tier Auto Mode are monitored.

• All Volumes: All volumes in the system are monitored.


• No Volumes: No volume is monitored.

4.3.5 Exporting the Easy Tier Summary Report

To export the Easy Tier Summary Report, complete the following steps:

1. Click the Home icon and click System Status. On the System Status window, select the appropriate Storage Complex and Storage Image. Click Storage Image Export Easy Tier Summary Report from the Action drop-down menu (Figure 4-35).

Figure 4-35 Exporting the Easy Tier Summary Report from the System Status window

Important: Do not turn off Easy Tier Monitor when Easy Tier Auto Mode is turned on. If Easy Tier Monitor is turned off and no new performance statistics are provided, Easy Tier Auto Mode cannot create migration plans. As a result, Easy Tier Auto Mode stops managing volumes.

Attention: Changing the Easy Tier monitoring mode affects the statistics collection and can lead to a reset (reinitialization) of the gathered monitoring data. This situation means that it might take up to 24 hours of collecting new performance statistics after Easy Tier monitoring has been enabled again until new migration plans are created. Easy Tier Automatic Mode can continue to automatically optimize storage performance and economics by relocating extents across and within storage tiers.


After the Easy Tier data has been retrieved the Export Summary Report (Figure 4-36), the File Download window opens (Figure 4-37).

Figure 4-36 Export Summary Report completion window

Figure 4-37 File Download window

2. Click Close in the Export Summary Report window and click Save in the File Download window to save the compressed file containing the Easy Tier Summary data files to your local workstation. These files must be extracted and processed by the Storage Tier Advisor Tool (STAT) (see Chapter 5, “Using IBM System Storage DS8000 Easy Tier” on page 119).

4.4 DS Storage Manager CLI support for Easy Tier

The following DS CLI commands support Easy Tier features:

chsi Controls Easy Tier Automatic Mode and Easy Tier monitoring.offloadfile Downloads Easy Tier monitoring data.manageckdvol Initiates CKD volume migration actions (dynamic volume relocation).managefbvol Initiates FB volume migration action (dynamic volume relocation).chextpool Performs dynamic extent pool merge.chrank Performs rank depopulation.

The following commands show Easy Tier status information:

� lskey� lsckdvol -l� lsfbvol -l� lsextpool -l� lsrank


� showckdvol� showfbvol� showextpool� showrank� showsi

The lskey command shows whether the Easy Tier LIC feature key is installed, not installed, or not enabled for a storage type (Example 4-1).

Example 4-1 Checking for Easy Tier license key (lskey) and control settings (showsi)

dscli> lskey IBM.2107-75TV181Activation Key Authorization Level (TB) Scope==========================================================================[...]IBM System Storage Easy Tier on All

dscli> showsi IBM.2107-75TV181Storage Unit IBM.2107-75TV180Model 951WWNN 500507630ABBC2AFState OnlineESSNet EnabledVolume Group V0Cache Memory 235.8 GBProcessor Memory 253.4 GBMTS IBM.2421-75TV180ETAutoMode tieredETMonitor all

The chsi or showsi commands change or show the control settings of Easy Tier Automatic Mode (ETAutoMode=all, tiered, or none) and Easy Tier monitoring (ETMonitor=automode, all, or none). When Easy Tier is supported on the DS8000 LMC level, Easy Tier monitoring can be activated on the storage facility image independently of whether the LIC feature is installed or enabled.

In addition, the lsextpool -l and showextpool commands show information about the number of tiers in an extent pool and if the pool is currently managed by Easy Tier Automatic Mode. See 4.5.4, “Easy Tier Automatic Mode” on page 115 for more information.

The volume configuration state of the volume commands has appropriate states reflecting volume migration and extent pool merge operations as follows:

� Merging: The volume’s extent pool is being merged into another pool.� Migrating: The volume is in the process of migrating.� Migration Paused: The volume migration is paused.� Migration Canceled: The volume migration was canceled.� Migration Error: An error has occurred during volume migration.� Transposition Error: An error has occurred while merging extent pool.

The rank configuration state has appropriate states relating to rank depopulation as follows:

� Depopulating: The rank is in the process of depopulating so that it can be unassigned from its extent pool.

� Depopulation Error: An error has occurred during rank depopulation.

� Unassigned Reserved: A rank is not assigned to any extent pools, and it is also reserved.


4.5 Easy Tier functions using the DS Storage Manager CLI

In this section, we provide examples of using the Easy Tier capabilities through the DS Storage Manager CLI. For a detailed description, see the Command-Line Interface User's Guide for the DS6000 series and DS8000 series, GC53-1127, or run help.

4.5.1 Dynamic volume relocation

Dynamic volume relocation allows a logical volume to be migrated from one extent pool to another pool (or even the same extent pool, using for example, manual volume rebalance) transparently to host operations. This capability is an Easy Tier Manual Mode capability. You can perform the following actions:

� Initiate a migration: Start a volume relocation in the same or another extent pool. The volume’s extent allocation method can be changed as part of the migration.

� Pause / resume a migration: Pause and resume an ongoing volume relocation. You might want to suspend a volume relocation temporarily to avoid additional workload on the back-end storage resources.

� Cancel a migration; Cancel an ongoing volume relocation. You might want to cancel a volume relocation because the volume migration is not needed anymore. After a volume relocation cancelation, the volume might end up in a transient state (migration canceled) at the time that the volume extents are spread between two extent pools. This situation must be corrected by initiating (and completing) a new volume relocation.

Dynamic volume relocation is performed using the managefbvol command for FB volumes and the manageckdvol command for CKD volumes. The command parameters are the same for both, FB and CKD environments. The syntax is:

managefbvol -action <action> -eam rotatevols|rotateexts -extpool TgtPoolID VolID

You can one of the following commands:

� migstart: Starts a volume migration.� migpause: Pauses a volume migration.� migresume: Resumes a paused volume migration.� migcancel: Cancels a volume migration.

The eam parameter is optional and can be used to change the volume’s EAM. It defaults to the volume’s current EAM. The extpool parameter is optional and defaults to the volume’s current extent pool.

Command output: Some of the command output lines in the following examples have been truncated to make the examples more readable.

Dynamic volume relocation considerations: Dynamic volume relocation is allowed only among extent pools of the same volume storage type, either CKD or Fixed Block (FB), and of the same server affinity. For example, you can migrate volumes between extent pools P0 and P2 (rank group 0 managed by storage server 0) or P1 and P3 (rank group 1 managed by storage server 1) but not between P0 and P1. Migration to the same extent pool or manual volume rebalance is not supported for any managed or hybrid (multitier) extent pool, even if the hybrid extent pool is currently not managed by Easy Tier Automatic Mode. Track space efficient (TSE) volume relocation also is not supported. See “Dynamic volume relocation” on page 15 and “Manual volume rebalance” on page 16 for more information.


The target extent pool must belong to the same rank group (0/1) as the volume’s current extent pool, which means that source and target extent pool need to be managed by the same DS8000 system. For example, a volume of LSS 10 located in Extent Pool P0 (rank group 0), which is managed by storage server 0, cannot be migrated to Extent Pool P1 (rank group 1), which is managed by storage server 1.

Volume migrationExample 4-2 shows volume migration from one extent pool to another using dynamic volume relocation. It also demonstrates the cancel operation. The source extent pool P0 contains three ranks and just the one volume to be migrated (volume ID 1000); the target extent pool P2 contains two ranks and is empty. This setup makes it easy to see where the volume extents are allocated.

Example 4-2 Manual volume migration to another extent pool

dscli> lsextpoolName ID stgtype rankgrp status availstor (2^30B) %allocated available reserved numvols===========================================================================================ITSO_FB_A P0 fb 0 below 4734 0 4734 0 1ITSO_FB_B P2 fb 0 below 1558 0 1558 0 0ITSO_FB_C P4 fb 0 below 388 0 388 0 0

dscli> lsrank -extpool p0 -lID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=========================================================================================R2 0 Normal Normal A2 5 P0 ITSO_FB_A fb 1582 4R3 0 Normal Normal A3 5 P0 ITSO_FB_A fb 1582 4R4 0 Normal Normal A4 5 P0 ITSO_FB_A fb 1582 4

dscli> lsrank -extpool p2 -lID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=========================================================================================R0 0 Normal Normal A0 5 P2 ITSO_FB_B fb 779 0R1 0 Normal Normal A1 5 P2 ITSO_FB_B fb 779 0

dscli> managefbvol -action migstart -extpool p2 1000CMUC00430I managefbvol: The migstart action for FB volume 1000 has completed.

dscli> lsfbvol -config not_normalName ID accstate datastate configstate deviceMTM datatype extpool cap (2^30B)=====================================================================================ITSO_J 1000 Online Normal Migrating 2107-900 FB 512 P0 12.0

dscli> managefbvol -action migcancel 1000CMUC00430I managefbvol: The migcancel action for FB volume 1000 has completed.

dscli> lsfbvol -config not_normalName ID accstate datastate configstate deviceMTM datatype extpool cap (2^30B)=========================================================================================ITSO_J 1000 Online Normal Migration Cancelled 2107-900 FB 512 P2 12.0

dscli> lsrank -extpool p0 -lID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=========================================================================================R2 0 Normal Normal A2 5 P0 ITSO_FB_A fb 1582 2R3 0 Normal Normal A3 5 P0 ITSO_FB_A fb 1582 2R4 0 Normal Normal A4 5 P0 ITSO_FB_A fb 1582 1

dscli> lsrank -extpool p2 -l


ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=========================================================================================R0 0 Normal Normal A0 5 P2 ITSO_FB_B fb 779 6R1 0 Normal Normal A1 5 P2 ITSO_FB_B fb 779 1

dscli> lsextpoolName ID stgtype rankgrp status availstor (2^30B) %allocated available reserved numvols===========================================================================================ITSO_FB_A P0 fb 0 below 4741 0 4741 0 0ITSO_FB_B P2 fb 0 below 1551 0 1551 0 1ITSO_FB_C P4 fb 0 below 388 0 388 0 0

dscli> managefbvol -action migstart -extpool p4 1000CMUN01131E managefbvol: 1000: The Migrate volume task cannot be initiated because the volume that you have selected has a volume configuration state of Migration_cancelled, and therefore the volume can only be migrated to the source or target extent pool of the cancelled migration task.

dscli> managefbvol -action migstart -extpool p2 1000CMUC00430I managefbvol: The migstart action for FB volume 1000 has completed.

dscli> lsfbvol -config not_normalCMUC00234I lsfbvol: No FB Volume found.

dscli> lsfbvol 1000Name ID accstate datastate configstate deviceMTM datatype extpool cap (2^30B)========================================================================================ITSO_J 1000 Online Normal Normal 2107-900 FB 512 P2 12.0

dscli> lsrank -extpool p2 -lID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=========================================================================================R0 0 Normal Normal A0 5 P2 ITSO_FB_B fb 779 6R1 0 Normal Normal A1 5 P2 ITSO_FB_B fb 779 6

Complete the following actions:

1. Check the initial extent pool status by running lsextpool and lsrank.

2. Start migration by running managefbvol -action migstart. The lsfbvol -config not_normal command can be used to quickly list all volumes in a configuration state other than Normal, for example, volumes in a Migrating state.

3. Cancel migration by running managefbvol -action migcancel. The lsfbvol command shows that the volume is in a Migration Canceled state. The lsrank commands show that the volume’s extents are now located in both extent pools. We also could verify this situation by running showfbvol -rank 1000 instead. The lsextpool command shows that the volume is considered to be in the target extent pool even though it has extents in both pools.

An attempt to start migration to a third extent pool (P4) fails.

4. Restart migration to the original target pool P2 by running managefbvol -action migstart. Eventually the migration completes and the volume returns to a Normal state. The last lsrank command shows that all the extents are now on the two ranks of P2.

Volume migration: If the same extent pool is used as source and target pool of a volume migration, for example, with manual volume rebalance, then the migration cancel action changes the state from Migrating to Normal, instead of Migration Canceled. All the volume extents still belong to only one extent pool.


Manual volume rebalanceManual volume rebalance is designed to redistribute the extents of volumes within a non-managed, single-tier (homogeneous) pool so that workload skew and hot spots are less likely to occur on the ranks. Manual volume rebalance is an enhancement of the dynamic volume relocation feature introduced with DS8000 R6.1 LMC. This feature is when a volume is migrated back into the same extent pool with the source extent pool being the target extent pool and the EAM being rotate extents (which means using storage pool striping). In this case, the algorithm tries to evenly spread the volume’s extents across all ranks in the extent pool.

Use manual volume rebalance when a rank is added to a pool, homogeneous extent pools have ben merged, or when large volumes with an EAM of rotate volumes are deleted and a rebalance can optimize a non-balanced extent distribution within a homogeneous pool. See “Manual volume rebalance” on page 16 for additional information.

With manual volume rebalance, only one extent at a time is allocated (rather than preallocating the full volume), so only the smallest number of extents required needs to be relocated. There only needs to be a little free capacity available on the ranks to achieve a balanced capacity distribution across all ranks. Iif a manual volume rebalance operation is executed in an existing pool with only partial capacity available on some ranks and no capacity available on other ranks, there might not always be a balanced distribution of the volume capacity across all ranks. An example is if one rank is fully allocated by another volume using an EAM of rotate volumes. In this case, you might need to perform the relocation in a certain sequence or in multiple steps to make sure to have at least some free extents available on each rank in the pool.

Example 4-3 shows the migration of nine FB volumes back into the same extent pool (P6). This migration shows the benefits of manual volume rebalance. For newly created volumes, the default extent allocation method (EAM) is rotate extents. The example also demonstrates the use of migration pause and resume.

Example 4-3 Extent pool capacity distribution before adding new capacity and initiating manual volume rebalance

dscli> lsextpool -l p6ID stgtype rankgrp status availstor (2^30B) %allocated available numvols numranks numtiers etmanaged====================================================================================================P6 fb 0 full 0 100 0 9 2 1 no

dscli> lsfbvol -l -extpool p6ID accstate datastate configstate deviceMTM datatype extpool sam captype cap (2^30B) eam ======================================================================================================4000 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4001 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4002 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4003 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4004 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4005 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4006 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4007 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4008 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts


Complete the following steps:

1. Check the initial extent pool and volume status by running lsextpool -l and lsfbvol -l. There are nine volumes (4000 - 4008), each using the rotate extents EAM. The extent pool (P6) has two ranks and no free extents available.

2. Add an empty rank to the extent pool (P6). The lsrank -l command shows that a new empty rank has been added to the extent pool (Example 4-4).

Example 4-4 Add new rank to extent pool and check extent allocation across ranks before manual volume rebalance

dscli> chrank -extpool p6 r13CMUC00008I chrank: Rank R13 successfully modified.

dscli> lsrank -l -extpool p6ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts encryptgrp===============================================================================================R8 0 Normal Normal A10 5 P6 tier1 fb 900 900 -R10 0 Normal Normal A18 5 P6 tier1 fb 900 900 -R13 0 Normal Normal A21 5 P6 tier1 fb 900 0 -

3. Check that the extent pool (P6) is not managed by Easy Tier Auto Mode by running lsextpool -l, and start manual volume rebalance by running managefbvol -action migstart. Because a target extent pool is not specified on the command, the target is the same as the current extent pool. The lsfbvol -l command shows that the volumes’ configstate is Migrating, and the lsrank -l command shows that additional extents have been allocated on the ranks to facilitate the volume rebalance (Example 4-5).

Example 4-5 Manual volume rebalance action is started

dscli> lsextpool -l p6ID stgtype rankgrp status availstor (2^30B) %allocated available numvols numranks numtiers etmanaged=====================================================================================================P6 fb 0 below 900 67 900 9 3 1 no

dscli> managefbvol -action migstart 4000-4008CMUC00430I managefbvol: The migstart action for FB volume 4000 has completed.CMUC00430I managefbvol: The migstart action for FB volume 4001 has completed.CMUC00430I managefbvol: The migstart action for FB volume 4002 has completed.CMUC00430I managefbvol: The migstart action for FB volume 4003 has completed.CMUC00430I managefbvol: The migstart action for FB volume 4004 has completed.CMUC00430I managefbvol: The migstart action for FB volume 4005 has completed.CMUC00430I managefbvol: The migstart action for FB volume 4006 has completed.CMUC00430I managefbvol: The migstart action for FB volume 4007 has completed.CMUC00430I managefbvol: The migstart action for FB volume 4008 has completed.

dscli> lsfbvol -l -extpool p6 -config not_normalID accstate datastate configstate deviceMTM datatype extpool sam captype cap (2^30B) eam ======================================================================================================4000 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4001 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4002 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4003 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4004 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4005 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4006 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4007 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4008 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts

dscli> lsrank -l -extpool p6ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts encryptgrp


===============================================================================================R8 0 Normal Normal A10 5 P6 tier1 fb 900 894 -R10 0 Normal Normal A18 5 P6 tier1 fb 900 893 -R13 0 Normal Normal A21 5 P6 tier1 fb 900 14 -

4. Pause volume rebalance by running managefbvol -action migpause. The lsfbvol command shows that the volumes are in a Migration Paused state (Example 4-6).

Example 4-6 Manual volume rebalance action is paused

dscli> managefbvol -action migpause 4000-4008CMUC00430I managefbvol: The migpause action for FB volume 4000 has completed.CMUC00430I managefbvol: The migpause action for FB volume 4001 has completed.CMUC00430I managefbvol: The migpause action for FB volume 4002 has completed.CMUC00430I managefbvol: The migpause action for FB volume 4003 has completed.CMUC00430I managefbvol: The migpause action for FB volume 4004 has completed.CMUC00430I managefbvol: The migpause action for FB volume 4005 has completed.CMUC00430I managefbvol: The migpause action for FB volume 4006 has completed.CMUC00430I managefbvol: The migpause action for FB volume 4007 has completed.CMUC00430I managefbvol: The migpause action for FB volume 4008 has completed.

dscli> lsfbvol -l -extpool p6ID accstate datastate configstate deviceMTM datatype extpool sam captype cap (2^30B) eam ===========================================================================================================4000 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4001 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4002 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4003 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4004 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4005 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4006 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4007 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4008 Online Normal Migration Paused 2107-900 FB 512 P6 Standard DS 200.0 rotateexts

5. Resume volume rebalance by running managefbvol -action migresume. The lsfbvol -l command shows that the volumes’ configstate is Migrating (Example 4-7).

Example 4-7 Manual volume rebalance action is resumed

dscli> managefbvol -action migresume 4000-4008CMUC00430I managefbvol: The migresume action for FB volume 4000 has completed.CMUC00430I managefbvol: The migresume action for FB volume 4001 has completed.CMUC00430I managefbvol: The migresume action for FB volume 4002 has completed.CMUC00430I managefbvol: The migresume action for FB volume 4003 has completed.CMUC00430I managefbvol: The migresume action for FB volume 4004 has completed.CMUC00430I managefbvol: The migresume action for FB volume 4005 has completed.CMUC00430I managefbvol: The migresume action for FB volume 4006 has completed.CMUC00430I managefbvol: The migresume action for FB volume 4007 has completed.CMUC00430I managefbvol: The migresume action for FB volume 4008 has completed.

dscli> lsfbvol -l -extpool p6ID accstate datastate configstate deviceMTM datatype extpool sam captype cap (2^30B) eam ======================================================================================================4000 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4001 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4002 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4003 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4004 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4005 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4006 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4007 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts


4008 Online Normal Migrating 2107-900 FB 512 P6 Standard DS 200.0 rotateexts

6. The lsfbvol -l command shows that the configstate is Normal and no longer Migrating. The lsrank command shows that the extents now have been evenly allocated on all the ranks (Example 4-8).

Example 4-8 Extent pool capacity distribution after adding new capacity and initiating manual volume rebalance

dscli> lsfbvol -l -extpool p6ID accstate datastate configstate deviceMTM datatype extpool sam captype cap (2^30B) eam ======================================================================================================4000 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4001 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4002 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4003 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4004 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4005 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4006 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4007 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts4008 Online Normal Normal 2107-900 FB 512 P6 Standard DS 200.0 rotateexts

dscli> lsrank -l -extpool p6ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts encryptgrp===============================================================================================R8 0 Normal Normal A10 5 P6 tier1 fb 900 600 -R10 0 Normal Normal A18 5 P6 tier1 fb 900 600 -R13 0 Normal Normal A21 5 P6 tier1 fb 900 600 -

The showfbvol, showckdvol, showextpool, and showrank commands show volume migration and rank allocation-related information (Example 4-9).

Example 4-9 Volume status information related to volume migration

dscli> showfbvol -rank 4000Name mz4000ID 4000accstate Onlinedatastate Normalconfigstate NormaldeviceMTM 2107-900datatype FB 512addrgrp 4extpool P6exts 200captype DScap (2^30B) 200.0cap (10^9B) -cap (blocks) 419430400volgrp -ranks 3dbexts 0sam Standardrepcapalloc -eam rotateextsreqcap (blocks) 419430400realextents 200virtualextents 0migrating 0perfgrp PG0migratingfrom -


resgrp RG0==============Rank extents==============rank extents============R8 66R10 67R13 67

dscli> showextpool p0Name ITSO_FB_AID P0stgtype fbtotlstor (2^30B) 4746availstor (2^30B) 4701resvdstor (2^30B) 0rankgrp 0numranks 3numvols 2status below%allocated 0%available 99configured 4746allowed 4725available 4701allocated 24reserved 0%limit 100%threshold 15virextstatus full%virallocated 0%viravailable 0virconfigured 0virallowed 0viravailable 0virallocated 0virreserved 0%virextlimit -%virextthreshold -encryptgrp -%allocated(ese) 0%allocated(rep) 0%allocated(std) 1%allocated(over) 0%virallocated(ese) -%virallocated(tse) -%virallocated(init) -%migrating(in) 1%migrating(out) 1

dscli> showrank r2ID R2SN -Group 0State Normaldatastate NormalArray A2RAIDtype 5extpoolID P0extpoolnam ITSO_FB_Avolumes 1000,1001


stgtype fbexts 1582usedexts 8widearrays 0nararrays 1trksize 128strpsize 512strpesize 0extsize 16384encryptgrp -migrating(in) 5migrating(out) 0

4.5.2 Dynamic extent pool merge

Dynamic extent pool merge allows one extent pool (the source extent pool) to be merged into another extent pool (the target extent pool) transparently to the attached host systems. This capability is an Easy Tier Manual Mode capability.

The operation is performed by running chextpool. Here is the command syntax:

chextpool -merge <sourcepool_ID> -quiet <targetpool_ID>

Both source and target extent pools can contain regular or thin provisioned (ESE) logical volumes. An extent pool merge is not supported if both extent pools contain TSE volumes or if there are TSE volumes on the SSD ranks.

When the merge is complete, the ranks and volumes assigned to the source pool are reassigned to the target pool, and the source extent pool is removed. The optional -quiet parameter disables the confirmation prompt for the –merge parameter.

Example 4-10 shows a dynamic extent pool merge. The source extent pool P2 contains two ranks and one logical volume (1000). The target extent pool P0 contains three ranks and two logical volumes (1001-1002).

Example 4-10 Dynamic extent pool merging

dscli> lsextpoolName ID stgtype rankgrp status availstor (2^30B) %allocated available reserved numvols===========================================================================================ITSO_FB_A P0 fb 0 below 4716 0 4716 0 2ITSO_FB_B P2 fb 0 below 1546 0 1546 0 1

Pool merges: You cannot merge two extent pools:

� If both extent pools contain TSE volumes.� If there are TSE volumes on the SSD ranks.� If you have selected an extent pool that contains volumes that are being migrated.

Important: The extent pool merge is allowed only among extent pools with the same server affinity. If you try to merge two extent pools with various server affinities, you receive an error message (CMUG00408E) indicating that the two extent pools are not serviced by the same server. Similarly, you receive the same error message if you try to merge a CKD extent pool in an FB extent pool or vice versa. For further details about dynamic extent pool merge, see “Dynamic extent pool merge” on page 13.


dscli> lsrank -l -extpool p0ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=========================================================================================R2 0 Normal Normal A2 5 P0 ITSO_FB_A fb 1582 10R3 0 Normal Normal A3 5 P0 ITSO_FB_A fb 1582 10R4 0 Normal Normal A4 5 P0 ITSO_FB_A fb 1582 10

dscli> lsrank -l -extpool p2ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=========================================================================================R0 0 Normal Normal A0 5 P2 ITSO_FB_B fb 779 6R1 0 Normal Normal A1 5 P2 ITSO_FB_B fb 779 6

dscli> lsfbvol 1000-1002Name ID accstate datastate configstate deviceMTM datatype extpool cap (2^30B)=========================================================================================ITSO_J 1000 Online Normal Normal 2107-900 FB 512 P2 12.0ITSO_J 1001 Online Normal Normal 2107-900 FB 512 P0 15.0ITSO_J 1002 Online Normal Normal 2107-900 FB 512 P0 15.0

dscli> chextpool -merge p2 p0CMUC00422W chextpool: Are you sure that you want to merge source extent pool p2 into target extent pool p0? [y/n]: yCMUC00001I chextpool: Extent pool P0 successfully modified.

dscli> lsextpoolName ID stgtype rankgrp status availstor (2^30B) %allocated available reserved numvols===========================================================================================ITSO_FB_A P0 fb 0 below 6262 0 6262 0 3

dscli> lsrank -l -extpool p0ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts=========================================================================================R0 0 Normal Normal A0 5 P0 ITSO_FB_A fb 779 6R1 0 Normal Normal A1 5 P0 ITSO_FB_A fb 779 6R2 0 Normal Normal A2 5 P0 ITSO_FB_A fb 1582 10R3 0 Normal Normal A3 5 P0 ITSO_FB_A fb 1582 10R4 0 Normal Normal A4 5 P0 ITSO_FB_A fb 1582 10

dscli> lsfbvol 1000-1002Name ID accstate datastate configstate deviceMTM datatype extpool cap (2^30B)=========================================================================================ITSO_J 1000 Online Normal Normal 2107-900 FB 512 P0 12.0ITSO_J 1001 Online Normal Normal 2107-900 FB 512 P0 15.0ITSO_J 1002 Online Normal Normal 2107-900 FB 512 P0 15.0

Complete the following actions:

1. Check the initial status by running lsextpool, lsrank, and lsfbvol.

2. Perform extent pool merging by running chextpool -merge p2 p0.

3. After the merge has been completed, check the final status. The source extent pool P2 has been removed and all five ranks are now assigned to extent pool P0. Volume extents are on the same ranks as before.


4.5.3 Rank depopulation

Rank depopulation allows a user to unassign a rank from an extent pool, even if there are extents allocated on that rank. In that case, Easy Tier automatically reassigns all allocated extents to the other ranks within the same extent pool. This operation is not apparent to the host operations, and is available for all types of ranks.

The operation is performed by running chrank. Here is the command syntax:

chrank -unassigned -quiet <rank_ID>

Ranks to be depopulated can contain extents from regular and thin provisioned logical volumes, either TSE or ESE. When the depopulation is complete, these extents are migrated to the other ranks in the extent pool, and the source extents are removed. The optional -quiet parameter disables the confirmation prompt for the –unassign parameter.

Example 4-11 shows a rank depopulation operation. The hybrid extent pool P4 contains 12 ranks (including R18, which is an SSD rank) and several logical volumes, including 4008, which is spread on the non-SSD ranks in the pool.

Example 4-11 Rank and volume status before rank depopulation

dscli> lsfbvol -l -extpool p4ID accstate datastate configstate deviceMTM datatype extpool sam captype cap (2^30B) eam===================================================================================================1000 Online Normal Normal 2107-900 FB 512 P4 Standard DS 25.0 managed4000 Online Normal Normal 2107-900 FB 512 P4 Standard DS 10.0 managed4001 Online Normal Normal 2107-900 FB 512 P4 Standard DS 10.0 managed4002 Online Normal Normal 2107-900 FB 512 P4 Standard DS 100.0 managed4003 Online Normal Normal 2107-900 FB 512 P4 Standard DS 50.0 managed4004 Online Normal Normal 2107-900 FB 512 P4 Standard DS 80.0 managed4005 Online Normal Normal 2107-900 FB 512 P4 Standard DS 150.0 managed4006 Online Normal Normal 2107-900 FB 512 P4 Standard DS 20.0 managed4007 Online Normal Normal 2107-900 FB 512 P4 Standard DS 25.0 managed4008 Online Normal Normal 2107-900 FB 512 P4 Standard DS 2400.0 managed

dscli> showfbvol -rank 4008 Name -ID 4008accstate Onlinedatastate Normal

Merging extent pools: After merging extent pools, you might want to redistribute volume extents on the merged extent pool by using the manual volume rebalance function, as described in “Manual volume rebalance” on page 103. This situation is especially true if the extents are unevenly distributed on the ranks. There must be enough free space on the ranks.

Extent allocation: If no extents are allocated on the specified rank in the extent pool, the rank is directly unassigned from the pool.

Attention: A rank depopulation operation is not allowed if there are not enough extents available on the remaining ranks of the extent pool. However, it is supported with regular volumes and with ESE and TSE thin provisioned volumes in the pool. See “Rank depopulation” on page 18 for more details.


configstate NormaldeviceMTM 2107-900datatype FB 512addrgrp 4extpool P4exts 2400captype DScap (2^30B) 2400.0cap (10^9B) -cap (blocks) 5033164800volgrp V1ranks 12dbexts 0sam Standardrepcapalloc -eam managedreqcap (blocks) 5033164800realextents 2400virtualextents 0migrating 0perfgrp PG0migratingfrom -resgrp RG0==============Rank extents==============rank extents============R7 200R9 200R10 200R19 200R20 200R21 200R22 200R23 200R24 200R25 200R26 200R27 200

dscli> lsrank -l -extpool p4 ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts====================================================================================R7 0 Normal Normal A13 5 P4 ITSO_FB_P4 fb 384 207R9 0 Normal Normal A15 5 P4 ITSO_FB_P4 fb 448 207R10 0 Normal Normal A16 6 P4 ITSO_FB_P4 fb 376 206R18 0 Normal Normal A3 5 P4 ITSO_FB_P4 fb 1574 192R19 0 Normal Normal A17 5 P4 ITSO_FB_P4 fb 448 361R20 0 Normal Normal A18 5 P4 ITSO_FB_P4 fb 448 349R21 0 Normal Normal A19 5 P4 ITSO_FB_P4 fb 448 205R22 0 Normal Normal A10 6 P4 ITSO_FB_P4 fb 312 206R23 0 Normal Normal A11 6 P4 ITSO_FB_P4 fb 312 209R24 0 Normal Normal A12 6 P4 ITSO_FB_P4 fb 312 208R25 0 Normal Normal A21 5 P4 ITSO_FB_P4 fb 448 207R26 0 Normal Normal A26 10 P4 ITSO_FB_P4 fb 255 206R27 0 Normal Normal A27 10 P4 ITSO_FB_P4 fb 255 207



1. Check the initial status by running lsrank, lsfbvol, and showfbvol. Volume 4008 has 200 extents allocated on rank R27.

2. Perform the rank depopulation on rank R27 by running chrank -quiet -unassign r27 (Example 4-12).

Example 4-12 Use the chrank command to initiate rank depopulation

dscli> chrank -quiet -unassign r27 CMUC00008I chrank: Rank R27 successfully modified.

3. kCheck the rank status after the depopulation has started (Example 4-13). The used extents on all remaining ranks have increased immediately after running chrank (caused by Easy Tier pre-allocation process). Only the number of used extents on rank R27 has decreased.

Example 4-13 Checking the rank status after rank depopulation has started

dscli> lsrank -l -extpool p4 ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts ===========================================================================================R7 0 Normal Normal A13 5 P4 ITSO_FB_P4 fb 384 226 R9 0 Normal Normal A15 5 P4 ITSO_FB_P4 fb 448 226 R10 0 Normal Normal A16 6 P4 ITSO_FB_P4 fb 376 225 R18 0 Normal Normal A3 5 P4 ITSO_FB_P4 fb 1574 192 R19 0 Normal Normal A17 5 P4 ITSO_FB_P4 fb 448 379 R20 0 Normal Normal A18 5 P4 ITSO_FB_P4 fb 448 367 R21 0 Normal Normal A19 5 P4 ITSO_FB_P4 fb 448 224 R22 0 Normal Normal A10 6 P4 ITSO_FB_P4 fb 312 225 R23 0 Normal Normal A11 6 P4 ITSO_FB_P4 fb 312 228 R24 0 Normal Normal A12 6 P4 ITSO_FB_P4 fb 312 227 R25 0 Normal Normal A21 5 P4 ITSO_FB_P4 fb 448 226 R26 0 Normal Normal A26 10 P4 ITSO_FB_P4 fb 255 225 R27 0 Depopulating Normal A27 10 P4 ITSO_FB_P4 fb 255 205

4. Check the rank status by running lsrank (Example 4-14). The used extents on all remaining ranks have not changed, and the number of used extents on rank R27 is still decreasing.

Example 4-14 Checking the rank status while rank depopulation is running

dscli> lsrank -l -extpool p4 ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts ==========================================================================================R7 0 Normal Normal A13 5 P4 ITSO_FB_P4 fb 384 230 R9 0 Normal Normal A15 5 P4 ITSO_FB_P4 fb 448 231 R10 0 Normal Normal A16 6 P4 ITSO_FB_P4 fb 376 230 R18 0 Normal Normal A3 5 P4 ITSO_FB_P4 fb 1574 192 R19 0 Normal Normal A17 5 P4 ITSO_FB_P4 fb 448 384 R20 0 Normal Normal A18 5 P4 ITSO_FB_P4 fb 448 373

Note: The number of used extents on the SSD rank R18 has not changed after running chrank -unassign because, in a hybrid pool, extent allocation always allocates extents on the ranks of the Enterprise class tier first until there is no space left. If no space is left on the Enterprise tier, then extent allocation continues on the ranks of the Nearline tier and finally on the SSD tier. See 3.3.2, “Guidelines for creating multitier extent pools” on page 53 for more details. Extent allocation automatically is balanced in a rotate extents like fashion across the ranks within a storage tier. See 2.5.9, “Extent allocation in hybrid and managed extent pools” on page 30 for more information.


R21 0 Normal Normal A19 5 P4 ITSO_FB_P4 fb 448 230 R22 0 Normal Normal A10 6 P4 ITSO_FB_P4 fb 312 230 R23 0 Normal Normal A11 6 P4 ITSO_FB_P4 fb 312 232 R24 0 Normal Normal A12 6 P4 ITSO_FB_P4 fb 312 231 R25 0 Normal Normal A21 5 P4 ITSO_FB_P4 fb 448 229 R26 0 Normal Normal A26 10 P4 ITSO_FB_P4 fb 255 228 R27 0 Depopulating Normal A27 10 P4 ITSO_FB_P4 fb 255 185

5. Check the status of volume 4008 by running showfbvol -rank (Example 4-15). The rank depopulation already has started to migrate some of the volume extents on R27 to the other ranks.

Example 4-15 Checking the status of volume 4008 while rank depopulation is running

dscli> showfbvol -rank 4008 Name -ID 4008accstate Onlinedatastate Normalconfigstate NormaldeviceMTM 2107-900datatype FB 512addrgrp 4extpool P4exts 2400captype DScap (2^30B) 2400.0cap (10^9B) -cap (blocks) 5033164800volgrp V1ranks 12dbexts 0sam Standardrepcapalloc -eam managedreqcap (blocks) 5033164800realextents 2400virtualextents 0migrating 0perfgrp PG0migratingfrom -resgrp RG0==============Rank extents==============rank extents============R7 202R9 202R10 202R19 202R20 202R21 202R22 201R23 201R24 202R25 202


R26 202R27 180

6. After some time, we check the extent pool P4 rank status by running lsrank -l (Example 4-16). Rank R27 is no longer part of the extent pool. Running lsrank -l r27 shows that the rank is now unassigned, which means that the depopulation process has ended.

Example 4-16 After rank depopulation rank is removed from the extent pool

dscli> lsrank -l -extpool p4 ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts ===========================================================================================R7 0 Normal Normal A13 5 P4 ITSO_FB_P4 fb 384 230 R9 0 Normal Normal A15 5 P4 ITSO_FB_P4 fb 448 231 R10 0 Normal Normal A16 6 P4 ITSO_FB_P4 fb 376 230 R18 0 Normal Normal A3 5 P4 ITSO_FB_P4 fb 1574 192 R19 0 Normal Normal A17 5 P4 ITSO_FB_P4 fb 448 384 R26 0 Normal Normal A26 10 P4 ITSO_FB_P4 fb 255 228 R22 0 Normal Normal A10 6 P4 ITSO_FB_P4 fb 312 230 R23 0 Normal Normal A11 6 P4 ITSO_FB_P4 fb 312 232 R24 0 Normal Normal A12 6 P4 ITSO_FB_P4 fb 312 231 R25 0 Normal Normal A21 5 P4 ITSO_FB_P4 fb 448 229 R26 0 Normal Normal A26 10 P4 ITSO_FB_P4 fb 255 228

7. Check the status of volume 4008 by running showfbvol -rank (Example 4-17). The extents that were depopulated from rank R27 have been migrated in a balanced manner across the other ranks of the Enterprise tier in the extent pool.

Example 4-17 Checking the volume status after rank

dscli> showfbvol -rank 4008 Name -ID 4008accstate Onlinedatastate Normalconfigstate NormaldeviceMTM 2107-900datatype FB 512addrgrp 4extpool P4exts 2400captype DScap (2^30B) 2400.0cap (10^9B) -cap (blocks) 5033164800volgrp V1ranks 11dbexts 0sam Standardrepcapalloc -eam managedreqcap (blocks) 5033164800realextents 2400virtualextents 0migrating 0perfgrp PG0migratingfrom -resgrp RG0


==============Rank extents==============rank extents============R7 218R9 218R10 218R19 218R20 218R21 218R22 218R23 218R24 218R25 219R26 219


Easy Tier Automatic Mode manages the volume capacity within managed extent pools to place the extents on the most appropriate storage tiers and ranks. When Easy Tier Automatic Mode is enabled, Easy Tier automatically handles data placement to optimize performance and storage economics. See Chapter 2, “IBM System Storage DS8000 Easy Tier concepts, design, and implementation” on page 7 for more details about the Easy Tier concepts and Easy Tier Automatic Mode.

Run chsi to turn on or off Easy Tier Automatic Mode and to start or stop Easy Tier monitoring. The command syntax is:

chsi -ETAutoMode tiered|all|none -ETMonitor automode|all|none storage_image_ID

The -ETAutoMode parameter specifies which volumes are managed by Easy Tier Automatic Mode. If the value is set to tiered and Easy Tier Monitoring is enabled (automode or all), then only the volumes in hybrid (multitier) extent pool are managed by Easy Tier Automatic Mode. If the value is set to all and Easy Tier Monitoring is enabled, then all volumes in both, hybrid (multitier) and homogenous (single-tier) pools are managed by Easy Tier Automatic Mode. If the value is set to none, no volumes are managed by Easy Tier Automatic Mode. You can still relocate data manually on a volume level using Easy Tier Manual Mode features, as long as the Easy Tier LIC feature is installed.

The -ETMonitor parameter specifies which volumes are monitored by the Easy Tier monitoring process. If the value is set to all, then all volumes on the DS8000 system are monitored regardless of the Easy Tier LIC feature being installed or not. If the value is set to automode and the Easy Tier LIC feature is installed, then only those volumes in managed extent pools that are managed by Easy Tier Automatic Mode are monitored. If the value is set to automode and the Easy Tier LIC feature is not installed, then no volumes are monitored. If the value is set to none, then no volumes are monitored on the system. Monitoring can be set to none even when the -ETAutoMode switch is set to tiered or all. See Table 2-3 on page 33 for more information.

Note: The rank depopulation is a low priority background process, which is optimized to avoid affecting performance on the extent pool. For that reason, the time window necessary for the rank depopulation is not precisely predictable.


Example 4-18 shows the enablement of Easy Tier Automatic Mode for hybrid or multitier extent pools and sets monitoring to on for all volumes. Run showsi command to view Easy Tier Automatic Mode information.

Example 4-18 Using DS CLI to enable Easy Tier Automatic Mode

dscli> chsi -ETAutoMode tiered -ETMonitor all IBM.2107-xxxCMUC00042I chsi: Storage image IBM.2107-xxx successfully modified.

dscli> showsi IBM.2107-xxxName ATS_04desc DS8000-R6ID IBM.2107-xxxStorage Unit IBM.2107-xxxModel 951WWNN 50050763xxxxC29FSignature 637b-583b-xxxx-xxxxState OnlineESSNet EnabledVolume Group V0os400Serial 29FNVS Memory 8.0 GBCache Memory 235.8 GBProcessor Memory 253.4 GBMTS IBM.2421-xxxnumegsupported 0ETAutoMode tieredETMonitor allIOPMmode Managed

Starting with DS8000 R6.2 LMC, the lsextpool -l, and showextpool commands show information about the number of tiers in an extent pool and if the pool is currently managed by Easy Tier Automatic Mode (Example 4-19). In this example, we have a DS8000 configuration with the ETAutoMode control switch set to tiered, and we can see that only hybrid, multitier extent pools are actively managed by Easy Tier Automatic Mode. All homogeneous single-tier extent pools (numtiers=1 or numtiers=0) are not managed by Easy Tier, which means that no automatic intra-tier performance management (auto-rebalance) is active in these single-tier extent pools.

Example 4-19 Easy Tier status and number of tiers information

dscli> lsextpool -lName ID stgtype rankgrp [...] numvols numranks encryptgrp numtiers etmanaged====================================================================================ckd_hybrid P0 ckd 0 2688 2 - 2 yes

Important: Do not turn off Easy Tier monitoring if Easy Tier Automatic Mode (ETAutoMode) is enabled. When Easy Tier monitoring is turned off, no new performance statistics are collected and Easy Tier Automatic Mode cannot create migration plans. As a result, Easy Tier Automatic Mode stops managing volumes in all managed extent pools.

Attention: Changing the Easy Tier monitoring mode affects the statistics collection and can lead to a reset (reinitialization) of the gathered monitoring data. This situation means that it might take up to 24 hours of collecting new performance statistics after Easy Tier monitoring has been enabled again until new migration plans are created. Easy Tier Automatic Mode can continue to automatically optimize storage performance and economics by relocating extents across and within storage tiers.


ckd_600_1 P1 ckd 1 2690 1 - 1 notst_62esp P2 fb 0 6 3 - 2 yespool_1tier P3 fb 1 0 0 - 0 nopool_3tier P4 fb 0 44 4 - 3 yes

dscli> showextpool p4Name pool_3tierID P4stgtype fb[...]numtiers 3etmanaged yes

dscli> showsi IBM.2107-75TV181Name ATS_04[...]ETAutoMode tieredETMonitor all

4.5.5 Exporting the Easy Tier Summary Report (etdata)

Example 4-20 shows how Easy Tier monitoring data, which is also referred to as the Easy Tier Summary Report, can be downloaded from the DS8000 system to your workstation by running offloadfile.

Example 4-20 Downloading the Easy Tier monitoring data

dscli> offloadfile -etdata C:\tempCMUC00428I offloadfile: The etdata file has been offloaded to C:\temp\SFxxxESS01_heat.data.CMUC00428I offloadfile: The etdata file has been offloaded to C:\temp\SFxxxESS11_heat.data.

See Chapter 5, “Using IBM System Storage DS8000 Easy Tier” on page 119 for instructions about how to post-process the downloaded monitor files with the (STAT) to obtain system utilization and volume heat map reports.


Chapter 5. Using IBM System Storage DS8000 Easy Tier

In this chapter, we provide usage considerations about how to use Easy Tier and how to interpret the data from the Storage Tier Advisor Tool (STAT). We also show how valuable the actual Easy Tier feature can be from an operational and usability viewpoint.

Easy Tier can help the storage administrator configure and allocate the storage capacity for optimal cost-to-performance ratio based on the performance data collected by the Easy Tier monitoring task and processed with the STAT.

5


5.1 Easy Tier usage

As its name already implies, Easy Tier was designed to provide an easy solution for managing IBM System Storage DS8000 back-end storage for optimal usage of available storage tiers and resources within a tier.

The best approach often is to use Easy Tier in Automatic Mode. In this mode, Easy Tier automatically and transparently relocates hot extents in hybrid extent pools to SSDs or Enterprise drives. Within one tier, Easy Tier can rebalance individual overloaded ranks. You do not need to make complex decisions or LUN configurations. You can leave all the complex work to the system, which can perform self-tuning within a given hardware configuration. When configuration or workload changes occur, the system automatically adjusts over time.

In Manual Mode, Easy Tier also offers significant advantages. It allows you to relocate volumes, merge extent pools, rebalance ranks on request, and depopulate ranks, in a way that is not apparent to host applications. Thanks to the statistical usage data collected and the STAT, which interprets the collected data, you get a clear understanding of what application data can benefit the most from being relocated to SSDs, Enterprise (SAS/FC) drives, or Nearline drives.

5.2 Obtaining and installing the Storage Tier Advisor Tool

To process the data gathered by the collection task in the DS8000 system, you need to download and install the STAT software.

After verifying your code bundle (run ver -l to view the firmware level and LMC), download the correct software package directly from the following IBM website:

ftp://ftp.software.ibm.com/storage/ds8000/updates/DS8K_Customer_Download_Files/Storage_Tier_Advisor_Tool/

For the IBM System Storage DS8800, you can also go to this alternative website:

http://www.ibm.com/support/docview.wss?uid=ssg1S4000982

For the IBM System Storage DS8700, you can also go to this alternative website:



ftp://ftp.software.ibm.com/storage/ds8000/updates/DS8K_Customer_Download_Files/Storage_Tier_Advisor_Tool/



The DS8000 STAT software is available for download as an ISO file. Using tools such as WinRAR or VCdControlTool.exe from Microsoft lets you directly initiate the software installation from the INSTALL.EXE file in the ISO file (Figure 5-1). On Windows, the software is in C:\Program Files\IBM\STAT by default.

Figure 5-1 STAT installation process

5.3 Collecting DS8000 Easy Tier volume heat data

Easy Tier performance data collection is available on DS8800 systems (with LMC level 7.6.10.xxx/R6.1 or higher) and DS8700 systems (with LMC level 6.5.1.xxx/R5.1 or higher), whether the Easy Tier LIC feature is applied.

The switch for the Easy Tier data collection (ETmonitor in DS CLI) is by default set to automode. In this mode, only performance statistics for managed volumes under control of Easy Tier Automatic Mode in managed extent pools are collected. Managed pools can either be multitier (hybrid) or single-tier (homogeneous) extent pools depending of the Easy Tier ETAutoMode setting. Single-tier pools can only be managed by Easy Tier starting with DS8000 R6.2 LMC level or higher with ETAutoMode set to all. On systems without Easy Tier automatic management enabled or without the Easy Tier feature installed, no volumes are monitored and no performance statistics (Easy Tier Summary Report) are collected that can be interpreted using the STAT.

The current scope of Easy Tier monitoring data collection task can be queried by running showsi (Example 5-1).

Example 5-1 Running showsi to query the Easy Tier monitoring mode

dscli> showsi ibm.2107-75tv181Name ATS_04desc DS8000-R6ID IBM.2107-75TV181Storage Unit IBM.2107-75TV180Model 951WWNN 500507630AFFC38FSignature 617b-573b-92f4-5caaState OnlineESSNet Enabled

Chapter 5. Using IBM System Storage DS8000 Easy Tier 121

Volume Group V0os400Serial 29FNVS Memory 8.0 GBCache Memory 235.8 GBProcessor Memory 253.4 GBMTS IBM.2421-75TV180numegsupported 0ETAutoMode tieredETMonitor automodeIOPMmode Managed

The second to last line in Example 5-1 on page 121 shows a current setting and scope for the Easy Tier monitoring task (ETMonitor). As stated before, ETMonitor set to automode indicates that only volumes that are in a managed extent pool under control of Easy Tier Automatic Mode are currently monitored.

The IOPMmode Managed attribute only refers to the I/O Priority Manager, which can coexist with Easy Tier. I/O Priority Manager and Easy Tier both provide independent benefits. For detailed information regarding the I/O Priority Manager, see DS8000 I/O Priority Manager, REDP-4760.

ETAutoMode set to tiered indicates that Easy Tier provides automated storage performance and economics management with extent inter-tier and intra-tier relocations only in hybrid or multitier extent pools. This setup creates the appropriate extent migration plans (see 2.5.4, “Migration plan creation” on page 25).

A DS8000 system without any hybrid pools, only a homogeneous drive configuration, or with multiple tiers installed in separate, dedicated single-tier extent pools, does not collect any data when ETMonitor control is set to automode. You must change the data collection scope to all if you want to collect data for all volumes. Process the information provided in the Easy Tier Summary Report (etdata) using the STAT to obtain guidelines about how adding SSDs or other storage tiers might contribute to storage performance and storage economics enhancements, for example, on a system that has only Enterprise drives installed.

Example 5-3 on page 123 show how to change the Easy Tier monitoring task by running chsi, which samples performance data for all volumes on a DS8000 system in managed and non-managed pools.

Example 5-2 Using the chsi command to enable Easy Tier performance monitoring on all volumes

dscli> showsi ibm.2107-75tv181...ETAutoMode tieredETMonitor automodeIOPMmode Managed

dscli> chsi -etmonitor all ibm.2107-75tv181CMUC00042I chsi: Storage image IBM.2107-75TV181 successfully modified.

dscli> showsi ibm.2107-75tv181...ETAutoMode tieredETMonitor allIOPMmode Managed


To enable Easy Tier Automatic Mode management for homogeneous, single-tier pools to use Easy Tier Automatic Mode intra-tier management (auto-rebalance), which reduces skew and avoids single rank hotspots, set ETAutoMode to all (Example 5-3).

Example 5-3 Using the chsi command to enable Easy Tier performance management for all pools

dscli> showsi ibm.2107-75tv181...ETAutoMode tieredETMonitor allIOPMmode Managed

dscli> chsi -etautomode all ibm.2107-75tv181CMUC00042I chsi: Storage image IBM.2107-75TV181 successfully modified.

dscli> showsi ibm.2107-75tv181...ETAutoMode allETMonitor allIOPMmode Managed

5.4 Offloading the Easy Tier Summary Report

You can offload the collected Easy Tier performance data from the DS8000 system at any time. See 2.5.4, “Migration plan creation” on page 25 to learn when to offload the Easy Tier Summary Report from the DS8000 system.

The data that you can download in the Easy Tier Summary Report is a summary of the overall data collection. This information is sufficient to understand the current I/O activity at the DS8000 back-end storage and provide appropriate volume heat information and configuration recommendations. Both DS8000 systems collect more detailed statistics internally that are only available to IBM support for debugging purposes and IBM-assisted investigations or in-depth evaluations. This level of detailed back-end performance data is not available to the user.

Example 5-4 shows the help text and parameters of the offloadfile command. The following parameters are required:

� -etdata, which refers to the Easy Tier Summary Report.

� A directory on your local PC where the files with the summary data from each of the two DS8000 systems or processor complexes (CPCs) are stored.

Example 5-4 DS CLI offloadfile command to download the Easy Tier Summary Report (etdata)

dscli> help offloadfileoffloadfile

The offloadfile command offloads the specified set of data files. This command is not supported on DS6000 models.

>>- offloadfile--+---------------------------+---- -etdata-----> '- -dev-- storage_image_ID-'>--+---------+-- directory------------------------------------->< '- -quiet-'


Parameters

-dev storage_image_ID (Optional) Specifies the storage image ID, which includes manufacturer, machine type, and serial number. The storage image ID is required if you do not specify a fully qualified LSS ID, do not set the devid variable in your profile or through the setenv command, and the HMC is aware of more than one storage image. Using the -dev parameter will temporarily override any defined value for devid for the current command.

-etdata (Required) Offloads two files containing the IBM System Storage Easy Tier summary data.

-quiet (Optional) Specifies that the system turns off the confirmation prompt before overwriting existing files.

directory (Required) Specifies the local directory path that is used as the destination for offloading files.

Example

An invocation example

dscli>offloadfile -dev IBM.2107-75FA120 -etdata C:\temp

The resulting output

Sun Apr 09 02:23:49 PST 2004 IBM DS CLI

Offloadfile: The etdata file has been offloaded to c:\temp\SF1300860ESS01_heat.data.Offloadfile: The etdata file has been offloaded to c:\temp\SF1300860ESS11_heat.data.

Offloading the data is also possible by using the DS8000 GUI, as explained in 4.3.5, “Exporting the Easy Tier Summary Report” on page 97.

Example 5-5 shows how to offload the Easy Tier Summary Report using the DS CLI.

Example 5-5 Offloading the Easy Tier Summary Report (etdata) using the DS CLI

dscli> offloadfile -etdata c:\tempCMUC00428I offloadfile: The etdata file has been offloaded to c:\temp\SF75TV180ESS01_heat.data.CMUC00428I offloadfile: The etdata file has been offloaded to c:\temp\SF75TV180ESS11_heat.data.

The offloadfile command creates two output files, one file for each DS8000 system (CPC). In normal operation, there is an affinity between the system’s back end and its CPCs. The even-numbered extent pools are related to server 0 (first complex), and the odd-numbered extent pools are related to server 1 (second complex). The performance summary files are collected for each DS8000 system.


The file name structure of the output files follows a strict naming scheme. Within the directory specified, two files are created (see Example 5-5 on page 124) with the following file name structure:

� SF indicates Storage Facility.

� Next is the device ID (serial number or Storage Facility), which is 75TV180 in Example 5-5 on page 124. This ID is not the Storage Facility Image ID (75TV181 in Example 5-5 on page 124).

� Both processor complexes or storage servers are referred to as ESS01 and ESS11.

� The last file name qualifier is fixed and constant (_heat.data).

5.5 Processing the Easy Tier Summary Report with the Storage Tier Advisor Tool

As described in 5.2, “Obtaining and installing the Storage Tier Advisor Tool” on page 120, you installed the STAT software in a certain directory. Switch to this directory and familiarize yourself with the syntax of the stat command. Example 5-6 shows the command syntax and the required parameters of the stat.

Example 5-6 Usage of the stat.exe executable program from the Storage Tier Advisor Tool

C:\Program Files\IBM\STAT>stat -hLicensed Materials - Property of IBM

2105/2107 5639-VC6 5639-VM1

(C) Copyright IBM Corp. 2010, 2011 All Rights Reserved

US Government Users Restricted Rights - Use, duplication ordisclosure restricted by GSA ADP Schedule Contract withIBM Corp.

STAT.exe (Version 8.1.0.0)

The STAT.exe command creates an HTML report of the IO distribution on DS8000, SAN Volume Controller or Storwize V7000 disk drives.

STAT.exe [-h] [-o output_path] input_filename

Parameters -h (Optional) Displays the help information. You cannot use this parameter with any other parameters.

-o output_path (Optional) Specifies the output directory for the report. If not specified the output goes to the directory of the executable.

input_filename (Required) Specifies the name of the DS8000, SAN Volume Controller or Storwize V7000 input file(s).

DS8000 input files are extracted using the "Export Easy Tier Summary Report" action on a storage image.SAN Volume Controller and Storwize V7000 input files are found under /dumps on the configuration node and are named "dpa_heat.node_name.time_stamp.data". The file must be offloaded manually using the CLI or GUI.

Important: These summary files can only be interpreted by the STAT.


Example

>STAT.exe inputfile1 inputfile2

In the output directory, open index.html.

The STAT requires two input files for DS8000 systems. With only a single file, the stat command returns an error message indicating that two input files are needed (Example 5-7).

Example 5-7 The Storage Tier Advisor Tool requires input files from both storage servers on a DS8000 system

STAT>stat.exe SF75TV180ESS01_heat.dataCMUA00016E The STAT.exe command has failed because the storage image is operating in dual cluster mode but you have specified only the single raw data file "SF75TV180ESS01_heat.data".

5.5.1 Starting the Storage Tier Advisor Tool

The input file names must follow a strict naming structure and need to contain _heat in the file name. If _heat is missing, then the command fails (Example 5-8).

Example 5-8 The stat command with improperly renamed input files

STAT>stat.exe SF75TV180ESS01heat.data SF75TV180ESS11heat.dataCMUA00001E The STAT.exe command cannot be initiated because the input filename "SF75TV180ESS01heat.data" that you have specified does not have a valid input filename format for the STAT.exe command.

In many cases, it might be useful to collect Easy Tier Summary Reports automatically at regular intervals and rename the data files to add a time stamp by adding a date and time (“20111201-1600”) in front of the file name. This file name format is accepted by the stat command as long as the _heat extension remains unchanged as part of the input file name(Example 5-9).

Example 5-9 The stat command with properly renamed input files

STAT>stat.exe 20111201-1600_SF75TV180ESS01_heat.data 20111201-1600_SF75TV180ESS11_heat.dataCMUA00019I The STAT.exe command has completed.

5.5.2 Storage Tier Advisor Tool output files

The STAT output is placed into the directory specified by the -o parameter. This directory is created automatically. Example 5-10 shows the execution of the stat command when specifying a target directory for output files, which is named “report” in this example. The output files located in this directory contain an index.html file and a data directory named Data_files, with additional data files containing the specific performance and volume heat information. If you omit the -o option and do not specify a target directory, then these files are created in the current directory.

Example 5-10 Output files of the stat command

STAT>stat -o report SF75TV180ESS01_heat.data SF75TV180ESS11_heat.dataCMUA00019I The STAT.exe command has completed.

STAT>dir reportDirectory of C:\Program Files\IBM\STAT62\report


14.12.2011 15:51 <DIR> Data_files29.09.2011 16:13 1 244 index.html

To view the Easy Tier Summary Report after the input files have been processed with the STAT, open the index.html file.

5.6 Interpreting the Storage Tier Advisor Tool output

The output of the STAT is based on data collected by the Easy Tier monitoring function A migration plan is only created after a data collection period of at least 24 hours. Heat maps and migration plans are reported for DS8000 systems when all volumes are monitored (Easy Tier Monitor Mode set to all) or when only volumes in hybrid extent pools are monitored (Easy Tier Monitor Mode set to automode).

When you open the index.html file, the System Summary window of the STSAT output opens. Figure 5-2 shows an example of a System Summary window where data has been offloaded from a DS8800 system. Easy Tier collects only statistics data for DS8000 back-end storage I/Os and its focus is on random (or small) I/Os.

Figure 5-2 Storage Tier Advisor Tool - System Summary window


The System Summary report contains the data that the Easy Tier monitor previously collected. This window is composed of three main parts:

� A general pane on the left side that contains two quick links. You can use these links to either return to this summary window from any other window, or to jump to the Systemwide Recommendation window, as shown in Figure 5-3.

Figure 5-3 Storage Tier Advisor Tool - Systemwide Recommendation window

� A summary table containing the following data:

– The total number of monitored pools.– The total number of monitored volumes.– The total capacity of monitored volumes.– The hot data capacity, shown both as extents number and percentage of total extents.


� A storage pool table that shows, for each monitored extent pool, the following data:

– The storage pool ID. Selecting one of those IDs opens the Storage Pool Performance Statistics and Improvement Recommendation window for the corresponding extent pool, as shown in Figure 5-4.

Figure 5-4 Storage Tier Advisor Tool: Storage Pool Statistics and Recommendations – Existing Tier Status

– The total capacity of the extent pool.

– The configuration of the pool. Depending on whether the pool is a hybrid pool on not, it shows one or two of the following options:

• SSD• Enterprise• NL (for: Nearline)

– The tier status, which potentially indicates whether a tier in the extent pool has skewed workload, and whether any rank is overloaded either in terms of IOPS or bandwidth.

Additionally, under the title, you can see two dates. The start date is the latest date when Easy Tier started to monitor the workload. It might be longer in the past than the long-term monitoring window, which is taken into account for the current migration plan. The second date refers to the time the last data collection was taken into account for the sliding short-term monitoring window. That date corresponds to the last migration plan generation, if it exists, and is at most 24 hours from the Easy Tier data offload.


Figure 5-3 on page 128 shows the Systemwide Recommendation window, which is opened by selecting the Systemwide Recommendation link in the left pane. In this example, using the statistics data offloaded from a DS8800 system, there are three different levels of systemwide recommendations that can appear on this window:

� Recommended SSD configuration: Shows a list of extent pools that can take advantage of promoting extents on either existing or additional SSD ranks, and the estimated system performance improvement that results from this relocation. We can see, for example, that the system performance can gain up to 10% performance improvement by adding one rank with 300-GB SSD disks to the extent pool with ID 0002. This performance improvement is based on 24 hours worth of activity and that improvement can be higher and lower in different time periods

� Recommended Enterprise configuration: Shows a list of extent pools that can take advantage of moving extents on either existing or additional Enterprise ranks, and the predicted IOPS improvement that results from this relocation. The example here shows that the extent pool ID 0002 can gain up to 24% IOPS improvement, by adding two additional 146-GB/15-K Enterprise ranks. In Figure 5-2 on page 127, we see that this pool is a hybrid extent pool, composed of Enterprise and Nearline ranks. We conclude that some hot extents on Nearline ranks might gain an advantage by moving to Enterprise ranks, but that there is enough capacity on those ranks to cover the relocation.

� Recommended NL (Nearline) configuration: Shows a list of extent pools that can take advantage of demoting extents to either existing or additional Nearline (7200 min–1) ranks, and the cold data capacity resulting from this cold demotion.

If the systemwide recommendation is about adding a rank, rank specifications, including disks size, RAID level, and characteristics, are also shown. In Figure 5-3 on page 128, for the extent pool ID 0002 of the Recommended SSD configuration table, the item “Performance Improved by Adding 1 STEC_300G_R5_W6 Rank(s)” indicates ranks with 300-GB SSD disks, RAID 5, and Width 6 (meaning 6 data disks+Parity+Spare, or 6+P+S) configuration. For the extent pool ID 0001 in the Recommended NL configuration table, the item “Adding 1 7.2K_NL_3T_R6_W6 Rank(s)” indicates a rank with 3-TB Nearline disks, RAID 6, and Width 6 (6+P+P) configuration.

The IDs in the storage Pool ID column are selectable, and clicking one of them shows the detailed statistics and systemwide recommendations for the corresponding extent pool.

Important: Predicted performance improvements are related only to back-end disk service times and do not indicate an overall storage system performance improvement.

Tips:

� Depending on workload during the monitoring phase, the Systemwide Recommendation window might only show a subset of these categories, or even none at all, which means that the workload on the DS8000 system during this period was perfectly balanced.

� You must select the Recommended... line to expand the display and show the corresponding table.

Migration cost: All storage pool IDs are selectable in this report, including those pools on which the system has not made any systemwide recommendations. You might even see some hot or warm extents for those last ones, which means that the Easy Tier algorithm has decided that the migration cost for those extents was too high compared to the benefit.


Figure 5-4 on page 129 shows an example of the window displayed when selecting pool ID 0004 the window shown in Figure 5-3 on page 128, and then selecting Existing Tier Status to expand the corresponding table.

The Existing Tier Status table shows the following characteristics for each rank in the selected extent pool:

� The rank ID.

� The number of IOPS thresholds exceeded, which represents the number of cycles since the last decision window where the rank IOPS exceeded the threshold IOPS specified for the rank’s device type.

� The usage of the rank IOPS, which represents, in the shape of three possible colored bars (blue, orange, and red), the current percentage of the maximum allowed IOPS threshold for the rank’s device type. The blue portion represents the percentage of IOPS below the tier average usage of the rank’s IOPS, the orange portion the percentage between the average and the maximum allowed IOPS for the rank’s IOPS, and the red portion the percentage above the maximum allowed IOPS for the rank’s IOPS.

� The projected usage of the rank IOPS, which represents the expected percentage of the maximum allowed IOPS threshold for the rank’s device type, after the current migration plan has been applied. The color code is the same as for the previous bullet. It is expected to show an improvement compared to the current usage, or at least the same percentage level.

In the example shown in Figure 5-4 on page 129, we can see that two Enterprise ranks in this pool have a usage below critical levels and are already balanced, so the projected usage after further rebalancing (right side) is identical to the current IOPS usage of these ranks.

Selecting Recommended SSD/Enterprise/NL Configuration expands the table containing the list of recommended SSD rank usage or add-ons, and shows the fields shown in Figure 5-5:

� The storage pool ID.

� The recommended configuration change for the specified type of rank, and the expected result. As with the main summary report, the characteristics of the rank are shown (disk capacity, RAID level, and width).

� The predicted pool performance improvement percentage compared to the previous configuration.

� The predicted system performance improvement percentage compared to the previous configuration (as shown in Figure 5-3 on page 128).

� The estimated migration time range, either in the case of using the existing SSD disks in the pool (within the current migration plan), or after adding SSD capacity to the extent pool.

� For Nearline: The cold data capacity that can be expected to be used on the proposed configuration.

� The predicted pool capacity increase after the potential add-on (no value if the systemwide recommendation was to use existing ranks in the tier).

Figure 5-5 Storage Tier Advisor Tool - Storage Pool Statistics and Recommendations – SSD Configuration


The drop-down menu (in the left pane of Figure 5-5 on page 131) allows you to change the display to another rank configuration, if any is proposed for that selected tier. Figure 5-6 shows the expanded menu.

Figure 5-6 Storage Tier Advisor Tool: Storage Pool Statistics and Recommendations – drop-down Enterprise menu

Three-tier configuration: If the Systemwide Recommendation window leads to a three-tier configuration for a specific extent pool and you are not able to apply all of it, you must make a choice, which might be either performance (SSD) or budget (HDD) related.


Clicking Volume Heat Distribution opens the heat distribution table and show, for each volume in the corresponding extent pool, the fields shown in Figure 5-7:

� The volume ID.

� The volume configured capacity.

� The three tiers (SSD, Enterprise, and NL (Nearline)), with the extent capacities already allocated on the respective tiers. This value is 0G if there is no extent of a tier allocated to that specific volume.

� The heat distribution of the volume, visible through color coded table cells.

� Depending on the number of volumes in the extent pool, the display is divided into pages, between which you can navigate by clicking < and > on the line below the heat map. It is also possible to change the number of displayed volumes, or to enter a page number and click the GO button to jump to that page.

Figure 5-7 Storage Tier Advisor Tool: Storage Pool Statistics and Recommendations – Volume Heat Distribution

In the heat distribution column, red colored parts indicate hot extents. When the entire cell is red, all extents of the corresponding volume are considered to be hot. Hot extents that are not already on the higher tier ranks are prioritized for promotion. The orange section indicates warm data, which corresponds to data that will be promoted to the higher tier if it is currently on a lower tier, after the hot data has been promoted and capacity is available. Cooler warm data may also be demoted with the extended cold demotion.

The blue section indicates extents that are considered to be cold and currently not candidates to be moved to the higher tier’s ranks, but might be moved on Nearline ranks (cold demotion), if applicable.


The DS8000 system works internally with a higher granularity for its Easy Tier heat buckets, but externally only these three categories, hot, warm, and cold, are shown.

5.6.1 Storage Tier Advisor Tool examples

The following examples are solely intended to show how the STAT can help you better understand the DS8000 configuration.

System with SSDs and further improvement with ongoing migration to SSD

Figure 5-8 shows a workload that has only recently changed and increased, with Easy Tier needing a few more hours to move extents to the existing one SSD rank, which brings a considerable performance improvement. Adding further SSDs is not listed as an option for this system and point, and thus not needed in this specific example.

Figure 5-8 Storage Pool Statistics and Recommendations – SSD Configuration

System with skew and additional need for SSDs for certain poolsThis example shows a DS8800 system with two SSD ranks and some pools containing Nearline ranks. Figure 5-9 shows the System Summary window.

Figure 5-9 System Summary window

You can see in Figure 5-9 that two pools have skew on the Enterprise ranks. Some ranks in the pools and within the Enterprise tier are much more loaded than other pools, and Easy Tier has not yet finished data relocation to rebalance the work evenly across the ranks of the same tier (auto-rebalance).

Mode setting: To obtain performance or capacity improvement, as predicted by the STAT tool, Easy Tier Automatic Mode must be set to tiered or all.


Figure 5-10 shows the other view on the system level, which is the Systemwide Recommendation.

Figure 5-10 Systemwide Recommendation showing possible improvements for all pools

We have two storage pools, 0000 and 0008 (extent pools P0 and P8), whose workload patterns would benefit from adding SSDs. The expected performance gain (up to 19% improvement for P0) might seem small at first, but when you view the projections for the respective pools, you see that the expected improvements can be much bigger for the workloads in this pool (Figure 5-11).

Figure 5-11 Storage Pool 0000 in need of SSDs (skewed)

You see that while the overall system-wide performance might only increase by up to 19%, when you add that one SSD rank, the Predicted Pool Performance Improvement for the workloads in this pool P0 is expected to be in the range of 66% to 86%. This change is a considerable performance improvement for the workloads in this pool when adding SSDs.


Note two other aspects:

� Pool with skew:

As noted by the orange area for Rank ID 5, some of the ranks are more loaded than others in this pool. You currently see 9% above average utilization of that rank. Easy Tier has not yet reached full steady state, and tries to balance this situation using auto-rebalance. It is a single-tier homogeneous pool (Enterprise HDDs only).

After Easy Tier migration occurs and your systems reaches a steady state, as shown in the right column labeled Projected Utilization of Rank IOPS, you see that this skew or imbalance decrease to a projected 4% above average utilization only, even when not adding any further SSDs. You see that Easy Tier has not eliminated the skew completely after reaching steady state, and that can be for two reasons:

– A slight imbalance is accepted by Easy Tier, if it is not harming the overall performance.

– The algorithm avoids toggling. If a small hotspot moves with the extent when migrated and leads to a similar imbalance on the receiving rank, it is not moved.

� Pool can benefit from adding Nearline drives:

As stated earlier, Enterprise pool P0 can benefit from adding one SSD rank. However, at the same time, the STAT recognizes that a major part of the workload on this pool is cold data. 2657 GiB of the pool’s 5008 GiB are cold. For such data, especially for amounts in terabytes, it is beneficial cost-wise to introduce Nearline drives and let the Easy Tier algorithm demote the cold data onto Nearline drives. One Nearline rank is enough given the capacity here, and might save costs by regaining space on the Enterprise ranks.

In general, if no better information is available, start with an Enterprise tier and a minimal SSD tier configuration (for example, only one SSD rank per pool) for a new installation. Let Easy Tier and STAT analyze the system. Then use large-capacity Nearline drives for capacity upgrades, if STAT detects major parts of the workloads as cold.

Similar considerations apply to Enterprise class pools without any SSD tiers where large-capacity Nearline drives can be considered for capacity upgrades, if STAT detects major parts of the workloads as cold. So you can maintain Enterprise class performance while shrinking footprint and reducing costs by combining Enterprise-class drives with large capacity nearline drives.

System with workload and only minor benefits from SSDsSSDs may be more beneficial for some workloads than for others. SSDs are especially good for small-block random and cache-unfriendly workloads, while for large-block sequential workloads, HDDs provide good performance results. The Easy Tier algorithm carefully analyzes all workload profiles for their random / sequential ratios, read and write ratios, block sizes, and back-end response time behaviors. Easy Tier then sorts them into the relevant heat classes. Certain workloads might benefit more from adding further HDDs than adding SSDs.


Figure 5-12 shows the P1 storage pool. This pool already has one SSD rank, and adding another SSD rank should provide 22% additional performance. This performance gain is small in comparison to the values in Figure 5-11 on page 135. If, however, you add four more additional HDD ranks as a RAID 10 configuration, the STAT tool projects a performance improvement of up to 75%. Given the cost ratios of SSDs versus HDDs for a given capacity, you should add a few HDD ranks to increase performance.

Figure 5-12 DS8700 showing major benefit from adding further Enterprise ranks

System with an overloaded SSD rankThis example shows the STAT results for a DS8800 system which Storage Pool 0004 contains one SSD rank that is bandwidth (BW) overloaded. There are huge large-block workloads taking place that put a high strain on the SSD rank, compromising its performance.


Figure 5-13 shows a bandwidth utilization of currently 97% for this one existing SSD rank. You not only get the predicted performance by adding another SSD rank, but you can also get an increase in performance up to 65 %. You also get a prediction that by adding one more SSD rank, the utilization level will likely fall to 50%, and by adding two more SSD ranks, the utilization per rank is expected to decrease to 30%.

Figure 5-13 DS8800 showing a pool with an SSD rank that is bandwidth-overloaded

5.7 Easy Tier Manual Mode usage scenarios

In this section, we primarily look into dynamic extent pool merge and dynamic volume migration (DVR), which is the manual approach to consolidating extent pools and manage volume placement in extent pools. Easy Tier Manual Mode migration is initiated at the volume level, not the extent level. Subvolume management on the extent level is only a capability of Easy Tier Automatic Mode.

Easy Tier Manual Mode provides dynamic volume relocation, dynamic extent pool merge, and rank depopulation capabilities. Easy Tier Manual Mode capabilities are user initiated.

These three features are called dynamic in the sense that the host applications are not aware of what is happening in the DS8000 back-end storage. Host applications and I/O do not need to be suspended during a dynamic extent pool merge or a dynamic volume relocation or volume migration. With the enhanced Easy Tier on R6.1 (LMC level 7.6.10.xxx), you are able to merge extent pools containing enterprise drives with extent pools containing Nearline drives. Starting with R6.2 (LMC level 7.6.20.xxx), extent pools containing repositories for thin provisioned volumes (ESE) can be merged.

A typical use case for dynamic volume relocation or volume migration is when a configured DS8000 system is already actively used in production and additional new capacity (disk space) is required. You can add new ranks to an existing pool or you can merge extent pools with unused ranks. After merging pools or adding new ranks to a pool, you should evenly redistribute the extents of the volumes across the ranks, especially in non-managed single-tier extent pools. You could accomplish this task by striping the capacity of the volumes across all ranks (storage pool striping) in the pool.


Keep in mind that dynamic volume relocation within the same pool is not allowed in any hybrid or managed extent pools. In managed pools, new capacity is automatically allocated by Easy Tier Automatic Mode when managing data relocation across tiers or rebalancing the workload across all ranks within a storage tier (auto-rebalance). Starting with DS8000 R6.2 (LMC level 7.6.20.xxx), auto-rebalance is also available for homogeneous single-tier extent pools and is the more convenient and preferred way to initiate a rebalancing after adding capacity. This rebalance is actually base on rank performance and not a simple capacity redistribution. It can be achieved by setting the ETAutoMode to all, that is, enabling the Automatic Mode for the single-tier homogeneous extent pool. This action performs the additional task to initiate manual restriping and achieves a balanced workload distribution across ranks based on actual performance aspects, not on capacity.

If the DS8000 system is not yet on DS8000 R6.2 LMC or you do not want to enable the Automatic Mode on single-tier pools, you can use the manual volume rebalance feature. This feature is available with DS8000 R6.1 LMC or higher, and is based on the dynamic volume relocation capability of Easy Tier Manual Mode.

5.7.1 Volume migration across extent pools

The major capability that comes with Easy Tier Manual Mode is the dynamic volume relocation feature, which allows the migration of volumes across extent pools transparent to host workloads. This capability provides you with ease of use and more flexibility for storage management on a DS8000 system. It also allows you to manually manage performance across extent pools by moving volumes from highly used extent pools to less used extent pools regarding capacity or performance needs.

Even in configurations with different storage tiers that have dedicated single-tier extent pools for each tier, and where Easy Tier Automatic Mode is not deployed or enabled, the dynamic volume relocation feature allows you to manually manage volumes and tiers by manually moving volumes to the appropriate storage tier represented by a given extent pool. Although automatic management with Easy Tier is encouraged, the dynamic volume relocation feature might be appropriate for environments where the administrator needs to determine and guarantee a certain storage tier for a given volume.

See “Dynamic volume relocation” on page 15 for more details and “Volume migration” on page 101 for an example.

Volume migrations can only take place between extent pools of the same rank group and managed by the same DS8000 system. It is not supported for migrating a volume across different rank groups, for example, migrating a volume from extent pool P0 to extent pool P1.

Tip: Easy Tier Automatic Mode and Manual Mode are not exclusive, meaning that you can use most Manual Mode capabilities even if Automatic Mode is active.

Tip: If you have Easy Tier automatically use added capacity (ranks) or react to workload profile changes in single-tier pools, in DS8000 R6.2 LMC or higher, set the ETAutoMode to all instead of using manual volume rebalance. In this case, Easy Tier and its auto-rebalance feature takes advantage of new capacity by managing data placement on extent level based on actual performance needs.


5.7.2 Consolidation of extent pools

Assume that we have three extent pools called P1, P7, and P9, with just one rank per extent pool, where:

� Extent pool P1 has one SSD rank. � Extent pools P7 and P9 contain only one HDD rank.

The following steps can be done using either the DS CLI or DS GUI to consolidate these pools.

Figure 5-14 shows the step-by-step approach to consolidate these three extent pools into one extent pool that contains all three ranks.

Figure 5-14 Consolidating extent pools

The target extent pool will then be a hybrid pool, which is managed by Easy Tier Automatic Mode.

Only extent pools of the same storage type (either FB or CKD) associated with the same rank group or managed by the same storage server can be merged. In our example, all extent pool have odd extent pool numbers (P1, P7, and P9) and are associated with rank group 1 and managed by storage server 1.

P7

P9

P9

P1

P1

1

2

SSD

HDD HDD

Target configuration

R9 R10

R6

R9

R10

R6



1. Run lsextpool to list the extent pools and show their characteristics (Figure 5-15). The pools (P1, P7, and P9) that will be merged are highlighted. These extent pools are allocated between 84% and 99%.

Figure 5-15 Initial status as shown with lsextpool command before consolidating three extent pools

2. Run lsckdvol list the volumes in extent pool P1 (Figure 5-16). The output shows eight volumes with 8 × 54 extents for a total of 432 extents in this SSD-based extent pool P1. The device IDs are 3100 – 3107.

Figure 5-16 Extent pool consolidation - Extent pool P1

Figure 5-17 shows 16 volumes with 54 extents each in P7, which amounts to 864 extents. Volume or device IDs are 3300 – 330F, all on rank R9.

Figure 5-17 Extent pool consolidation - Extent pool P7


Figure 5-18 shows extent pool P9 with the same number of volumes and used extents as extent pool P7, with device IDs 3500 – 350F.

Figure 5-18 Extent pool consolidation - Extent Pool P9

All three extent pools hold 2160 used extents, with 40 volumes in total.

Consolidating extent pools is a quick exercise and does not cause any real data movement (except for a few metadata and control structures).

3. Run chextpool to merge extent pools P7 and P9 (Figure 5-19). P7 is merged into P9. You can view the resulting P9 pool by running lsextpool. The P9 extent pool contains two ranks.

Figure 5-19 Extent pool consolidation - Merge extent pool P7 into extent pool P9

4. Run chextpool to merge extent pool P9 into extent pool P1 (Figure 5-20). Another list command now shows three ranks within extent pool P1.

Figure 5-20 Extent pool consolidation - Merge extent pool P9 into extent pool P1

Important: No actual data movement is performed during a dynamic extent pool merge; only logical definition updates occur.


5. Run lsextpool and then chextpool to alter the name of the extent pool P1 and reflect that it is now a hybrid extent pool (Figure 5-21).

Figure 5-21 Extent pool consolidation: Status after extent pool merge

No extents have been moved and each rank still holds the same volumes and extents as it did before the merge (Figure 5-22).

Figure 5-22 Extent pool consolidation - New extent pool P1 with two storage tiers

Easy Tier Automatic Mode, once enabled, now starts relocating the extents based on their workload pattern to the most appropriate tier (cross-tier performance management) and rank (intra-tier performance management using auto-rebalance).

Figure 5-23 shows that all 40 volumes are now in the newly created multitier extent pool, P1:

� Addresses 31xx are in R6.� Addresses 33xx are in R9.� Addresses 35xx are in R10.

Figure 5-23 Extent pool consolidation - All volumes are now in the new multitier extent pool P1


5.7.3 Restriping of volumes in non-managed extent pools

Right after consolidating multiple extent pools into one homogeneous and single-rank extent pool, the capacity distribution of all volumes is highly unbalanced across the ranks, because no volumes were moved during the extent pool merge. All volumes and their extents are still on the ranks they have initially been created on.

The same situation applies if capacity or new ranks have been added to an existing single-tier extent pool. All existing volumes on the previous ranks leave the new capacity unused.

Restriping the volumes evenly across all ranks in the pool improves performance and reduces workload skew and rank hot spots.

If the new single-tier pool is not managed by Easy Tier Automatic Mode, then manual volume rebalance provides a way to restripe the volumes across all ranks in the non-managed extent pool. Manual volume rebalance is only available in non-managed, single-tier (homogeneous) extent pools and allows a balanced redistribution of the extents of a volume across all ranks in the pool. This feature is not available in managed or hybrid pools. The reason is that Easy Tier Automatic Mode manages the placement of the extents based on their actual workload pattern and appropriate storage tier and rank utilizations.

With Easy Tier Automatic Mode enabled for single-tier extent pools, for DS8000 R6.2 LMC or higher, you can benefit from Easy Tier automated intra-tier performance management (auto-rebalance), which relocates extents based on rank utilization, and reduces skew and avoids rank hot spots. Easy Tier relocates subvolume data on extent level based on actual workload pattern and rank utilization rather than balance the capacity of a volume across all ranks in the pool.

If the single-tier extent pool is not managed and the volume distribution is unbalanced, for example, after an extent pool consolidation using extent pool merge, use manual volume rebalance to rebalance the capacity of all volumes evenly across all ranks in the pool. See “Manual volume rebalance” on page 16 for more details and “Manual volume rebalance” on page 103 for examples using DS CLI.

Manual volume rebalance was introduced with DS8000 R6.1 LMC as an enhancement of the dynamic volume relocation feature. This feature needs a little free movement space in the extent pool to achieve a balanced redistribution of the volumes across all ranks in the non-managed pool. The feature only allocates one extent at a time instead of pre-allocating the full capacity of a volume.

5.7.4 Removing ranks from extent pools

Easy Tier Manual Mode also provides the capability to remove a rank with allocated capacity from a given extent pool. This capability can be used, for example, to move a rank from a less used extent pool to a higher used extent pool regarding performance or capacity. It can also be used to change an initial configuration to build new hybrid pools.

To remove a rank on a DS8000 system with microcode R6.1 and higher with the Easy Tier feature enabled, perform an unassign of the particular rank, for example, by running chrank -unassign. Easy Tier automatically depopulates all extents from the rank and moves them to the remaining ranks in the pool. See “Rank depopulation” on page 18 or more details and 4.5.3, “Rank depopulation” on page 110 for an example.


Appendix A. IBM System Storage DS8000 Easy Tier and IBM z/OS system-managed storage

In this appendix, we compare system-managed storage (SMS) in z/OS configurations with the Easy Tier Automatic Data Relocation (ADR). This brief review of system-managed storage shows how Easy Tier, with its ability to dynamically relocate extents, can complement, or even be a substitute, for SMS.

A


z/OS system-managed storage

The concept of storage tiering and managing various storage technologies in an automated fashion was implemented in IBM MVS™ / DFP V3.0. It was introduced to the marketplace in 1989 as system-managed storage (SMS) and data facility storage management subsystem (DFSMS), which consists of the following components:

� MVS/DFP: Software kernel of SMS with volume selection besides “I/O drivers” providing data and device management.

� DFSMShsm: Hierarchical storage manager.

� DFSMSdss: Highly tuned data set services to copy and move data from any device to any other device.

� DFSMSrmm: Removable media manager that keeps track where all the files are on tape.

DFSMS is an part of z/OS.

SMS provides support for any level of tiered storage, including disk, tape, and optical storage systems.

By using Automatic Class Selection routines (ACS routines), SMS automatically assigns service level constructs to newly allocated data sets. These constructs imply certain service level requirements. Storage Class constructs are used to indicate service levels in terms of required response times or bandwidth requirements. Based on these Storage Class attributes, SMS selects, by using the Storage Group ACS routine, which storage tier best matches the service level requirements and selects appropriate volumes.

Figure A-1 shows the concept of SMS with its ACS routines to assign service level-based constructs to a newly created file, data set, or object.

Figure A-1 System-managed storage in z/OS - assign performance service level to new file

PROC STORCLAS / * SC ACS r out i ne change hi st or y * /

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .FI LTLI ST TEMPDS I NCLUDE( * * . * TEMP* . * * , SORTWK* , * * . SPFLOG* . * * ) FI LTLI ST I XGLOGR I NCLUDE( I XGLOGR. * * ) FI LTLI ST DB2 I NCLUDE( DSN%. * * ) FI LTLI ST I MPORTANT I NCLUDE( SYS%. * * , TCPI P. * * ,

DSN510. * * , DSNCAT. * * , CATALOG. * * ). . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SELECT WHEN ( &DSTYPE = ' TEMP' ) SET &STORCLAS = ' XLARGE' WHEN ( &DSN = &I XGLOGR OR &DSN = &DB2)

SET &STORCLAS = ' AVERAGE'WHEN ( &DSN = &I MPORTANT) SET &STORCLAS = ' CRI T'

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .OTHERWI SE SET &STORCLAS = ' AVERAGE'

END END

ACS RoutineStorage Class

Storage GroupACS Routine

Management ClassACS Routine

Data ClassACS Routine

Storage hierarchy (tiers)

SSD

FCP

SATA


Particular data sets, such as system data sets or IBM DB2® data sets that are critical to performance, are automatically considered for placement on the best performing devices available in the configuration.

Figure A-2 shows a fragment of a Storage Group ACS routine that maps the service level requirements, expressed through Storage Classes, to the existing storage subsystems and technology within the configuration.

Figure A-2 Map performance service level requirements to I/O environment

The actual volume selection is a sophisticated process in DFSMSdfp and takes storage technologies into consideration to map the file to the correct storage tier.

For our discussion, granularity is at the level of a z/OS data set or file.

A data set can be as small as a track (56 KB) or in the range of terabytes. Volumes and technology are selected for the entire data set. Placing a large data set on to expensive high performance devices can impact many smaller data sets because the high performance space is taken. Additionally, DFSMS does not generally automatically move data sets through the storage hierarchy when the need for high performance changes over time.

Easy Tier storage management

Compared to SMS, the advantage of Easy Tier is the finer granularity (one extent) that it uses to manage the storage tiers within the DS8700 system. Also, Easy Tier can dynamically manage the I/O activity and adapt data placement over time.

PROC STORGRP /* SG ACS Routine change history */......................... SELECTWHEN (&STORCLAS = 'CRIT') SET &STORGRP = 'SSD' WHEN (&STORCLAS = 'AVERAGE') SET &STORGRP = 'FCP'WHEN (&STORCLAS = 'XLARGE' OR 'BATCH') SET &STORGRP = 'SATA'......................... .OTHERWISE SET &STORGRP = 'SATA','FCP'

ENDEND

SSD

FCP

SATA

SMS Storage Groups

Storage hierarchy Storage tiers

=

Appendix A. IBM System Storage DS8000 Easy Tier and IBM z/OS system-managed storage 147

There might be still a need to guarantee that certain data must be on the most responsive device, no matter how active (or inactive) this data set is. It might be feasible to dedicate an SSD rank in an isolated extent pool and deliberately create only volumes that exclusively hold these data sets and no other data. However, this method is a manual one and implies intensive and time consuming system management tasks, which can only be done by highly skilled experts. From this perspective, Easy Tier is certainly a better approach.

Hybrid pools in z/OS system-managed storage environments

It remains possible in a z/OS configuration to combine the SMS approach with its storage groups and the Easy Tier capabilities (Figure A-3).

Figure A-3 Include new storage group in system-managed configurations

Isolate SSD-based logical volumes for particularly demanding service levels for specific data and applications in the configuration. SMS strictly controls which applications can have data placed in this expensive storage group. The next storage group, HYBRID, combines DS8700 HDD and SSD ranks. Within that storage group, Easy Tier can come in to play to relocate data at an extent level to SSD or HDD. In the SMS HYBRID storage group, you can combine all hybrid extent pools (even from multiple DS8700 systems).

The next level in this storage hierarchy might be a storage group named FCP, which contains all the logical volumes that are in extent pools with FCP-based HDDs for all of your DS8700 systems. Finally, the SATA Storage Group is composed of all the extent pools with SATA technology or lower spinning but large HDDs.

Storage hierarchy (tiers)

SSD

SATA

FCP

HYBRID


Appendix B. IBM System Storage DS8000 storage virtualization

In this appendix, we describe the internal virtualization concepts as they apply to the IBM System Storage DS8000 series. This material is included for the reader’s convenience. For a detailed description, see Chapter 5, “Virtualization concepts”, in IBM System Storage DS8000: Architecture and Implementation, SG24-8886.

We describe the following DS8000 storage virtualization topics:

� Array sites� Arrays� Ranks� Extent pools� Logical volumes

B


The abstraction layers for disk virtualization

In this chapter, when talking about virtualization, we mean the process of preparing several physical disk drive modules (DDMs) to become an entity that can be used by an operating system.

The way DDMs are physically installed slightly differs between the IBM System Storage DS8700 and IBM System Storage DS8800 systems:

� The DS8700 system is populated with switched Fibre Channel arbitrated loop (FC-AL) disk drives that are mounted in 16 DDM slots disk enclosures. You can order disk drives in groups of 8 or 16 drives of the same capacity and revolutions per minute (rpm). The options for eight-drive sets only apply for the 600-GB solid-state drives (SSDs).

� The DS8800 system uses a switched point to point topology using serial attached SCSI (SAS) disk drives that are mounted in 24 DDM slots disk enclosures. Disk drives can be ordered in groups of 8 or 16 drives of the same capacity and rpm. The option for eight-drive sets only apply for the 300-GB solid-state drives (SSDs).

The disk drives can be accessed by a pair of device adapters. This physical layer serves as the base for virtualization.

Array site

An array site is the first layer in the DS8000 logical volume virtualization stack and consists of a group of eight DDMs. Which DDMs are forming an array site is predetermined automatically by the DS8000 system. The DDMs in an array site are of the same DDM type, which means the same capacity and the same speed (rpm).

Array

An array is created from one array site. Forming an array means defining a specific RAID type. The supported RAID types are RAID 5, RAID 6, and RAID 10. For each array site, you can select a RAID type, but solid-state drives can only be configured as RAID 5 and RAID 10 (RPQ only), and Serial Advanced Technology Attachment (SATA) drives can only be configured as RAID 6 or RAID 10.

Arrays: In the DS8000 implementation, one array is defined using one array site.


Figure B-1 shows the creation of a RAID 5 array with one spare, also called a 6+P+S array (capacity of six DDMs for data, capacity of one DDM for parity, and a spare drive). According to the RAID 5 rules, parity is distributed across all seven drives in this example.

Figure B-1 Creation of an array

On the right side in Figure B-1, the terms D1, D2, D3, and so on, stand for the set of data contained on one disk within a stripe on the array. If, for example, 1 GB of data is written, it is distributed across all the disks of the array.

Ranks

In the DS8000 virtualization hierarchy, the next logical construct is called a rank. When defining a new rank, its name is chosen by the DS Storage Manager, for example, R1, R2, R3, and so on. You must add an array to a rank.

Array Site

RAIDArray

Spare

Data

Data

Data

Data

Data

Data

Parity

Creation ofan array

D1 D7 D13 ...

D2 D8 D14 ...

D3 D9 D15 ...

D4 D10 D16 ...

D5 D11 P ...

D6 P D17 ...

P D12 D18 ...

Spare

Ranks: In the DS8000 implementation, a rank is built using one array.

Appendix B. IBM System Storage DS8000 storage virtualization 151

The available space on each rank is divided into extents. The extents are the building blocks of the logical volumes. An extent is striped across all disks of an array (Figure B-1 on page 151) and indicated by the small squares in Figure B-2.

Figure B-2 shows an example of an array that is formatted for fixed block (FB) data with 1-GB extents (the squares in the rank indicate that the extent is composed of several blocks from various DDMs).

Figure B-2 Forming an FB rank with 1-GB extents

Extent pools

An extent pool is a logical construct to combine the extents from a set of ranks, forming a domain for extent allocation to a logical volume. One or more ranks with the same extent type (FB or CKD) can be assigned to an extent pool. One rank can be assigned to only one extent pool.

There can be as many extent pools as there are ranks.

FB Rankof 1GBextents

Creation of

a Rank

D1 D7 D13 ...

D2 D8 D14 ...

D3 D9 D15 ...

D4 D10 D16 ...

D5 D11 P ...

D6 P D17 ...

P D12 D18 ...

RAIDArray

Spare

Data

Data

Data

Data

Data

Data

Parity

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

1GB 1GB 1GB 1GB

Important: Do not intermix ranks with various drive classes, RAID types, or disk RPMs in an extent pool, unless you want to enable the Easy Tier Automatic Mode facility.


Here are the main considerations regarding how many ranks can be added in an extent pool:

� Storage pool striping allows you to create logical volumes striped across multiple ranks. This setup typically enhances performance. To benefit from storage pool striping, more than one rank in an extent pool is required.

� Storage pool striping can enhance performance a great deal. However, hen you lose one rank (in the unlikely event that a whole RAID array failed due to a scenario with multiple failures at the same time), you lose all the data of this rank and all data in this extent pool. This loss occurs because data is striped across all ranks. To avoid data loss, mirror your data to a remote DS8000.

Dynamic extent pool merging, rank depopulationDynamic extent pool merging, and rank depopulation are capabilities provided as part of the Easy Tier Manual Mode facility. The Easy Tier facility is enabled by LIC feature 0713 and 7083. These topics are further described in 2.4.2, “Easy Tier Manual Mode” on page 13.

Automatic Data Relocation (ADR), auto rebalance, warm demotion, and hot demotionADR, auto rebalance, warm demotion, and hot demotion are features provided as part of the Easy Tier Automatic Mode facility. The Easy Tier facility is enabled by LIC feature 0713 and 7083. Those topics are further described in 2.5, “Easy Tier design” on page 21

Logical volumes

A logical volume is composed of a set of extents from one extent pool.

A logical volume cannot span multiple extent pools, but it can have extents from various ranks within the same extent pool. You can create FB volumes (also called LUNs) from FB extent pools, or CKD volumes from CKD extent pools.

Important: For best performance, balance the capacity between the two DS8000 systems. Create at least two extent pools, with one per server.


The allocation process for FB volumes is shown in Figure B-3.

Figure B-3 Creation of an FB LUN

The DS CLI refers to the servers as “rank groups,” where rank group 0 means server0 and rank group 1 means server1.

Space-efficient volumes

When a regular (fully provisioned) LUN is created, it allocates extents on the physical drive to match the capacity defined. The DS8000 system has two types of space-efficient volumes that can be created:

� Extent space-efficient volumes (ESE; only FB)� Track space-efficient volumes (TSE; both FB and CKD)

A space-efficient volume does not occupy physical capacity when it is created. Space is allocated when data is written to the volume. The amount of space that is physically allocated depends on the amount of data written to the volume, or changes performed on the volume. The sum of the capacities of all defined space efficient volumes can be larger than the physical capacity available. This feature is called over-provisioning or thin provisioning.

Space efficient volumes, like standard volumes (which are fully provisioned), can be mapped to hosts. Extent space efficient volumes are supported in combination with the FlashCopy function.

Track space efficient volumes are supported as IBM FlashCopy target volumes only.

Important: All ranks and volumes that are assigned to even extent pools (P0, P2, and P4) are managed by server0. All ranks and volumes assigned to odd extent pools (P1, P3, and P5) are managed by server1.

1 GB 1 GB 1 GB 1 GBfree

Extent Pool FBprod

Rank-a

Rank-b

3 GB LUN

1 GBfree

1 GBfree

used used Allocate a 3 GB LUN

1 GB 1 GB 1 GB 1 GBused

Extent Pool FBprod

Rank-a

Rank-b

3 GB LUN

1 GBused

1 GBused

used used

Logical 3 GB LUN

2.9 GB LUNcreated

100 MB unused


For additional information about this topic, see IBM System Storage DS8000: Architecture and Implementation, SG24-8886.

Allocation, deletion, and modification of LUNs or CKD volumes

All extents from the ranks assigned to an extent pool are independently available for allocation to logical volumes. The extents for a LUN / volume are logically ordered, but they do not have to come from one rank and the extents do not have to be contiguous on a rank.

Using fixed extents to form a logical volume in the DS8000 allows for flexibility in the management of the logical volumes. You can delete LUNs or CKD volumes, resize LUNs or volumes, and reuse the extents of those LUNs to create other LUNs or volumes, of various sizes. One logical volume can be removed without affecting the other logical volumes defined on the same extent pool.

The extents are cleaned after a LUN or CKD volume has been deleted, so it might take time for the extents to be available for reallocation. The reformatting of the extents is a background process.

There are two extent allocation methods (EAMs) for the DS8000 system:

� Rotate volumes allocation method:

Extents can be allocated sequentially. All extents are taken from the same rank until there are enough extents for the requested volume size or the rank is full, in which case the allocation continues with the next rank in the extent pool.

If more than one volume is created in one operation, the allocation for each volume starts in another rank. When allocating several volumes, we rotate through the ranks.

� Storage pool striping - rotate extents allocation method:

The second and preferred storage allocation method is storage pool striping. This method allocates capacity to a logical volume, spreading the capacity over the set of ranks in the logical volumes extent pool. This method provides improved performance by distributing volumes over all ranks evenly, avoiding hot spots that have been caused by workload skew between logical volumes.

With the rotate extents EAM, the first rank in the list is randomly picked every time the storage subsystem starts. The DS8000 system tracks the rank in which the last allocation started. The allocation of a first extent for the next volume starts from the next rank in that sequence. The next extent for that volume is taken from the next rank in sequence, and so on. So, the system rotates the extents across the ranks.

Important: Starting with microcode R6.2, ESE volumes are fully supported by Easy Tier. The TSE volumes are not supported by Easy Tier, with the only exception being the rank depopulation capability.


Figure B-4 shows an example of how volumes are allocated within the extent pool.

Figure B-4 Extent allocation methods

Dynamic volume expansionThe size of a LUN or CKD volume can be expanded without deleting the data. On the DS8000 system, additional extents are added to the volume. The operating system must support volume resizing.

If the volume was created using the storage pool striping option, then the extents that are used to increase the size of the volume are striped across the ranks of its extent pool. If a volume was created without striping, using the rotate volume option, the system tries to allocate the additional extents within the same rank that the volume is already in.

Due to the risk of data corruption, you cannot shrink a volume. The DS8000 configuration interfaces DS CLI and DS GUI do not allow you to change a volume to a smaller size.

Tip: Rotate extents and rotate volume EAMs provide distribution of volumes over ranks. Rotate extents does this task at a granular level, which is the preferred method to minimize hot spots and improve overall performance.

Allocation: In a Easy Tier managed extent pool, the storage pool striping EAM is used independently of the requested EAM and the volume’s EAM attribute is set to managed.

Tip: When configuring a new DS8000 system, do not mix, in the same extent pool, volumes using the storage pool striping method (rotate extents) and volumes using the rotate volumes method.

Attention: Before you can expand a volume, you must delete any Copy Services relationship involving that volume.

ExtentPool

Ranks

6

1

2

4

5

3

7

8…….12

13

16

15

Where to start with the firstvolume is determined at power on (say, R2)Striped volume with twoExtents created

Next striped volume (five extents in this example)starts at next rank (R3) from which the previous volume was started

Non-striped volume createdStarts at next rank (R1), going in a round-robin

Striped volume createdStarts at next rank (R2)(extents 12 to 15)

Extent

6

8 9 10

11

13

14

13

R1

R2

R3

12

15


Dynamic volume relocation (DVR)DVR is a feature that allows users to nondisruptively migrate volumes between extent pools, and is provided as part of the Easy Tier Manual Mode facility. This topic is further described in 2.4.2, “Easy Tier Manual Mode” on page 13.

Managed extents policyStarting with Licensed Machine Code level 7.6.1.xx.xx, a volume in a Easy Tier managed extent pool sees its EAM configuration state automatically changed to “managed”, whatever the original EAM was. This feature is further described in 2.5.9, “Extent allocation in hybrid and managed extent pools” on page 30.

Summary of the virtualization hierarchy

Here we summarize the virtualization hierarchy:

� At the top of the virtualization hierarchy, you have just a bunch of disks (JBOD) that are grouped in array sites.

� An array site is then transformed into an array, with spare disks.

� The array is further transformed into a rank with extents formatted for FB data or CKD.

� The extents from selected ranks are added to an extent pool. The aggregated extents from those ranks in the extent pool are used for subsequent allocation to one or more logical volumes. The Easy Tier Automatic Mode function automatically manages the capacity assigned to logical volumes in up to three tiers in a hybrid extent pool. This function places the extents of each logical volume on the most appropriate storage within the extent pool. Additionally, the Easy Tier Automatic Mode function automatically rebalances extents within the same tier in a managed pool to continuously provide good performance by balancing the load on the ranks and on DA pairs.

� Dynamic extent pool merging allows one extent pool to be merged into another extent pool while the logical volumes in both extent pools remain accessible to host access.

� Rank depopulation allows you to unassign a rank that contains used extents by automatically relocating those extents away to the other ranks in the same pool, while the affected logical volumes remain accessible to host access.

� Logical volumes are created from extents within the extent pools (optionally striping the volumes), assigning them a logical volume number that determined which logical subsystem they can be associated with and which server can manage them. This function is the same for both standard volumes (fully allocated) and extent space efficient volumes. Track space-efficient volumes for use with FlashCopy SE can only be created within the repository of the extent pool.

� The logical volume configuration state reflects the condition of the logical volume relative to user requested configuration operations, or to the Easy Tier automatic extent relocation (managed EAM state).

� Dynamic volume relocation allows data stored on a logical volume to be migrated from its currently allocated storage to newly allocated storage while the logical volume is online to attached hosts.

To complete the virtualization hierarchy:

� The LUNs can then be assigned to one or more volume groups.

� The host HBAs are configured into a host attachment that is associated with a volume group.


Figure B-5 summarizes the virtualization hierarchy.

Figure B-5 Virtualization hierarchy

This virtualization concept provides much more flexibility than in previous products. Logical volumes can dynamically be created, deleted, and resized. They can be grouped logically to simplify storage management. Large LUNs and CKD volumes reduce the total number of volumes, and this configuration also contributes to a reduction of management effort.

Ser

ver0

Array Site

RAIDArray

Spare

Data

Data

Data

Data

Data

Data

Parity

RankType FB

1 G

B F

B

1 G

B F

B

1 G

B F

B

1 G

B F

B

1 G

B F

B

1 G

B F

B

1 G

B F

B

1 G

B F

B

1 G

B F

B

ExtentPool

LogicalVolume

LSSFB

AddressGroup

X'2x' FB4096

addresses

LSS X'27'

X'3x' CKD4096

addresses

VolumeGroup

Host Attachment


Glossary

Cold extent on HDD An extent that does not benefit if it is moved from HDD to SSD media.

Cold extent on SSD An extent that can provide overall performance improvement if it is replaced by a hot extent from an HDD.

Easy Tier The capability of a DS8700 system to manage data placement in a hybrid pool. Easy Tier optimizes the data placement in a hybrid pool by using the SSD media sparingly, while taking advantage of its performance superiority to achieve an overall performance improvement.

Heat map I/O statistics collected during the learning period, which can be used to generate a heat map using the STAT tool. The heat map identifies the amount of hot data in each volume.

Hot extent An extent that can benefit if it is moved from an HDD to SSD. Moving the extent to SSD improves the I/O performance on the extent by reducing the latency time.

Hybrid pool A storage pool that contains both SSD and HDD drives.

Latency The time that elapses between the initiation of a request for data from a rank and the start of the actual data transfer to the rank.

Learning window/period A learning algorithm developed by IBM. This algorithm runs for a 24-hour period, monitoring and assigning “heat” values to each extent in a storage device. The heat values, hot or cold, indicate what tier the data must be on.

Migration plan Based on the data from the heat map, a migration plan is created to migrate cold extents to HDDs and hot extents to SSDs.

Storage Tier Advisor Tool (STAT) This tool requires the I/O statistics input file gathered during the learning window. With this data, the STAT tool provides a heat map of each volume.

© Copyright IBM Corp. 2010, 2012. All rights reserved.
159

Related publications

The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this paper.

IBM Redbooks publications

For information about ordering these publications, see “How to get Redbooks publications” on page 162. Note that certain documents referenced here might be available in softcopy only.

� DS8000 I/O Priority Manager, REDP-4760

� DS8800 Performance Monitoring and Tuning, SG24-8013

� IBM System Storage DS8000: Architecture and Implementation, SG24-8886

� IBM System Storage DS8000 Host Attachment and Interoperability, SG24-8887

Other publications

These publications are also relevant as further information sources:

� Driving Business Value on Power Systems with Solid State Drives, POW03025USEN

� IBM DS8000 Storage Virtualization Overview Including Storage Pool Striping, Thin Provisioning, Easy Tier, WP101550

� IBM Easy Tier on DS8000, SVC and Storwize V7000 Deployment Considerations Guide January 2011, WP101844

� IBM System Storage DS8700 and DS8800 Introduction and Planning Guide, GC27-2297-07

� IBM System Storage DS8700 and DS8800 (M/T 242x) delivers DS8000 I/O Priority Manager and advanced features to enhance data protection for multi-tenant copy services, ZG11-0282

� IBM System Storage DS8700 and DS8800 (M/T 242x) delivers DS8000 I/O Priority Manager and advanced features to enhance data protection for multi-tenant copy services, ZG11-0130

� IBM System Storage DS8700 (Machine types 2421, 2422, 2423, and 2424) is designed to deliver new Easy Tier solid-state storage workload optimizer, advanced features supporting business continuity, and new drive options that can help to double the system's maximum capacity, ZG10-0125

� IBM System Storage DS8700 Performance with Easy Tier, WP101675

� IBM System Storage DS8700 and DS8800 Performance with Easy Tier 2nd Generation, WP101961

� IBM System Storage DS8800 and DS8700 Performance with Easy Tier 3rd Generation, WP102024

� IBM System Storage DS8700 Performance Whitepaper, WP101614

� IBM System Storage DS8800 Performance Whitepaper, WP102025


Online resources

These website and URLs are also relevant as further information sources:

� IBM Disk Storage Feature Activation (DSFA):


� Documentation for the DS8000 system:

http://www.ibm.com/systems/storage/disk/ds8000/index.html

� IBM System Storage Interoperation Center (SSIC):

http://www.ibm.com/systems/support/storage/config/ssic/index.jsp

� IBM Announcement letters (search for R6.2):

http://www.ibm.com/common/ssi/index.wss

� IBM Techdocs Library - The IBM Technical Sales Library:

http://www.ibm.com/support/techdocs/atsmastr.nsf/Web/Techdocs

� IBM Almaden Research Center, Easy Tier:


How to get Redbooks publications

You can search for, view, or download Redbooks publications, Redpapers publications, Technotes, draft publications, and Additional materials, and order hardcopy Redbooks publications, at this website:

ibm.com/redbooks

Help from IBM

IBM Support and downloads:

ibm.com/support

IBM Global Services:

ibm.com/services




http://www.ibm.com/support/

http://www.ibm.com/support/

http://www.ibm.com/services/

http://www.ibm.com/services/

http://www.ibm.com/support/techdocs/atsmastr.nsf/Web/Techdocs


http://www.ibm.com/systems/storage/disk/ds8000/index.html


http://www.ibm.com/systems/support/storage/config/ssic/index.jsp

http://www.ibm.com/common/ssi/index.wss

Index

AAccess state field 73auto mode 34Automatic Class Selection 146Automatic Class Selection (ACS)

routines 146automatic data relocation 21Automatic Data Relocation (ADR) 12Automatic Mode 13automatic performance rebalance 28automode 115auto-rebalance 3, 8, 13, 24, 26–28, 49, 54–56, 116, 123, 134, 136, 139auto-rebalance migration plans 25

Bbucket 10

Ccache 21cache hit 21capacity rebalance 16chextpool 98, 142chrank 98, 110chsi 98, 115CMUG00408E 90CMUN80204E 18cold data 10cold demote 27cold demotion 29cold extents 23, 133configstate 106Configuration State 72Copy Services 59cross-tier management 12cross-tier migration 24–25cross-tier migration plans 25

Ddata granularity 9data relocation 9data set 21, 147Data state field 73data_files directory 126decision windows 25depopulating 99Depopulation Error 99device adapters (DA) 35DFSMSdfp 147disaster recovery 35Disk Magic 20, 64DS CLI 72

© Copyright IBM Corp. 2010, 2012. All rights reserved.

DS GUI 72dynamic data relocation 2dynamic extent pool merge 4, 13–14, 49, 72, 108, 138, 142dynamic volume relocation (DVR) 4, 13, 15, 17, 20–21, 30, 54, 57, 72, 78, 98, 100–101, 138

EEasy Tier 2

Automatic Mode 3, 8, 10–11, 21, 23–24, 32, 35, 38, 50, 53, 72, 75, 94, 96, 99, 115–116, 121–123, 143

all 32control 32enable 32tiered 32

controls 72decision window 26ETAutoMode 122first generation 4, 8, 37Managed field 75Manual Mode 4, 8, 13, 18, 20–21, 32–33, 50, 54, 75, 115, 138–139Monitor 96

automode 122Monitor Mode 32, 127monitoring 21, 23, 50, 96, 99, 115, 122

automode 122performance data collection 4, 121second generation 4, 8, 37Summary Report 97, 117, 121–125third generation 4, 8, 37

encryption 38Enterprise HDDs 37ETAutoMode 115, 139

all 139etdata 117, 122–123ETMonitor 115

all 121–122automode 121

ETmonitor 121expanded cold demote 27, 30extent allocation method (EAM) 16–17, 30, 52, 57, 81, 100, 103

change 52managed 30–31, 52–53, 121, 156rotate extents 30–31, 52, 56, 155–156rotate volume 156rotate volumes 30, 52, 56, 155

Extent Pool Merge window 72extent pools

consolidation 14homogeneous 10–11hybrid 8, 10–11, 14, 53multitier 8, 10–11, 53

163

single-tier 10–11source 13target 13

extent space-efficient (ESE) 16, 38, 57storage allocation method 8, 11, 13, 18, 57volumes 154

Ffailure boundaries 55FlashCopy 59FlashCopy Space Efficient 60FLASHDA tool 21

GGlobal Mirror 59

HHDD performance 22heat 23heat maps 11, 21home tier 31, 52, 57–59hot data 10hot extents 23, 133hot volumes 20hotspots 28hybrid extent pools 14hybrid pools 49

II/O characteristics 9I/O Priority Manager 122IBM i 21index.html file 126inter-tier management 12inter-tier migration 24–26intra-tier 24intra-tier management 12intra-tier migration 24–26intra-tier migration plans 25intra-tier rebalancing 28IOPMmode

Managed 122

Llatency 23lsckdvol 98lsextpool 98lsfbvol 98lskey 98lsrank 98

MMachine Type Serial (MTS) 78manageckdvol 98, 100managefbvol 98, 100manual rebalance 8

manual volume rebalance 8, 13–14, 16–17, 29, 52, 56, 103, 139, 144Merge Extent Pool window 77merging 99Metro Mirror 23, 59micro-tiering 13, 49, 54–55migcancel 100migpause 100Migrate Volume windows 72Migrating state 99migration 15

cancel 78initiate 78pause or resume 78

Migration Cancelled 99Migration Error 99Migration Paused 99migration plans 10, 23–25, 122migration queues 25migresume 100migstart 100

NNearline drives 37Number Of Tiers 75

Ooffloadfile 117, 123

Pperformance metrics 11performance monitoring 11planning tools 61priority levels 10promote migration type 27

RRAID types 54rank configuration state 99rank depopulation 4, 18, 57, 72, 110, 138rank states 20Redbooks website 162

Contact us xiirequest for price quotation (RPQ) 40, 54Resume Migration window 86rotate extents 15, 30, 56rotate volumes 15–16, 103

SSCORE 40, 54server affinity 13, 15showckdvol 31, 98–99showextpool 98–99showfbvol 31, 98–99showrank 98–99showsi 99, 121skew 28, 66, 134


skew-factor 66solid-state drives (SSD) 35, 37

performance 22stat.exe 126storage allocation method (SAM) 11Storage Class 146storage economics 8storage economics management 4, 8Storage Group ACS 146storage performance 8storage performance management 4, 8storage pool striping 15, 17, 31, 52, 138, 155storage pool stripoing 56Storage Tier Advisor Tool (STAT) 5, 11, 20–21, 25, 33, 53, 61–63, 72, 98, 117, 119–120, 122, 126, 134

download 5, 62, 120Existing Tier Status 131Existing Tier Status window 129input file names 126interpret 127invoke 126processing 125Recommended NL configuration table 130Recommended SSD configuration table 130rename input files 126System Summary window 127Systemwide Recommendation window 130

swap migration type 27system-manage storage (SMS) 146

Ttemperature 11, 23thin provisioning 11, 18, 154thin-provisioned volumes 11Tivoli Storage Productivity Center 21Tivoli Storage Productivity Center for Disk 21track space-efficient (TSE) 16

storage allocation method 13, 15, 18volumes 154

Transposition Error state 99

UUnassigned Reserved state 99Unassigned state 19

VVolume Heat Distribution 11, 64, 133volume restriping 16, 56volumes

configuration state 99extent pools 38migration 15, 52, 72, 88, 98, 138space-efficient 38

Wwarm data 10warm demote 27warm demotion 29

warm extents 23, 133workload planning 60workload skew 66

Index 165

®

REDP-4667-02

INTERNATIONAL TECHNICALSUPPORTORGANIZATION

BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE

IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.

For more information:ibm.com/redbooks

Redpaper™


Smart monitoringthree-tier support

Intra-tier and cross-tier performance management

Automated subvolume data relocation and rebalancing

This IBM Redpaper publication describes the concepts and functions of IBM System Storage DS8000 Easy Tier and explains its practical use with the IBM System Storage DS8700 and the IBM System Storage DS8800 systems.

The IBM System Storage DS8000 Easy Tier feature and the accompanying Storage Tier Advisor Tool (STAT), as described in this paper, are available for the DS8700 with Licensed Machine Code (LMC) level 6.6.2x.xxx or higher and the DS8800 with Licensed Machine Code (LMC) level 7.6.2x.xxx or higher.

IBM System Storage DS8000 Easy Tier is designed to automate data placement throughout the storage system disks pool. It enables the system, automatically and without disruption to applications, to relocate data (at the extent level) across up to three drive tiers. The process is fully automated. Easy Tier also automatically rebalances extents among ranks within the same tier, removing workload skew between ranks, even within homogeneous and single-tier extent pools.

Easy Tier supports a Manual Mode that enables the user to relocate full volumes. Manual Mode also offers the ability to merge extent pools and to restripe volumes within an extent pool. A rank depopulation function is also offered. Easy Tier now fully supports thin provisioned fixed-block volumes (ESE) in Manual Mode and Automatic Mode.

This paper is aimed at those professionals who want to understand the Easy Tier concept and its underlying design. It also provides guidance and practical illustrations for users who want to use the Easy Tier Manual Mode capabilities.

Back cover




ibm storage easy tiers

Documents