db2 11 for z/os: migration, experiences and...

IBMDB2forz/OSDevelopment

MichaelDewert– dewert@de.ibm.com

DB211forz/OS:Migration,ExperiencesandTransition

Disclaimer:

Informationregardingpotentialfutureproductsisintendedtooutlineourgeneralproductdirectionanditshouldnotbereliedoninmakingapurchasingdecision.TheInformationmentionedregardingpotentialfutureproductsisnotacommitment,promise,orlegalobligationtodeliveranymaterial,codeorfunctionality.Informationaboutpotentialfutureproductsmaynotbeincorporatedintoanycontract.Thedevelopment,release,andtimingofanyfuturefeaturesorfunctionalitydescribedforourproductsremainsatoursolediscretion.

PerformanceDisclaimer:

Thisdocumentcontainsperformanceinformationbasedonmeasurementsdoneinacontrolledenvironment.Theactualthroughputorperformancethatanyuserwillexperiencewillvarydependinguponconsiderationssuchastheamountofmultiprogrammingintheuser’sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed.Therefore,noassurancecanbegiventhatanindividualuserwillachievethroughputorperformanceimprovementsequivalenttothenumbersstatedhere.

2

Objectives• Sharelessonslearned,surprises,pitfalls

• Providehintsandtips

• Addresssomemyths

• Provideadditionalplanninginformation

• Provideusageguidelinesandpositioningonnewenhancements

• Helpcustomersmigrateasfastaspossible,butsafely

3

Agenda

• Introduction

• MigrationConsiderations

• APPLCOMPAT

• PlanManagement

• Utilities

• Availability

• DataSharing

• PerformanceandScalability

• DB2andREALSTORAGEmanagement

• EXTENDEDRBA/LRSN

• Summary

4

INTRODUCTION

5

DB211MajorThemes• Out-of-the-boxCPUSavings

– Improvingefficiency,reducingcosts,noapplicationchanges

– Upto10%forcomplexOLTP

– Upto10%forupdateintensivebatch

– Upto40%forqueries

– AdditionalperformanceimprovementsthroughuseofnewDB211features

• EnhancedResiliencyandContinuousAvailability– Improvedautonomicswhichreducescostsandimprovesavailability

– Makingmoreonlinechangeswithoutaffectingapplications

– OnlineREORGimprovements,lessdisruption

– DROPCOLUMN,onlinechangeofpartitionlimitkeys

– Extendedlogrecordaddressingcapacity- 1yottabyte(or1Bpetabytes)

– BIND/REBIND,DDL,OnlineREORGtobreakintopersistentthreads

• Enhancedbusinessanalytics– ExpandedSQL,XML,andanalyticscapabilities

– TemporalandSQLPLenhancements

– Hadoopintegration,NoSQLandJSONsupport

– Transparentarchiving

• Simpler,fasterDB2versionupgrades– Improvedproductquality/reliability– throughiterativeapproachon3monthlycycle(1:Nrallies)toFVT,SVT,Performancetesting,and

stabilizationphaseaheadofstartofESP

– ApplicationchangesdivorcedfromDB2systemupgrade(APPLCOMPAT)

– Accesspathstabilityimprovements

6

Announced Oct. 1, 2013

GA Oct. 25 2013

MIGRATIONCONSIDERATIONS

7

SoftwareRequirements• z/OSV1.13BaseServices(5694-A01)atminimum• DFSMSV1R13– theDB2Catalog isSMSmanaged• LanguageEnvironmentBaseServices• z/OSVersion1Release13SecurityServer(RACF)• IRLMVersion2Release3(ShippedwithDB211forz/OS)• IBMInfoSphere DataReplication(IIDR)10.2.1

– Noticethatthisisapre-req forentryintoV11ifrunningCDCand/orQ-Rep

• IBMDB2AnalyticsAcceleratorV4• IBMDB2ToolsCompatibility• DB2LUW/Connect/DataServerDriver/JDBC/ODBC

– Anyin-supportIBMInformationManagementsclients/drivers– V10.5FP2advisedfornewfeatures(seamlessupgrade9.7fp6,10.1fp2,10.5fp4)– SeetheProgramDirectoryformoredetails(exDB2forz/OSclienttoDB2LUWserver)– SeethefollowingslidesonDB2Connectinthissection

8

PrerequisiteSummary&Planning...• z/Architecture(z10andsubsequent64-bitz/ArchitectureProcessors)

• DataSharing:CFLEVEL12,with13or14recommended– GBPEnhancementsrequireCFLEVEL17orhigher

• 2-Gigabyteframesizeforbufferpools– RequireszEnterprise EC12processorandAPAROA40967forz/OSVersion1Release13

• Pageable 1-Megabytelargepages– RequireazEnterprise EC12processorwithFlashExpresscapabilityplusRSMEnablement

Offering(FMIDJDB778H)

• z/OSUnicodeServicesandappropriateconversiondefinitionsarerequired

• Checkthefollowingdocumentationformoreinformationaboutsoftwareandhardwarepre-requisites:

– TheDB211ProgramDirectory:http://publibfp.boulder.ibm.com/epubs/pdf/i1089450.pdf

– TheDB211AnnouncementLetter:http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=AN&subtype=CA&htmlfid=897/ENUS213-376&appname=USN

9

PrerequisiteSummary&Planning...• PID5615-DB2

– FMIDsHDBBB10,HIYBB10,HIZBB10,HIR2230,HDREB10• Check/correctincompatibilities

– RunacurrentV10DSNTIJPBpre-migrationjob• DeliveredinPM94057,currentlevelPI37038• SameasV11DSNTIJPM• Checkmaintenanceforcurrencyfixcat DB2MIGV11/K

– ReleaseIncompatibilitiesdocumentedin:• InstallationGuide• ApplicationProgrammingandSQLGuide

• Checkprogramminglanguagerequirements– Precompiler DSNHPC7includedinthebaseforolderCOBOLandPL/I

• RestrictsSQLtoDB2forz/OSV7level• SeetheProgramDirectory

• MigrationonlyallowedfromDB210forz/OSNFM– WithFallbackSPEPM31841withassociatedcoreqs &prerequisites– TheDSNHDECPparametermodulemustberecompiled– SeeInformationalAPARsII14660/II14745(Migration),II14730/II14732(Client&DDF)

• EstablishperformancebenchmarksatDB210NFM– SeethelatersectiononPerformanceandScalability

10

PrerequisiteSummary&Planning...• Inz/OSparametermemberIEASYSxx,configureaminimumof:

– 1TBofcontiguoussharedprivateperDB2– HVSHARE• z/OSdefaultis510TB

– 6GBofcontiguoussharedextendedprivateperDB2– HVCOMMON• SameasDB210• Default64GB

• Nosignificantstorageusagedifferences– On-demandBufferPoolallocationfromDB210wasremoved

• Thisprobablydoesnotrepresentastoragerequirement• VPSIZEisallocatedwhenBufferPoolisallocated

– Earlytestingindicates0– 5%additionalmemorydependentonqueryworkload• Consider10%-15%cushionforpotentialgrowthwithlesswell-behavedworkloads

– Extra37Kperopenindexpartasusedbyautonomicpseudodeletecleanup• SMSmanagedcataloganddirectory

– ThiswasarequirementforDB210• DataClassattributesof

– ExtendedFormat– ExtendedAddressability

– NewareasoftheCat/DirmovedunderSMSmanagementduringmigrationto11– AdditionalmigratedareasREORGed afterV10ENFMwouldalsohavebeenconverted– SDSNSAMPmemberDSNTIJSS isprovidedasasampleforconfiguration

11

DB211forz/OSRelatedNoChargeProduct• IBMDB2AccessorySuiteforz/OSV3.3(12/11/15)

– 5697-Q04- http://www-01.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_sm/4/872/ENUS5697-Q04/index.html&lang=en&request_locale=en

– IBMDB2AccessoriesSuiteforDB211feature• Spatialdata,UnicodeandInternationalizationsupportfor

– IBMTextSearchforDB2forz/OS–DataStudio4.1(4.1.2isnewest)

• EnhancedsupportforDB2asJSONDocumentstore• InstallableInformationCenterFramework

–Nolonger includedbutstillcurrentlysupported• SeeProgramDirectoryGI10-8957• DB2Adaptorforz/OSConnect

– IBMDB2AccessoriesSuiteforDB210feature• IncludeswhatisintheV11AccessoriesSuite• JSONSupportviaAPARII14727forDB210

12

PRE-MIGRATIONPLANNING

13

Pre-migrationSummary&Planning• Developconversionandcoexistencegoals

– HowdidyourV10testplanswork?– Reuseandimproveuponyourexperiences

• Migrationoccursinthreefamiliarphases– ConversionMode(CM)– EnableNewFunctionMode(ENFM)– NewFunctionMode(NFM)

• UsetheproactivePMRprocess– ThisistonotifyIBMDB2Supportofyourupcomingmigrationactivity

• ItistypicallyroutedtothesupportDutyManagerandbroadcasttothedutyteam– Ifproblemsoccurduringthemigration,openanewPriority1Severity1PMR

• REBIND/REGENERATEwhileinCM– UseREBINDPlanManagement(Package/BindStability)

• ConsiderFREEing OriginalPackagestoestablishanewDB210backup– Consider:

• REBIND...EXPLAIN(YES)...APREUSE(WARNorERROR)• REBIND...EXPLAIN(YES)...APCOMPARE(WARNorERROR)

14

Pre-migrationplanning

• RunDSNTIJPMorDSNTIJPB,toidentifythem– DSNTIJPMshipswithDB211andshouldberunonDB210toidentifypre-migrationcatalogclean-uprequirements• DSNTIJPMmayprovideDDLorutilitystatementsfortheclean-up

– DSNTIJPBisthesamejobandisshippedforDB210tomaximizepreparetime

15

Importantpreparation• OldplansandpackagesbeforeV9->REBIND• DonotrelyonAUTOBINDofinvalidplansandpackagesonentrytoV11CM.

– DealwiththeissuebeforeleavingV10NFM.

• Views,MQTs,andTablefunctionswithPeriodSpecification->DROP– ThosecreatedinV10arenotsupported

• DSNUTILSstoredproceduredeprecated• Synonymsaredeprecatedà but,stillsupportedinfuturereleasesasconsideredasa

incompatiblechange– ShouldbelookingatusingAliases

• Simpletablespacesà needtotracktheiruse,keepmovingtowardUTS• ASUTIMEmaybeapplieddifferentlyfordynamicstatementsofnestedroutinesinNFM

– SQLCODE-905returnedislimitfornestedroutineisexceeded– Previouslyhadtoexceedthetoplevelpackagelimit

• ALTERLimitKey– NowaPendingAlterONLYifitisTABLECONTROLLED

• PREVENT_ALTERTB_LIMITKEY

• ClientAccountingInfoSpecialRegisterExpansion

16

Itemsdeprecatedinearlierversions– Noweliminated• Passwordprotectionforactivelogandarchivelogdatasets

• DSNHCLISTNEWFUNvaluesofV8andV9– UseV10orV11

• SomeDB2suppliedroutines– SYSPROC.DSNAEXP–>UsetheEXPLAINPrivilegeandissueEXPLAINdirectly

– AMI-basedDB2MQ(DB2MQ)functions–>usetheMQI-basedfunctionsinschema(seeAPARPK37290forguidance)• DB2MQ1C.*, DB2MQ2C.*

• DB2MQ1N.*, DB2MQ2N.*

• CHARSETapplicationprogrammingdefaultvalue(KATAKANA)– useCCSIDs

• BINDPACKAGEoptionsENABLEandDISABLE(REMOTE)REMOTE(location-name,...,<luname>,...)-- specificnamescannotbespecified

• SysplexQueryParallelism– Singlememberparallelismisstillsupported

• DSN1CHKR– TherearenolongeranylinksintheCatalogorDirectory

17

MIGRATIONOVERVIEW:DB210->DB211

18

MigrationOverviewDB210->DB211

19

DB2 11 Enabling New Function Mode (ENFM)

DB2 11 Catalog

Data Sharing Coexistence

DB2 11 Conversion Mode (CM)

DB2 11 New Function Mode

(NFM)

DSNTIJTC(CATMAINT UPDATE)

DSNTIJNF(CATENFM

COMPLETE)

DSNTIJEN(CATENFM

START)

DB2 10 Catalog

DB2 11 Libraries

DB2 10 Libraries

DB2 10 New Function Mode

(NFM) With SPE

1 – 2 months

1 week

Minutes

Use APPLCOMPAT(V10R1) here

PreparingyourcurrentDB210NFMforMigrationtoDB211CM• ApplytheFallbackSPEAPAR,PM31841andanyprerequisitefixes

– YourDB210systemMUSTbeattheproperservicelevel

– SeeInfoAPARsII14660

• Non-DataSharing– CurrentDB210mustbestartedwiththeSPEapplied,ormigrationtoDB211willterminate

• DataSharing– BeforemigratingamembertoDB211,allotherstartedDB210membersmusthavethefallbackSPEapplied

– ThefallbackSPEmustbeonallactiveDB210groupmembersforDB211tostart

20

Important – Apply SPE to ALL Data Sharing Members Before Starting Migration!

Otherrecommendations• RunOnlineREORGsagainstCatalogandDirectoryobjectspriortotheENFM/NFMmigration

– CheckthatREORGcanbreakin

– CheckdataconsistencyofCatalogandDirectory

– ImprovetheperformanceoftheENFMprocess

• CATMAINTandENFMwillnotexecuteifentriesfoundinSYSUTILX– DB2willnolongerblindlyre-initializeit

21

MigrationPreparations• RunhealthchecksonDB2CatalogandDirectorytoflushoutanylatentinconsistencyandcleanupbeforethemigration

• QueriesfrommigrationjobDSNTESQ- shouldalwaysreturnzerorows

• REPAIRDBDTESTorDIAGNOSE

• COPY,orDSN1COPYwithCHECKoption

• BasicRUNSTATSonallobjects

• ForobjectswithLOBcolumns:– CHECKLOB

– CHECKINDEXontheAUXindex

– CHECKDATAonthebasetablespaceusingSCOPEAUXONLYAUXERRORREPORT

• CHECKINDEXonallindexes

• CHECKDATAforDSNDB01.SYSUTILX,DSNDB06.SYSPKAGE,DSNDB06.SYSSTATS,DSNDB06.SYSSTR

22

APPLCOMPAT-APPLICATIONCOMPATIBILITY

23

APPLCOMPAT– ApplicationCompatibility• Requirements

– De-coupletheneedforapplication programchangestodealwithincompatibleSQLDMLandXMLchangesfromtheactualDB2systemmigrationtothenewDB2releasewhichintroducedtheincompatibleSQLDMLandXMLchanges

– ProvideamechanismtoidentifyapplicationprogramsaffectedbyincompatibleSQLDMLandXMLchanges

– Provideamechanismtointroducechangesatanindividualapplicationprogram(package)level• Enablesupportsothatapplicationprogramchangescanbephasedinovermuchlongertime

• EnablesupportformixedDB2releaseco-existenceindatasharing

• EnablesupportforuptotwobacklevelreleasesofDB2(N-2)

• Solution– APPLCOMPATwhichseparatesDB2systemmigrationtothenewDB2releasefromapplicationprogrammigrationtodealwithincompatibleSQLDMLandXMLintroducedbythenewrelease

24

The image part

APPLCOMPAT– ApplicationCompatibility...• APPLCOMPATZPARMprovidesdefaultforBIND/REBIND

– V10R1forDB210SQLDMLbehaviour

– V11R1forDB211SQLDMLbehaviour

– DefaultisV11R1fornewinstalls,V10R1formigration

• APPLCOMPAToptiononBIND/REBINDtooverrideZPARMdefault

• CURRENTAPPLICATIONCOMPATIBILITYspecialregisterandDSN_PROFILE_ATTRIBUTESforDDF

– FordynamicSQL

– DynamicJCCpackages(i.e.SYSNLxxx)needtobereboundtoinitiatespecialregister

• DoesnotaddressissueswithnewreservedwordsorotherincompatibilitiesthatcouldonlyberesolvedbyhavingmultiplelevelsoftheDB2parser

• BIF_COMPATIBILITYZPARMisindependentofAPPLCOMPAT

• NewSQLfunctionalityavailableinV11NFMcannotbeuseduntilpackageisboundwithAPPLCOMPATvalueofV11R1

– RememberifapackagewasboundinCMithas‘V10R1’isSYSPACKAGEandyoumustconsciouslyrebinditwith‘V11R1’tousenewfunctionality

25

The image part

APPLCOMPAT– ApplicationCompatibility...• MigrationautomaticallysetsV10R1priortoNFM…otherwise

– DSNT225I-DSNBINDERRORFORPACKAGElocation.collid.memberAPPLCOMPAT(V11R1)OPTIONISNOTSUPPORTED

– IFCID376– SummaryofV10functionusage• IT is recommended to use use IFCID 376 under V11 CM in Development/Test and

Production (include PI61920, PI21454, OA49474)

– WeexpectchangesnecessarytoavoidV10R1usagetohappenafterreachingNFM

• WorkaroundtodistinguishpackageswhichhavetoabsolutelyrunasV10R1untiltheyarecorrected

– AnnotatethepackageusingSQLCOMMENTONPACKAGEcolid.name.”version”IS‘V10R1’• Ifversionisapre-compilertimestampthenthedoublequotesarenecessary

– StoredintheREMARKScolumninSYSIBM.SYSPACKAGEtable• Canbequeriedandbeexploitedbyhousekeeping

26

APPLCOMPATvs.BIF_COMPATIBILITY• BIF_COMPATIBILITY=V9|V9_DECIMAL_VARCHARisstillhonoredinallmodesofV11

– The‘undocumented’timestampsupportisbackagainwithV11CMonlywithAPPLCOMPAT(V10R1)e.g.,• EURdateformatconcatenatedtotheTIME(andmicroseconds)

• AlsoDDF_COMPATABILITYZPARM– IDNTFY_V11_PRIOR_VER

• DB211CMmemberisidentifiedasDB210NFMmember

27

PLANMANAGEMENT

28

AccessPathStabilityImprovements(APSI)...• PlanManagementPolicies(PLANMGMT)

– DB29deliveredAccessPath/BINDStability(PLANMGMT)viaREBIND• PLANMGMTRebindkeywordorSystemParameter

–BASIC:Keeps2copiesofapackage(CurrentandPrevious)

–EXTENDED:Keeps3copies(Current,Previous,andOriginal)• UsetheProfiletablestosetupautomaticPlanManagementPolices

– SystemParameteroptionsarethesameasV9withdefaultchangeinCM• OFF(wasV9default)• BASIC• EXTENDED(V10default)

– CopiesarekeptbydefaultinDB210

29

APSI/PlanManagementPolicies...• PLANMGMT

– DB210evolvesthiscapability• MoreformalCatalogimplementationforPLANMGMT(V9delivery)

–SYSPACKCOPYispopulatedaspartoftheENFMprocess» COPYIDcolumn

» 1:IndicatesaPreviouscopy» 2:IndicatesanOriginalcopy

• OldcopiesthatarefrombeforeV6willstillbemarkedinvalid

– REBINDchanges•APRETAINDUPforPLANMGMT(BASICorEXTENDED)

–APRETAINDUPkeyworddefaultstoYES–NO:AnewaccesspathidenticaltoaCurrentaccesspathdoesnotgetcopiedtoaPreviousorOriginal

30

APSI/PlanManagementPolicies• PLANMGMTSCOPE

– SystemParameterandProfileAttribute

– Fornow,theonlyoption(anddefault)isSTATIC(V9behavior)

• PLANMGMTextendedtoNativeSQLProcedures

• EXPLAINPACKAGEcommand

– ToretrievePLAN_TABLErecordsforexistingpackage

•Originalbind/rebindmayhavebeenEXPLAIN(NO)

•MusthavebeenboundonV9orV10towork,needExplainDataBlock

•PopulateexplaintableswithoutBINDorREBIND.

31

>>-EXPLAIN----PACKAGE----------->

>>-----COLLECTION--collection-name--PACKAGE--package-name--------->

>----+--------------------------+----+-------------------+-------->| | | |+---VERSION-version-name---+ +---COPY--copy-id---+

AccessPathStabilityImprovements...• APCOMPARE/APREUSE

– OptionstostabilizeaccesspathsacrossBINDorREBIND

– ThereplacedpackagemustbefromDB29orlater

– APCOMPARE• StructuralcomparisonoftheaccesspathacrosstheBIND/REBIND

• Whendifferencesarediscovered(forexistingstatements),theoutcomedependsontheoptionused

– WARN:PackageiscreatedwithanRC=04

– ERROR:NopackageiscreatedwithanRC=08

– APREUSE• Hintsallstatementsforthenewpackageusingtheaccesspathfromthepackagetobereplaced

• Whendifferencesarediscovered(forexistingstatements):– WARN:PackageiscreatedwithanRC=04,somestatementshavenewaccesspath

– ERROR:NopackageiscreatedwithanRC=08,BIND/REBINDfails

32

AccessPathStabilityImprovements• APCOMPARE/APREUSE

– Comparisonsaredoneatthestatementlevel

– FeedbackisplacedinthePLAN_TABLEifEXPLAINYES orONLY specified

• HINT_USEDcontains‘APREUSE’whenhintsucceeds

• REMARKSusedforreuseorcomparisonfailureinformation

• Theseareplacedonthe‘new’PLAN_TABLErows

• Ifan‘old’PLAN_TABLErowisunmatchedinthe‘new’setofrows– AnewrowiscreatedtoshowtheREMARKS

– Thesearenot“sticky”;mustspecifyforeachBIND/REBIND

– AUTOBINDdefaultstoNO

– HINTcanfailduetocolumnorderofjoins,merging,correlation/decorrelation

33

AccessPathStabilityImprovements• APCOMPARE

– DSNT285Ilists

• #ofstatementssuccessfullycompared

• #ofstatementswithunsuccessfulcomparison

• #ofstatementsthatcouldnotbecompared• APREUSE(NO/NONE,WARN,ERROR)

– InvokesAPCOMPARE– AppliesahintfromtheEDBforthenewpackage– CangetthesameerrorsaswhenusingOPTHINTs

• +395/ReasonCode• ReasonCode50addedforcomparisonfailure

• MigrationandAPCOMPARE/APREUSE– ConsiderusingREBIND...EXPLAIN(YES)...APREUSE(ERROR)

– ThenanalyzetheREBINDfailuresoruseAPREUSE(WARN)

34

APREUSE• DB211providesAPREUSE(WARN)

– Attemptstohintthesameaccesspathformatchedstatements– Whentheaccesspathcannotbemaintained,anewoneiscalculatedforthosespecificmatchingstatements

– Effectivelyoperatesatthestatementlevel– ThesameconsiderationsfromDB210carryforward

• UsestheExplainDataBlock,asdoesAPREUSE(ERROR)andAPCOMPARE(WARN/ERROR)• SomePLAN_TABLEcolumnsarenothint-able(MATCHCOLS)

– APREUSE(WARN)willallowtheBIND/REBINDtocontinue

• DB210APREUSE(ERROR)...EXPLAIN(ONLY)mayrepresentinvalidplan– InDB211,thiswillberolledbacked

35

APPLCOMPAT-ApplicationCompatibilityandPlanManagementSummary• DifferentiateAPPLCOMPAT-ApplicationCompatibilityandPlanManagement• UnderstandAPPLCOMPATwhichseparatesDB2systemmigrationfromapplicationprogrammigration

– MonitorandprepareforincompatiblesusingIFCIDTRACINGandsettherightBINDoption» UseIFCID376underV11CMinDevelopment/TestandProduction.

• APPLCOMPATisastickyoption.– WillneedtouseitexplicitlyonBINDREPLACEand REBINDofexistingpackage

togettoV11R1.• UseAccessPlanStabilityImprovementsasoptionstostabilizeaccesspathsacrossBIND

orREBIND– ConsiderDB2V11withAPCOMPARE/APREUS– ThereplacedpackagemustbefromDB29orlater– ConsiderbindingpackagelikeSPUFI,TEP2,etc withAPPLCOMPAT(V11R1)onentrytoV11

NFM.

• DonotrelyonAUTOBINDofinvalidplansandpackagesonentrytoV11CM.– DealwiththeissuebeforeleavingV10NFMusingPLANMANAGEMENT.

36

UTILITIES

37

REORGEnhancements

• SWITCHphaseETreductionof91%measuredwhenreorging 20parts

– ReductionattributedtointernalLOG/SWITCHphaseprocessingchanges

• NewSWITCHTIMEoptionavoidstheneedformultiplejobstocontrolthestartofthedrain

– PriortoDB211hadtocontrolSWITCHphasewithMAXRODEFERandaseparatejobtoissue–ALTERUTILITY…MAXRO(n)

– NewSWITCHTIMEparameterusedtodetermineearliestpointatwhichdrainprocessingwillbeattempted• CanalsosetNEWMAXRO-integerforsecondstowaitafterSWITCHTIME

38

REORGEnhancements…• Fasterdrainacquisitionforpart-levelREORGinCM

• Preventnewclaimsonalltargetpartitionswhilewaitingfordrains– IfREORGPART1-10,priortobeginningdrainonpart1(outof10),willsetflagtopreventclaimsonanyofthe10partsinvolvedinREORG.

– Willthendrainpart1,part2andsoontillall10partsdrained

• UseofDRAIN_ALLPARTSYESoption(notdefault)hasthepotentialtosignificantlyreducethe‘outage’

– AvoiddeadlocksbetweendrainsandclaimsacrossNPIsandpartitionswhenreorganizingsubsetofpartitions

– TemporarilydrainentiretablespaceuntilwegetdrainonNPI,thenreleasealltablespacepartsexceptthoseinvolvedinREORG

– MorelikelytobesuccessfulingettingsuccessfulDRAINtomaketheSWITCH

– BigreductionsseenintheelapsedtimetocompleteDRAINandSWITCH

– REORGsshouldrunwithfewerproblemsusingthisfeature

• REORGmessageoutput- DSNU1138Iprovidesdrainbegin/endinformation

39

REORGEnhancements…• Part-levelinlineimagecopywhenreorganizingasubsetofpartitions

– Tapesupportadded,butnosupportyetforSTACKYES

– Changesrequiredtoexistingjobs• Unlesspartition-levelinlineimagecopiesusingTEMPLATEwith&PAor&PART

• REORGSORTDATANORECLUSTERYES|NO– RECLUSTERNOwillbypasssort(andspeedupconversiontoextendedformat)

– RECLUSTERNOisenforcedforSHRLEVELCHANGEwithSORTDATANO• SpecifySORTDATAtogetreclustering (unloadedthroughclusteringindex)

– DSNU2904IDATARECORDSWILLBEUNLOADEDVIAunload-method• CLUSTERINGINDEX

• TABLESCAN

• TABLESPACESCAN

40

REORGEnhancements…• Automatedbuildingofthemappingtablewithnew10byteLRBA/LRSN

– V11CMbehavior• AnexistingmappingtableinV10(6-byteLRBA/LRSN)orV11(10-byteLRBA/LRSN)formatisreused

• Ifthemappingtabledoesnotexist,amappingtableisautomaticallytemporarilycreated

– V11NFMbehavior• AnexistingmappingtableinV11formatisreused

• IfamappingtableexistsbutisinV10format,aV11formatmappingtableisautomaticallycreatedinthesamedatabaseastheoriginalmappingtable

• Ifmappingtabledoesnotexist,mappingtableisautomaticallycreatedindatabasespecifiedbyZPARMREORG_MAPPING_DATABASE,orindeclareddatabaseorinDSNDB04

– AutomaticallycreatedmappingtablesaredroppedattheendofREORGortheendofthelastREORGiftherearemultipleREORGsinthejobstep

– Recommendations• PredefineandkeepmappingtablestoavoidSQLDDLcontentionontheCatalog

• UseasinglespecificdatabaseasspecifiedinZPARMforallmappingtables

• ModifyschemaofexistingmappingtablestoV11formataspartofmigrationprocesstoNFMi.e.,ALTERTABLETBMAPALTERCOLUMNLRSNSETDATATYPECHAR(10)NOTNULL;

41

REORGEnhancements…• PARALLEL(maximumnumberofsubtasks)optiontocontrolthenumberofsubtasksusuallybasedoneitherpartitionsorparallelindexbuild

– OverridesZPARMPARAMDEG_UTIL

• REORGLISToptionLISTPARTSn(NFM)– limit#ofpartitionstobeprocessedinasingleREORG

• Ifinputisapart-levelLISTDEF

– OverridesZPARMREORG_LIST_PROCESSING– DefaultvaluecorrespondstosettingofZPARMREORG_LIST_PROCESSING

• PARALLEL=Allpartsreorganizedinoneexecution- SameasLISTPARTS[unlimitedor#ofpartitions]

–DefaultZPARMsetting• SERIAL=Partsreorganizedoneatatime- SameasLISTPARTS1

– TakestheplaceofPARALLELYES/NOonREORGLIST

• Beawareofchangeddefaultse.g.,NOPADYESforREORGDISCARD• LOGRANGESNOoptionshouldonlybeusedwhenSYSLGRNXisknowntobelogicallycorruptedand/orhastobereset

42

REORGEnhancements…• REORGREBALANCE

– NowsupportsSHRLEVEL(CHANGE)– bigstepforwardfor24*7

– Cannowdealwithpartitionsthatwereempty(ordidnotcontainenoughdataforacompressiondictionarytobebuiltduringtheUNLOADphase)beforetheREORG• Willnowbuildasinglecompressiondictionarythatwillgetappliedtoalltargetpartitions

• ThereisnolongeraneedforsubsequentREORGtogaincompression

– GoodchanceofbreakinginonpersistentthreadsrunningRELEASE(DEALLOCATE)packages

• PartitionpruningforUTSPBGtablespaces– OptiontophysicallyremoveandcontractthenumberofUTSPBGpartitions

– OnlyperformedwhenZPARMREORG_DROP_PBG_PARTS=ENABLE

– Disabledbydefault

– ThereisnosupportforPITrecoverytoapointintimepriortoSWITCHphaseforaprunedtablespace

43

ReservedSpaceforUpdate

• Indirectreferencesarecausedbyupdatestoatablewhentherownolongerfitsontheoriginalpage…sothereisapointertowhereitnowresides

• Impactcausedbyindirectreferences(overflowrecords)– Additionalgetpages,potentiallyadditionalI/Os totheoverflowpages

– DegradedclusteringratioàMorefrequentREORGTablespace

– InDB210synclogwritesforGBPdependentpagesetswithoverflowrows

44

SG24-8222 – V11 Performance Topics

• DB211:– INSERTwillreservespaceforupdaters

– NewDDLkeywordforCREATETABLESPACEonPCTFREEclausetospecify%offreespacetoleaveineachdatapagebyINSERT,LOADorREORG

• FORUPDATEn|AUTO

– n isaspecifiedpercentage0to99

– AUTOusesRTSvaluestocalculatethe%foryou

– NewZParm:PCTFREE_UPD• SystemdefaultforFORUPDATEvaluewhenitisnotspecified

• Ifnotspecified,samebehaviorasDB210

REORGofLOBdata• SupportREORGofLOBdataeventhoughauxindexisunavailable

– InV10ifLOBtablespaceisREORPandindexisRBDPLOBscan’tbeREORG’d andindexcan’tberebuilt

• REORGperformanceimprovementforLOBtablespaceofupto70%elapsedand40%CPUreduction

– http://www-01.ibm.com/support/docview.wss?uid=swg1PI17945

• REORGSHRLEVELNONEforLOBschangedtoRC=8from11CMonwards– NotsupportedinV10NFM,butreturnsRC=0withMSGDSNU126I

45

CompressionDictionaries• AvoiddecompressionfailuresforIFCID306readerswhennewcompressiondictionarybuiltbyREORG/LOAD

– Oldcompressiondictionarystoredonlog• NewSYSCOPYrecordpointstooldcompressiondictionaryforCDCtables

– IndicatedinSYSIBM.SYSCOPYcolumnICTYPE=J• IFI306readautomaticallyretrievesoldcompressiondictionaryifnecessary• Avoidneedforreplicationtargetrefreshwhendictionarychanges

46

RUNSTATSandRTSEnhancements• InlineStatisticsareestimatesandshouldnotbecomparedwithstandaloneRUNSTATS• NowpossibletoavoidDSNU602ISTATISTICSARENOTCOLLECTEDFORNONPARTITIONED

INDEXonREORGPARToperation*– REORGsortsallofthenon-partitionedindexkeysbecausetheamountofdatathatwasbeing

reorganizedrelativetothesizeofobjectsexceededinternalthresholds– WhenSORTNPSIoptiononREORGjoborREORG_PART_SORT_NPSIZPARMsettoYESorAUTO:AUTO

specifiesthatifsortingallkeysofthenon-partitionedsecondaryindexesimprovestheelapsedtimeandCPUperformance,allkeysaresorted

• NewRESETACCESSPATHoption– Resetmissingand/orconflictingaccesspathstatisticsintheCatalog– DoesnotaffectspacestatisticsintheCatalogorRTS

• AvoidDSNU1363ITHESTATSPROFILEFORTABLEtable-name NOTFOUND– Willusefixeddefaults

• NosupportforUSEPROFILEwithinlinestatisticsinREORGandLOAD• CanadditionallyExternalizeRTSin-memoryblocksviathefollowingcommand

– –ACCESSDATABASE(DB)SP(TS)MODE(STATS)command– ConsiderexecutingjustbeforerunninghousekeepingwhichwilluseRTStomakedecisions.

47

StatisticsandzIIP Enhancements…• MorezIIP offloadforRUNSTATSdistributionstatistics

– Columngroupdistributionstatisticscollection– RUNSTATSutilityrunsunderanenclaveSRBandisthereforezIIP eligible– Upto80%zIIP-eligible

• EnhanceinlinestatisticsforRUNSTATSavoidance– Inlinehistogramstatistics– InlineDSTATS

• AddedCOLGROUPtoinlinestats• Uptoadditional30%offloadtozIIP (comparedtoV10)

• zIIP offloadforLOADREPLACEPARTclearingofNPIs– 100%forLOADREPLACEDUMMYinput

48

RECOVERenhancements• FastLogApply(FLA)nowimplementedforRECOVERINDEX

– PreviouslyDB2wouldwaituntilalogrecordwastobeappliedbeforereadingtheassociatedindexpageintothelocalbufferpoolwhereitwouldthenbecached

– NowDB2willuselistprefetchtoreadalltheindexpagesthatareneededtoapplylogrecordsfor,beforeapplyinganylogrecord

– Potentialforsignificantsavingsinelapsedtime

– Shouldnowreconsiderdecision:runRECOVERINDEXinparallelwithRECOVERTABLESPACE[PART]vs.waitforRECOVERTABLESPACE[PARTs]tocompleteandthenrunREBUILDINDEX

– EnhancementtakenbacktoV9andV10viaAPARPI07694

• Optimizationtopoint-in-timeRECOVERlistofobjects– RecoverobjectsonlywhennecessarywhenperformingPITrecoverywhenTOLOGPOINTorTORBAarespecified

– Itdoesnotapplytologonlyrecoveries,RECOVERBACKOUT,andrecoverstocurrent

– DIAGNOSETYPE(607)isrequiredtoactivatethisbehavior

49

RECOVEREnhancements• RecovertoPITfortablespacewithmaterializedpendingalters

– NowsupportedformostALTERs• ForPBRtablespace:SEGSIZE,DSSIZE,BPID,MEMBERCLUSTER

– MustissueaRECOVERthensubsequentREORG• ObjectinhardREORPafterrecover,dataisrecoveredbutnotDDL

• WhenyoudroppendingALTERsinV11AREORstatusremoved

50

time

1> PendingALTER issued

Alter is deferred

3> RECOVER to PIT start

2> REORGmaterialize changes

4> REORG SHRLEVEL

REFERENCE

Backup&RecoveryEnhancements• Fastercatalog/directoryrecovery

– DB211EnablesSYSLGRNXrecordingfortheseobjects(inCM)• DSNDB01.SCT02• DSNDB01.SPT01• DSNDB01.SYSSPUXA• DSNDB01.SYSSPUXB• Indexesonthesetablespaces

– RECOVERutilitywillnottakeadvantageoftheSYSLGRNXrecordsuntilNFM• NewVCATnametranslationonRESTORESYSTEMforsystemcloning

– DB211enhancedtheRESTORESYSTEMutilitytoacceptoldandnewVCATaliases,andtoswitchtothenewVCATaliasesduringlogapplyprocessing• SupportlogapplywhenRESTORESYSTEMusedforcloningpurposes

• ImprovedrecoverabilitywithCOPY-REORGconcurrency– AllowCOPYSHRLEVELCHANGEtorunatthesametimeasREORGSHRLEVELCHANGE,untilREORGisabletodraintheclaimers

51

Partition-levelinlineimagecopy• Fasterpartition-levelrecoveryfrominlineimagecopy

• Createpartition-levelinlineimagecopiesifusingTEMPLATEwith&PAor&PART– Separateoutputdatasetcreatedforeachpartition

• PreV11usedsingleoutputdatasetforALLpartitions

– NonewoptionorkeywordneededonREORG

• RECOVERofsinglepartitionofa20partitiontablespace– ETreducedby28%

– CPUreducedby49%

52

EnhancementsintheMaintenanceStream• REORGLOGphaseperformanceimprovement

– APARPI32491

• SupportLOADSHRLEVELCHANGEPARALLELforPBGpagesets inV11– APARPI44674

• LOADutilityIGNORE(WHEN)option– AvoiddiscardprocessingforrecordsthatdonotsatisfyWHENclause

• APARsPI44415&PI44465

• AvoidrepeatedtapemountstoretrievecompressiondictionaryforIFI306reads– APARPI54885

• Reducedindexpagep-lockingindatasharingforREORGINDEX,REORGTABLESPACEpartlevel withNPSIs&RUNSTATSINDEX

– Example:RUNSTATSETcutby70%,CPUby25%,indexleafpagep-lockscutfrom2.6mtozero• APARPI46320

53

UtilitiesSummary• EnsurethatRUNSTATSaspartofREORGwasatleastcollectedunderV9

– IfnottheDataRepeatFactorwillbeoutlinedDRF=-1!Queryoptimisation!• ManyREORGEnhancements,e.g.

– NewSWITCHTIMEoption– Part-levelinlineimagecopywhenreorganizingasubsetofpartitions– REORGSORTDATANORECLUSTERYES|NO– Automatedbuildingofthemappingtablewithnew10byteLRBA/LRSN– AbilityofusingDRAIN_ALLPARTSYESoption– REORGREBALANCEsupportsSHRLEVEL(CHANGE)– SupportREORGofLOBdataeventhoughauxindexisunavailable– REORGofLOBbigperformanceimprovement

• ConsiderRUNSTATSandRTSEnhancements• LookforStatisticsandzIIP offload• FastLogApply(FLA)nowimplementedforRECOVERINDEX• PendingaltersandRecovertoPITenhancements• Fastercatalog/directoryrecoveryandFasterpartition-levelrecoveryfrominlineimagecopy

54

AVAILABILITY

55

BIND/REBIND/DDL/OnlineREORGbreakingintopersistentthreadrunningpackagesboundwithRELEASE(DEALLOCATE)

• PersistentthreadswithRELEASE(DEALLOCATE)whichwerepreviousblocking– e.g.,IMSPseudoWFI,CICSProtectedENTRYthreads,etc

• TypesofREORGswhichinvalidatedpackageswerepreviouslyblocked– REORGREBALANCE

– MaterializingREORG

• The'break-in'behaviorisONbydefault(ZPARMPKGREL_COMMIT=YES)

• Break-inisperformedona“bestefforts”basis

• Break-inmechanismcanhandleidlingthreadsatatransactionboundary(i.e.,wherecommitorabortisthelastthingperformed)

56

BIND/REBIND/DDL/OnlineREORGbreakingintopersistentthreadrunningpackagesboundwithRELEASE(DEALLOCATE)…

• Severalfactorscomeintoplayforasuccessfulbreak-in– PersistentthreadmustCOMMIT

– ThetimingoftheCOMMITandthefrequencyoftheCOMMITsarebothkey

– IncreasingtheZPARMforIRLMresourcetimeout(IRLMRWT)helpstokeeptheBIND/REBIND/DDL/OnlineREORGoperationwaitingtoincreasethechancesofasuccessfulbreak-in

• Thebreak-inmechanismdoesnotapplywhen– RunningpackagesboundKEEPDYNAMIC(YES),or

– OPENcursorsdefinedWITHHOLDatthetimeofCOMMIT,or

– IftheCOMMIThappensinsideastoredprocedure

• RELEASE(COMMIT)wouldalsonotbreak-infortheaboveconditions

57

BIND/REBIND/DDL/OnlineREORGbreakin- Howdoesitwork1. BIND/REBIND/DDL/OnlineREORGisinitiatedandwaitsonapackagelock

– Willtimeoutafter3xIRLMtimeoutlimit(IRLMRWT)

2. At1/2oftheIRLMtimeoutlimit,DB2willgetnotifiedbyIRLMthatsomeoneisstuckonapackagelock– IfDB2hasanS-holder,DB2willpostasystemtasktotakefurtheraction

3. DB2systemtaskisawakenedandcheckstoseeifa‘recycle’oflocallyattachedthreadshasbeendoneinthelast10seconds– Ifnot,thebreak-inoperationwillproceed– DB2istryingtoavoidabatteringofthesystemviaBIND/REBIND/DDL/OnlineREORG

4. SendbroadcasttoallDB2memberstoperformarecycleoflocallyattachedthreads5. Iftaskproceeds,itwillloopthroughalllocallyattachedthreads(notDIST!)andseeif

theywerelastcommitted/abortedin>1/2oftheIRLMtimeoutlimit– Ifso,theBIND/REBIND/DDL/OnlineREORGislikelywaitingonthem

6. ThenexttestistoseeifDB2candoanythingaboutit?– Eachthreadmustbeatatransactionboundary(i.e.,commitorabortisthelastthing)– Ifso,DB2canprocessthethread

58

BIND/REBIND/DDL/OnlineREORGbreakin- Howdoesitwork…7. DB2willfencetheAPIforthethread,grabtheagentstructureanddrivea‘dummy

COMMIT’– ThecommitistransactionallyOKsinceweareatatransactionboundary– DB2willbethecoordinatorasthisissingle-phasecommitandgetout– OntheCOMMIT,RDSseesthatthereisawaiterforapackagelockheldbythisagentandwill

switchtoRELEASE(COMMIT)forthiscommitcycle– ThelockisfreedandDB2isonestepclosertotheBIND/REBIND/DDL/OnlineREORGbreaking

in

8. Repeatforallqualifyingthreads9. BIND/REBIND/DDL/OnlineREORGshouldbreak-inprovidedtherearenoblockersthat

hadtobeexcludede.g.,longrunningreadonlyapplicationprocesswithoutacommit10. Iftheapplicationstartsusingthethreadduringtherecycleprocessing,itwillbeblocked

attheAPIlevel– DB2willspinthethreadinatimedwaitloopuntiltherecycleisdone– DB2willwaitamillisecondapproximatelybetweenpolls– DB2hasalsotakencaretofenceend-of-task(cancelapplicationTCB),end-of-memory(force

downthehomeASIDduringrecycle),associate,dissociate,etc

59

BINDBreak-in– Simplecustomertest

60

Time-----------à

Thread: T1 Treadreuse

BIND: BIND

1 transaction

One CICS ENTRY Thread

Break in

BIND waits approximately 30 sec before it breaks into an idle thread

30 sec is half the transaction time out interval

BINDbreakin– additionalcustomertesting

61

Action Threads Result

BIND BatchNoCommit Nobreak-in

BIND BatchfrequentCommit Break-in

BIND 50*CICS ENTRY Break-in

DDL:

CreateIndex CICS ENTRY Break-in

DropIndex CICS ENTRY Break-in

AlterTableAddColumn CICS ENTRY Break-in

Alterindex(NO)cluster CICS ENTRY Break-in

AlterTablespacetoUTS CICS ENTRY Break-in

AlterPartition CICS ENTRY Break-in

ALTERLIMITKEYenhancement• Behaviorisdifferentdependingonhowthetablepartitioningiscontrolled• Withtable-controlledtablepartitioning,thisisapendingalteration

– Droppingofthesealterscanoccuratanytime

• Withindex-controlledtablepartitioning– IfalterisdoneviaALTERINDEXALTERPARTITION

• Partitiongoesinto‘hard’reorgpending(REORP)!• Tablespaceremainsindex-controlled• Altercannotbewithdrawn!

– IfthealterisdonebyALTERTABLEALTERPARTITION• Ifthepartitionisnotempty

–Partitiongoesinto‘hard’reorgpending(REORP)!–Tablespaceisconvertedtotable-controlledpartitioning!–Altercannotbewithdrawn

• Ifthepartitionisempty

–Alterisexecutedimmediately–Tablespaceisconvertedtotable-controlledpartitioning

62

ALTERLIMITKEYenhancement…• TwonewZPARMsintroduced

– PREVENT_ALTERTB_LIMITKEY• ALTERINDEXALTERPARTITIONisstillpossible– donotuseitbecauseofREORP

• ALTERTABLEALTERPARTITIONleadstoSQLCODE-876

– PREVENT_NEW_IXCTRL_PART• Cannolongercreatenewindex-controlledpartitionedtablespaces

• MaterializingREORGcannowbreak-intoapersistentthreadrunningRELEASE(DEALLOCATE)package

• REORGREBALANCE– NotpossibleforpartitionswithpendingALTERLIMITKEYchanges

– Willworkfortheotherpartitions

– Willworkforpartitionswhichare‘hard’reorgpending(REORP)

63

DROPCOLUMN• Workswell

– CanconverttoUTSandconcurrentlyDROPCOLUMN

– MaterializingREORGcanberunatthepartitionlevel– ifallpartitionsarecovered

– Allpackagestouchingthetablewillbeinvalidated

• Restrictions– CannotuseDROPCOLUMNinclassictablespacetype(SQLCODE-650)

– Cannotdropacolumncontainedinanindexorview(SQLCODE-478)

– CannotaddadroppedcolumnbeforethematerializingREORG(SQLCODE-20385)

– Cannotcreateaviewwithadroppedcolumn(SQLCODE-20385)

– CannotdropthesamecolumnasecondtimebeforethematerializingREORG(SQLCODE-205)

– CannotunloadfromanimagecopytakenbeforethematerializingREORG(DSNU1227I)

– CannotrecovertoaPITbeforethematerializingREORG(DSNU556I)

64

AvailabilitySummary• ConsiderusingthenewDB2V11Bind,Rebind,DDLandonlineREORGbreakintopersistedthreadwithRELEASE(DEALLOCATE)mechanism

– Understandthebreakinbehavioranditsdesign

• BeawareofimplicationswithTablepartitioningcomparedtoIndexcontrolledpartitioningandALTERLIMITKEY

• DROPColumnfeatureandtherestrictions

65

PERFORMANCEANDSCALABILITY

66

PerformanceEnhancements- noREBINDneeded(CM)• DDFperformance improvements

– Reduced SRBscheduling onTCP/IPreceive using new CommServer capabilitiesimproved autocommit OLTPperformance

• xProcs above the bar

• zIIPenablementforallSRB-modeDB2systemagentsexceptP-Locknegotiation

• Avoid cross-memoryoverhead for writing logrecords

• Datadecompression performance improvement

• INSERTperformance– Latch contention reduction

– CPUreduction for Insertcolumn processing and logrecord creation

– Datasharing LRSNspin avoidance

– Pagefix/free avoidance inGBPwrite

67

PerformanceEnhancements- noREBINDneeded(CM)...

• Sort performance improvements

• DPSIperformance improvements for merge

• Performanceimprovements with largenumber of partitions

• XMLperformance improvements

• Optimize RELEASE(DEALLOCATE)execution sothat it is consistently betterperforming than RELEASE(COMMIT)

• IFI306filtering capabilities to improve QReplication capture performance

• Utilitiesperformance improvements

• Automatic index pseudodelete clean-up

• ODBC/JDBCType2performance improvements

• Javastored procedures

– Multithreaded JVMs,64-bitJVM– more efficient

68

PerformanceEnhancements– noREBINDneeded(CM)...

• ACCESSDATABASEcommand performance

• DGTTperformanceimprovement– AvoidincrementalbindsforreducedCPUoverhead

• P-procs for LIKEpredicates against Unicodetables

• Improved performance for ROLLBACKTOSAVEPOINT

• zEC12exploitation

• Datasharing performance improvements

69

PerformanceEnhancementsrequiringREBIND(CMwithorwithoutAPREUSE)

• MostIn-memorytechniques

• Noncorrelatedsubquerywithmismatchedlength

• Selectlistdo-once

• Columnprocessingimprovements

• RIDoverflowtoworkfile handledforDataManagersetfunctions

• Performanceimprovements for common operators

• DECFLOATdata typeperformance improvements

70

PerformanceEnhancementsrequiringREBIND(CMwithoutAPREUSE)

• Querytransformationimprovements– lessexpertiserequiredtowriteperformantSQL

• Enhancedduplicateremoval

• DPSIandpagerangeperformanceimprovements

• OptimizerCPUandI/Ocostbalancingimprovements

71

PerformanceEnhancements– DBAorapplicationeffortrequired(NFM)

• Suppress null-indexes

• NewPCTFREEFORUPDATEattributetoreduceindirectreferences

• DGTTperformanceimprovements

• Globalvariables

• Optimizerexternalizationofmissing/conflictingstatistics

• Extendedoptimization- selectivityoverrides(filterfactorhints)

• Opendatasetlimitraisedto200K

72

OptionalEnhancementsneedNFM+DBAeffort

• DSNTIJCB– Optional– ConvertBSDSforextended10-byteRBAs– - STOPDB2MODE(QUIESCE)

• DSNTIJCV– Optional– ConvertCatalog andDirectorytableandindexspacestoextended10-byteRBAformat

– ReorgsallCatalog andDirectorytablespacesSHRLEVELCHANGE

– Canbesplituptorunreorgsinparallel

73

ExampleofCustomerPerformanceTesting• DB210NFMbaseline

• DB211CMbeforeREBIND

• DB211CMafterREBIND

• DB211NFM(noneedforfurtherREBIND)

• DB211NFMafterREORG(tomigrateobjecttoextendedLRSN)

• DB211NFMExtendedLRSN

74

ExampleofCustomerPerformanceTesting...• MakesurethattheCPUnumbersarenormalizedacrossthoseintervalsi.e.,useCPUmillisecondspercommit

• EasytocombinestatisticsandaccountingbystackingthevariouscomponentsofCPUresourceconsumption:

– MSTRTCB/(commits+rollbacks)

– MSTRSRB/(commits+rollbacks)

– MSTRIIPSRB/(commits+rollbacks)

– DBM1TCB/(commits+rollbacks)

– DBM1SRB/(commits+rollbacks)

– DBM1IIPSRB/(commits+rollbacks)

– IRLMTCB/(commits+rollbacks)

– IRLMSRB/(commits+rollbacks)

– AverageClass2CPCPU*occurrences/(commits+rollbacks)

– AverageClass2SECPU*occurrences/(commits+rollbacks)

75

CICSTestTransactionProfile

Avg.DML• 3 Insert

• 7 Select

• 5 Open

•103Fetch

Avg.Bufferpool•65 Getpages•13 SyncRead

CPUconsumption•Class1: 3.2msec

•Class2: 2.3msecWithCP-Speed802MIPS

76

Fetch Intensive

• Transaction types– E-Banklogon

– Balancecheck

– FinancialStatementhistory

– Accountsearch

CICStransactionCPUtime

77

0,00

0,50

1,00

1,50

2,00

2,50

3,00

3,50

10NFM 11CM 11RBND 11NFM 11Reo 11EL

CPUmsec

DB2ASzIIPperTRAN

DB2ASGCPperTRAN

CL2CPUperTRAN

5% Activity moved to zIIP

In CM

DB2systemaddressspaceCPUperCICStransaction

78

0,00

0,10

0,20

0,30

0,40

0,50

0,60

0,70

10NFM 11CM 11RBND 11NFM 11Reo 11EL

CPUTimemSec

IRLMSRB

IRLMTCB

DBM1zIIP

DBM1SRB

DBM1TCB

MSTRzIIP

MSTRSRB

MSTRTCB

Activity moved to zIIP

in CM

TestBatchjobprofile

DML• Commits:4528

• Delete: 37613• Rows:482836

• Update: 45099

• Select: 119773

• Insert: 525884

• Fetch: 548947

79

Bufferpool• Getpage: 4.8M

• SyncRead: 133K

• DynPref: 57K

CPU• Class1: 69Sec

• Class2: 67SecWithCP-Speed802MIPS

Elapsed:• Class1: 07:02min

Insert and Delete Intensive

BatchtestCPUtime

80

0,00

20,00

40,00

60,00

80,00

100,00

120,00

10NFM 11CM 11RBND 11NFM 11Reo 11EL

CPUSec

DB2ASzIIPperBATCH

DB2ASGCPperBATCH

CL2CPUperBATCH

LRSN spin loop eliminated

in V11 E LNo credit to V11

For V11 Reo improvement

DB2systemaddressspaceCPUforBatch

81

0,00

5,00

10,00

15,00

20,00

25,00

10NFM 11CM 11RBND 11NFM 11Reo 11EL

CPUsec

IRLMSRB

IRLMTCB

DBM1zIIP

DBM1SRB

DBM1TCB

MSTRzIIP

MSTRSRB

MSTRTCB

Activity moved to zIIP

In CM

LRSN spin loop eliminated in 11 EL

reduces MSTR SRB

AutomaticPseudoDeletedIndexEntryClean-up• Recaponimpactofpseudodeletedindexentries

– Indexsizegrowswithincreasingnumberofpseudo-deletedindexentries• Moregetpagesandlockrequestsrequired• IncreasedCPUcostandpossiblylongerelapsedtimesforaccessviaindexsearch

– ApplicationsmayencounterdeadlocksandtimeoutsduringINSERT/UPDATE/DELETE• Collisionswithcommittedpseudo-deletedindexentries• RIDreusebyINSERTfollowingDELETE=>deadlock

• PriortoV11,howaretheycleanedup– DB2removespseudo-deletedentriesduringmainlineoperations

• Insert/deleteoperationsremovepseudo-deletedentriesfromindexpages• SQLrunningwithisolationlevelRRremovespseudo-deletedentries

– Pagesthatonlycontainpseudo-deletedindexentriesarecalledpseudo-empty• DB2attemptstocleanuppseudo-emptyindexpagesaspartofDELETEprocessing

– REORGINDEXremovespseudo-emptyindexpagesandpseudo-deletedentriesthatwerenotcleanedupbythemainlineprocessing

82

AutomaticPseudoDeletedIndexEntryClean-up…• AutonomicsolutionprovidedinCMandturnedonautomaticallyforallindexes24*7

– Automaticclean-upofpseudo-deletedindexentriesinindexleafpages– Automaticclean-upofpseudo-emptyindexpages– DesignedtohaveminimalornodisruptiontoconcurrentDB2work– Clean-upisdoneundersystemtasks,whichrunasenclaveSRBsandarezIIP eligible– Parentthread(oneperDB2member)loopsthroughRTStofindcandidateindexes– Childclean-upthreadsonlycleanupanindexifitalreadyisopenedforINSERT,

UPDATEorDELETEontheDB2member• AvoidcreatingGBPdependencyonindexes

• Potentialdisruptioncanbeminimizedbymanagingdownthenumberofclean-upthreadsorspecifyingtimewhenindexesaresubjecttoclean-up

– Cancontrolthenumberofconcurrentclean-upthreadsordisablethefunctionusingZPARMINDEX_CLEANUP_THREADS

• 0=Disable,1-128,10isdefault

– EntriesinnewCatalogtableSYSIBM.SYSINDEXCLEANUP• Definewhen/whichobjectsaretobeconsideredinagenericway

83

AutomaticPseudoDeletedIndexEntryClean-up• Upto39%DB2CPUreductionpertransactioninDB211comparedtoDB210

• Upto93%reductioninPseudodeletedentriesinDB211

• ConsistentperformanceandlessneedofREORGinDB211

84

0

500000

1000000

1500000

2000000

2500000

0

0,0005

0,001

0,0015

0,002

0,0025

0,003

0,0035

Day1 Day2 Day3 Day4 Day5#o

fpseud

o_de

leteden

tries

CPUtime(sec)

WASPortalWorkload5DaysPerformance

V10TotalCPUtime

V11TotalCPUtime

V10sumofREORGPSEUDODELETES

V11sumofREORGPSEUDODELETES

PerformanceEnhancements• Qreplication logfiltering

– ReduceIFIlogreadcostbyqualifyingtheobjectswithDBID/PSID

– Additionalbenefitifobjectsarecompressed

– MovefilteringfromQreplication capturetasktoDB2engine

– Potentialforverysignificantreductioninthenumberoflogrecordsreplicated

– RequiresIBMInfosphere DataReplicationQReplicationorChangeDataCapture10.2.1

• ArchiveTransparency

– Veryusefulnewfeature

– NeedtocarefullyexaminetheadditionalcostofORDERBYsortwhenaccessingthearchive

– Whenapplicationonlyfetchesalimitednumberofrowsfromtheresultsetthenthecostcanincreasesignificantlywhenalsoaccessingthearchive

– Willtypicallybeusedbycustomersselectivelyonacasebycasebasis

85

zIIPExpansion• NotexpectedtoassistasmuchasV10,butcanprovideincreaseuse• Upto100%ofprocessingforasynchronousenclaveservicerequestblocksthatexecuteintheDB2ssnmMSTR,ssnmDBM1andssnmDIST addressspaces.

– Prefetch (V10)– Deferredwrite(V10)– Logwriteandread– GBPwriteandcastout– Indexpseudodelete/XMLcleanup– DPSIparallelaccess(PARAMDEG_DIPSI)

• ExpandedRUNSTATSandLOADredirect– EnclaveSRBtasksforLOAD,REORG,andREBUILDINDEXInlineStats– EnclaveSRBtasksforRUNSTATSColumnGroupDistributionStats– EnclaveSRBtasksforLOADREPLACEindexmanagement

• IIPHONORPRIORITY=NO– Disablesallsystemagent(includingprefetch)redirect

86

P-LockNegotiation NOT zIIP Eligible

SynchronousReceive- Allremoteapplications…

• PreDB211DDFusesasynchronousreceive,whichrequiresexpensiveSRBswitchwhendatabecomesavailable

– ClientconnectionterminationsoDDFcancancelthread• DB211DDFwillexploitsynchronousreceive

– AvailableinCMwithoutapplicationchangesorREBINDsrequired– z/OSCommunicationsServerprovidesnewTerminationNotification– DDFwillbenotifiedofanysocketfailure,evenwhileinsyncreceivemode– DDFcanabortthreadsrunningsynchronously– PreliminarymeasurementsusingthedistributedIRWW*suite

• 6%to8%ITR**improvement• 4%to6%CPUimprovement

• Onlyaffectsnon-storedprocedureworkloads

87

*IBM Relational Warehouse Workload**Internal Throughput Rate

CancelthreadandInterruptstatement…• PreDB211DDFCANCELTHREADmaynotworkinsomecases

– Historically"soft",orreactiveinnature

– Canceldetectionpointmaynotbeencounteredinatimelymanner

• DB211NewFORCEkeywordonCANCELTHREADcommand– Purges thread ofaremoteconnectionintheDB2

– FORCE optionacceptedonlyafterCANCEL THREADissuedwithoutthe FORCE option

– AvailableinCM

• PurgerunningDDFSRBusingnewz/OSpurgeSRBfunction– Requiresnewz/OSCALLRTMTYPE=SRBTERMservice

• Availableinz/OS1.13orlaterviaAPAROA39392

88

WithSyncReceiveandIDTHTOIN• (DB211)- NowwhenidlethreadtimeoutishitDDFmustissueTCPIP.DROPcommandtokillthesocketassociatedwiththethread

– Customersawtimeoutsandhadtomanuallycancelthreadstoletbatchbreakin..– SodidyouseeIDTHTOINpopinthelog,whataboutinthepreviousrelease?

• 00D3003BinthelogforthreadsthathitIDTHTOIN

• IfthreadsareremaininginthesystemlongerthanonDB210,andtheidlethreadsarenotbeingcanceled(causingtimeoutsorcontentionwithotherprocesses),thenMVS.VARY.TCPIP.DROPOPERCMDSmissing

– GetDSNL512I-111RC=77E800DC(EACCES/JRSAFNotAuthorized)

• ProcessisdescribedinsettingupDDFandUNIXsystemservicessectionoftheinstallationguide

– http://www-01.ibm.com/support/knowledgecenter/SSEPEK_11.0.0/com.ibm.db2z11.doc.inst/src/tpc/db2z_enableddf4uss.dita?lang=en

• PI06325– messageDSNL512Iisenhancedtoshowsocket=EZBNMIF4_DROPCONtoalertyouthatservicefailed

– http://www-01.ibm.com/support/docview.wss?uid=swg1PI06325

89

BufferPoolenhancements…• BPReclassification

– DB29/DB210• PagesremainclassifiedasprefetchevenafterrandomI/Oagainstpage

– DB211• Prefetchedpagescanbereclassifiedasrandomifrandomgetpageagainstpage

• PotentiallylesssyncI/Osformixedaccess(prefetch+random)

– Mayneedtore-evaluateVPSEQTsettingforcertainbufferpools

– VPSIZEMINandVPSIZEMAXattributesforAUTOSIZE(YES)bufferpools• Setahardupperandlowerlimitforautonomicsizing

90

BufferPoolenhancements• OnDemandStorageAllocation

– DB29• EntirebufferpoolallocatedusingVPSIZEforvirtualandrealstorage

– DB210• Bufferpoolallocatedondemand

– Ifaquerytouchesonlyafewdatapages,onlyasmallamountofvirtualandrealbufferpoolstorageisallocated

–CouldcauseperformanceproblemswhentryingtoquicklyaddmorepagesforadditionalGETPAGEs

– DB211• SimilartoDB29andprior

• EntireVPSIZEofbufferpoolvirtualstorageallocatedwhenbufferpoolbecomesactivebutonlyenoughrealstorageforcurrentGETPAGEs

• OvercomesDB210performanceissuewithoutusingrealstorageifnotnecessary

91

BPSimulation(NFMmode)• AbilitytosimulatechangingVPSIZEandVPSEQT

– AssumesLRUmechanisminplace

• Storagecostforasimulatedbufferpoolislessthan2%for4KpagessimilartoHiperpoolexceptnotallocatingactualbuffers

– Simulationisdonewithaseparatebuffersearchhashtablewithbuffercontrolblocks

– For4KpagesizeBP,thestorageoverheadis2%ofSPSIZE*4K(e.g.20MBfor1GBsizebufferpool)

– CPUoverheadisbetween1-3%whenrunninglab.Workloads

– ShowsupinDSNB431I-432I,andOMPE(IFCID002)

92

SelectiveDecompression• Compression/Decompressionisthesinglemostexpensiveinstructioninz/OS

• DB2usesrowlevelcompression,henceeachrowwascompletelydecompressedwhenitwasaccessed

• V11usessoftwareinstructionstoaugmenttheHWdecompression– AvailableinCMbutpartialrowdecompressionneedsarebind(RRFformat)

93

Otherperformancerecommendations• MakesureHVCOMMONinIEASYSxx canaccommodatelogoutputbuffer

• Configureadditional1MBLFAREA(z/OSparameterinIEASYSxx)formaximumbenefit

• LRSNspinavoidancerequiresbothBSDSandobjectsconversioninNFM

• MonitorlogI/Operformanceduetologrecordsizeincrease– 3%to40%increaseinlogrecordsizeobservedfollowingBSDSconversion

• EssentialtomakesureenoughzIIP capacityavailablebeforeV11CMmigration– zIIP ‘HelpFunction”IIPHONORPRIORITYshouldbesettoYESincasethereisashortageofzIIP capacity

– ContinuetomonitorzIIP capacitythereafter

• MRU(MostRecentlyUsed)usedforpagesbroughtinbyutilities

• NewFRAMESIZEparameterindependentfromPGFIXparameter– IfyouPGFIXafterV11FRAMESIZE1MisneededtouseLFAREA

94

PerformanceSummary• Opportunityforimprovedperformanceforlegacyapplicationprograms

• REBINDofstaticSQLpackagesisveryimportant

• GoodvalidationofpotentialfromESPcustomersandIBMinternalworkloads

• YourmileagewillvarybasedonyourSQLapplicationworkloadascertainfeaturesonlyapplytocertainworkloads

• ImpressiveCPUsavingsobservedforsomeworkloads

• HighlyoptimizedstaticSQLand/orsimpleSQLmaynotseemuchbenefit

• MorebenefitformorecomplexSQLi.e.,notreadasinglerowbyprimarykey

• Donotsell(orbuy)thesavingsbeforeyouhaveseenthemforyourworkload

• BestthingforCPUperformanceonentrytoV11CMistorebindfrequentlyusedpackages(re-enableSPROCs,newxPROCs,avoidpuffing).

9595

REALSTORAGEMANAGEMENT

96

Agenda• LargeMemoryValue

• RealandVirtualConcept

• Runningsafely

• RealStorageManagementControls

• ReportingandMonitoring

• Bufferpool ConsiderationsandLFAREA

• Summary

97

Largerealmemoryvalue• SubstantiallatencyreductionforOLTPworkloads

– Significantresponsetime reductions

– Increasedtransaction rates

• In-memorydatabasesfordramaticgainsinresponsetimesbyavoidingIOwaits

• Batchwindow reductions

– Improvesinglethread performance

– Canprocessdatamoreefficientlyhelpingorganisations keeppacewithinfluxof data

– Reducingtimetoinsightforanalytic workloads

– Reducestimeittakestogetbusinessinsightfromraw data

• CPUperformance improvements

– ReducesCPUresourceconsumptionper transaction

– z/OSlabmeasuredcostofI/Oat20us-70usofCPU

– Improvesresponsetimesandshrinksbatch windows

– Reducetheneedforapplicationand/orsystemredesigntomeetservice goals

98

DB2forz/OSandLargeReal Memory

99

“Memoryischeaporone time charge, CPUsare expensive”“ForeveryI/Othatyousave,youavoidthesoftwarechargefor

theCPUthatittooktootherwisedothat I/O”

77

CPUCostSavingbyReducingDB2SyncI/Os• Banking(60Maccount)workloadwith2waydatasharing:

• 11%responseand6%CPUreductionfrom52GBGBPto398GBforbothmemberswithsameLBPsize(60GB)

• 40%responseand11%CPUreductionfrom30GBLBPto236GBLBP forbothmemberswithsamereasonableGBPsize(60GB)

100

Summary• SyncI/OreductiontiescloselywithCPUreduction

– Thiswasdoneinz/OS2.1,zEC12,DB211

• Forthisparticularworkload,thehigherbenefitoninvestingonlocalbufferpools– NotalltheobjectsbecomeGBP-dependent

– Largetablespaces definedwithMemberCluster

– Mostofaccesspatternisfairlyrandom

• GBPhastobebigenoughtosupportenoughdirectoryentries

• ThebestperformerwaswithbothlargeLBPandlargeGBP

101

IBMBrokerageWorkload– EnlargingLocalBuffer Pool

102

l 10->24GB14%CPUreduction,3%Throughputimprovement(26->13sync I/O)l 24 ->70GB 3%CPU reduction,15% Throughputimprovement (13->8sync I/O)

• Globaldynamicstatementcache– EDMSTMTCupto4GwithDB211,default110MB

– ReductionofCPUtimebyavoidingfullprepare

• Localstatementcache– MAXKEEPDupto200KstatementswithDB211,default

5000

– ReductionofCPUtimebyavoidingshortprepare

• In-memorydatacacheforsparseindex– MXDTCACHupto512MBperthread,default20MB

– ReductionofCPUandelapsedtimewithpotentiallybetteraccesspathselectionwithDB211

• RIDPool(MAXRBLK)– LowerCPUtimeifRIDlistsdon’tspillintoworkfiles

• DB2localandgroupbufferpools– ReductionofelapsedtimeandCPUtimebyavoiding

I/Os

– PGSTEAL(NONE)inDB210=Inmemorydatabase

– CPUreductionfromPGFIX=YESandlargepageframes

• ThreadreusewithIMSorCICSapplications

– ReductionofCPUtimebyavoidingthreadallocationanddeallocation

• ThreadreuseandRELEASE(DEALLOCATE)

– ReductionofCPUtimebyavoidingpackageallocationandparentlocks

– DDFHighperformanceDBATssupportwithDB210

– Abilitytobreak-inpersistentthreadwithDB211

DB2 Areas that Benefit from Larger Memory

103

REALSTORAGEMANAGEMENTANDCONTROLS

104

8

DB2pushedouttoAUXstorage• AnypagingofDB2isbad

– PagingaddstodirectCPUcostandhurtsresponsetime

– FlashExpressmemoryhelpsmitigatetheproblem,butdoesnotsolveit

• DB2dumpswilltakelongerifz/OShastoreadpagesinfromAUXstorageorFlashExpressmemory,impactingtheavailabilityofDB2system

– Heldlocksmagnifiestheproblem,causingsympathysicknessonotherLPARsinthesysplex

• DFSORTmayconsumeallavailablerealmemoryandpushDB2framesout– RiskofworkloadspikesoccurringwhileDFSORTisrunning

• Donotoversizeyourlocalbufferpools– Ifyouregularlyseepaging,reducethesizeofyourbufferpools(andotherDB2memorypools)

– UsingPGFIX(YES)andlargesizeframessavesCPUtime,butbecarefultomakesureyouhaveenoughrealmemorysothatthereisnopaging

105

Whataffectsthestorage?• 64-bitprivate

– Mostlybufferpools

– Fixed-RIDpool

– EDMpool

• 64-bitsharedthreadandsystem– UsedtoexecuteSQL

– Variable-dynamicstatementcaches,CT,PT,SKCT,SKPT

– *Comparetonumberofsystemanduseragents/threads,parallelism

• 64-bitstack– JustasinV9,usedbythreadtoexecuteSQL

– *same

• 64-bitcommon– Distributedagents,packageaccounting,rollup

• AllareaffectedbyREALSTORAGE_MANAGEMENT

106

MonitoringREAL/AUXstorageusage– Mappingforreference

IFCIDFIELD OMPEFIELD OMPEPDBCOLUMNNAME MEMU2Description

QW0225RL QW0225RL REAL_STORAGE_FRAME DBM1REALinusefor31and64-bitpriv (MB)

QW0225AX QW0225AX AUX_STORAGE_SLOT DBM1AUXinusefor31and64-bitpriv (MB)

QW0225HVPagesInReal SW225VPR A2GB_REAL_FRAME DBM1REALinusefor64-bitpriv (MB)

QW0225HVAuxSlots SW225VAS A2GB_AUX_SLOT DBM1AUXinusefor64-bitpriv (MB)

QW0225PriStg_Real SW225PSR A2GB_REAL_FRAME_TS DBM1REALinusefor64-bitpriv w/oBP(MB)

QW0225PriStg_Aux SW225PSA A2GB_AUX_SLOT_TS DBM1AUXinusefor64-bitpriv w/oBP(MB)

QW0225RL QW0225RL DIST_REAL_FRAME DISTREALinusefor31and64-bitpriv (MB)

QW0225AX QW0225AX DIST_AUX_SLOT DISTAUXinusefor31and64-bitpriv (MB)

QW0225HVPagesInReal SW225VPR A2GB_DIST_REAL_FRM DISTREALinusefor64-bitpriv(MB)

QW0225HVAuxSlots SW225VAS A2GB_DIST_AUX_SLOT DISTAUXinusefor64-bitpriv(MB)

QW0225ShrStg_Real SW225SSR A2GB_SHR_REALF_TS REALinusefor64-bitshared(MB)

QW0225ShrStg_Aux SW225SSA A2GB_SHR_AUXS_TS AUXinusefor64-bitshared(MB)

QW0225ShrStkStg_Real SW225KSR A2GB_SHR_REALF_STK REALinusefor64-bitsharedstack(MB)

QW0225ShrStkStg_Aux SW225KSA A2GB_SHR_AUXS_STK AUXinusefor64-bitsharedstack(MB)

QW0225ComStg_Real SW225CSR A2GB_COMMON_REALF REALinusefor64-bitcommon(MB)

QW0225ComStg_Aux SW225CSA A2GB_COMMON_AUXS AUXinusefor64-bitcommon(MB)

QW0225_REALAVAIL S225RLAV QW0225_REALAVAIL REALAVAIL(MB)(S)

107

Note:AllREAL/AUXstoragefieldsinIFCID225andOMPEperformancedatabaseareexpressedin4KBframesorslots– theyshouldbeconvertedtoMB(conversionisalreadydoneinMEMU2)

RealStorageManagementandControls…• MakesureMAXSPACEissetproperlyanddefensively

– RepresentsthetotalamountofstorageforcaptureddumpsfortheentireLPAR– MAXSPACEvalueshouldnotbesetsohighthatpagingcanoccurcausingmassiveissuesto

theLPAR– IfmultipleDB2sonsameLPARcanwildcardtothesamedump,thenMAXSPACEneedsto

besetappropriately

• MAXSPACE=16Gisagoodstarttocopewithmorethan90%ofallcases– MAXSPACEat16GmaybetoobigforsmallLPARs/DB2subsystems.

• LookatsizeofexistingdumpsandcutbackonMAXSPACE.

• MAXSPACErequirementshouldbe– (DBM1– Bufferpools)+Sharedmemory+DIST+MSTR+IRLM+COMMON+ECSA

• z/OSparameterAUXMGMT=ON– NonewdumpsareallowedwhenAUXstorageutilizationreaches50%– CurrentdumpdatacapturestopswhenAUXstorageutilizationreaches68%– Oncethelimitisexceeded,newdumpswillnotbeprocesseduntiltheAUXstorage

utilizationdropsbelow35%

108

RealStorageManagementandControls…• DB2pagespagedoutandadumphappens?:

• Whatismyexposure?– IncreasedMASTERCPUtime,z/OStriestostealframestomeettheexcessivedemandcausedbythedump

– Elongateddumptimes

• Auxiliarystoragenumber>0– Intheoryeachslotcouldhavetobepagedin

• z/OScanhavethepageinbothplaces,REALandAUX

• Iftotalsizeacrossallbufferpools ismorethan800MBthenthebufferpools arenotdumped

– Noprefetch onAUXstorage,soallI/Os aresynchronous

– Worstcaseisthenumberofpages*page-inI/Otime• Forexample2GBof4Kpages*3ms=524288*0.003=26mins

– Guinnessworldrecordforadump37mins

– FullSysplex hangresulted

109

RealStorageManagementandControls…• SYSPLEXsympathysickness:

• Slowdownandnoapparentreasonwhy?

• DB2takingthedumpmayhaveTCBsnon-dispatchable

• P-Locknegotiationaffected

• Locksnotreleasedinatimelymanner

• ExcessivedumptimecausedbypagingontheLPARmaycausemassivesympathysicknessslowdowns

• HowcanamemberslowdownwhenthereisplentyofCPU/StorageontheLPAR– MaybetheownerofaP-lockisbeingdumpedorispagingalot

110

TheDUMPeffect– notransactionsbeingprocessed

111

0

1000

2000

3000

4000

5000

6000

7000

NUM_ENDED NUM_ARRIVED

One Flash Express Card

CPUnanos

Flash -microseconds

Disk -milleseconds

Access Time

Ifyouhavenochoicebuttopage– useFlashExpress

• IfthereisnotenoughspareREALmemoryontheLPARtoavoidpaging;considerprovisioningFlashExpress

– FlashExpressisaPCIe IOadapterwithNANDFlashSSDs

– Usedtodeliveranewtierofmemory- storageclassmemory

– SizedtoaccommodateallLPARpaging• Eachcardpairprovides1.4TBusablestorage

• Maximum4cardpairs(4X1.4=5.6TB)

• FLASHExpresshelps1MBpageable frames• LargepageswillbepagedouttoFLASH

• Retain‘largepage’attributewhenpagedbackin

• VeryimportantifyouhaveHyperswap andz/OSCRITICALPAGING=YES

– HVSHAREandotherstorageareasprotectedfrompaging

• AlsousefulifMAXSPACEcannotbebackedwithREAL

112

RealStorageManagementandControls…• ControluseofstoragebyDFSORT

– SetEXPMAXdowntolimitmaximumDFSORTusage

– SetEXPOLD=0topreventDFSORTfromtaking"old"framesfromotherworkloads

– SetEXPRES=%orn{reserveenoughforMAXSPACE}

113

0

5000

10000

15000

20000

25000

30000

2013-07-29-00.02.00.000161

2013-07-29-04.08.00.000157

2013-07-29-08.14.00.000209

2013-07-29-12.20.00.018877

2013-07-29-16.26.00.000163

2013-07-29-20.32.00.000178

2013-07-30-00.38.00.000166

2013-07-30-04.44.00.000904

2013-07-30-08.50.00.016955

2013-07-30-12.56.00.000175

2013-07-30-17.02.00.000175

2013-07-30-21.08.00.000310

2013-07-31-01.14.00.059911

2013-07-31-05.20.00.019601

2013-07-31-09.26.00.000123

2013-07-31-13.32.00.000499

2013-07-31-17.38.00.000158

2013-07-31-21.44.00.000342

2013-08-01-01.50.00.370531

2013-08-01-05.56.00.093954

2013-08-01-10.02.00.130884

2013-08-01-14.08.00.000155

2013-08-01-18.14.00.000274

2013-08-01-22.20.00.015079

2013-08-02-02.26.00.000102

2013-08-02-06.32.00.000152

2013-08-02-10.38.00.004328

2013-08-02-14.44.00.003906

2013-08-02-18.50.00.036134

2013-08-02-22.56.00.000143

2013-08-03-03.02.00.000214

2013-08-03-07.08.00.000181

2013-08-03-11.14.00.000159

2013-08-03-15.20.00.014000

2013-08-03-19.26.00.000364

2013-08-03-23.32.00.000201

2013-08-04-03.38.00.000168

2013-08-04-07.44.00.000197

2013-08-04-11.50.00.033578

2013-08-04-15.56.00.000174

2013-08-04-20.02.00.000151

(MB)CBP4- REALstorageavailable

RealStorageSummary• UseREAL_STORAGE_MANAGEMENT=AUTO

• KeeptheLPARwellprovisionedwithREALstorage

– Haveenoughfor4Kandusewhatisleftfor1M!

• AvoidPagingtoAUXorFlashExpress

• WatchoutforMAXSPACEandlargedumpsthatmaycausesystemtopage

• Use‘commoncurrency’forreportingandmonitoringREALandAUXusage

• ItisimpossibletoknowhowmuchrealstorageDB2isaactuallyusinguntiltheLPARstartstopage(z/OSRSMinstrumentation).

• CPUperformanceadvantageusingREALSTORAGE_MANAGEMENT=NO/AUTO

– REALSTORAGE_MANAGEMENT=NOonlyifrealmemoryisverygenerouslyallocatedontheLPAR

– REALSTORAGE_MANAGEMENT=AUTOisdesignforeffectivethreadreuseforhighvolumetransactionstomitigateCPUoverheadofdiscard

114

EXTENDEDLRBA/LRSN

115

ExtendedLRBA/LRSN• WhatyouneedtoknowforDB211CMandDB210NFM?

– 6byteformat- LRBA/LRSNbeforeDB211è x‘LLLLLLLLLLLL’

– 10byteextendedformat– LRBA/LRSNhasaddressingcapacityof1yottabyte(2**80)• 10byteextendedformat- LRSNwithDB211 è x‘00LLLLLLLLLLLL000000’

• 10byteextendedformat- LRBAwithDB211 è x‘00000000RRRRRRRRRRRR’

– WheredowefindLRBA/LRSN?• DB2Catalog è SYSCOPY,SYSxxxxPART,…..

• DB2Directory è SYSUTILX,SYSLGRNX,….

• BSDS è pointer,Active&ArchiveLogvalues..

• DB2Logs è active&archivelogs

• DB2Pagesets è Catalog&Directoryandalluser-pagesets

116

ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211CMandDB210NFM?…

– DB211CM• DB2internalcodingdealswith10byteextendedformatLRBA/LRSNvaluesonly

• LRSNinUtilityoutputisshownin10byteextendedformatwithprecision‘000000’ except– QUIESCEutility,whichexternalizesLRSNin10byteextendedformatwithprecision‘nnnnnn’

• RECOVERutilityhandles10byteextendedformatLRBA/LRSNinput

• Column‘RBA_FORMAT’inSYSIBM.SYSxxxPART issetto‘B‘fornewdefinedorobjects,whicharereorged orloadedwithreplace-option(possiblevalues‚B,blank‚U,E)

– DB211CM/DB210NFMcoexistenceindatasharing• Fulltolerationof10byteextendedformatLRBA/LRSNvalueasinputtotheRECOVERUtility

• Sanitychecksincludedfor‘wronglyused6byteformatLRBA/LRSN’

117

ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211NFM?

– MigrationtoDB211NFM(viaDSNTIJEN)• Catalog&DirectoryTable‘LRBA/LRSNColumns’arealteredto10byteextendedformat

• SYSIBM.SYSLGRNXentriesarenowstoredas10byteextendedformatLRBA/LRSNvalues

• SYSIBM.SYSCOPY– ConversionofallLRBA/LRSNvaluesisdoneforexistingdatato10byteextendedformatwith

leadingbyte‘00’andprecision,‘000000’forLRSNandrightjustifiedrightwithleading‘00000000’forLRBAvalues

– Newdataisstoredin10byteextendedformatwithprecision‘nnnnnn’

• LRBA/LRSNforallUtilitiesusenow10byteextendedformat

• LRBA/LRSNvaluesarestillwrittentoDB2logsin6byteformat

• LRBA/LRSNvaluesarestillwrittentoDB2pagesetsin6byteformat

118

ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211NFM?…

– BSDSconvertedto10byteextendedformatLRBA/LRSNinNFMonly(DSNJCNVT)• ThereisnowaybackforBSDS!

• NowLRBA/LRSNvaluesarewrittentoDB2logsofthesubjectDB2membernowin10byteextendedformatwithprecision‘nnnnnn’

• LRBA/LRSNvaluesarestillwrittentoDB2pagesetsin6byteformat– Conversion(10to6or6to10byte)hastobedone

– LRSNSpincanstillhappen

– DSN1LOGPandREPORTRECOVERoutputwillshow10byteextendedformatLRBA/LRSNalthoughneverexternalizedtopagesets(differentoutput,forDSN1PRNTofpagesets)

• Canbedone,wheneveryouwanttodoitafterentrytoV11NFM,regardlessofpagesetformats

119

ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211NFM?…

– ReorgCatalogandDirectorypagesetsto‘extendedformat’(inNFMonly!)• Canbedonewheneveryouwantto,regardlessofBSDSanduserpagesetformats

• NowLRBA/LRSNvaluesarewrittentoconvertedpagesetsin10byteextendedformat– LRSNwithprecision‘nnnnnn’,ifupdateisdoneonaDB2memberwith10byteextendedformat

BSDS

– LRSNwithprecision‘000000’,ifupdateindoneinamemberwith6byteformatBSDS

• Column‘RBA_FORMAT’inSYSIBM.SYSxxxPARTisupdatedto‘E’

• LRSNSpincouldstillhappenforDB2memberwith6byteformatBSDS

• Canbeconvertedbackto6byteformat(alloratpartlevel)

120

ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211NFM?…

– ReorgUserpagesets to‘extendedformat’(inNFMonly!)• Canbedonewheneveryouwantto,regardlessofBSDS,Catalog&Directorypageset

formats

• NowLRBA/LRSNvaluesarewrittentoconvertedpagesets in10byteextendedformat– LRSNwithprecision‘nnnnnn‘,ifupdateisdoneinamemberwith10byteextendedformatBSDS

– LRSNwithprecision‘000000‘,ifupdateisdoneinamemberwith6byteformatBSDS

• Column‘RBA_FORMAT’inSYSIBM.SYSxxxPART issetto‘E‘

• LRSNSpincouldstillhappenforaDB2memberwith6byteformatBSDS

• Canbeconvertedbackto6byte(alloratpartlevel)

• IsdonebyREORG,LOAD..REPLACEorREBUILDwith‘RBALRSN_CONVERSIONEXTENDED’orifZPARMOBJECT_CONVERTED=EXTENDED

• ‘RECOVER...TOCOPY...’usinga6byteCopycanresetformatbackto‘basic’

121

ExtendedLRBA/LRSN…• Enhancementstoimproveusabilitycharacteristicsbasedon6byte/10byte

formatLRBA/LRSNhandling– PreventDSNJCNVTfromconvertingDB210NFMBSDStoextendedformat– Support10byteextendedformatinputtoRECOVERinDB210– PerformsanitycheckstoguardagainstinvalidLRSNvaluesi.e.,6byteLRSNvalues

withleadingbyteofzeros,topreventPITrecoveriesusingbadRBA/LRSNfromfailing(RC=8inUTILINITphaseinstead)

– SanitycheckalsoperformedinDB210(coexistence)– Supportfor‘NOBASIC’valueforOBJECT_CONVERSIONZPARMtopreventconverting

backpagesets inextendedformat,andto‘EXTENDED’asdefaultif‘NOBASIC’issetandcatalogcolumnis<>‘E’

– AddLRSNvaluestoarchiveloginformationinREPORTRECOVERYutilityoutput– Technicalwhitepaperproducedexplainsabout‘6/10byteLRBA/LRSNhandling’– SeveralenhancementstoDB211books

122

ExtendedLRBA/LRSN…• Recommendedbestpracticemigrationstrategy

1. Runpre-migrationjobsandstepstoclean-up2. MigrationtoDB211CM3. MigrationtoDB211NFM4. ConvertALL BSDSofdatasharinggroupwithin‘n’weekends5. ReorgALL Directory&CatalogPagesets to‘extendedLRBA/LRSNformat’6. SetOBJECT_CREATEandUTILITY_CONVERSIONZPARMstoEXTENDED

- Newobjectswillbecreatedin10byteextendedformat- REORG,LOADREPLACEandREBUILDwillconvertuserobjectstoextendedformatwithout

needtochangeutilitycontrolstatements

7. ReorgallobjectstoextendedLRBA/LRSNformatbyexecutingnormalreorgjobsorsomeadditionaljobs• PerformregularcheckforongoingprogressbyselectingrowswhereRBA_FORMAT=‘E’in

SYSIBM.SYSxxxxPART

8. Ifalldone,setOBJECT_CONVERSIONZPARMtoNOBASIC

123

ExtendedRBA/LRSN– Conversionstrategies

124

DB211CMDB210NFM DB211NFM ConvertBSDS

DataSharing– EndLRSNproblem

ConvertDB2Objects

DB211CMDB210NFM DB211NFM ConvertDB2Objects

NonDataSharing– EndlogRBAproblem

ConvertBSDS

DB211CMDB210NFM DB211NFM ConvertBSDS

DataSharing– EndlogRBAproblemonlyononeormoremembers

• ConvertingtheBSDSimplicitlyconvertstherecoverylogtoextendedformat.

ConvertingtheBSDSupfrontwillprovideincrementallybenefitfromLRSNspinavoidanceasobjectsaregraduallyconvertedtoextendedformatbyREORGutilities

LeaveconvertingoftheBSDSuntiltheveryendwillavoidadditionalloggingacceleratingtowardstheendoflogRBA

ExtendedLRSNSummary• DistinguishbetweenendofLogRecordSequenceNumber(LRSN)andendofLogRelativeByteAddress(RBA)

• WithDB2V11internalcodingdealswith10byteextendedformatLRBA/LRSNvaluesonly

• MonitortheboundariesofLRBAandLRSNvaluesandtakeactionifneeded.– DifferentchoiceswithRBAlimits,butonlyoneforendofLRSN

• NoticethatallDB2objectsofagivenneedtoREORGEDtogetconvertedto10ByteLRSN

• FollowbestpracticeshaveasolidmigrationstrategyandmigrationplanforRBAandLRSNconversionto10Byte.

• CPUreductionfromavoidingLRSNspinisatmostcustomerinstallationnotthatsignificant.Thereshouldbeabalance,e.g.

– IfnotenforcedbyapproachingLRSNand/orRBAwhygoingthroughprocess of converting DB2 Cat/Dir and application objects to extended format.

125

OVERALLSUMMARY

126

OverallSummary• Sharelessonslearned,surprises,pitfalls

• Providehintsandtips

• Addresssomemyths

• Provideadditionalplanninginformation

• Provideusageguidelinesandpositioningonnewenhancements

• Helpcustomersmigrateasfastaspossible,butsafely

127

DB211Resources• IBMInformationCenter/KnowledgeCenter• DB211TechnicalOverviewRedbook(SG24-8180)• DB211forz/OSPerformanceTopics(SG24-8222)• DB211links:https://www.ibm.com/software/data/db2/zos/family/db211/

– LinkstoDB211AnnouncementLetter,webcastsandcustomercasestudies– IBMDB211extendedLRSNandRBAformatsupportenhancesavailabilityandaddresses

businessrisk– Whitepaper:“DB211forz/OS:UnmatchedEfficiencyfor

BigDataandAnalytics”– Whitepaper:“HowDB211forz/OSCanHelpReduce

TotalCostofOwnership”

• DB211MigrationPlanningWorkshop– http://ibm.co/IIJxw8

• FreeeBookavailablefordownload– http://ibm.co/160vQgM

• “DB211forSAPMissionCriticalSolutions”– http://scn.sap.com/docs/DOC-50807

128

129