© 2012 IBM Corporation
Using Shadow Jobs to Manage Cross-Platform Dependencies
© 2012 IBM Corporation2
Workload Automation
Agenda
• Introduction• Cross dependencies flow• Binding process• Shadow job status changes• Setup• Demo / Q&A
© 2012 IBM Corporation3
Workload Automation
Introduction (I)• Workload is sometimes spread across different scheduling
environments, often located in different sites Scheduling environments managed independently Scheduling environments need to interoperate
• Using cross dependencies Synchronize activities between different scheduling environments
© 2012 IBM Corporation4
Workload Automation
Introduction (II)• Job B (Local Engine) logically dependent on Job A (Remote Engine)• Job A’ (on Local Engine) represents Job A (on Remote Engine)• Local Engine
Tivoli Workload Scheduler Distributed Tivoli Workload Scheduler for z/OS (Session 3603)
• Remote Engine Tivoli Workload Scheduler Distributed Tivoli Workload Scheduler for z/OS
© 2012 IBM Corporation5
Workload Automation
Cross dependencies flow
Shadow Job on the local engine is free from dependencies
© 2012 IBM Corporation6
Workload Automation
Cross dependencies flow
Broker of the local engine sends binding request to the remote engine
© 2012 IBM Corporation7
Workload Automation
Cross dependencies flow
Broker of the remote engine sends binding response to the local engine Broker of the local engine updates the status of the Shadow Job
© 2012 IBM Corporation8
Workload Automation
Cross dependencies flow
Remote Job starts running Broker of the remote engine sends the status update to the local engine Broker of the local engine updates the status of the Shadow Job
© 2012 IBM Corporation9
Workload Automation
Cross dependencies flow
Remote Job completes successfully Broker of the remote engine sends the status update to the local engine Broker of the local engine updates the status of the Shadow Job Successor of the Shadow Job is released
© 2012 IBM Corporation10
Workload Automation
Binding request (remote engine = TWSd)<jmgr:JobDefinitionDocument>
<jsdl:jobDefinition>
<jsdl:application name="distributedShadowJob">
<dshadow:DistributedShadowJob>
<dshadow:Timezone>Europe/Brussels</dshadow:Timezone>
<dshadow:StartOfDay>2300</dshadow:StartOfDay>
<dshadow:JobStream>JS1</dshadow:JobStream> <dshadow:Workstation>WS1</dshadow:Workstation> <dshadow:Job>XYZ</dshadow:Job> <dshadow:matching> <dshadow:sameDay/> </dshadow:matching> <dshadow:ScheduledTime>2012-04-12T13:00:00Z</dshadow:ScheduledTime> <dshadow:SchedDate>2012-04-12T00:00:00Z</dshadow:SchedDate>
</dshadow:DistributedShadowJob>
</jsdl:application>
<jsdl:resources/>
</jsdl:jobDefinition>
</jmgr:JobDefinitionDocument>
<jmgr:Alias>TWS3_REM_ENG#JOBS.TEST_L1636311662.JNUM-560798319</jmgr:Alias>
<jmgr:JobId>2f39b341-0ec6-311c-a9b7-6f27e33a2229</jmgr:JobId>
<jmgr:ClientNotifyURI>https://tws1.swg.be.ibm.com:31116/JobManagerRESTWeb/JobScheduler/job</jmgr:ClientNotifyURI> <jmgr:ClientNotifyURI>https://tws2.swg.be.ibm.com:31116/JobManagerRESTWeb/JobScheduler/job</jmgr:ClientNotifyURI>
© 2012 IBM Corporation11
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
© 2012 IBM Corporation12
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
If a matching job stream instance in found in the Preproduction Plan
© 2012 IBM Corporation13
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”
© 2012 IBM Corporation14
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”
Look for the job in matching job stream instance in Current Plan
(Example : XYZ )
© 2012 IBM Corporation15
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”
Look for the job in matching job stream instance in Current Plan
(Example : XYZ ) If found
Send INFO notification back to the local engine (Shadow Job = BOUND).
© 2012 IBM Corporation16
Workload Automation
Binding process (remote engine = TWSd)
© 2012 IBM Corporation17
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”
Look for the job in matching job stream instance in Current Plan
(Example : XYZ ) If found
Send INFO notification back to the local engine (Shadow Job = BOUND). Else
Send ERROR notification back to the local engine (Shadow Job = ERROR).
© 2012 IBM Corporation18
Workload Automation
Binding process (remote engine = TWSd)
© 2012 IBM Corporation19
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”
Look for the job in matching job stream instance in Current Plan
(Example : XYZ ) If found
Send INFO notification back to the local engine (Shadow Job = BOUND). Else
Send ERROR notification back to the local engine (Shadow Job = ERROR). Else
Send INFO notification back to the local engine (Shadow Job = BOUND).
© 2012 IBM Corporation20
Workload Automation
Binding process (remote engine = TWSd)
© 2012 IBM Corporation21
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”
Look for the job in matching job stream instance in Current Plan
(Example : XYZ ) If found
Send INFO notification back to the local engine (Shadow Job = BOUND). Else
Send ERROR notification back to the local engine (Shadow Job = ERROR). Else
Send INFO notification back to the local engine (Shadow Job = BOUND). Else
Send ERROR notification back to the local engine (Shadow Job = ERROR).
© 2012 IBM Corporation22
Workload Automation
Binding process (remote engine = TWSd)
© 2012 IBM Corporation23
Workload Automation
Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”
If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”
Look for the job in matching job stream instance in Current Plan
(Example : XYZ ) If found
Send INFO notification back to the local engine (Shadow Job = BOUND). Else
Send ERROR notification back to the local engine (Shadow Job = ERROR). Else
Send INFO notification back to the local engine (Shadow Job = BOUND). Else
Send ERROR notification back to the local engine (Shadow Job = ERROR).
• Else Delay notification to the local engine (Shadow Job remains in WAIT ).
© 2012 IBM Corporation24
Workload Automation
Binding request (remote engine = TWSz)<jmgr:JobDefinitionDocument>
<jsdl:jobDefinition>
<jsdl:application name=“zShadowJob">
<zshadow:zShadowJob>
<zshadow:Timezone>Europe/Brussels</zshadow:Timezone>
<zshadow:JobStream>APP1</zshadow:JobStream> <zshadow:JobNumber>002</zshadow:JobNumber>
<zshadow:matching> <zshadow:previous/> </zshadow:matching> <zshadow:ScheduledTime>2012-04-15T15:00:00Z</zshadow:ScheduledTime>
</zshadow:zShadowJob>
</jsdl:application>
<jsdl:resources/>
</jsdl:jobDefinition>
</jmgr:JobDefinitionDocument>
<jmgr:Alias>TWS4_REM_ENG#JOBS.TEST_L1636311663.JNUM-560799234</jmgr:Alias>
<jmgr:JobId>2f39b341-0ec6-311c-a9b7-6f27e34r2229</jmgr:JobId>
<jmgr:ClientNotifyURI>http://tws1.swg.be.ibm.com:31115/JobManagerRESTWeb/JobScheduler/job</jmgr:ClientNotifyURI> <jmgr:ClientNotifyURI>http://tws2.swg.be.ibm.com:31115/JobManagerRESTWeb/JobScheduler/job</jmgr:ClientNotifyURI>
© 2012 IBM Corporation25
Workload Automation
Binding process (remote engine = TWSz)• If “CP end time” < “Scheduled time of Shadow Job” ≤ “LTP end time”
Search Long Term Plan Interval searched : “CP end time” < x ≤ “Scheduled time of Shadow Job”
© 2012 IBM Corporation26
Workload Automation
Binding process (remote engine = TWSz)• If no instances are found in the LTP interval
Search Current Plan Interval searched : x ≤ “Scheduled time of Shadow Job”
© 2012 IBM Corporation27
Workload Automation
Binding process (remote engine = TWSz)• Search Long Term Plan
Matched APP1 found in LTP interval Shadow job set to BOUND
© 2012 IBM Corporation28
Workload Automation
Binding process (remote engine = TWSz)• Search Current Plan
Matched APP1 found in CP interval and it contains 002 Shadow job set to BOUND
© 2012 IBM Corporation29
Workload Automation
Binding process (remote engine = TWSz)• Search Current Plan
Matched APP1 found in CP interval and it contains 002 (but deleted) Shadow job set to SUCC
© 2012 IBM Corporation30
Workload Automation
Binding process (remote engine = TWSz)• Search Current Plan
Matched APP1 found in CP interval and it contains 002 Shadow job set to BOUND
© 2012 IBM Corporation31
Workload Automation
Binding process (remote engine = TWSz)• Search Current Plan
Matched APP1 found in CP interval but it does not contain 002 Shadow job set to ERROR
© 2012 IBM Corporation32
Workload Automation
Binding process (remote engine = TWSz)• Search Current Plan
No match found in CP interval Shadow job set to ERROR
© 2012 IBM Corporation33
Workload Automation
Binding process (remote engine = TWSz)• Search Current Plan
Instances APP1 deleted in LTP interval Shadow job set to BOUND
© 2012 IBM Corporation34
Workload Automation
Shadow job status changes
© 2012 IBM Corporation35
Workload Automation
Setup• Remote Engine
Tivoli Workload Scheduler Distributed: Nothing Tivoli Workload Scheduler for z/OS
Update ROUTOPTS in controller PARMLIB
© 2012 IBM Corporation36
Workload Automation
Setup• Local Engine
Define Remote Engine workstation Remote Engine = Tivoli Workload Scheduler Distributed
Remote Engine = Tivoli Workload Scheduler for z/OS
© 2012 IBM Corporation37
Workload Automation
Setup• Local Engine
Define Shadow Job Remote Engine = Tivoli Workload Scheduler Distributed
Remote Engine = Tivoli Workload Scheduler for z/OS
© 2012 IBM Corporation38
Workload Automation