archiving wf work item
TRANSCRIPT
Archiving of Workflow Work Items
David Weston - Clockwork
Tony Gramelspacher – Anhueuser-Busch
Marcel Landry – New Brunswick Power
William Reilly - IBM
© Copyright 2000
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 - 2 7 , 2 0 0 0W e s t i n I n n i s b r o o k R e s o r t a m p a F l o r i d a
A M E R I C A S ’ S A P U S E R S ’ G R O U P ©Copyright 2000
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Data Archiving: General Principles Types of Work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch– IBM– NB Power
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Data Archiving Process
Archive Files
Business Transactions /Reporting
Read Archived Data
Data Objects
Delete from Database
Database
Application Data
Archive to Archive Files
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
What is an Archiving Object?
SARA Customizing
Programs
Data
1010101011101011000110110010000110101000111010101001101011100011000010111010110110011000111000101001011010101011
WORKITEM
ArchivingObjects
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Storage Alternatives for Archive Files
Database
SAP System
Storage System
File System
Archive Files
Tape
ApplikationsserverApplikationsserverApplikationsserverApplikationsserver
Application ServerApplication Server
SAP ArchiveLink /Content Management
Service (CMS)
SAP ArchiveLink /Content Management
Service (CMS)
HierarchicalStorage
Management(HSM)
HierarchicalStorage
Management(HSM)
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving Work items
Data Archiving: General Principles Types of Work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA for
archiving object WORKITEM SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch– IBM– NB Power
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Types of Work items Work item types to be archived:
– W - Work item with dialog. Runtime representation of single-step tasks that require interaction with the user and runtime representation of user decisions
– D - Missed deadline. Work item for notification of missed deadline– A - Work Queue. A work queue is a list of objects to be processed once and
together in a limited time frame.– F - Workflow. Runtime representation of a multistep task.– B - Batch item. Runtime presentation of a single-step task that runs in the
background– P - Remote work item – Work item that represents a remote work item (proxy
WI)– N - Notification item – no longer used.– E - Wait step work item. Runtime representation of a wait step in the workflow
definition– C - Container anchor. This type of work item is required to enable the links
between the SAP business applications and an IDoc. It does not normally appear in the workflow environment. Work items of this type should be regularly deleted or archived.
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Handling of Type C Work items
Type C work items are not archived and deleted using the WORKITEM archiving object using SARA.
Up to release 4.6A Type C work items form the link between the SAP Business Applications and the IDoc e.g. Purchase Order, Invoice etc.
Type C work items ONLY must be deleted separately using a new ABAP called RSWWCIDE as the existing deletion ABAP called RSWWWIDE is too slow. See SAPNet note 153205 for the ABAP source code. The RSWWWIDE report deletes all types of work item.
As of Release 4.6 links are stored in table IDOCREL. Entries can be deleted from this table using report RSRLDREL. The report is available as of Release 4.6B.
See SAPNet notes : 49545, 126678, 149367, 153205
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Data Archiving: General Principles Types of Work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA for
archiving object WORKITEM SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch– IBM– NB Power
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
When to archive ?, when to delete work items ?
Deletion of work items– Deletion of work items is carried out using ABAP report RSWWWIDE or
transaction SWWL.– Delete the work item history using report RSWWHIDE. RSWWHIDE
deleted a work item's history, even if the work item still exists (you can see the work item but not its history). RSWWWIDE can also delete the work item history by selecting the appropriate selection screen parameter.
– Should only be run in NON-PRODUCTION R/3 systems. – Should only be run for type C work items, even better is to run report
RSWWCIDE as explained previously which only deletes type C work items.
– Work items should really be archived in production systems using the archiving object WORKITEM using transaction SARA except for those of type C.
– Archived work items cannot be reloaded into the production system.
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
When to archive ?, when to delete work items ? (RSWWWIDE)
Select work items by:– Type– Task– Status (Error,
Completed etc)– Agent
Choose test run and delete log file option
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Data Archiving: General Principles Types of work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA for
archiving object WORKITEM SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch– IBM– NB Power
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving Object WORKITEM
For archiving object WORKITEM in SARA:– Customizing– Network Graphic –
application dependencies
– Job Overview– DB Tables– Information System– Archive Object
Management
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Customizing in SARA
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
DB Tables (DB15)
Space info for tables Online space Space statistics
Which archiving object(s)for selected DB table(s)?
Which tables arearchived?
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Work Item Archiving Steps
In transaction SARA:– ‘Archive’
Create a variant to create the archive files. Execute report RSWWARCA.
– ‘Delete’ Choose the archive run data to delete. Select a start date and the spool parameters. Execute report RSWWARCD.
– ‘Analyze’ Execute report RSWWARCR to read and display work items from the
archive.– Once work items are archived you can use transaction SWW_ARCHIV
(RSWWARCP) to read work items from the archive, starting from an object or task
– THERE IS NO RELOAD ARCHIVING REPORT !! Next steps: Archive the associated R/3 business application data
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
General Rules The system archives all data that belongs to a work item and that is not purely
runtime data. Work items should not be archived without an application reference.
– Therefore, in a production system not only the work items but also the application archiving objects associated with these work items should be archived.
– The data archived includes log data, workflow manager data, and work items that depend on another work item (for example, the work items in a workflow or the lines of a work queue).
– The objects in the container of a work item are archived as references only. The same applies to the work item attachments. The attachments themselves are not archived because attachments can be objects outside R/3, such as Microsoft Word files. Archiving is not currently supported for such data.
– When the archived work items are deleted, the attachments (not archived) are also deleted.
You can only archive work items that have one of the following statuses:– Completed – Logically deleted (cancelled)
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Data Archiving: General Principles Types of work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA for
archiving object WORKITEM SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch– IBM– NB Power
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
SAPNet Notes
– 49545 - Deleting unnecessary work items– 67366 - Deletion of work items is incomplete– 76431 - Measures for optimizing the upgrade runtime 4.x– 117191 - Work item log is not deleted from archiving– 126678 - Archiving or deleting work items type C– 133943 - Update termination when deleting a work item– 145291 - Performance of deletion from archive file– 149367 - IDoc: Link and archiving – 153205 - Direct deletion of type C work items – 159065 - Data selection for work item archiving– 180683 - Work items in Subworkflows are not deleted – 182204 - Short dump when archiving work items– See also the standard upgrade notes in application area BC-UPG-RDM
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Data Archiving: General Principles Types of work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA for
archiving object WORKITEM SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch– IBM– NB Power
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Data Archiving: General Principles Types of work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA for
archiving object WORKITEM SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch – Tony Gramelspacher– IBM – William Reilly– NB Power – Marcel Landry
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items at Anheuser-Busch
OSS note research and application– Test – Review OSS for updates
Security profiles– Opened security for all archive objects– Separated security by archive object, i.e. WORKITEM
Retention period of work items– Identified stakeholders– Attained user agreement on retention period– Strived for one period for all work items (60days)
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items at Anheuser-Busch
Testing
– Development test Functional verification Identify security issues
– Consolidation (system) test Simulate production run Estimate production performance
– Plan carefully Walk through procedures step by step to avoid duplicate runs Use small amounts of data - once it’s gone, it’s gone! Separate archive and delete steps Use alternative system test environments
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items at Anheuser-Busch
User sign offs– Educated stakeholders on what will be archived
Only the work item is purged Work item purge does not remove the business object, i.e. actual
purchase orders or invoices are not removed– Attained sign off from stakeholders via accept/reject email
Clean ups / purges– Identified end users with large inboxes
Select from table V_ORG_WI Communicate importance of executing “notification” work items
to end users Completed/canceled work items either manually or through
custom program
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items at Anheuser-Busch
Clean ups / purges (continued)– Executed before setting up recurring archive jobs– Eliminated 3.5 years worth of work items– Recovered 17.6 GB of disk space– Reorganized SWW… tables at the database level to fully
recover space– Noticed workflow runtime improvement
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items at Anheuser-Busch
Most Critical Issues / Lessons Learned
– Attaining retention period agreement from stakeholders
– Auditing point - purging data from system for good, no way to restore work items
– User involvement - explanation of work items, status of a work item - in progress, reserved, completed, canceled, etc.
– Purging only work items, not the business objects associated with the work item, i.e. P/Os, Invoice, Idoc
– SARA work item purge program limitation Internal table filling up due to volume of data being purged SARA jobs would terminate Had to reduce the deletion period down to a day in some cases
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Data Archiving: General Principles Types of work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA for
archiving object WORKITEM SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch – Tony Gramelspacher– IBM – William Reilly– NB Power – Marcel Landry
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Incentives needed to start Work Item Archiving Started with “C” Type Work Items (IDOC related Work Items) Gathered Business Retention Requirements for Workflow Work
Items Archiving Workflow Work Item Procedures Lessons Learned, Problems encountered
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Incentives needed to start Work Item Archiving
Performance problems, especially with IDoc processing Runtime Utilization Table status (SWUF) showing large number of
records - SWW_CONTOB > 45M Could not perform upgrade to SAP 4.5B due to large table sizes Users Inboxes (especially those of administrators) too full to be useful,
or to read in some cases
Better be a good reasonfor this!
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Started with C Type Work Items
Ran RSWWCIDE to ‘catch up’ with old C type work items
– OSS Note 153205 for 3.1 - 4.5
– Much faster than RSWWWIDE
– Can run multiple jobs in parallel, separated by From / To Dates or Work Item Ids
– uncheck “DISP” when deleting After ‘catch up’, linked to existing IDoc
archiving strategy
– OSS note 126678 to 3.1 system; already in 4.x
– When Archive IDoc then related C type work item set to COMPLETED
– Ran RSWWWIDE and RSWWHIDE on C type work items with status COMPLETED
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Gathered Retention Reqs. for WF Work Items Needed retention requirements for the following areas which utilized SAP
Workflow– Material Management– Pricing– Order Management– Credit Management– Export Regulations– Contracts– IDOC Error Handling (IT support)– Billing– Scheduling
Potential of variant / batch job for each areas’ Workflow(s) Most functional areas wanted 3-6 months retention period due to work
item reporting and Inbox retention Exception was IDOC Error Handling
– 1 month or less retention - Not as much use for history after IDOC errors resolved
– Occasional specific runs to remove large number of work items due to large number of related IDOC errors
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving Workflow Work Item Procedures
Decided not to run SARA to archive work items Setup batch job and variant for RSWWWIDE; our variants setup
as follows:– ID of work item (default)– Type of work item = W, B, F, D, A, E (and P for 4.x)– Status of work item = Error, Committed, Completed, Cancelled– Date created / Creation date = desired retention period, e.g.
current date - 90, current date– Time created / Creation time (default)– Executive agent, ID of task = <blank> (default)– No. of WIs to be delet. Per LUW = 50 (default)– In initial test runs, can run with “Display list only” checked to
ensure correct work items list to be deleted; after that remove this flag to run deletion
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving Workflow Work Item Procedures (conted)
Setup batch job and variant to run RSWWHIDE– Will remove the Work Item Log History of work items that were
previously deleted (tables: swwlogpara, swwloghist)– ID of work item = 1 to 999999999999; Uncheck “Display list only”
Run re-organization of runtime table indexes– See OSS Note 72873 for table indexes– Run after each archiving run, performance gains are realized here
‘Ad-hoc’ deletion of work items– Use RSWWWIDE (and then RSWWHIDE) to delete specific Tasks’
and / or Workflows’ work items (e.g. TS00008068, WF550000062)– Can end specific Workflow or Tasks work items (“end manually” or
“Logically delete”) via SWI2 (work item reporting) or Inbox– Deleting of WF or WS work items will delete the lower level WF,
WS, T and TS work items
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Lessons Learned, Problems encountered
Have a procedure to cleanup the Work Items early, do not wait for performance problems
If have heavy IDoc traffic, will have large number of C type work items
Attaining agreement for deletion procedures and attaining the retention period from functional areas proved to be most difficult obstacles
Decide on whether to archive work item data– Can not re-load deleted work items– Determine requirements for data retention and access– SARA selection criteria does not allow for ‘Type of work item’
or ‘Status of work item’– Archiving can be performance intensive– Enhance example report (RSWWARCR) which reads
archived work items for more useful information if decide to archive
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving work items
Data Archiving: General Principles Types of work items When to archive ?, when to delete work items ? Work item Archiving process using transaction SARA for
archiving object WORKITEM SAPNet Notes Screencam Demo Customer Experiences:
– Anheuser Busch – Tony Gramelspacher– IBM – William Reilly– NB Power – Marcel Landry
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Environment at NB Power
– Electric Utility: 300,000 customers.– 1997 - SAP R/3 Rel 4.0B FI/CO/MM/SD (No Workflow).– 1999 - IS-U/CCS 1.2 Add-on module + workflows.– OS / DB: Windows NT / Microsoft SQL Server– Users: 900+ – Majority of workflows originate from IS-U/CCS module.– Average volume: 2000 new workflows / day.– First archiving exercise almost one year after ‘go live’.
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Issues Encountered
– Workflow and container tables growing at rapid pace.– WF Table monitoring (swuf) showing yellow lights on most
tables.– Neglect - Always saw archiving as ‘something to do later”… – No policy in place for WI’s - had to fast track our approach.– Ownership of WF data not well defined.– ‘Archiving’ misunderstood by client - actually purging.– Client concerned about impact on WIS. – Large volume of ‘stale’ workitems in place as a result of initial
data creation. – Had used WI deletion program to manage size of WF inbox -
Ooops!
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Preparation / Testing
– Applied OSS notes to improve archiving performance.– Educated client on WI log data and it’s usage - not impacted.
No business documents removed as result of WI archiving.– Enabled security for object Workitem only.– Confirmed that WIS in not impacted by archiving - tested in DEV
environment.– Agreed on data retention period of 8 weeks from completion
date. (client uses WF logs for investigation of WF errors)
– Volume tested on QA system (Prod Copy) - measured performance.
– Ensured that enough space was available for creation and storage of archive files.
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Archiving - The Results
– Initial archiving run on weekend outside of batch window.– Limited size of archive files to 25 Mb each (config).– Archived one month at a time (limit impact in case of failure)– Timing:
Archive selection and identification: 6 hours. Deletion of identified records: 2 hours
– 1.5Gb of space required to store all archive files (60+).– Total of 16,2 Million records removed from WF tables.– Less data archived than expected
...many stale “in process” WI’s.– Many workitems of type ‘C’ in tables
...not removed by archiving.
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Ongoing Activities and Future Steps
– Archiving being done manually bi-monthly.– Have report to change status of stale WI to
‘cancelled” Will allow archiving to pick up old items..
– Setting-up batch job for archiving on weekly basis - small volumes.
– Planning the removal of Type ‘C’ workitems.– Developing strategy for archiving of IDOCS.
2 0 0 0 A S U G B u s i n e s s W o r k f l o w F a l l M e e t i n g O c t o b e r 2 5 – 2 7 , 2 0 0 0 ©Copyright 2000
Thanks !