archiving @ dhlotmsig.communities.oaug.org/multisites/otm/media/euro_2014/c14e-08... · 9 archiving...

ARCHIVING - PURGING IN DHL Krishna Sabbineni & Anand Vasudevan Xchanging/DHL March, 2014

© 2013 Xchanging plc

AGENDA

26/03/2014 Archiving @ DHL – An insight 2

Migration Strategy to new version

Implementation & Design Components

Q&A Session

DHL Background & OTM Usage

Options Explored, Timeline & Scope

Post Go-Lives Issues & Challenges

Web Solution Screens

Archiving Solution Design

Business Case & Challenges

© 2014 Xchanging Plc

© 2013 Xchanging plc 26/03/2014 Archiving @ DHL – An insight 3

WHERE WE USE OTM WITHIN DPDHL?

Archiving implemented for this OTM setup in DGF



BUSINESS CASE

!  Requirement to archive data after six months.

!  Ability to do a track and trace on purged data.

!  Ability to run reports on archived data.

!  Minimal impact to the Production environment

!  Solution to be fast & highly reliable

26/03/2014 Archiving @ DHL – An insight 4 © 2014 Xchanging Plc


OTM @ DGF

Xchanging offers the below OTM servcies:-

Key services are:

!  Order Management & Visibility

!  Optimized Transports

!  Improved Performance

!  Traceability & Alert Management

!  Individual care

!  X-Dock and Merge In Transit

!  Data Archiving and Reporting

SCI (OTM) tool is the key application to support the service delivery towards the customers. SCI needs to be fully embedded into the current business and to provide efficient services to the customer.


•  OTM Implementation •  Custom Development •  24 x 7 – Support and

Maintenance •  Version Upgrade


DHL DATA VOLUMES

!  Average Inbound EDI Messages per day (approx.) –  Shipment Updates – 3,400

–  Inbound Orders – 10,000

–  Transportation Requests – 3,000

–  Inbound EDI Transactions - 80,000

–  Outbound EDI Transactions – 4,200



CHALLENGES

!  Single database for multiple customers hence Rate of Data accumulation is high

!  Huge amount of data gets generated which needs periodic purging

!  Sensitive Production environment – Any performance degradation causes Business Impact

!  Compatible with later OTM versions

!  OTM Migration planned – but to which version?

!  Maintain Data Integrity

!  Provide a web interface that is close to the existing SCI branding



PRESSURE POINTS


"  Existing customers demand archiving and reporting and even linked it to contractual obligations

"  Customers and users request a high performing application e.g Cust A requests a response time within 4 seconds

"  Future strategic business development clearly demands for the Service

"  Flexible and customisable reporting capability for dedicated customers

"  Current workarounds to keep data longer than 6 months are very complex to manage

"  Performance issues due to heavy workload on the operational database

"  Bad data quality caused by inconsistent purging

"  No structured and synchronous information exchange/replication

! Need for one Archiving

environment



OPTIONS EXPLORED

!  Traditional Approach: Using Standard Oracle functionality to create a Real-time Operational Schema

– Recycling primary keys

–  Performance issues

– Migration

–  Front-end will be an OTM one but needed a leaner front-end

–  Archive solution should work seamlessly during version migration where migrations are done in batches



ETL TOOLS CONSIDERED – GENIO/GOLDEN GATE & CUSTOM BUILD SCRIPTS

Key Factors which influenced the selection of GG

!  Robustness of the solution

!  Complexity of Set-Up/Maintenance

!  Prior experience of some of the tools

!  Compatibility/Support with OTM

!  Availability of Resources



TIMELINE & SCOPE

!  Data Archiving Solution

!  Web Front End to view data


SCI Version Migration Completed February 2014

Archive Migration Completed January 2014

Archive Migration Initiated August 2013

Final Solution Completed (5.5) June 2012

Proof of Concept Signed off: 28 August 2011

BRS and first cost proposals: Signed off: 30 July 2011

Initial talks and presentations conducted in May 2011

SCI Version Migration First customer on 6.3 June 2013



SCI – BUSINESS OBJECTS FOR ARCHIVING

!  Order Base

!  Order Releases

!  Order Movements

!  Shipments

!  Invoices

!  SKU



SOLUTION DIAGRAM - OVERVIEW



DESIGN – LOGICAL COMPONENTS

!  Replicated Online Database – ROD

!  Non-OTM Front End

!  Golden Gate - allows operating, managing and controlling data processing on each server

!  Golden Gate Director - a stand alone application orchestrating all Golden Gate components spread across source, target and any other intermediate servers. It is a single point to Administer and Monitor SCI Archive data processing



ARCHIVING TOOL - GOLDEN GATE

!  Provide close to real time data replication for ROD database.

!  Does not cause performance impact on OTM production database. Golden Gate uses transaction logs for data extraction from the source database, so does not need to run any queries on production data.

!  Perform online data transformation before loading into Archive databases. Transformation can be performed on target database server



REPLICATE OPTIONS


Top Option Chosen due to the size of database



IMPLEMENTATION – DATA SIZE

!  Daily Redo Data size – 15-30 GB/day

!  TMS Database size = 320 GB (approx.)

!  Initial Data Load and REPLICAT Option chosen

!  Golden Gate Director – Basic Monitoring –  View process information and statistics, including data latency between source and

target databases

–  View GoldenGate process log

–  Configure e-mail notification about alerts



DESIGN – DATA FLOW DIAGRAM



WEB SOLUTION SCREENS – LOGIN PAGE



WEB SOLUTION – SEARCH CRITERIA

!  Search window popup –  Search window popup comes when clicked against any field

!  Search criteria help – 

–  Begins With : Searches for records with field text that appears at the beginning of the text string.

–  Ends With : Searches for records with field text that appears at the end of the text string.

–  Is Null : Searches for records with field text that is blank.

–  Not Null : Searches for records with field text that not blank.

–  One Of : Finds a series of specific values.

–  Not One Of : Eliminates a series of specific values from the search.

–  Contains : Searches for records with field text that appears anywhere within a text string.

–  Same As : Searches for records with field text that is an exact match.

–  Not Same As : Eliminates field text to be considered in the search.



WEB SOLUTION- SHIPMENT SEARCH

!  Search Shipment –  Enter any search criteria for Shipment and click ‘Enter’

–  Multiple Tabs (Shipment, Order Release, Status, Advanced) on which the search criteria can be applied



WEB SOLUTION – SHIPMENT VIEW

!  View –  Click on the hyperlink of a any particular Shipment in the Result screen which need to be

viewed in detail



MIGRATION FROM 5.5 TO 6.3

!  Parallel Project for TMS Migration 5.5 - > 6.3 with a finish line of 6-8 months

!  Not all customers use Archive solution

!  Current TMS stores data for 6 months data

!  Customers connect to ROD for data older than 6 months



POST GO-LIVE SUPPORT

!  Periodic monitoring of the application by one of the monitoring technique provided by the Golden Gate application tool

!  Random checks to ensure that replication is being done correctly

!  Cross-checking selected data for key transaction tables – Orders, Order Release or Shipment tables

!  Counting the number of row of random or key tables



ISSUES AFTER GO-LIVE / CHALLENGES

!  Old Shipment updates being received – OTM creates a new record which means a duplicate entry in Archive database

!  Foreign Key Constraints – Locations or Contacts

!  Missing Archived Logs in case of any break in Archiving. The rate of Archive ReDo logs creation is quite high and the restriction of disk space means backing up to tape drives for later retrieval is complex

!  Experienced Golden Gate Resources



NEXT STEPS

!  Include all OTM modules - all set of tables

!  Include the SCI VIS Solution database

!  Include the BI reports to be run from BI Publisher



XCHANGING OVERVIEW

27

Industry Sectors that we Serve

Our Core Principles Our Global Presence

Servicing our customers

What We Are

Xchanging provides business processing, technology and procurement services internationally for customer across multiple industries.

What We Do

Xchanging brings innovation, thought leadership and passion to its customers’ business so as to enhance performance and value. Our Values are embedded into everything we do.

What We Want To Be

Xchanging wants to be regarded as the best provider in its chosen markets by delivering services that are recognised for outstanding quality, reliability and innovation.

8,000+ employees Customers in 48 countries

* Upcoming site

*

Aerospace & Defence

Automotive Banks & Financial Services

Energy & Utilities FMCG

Food & Beverage

Healthcare Insurance Manufacturing Public sector

Real Estate Retail Logistics Tele-communications

Transportation, Travel & Leisure


TECHNOLOGY SERVICE OFFERINGS

28

Development to Production •  Product Requirements Analysis & Definition •  Product Architectural Design, Detailed Design, •  Product Quality Management, •  Test Strategy, Test Automation, System & User testing, •  Release Planning, Product deployment, Acceptance

Application Implementation •  Implementation of enterprise application

software such as Oracle Apps, OTM, SAP •  Workflow / database management systems •  End to end Industry specific suites •  ERP, BPM, EAI and b2bi software

Business Consulting / Strategy •  Product strategy, Consulting, Business requirements, •  Applications rationalization, Technology evaluation, •  System Architecture, Product Roadmap, budgetary

assessment, •  Proof of Concept, Business validation

Support & Infrastructure services •  L1/L2/L3 support, Backlog reduction, •  IV & V •  Application Performance Management, Customizations,

Upgrade, migration, security, patch & Incident Mgt

Genius IRIS

ELGAR Brokasure NetCommex

TESA XMM

CRONUS

Application Development •  Requirements Analysis & Definition •  High Level Design (HLD), Low Level Design (LLD) •  Build and Unit Testing •  Test Strategy, Test Automation, System testing, User

Acceptance testing & Support •  Deployment, Go Live

Tools IP

Processes Methodology

THANK YOU

archiving @ dhlotmsig.communities.oaug.org/multisites/otm/media/euro_2014/c14e-08... · 9 archiving...

Documents