1

Commercial Applications Services

867 Processing Issues

Outages/Service Availability

(05/24/06 – 07/27/06)– Planned Outages

– Unplanned Outages

– Service Availability

2

867 Processing Issues

Paperfree Process Servers

Key:

PROXIES

INTERNET

OUTBOUND

NAESB

TCH

EAI

TCH Database

Retail Database Server

(Oracle Databases)

PAPERFREE

SIEBEL

Bi-Directional Data Flow

Siebel DatabaseNAESB Database Paperfree DatabaseOutbound Data Flow

Inbound Data Flow

Retail Systems

INBOUND

Paperfree File Server

FIREWALL

Solaris

W2K

W2K

HP

W2K

W2K

HP

IN/OUT

DMZ

SWITCH

Single Point of Failure

3

867 Processing Issues

Issue No. 1: Slowdown pulling 867’s from TCH interface table (Began Friday 07/17 6:30PM)

TCH EAI

TCH Database(Interface Table for

867's)

Pu

lls 8

67

's

PAPERFREESIEBEL

Writes 867s

4

867 Processing Issues

TCH EAIPAPERFREE

SIEBEL

Issue No. 2: EAI Component unable to update Siebel resulting from db clean-up efforts (Issue No. 1)

5

Outages/Service Availability

Planned Outage Summary

• Retail Systems: Saturday, 07/08– Siebel SIR 10698 implemented to address duplicates created in

Siebel

– 12:00PM – 6:30PM

– Siebel SIR 10698 rolled back Saturday, 07/12

– Created internal Siebel errors and rejected some 814_16s

– No outage / Transparent

6

Outages/Service Availability

Unplanned Outage Summary• PaperFree: Sunday, 05/28

– Processing issues identified by Operations personnel– The PaperFree processor servers were unable to start up the two services used in processing data on the file

server– DNS related issues– 9:50PM – 12:42AM (172 minutes)

• Siebel/TCH: Friday, 07/07– Siebel Database Performance issue/degradation identified by Operations personnel– Performance brought processing to a halt– Statistics on database were run at an inopportune time– 9:30PM to 10:30PM (60 minutes)

• NAESB: Tuesday, 07/11– Processing issues identified by Operations personnel– Several directories missing from the server– Services were restored following recovery of data– 9:15PM to 12:05AM (170 minutes)

• NAESB: Friday, 07/21– Processing issues identified by Operations personnel– Service account locked out– Root cause underway– 4:50PM to 6:04PM (74 minutes)

7

Outages/Service Availability

Service Availability

Date Range Services  

Gross Available Minutes

Planned Outage Minutes

Net Available Minutes

Unplanned Outage Minutes

Percent Availability

January

Retail Transaction Processing 44,640 840 43,800 105 99.76

February

Retail Transaction Processing 40,320 870 39,450 556 98.59

March

Retail Transaction Processing 44,640 960 43,680 0 100

April

Retail Transaction Processing 43,200 660 42,540 0 100

May

Retail Transaction Processing 44,640 6,135 38,505 402 98.96

June

Retail Transaction Processing 43,200 0 43,200 0 100

July

Retail Transaction Processing 44,640 390 44,250 304 99.31

Year to Date

Retail Transaction Processing

305,280(to July 31) 9,855 295,425 1,367 99.54