e-business suite on linux rac (platform transition through upgrade)
DESCRIPTION
e-Business Suite on Linux RAC (Platform Transition through Upgrade). Chakri Reddy Director of Delivery Zensar Technologies San Jose, CA. Ram Laxmanan Director, PMO Office Zensar Technologies San Jose, CA. March 2005. Agenda. Who we are Description of project Goals - PowerPoint PPT PresentationTRANSCRIPT
e-Business Suite on Linux RAC (Platform Transition through
Upgrade)
March 2005
Chakri Reddy Director of DeliveryZensar TechnologiesSan Jose, CA.
Ram LaxmananDirector, PMO OfficeZensar TechnologiesSan Jose, CA.
Agenda
• Who we are• Description of project Goals• Hardware Architecture & Scaling Plan• 11i Linux / RAC Architecture• Upgrade Process Overview • Upgrade Takeaways• Infrastructure Issues• Global Team Tactics• Questions & Answers
Who we are
• Zensar Technologies (www.zensar.com)– Global information technology organization– Decade of experience of providing IT and IT enabled service – Gained over 7500+ person years of experience and executed over 1250+
engagements– Leading software exporter in India, over 2000 skilled resources across six
delivery centers– 23 offices worldwide
• Zensar’s established Oracle Applications Practice– Enterprise Applications Group (EAG) is a focus area in packaged
solutions– Dedicated Oracle competency center, with over 600 skilled resources– Engagements include implementations, upgrades, customizations, 3rd
party integration and on-going support– Customers include Cisco, Logitech, Fujitsu, Fluke, PO Ned Lloyd and
Sprint
About Electronic Arts, our customer
• The #1 video game publisher in the US• Over a hundred popular titles such as NBA Street, Madden
NFL 2004, SimCity 4, Need for Speed, and Medal of Honor.
• Has also published extremely successful games based on Hollywood franchises, including The Lord of the Rings, Harry Potter and James Bond
• Total revenue for year 2003-04 : $3 billion• Has offices throughout the Asia-pacific, Europe and North
America – EU territories contributed to over $1 billion of revenue last year
Project Goals (Business)
• Move all business applications from RISC platform to Linux
• Plan for future growth and scalability• Cost savings in terms of licensing and hardware• Rapid deployment• Save $$$’s• OM BPR• Review and update customisations
Project Goals (Technical)
• Standardize the installation of Oracle Applications 11.5.9 across 4 global installations in Electronic Arts
• Upgrade to Oracle 11.5.9 e-business suite of products• Transition to e-business suite running on Linux RAC
– Along with the upgrade• Produce a common and extensible hardware environment
– That meets EA’s user and performance criteria, currently and into the future
• Introduce a hardware and software architecture that would permit both scaling of capacity and continuous operations
• The upgrade and platform transition, including user testing must be accomplished inside of 130 hours
Fact sheet
• Organizations, 1 in US, 21 in EU• Languages, 1 in US, 4 in EU• Database size, 200Gb in US, 350 in EU• Users, 175 in US, 350 in EU• Transaction volumes
– 9000 order lines booked per day– 30000 picking line details per day– 10000 concurrent requests per day
• Boundary systems, 15 in US, 25 in EU• Customizations, 1000 each in US and EU• Dual on-shore / off-shore delivery model
Hardware Architecture & Scaling Plan• Dell was selected as the vendor of choice at EA for world wide
deployment of Oracle 11.5.9• RedHat 2.1 was the operating system at all tiers of the architecture• “Multi-node” & “Multi-tier”• Combination of both local storage and SAN storage
– Intelligently deployed to get the highest performance and greatest economy from Oracle Applications
• Hardware load balanced Oracle Applications servers– Easily configured and robustly implemented.
• Dual concurrent process servers– Assuring rapid process completion – Confident & continuous operations, deployed with PCP (parallel concurrent
processing)
• Dual RAC nodes
11i Linux / RAC Architecture
IBM Compatible
IBM Compatible
IBM Compatible
IBM Compatible
User Tier
Load Balancer
Applications Server 01
Applications Server 02
SAN Storage Device
Application Tier Database Tier
Concurrent Server 01
Concurrent Server 02
DB Server 01
DB Server 02
Linux Production Hardware Diagram
EU Production Configuration
2 Applications Servers = Dell 2 processors 4 gig memory2 Concurrent Mgr Servers = Dell 2 processors 4 gig memory2 Database Servers = Dell 2 processors 4 gig memory
11i Linux / RAC Architecture
Storage
App 1
CM 1
DB Node1
App 2
CM 2
DB Node2
DatabaseDatafiles
L4 Switch
Load Balancer
RAC
Upgrade Process - Overview• Source
– Oracle Applications Release 11.03 with 8i DB running on HPUX– Single Node Configuration– 8 Processors at 575 MHz Speed
• Target– Oracle e-Business Suite Release 11.5.9– 2 Application Servers (H/W Load Balance)– 2 Concurrent Manager Servers with PCP Configuration– 2 RAC Load Balanced DB Nodes
• Upgrade Starts With– Single Node RISC Server Running 11.03– One Intel Linux Application Server– One Linux CM Server– One Linux DB Server
Upgrade Process - Overview
Fina
l Bac
kup
CA
T 1
& 2
Fini
shin
g St
eps
and
Test
ing
Rol
l Out
C
onne
ctiv
ity
Info
rmat
ion
Oracle 11.5.9 9.2.0.4
EA EU CRP-2 Linux Upgrade Model
Oracle 11.0.3 8.1.7.3 DB
eucr1db1
Rec
eive
and
Val
idat
e Li
nux
Har
dwar
e Pa
latfo
rm a
nd
HP
DB
ser
ver
Linu
x A
pp S
VR a
nd C
M
SVR
Con
figur
atio
n
Rec
eive
Sin
gle
Nod
e Su
rce
DB
& A
pp F
rom
EA
Linu
x R
AC
Sof
twar
e In
stal
l an
d C
onfig
urat
ion
Prep
are
for F
unct
iona
l and
D
BA
Cat
egor
y St
eps
RA
PID
Inst
all A
pps
CA
T 3
DB
Upg
rade
8.1
.7.4
Firs
t Bac
kup
Oracle 11.0.3 8.1.7.3 DB
eucr1db1 App SVR 01
App SVR 02
CM SVR 01
CM SVR 02
Pre
Aut
oUpg
rade
Valid
ate
Ora
cle
9.2.
0.4
& R
AC
Inte
rnal
Use
r Te
stin
g
Con
duct
H
ando
ver
Act
iviti
es
Gai
n A
ppro
val t
o R
elea
se th
e In
stan
ce
Aut
o - U
pgra
deS
econ
d B
acku
pA
uto
- Pat
ch
CA
T 4,
5, 6
Fun
ctio
nal
CA
T 4,
5, 6
DB
A +
NLS
Para
llel E
xpor
t/Im
port
DB
Inde
x C
reat
e
Oracle 11.0.3 8.1.7.4 DB
eucr1db1 App SVR 01
App SVR 02
CM SVR 01
CM SVR 02
Linux 11.5.9 APPS
Third
Bac
kup
Four
th B
acku
p
Oracle 11.0.3 8.1.7.4 DB
eucr1db1 App SVR 01
App SVR 02
CM SVR 01
CM SVR 02
Linux 11.5.9 APPS
Linux CM Servers
Linux 11.5.9 App SVR
Linux 9.2.0.4 RACDB
Phase 1
Phase 2
Phase 3
Linux 9.2.0.4 RACDB
Linux CM Servers
Linux 11.5.9 App SVR
Oracle 11.5.9 9.2.0.4
Downtime begins
4 hr
4 hr
5 hr
5 hr
*
13 h
r
23 h
r6
hr *
46 h
r
6 hr
*
18 h
r
6 hr
*
22 h
r 15 h
r
6 hr
*
4 hr ? hr
? hr
? hr
? hr
Downtimeends
11.5
.9.2
Cum
ulat
ive
Patc
h
cv
10 h
r
Mul
ti-no
de +
RA
C2
hr
Notes:
* Indicates actual backup time. Expectation is ~15 min backup time using mirror/split
Estimate times in italics e.g. Additional Setup
Upgrade = 188Backup = 29Total = 217
Tran
sfer
to V
irgin
ia1
hr
Post
Aut
opat
ch5
hr
Post
Impo
rt6
hr
Add
ition
al S
etup
8 hr
Pre
Expo
rt2
hr
Upgrade Process - Overview
HP ServerSingle-node Linux 11.5.9
App 1
Oracle 11.0.38.1.7.3
Oracle 11.5.99.2.0.4
Cat 1-3 Auto Upgrade
Auto Patch
11.5.9CM2 Cat 4-6
CM 1
Ora
cle
App
s 11
i – L
inux
App
s/C
M, H
P D
BExport & Import
Linux 11.5.9
App 1 App 2
CM 1 CM 2
DB Node1 DB Node2
ExportImport
IndexBuild
RAC (DB2)App2CM2
Upgrade Process - Overview• Customized Oracle Upgrade Approach
– Heterogeneous upgrade approach• Admin node running on Linux and DB node running on RISC server
– Pre upgrade steps• Rapid install (Linux application server & CM server)• 9i DB Installation (Linux DB server)
– Category 1, 2 and 3 steps• Perform all relevant category steps on RISC server running 11.03• Gather schema statistics (30%)• Backup applications
– Auto Upgrade• Admin node running on Linux and DB node running on RISC server• Batch size of 10000 & max workers of 20• Backup applications
– Auto Patch• Gather schema statistics (30%)• Backup applications
Upgrade Process - Overview• Customized Oracle Upgrade Approach
– Patch the applications• Apply consolidated update patch 2• Apply one off patches
– Category 4, 5 and 6 steps• Perform all relevant category steps• Apply customizations (if any)• Apply conversions (if any)
– e-Business suite functioning with• Linux application server• Linux CM server• RISC DB server
Upgrade Process - Overview• Customized Oracle Upgrade Approach
– Export source DB• Apply applications consolidated export import patch
(2981815 ) for 9i on SOURCE admin node. • Direct path full DB export
– Import DB• Apply patch # 2873456• Point your APPL_TOP to the 9i DB• Full system import or• Two phased import process where structure is imported first
and data is subsequently imported.• Index builds are always last with either of the import choice• Post import activities
– Gather schema statistics– Modify TNS names on Admin and CM nodes to point to 9i DB
Upgrade Process - Overview• Customized Oracle Upgrade Approach
– Converting to RAC DB• Modify ORACM configuration file• Follow Metalink documentation
– Adding Second Application Server• Change the application server name to load balancer• Review H/W load balancer setup• TAR and UNTAR the APPL_TOP onto second node
– Adding Second CM Server• Create second CM node by cloning from first CM• Setup Internal Monitor (ICM)• Define Primary & Secondary nodes for all Concurrent Managers• Define the transaction managers for all the instances in RAC cluster
Upgrade Takeaways
The Upgrade – Category 0The Upgrade – Category 0
• Total Production downtime of 98 hours• Perform as many iterations as possible
• 12 practice iterations were performed• Impact of hardware• Import process needs lot of attention• Impact of Orgs and Languages
• Be generous with your instance planning – you will need them!• Refine backup methodology, to provide total database backups as
rapidly as technically achievable• Perform Validate APPS Schema script on production regularly and fix
the issues• Make time for high quality testing of both standard and custom Oracle
Applications code– 20 One off patches applied on consolidated patch 2– Perform load test and regression testing
Infrastructure Issues
• Instance Planning– Differences from Traditional Unix to Linux platforms– Impact of server numbers– Don’t forget the disk space!
• Important considerations– Data Archiving– Getting it all to work! e.g. load balancing, DNS etc.– Log/out files, UTL, FTP– Individual server memory footprint
Global Team Tactics
• Global Project technical, functional and support teams• Managing communication• Coping with time zones
– Make it work to your advantage
• Sharing of knowledge– Client/Vendor sharing
• Examples of success
Agenda Revisited
• Who we are• Description of the Goals• Hardware Architecture & Scaling Plan• 11i Linux / RAC Architecture• Upgrade Process Overview • Upgrade Takeaways• Infrastructure Issues• Global Team Tactics• Questions & Answers