1 copyright © 2013, oracle and/or its affiliates. all ... · oms’ so that the certificate on the...

Best Practices for Maximizing Oracle Enterprise Manager 12c Availability

Mark McGill

Consulting Product Manager

Keith Meeks

FedEx


Program Agenda

Enterprise Manager Platform and Plug-in Model

Enterprise Manager Availability Challenges

– Planned Downtime

Plug-in Management

Platform Management

– Unplanned Downtime

High Availability and Disaster Recovery

Customer Case Study – FedEx


Oracle Enterprise Manager 12c

Complete Cloud

Lifecycle Management

Integrated Cloud

Stack Management

Business-Driven Application

Management

Self-Service IT Simple and Automated Business Driven | |


12c Platform and Plug-in Model

Enterprise Manager 12c is built

using a Platform/Plug-in model

– Platform contains core EM services and

Target agnostic UI pages

– Each Plug-in contains Target specific

management or other vertical functionality

– Fundamental change from prior releases

of Enterprise Manager

Allows new features to be added

outside of platform release cycle

Enterprise Manager

11g

Enterprise Manager

12c

Platform

Plug-in Plug-in

Plug-in Plug-in


Enterprise Manager Platform

Platform consists of:

– OMS Services

Loader, Metrics, Notification, Jobs, etc

– Target agnostic UI pages

All metrics, Jobs, Notifications, etc

– Core Agent

Platform versions

– EM 12c Release 1 (12.1.0.1 Platform)



Concepts

EM Platform

Platform Background Services

(PBS)

Loader

Jobs

Notifications

Console

Core UI Pages

Core Agent


Plug-ins

Target specific management is contained

in Plug-ins

Plug-ins can be created by Oracle or 3rd

party integrators using Extensibility

Development Kit (EDK)

Additional vertical Enterprise Manager

functionality (e.g.: Cloud Application

Plug-in) is also delivered using Plug-ins

Plug-ins have their own lifecycle and

version independent of the EM platform

Concepts

Target Plug-ins Solution Plug-ins

Plug-in

EM Platform


Solution Plug-ins

Provide non-Target specific, vertical

functionality within Enterprise

Manager

– Chargeback and Capacity Planning

– Cloud Application (self-service)

– My Oracle Support (MOS) Integration

Deployed only on OMS tier

Solution Plug-ins

Oracle Management Service

Chargeback /

Capacity

Planning Cloud

Application MOS

Integration


Target Plug-ins

Contain everything specific to

managing specific targets

– Target definition, Default collections

– Configuration Standards

– Job definitions

– Custom UI Pages

– Reports

– Discovery Scripts

Has OMS and Agent side content

Target Plug-ins

Discovery +

Management

Content

Discovery

Content

Discovery

Content


Agent

Managed

Target

Discovered

Target


Target Plug-ins

Agent side content consists of:

– Discovery Content

– Management Content

Discovery content is deployed on to

agents when the agent is pushed

– Discovers targets so that they can be

promoted to managed targets

Management content for each target

type is pushed when administrator

makes a discovered target a managed

target

Target Plug-ins

Discovery +

Management

Content

Discovery

Content

Discovery

Content


Agent

Managed

Target

Discovered

Target


OMS and Agent Plug-in Compatibility

Plug-ins are always updated on the

OMS first

– The Agent side Plug-in version cannot be

higher than the OMS version

– After Plug-in has been updated on the OMS

the Plug-in can be updated on the Agents

The version deployed on the OMS must

be >= the versions present on Agents

– Enables phased update of Agents

DB Plug-in

12.1.0.4

DB Plug-in

12.1.0.4

DB Plug-in

12.1.0.3


Agent

DB Plug-in

12.1.0.3 DB Plug-in

12.1.0.4

DB Plug-in

12.1.0.4

✗ ✓ ✓


Managing Plug-ins

Plug-in model means

– Modular software updates

– Updates aligned with release cycles of managed

targets

Administrator needs to be able to

– Be notified of plug-in updates

– View Plug-in details

– Download and Deploy Plug-ins across all

components with minimal disruption

Find

View

Download

Deploy


Managing Plug-ins Finding Plug-in Updates

Plug-in updates from Oracle can

be discovered using Self-Update

– Supports Online and Offline Modes

– Updates discovered every 24 hours

in online mode

Use Enterprise Manager

Extensibility Exchange to find 3rd

party created Plug-ins

– Import Plug-ins using “emcli

import_update” command


Managing Plug-ins Viewing Plug-in Information

Plug-in Information page

shows:

– Version/Revision

– Certified targets and

operating systems

– Dependencies

– OMS and Agent

Deployments


OMS Plug-in Deployment

All Plug-ins that are used must be

deployed on the OMS

– Best Practice: Deploy Target

Plug-ins to OMS prior to pushing

agents to ensure discovery content

is present

Deployment of Plug-ins to OMS can

be done using

– Plug-in Manager

– EM CLI (allows deployment of

multiple Plug-ins in one go)


Availability Planned Downtime


Plug-in Deployment and OMS Restarts

Some Oracle provided Plug-ins can include Java

code, PL/SQL, etc. (examples, DB, Middleware)

– These Plug-ins require an OMS restart during

deployment/update

– Run Plug-in deployment pre-req to determine if a

restart is needed

All 3rd party Plug-ins are Metadata driven

– Do not require an OMS restart during

deployment/update


Platform and Oracle Plug-in Release Cycle Regular “4-5-4” Development Cycles Follow Major Releases

12.1.0.2

(Release 2)

Plug-In Update

(DB, VT, SSA, SMF)

12.1.0.3

(Release 3)

Plug-in Update

(DB, FMW, SSA, VT) 12.1.0.4

(Release 4)

Sept’ 12 Feb’ 13 June’ 13 Oct’ 13 Mar’ 14

(planned)

~4 months ~5 months ~4 months ~5 months

All Oracle built Plug-ins released in “Bundles”

2 bundles per Platform release


Enhancements for recovery of failed OMS

Plug-in updates

1. Plug-in update failure can be rolled back using

database recovery mechanisms

Flashback or restore database following failure

SCN provided to administrator in ‘emctl status

oms –details’ output

2. Plug-in update failure can alternatively be resumed

after fixing the root cause of the update failure

‘emctl resume_plugin_upgrade’

Enables administrator to quickly recover to a

known state and restart Enterprise Manager

Plug-in Update Rollback/Resume

Flashback / Restore

Database

Start OMS

Fix Root Cause

Resume Plug-in Update

Start OMS

Resume Rollback


Minimizing OMS Plug-in Deployment Impact

Schedule maintenance windows for Enterprise Manager updates

When updating a Plug-in, apply all other available Plug-in updates

Use EM CLI to deploy Plug-ins that require a restart in batch to

avoid multiple OMS bounces

Use Flashback Database for fast recovery from deployment issues

Tips for OMS Plug-in Deployment


Minimizing Downtime for Platform Updates

Stop OMS

Start OUI and chose Upgrade

Create New Home

Configure

Start OMS

Upgrade Agents

Platform Patchset Update

OUI

Upgrade

Software

only Install

Start OUI and chose Software Only Install

Create New Home

Stop OMS

Configure New Home

Start OMS

Upgrade Agents

Patchset update requires downtime

for all OMSs

Best Practice: Use Software Only

Install in OUI to install updated

binaries while OMS is still up and

running

– Reduces OMS downtime

OMS Up

OMS Down


Availability Unplanned Downtime


High Availability

Critical components in Enterprise Manager infrastructure

are:

– Repository - Persistent store for all Enterprise Manager

data

– OMS - Central application accessed by Agents and end-

users

– Software Library - Filesystem repository used to store

software entities

All of the above should be configured for High Availability

if availability of Enterprise Manager is critical


High Availability

Oracle RAC provides a

standard HA solution for the

EM repository

Best Practice: Configure

RAC prior to EM installation

Best Practice: Use SCAN

and DB Services for OMS to

Repository connect strings

Repository

OMS

Repository

emctl config oms -store_repos_details

-repos_conndesc

“(DESCRIPTION=

(ADDRESS=(PROTOCOL=TCP)

(HOST=emrep-cl-scan.example.com)(PORT=1521)

)

(CONNECT_DATA=(SERVER=DEDICATED)

(SERVICE_NAME = emrep)))"

-repos_user sysman


High Availability OMS

Additional OMSs can be

deployed behind a Server Load

Balancer (SLB) for OMS High

Availability

Agents and Users communicate

with OMS via load balancer

Software Library must be

accessible from all OMSs

Agent

Console

/EM CLI

Software

Library

OMS

Agent

Console

/EM CLI

Load

Balancer

Software

Library

OMS

OMS

No HA

HA


High Availability Additional OMS Deployment

“Add Management Service”

Deployment procedure used to

install additional OMSs

– Clones first OMS onto

additional servers

– Provides EM Administrator with

storage and SLB setup

information

– Performs Software Library

checks


High Availability SLB Requirements

SLB must have support for the following:

– Virtual Server Ports

Individual Ports must be configured for HTTP/HTTPS Upload and

Console services

– Persistence

All Console traffic for an individual session should be directed to the

same OMS

– Application Monitoring

SLB must be able to recognise when the application is down


SLB Setup SLB Configuration

Configure Virtual Servers on the SLB for the Console and Upload

Services

– Virtual Servers must use a common IP address and different port for each service

– Port mapping can be done on the SLB (e.g. Use default secure HTTP port of 443 on Virtual Server

and port 7799 on OMS servers)

Set Session Persistence for Console services

– Use Source IP based persistence

Configure Application Monitors for each service

Service Monitor Send String Monitor Receive String

HTTPS Console GET /em/consoleStatus.jsp Enterprise Manager Console is UP

HTTPS Upload GET /empbs/upload Http Receiver Servlet active!


SLB Setup OMS and Agent Configuration

Resecure all OMSs using ‘emctl secure

oms’ so that the certificate on the OMS

contains the SLB hostname

– Management Agents communicating using the

SLB will reject certificates that don’t use SLB

hostname

– Use ‘–console’ to also generate certificate for

console

Resecure the Agents using ‘emctl secure

agent’ command

– Reconfigures agents to send traffic to the SLB

instead of directly to the OMS

./emctl secure oms –sysman_pwd xxxxxx -host

slb.example.com -slb_console_port 443 –

secure_port 4900 –slb_port 4900 -console -

lock_console -lock_upload

.

.

Securing OMS... Started. Enter

Agent Registration Password : Securing

OMS... Successful Restart OMS

emctl secure agent

-emdWalletSrcUrl

https://oms.example.com:4900/em


High Availability

Software Library is a critical component in Enterprise

Manager 12c

– Used for patching, agent deployment, provisioning, self-

update

Must be accessible Read/Write from all active OMSs

Should be deployed on highly available storage

– Examples

ZFS Appliance

NetApp Filer

Software Library High Availability


High Availability End-to-End HA Topology

Agent

Console

/EMCLI

Load

Balancer

Software

Library

Active OMS

Active OMS

Repository

All OMS, Repository

and Software Library

components are active

within the same Data

Center

Follows Oracle

Maximum Availability

Architecture (MAA)

practices

Not a Disaster

Recovery (DR) solution


Disaster Recovery

Protects applications from

catastrophic failures

Keeps data on primary site

synchronized with a standby

Allows applications to failover to the

standby site

Primary

Site

Standby

Site


Disaster Recovery

Data Guard Physical Standby Database

provides Disaster Recovery solution for

Repository

Chose protection mode based on recovery

objectives and network performance

– Max Protection (SYNC writes to Standby)

– Max Availability (SYNC writes to Standby)

– Max Performance (ASYNC writes to Standby)

Use Data Guard Broker to manage

switchover/failover of database

Repository

Primary Site

(active)

Standby Site

(passive)

Data Guard

Physical

Standby


Disaster Recovery

Deploy Standby (Passive) OMSs on

Standby Site

Use application virtual hostname to direct

clients to the active site

– Use DNS to point to active site (Global Traffic

Manager is optional)

– Secure OMSs using this hostname

Two options for Standby OMSs

– Standby OMS using Standby WebLogic Domain

– Standby OMS using Storage Replication

OMS

Agent

Console

/EMCLI

DNS/

GTM

Primary Site

Standby Site


Disaster Recovery Standby OMSs Using Storage Replication

Primary OMS

Replication

Standby OMS

Best Practice: Storage Replication is

recommended method for 12c Release 3

– No manual application of Plug-ins or

OMS patches at Standby Site

– No rebuild of Standby site needed after

upgrades

Uses Storage Replication to replicate

entire installation at standby site

Requires configuration of Network

aliases at Standby site

Primary Site

Storage

Standby Site

Storage


Disaster Recovery Standby OMSs Using Standby WebLogic Domain

Primary OMS

(GCDomain)

Clone

Standby OMS

(GCDomainSby)

Creates a new WebLogic

Domain at the Standby site

– Requires manual application of

Plug-ins or OMS patches at

Standby Site

– Rebuild of Standby site

needed after upgrades

Solution for versions prior to

12c Release 3


Disaster Recovery

Software Library should be made available on

standby site

Variety of technologies available (SYNC or ASYNC):

– ZFS Remote Replication

– NetApp SnapMirror

Sync changes at least hourly

Software Library

1 hr


High Availability and Disaster Recovery HA/DR Topology

Console

/EMCLI

DNS/

GTM

Full protection from

site failure

Failover to DR site in

minutes

Zero or minimal data

loss depending on

configuration

SLB

SLB

Primary Site

Standby Site


High Availability and Disaster Recovery Levels of Availability

Description Load Balancer

Requirements Cost

Level 1 OMS and repository on their own hosts.

No failover.

Repository backup and OMS exportconfig

None $

Level 2 OMS installed on shared storage with a VIP based failover.

Database protected with Local Data Guard. None $$

Level 3 OMS in Active/Active configuration.

RAC repository with Local Data Guard SLB at primary $$$

Level 4

Primary OMS in Active/Active configuration

RAC Repository on primary site.

Standby OMSs at DR site

Standby RAC database at DR Site site.

SLB at primary and

standby sites

Optional GTM

$$$$

Global EM Architecture

Keith Meeks

09/24/2013

About FedEx Services

FedEx Corporate Services, Inc. (“FedEx Services”) began operations in

June 2000 to provide information technology, sales and marketing

support for FedEx Corp. subsidiaries FedEx Express and FedEx

Ground.

Today FedEx Services coordinates sales, marketing, information

technology, customer service, and worldwide supply chain services

support for the global FedEx brand. This includes the data management

and networking expertise behind the package tracking capabilities for

FedEx Express, FedEx Ground and FedEx Freight, along with e-

commerce services, customer contact services, and other functions of

the corporation’s professional services company.

2 October 2013 41

Enterprise Manager at FedEx Services

Standardized approach to monitoring and alerting

Centralized view of the Database farm

Minimal effort to scale as the enterprise grows

At a glance system health check

Real time and historical views of performance and SQL activity

Inventory and resource tracking

Manage more with less

2 October 2013 42

What is Enterprise Manager Monitoring?

Agents: 1,600+

Hosts: 1,600+

Cluster Databases: 230+

Database Instance: 1185

Applications supported include:

Air / Ground Operations

Revenue Systems

Customs Critical

Fedex Ground

Fedex.com

2 October 2013 43

Requirements for High Availability

Datacenters in Colorado & Tennessee for Disaster Recovery

EM is monitoring mission critical applications that run FedEx

Required to be able to continue monitoring in event of disaster

Increased dependence on EM for running jobs, administration and

performance tuning

Recent wildfires in Colorado Springs prompted for “prepare for

action” on Disaster Recovery systems

2 October 2013 44

Current EM 12c Architecture

2 October 2013 45

Future EM 12c Architecture

2 October 2013 46


Summary

Enterprise Manager 12c is based on a Platform/Plug-in

architecture

Various techniques available to reduce downtime during Platform

and Plug-in maintenance:

– Apply plug-in updates in batch

– Use flashback database to recover quickly from plug-in deployment

issues

– Install new Platform binaries while old version is still running

Unplanned downtime can be reduced by following the

documented highly available and disaster tolerant configurations


Resources

Install and Upgrade collateral page(PPT, Recorded demo,

Whitepapers) on OTN

– http://www.oracle.com/technetwork/oem/install-upgrade/index.html

White Paper: Deploying a Highly Available Enterprise Manager

12c Cloud Control

– http://www.oracle.com/technetwork/oem/framework-infra/wp-

em12c-building-ha-level3-1631423.pdf

White Paper: Enterprise Manager Cloud Control 12c Disaster

Recovery with Storage Replication

– http://www.oracle.com/technetwork/oem/cloud-mgmt/wp-em12cr3-

dr-solution-1965514.pdf

http://www.oracle.com/technetwork/oem/install-upgrade/index.html



http://www.oracle.com/technetwork/oem/framework-infra/wp-em12c-building-ha-level3-1631423.pdf













http://www.oracle.com/technetwork/oem/cloud-mgmt/wp-em12cr3-dr-solution-1965514.pdf












Resources

White Paper: Configuring OMS High Availability with F5

BIG-IP Local Traffic Manager

– http://www.oracle.com/technetwork/oem/framework-infra/wp-

em12c-config-oms-ha-bigip-1552459.pdf

White Paper: Configuring OMS Disaster Recovery with F5

BIG-IP Global Traffic Manager

– http://www.oracle.com/technetwork/database/availability/oms-

dr-f5-big-ip-gtm-1880830.pdf

Oracle University Course

– Enterprise Manager Advanced Configuration

http://www.oracle.com/technetwork/oem/framework-infra/wp-em12c-config-oms-ha-bigip-1552459.pdf















http://www.oracle.com/technetwork/database/availability/oms-dr-f5-big-ip-gtm-1880830.pdf














Oracle Enterprise Manager Sessions - Tuesday

Session Session Title Day Time Location

CON3991

Deploying Oracle Enterprise Manager 12c in a Complex Network Without

Compromising Security Tuesday 5:00 PM Moscone South - 300

CON9586 DBA’s New Favorite: Lifecycle Management with Oracle Enterprise Manager 12c Tuesday 5:00 PM Moscone South - 306

CON9574

Proactively Monitor Oracle WebLogic User Activity and Solve Issues Before End

Users Complain Tuesday 5:15 PM Moscone North - 130


Oracle Enterprise Manager Sessions - Wednesday Session Session Title Day Time Location

CON11255 Real-World Operating Excellence with Oracle Enterprise Manager 12c Wednesday 10:15 AM

Westin San Francisco -

Metropolitan I

CON9480 Simplify and Enhance Siebel Management Technologies with Oracle Enterprise Manager Wednesday 10:15 AM Moscone West - 3001

CON9566

Get Up and Running with Oracle Enterprise Manager for SOA and Oracle Fusion

Middleware Wednesday 10:15 AM Moscone North - 130

CON9584 Deploying Applications in a Private Cloud with Oracle Enterprise Manager Wednesday 10:15 AM Moscone South - 308

GEN8792

General Session: Database Management Innovations—Oracle Database 12c

Manageability Highlights Wednesday 10:15 AM Moscone South - 103

CON4101

Take Your Oracle WebLogic Applications to the Next Level with Oracle Enterprise

Manager 12c Wednesday 11:45 AM Moscone North - 130

CON9581 Optimize Oracle Enterprise Manager Monitoring for Your Global Data Center Wednesday 11:45 AM Moscone South - 308

CON9579 Step-by-Step Cookbook for Identifying and Tuning SQL Problems Wednesday 1:15 PM Moscone South - 103

CON9589 Building a SPARC Cloud with Oracle Enterprise Manager Ops Center 12c Wednesday 1:15 PM

Westin San Francisco -

Metropolitan I

CON4666

Oracle Enterprise Manager 12c Database Lifecycle Management Automatic Provisioning

and Patching Wednesday 3:30 PM Moscone North - 131

CON8768 DBA Best Practices for Protecting Data Privacy with Oracle’s Data Masking Wednesday 3:30 PM Moscone West - 2024

CON9577 Active Session History Deep Dive: Advanced Performance Analysis Tips Wednesday 3:30 PM Moscone South - 104

CON3255 Being Sure: Confident Consolidations with Oracle Real Application Testing 12c Wednesday 5:00 PM Moscone South - 306

CON6217

Using Oracle Enterprise Manager 12c to Manage Engineered Systems and Oracle

Fusion Middleware Wednesday 5:00 PM Moscone North - 130

CON9588 Get Proactive: Best Practices for Upgrading to Oracle Enterprise Manager 12c Wednesday 5:00 PM Moscone South - 308


Oracle Enterprise Manager Sessions - Thursday

Session Session Title Day Time Location

CON9605

Databases “On the Fly”: Unravel the Cloud Potential in Oracle Enterprise

Manager 12c Thursday 11:00 AM Moscone South - 308

CON3103 Real Oracle Real Application Testing: What to Expect and Prepare For Thursday 12:30 PM Moscone South - 104

CON6129

Migrating Oracle Enterprise Manager to a New Data Center with Near-Zero

Downtime Thursday 12:30 PM Moscone South - 308

CON9575

Keyword-Driven Testing with Oracle Application Testing Suite: Application

Testing Reinvented Thursday 12:30 PM Moscone West - 3018

CON5677

Bank of America and Oracle Enterprise Manager 12c: Delivering Productivity

and Efficiency Thursday 2:00 PM Moscone South - 308

CON9571 JVM Diagnostics: Java Profiling in Production Environments Thursday 2:00 PM Moscone North - 130

CON9587

Manage Beyond Limits: Oracle Enterprise Manager CLI and Other Extensibility

Features Thursday 3:30 PM Moscone South - 308


Oracle Enterprise Manager DEMOgrounds Session Title Location

Application and Infrastructure Testing Moscone West - W-098

Automatic Application and SQL Tuning Moscone South, Left - SL-001

Automatic Fault Diagnostics Moscone South, Left - SL-005

Automatic Performance Diagnostics Moscone South, Left - SL-006

Complete Data Center Monitoring Moscone South, Left - SL-018

Complete Database Lifecycle Management Moscone South, Left - SL-014

Data Masking and Data Subsetting Moscone South, Left - SL-015

Delivering and Managing Database as a Service Moscone South, Left - SL-017

End-User Diagnostics Moscone South, Right - SR-222

End-to-End Management of Oracle E-Business Suite Moscone West - W-014

Identity Management Monitoring with Oracle Enterprise Manager Moscone South, Right - SR-250

Infrastructure as a Service Moscone South, Center - SC-175

Oracle Applications Management Moscone West - W-062

Oracle Enterprise Manager Cloud Control 12c Overview Moscone South, Left - SL-013

Oracle Real Application Testing Moscone South, Left - SL-002

Oracle WebLogic Server, Oracle Coherence, and Middleware Cloud Management Moscone South, Right - SR-206

SOA Management Moscone South, Right - SR-224

Zero to Cloud: Infrastructure to Testing as a Service Moscone South, Left - SL-016


Oracle Enterprise Manager Hands-on Labs Marriott Marquis - Salon 14/15

HOL

Session Hands-on Lab Title Day Time

HOL11305 Oracle Database 12c Lifecycle Management with Oracle Enterprise Manager Wednesday 10:15 AM

HOL11304 Maximizing Oracle Database 12c Performance with Oracle Enterprise Manager Wednesday 11:45 AM

HOL11284 Database as a Service with Oracle Enterprise Manager and Oracle Database 12c Wednesday 01:15 PM

HOL11734 Middleware as a Service using Oracle Enterprise Manager 12c Wednesday 03:30 PM


Enterprise Manager Resources Oracle.com: http://www.oracle.com/enterprisemanager

– Enterprise Manager Customer References

OTN: http://www.oracle.com/technetwork/oem

– Demos on Demand

– Enterprise Manager Forums

Enterprise Manager Training from Oracle University

– Oracle Enterprise Manager 12c Learning Library

– Oracle Enterprise Manager Cloud Control 12c: Install and Upgrade

– Using Oracle Enterprise Manager Cloud Control 12c Ed 1

– Oracle Enterprise Manager 12c: Overview Bundle Self-Study

– Oracle Enterprise Manager 12c: Management Bundle Self-Study

Partners

– List of Enterprise Manager Specialized Partners

– Oracle Enterprise Manager OPN Knowledge Zone

– Oracle Enterprise Manager OPN Specialization

– Oracle Enterprise Manager Application Quality Management OPN Specialization

– IOUG Oracle Enterprise Manager Special Interest Group

Social Media

– Twitter, Facebook, YouTube, Linkedin, Blog

http://www.oracle.com/enterprisemanager

http://www.oracle.com/us/corporate/customers/customersearch/index.html?xProductServices=12/44

http://www.oracle.com/technetwork/oem/em12c-screenwatches-512013.html

http://www.oracle.com/technetwork/oem

http://www.oracle.com/technetwork/oem/em12c-screenwatches-512013.html

http://education.oracle.com/pls/web_prod-plq-dad/db_pages.getCourseDesc?dc=D73067GC10

https://forums.oracle.com/forums/category.jspa?categoryID=70


https://apex.oracle.com/pls/apex/f?p=44785:141:104974085335854::::P141_PAGE_ID,P141_SECTION_ID:16,92




http://www.oracle.com/partners/en/knowledge-zone/database/enterprise-manager-021472.htm?ssSourceSiteId=ocomen


http://education.oracle.com/pls/web_prod-plq-dad/db_pages.getCourseDesc?dc=D74510







https://solutions.oracle.com/scwar/sc/index.html?canned_search=PP_SPEC_OENTM



https://competencycenter.oracle.com/opncc/spec_status.cc?spec_id=18

https://competencycenter.oracle.com/opncc/spec_status.cc?spec_id=16

http://www.ioug.org/p/cm/ld/fid=148&gid=440

http://twitter.com/

https://www.facebook.com/oracleenterprisemanager

http://www.youtube.com/OracleEnterpriseMgr

http://www.linkedin.com/groups?about=&gid=2688744

http://blogs.oracle.com/oem/

1 copyright © 2013, oracle and/or its affiliates. all ... · oms’ so that the certificate on the...

Documents