sungard higher education banner on dell servers …the database was managed by oracle enterprise...

April 2008

SUNGARD HIGHER EDUCATION BANNER ON DELL SERVERS AND STORAGE THE DELL-SUNY PROOF OF CONCEPT

By Dave Jaffe and Kai Yu Dell Inc. and Dan Brint State University of New York (SUNY) Information Technology Exchange Center

April 2008 Dell Reference Architectures

Contents

Executive Summary ....................................................................................................... 4

Introduction .................................................................................................................... 5

SunGard Higher Education Banner Unified Digital Campus ..................................... 7

The Physical Infrastructure ........................................................................................... 9

Load Generators ...................................................................................................... 9

F5 Load Balancer ................................................................................................... 10

Application Servers ............................................................................................... 10

Database Servers ................................................................................................... 10

Storage .................................................................................................................... 10

The Oracle RAC Database ........................................................................................... 12

Using the Oracle Grid Computing Model to Provide Database Services .......... 12

SUNY Multiple Campus Banner Applications POC Grid Implementation ......... 12

Software Stack and Databases Configuration: ................................................... 13

Database Shared Storage Configuration ............................................................. 14

SUNY POC Grid Database Implementation Architecture ................................... 15

Oracle Enterprise Manager Configuration ........................................................... 16

Dynamic Scaling of the Oracle RAC Cluster ....................................................... 16

Test Methodology ........................................................................................................ 17

Results .......................................................................................................................... 18

Conclusions .................................................................................................................. 20

Acknowledgements ..................................................................................................... 21

Tables

Table 1: Banner Components .......................................................................................................... 8

Table 2: Dell PowerEdge Servers Used in Proof of Concept ........................................................ 11

Table 3: SUNY POC Shared Storage Database LUN Configuration ............................................ 15

Table 4: Simultaneous Simulated Users per Student Function ..................................................... 17

Table 5: University Sizes Tested ................................................................................................... 17

Table 6: Total Response Time and CPU Utilization ...................................................................... 19

Table 7: Total Registrations on 5+5 Configuration ........................................................................ 19

Figures

Figure 1: Block Diagram of Proof of Concept Components ............................................................ 9


Figure 2: SUNY POC Grid Computing Model ............................................................................... 12

Figure 3: Eight-Node Oracle 10g RAC Cluster Configuration ....................................................... 13

Figure 4: Database Instance Map on the Grid .............................................................................. 14

Figure 5: SUNY POC Grid Database Implementation Architecture .............................................. 16

Figure 6: Response Time Breakdown for Traditional Student Registration .................................. 18


Section 1 Executive Summary

The State University of New York (SUNY) Information Technology Exchange Center (ITEC) wanted to demonstrate the power of running SunGard Higher Education’s Banner® Unified Digital Campus enterprise resource planning software on Dell servers and storage. To achieve this goal they collaborated with Dell Inc.’s Reference Architecture team in a proof of concept test. Using a database taken from a previous university study, the POC showed that Banner running on Dell™ PowerEdge™ servers and Dell/EMC® storage could scale to handle 6.25 Texas Tech-sized databases or a total of 175,000 students. In the largest simulation over 11,000 students simultaneously registered for courses and performed other tasks, with nearly 70,000 courses selected in an hour, 37 times the actual number seen at a SUNY school of 11,000 students.


Section 2 Introduction

Banner® Unified Digital Campus (UDC) from SunGard Higher Education is an enterprise resource planning (ERP) solution for higher education, in use in over 900 institutions world-wide. Banner UDC is a suite of administrative applications that work seamlessly with other SunGard Higher Education applications, third party applications, or user-developed applications to supply the basic information processing needs of higher-education institutions. Banner uses the Oracle® database to store all data.

In November 2007, SunGard Higher Education announced that Dell was a partner in its new Unified Digital Campus Test Center and that Dell servers and storage would comprise the hardware reference platform for Windows and Linux Banner solutions (http://www.sungardhe.com/about/news/PressReleases/ Article.aspx?id=2478). SunGard Higher Education and Dell’s partnership in the UDC Test Center is intended to drive interoperability testing between SunGard Higher Education software and Dell hardware, test the performance of SunGard Higher Education applications on Dell hardware, and develop tested-and-validated reference architectures of Banner running on Dell hardware, thus simplifying an institution’s adoption of Banner.

To demonstrate the effectiveness of running Banner on Dell hardware, the Dell Reference Architecture team partnered with the State University of New York’s Information Technology Exchange Center (SUNY ITEC) in a proof of concept (POC) test in Dell’s Austin lab in late 2007 and early 2008. SUNY ITEC, a central IT resource for the 64 separate institutions of higher learning and research that comprise the State University of New York, manages servers at many of the state university’s campuses as well as hosts applications for several of the campuses. To demonstrate its scope, SUNY ITEC currently supports Banner for 24 campuses and manages over 140 Oracle databases and over 50 Oracle application servers providing a variety of applications in addition to Banner.

The Dell-SUNY proof of concept extends earlier testing performed by Dell and SunGard Higher Education with Texas Tech University (http://www.dell.com/content/ topics/global.aspx/casestudies/fy2008_q3_id706?c=us&cs=RC956904&l=en&s=hied). Whereas the earlier testing paved the way for installation of Banner on Dell to serve the needs of Texas Tech’s 28,000 students, the Dell-SUNY testing was intended to prove that Banner running on Dell™ PowerEdge™ servers and Dell/EMC® storage could be scaled to handle more than 6 times that load to satisfy the needs of SUNY ITEC. This result was indeed achieved. Through the use of Oracle Real Application Clusters (RAC) with Oracle Enterprise Manager Grid Control to manage the large databases, virtualization software from VMware Inc. to help manage the application servers, the Big-IP® load balancer from F5 Networks Inc. to send the requests to the application servers and Spotlight® from Quest Software Inc. to help manage the Oracle RAC databases, a workload representing 175,000 students was successfully handled by Banner on Dell servers and storage.

The Banner application is discussed in detail in the next section, followed by a discussion of the hardware architecture used in the POC in Section 4, which is


followed by a detailed look at the Oracle database in Section 5. Section 6 describes the methodology used in test, with the results shown in Section 7.


Section 3 SunGard Higher Education Banner Unified Digital Campus

Banner (http://www.sungardhe.com/Products/Product.aspx?id=832) is a tightly integrated suite of higher education applications, running on a single database. The Banner components provide the university with tools to manage students, financial aid, finance, human resources, enrollment, and constituents. Additional components like Luminis® and Banner XtenderSolutions® provide portal and document imaging and management features.

Banner Student® provides secure online 24x7 administrative and academic functions to college prospects, students, and faculty. Prospects can use the Banner Student component to apply for admission. Current students use it to search and register for classes, and obtain financial aid data. Faculty members use it to manage their courses, class rosters, and grades.

Banner Financial Aid® provides for seamless management of financial aid from multiple sources. From the initial web-based communication with students to the final dispersal of aid, the Banner Financial Aid program aligns with the university’s admission calendar to provide financial aid services during early and regular admissions, as needed by the university.

Built using higher education fund accounting principles, Banner Finance® provides accurate financial information to all key institution stakeholders. Departments can be given access to only the information they need, whether that be procurement, accounts payable, budgets, research funding or endowment management.

Banner HR® provides appropriate, secure access to human resources information to both institution administrators and employees. Banner HR handles all aspects of personnel management including recruitment, hiring, career management and retention.

Banner Enrollment Management Suite® is a relationship management program tailored for the needs of a university. Focused on building strong relationships with prospects, students, parents and alumni, the Enrollment Management Suite provides contact, communication, campaign, event and performance management capabilities.

Banner Advancement® manages the interaction of the university’s alumni, donors, friends, community members and staff with each other and with the university. Fund raising is managed through functionality to manage campaigns, events, annual giving programs and corporate/foundation programs. Self-service features make it easy for alumni to donate and to keep in touch with classmates.

Table 1 lists the Banner components installed for the Dell-SUNY proof of concept.


Component Version

Banner Financial Aid 7.5

Banner Financial Aid Self Service 7.3

Banner General 7.3

Banner Web General 7.3

Banner Web Tailor 7.3

Banner Student 7.3

Banner Student Self Service 7.3

Banner Faculty and Advisor Self Service 7.3

Banner Accounts Receivable 7.3

Banner Accounts Receivable Self Service 7.3

Banner Advancement 7.3

Banner Advancement Officers Self Service 7.3

Banner Advancement Self Service 7.3

Banner Finance 7.2

Banner Finance Self Service 7.2

Banner Human Resources 7.2

Banner Position Control 7.3

Banner Employee Self Service 7.3

Table 1: Banner Components


Section 4 The Physical Infrastructure

To model the large Banner configuration required by SUNY ITEC the complete configuration was built in the Dell Reference Architecture lab along with sufficient load generator servers to model the work generated by 175,000 students. The components used in the Dell-SUNY proof of concept are shown in Figure 1. The role of each component type is described below.

Figure 1: Block Diagram of Proof of Concept Components

Load Generators

During the Texas Tech POC, SunGard Higher Education and Dell captured the input keystrokes and web clicks for many of the key Banner user actions using HP’s LoadRunner tool. This data was then sanitized to remove real names and other data, and used for the SUNY POC. During the tests the workload was


played back by LoadRunner Virtual User Generators running on eight PowerEdge 1855 blade servers under the control of a ninth PowerEdge 1855 serving as the LoadRunner console. Details of all the PowerEdge servers used in the POC are included in Table 2.

F5 Load Balancer The Big-IP load balancer from F5 Networks, Inc., was used to steer the simulated user requests to the application servers in a balanced manner. The Big-IP model used in this test was the 6400.

Application Servers

The key Banner components SSB (Self-Service Banner) and INB (Internet Native Banner) run on the Oracle Application Server platform. Other Banner functions such as batch processing, compilers, etc., run on what are termed Banner Application servers in Figure 1. To provide maximum server utilization and flexibility, the ESX® Server virtualization software from VMware, Inc. was installed on 10 PowerEdge 1955 blade servers and 3 PowerEdge 1950 rack servers to create a virtual machine (VM) farm. For the test 20 VMs running 32-bit Red Hat Enterprise Linux 4 were created on the 10 PowerEdge 1955s to serve as the Oracle Application Servers and 10 VMs, also 32-bit Red Hat Enterprise Linux 4, were created on the 3 PowerEdge 1950s to serve as the Banner Application servers. The use of VMware’s live migration facility, VMotion, provided maximum flexibility to move the VMs around on the ESX Server hosts to balance the load appropriately.

Database Servers The Banner components used in the POC stored their data in an 8-node Oracle Real Applications Cluster (RAC) database running as an Oracle Grid. Each node was a PowerEdge 2950 running 64-bit Red Hat Enterprise Linux 4. The database was managed by Oracle Enterprise Manager as well as Quest Software’s Spotlight tool. Ten database instances representing ten separate campuses were built for the test and stressed in various combinations as described in Section 6. The database is described in detail in the next section.

Storage The Oracle RAC database was stored on a Dell/EMC CX3-40 storage area network (SAN) with three DAE4P disk pods with 15 146GB disks each. The VMs used for application servers were stored on a Dell/EMC CX300 SAN. The use of shared storage for the VMs enables VMotion.


Server Model Role CPUs Memory Operating System Count

PowerEdge 1855 LoadRunner Console

2x 2.8GHz Intel® Xeon® Single-Core

4 GB Windows 2003

Server 32 bits

1

PowerEdge 1855 LoadRunner Generators

2x 3.6 GHz Intel® Xeon® Single-Core

2 GB Windows 2003

Server 32 bits

8

PowerEdge 1955 ESX Hosts for

Application Servers

2x 3.0 GHz Intel® Xeon®

Dual-Core 8 GB ESX 3.02 10

PowerEdge 1950 ESX Hosts for Banner Servers

2x 3.0 GHz Intel® Xeon®

Dual-Core 8 GB ESX 3.02 3

PowerEdge 2950 Oracle RAC Nodes

2x 2.6 GHz Intel® Xeon® Quad-Core

32 GB

Red Hat Enterprise

Linux 4 64 bits

8

Table 2: Dell PowerEdge Servers Used in Proof of Concept


Section 5 The Oracle RAC Database

Using the Oracle Grid Computing Model to Provide Database Services Unlike the traditional database model where each database resides on its own set of servers and storage, in the Oracle Grid Computing model multiple databases share a single infrastructure of servers and storage and provide database services to applications. In this way Oracle Grid provides dynamic resource sharing and allocation and continuous high availability for all the databases in the Grid.

The Oracle Grid Computing model is well suited for the SUNY multiple campus Banner installation. In this Grid infrastructure, ten application databases were consolidated in an eight-node Oracle 10gR2 Real Application Cluster (RAC) to provide the database services to the corresponding ten Banner applications each of which serves one campus.

Figure 2: SUNY POC Grid Computing Model

SUNY Multiple Campus Banner Applications POC Grid Implementation In addition to RAC and database services, Oracle 10g offers features to implement the Grid computing model including Automatic Storage Management (ASM), Oracle Enterprise Manager Grid Control (OEM) and load balancing. The Grid configuration used in the Dell-SUNY POC included the following hardware (see Figure 3):

• Eight database servers: Dell PowerEdge 2950, each with two 2.6 GHz Intel Xeon Quad-Core processors and 32 GB RAM.

• Two private interconnect network switches connecting the eight database servers formed the fully redundant interconnect heartbeat between the database nodes.


• Fibre channel storage connections with dual host bus adapters (HBAs) per database server and dual fibre channel switches connecting the servers to a Dell/EMC CX3-40 SAN provided full redundancy for high availability and input/output (IO) workload balance.

• The Dell/EMC CX3-40 SAN with 37 spindles provided the shared storage and IO bandwidth for the ten databases.

Figure 3: Eight-Node Oracle 10g RAC Cluster Configuration

Software Stack and Databases Configuration:

The following software stack was configured on each Oracle RAC node:

• Red Hat Enterprise Linux 4.5 (64 bit)

• Oracle 10gR2 Clusterware

• Oracle 10gR2 RAC software

• Oracle ASM instance

Based on these software stacks, ten database instances were created on the Oracle RAC. Five of the database instances were “large” databases equal to one Texas Tech-sized database (able to handle 28,000 students) and five were “small” databases equal to one fourth of a Texas Tech-sized database (7000 students).

To distribute the workloads across the cluster nodes, each of the ten databases instances was initially configured to run on three of the eight RAC nodes, as illustrated in Figure 4. An instance can be added or dropped from a RAC node by defining which database services run on which nodes. Note: Node 8 was reserved for future expansion.


Figure 4: Database Instance Map on the Grid

Database Shared Storage Configuration The Dell/EMC CX3-40 with 37 disk drives provided the shared storage for all ten database instances on the Grid. To evenly distribute IOs, twenty storage logical units (LUNs) (two LUNs for each database instance) were created in such a way as to spread the IOs across the CX3-40’s two storage processors and 32 disks, as shown in Table 3. (The first 5 disks on Enclosure 0 were reserved for use by the CX3-40’s FLARE operating code). The storage was managed by Oracle Automatic Storage Management (ASM).


Enclosure # SP Disks LUN RAID

Group ASM Disk

Group Database

0 B 9, 10, 11, 12 2 1 +DG1 DB1

1 A 10, 11, 12, 13 8 4 +DG1 DB1

2 A 10, 11, 12, 13 11 5 +DG2 DB2

2 B 3, 4, 5, 14 13 7 +DG2 DB2

2 A 6, 7, 8, 9 17 6 +DG3 DB3

2 B 3, 4, 5, 14 19 7 +DG3 DB3

1 A 10, 11, 12, 13 9 4 +DG4 DB4

2 B 6, 7, 8, 9 15 6 +DG4 DB4

2 B 6, 7, 8, 9 16 6 +DG5 DB5

2 A 6, 7, 8, 9 14 6 +DG5 DB5

0 A 5, 6, 7, 8 0 0 +DG6 DB6

2 B 10, 11, 12, 13 12 5 +DG6 DB6

0 A 5, 6, 7, 8 1 0 +DG7 DB7

0 B 9, 10, 11, 12 3 1 +DG7 DB7

1 A 0, 1, 2, 3 4 2 +DG8 DB8

1 B 4, 5, 6, 7 6 3 +DG8 DB8

1 A 0, 1, 2, 3 5 2 +DG9 DB9

1 B 4, 5, 6, 7 7 3 +DG9 DB9

2 A 10, 11, 12, 13 10 5 +DG10 DB10

2 B 3, 4, 5, 14 18 7 +DG10 DB10

Table 3: SUNY POC Shared Storage Database LUN Configuration

SUNY POC Grid Database Implementation Architecture By using Oracle RAC, ASM and Dell/EMC storage, the SUNY POC Oracle Grid was implemented in four layers, each of which offers services to the layer above, as shown in Figure 5.


Figure 5: SUNY POC Grid Database Implementation Architecture

Oracle Enterprise Manager Configuration Oracle Enterprise Manager Grid control 10.2.0.4 was configured to manage the Grid infrastructure and monitor applications server and database server performance in the Grid.

Dynamic Scaling of the Oracle RAC Cluster One of the important benefits of using the Grid Computing infrastructure is to be able to dynamically scale out the database resources whenever required by the application workload. Oracle 10g RAC and Oracle Enterprise Manager provisioning pack allow administrators to provision and scale the database resource with a few simple clicks on the Enterprise Manager’s console to add or drop a node to/from a specific database instance or to expand the RAC by adding a new node.


Section 6 Test Methodology

From the many Banner functions described in Section 3 it was decided to focus on five of the most common student requests for the POC testing: student registration (both the traditional one-class-at-a-time registration and registering at one time for several courses saved to a shopping cart), viewing class lists and grades, and adding or dropping classes. To model a typical student workload the LoadRunner workload generators were programmed to simulate a specified number of users performing the various functions at any given time. These numbers are shown in Table 4 for the two different database sizes defined in the previous section.

Table 4: Simultaneous Simulated Users per Student Function

To show scaling from the previous proof of concept with Texas Tech, the large database (representing 28,000 students) and small database (representing 7,000 students) were combined as shown in Table 5 to model various sizes of universities. With the numbers of simultaneous users defined above for each size database this amounts to about 6% of the student body accessing Banner at one time (Banner has a feature that enables staggered registration dates to limit student load).

Databases Used Total Enrollment Modeled Simultaneous Simulated Users

1 Large Campus + 1 Small Campus 35,000 students 2,250

3 Large Campuses + 3 Small Campuses 105,000 students 6,750

5 Large Campuses + 5 Small Campuses 175,000 students 11,250

Table 5: University Sizes Tested

Scenario Users (Large Database) Users (Small Database)

Student Registration (Shopping Cart) 320 80

Student Registration (Traditional) 320 80

View Class List 200 50

Add/Drop Class 160 40

View Grades 800 200

Total 1,800 450


Section 7 Results

Using LoadRunner the workloads featuring the five Banner student functions shown in Table 4 were driven against the database configurations shown in Table 5 with the specified number of simulated users. Each run lasted 90 minutes including 10 minutes of ramp-up time. The key metrics were response time, CPU utilization on the database servers, and total registrations per time.

To best model actual behavior, pauses of several seconds representing realistic “think times” were inserted by the load generator in between the actions of the simulated users. The response time to an action is defined as the time from when the simulated user submits the request to the Banner web page to the time that Banner has sent the response back to the simulated user session. The average response times for the steps in the traditional student registration are shown in Figure 6. Note that the actual (sub-second) response times are much less than the two or five second requirement specified originally by Texas Tech.

Figure 6: Response Time Breakdown for Traditional Student Registration


The total response time for a traditional student registration is shown in Table 6 for the three tests. As seen, the time an average student waited for the computer to respond was about a second total for the entire set of registration actions. Table 6 also lists the average CPU Utilization of the database servers used in each of the three tests. Ranging from 25-30% utilized, these servers had plenty of headroom for user spikes.

Workload

Total Response Time for

Traditional Student

Registration Process

Number of Database Servers

Average CPU Utilization

1 Large Campus + 1 Small Campus 0.7 sec 3 25%

3 Large Campuses + 3 Small Campuses 1.1 sec 5 31%

5 Large Campuses + 5 Small Campuses 0.8 sec 6 30%

Table 6: Total Response Time and CPU Utilization

The best way to appreciate the throughput of the largest Banner configuration tested here (5 large campuses plus 5 small campuses) is to compare the student course registrations for both a single 15 minute period as well as for a one hour period versus actual data measured during the enrollment period of Buffalo State, a SUNY member school with 11,000 students. As seen in Table 7, the total number of registrations achieved by the Dell-SUNY proof of concept at its peak (in the 5 small campuses + 5 large campuses test) was 18 times higher than the highest 15 minute period seen at Buffalo State and 37 times higher than the busiest hour-long period.

Dell-SUNY POC Buffalo State (November 14, 2007)

Multiplication Factor

Maximum Student Course

Registrations in a single 15 minute

period

20,267 1,172 17x

Maximum Student Course

Registrations over a one hour

period

67,807 1,820 37x

Table 7: Total Registrations on 5+5 Configuration


Section 8 Conclusions

The results of the SUNY-Dell proof of concept achieved the intended goal of showing that SunGard Banner software running on Dell PowerEdge servers and Dell/EMC storage running Oracle Real Application Clusters could be scaled out to handle the needs of multiple large campuses. The Banner on Dell configuration was able to handle a user load representing approximately 11,000 students simultaneously registering for classes and performing other typical tasks with sub-second response times. Nearly 70,000 courses were selected in an hour, 37 times the actual number seen at a SUNY school of 11,000 students. The Oracle RAC database could handle more than 6 Texas Tech-sized databases representing approximately 175,000 students it total.

THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.

Dell, PowerEdge and PowerVault are trademarks of Dell Inc. Intel and Xeon are registered trademark of Intel Corp. Red Hat is a registered trademark of Red Hat Inc. Linux is a registered trademark of Linus Torvalds. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell disclaims proprietary interest in the marks and names of others

Pricing, specifications, availability, and terms of offers may change without notice. Taxes, fees, shipping, handling and any applicable restocking charges are extra, and vary.

©Copyright 2008 Dell Inc. All rights reserved. Reproduction in any manner whatsoever without the express written permission of Dell Inc. is strictly forbidden. For more information, contact Dell.

Information in this document is subject to change without notice.


Section 9 Acknowledgements

The authors would like to thank the following contributors to the POC:

• ITEC: Daniel Brint, Luke Clutter, Jeff Caughel, Paul Hebert, Erik Snyder, Mike Radomski, Mike Notarius, Ron Brown

• Sicas - Sue Smith, Pete Andrusyszyn, Charlie Young

• Dell - Aaron Burns, Orlando Gallegos, Erik Barnes, Jack Davis, Ericka Villalobos

• Oracle - John MacDougal, Marc Kelberman, Austin Laird, Tom Kopec

• SunGard - Pat Masterson, Rob McQue, Jennifer O'Brien

• Maranatha Group - Eveline Taylor

• F5 - Keith Keeling

• Performance Tuning Corporation - Alex Chaniotis

sungard higher education banner on dell servers …the database was managed by oracle enterprise...

Documents