About the EnLightened Computing ProjectPrepared for Grid Computing Classat UNCW and UNCC

Carla S. HuntSenior Solutions ArchitectMCNCApril 12, 2007

Your Presenter

Education: B.S. Computer Science, University of Wilmington M.S. Information Sciences, North Carolina Central

University Industry Experience

IBM, Network and Systems Engineer Robert Taylor Associates, Chief Internet Services

Engineer Sentinel Services, Information Technology

Consultant MCNC, Senior Solutions Architect

MCNC & NCREN

Who we are: MCNC (Microelectronics Center of NC)

We own and operate NCREN (NC Research and Education Network) One of the most advanced statewide research and education networks in

the country Provide Internet, video and related services to University of North

Carolina’s 16 campus system, Duke, Wake Forest, etc. Non-profit organization, about 50 employees

History: Commissioned by the General Assembly in 1980 to further economic

development in the state of North Carolina What we do:

Provide non-commodity Internet and related services to more than half a million users daily!

Network has an optical ring based architecture with speeds up to 2.5 Gbps

EnLIGHTened Computing Project

International recognition and collaboration NSF seed funded 10 Gbps national research testbed Grid Middleware developed to co-schedule and co-allocate

network and compute resources

Major Accomplishments: First ever demonstration of trans-pacific, inter-domain co-

scheduling of network and compute resources across two national grid computing research testbeds

First ever demonstration of External Network to Network Interface (E-NNI) capability to establish on-demand end-to-end light paths via Calient optical switches

First ever high definition video broadcast over the Internet of classroom instruction—LSU high performance computing class

EnLightened and G-Lambda GLIF Demonstration

Who: EnLIGHTened (US) and G-Lambda (Japan)

Where: 11th floor of AIST building in Tokyo, Japan

When: September 11th-13th, 2006 What: Researchers from US and Japan

demonstrated trans-pacific, inter-domain advance reservation of coordinated network and compute resources across two national research testbeds during GLIF’s 6th Annual Global Lambdagrid Workshop

Organization of Research Teams

Testbed Network and Compute Resources

ApplicationMiddlewarePerformance and monitoring

EnLIGHTened Testbed

Testbed Components

Compute Resources:ClustersStorageClient machines/displays

Network Resources:Core Optical SwitchesOther Network Devices

US Domain: Compute Resources

X1U

RA1

BT2

CH1

VC1

LA1

BT1

Caltech—LA1

4 nodes (LA1-LA4)2 nodes---4x2.2GHz Sun Opteron2 nodes---4x2.4GHz Tyan S2891 Opteron

RHELv4, 64 bitTorque and MauiGlobus Toolkit10 Gig Nic cardsShared resource sometimes

CCT at LSU: BT1

Pelican.cct.lsu.edu14 nodes8 x 1.9 GHz IBM POWER5 processors16 GB RAM per nodeAIX 5.2 w/ LoadlevelerShared resource

CCT at LSU: BT2

Santaka.cct.lsu.eduSGI shared memory32 – 1.5 GHz Intel Itanium2 processors128 GB RAM per systemSUSE Enterprise 9/ PBSPro10 Gig Nic card (I think)Shared resource

MCNC: RA1

Kite1.enlightenedcomputing.orgApproximately 40 nodes available2x2.8GHz Intel Xeon4 GB RAMRHELv4Torque/MauiGlobus Toolkit v 4.0.2Shared resource

VCL at NCSU: VC1

vclb5-1.hpc.ncsu.eduIBM Blade CenterRHELv4Torque/MauiGlobus Toolkit v 4.0.2Dedicated Resource

Coordinating Compute Resources

Policy for shared & borrowed resources User accounts Security

Installation and configuration of Operating systems Cluster management software (scheduler, resource mgr) Grid middleware (globus toolkit and grid security)

IP address management Other

Application requirements (software libraries or pkgs) Performance tuning (tcp parameters for network card)

KDDINRM

CRM

Cluster

CRM

Cluster

CRM

Cluster

NTTNRM

CRM

Cluster

CRM

Cluster

ELNRM

CRM

Cluster

CRM

Cluster

CRM

Cluster

JAP

AN

CRM CRM CRMCRM CRM CRM

USApplication

(MPI)Application

(Visualization)

Request Networkbandwidthand Computers

ReservationFrom xx:xx to yy:yy

Request Networkbandwidthand Computers

ReservationFrom xx:xx to yy:yy

JapanApplication

GLGrid Resource

Scheduler

EL→GLwrapper

KDDINRM

CRM

Cluster

CRM

Cluster

CRM

Cluster

NTTNRM

CRM

Cluster

CRM

Cluster

ELNRM

CRM

Cluster

CRM

Cluster

CRM

Cluster

USApplication

ELGrid ResourceCoordinator/

App. Launcher

EL→GLGNS-WSIwrapper

GNS-WSI

JAP

AN

US

GL: G-lambdaEL: Enlightened

GL request

EL request

G-lambda/Enlightened middleware coordination diagram

EL→GLCRM

wrapperHARC

Acceptor

Resource map

X2S

Japan South

Japan North US

X2N

X1N X1U

X1S

KMFFUK

KHN

AKB

RA1

BT2

KAN

TKB

OSA

CH1