Cluster Software Overview

Cluster Software Stack

Linux Kernel/Environment RedHat, SuSE, Debian, etc.

Cluster Software Stack

HPC Device Drivers Interconnect driver (e.g., Myrinet, Infiniband, Quadrics) Storage drivers (e.g., PVFS)

Cluster Software Stack

Job Scheduling and Launching Sun Grid Engine (SGE) Portable Batch System (PBS) Load Sharing Facility (LSF)

Cluster Software Stack

Cluster Software Management E.g., Rocks, OSCAR, Scyld

Cluster Software Stack

Cluster State Management and Monitoring Monitoring: Ganglia, Clumon, Nagios, Tripwire, Big Brother Management: Node naming and configuration (e.g., DHCP)

Cluster Software Stack

Message Passing and Communication Layer E.g., Sockets, MPICH, PVM

Cluster Software Stack

Parallel Code / Web Farm / Grid / Computer Lab Locally developed code

Cluster Software Stack

Questions: How to deploy this stack across every machine in the cluster? How to keep this stack consistent across every machine?

Software Deployment

Known methods: By hand “Add-on” method

Bring up a frontend, then add cluster packages OpenMosix, OSCAR, Warewulf

Integrated Cluster packages are added at frontend installation time

Rocks, Scyld

OpenMosix

Overview Single system image - all nodes look like one large

multiprocessor Jobs migrate from machine to machine (based on machine

load) No changes required for apps to use system

Processors supported x86 Opteron port in progress Itanium port unclear

Interconnects supported All IP-based networks

OpenMosix Installation

Install all nodes with Linux Can use RedHat, SuSE, Debian, Gentoo, etc.

Download and install OpenMosix add-on packages on each node (or configure and compile custom kernel) Kernel patches

Supports diskful and diskless

Reboot all nodes Now running OpenMosix kernel

OSCAR

Overview Imaged based installation

Bring up frontend by hand Add OSCAR packages to frontend Construct image for compute nodes within a directory

on the frontend Redhat 9.0 and Mandrake 9.0 support Includes LAM/MPI, MPICH, PVM, Maui/PBS, PVFS, Clumon

Processors supported: x86, Itanium (in beta) Interconnects: Ethernet, Myrinet

OSCAR

Diskless support in development Opteron support in development High-availability support in alpha testing

Warewulf Overview

Install frontend first Recommend using RPM-based distribution

Imaged based installation “Virtual node filesystem”

Attacks problem of generic slave node management

Standard cluster software not included Added separately Use ‘chroot’ commands to add in extra software

Supports diskful and diskless Processors supported: x86, Opteron

Scyld

Commercial distribution Single system image design

“Bproc” Based on Red Hat Enterprise Server 3.0

Processors: x86 and Opteron Itanium in development

Interconnects: Ethernet and Infiniband MPI and PVM Diskful and diskless support

Rocks

Cluster on a CD Based on Red Hat Enterprise Workstation 3.0

Processors: x86, Opteron, Itanium Interconnects: Ethernet and Myrinet

Infiniband in development

Compute node management via Red Hat’s kickstart mechanism

Diskfull only

Many Others

Clustermatic Uses LinuxBIOS

Clic From mandrakesoft

Fully Automatic Installation Debian-based

Callident Commericial version of Rocks

Qlusters Commercial version of openMosix