Download - Alice May02
-
8/2/2019 Alice May02
1/37
-
8/2/2019 Alice May02
2/37
26 May 2002 Teo Yong Meng, NUS 2
SETI@Home
-
8/2/2019 Alice May02
3/37
26 May 2002 Teo Yong Meng, NUS 3
Cost of I dle Comput ing Cycles
Desktop Processor Utilisation
$ / processor(desktop) $ / used
$ / usedprocessor
cost of unused cycles
one desktop $1200 $300 $150 $10501000 desktops $1,200,000 $300,000 $150,000 $1,050,000
Source: Adapted from Internet Infrastructure & Services by Bear, Stearns & Co., May2001. Based on IDA tender price, the cost of a Pentium PC desktop is estimated to be$1200.
Assumpt ions:1. Deskt op ut ilizat ion is 25%; 8 hr s/ 24 hr s = 33%; f act or ing in lunch,
r est r oom, et c. a deskt op can be idle up t o 90%
2. When a pr ocessor is in used, assume a peak ut ilizat ion of 50%
-
8/2/2019 Alice May02
4/37
26 May 2002 Teo Yong Meng, NUS 4
Out line! Gr id comput ing over view! Over view of Globus! ALiCE Middlewar e
! Key f eat ur es!
Pr oducer -consumer model! Templat e-based gr id pr ogr amming! ALiCE applicat ions
! ALiCE vs Globus! Super comput er , Physical and Vir t ual
Clust er
-
8/2/2019 Alice May02
5/37
26 May 2002 Teo Yong Meng, NUS 5
Gr id Comput ing (1)
Client / Server ModelGrid Model
-
8/2/2019 Alice May02
6/37
26 May 2002 Teo Yong Meng, NUS 6
Gr id Comput ing (2)
Flexible, secur e, coor dinat ed r esour ce shar ing amongdynamic collect ions of individuals, inst it ut ions, and
resourceFrom The Anat omy of t he Gr id: Enabling Scalable Vir t ual Or ganizat ions
Enable communit ies (vir t ual or ganizat ions) t o shar egeogr aphically dist r ibut ed r esour ces as t hey pur suecommon goals -- assuming t he absence of cent r al locat ion,
cent r al cont r ol, omniscience, exist ing t r ust r elat ionships
-
8/2/2019 Alice May02
7/37
26 May 2002 Teo Yong Meng, NUS 7
Gr id Comput ing (3) Resour ce shar ing
Comput er s, st or age, sensor s, net wor ks,dat abases,
Shar ing always condit ional: issues of t r ust , policy,negot iat ion, payment ,
Coor dinat ed pr oblem solving Beyond client -ser ver : dist r ibut ed dat a analysis,comput at ion, collabor at ion,
Dynamic, mult i- inst it ut ional vir t ual or gs Communit y over lays on classic or g st r uct ur es
Lar ge or small, st at ic or dynamic
-
8/2/2019 Alice May02
8/37
26 May 2002 Teo Yong Meng, NUS 8
Gr id Comput ing (4)
A d v a n t a g e s sharing and a g g r e g a t i o n
of resources l e v e r a g i n g on resourcesyou dont own
co m p u t i n g o n d em an d - focus on business rather than
technology- reduce business costs
r e m o t e a cce ss to expensiveresources (proprietary datasets,..)
capability and scalability fault tolerance ..
Cha l l enges heterogeneity
distributed ownership dynamic behavior internetbased on best effort packetdelivery
security ease of use .
-
8/2/2019 Alice May02
9/37
26 May 2002 Teo Yong Meng, NUS 9
Design Complexit ies
consist ency GridSyst em
p e r f o r m a n c e
s h a r e d
d a t a
maint ainabilit y
s e c u r i t y
s c a l a b i l i t y
e a s e
of
u s e d e
c e n t
r a l i z
a t i o na v a i l a b i l i t y
a t o m i c i t y
m o b i l i
t y
Too manyobjectives
Not enoughprinciples!
-
8/2/2019 Alice May02
10/37
26 May 2002 Teo Yong Meng, NUS 10
Thr ee Main Obst acles
in Gr id Comput ing1) New appr oaches t o pr oblem solving
Dat a Gr ids, dist r ibut ed comput ing, peer -t o-peer ,collabor at ion gr ids,
2) St r uct ur ing and wr it ing pr ogr ams
Abst r act ions, t ools
3) Enabling r esour ce shar ing acr oss dist inctinst it ut ions
Resour ce discover y, access, r eser vat ion, allocat ion;aut hent icat ion, aut hor izat ion, policy; communicat ion; f aultdet ect ion and not if icat ion;
Pr o g r a m m i n g Pr o b l e m
Sy s t e m s Pr o b l e m
-
8/2/2019 Alice May02
11/37
26 May 2002 Teo Yong Meng, NUS 11
Globus Layer ed Gr id Ar chit ect ur e
Co n t r o l l i n g t h i n g s l o ca l l y :Access to, & control of, resources
Ta lk i n g t o t h i n g s :communication (Internet protocols)& security
Sh a r i n g s i n g l e r e so u r ce s :negotiating access, controlling use
Co o r d i n a t i n g m u l t i p l e r e s o u r c es :ubiquitous infrastructure services, app-specific distributed services
Resource
Fabric
Connectivity
Collective
Application
InternetTransport
Application
Link
I n
t er n
e t P r
o t o c ol A r
c h i
t e c t ur
e
-
8/2/2019 Alice May02
12/37
26 May 2002 Teo Yong Meng, NUS 12
Globus Layer ed Gr id Ar chit ect ur e
Applications
Core ServicesMetacomputing
DirectoryService
GRAMGlobus
SecurityInterface
ReplicaCatalog
GASS
GridFTP
LocalServices
LSF
Condor MPI
NQEPBS
TCP
AIXLinux
UDP
High-level Services and Tools
Cactus Condor-GMPI Nimrod/Gglobusrun PUNCH
Grid Status
I/O
Solaris
DRM
-
8/2/2019 Alice May02
13/37
26 May 2002 Teo Yong Meng, NUS 13
Globus Toolkit
Sof t war e t oolkit def ines a set of ser vices (gr id pr ot ocols and API s) (par t ially) implement ed as of a collect ion of t ools
Focus is on int er -domain issues, not clust er ing suppor t s collabor at ive resour ce use spanning mult iple
or ganizat ions
int egr at es wit h int r a-domain ser vices
-
8/2/2019 Alice May02
14/37
26 May 2002 Teo Yong Meng, NUS 14
Key f eat ur es
Pr oducer -consumer model
Templat e-based pr ogr amming
ALiCE Applicat ions
Globus vs ALiCE
Super comput er , physical and vir t ual gr id
-
8/2/2019 Alice May02
15/37
26 May 2002 Teo Yong Meng, NUS 15
What is ALiCE (Adapt ive and sca LableI nt er net -based Comput ing Engine)?
Client / Server Model
Grid Model
Brokered Grid ModelALiCE
server
-
8/2/2019 Alice May02
16/37
26 May 2002 Teo Yong Meng, NUS 16
ALiCE (Adapt ive and scaLableint er net - based Comput ing Engine)
" Suppor t f or development and deployment of gr idapplicat ions
" Templat e-based pr ogr amming t o mask complexit y of gr idinf r ast r uct ur e" J ob-par allelism t o maximize t hr oughput
" (J ava) obj ect -par allelism t o maximize per f or mance" Dist r ibut ed load-balancing algor it hm" Task r eplicat ions f or f ault -t oler ant and meet ing
per f or mance deadline" Dif f er ent iat ed levels of secur it y (code, dat a and r esult ) at
var ying cost s
" I mplement ed in Java and J ava J ini/ J avaSpaces
-
8/2/2019 Alice May02
17/37
26 May 2002 Teo Yong Meng, NUS 17
ALiCE Pr oducer - Consumer Model
Co n s u m e r s ( C)" interface to users" launch point for applications" collection point for results(visualization)
Re so u r ce Br o k e r ( RB )" authentication
" application execution control" resource management
" scheduling" load balancing
"
Pr o d u ce r s ( P)" provide computing power" executes tasks
-
8/2/2019 Alice May02
18/37
26 May 2002 Teo Yong Meng, NUS 18ConsumerConsumer
# J ava SDK 1.3# J ava J ini 1.1/
J avaSpaces# J ava Ref lect ion
API# Swing
ProducerProducer
Resource BrokerResource Broker
task result Task PoolTask Pool
# J ava SDK 1.3# J ava J ini 1.1/
J avaSpaces# Swing
# J ava SDK 1.3# J ava J ini
1.1/ J avaSpaces
ALiCE I MPLEMENTATI ON
-
8/2/2019 Alice May02
19/37
26 May 2002 Teo Yong Meng, NUS 19
ALiCEALiCE Consumer GUIConsumer GUI
ControlControlPanelPanel
MessagesMessages
ResourceResourceBroker InfoBroker Info
UserUserRequirementsRequirements
Task InputTask Input
-
8/2/2019 Alice May02
20/37
26 May 2002 Teo Yong Meng, NUS 20
ALiCEALiCE Producer GUIProducer GUI
Control PanelControl Panel
MessagesMessages
PerformancePerformance
TaskTaskInformationInformation
J b E i
-
8/2/2019 Alice May02
21/37
26 May 2002 Teo Yong Meng, NUS 21
J ob Execut ion
1. JobLaunchersends jar file via TCP
-
8/2/2019 Alice May02
22/37
26 May 2002 Teo Yong Meng, NUS 22
Types of Applicat ions Suppor t ed
1. Sequent ial J obs (par amet r ic comput at ion) suppor t s single-t asking pr ogr ams wit h well-
def ined met hods like main() or r un()
2. Par allel J obs - Obj ect - level Par allelism suppor t s var ious par allel progr amming modelsvia pr ogr amming t emplat es
allows t ask and result obj ect s t o beexchanged bet ween consumer s and pr oducer st hr ough r esour ce br oker
-
8/2/2019 Alice May02
23/37
26 May 2002 Teo Yong Meng, NUS 23
Templat e- based Pr ogr amming
I nt er f ace f or pr oducer t o inst ant iat eand r et ur n r esult
Result
Specif y f unct ions t o execut e atpr oducer
Ret ur n a Result obj ect
Task
Visualizer t o be invoke at consumer
Met hod t o r et r ieve r esult s
Result Collect or
I nvoked at r esour ce br oker Met hod t o send t asks t o pr oducerTaskGener at or
Funct ionTemplat e
-
8/2/2019 Alice May02
24/37
26 May 2002 Teo Yong Meng, NUS 24
J ob Execut ion
process ( )
collectResult()
Result
-
8/2/2019 Alice May02
25/37
26 May 2002 Teo Yong Meng, NUS 25
Task Generator TemplateTask Generator Template// Task Generator Templateimport alice.application.*; // im por t t he t em pla t es
public class CLASSNAME extends TaskGenerator {
// p lac e your v ar ia bles h ere
// Constructorpublic CLASSNAME(){};
// The no param et e r co ns t ruc tor i s a MUST.
public void init() { // p lac e your in i t i a l i za t ion c ode here
}// init()
/*** generateTasks() - generates tasks***/
public void generateTasks() {
// Th is i s w here th e t ask s a re genera t ed // Usua l ly t ask s a re genera t ed in a loop , // and in t h i s loop eac h ta sk i s sen t fo r // p roc ess ing by c a l l ing t he // // pub l i c vo id p roces s (Task t ) me t hod
} // generateTasks()
/*** main method**/
public static void main(String args[]) {CLASSNAME m = new CLASSNAME();m.init();m.generateTasks();
}
} // end class
-
8/2/2019 Alice May02
26/37
26 May 2002 Teo Yong Meng, NUS 26
Result Collector TemplateResult Collector Template// Template for ResultCollector//import alice.application.*; // im port t het e m p l a t e s
public class CLASSNAME extendsResultCollector {
// p lac e your v ar iab les her e //
public static void main(String args[]){
CLASSNAME MV = new CLASSNAME ();MV.init();MV.collectAllResults();
}
// th e no a rgument c ons t ruc to r MUST ex is t
public CLASSNAME() {}
public void init() { // p lac e your in i t c odes here //
}
public void collectAllResults() {
// Here i s th e resu l t handl ing c ode . // Usua l ly resu l t handl ing invo lves a loo p // t ha t repea t ed ly c a l l s the c o l lec t Resu l t () // me t hod o f t he Resu l tCo ll ec t o r supe rc l a s s .
// // Th is m eth od re t u rns a Resu l t Objec t . // // The c on t en t s o f th i s Resu l t Objec t c an be // inspec t ed fo r resu l t handl ing /proce ss ing .
}}
-
8/2/2019 Alice May02
27/37
26 May 2002 Teo Yong Meng, NUS 27
Task TemplateTask Template // Template for Task
import alice.application.*; // im por t t he t em pla t e s
public class CLASSNAME implements Task {
// p lac e you r var i ab les here // public CLASSNAME () {}
public Result execute() {
// Th i s i s w here you do your c a l c u la t ion // The r e su l t s a re s t o red in t he Resu l t c l a s s // w h i c h f u n c t i o n s a s a d a t a s t r u c t u r e // w i t h w h i c h y o u c a n s t o r e r e s u l t s o f a n y O b j e c t t y p e
}
public String toString() { // r e tu r ns a S t r ing t ha t c an be used t o ID your t a sk
}}
-
8/2/2019 Alice May02
28/37
26 May 2002 Teo Yong Meng, NUS 28
ALiCE Applicat ions
distributed ray tracing
georectification of satellite images (CRISP)
mandelbrot setN-body problem
distributedequation solver
protein alignmentand matching (BII)
Primer Search inChromosome Sequences(Nanyang Polytechnics)
G if i iG if i i f S lli If S lli I
-
8/2/2019 Alice May02
29/37
26 May 2002 Teo Yong Meng, NUS 29
Geor ect if icat ionGeor ect if icat ion of Sat ellit e I magesof Sat ellit e I mages
-
8/2/2019 Alice May02
30/37
26 May 2002 Teo Yong Meng, NUS 30
Geor ect if icat ionGeor ect if icat ion ALiCEALiCE Pr ogr amPr ogr am
l
-
8/2/2019 Alice May02
31/37
26 May 2002 Teo Yong Meng, NUS 31
ALiCEALiCE Applicat ionsApplicat ionsPr ot ein Alignment and Mat ching 50 MBchromosome dat abase, swisspr ot (NCBI BLAST d/ bser ver ), P2, 450MHz, 256MB memor y
- Sequent ial ~ 2 hour s
- ALiCE wit h 8 pr oducer s, 4000 sequences/ t ask
~1400 secondsN- body pr oblem (n=20, 000 bodies) pr edict ing t hemot ion of ast r onomical bodies in space. - Sequent ial 9428 seconds- ALiCE: 2 pr oducer s = 1109 sec, 8 pr oduces = 457sec
-
8/2/2019 Alice May02
32/37
26 May 2002 Teo Yong Meng, NUS 32
Globus vs ALiCE
Globus is a Gr id Toolkit (pr ovides set of ser vices)
I s an open syst em (user s can develop higher -level ser vices on t op of basic ser vices)
Adv: modular it y & r eusabilit y of ser vicesDis: Gr id I nf r ast r uct ur e set up is complex
app development / deployment is complex
- is lar gely plat f or m dependent (most ly UNI X)
- Resour ce Management (via GRAM) at J ob Level
-
8/2/2019 Alice May02
33/37
26 May 2002 Teo Yong Meng, NUS 33
Globus vs ALiCE ALiCE
User -Or ient ed Gr id Comput ing Engine I nt egr at es all ser vices (basic+higher level)
t o f acilit at e: Ease of inst allat ion, deployment and administ r at ion Ease of Gr id App Development / Deployment using obj ect
pr ogr amming t emplat e
- Plat f or m I ndependent (cor e ser vices implement ed inJ ava)
- Resour ce Management at Obj ect -level (f ine-gr aincont r ol)
-
8/2/2019 Alice May02
34/37
26 May 2002 Teo Yong Meng, NUS 34
Super comput er s
J une 29, 2000 I BM ASCI Whit e 8192 RS/ 6000 pr ocessor s, 12.3 TFLOPS 6 TB memor y, 160 TB disk st or age US$ 110m, 106 t ons, 28 t r act or -t r ailer
trucks
Apr il 20, 2002 NY Times J apanese Comput er is Wor ldsFast est
NEC
US$350m, occupies 4 t ennis-court
640 specialized nodes, 5104pr ocessor s
achieved 35.6 TFLOPS ver sus7 TFLOPS in ASCI Whit e
-
8/2/2019 Alice May02
35/37
26 May 2002 Teo Yong Meng, NUS 35
Cost of a Super comput ing, a Physical Clust er
and a Vir t ual Gr id of 100, 000 PCs
" I BM ASCI Whit e (2000) US$ 110m
" NEC (2002) US$ 350m
" Physical clust er of 100K I nt el P4 ~ US$200m + US$13m
(elect r icit y)
" Vir t ual clust er cost is dist r ibut ed and absor bed by PC owner s
-
8/2/2019 Alice May02
36/37
26 May 2002 Teo Yong Meng, NUS 36
Acknowledgement s
Collabor at or s:" Cent r e f or Remot e I maging, Sensing and Pr ocessing (CRI SP)" BioI nf or mat ics I nst it ut e" Nanyang Polyt echnic (School of Lif e Sciences)" The Royal I nst it ut e of Technology, Sweden
Acknowledgement : Sun Micr osyst ems
-
8/2/2019 Alice May02
37/37
26 May 2002 Teo Yong Meng, NUS 37
Thank you.Questions & Answers