proactive computing: adaptability, autonomic behavior, dependability gustavo alonso computer science...
TRANSCRIPT
Proactive computing:Adaptability, autonomic behavior, dependability
Gustavo AlonsoComputer Science DepartmentSwiss Federal Institute of TechnologyETH Zü[email protected]://www.inf.ethz.ch/department/IS/iks/
©Gustavo Alonso. ETH Zürich. 2
Proactive computing
CENTRAL TOPIC in DeFINE
“The challenge is to build proactive systems that regulate themselves and reduce the involvement of humans, whether these be administrators, operators or end users. Humans can thus concentrate on the main task instead of dedicating unnecessary efforts to tasks that can be performed by computers” (Define documentation 18.11.02)
©Gustavo Alonso. ETH Zürich. 3
The All vs. All Problem: Compute a cross-
comparison of the SwissProt v38 protein database.80‘000 entries.Average sequence
length: 400 amino acid residues.
1.5 years (cpu time) on a single machine.
Several months to run just updates (using a medium size cluster).
<E><ID>CY2_RHOVI</ID><AC>P00083;</AC><DE>CYTOCHROME C2 PRECURSOR.</DE><OS>R HODOPSEUDOMONAS VIRIDIS.</OS> <OC>PROKARYOTA; GRACILICUTES; ANOXYPHOTOBACTERIA; PURPLE BACTERIA; RHODOSPIRILLACEAE.</OC><DR>PDB; 1CRY;</DR><KW>ELECTRON TRANSPORT; PHOTOSYNTHESIS; HEME; SIGNAL; 3DSTRUCTURE.</KW><SEQ>MRKLVFGLFVLAASVAPAAAQDAASGEQVFKQCLVCHSIGPGAKNKVGPVLNGLFGRHSGTIEGFAYSDANKNSGITWTEEVFREYIRDPKAKIPGTKMIFAGVKDEQKVSDLIAYIKQFNADGSKK</SEQ><RES>1.6</RES></E>
lengths=97,110 simil=275.3, PAM_dist=85.9883, offsets=5978923,5974267, identity=40.4%, similarity=12.3%
QDAASGEQVFKQCLVCHSIGPGAKNKVGPVLNGLFGRHSGTIEGFAYSDANKNS___GITWTEEVFREYIRDPKA_____|||.:||.|||||!.||. ..||.|||.|.|:.||.:||..||.||..|.|| |:.||.! . .|:.||.| QDAKAGEAVFKQCMTCHR___ADKNMVGPALGGVVGRKAGTAAGFTYSPLNHNSGEAGLVWTADNIINYLNDPNAFLKKF_________KIPGTKMIFAGVKDEQKVSDLIAYI .!. |||.|. :.:||:..|:!||:LTDKGKADQAVGVTKMTFK_LANEQQRKDVVAYL
©Gustavo Alonso. ETH Zürich. 4
Proactive vs. manual computing
0
10
20
30
Dec
17
Dec
18
Dec
19
Dec
20
Dec
21
Dec
22
Dec
23
Dec
24
Dec
25
Dec
26
Dec
27
Dec
28
Dec
29
Dec
30
Dec
31
Ja
n 0
1
Ja
n 0
2
Ja
n 0
3
Ja
n 0
4
Ja
n 0
5
Ja
n 0
6
Ja
n 0
7
Ja
n 0
8
Ja
n 0
9
Ja
n 1
0
Ja
n 1
1
Ja
n 1
2
Ja
n 1
3
Ja
n 1
4
Ja
n 1
5
Ja
n 1
6
Ja
n 1
7
Ja
n 1
8
Ja
n 1
9
Ja
n 2
0
Ja
n 2
1
0
10
20
30
Nu
mb
er o
f pro
ces
so
rs
Available processors Running jobs Succesful jobs Failed jobs Jobs on CPU
Other user
needs cluster
Disk space
shortage
Cluster busy with other
jobs
Server down for
maintenanceCluster failure
All jobs manually
killed
©Gustavo Alonso. ETH Zürich. 5
PROPOSAL A RESEARCH AGENDA ON PROACTIVE COMPUTING
©Gustavo Alonso. ETH Zürich. 6
The challenge is to do it all
Models and instrumentation
Flexible, efficient architectures
High level representations
PROACTIVE COMPUTING
©Gustavo Alonso. ETH Zürich. 7
The basis for proactive computing High level representations of complex systems and computations
programming languages are at the wrong level for doing this processes, new languages for composition, visual languages
Flexible and efficient architectures in which almost any aspect of the system can be dynamically adapted (beyond typical tuning knobs) separation of concerns and fully modular architectures reflective middleware dynamic AOP
Models of behavior and adequate instrumentation (which, in fact, is an adaptation) for acquiring the data that will be processed by adaptation algorithms that will use the flexibility of the architecture to dynamically change the system as needed better understanding and tested models of non-functional properties ability to cope with complex data sets and algorithms for mining this
information both online and offline
©Gustavo Alonso. ETH Zürich. 8
©Gustavo Alonso. ETH Zürich. 9
These are hard problems A consensus is emerging on what is needed, but many of these
goals contradict each other (an improvement here makes life very difficult there): lack of models, programming paradigms, and suitable
architectures autonomic behavior and automatic reconfiguration makes it
very complex to understand what is going on (code evolution, instrumentation, monitoring)
dynamic AOP, reflective middleware, and application extensibility change the nature of the application on the fly making it more difficult to monitor and instrument
programming for composition may actually lead to new programming paradigms where extensibility is an essential part of the language and not a middleware feature (what is better?)
end to end understanding of the problem is the key but we don’t know how to deal with composite systems
©Gustavo Alonso. ETH Zürich. 10
Dependability: autonomic behavior True dependability encompasses a broad range of issues usually ignored by
research:
Advance management support: change almost every aspect of the system at run time monitoring and dynamically alter the configuration of the system decision support mechanisms:
• what needs to be stopped if machine A is taken off-line?
• given the current load, what is the best schedule for maintenance of each sub-cluster?
Autonomic behavior: automatic rejuvenation (fast kill and reboot) automatic recovery of complex distributed computations add and remove nodes without stopping computations low level instrumentation for run time awareness model (off line nodes,
over loaded nodes, defective networks, load prediction, QoS, etc) instrumentation as a dynamic extension