17 th october, 2006pragma 11, beautiful osaka, japan complaints to resource group habibah a wahab,...
TRANSCRIPT
![Page 1: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/1.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
COMPLAINTS TO RESOURCE GROUPCOMPLAINTS TO RESOURCE GROUP
Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat
School of Pharmaceutical Sciences,
Unversiti Sains Malaysia
![Page 2: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/2.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
MIGRATING AMBER to GRID
• SYSTEM REQUIREMENTSYSTEM REQUIREMENT– Software: Globus 2.x, 3.x or 4.x Fortran 90 compiler
– Hardware: ~50GB of disk space Linux on 32bit Intel machine
![Page 3: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/3.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
HOW WE BEGAN…
• Contact Cindy for testing resources.• Allocated Resources:
– USM – hawk.usm.my– USM – aurora.cs.usm.my– ROCK- 52 – rock-52.sdsc.edu– ASCC – pragma001.grid.sinica.edu.tw– IOIT-HCM – venus.ioit-hcm.ac.vn– UNAM – malicia.super.unam.mx
– Thank You, Cindy!
![Page 4: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/4.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
HOW WE BEGAN…
• Contact Cindy for testing resources.• Allocated Resources:
– USM – hawk.usm.my– USM – aurora.cs.usm.my– ROCK- 52 – rock-52.sdsc.edu– ASCC – pragma001.grid.sinica.edu.tw– IOIT-HCM – venus.ioit-hcm.ac.vn– UNAM – malicia.super.unam.mx
– Thank You, Cindy!
Contacting th
e syste
m administrators
are fine, but is
there any sy
stem th
at
we could just
submit our jo
b without
worrying about w
here they will b
e
executed ?
![Page 5: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/5.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
WHAT WE ENCOUNTERED….
• Hardware:– Heterogeneous architecture between clusters
• Globus Authentication:– Requires users account in all clusters– Globus’s user certificate setup on each cluster – The cert need to be signed by institution CA admin. – User have to know all clusters in PRAGMA (host address
and total of nodes on each site).– Certain port cannot be accessed.
• e.g: gsiftp port – for file transfer
![Page 6: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/6.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
WHAT WE ENCOUNTERED….
• Hardware:– Heterogeneous architecture between clusters
• Globus Authentication:– Requires users account in all clusters– Globus’s user certificate setup on each cluster – The cert need to be signed by institution CA admin. – User have to know all clusters in PRAGMA (host address
and total of nodes on each site).– Certain port cannot be accessed.
• e.g: gsiftp port – for file transfer
This is o
kay, a lo
t of w
ork but we wish
this process
could be simpler…
..
![Page 7: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/7.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
more encounters….
• MPICH/MPI– No standard parallel software on the grid– e.g: MPICH (ASCC, UNAM, hawk, IOIT-HCM, aurora), LAM
(rocks-52) – User need to know whether mpich/lam is configured by
ssh/rsh
• rsh or ssh?– setting up rsh/ssh without password between execution
nodes. – non-standardized usage of rsh/ssh on the grid. Some
clusters are using rsh and others are using ssh. – e.g :
– rsh – IOIT-HCM – ssh – hawk, aurora, ASCC, UNAM, rocks-52
![Page 8: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/8.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
more encounters….
• MPICH/MPI– No standard parallel software on the grid– e.g: MPICH (ASCC, UNAM, hawk, IOIT-HCM, aurora), LAM
(rocks-52) – User need to know whether mpich/lam is configured by
ssh/rsh
• rsh or ssh?– setting up rsh/ssh without password between execution
nodes. – non-standardized usage of rsh/ssh on the grid. Some
clusters are using rsh and others are using ssh. – e.g :
– rsh – IOIT-HCM – ssh – hawk, aurora, ASCC, UNAM, rocks-52
How we wish th
ere is a st
andard
parallel so
ftware and rs
h/ssh ru
nning
on all the cluste
rs in pragma
testbed….
![Page 9: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/9.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
still more …..
Compiling parallel AMBER– Unable to compiled with
mpich/lam in the cluster.– Can compile amber-mpich
in rocks-52, BUT… 1. CANNOT BE EXECUTED USING
GLOBUS (Figure 1)2. CAN BE EXECUTED USING
GLOBUS, but run on one node only
![Page 10: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/10.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
But there is hope for us….
• executable file can be copied between clusters with similar architecture and mpich configuration.– executables copied from HAWK to UNAM,
aurora, IOIT-HCM (mpich-configured with rsh)– executables copied from rocks-52 to ASCC
(mpich-configured with ssh )Wilfred sa
id that G
farm can overcome
this problem… Is
it true Tatebe-sa
n?
![Page 11: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/11.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Testing AMBER with Globus
• Testing execution on each cluster, using globus from hawk to all sites.
• Testing gsiftp for sending and receiving files using from hawk-other cluster.
• Network Condition– Globus submission depends on the network condition.– Globus submission may fail, yet, the user will not know…
• Cluster reliability– unexpected cluster problem. System may down or cannot be
access due many factors.
• Or… globus was just not working.
![Page 12: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/12.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Testing AMBER with Globus
• Testing execution on each cluster, using globus from hawk to all sites.
• Testing gsiftp for sending and receiving files using from hawk-other cluster.
• Network Condition– Globus submission depends on the network condition.– Globus submission may fail, yet, the user will not know…
• Cluster reliability– unexpected cluster problem. System may down or cannot be
access due many factors.
• Or… globus was just not working.
Cindy, Sue gave up. In
stead of w
orking on 6 cluste
rs you
allocated to
us:
USM – aurora.cs
.usm.m
y
ROCK- 52 – r
ock-52.sdsc.
edu
ASCC – pragma001.grid
.sinica
.edu.tw
IOIT-HCM – venus.io
it-hcm
.ac.vn
UNAM – malici
a.super.u
nam.mx,
She just
work with 4 clu
sters:
Aurora – 300K
ASCC – 373K, 5
00K
IOIT-HCM – 4
00K
UNAM – 473K
I think you know why…..
![Page 13: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/13.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Web Interface?
– Too many commands to remember & things to do to run AMBER on the grid
– Web is more user-friendly. – But, it employs dynamic programming to
process user’s command to run on the grid – But, must understand the application (amber)
work flow and input files.– With this user can simply run and concentrate
on the simulation.
![Page 14: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/14.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
AMBER Work Flow
Structure
Coordinates
Force Field &
Topology
Creator
Minimiser/
MD
simulator
Trajectory
Analyser
PDB, XYZ, Internal Coord.
Junk in, Junk out!
Prmtop, prmcrd Mdin Md.OutEn.outTrj.files
Grid MiddlewareUser
Simulator Engine
![Page 15: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/15.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
User interface
Hawk
Rocks-52
ASCC
Aurora
IOIT-HCM
Gsiftp
inpu
ts &
resu
ltsGlo
bus-
subm
it
jobs
Gsiftp inputs & results
Globus-submit jobs
Gsiftp inputs & resultsGlobus-submit jobs
Gsiftp inputs &
results
Globus-subm
it jobs
Upload files/submit jobs
Download & view results
![Page 16: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/16.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
http://hawk.usm.my/AMEXg
TESTING…..
Thermo-effects of Methionine Aminopeptidase:Molecular Dynamics Studies
![Page 17: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/17.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Globus-job-submit….
• submitted 5 jobs(5 different temperatures of the same system) to 4 different clusters.
• Each job will occupy any empty cluster. • List of clusters and jobs:
– Aurora – 300K– ASCC – 373K, 500K– IOIT-HCM – 400K– UNAM – 473K
• Simulation time: 20ps
![Page 18: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/18.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Benchmarking
• AMEXg Benchmark:• Submit 4 different temperatures for the
same system to 4 different clusters.• List of clusters and jobs:
– Aurora – 300K [Running on 16 nodes]– ASCC – 373K [Running on 4 nodes]– IOIT-HCM – 400K [Running on 8 nodes]– UNAM – 473K [Running on 8 nodes ]
• Simulation time: 20ps
![Page 19: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/19.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Checking……
• Transferring input files from hawk to other clusters
![Page 20: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/20.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Checking……
Aurora clusterAurora cluster
Receiving files from hawk
Job submitted from hawk
![Page 21: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/21.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Checking……
Receiving files from hawk
Job submitted from hawk
ASCC clusterASCC cluster
![Page 22: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/22.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Checking……
Receiving files from hawk
Job submitted from hawk
IOIT-HCM clusterIOIT-HCM cluster
![Page 23: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/23.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Checking……
Receiving files from hawk
Job submitted from hawk
UNAM clusterUNAM cluster
![Page 24: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/24.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Checking……
Receiving files from hawk
Transferring/copying output files from clusters to hawk
![Page 25: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/25.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Interface displayed after uploading input files using AMEXg
![Page 26: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/26.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Aurora clusterAurora cluster
Transferring output files to hawk
![Page 27: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/27.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
ASCC ASCC clustercluster
Transferring output files to hawk (cont.)
![Page 28: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/28.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
IOIT-HCM clusterIOIT-HCM cluster
Transferring output files to hawk (cont.)
![Page 29: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/29.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
UNAM clusterUNAM cluster
Transferring output files to hawk (cont.)
![Page 30: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/30.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
List of output files
Result for MD simulationResult for MD simulation
![Page 31: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/31.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Benchmarking
Aurora – 300KASCC – 373K
UNAM – 473K IOIT-HCM – 400K
![Page 32: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/32.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Benchmarking
Aurora – 300KASCC – 373K
UNAM – 473K IOIT-HCM – 400K
This is f
ar from perfe
ct…. W
e are working
with Grid
Sphere with Chan Huah Yong. B
ut
we are extremely happy th
at we can ru
n
our applicatio
ns on th
e grid. If
it is o
kay, we
would like to
run th
e applications f
rom time
to time on th
e testb
ed…. But s
oon, we
need to th
ink about the lic
encing issue,
because AMBER is not fr
ee….
![Page 33: 17 th October, 2006PRAGMA 11, Beautiful Osaka, Japan COMPLAINTS TO RESOURCE GROUP Habibah A Wahab, Suhaini Ahmad, Nur Hanani Che Mat School of Pharmaceutical](https://reader035.vdocument.in/reader035/viewer/2022070305/55149c10550346b2598b58a4/html5/thumbnails/33.jpg)
17th October, 2006 PRAGMA 11, Beautiful Osaka, Japan
Sipadan Island, Sabah, Malaysia
Thank you!