the use of cluster and parallelism to support the ...€¦ · workshop open modeller - cluster...
TRANSCRIPT
Workshop Open Modeller - Cluster
The use of Cluster and parallelism to support the execution of openModeller.
Workshop Open Modeller - Cluster
� Agenda:
− LAHPC;
− Cluster;
− Activities;
− Experiments;
− Results;
− Future Work;
Workshop Open Modeller - Cluster
� Agenda:
− LAHPC;
− Cluster;
− Activities;
− Experiments;
− Results;
− Future Work;
Workshop Open Modeller - Cluster
� LAHPC (Laboratory of Architecture and High Performance Computing)
− Tools for parallel and distributed programming, respectively to shared memory machine and homogeneous and heterogeneous clusters,
− Performance analysis to parallel and distributed programs,
− Tools to parallel database,
− Memory management techniques,
− Parallel and distributed architectures;
− Grid computing.
Workshop Open Modeller - Cluster
� Agenda:
− LAHPC;
− Cluster;
− Activities;
− Experiments;
− Results;
− Future Work;
Workshop Open Modeller - Cluster
� Cluster Hardware:
− 11 nodes: 2 quad intel Xeon 2.0 GHz and 8 GB ram memory;
− Network interconnection: Gigabit and Infiniband;
− Local File System and Network File System.
� Cluster Software:
− Operational System: SUSE Linux Enterprise 10, 64 bits, XFS;
− Compilers: C, C++ and Fortran;
− MPI: LAM, Voltaire, Intel;
− Condor job scheduler;
− Ganglia cluster monitor;
− Apache Server;
− Tomcat;
Workshop Open Modeller - Cluster
Workshop Open Modeller - Cluster
Workshop Open Modeller - Cluster
Workshop Open Modeller - Cluster
Workshop Open Modeller - Cluster
� Agenda:
− LAHPC;
− Cluster;
− Activities;
− Experiment;
− Results;
− Future Work;
Workshop Open Modeller - Cluster
� Improve the performance of openModeller
− Parallelism;
− Cluster (node: multi-core architecture);
− Infiniband;
� Enable the use of cluster power processing on openModeller Desktop;
Workshop Open Modeller - Cluster
� Condor:
− Specialized workload management system for compute-intensive jobs.
� LAM-MPI:
− LAM (Local Area Multicomputer) an open source implementation of the Message Passing Interface (MPI) standard. The MPI standard is the de facto industry standard for parallel applications.
� SOA (Service Oriented Architecture):
− Architectural model to enable the integration of loosed coupling systems.
Workshop Open Modeller - Cluster
� Integration with openModeller desktop using web services;
� openModeller execution as a job:− openModeller Library build on Front Node (NFS);− Layers replicated in all the nodes;− Condor support;
� openModeller parallelization:− openModeller components parallelization using LAM;− Condor support;
Workshop Open Modeller - Cluster
� Agenda:
− LAHPC;
− Cluster;
− Activities;
− Experiment;
− Results;
− Future Work;
Workshop Open Modeller – Cluster
Workshop Open Modeller – Cluster
Workshop Open Modeller – Cluster
Workshop Open Modeller – Cluster
Workshop Open Modeller – Cluster
Workshop Open Modeller - Cluster
� Experiment:− Species: Furcata boliviana
− Layers:� WorldClim (bioclimáticos);
− Output Mask: /application/scratch/layers/WORLDCLIM/bioclimaticos/bio_10min/bio_1
− Algorithm: BioClim
− Format: GreyTiff100
− Average time 1 min 55 segundos. (serial)
− Average time 55 segundos. (parallel) 10 nodes;
Workshop Open Modeller - Cluster
� Agenda:
− LAHPC;
− Cluster;
− Activities;
− Experiment;
− Results;
− Future Work;
Workshop Open Modeller - Cluster
� Experiments:
− Furcata Boliviana:
� Algorithms:
− Bioclim
− DG GARP
− GARP
− Envelope Score
− Support Vector Machines
− Environmental Distance
− Environmental Distance (Chebyshev)�
− Environmental Distance (Mahalanobys)�
� Executions:
− From 1 to 15 process running 5 times each.
Workshop Open Modeller - Cluster
• Average Execution Time (om_project):
1 3 4 5 6 7 8 9 10 11 12 13 14 15
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
20.00
BioclimDG GARPGARPEnvelope ScoreSupport Vector M achinesEnvironmental DistanceEnvironmental Distance (Chebyshev)Environmental Distance (M ahalanobys)
Workshop Open Modeller - Cluster
� Discussion:
− Degradation level;
− Network communication;
− Map Generation;
− Layer file reading.
Workshop Open Modeller - Cluster
� Agenda:
− LAHPC;
− Cluster;
− Activities;
− Experiment;
− Results;
− Future Work;
Workshop Open Modeller - Cluster
� Future Work:
− Performance evaluations;
− Port new algorithms to openModeller: AdaptGarp, Pgarp, ... ;
− Implement an authorization infra-structure based on credentials;
− Enforcement of web services invocation by known users identified by credentials;
Workshop Open Modeller - Cluster
� Questions ?