workshop on hpc in india prof: n.pradhan department of psychopharmacology national institute of...

Workshop on HPC in India

Prof: N.PradhanDepartment of PsychopharmacologyNational Institute of Mental Health &

Neurosciences, Bangalore 560029 (India)[email protected]

ATIP 1st Workshop on HPC in India @ SC-09

HIGH PERFORMANCE COMPUTING AT NIMHANS AND ITS IMPACT ON MENTAL

HEALTH RESEARCH IN INDIA.

•Introducing to the Need for HPC in Mental Health Research

•Brief Summary of Research Areas using HPC at NIMHANS

•How we built our HPC and its performance

• Conclusion

HIGH PERFORMANCE COMPUTING AT NIMHANS AND ITS IMPACT ON

MENTAL HEALTH RESEARCH IN INDIA.

N PRADHAN ATIP 1st Workshop on HPC in India @ SC-09 2

Introducing to the Need for HPC in Mental Health Research

•MENTAL HEALTH IS MULTI-DIMENSIONAL ISSUES RANGE FROM MOLECULE TO PSYCHOLOGY, SOCIAL SCIENCES, LAW AND HUMAN RIGHTS.

MENTAL HEALTH INSTITUTES / NEUROSCIENCE INSTITUTES IN INDIA

NONE OTHER THAN NIMHANS DEAL WITH HUMAN BRAIN & BEHAVIOUR

UNDERSTANDING HUMAN BEHAVIOUR HPC“MOLECULAR AND SYSTEMS LEVEL” IN HEALTH AND DISEASE STATES: Diagnosis, Monitoring, Therapeutic Response Evaluation and Prognostic.


NIMHANS AND HPC

HUMAN BRAIN OCCUPIES THE CENTER STAGE EVERYWHERE

IT IS TOO COMPLEXImmense Synaptic ConnectivitiesUnderstanding Brain Functions Remains Descriptive Medical Observation with Brain Lesions.(No Definite Model for Any Function)THE HPC NEEDHIGH RESOLUTION MAPPING

Both Structural and Functional: Example Epileptic Surgery

MOLECULAR APPROACH TO DIAGNOSIS AND THERAPY

Example: Myopathies, Degenerative disorders


NIMHANS AND HPCIT IS IMPLIED THAT WHILE WE ADDRESS THE ISSUES OF THE BRAIN HPC IS AN IMPORTANT TOOL OF INVESTIGATIVE NATURE

BRAIN DOES NOT COMPUTE : WE MAKE A CARICATUTRAL MODEL OF THE BRAIN

BEHAVIOURAL SCIENCES AND MENTAL HEATH HAVE BEEN KEPT AWAY FROM SUPERCOMPUTING.

SUPERCOMPUTERS ARE EXPENSIVE AND NONETHELESS INFINITESIMALLY SMALL IN COMPUTING POWER TO ADDRESS ANY QUESTION RELATED TO BRAIN AND ITS BEHAVIOUR.

IT NOT POSSIBLE TO EMULATE HUMAN PERCEPTUAL-COGNITIVE BEHAVIOUR AND EMOTIONS IN ANY SUPERCOMPUTER

HOWEVER REFINEMENT OF CERTAIN ROUTINE CLINICAL PROCEDURES NEED HPC NOW (DATA CENTERS AND HMS)


Summary of Research Areas using HPC at NIMHANS

PURPOSE OF BUILDING OUR HPC

MASSIVE SCALE BRAIN SIGNAL PROCESSING(BRAIN DYNAMICS)

fMRI MAGE PROCESSING

TRAINING AND RESEARCH IN MOLECULAR PHARMACOLOGY

BIOINFORMATICS AND MOLECULAR DYNAMICS

NEURAL NETWORKS SIMULATIONS


BRAIN SIGNAL PROCESSING

MULTICHANNEL BIOMEDICAL SIGNALS

32 – 256 CHANNELS OF DATASAMPLING 256 TO 40KDURATION 10 SEC TO 8 HOURS TO DAYS

Total Data size could in GBs MEG may be added this Year

EPILEPSY MONITORINGICU (COMA, STROKE ETC)ANAETHESIASLEEP ELECTRO-CONVULSIVE THERAPY AUTONOMIC DYSFUNCTIONS


BRAIN SIGNAL PROCESSINGS

THERE IS NO LIMIT TO HOW MUCH OF MATHEMATICS BE APPLIED

TO UNDERSTAND THE PATTERN FEATURES OF EEG IN VARIOUS NEUROLOGICAL AND PSYCHIATRIC CONDITIONS

The Fundamental Requirements•SOURCE localization•Functional Characterization (Multimodal)• Modeling •Simulation

In Conjunction with SPECT MRI / fMRI


NIMHANS AND HPC (APPLICATIONS)


BRAIN SIGNAL PROCESSINGBRAIN DYNAMICS

PRE-PROCESSING:

FILTRATION AND ARTIFACT REMOVALIN BAND NOISE CANCELLATION

POST -PROCESSING:

LINEAR METHODSNONLINEAR METHODS


NIMHANS AND HPC APPLICATIONS

•NONLINEAR DYAMICAL INVARIANTS


NIMHANS AND HPC APPLICATIONS •NONLINEAR DYAMICAL INVARIANTS



fMRI Image Processing



•fMRI Image Processing



IMAGING BRAIN ACTIVITY

A 3-D View of the Brain Imaging software combines data from several imaging technologies to create an interactive 3-D map of the brain.

Reading Thoughts with Brain Imaging

Researchers use fMRI to determine the contents of short-term memory. fMRI and Yoga



MOLECULAR DYAMICS NMDA



MOLECULAR DYAMICS α-synuclein



MOLECULAR DYAMICS GAD



MOLECULAR DYAMICS INHA-NADH


NIMHANS AND HPC (BUILDING THE HPC)

IBM server 326 16 Nodes



SUN V40Z TWO Nodes


THE HPC The Sever In place is Cluster Built on Octal SMP Servers of HP DL 785

(8 No)

EACH NODE contained (In addition to Standard features)

8x 8384 Opteron 2.7 GHZ Quad Core Processors 128 GB RAM (Opeating at 533 GHZ –Fully Populated DIMMS) 4x 300GB highest supported speed SAS HDD 1xDVD ROM,

IB Inter-Conncect: Infiniband Card 4xDDR PCIe-dual port Con-X IB Card.

Manually Clustered without any Clustering Kit or software on Fedora 10 (Kernel 2.6.27)

Non-Reducing Loop Topology.



HP DL875 8 NODES



ALPHA –PROCESSOR-NON Availability which made us look for other Processors

CHOICE: (COST FACTOR ONLY) BUILDING USING THE AMD OPTERON

SIMPLEST WAY - TOTAL OPEN SOURCE STARTING FEDORA 6 through FEDORA 10

TECHNIQUE:DETECT HARDWARES

MAKE IB OVER IPALL NODES ARE ON ONE SUBNET

OPENSM - Infiniband subnet manager and administration (SM/SA) Opensm 3.3.2 TO SCAN IB HARDWAREFABRIC AND MAKE SUBNET UP LAMBOOT PROCESS to make Resources available as the user desires.LAM MPI 7.1 .5b (LAM-MP connects well on Open IB)



Development of HPC step Wise Vector Length of the Test problem to be decided Processor Core status: Core to Core SMP Steams

are synchronous Physics of Problem. The length of the iterations in,

femto, Pico seconds time steps to Nano Seconds Path Modulation by streaming data in Multi Channel

Modethrough Dual Path IB Interconnect

Core path structure of Folding and Unfolding in Protein complexes

Protein Dock to Undock Rendezvous iterations in multiple data inputs and Outputs

Number of the Compute Cores in Cycle Change -256. Data analysis of foot prints in Molecular Motion.


THE BENCHMARKThe MD Simulation benchmark scaling is nearly linear and 100%, despite other users running jobs concurrently.

ONE NODE: 32 Computing core: NODE (s) Real (s) (%) Time: 3398.600 3399.000 100.0 56:38 (Mnbf/s) (GFlops) (ns/day) (hour/ns) Performance: 34.273 1.207 0.254 94.387 FOUR NODES: 128 Computing Cores (IB Connect) Average load imbalance: 4.2 % Part of the total run time spent waiting due to load imbalance: 2.2 % Parallel run - timing based on wallclock. NODE (s) Real (s) (%) Time: 696.000 696.000 100.0 11:36 (Mnbf/s) (GFlops) (ns/day) (hour/ns) Performance: 167.316 5.880 1.242 19.329 EIGHT NODES: 256 Computing Cores (IB Connect) Average load imbalance: 7.9 % Part of the total run time spent waiting due to load imbalance: 3.8 % Parallel run - timing based on wallclock. NODE (s) Real (s) (%) Time: 370.000 370.000 100.0 6:10 (Mnbf/s) (GFlops) (ns/day) (hour/ns) Performance: 314.768 11.060 2.336 10.276



CONCLUSION

WE have shown

HPCNEEDHOW TO BUILDWhat is its Performance

LOW COST HPC can be built manually using Total Open Source. One need not pay extraordinary price for proprietary interconnects.


THE HPC The Sever In place is Cluster Built on Octal SMP Servers of HP DL 785

(8 No)

EACH NODE contained (In addition to Standard features)

8x 8384 Opteron 2.7 GHZ Quad Core Processors 128 GB RAM (Opeating at 533 GHZ –Fully Populated DIMMS) 4x 300GB highest supported speed SAS HDD 1xDVD ROM,

IB Inter-Conncect: Infiniband Card 4xDDR PCIe-dual port Con-X IB Card.

Manually Clustered without any Clustering Kit or software on Fedora 10 (Kernel 2.6.27)

Non-Reducing Loop Topology.


workshop on hpc in india prof: n.pradhan department of psychopharmacology national institute of...

Documents