IndIGOIndian Inititative in Gravitational-wave Observations

Tarun Souradeep

Preparing India for Gravitational wave Astronomy

IUCAA Retreat 2011April 21, 2011

GW Astronomy with Intl. Network of GW ObservatoriesGW Astronomy with Intl. Network of GW Observatories

LIGO-LLO: 4km

LIGO-LHO: 2km, 4kmGEO: 0.6km VIRGO: 3km

TAMA: 0.3km

LIGO-Australia?

1. Detection confidence 2. Source direction 3. Polarization info.

Courtesy: S. Dhurandhar

Courtesy: B. SchultzGWIC Roadmap

Gravitational wave Astronomy :

Courtesy: B. Schultz: GWIC Roadmap Document

GWIC: Gravitational Wave International Committee

INDIGO: the goals• Partnership in LIGO-Australia

– Advanced LIGO hardware for 1 detector shipped to Australia at the Gingin site, near Perth. NSF approval

– Australia and International partners find funds (equiv to half the detector cost ~$200M) within a year.

– Indian partnership at 15% with full data rights.

• Consolidated IndIGO membership of LSC + propose creating a Tier-2 data centre for LSC in IUCAA

• Provide a common umbrella to initiate and expand GW related experimental efforts– 3m prototype detector in TIFR (funded). Unnikrishnan– New IndIGO partners : Laser expt. IIT M, IIT K– UG summer internship at Intl GW labs & observatories.

IndIGO: Why is it a good idea?• Have a 20 year legacy and wide recognition in the Intl. GW

community. (Would not make it to the GWIC report, otherwise!)– AIGO/LIGO/EGO strong interest in fostering Indian community– GWIC invitation to IndIGO join as member (Jul 2011)

• Jump start direct participation in GW observations/astronomy– going beyond analysis methodology & theoretical prediction --- to full

participation in data acquisition, analysis and astronomy results.• For once, may be perfect time to a launch into a promising field (GW

astronomy) well before it has obviously blossomed. • Provides an exciting challenge at an International forefront of

experimental science. Can tap and siphon back the extremely good UG students trained in India. (Sole cause of `brain drain’).– 1st yr summer intern 2010 MIT for PhD– Indian experimental scientist Postdoc at LIGO training for Adv. LIGO subsystem

• Scattered , small scale, individual experimental expertise related to GW observatories have a much better prospect to thrive under the INDIGO umbrella. (Easier to conceive, plan and implement in light of a major initiative)– Sendhil Raja, RRCAT, Anil Prabhakar, EE, IIT Madras, Pradeep Kumar, EE, IITK

Photonics– Vacuum expertise with RRCAT , IPR (Ajai Kumar)

Multi-InstitutionalConsortium

1. IUCAA2. TIFR3. RRI /ICTS (?)4. RRCAT5. IPR6. CMI7. Delhi University8. IISER Kolkata9. IISER Trivandrum

• IIT Chennai?• IIT Kanpur?• Jamia Milia ?

Courtesy: Unnikrishnan

The IndIGO Consortium:Sanjeev Dhurandhar (Council Spokesperson) IUCAA, Pune Tarun Souradeep (Council) IUCAA, Pune Bala Iyer (Council chair) RRI, Bangalore C. S. Unnikrishnan (Council) TIFR, Mumbai Badri Krishnan Albert Einstein Institute, Germany Rana Adhikari Caltech, Pasadena P Ajith Caltech, Pasadena B Sathyaprakash Cardiff University T R Seshadri Delhi University Patrick Dasgupta Delhi University Anand Sengupta Delhi University ?Biplab Bhawal Independent Rajesh Nayak IISER, Kolkata Archana Pai IISER, Trivandrum Suresh Doravari Caltech, Pasadena. Ajai Kumar IPR, Gandhinagar Ranjan Gupta IUCAA, Pune Sanjay Jhingan Jamila Milia, DelhiBhim Prasad Sarma Tezpur Univ . ?Sanjit Mitra JPL/LIGO, Caltech IUCAAJiwan Mittal RRCAT, Indore S Shukla RRCAT, Indore G Rajalakshmi TIFR, Mumbai A Gopakumar TIFR, Mumbai Soumya Mohanty UTB, Brownsville Sukanta Bose Washington University, Pullman K G Arun Chennai Mathematical Institute, Chennai

Significant IUCAA presence:

• 50% of council• 3+1 faculty members• 3 +2 associates• 8 Alumni

IndIGO & IUCAA

Courtesy: Unnikrishnan

Committees:

National Steering Committee:Kailash Rustagi (IIT, Mumbai) [Chair]Bala Iyer (RRI) [Coordinator]Sanjeev Dhurandhar (IUCAA) [Co-Coordinator]D.D. BhawalkarP.D. Gupta (RRCAT)J.V. Narlikar (IUCAA)G. Srinivasan

International Advisory Committee

Rana Adhikari (LIGO, Caltech, USA)David Blair (AIGO, UWA, Australia)Adalberto Giazotto (Virgo, Italy)P.D. Gupta (Director, RRCAT, India)James Hough (GEO, GWIC Chair; Glasgow, UK)Kazuaki Kuroda (LCGT, Japan)Harald Lueck (GEO, Germany)Nary Man (Virgo, France)Jay Marx (LIGO, Director, USA)David McClelland (AIGO, ANU, Australia)Jesper Munch (Chair, ACIGA, Australia)B.S. Sathyaprakash (GEO, Cardiff Univ, UK)Bernard F. Schutz (GEO, Director AEI, Germany)Jean-Yves Vinet (Virgo, France)Stan Whitcomb (LIGO, Caltech, USA)

IndIGO & IUCAA

Courtesy: Unnikrishnan

IndIGO & IUCAA• Partnership in IGO-Australia

– Advanced LIGO hardware for 1 detector shipped to Australia at the Gingin site, near Perth. NSF approval

– Australia and International partners find funds (equiv to half the detector cost ~$200M) within a year.

– Indian partnership at 15% with full data rights.

• Consolidated IndIGO membership of LSC + Tier-2 data centre for LSC in IUCAA + IUSSTF IndoUS joint Centre at IUCAA (with Caltech)

• Provide a common umbrella to initiate and expand GW related experimental efforts– 3m prototype detector in TIFR (funded). Unnikrishnan– New IndIGO partners : Laser expt. IIT M, IIT K– UG summer internship at Intl GW labs & observatories.

The IndIGO data analysis centre

Propose for a high-throughput Computation and GW Data Archival Centre.

Tier -2 centre with data archival and computational facilities

Inter-institutional proposal for facility

Will provide fundamental infrastructure for consolidating GW data analysis expertise in India.

Courtesy: Anand Sengupta

Objectives of the data centre

LIGO Data Grid as a role model for the proposedIndIGO Data Analysis Centre.

Courtesy: Anand Sengupta

13Why IndIGO Data Centre?

Scientific pay-off is bounded by the ability to

perform computations on the data.

Scientific pay-off is bounded by the ability to

perform computations on the data.

Maximum scientific exploitation requires data analysis to proceed at same rate as data acquisition

Low latency analysis is needed if we want opportunity to provide alerts to astronomical community in the future

Computers required for LIGO flagship searches Stochastic = 1 unit (3 GHz workstation day per day of data) Bursts = 50 Compact binary inspiral = 600 (BNS), 300 (BBH), 6,000 (PBH) ...... All sky pulsars = 1,000,000,000 (but can tolerate lower latency

& ..... )

Data CentreData Centre

Courtesy: Anand Sengupta

How big is big enough?

IndIGO has world expertise in coherent analysis of gravitational wave data. This is the holy grail of GW data analysis with many advantages. Archana Pai (IISER Tvm), Anand Sengupta (Univ. of Delhi) and

K.G. Arun (CMI) have recently secured Indo-Japanese DST project for developing and testing efficient coherent methods to analyze GW data.

Niche area, would like to take lead in this

Real time zero-lag data analysis will require 10 TFlops of computation Real time can mean months or years of continuous data But this is not all we do with the data

X 100 passes for time slides (background estimation) X 1000 passes for Monte Carlo injection studies, pipeline

tuning

Target: Somewhere in the ball park of 100 Tflops.Courtesy: Anand Sengupta

Need for a IndIGO data centre Large Tier-2 data/compute centre for archival of g-wave data and

analysis Bring together data-analysts within the Indian gravity wave

community. Puts IndIGO in the global map for international collaboration

LSC wide facility would be useful for LSC participation

Functions of the IndIGO data centre Data archival: Tier-2 data centre for archival of LIGO data. This

would include data from LIGO-Australia. LIGO Data-Grid Tools for replication.

Provide Computation Power: Pitch for about 8000 cores Compare with AEI (~5000 cores), LIGO-Caltech (~1400 cores),

Syracuse cluster (~2500 cores).

Main considerations for data centre design Network: gigabit backbone, National Knowledge Network. Indian

grid! Dedicated storage network: SAN, disk space Electrical power, cooling, Air-Conditioning: requirements and design Layout of rack, cabling Hardware (blades, GPUs etc.), middleware (Condor, Globus),

software (Data Monitoring Tools, LALApps, Matlab)

IndIGO Data Centre@IUCAA

Courtesy: Anand Sengupta

Logistics involved

Courtesy: Anand Sengupta

Summary: data centre requirements 100 Tflops = 8500 cores x 3 GHz/core

Need 8500 cores to carry out a half decent coherent search for gravitational waves from compact binaries.

(1 Tflop = 250 GHz = 85 cores x 3 GHz / core)

Storage: 4x100TB per year per interferometer.

Cost ~ 25 crores (Comp. hardware alone)

3/4 crores startup - to facilitate the close Intl. interactions required with existing LSC data centres & labs . Large scale LD analysis tools training required. Summer internships, meetings/conference/schools,…

As part of planned HPC data centre at IUCAA ?

(AKK, Dipankar ‘s talk)

Courtesy: Anand Sengupta

National Knowledge Network

IndIGO data centre will need a high bandwidth backbone connection for data replication from Tier-1 centres as well as for users to use the facility from their parent institutions. NKN can potentially provide this facility between IndIGO member

institutions. Outstanding issues: International connections, EU-India Grid

The philosophy of NKN is to build a scalable network, which can expand both in the reach (spread in the country) and Speed.

Setting up a common network backbone like national highway, wherein different categories of users shall be supported.

Courtesy: Anand Sengupta

Future GWDA Plans of IndIGO (as part of LSC)

Project leads: Sanjit Mitra, T. Souradeep, S. Dhurandhar …

Extend GW radiometer work (Mitra,Dhurandhar, TS,…2009)

Implementation of the cross-correlation search for periodic sources (Dhurandhar + collab.)

Burst Sources • Formulation• Implementation

Courtesy: S. Dhurandhar

Vetoes for non-Gaussian noise for coherent detection of inspirals

• Project leads: Anand Sengupta, Archana Pai, M K Harris.

Non-Gaussian noise plagues the detector data

Vetoes have been developed in LSC for removal of non-Gaussian noise in the single detector case

For coincidence search the veto is obvious but for coherent not so.

Developing a veto for coherent is crucial – chi squared

Scope for improving the current chi squared test – Japanese collaboration

8th February Delhi Courtesy: S. Dhurandhar

Tests of General Relativity using GW observations

Project leads:   K G Arun, Rajesh Nayak and Chandra Kant Mishra, Bala Iyer

GWs are unique probes of strong field gravity. Their direct detection would enable very precise tests of GR in the dynamical and strong field regime.

Preparing data analysis algorithms for AdvLIGO in order to test GR and its alternatives is one of the important and immediate goals of LSC.

Plan to take part in the activity to develop parameter estimation tools based on Bayesian methods.

Possible collaboration with B S Sathyaprakash (Cardiff University) & P Ajith (Caltech).

Courtesy: S. Dhurandhar

Indo-US centre for Gravitational Physics and Astronomy

• Centre of Indo-US Science and Technology Forum (IUSSTF)

• Exchange program to fund mutual visits and facilitate interaction.

• Nodal centres: IUCAA , India & Caltech, US.

• Institutions:

Indian: IUCAA, TIFR, IISER, DU, CMI - PI: Tarun Souradeep US: Caltech, WSU - PI: Rana Adhikari

APPROVED for funding (Dec 2010)

LIGO-Australia: Idea and Opportunity

• The NSF approved grand decision to locate one of the planned LIGO-USA interferometer detector at Gingin site, W. Australia to maximize science benefits like baseline, pointing, duty cycle, technology development and international collaboration.

• The proposal from Australian consortium envisages IndIGO as one of the partners to realize this amazing opportunity.

- Indian contribution in hardware (end station vacuum system, and controls), Data centre, manpower for installation and commissioning.

Courtesy: Sucheta Koshti

Collective wisdom (contd.)

How much space is required for a data centre of this size Specific to the data centre design and density of racks uses Here is an example of University of Wisconsin Milwaukee’s NEMO

cluster 780 CPUs x 2 cores per CPU = 1560 cores. AMD Opteron (dated). 1400 Sq feet, 100 ton AC units This was 5-6 years ago. Now we have much higher density racks. Take 12 core per CPU (available today) = 9360 cores in the same

space! This means that a size of around 1400 sq feet would be sufficient for our purposes.

Interconnect Infiniband is NOT a requirement This brings down the cost of the data centre substantially Gravity wave analysis is Data parallel [high throughput, high data

volume driven] rather than task parallel. GigE switches will be sufficient, although high speed storage will be a

requirement.