introduction to the compact binary coalescence search...
TRANSCRIPT
Introduction to the Compact Binary
CoalescenceSearch Pipeline
Duncan BrownSyracuse University
Overview of Lectures
Overview of Lectures
Dr. Marion
CBC Gravitational WaveData Analysis
Overview of Lectures
Dr. Marion
CBC Gravitational WaveData Analysis
Dr. Koranda
The LIGO Data Grid
Overview of Lectures
Dr. Marion
CBC Gravitational WaveData Analysis
Dr. Koranda
The LIGO Data Grid
These Lectures:
How do we put everything together to search for
CBCs?
Overview of Lectures
Overview of Lectures
• Lecture I: Introduction to CBC pipelines
Overview of Lectures
• Lecture I: Introduction to CBC pipelines
• Lecture II: Interpreting the results
Overview of Lectures
• Lecture I: Introduction to CBC pipelines
• Lecture II: Interpreting the results
• Lecture III: Setting up and running the ihope pipeline
Lecture I:CBC Pipelines
Search Pipelines
Search Pipelines
• Getting from raw detector data to a detection statement requires many related steps which depend on each other
Search Pipelines
• Getting from raw detector data to a detection statement requires many related steps which depend on each other
• Putting these steps together is and executing them in the correct order is the job of the search pipeline
Terminology
Pipeline
Terminology
PipelineWhat we want to do and
how we want to do it.
Terminology
Pipeline
Workflow
What we want to do and how we want to do it.
Terminology
Pipeline
Workflow
What we want to do and how we want to do it.
A specific realization of a pipeline for given detector(s) and time(s)
Terminology
Pipeline
Workflow
Condor DAG,Condor submit files &
Executables
What we want to do and how we want to do it.
A specific realization of a pipeline for given detector(s) and time(s)
Terminology
Pipeline
Workflow
Condor DAG,Condor submit files &
Executables
What we want to do and how we want to do it.
A specific realization of a pipeline for given detector(s) and time(s)
The code that runs on the cluster and does the actual work
Terminology
Pipeline building blocks
Choose your detectors
Choose a source
m1(M⊙)
m2(M
⊙)
Choose a source
m1(M⊙)
m2(M
⊙)
Low Mass CBC
Choose a source
m1(M⊙)
m2(M
⊙)
Low Mass CBC
High MassCBC
Choose a source
m1(M⊙)
m2(M
⊙)
(1, 1)Low Mass CBC
High MassCBC
Choose a source
m1(M⊙)
m2(M
⊙)
(1, 1)
M = 35
Low Mass CBC
High MassCBC
Choose a source
m1(M⊙)
m2(M
⊙)
(1, 1)
M = 35
M = 100
Low Mass CBC
High MassCBC
Choose your waveforms
m1(M⊙)
m2(M
⊙)
Filtering Engine
• The matched-filtering engine for the LSC-Virgo CBC pipeline is FindChirp
Filtering Engine
• The matched-filtering engine for the LSC-Virgo CBC pipeline is FindChirp
• FindChirp generates the signal-to-noise ratio (SNR) and chisq-veto time series using data from a single detector
Filtering Engine
• The matched-filtering engine for the LSC-Virgo CBC pipeline is FindChirp
• FindChirp generates the signal-to-noise ratio (SNR) and chisq-veto time series using data from a single detector
• It then generates triggers: times when the SNR and chisq-veto satisfy certain conditions
Filtering Engine
Coincidence Test
Coincidence Test
• Coincidence is one of the most powerful methods for eliminating false alarms
Coincidence Test
• Coincidence is one of the most powerful methods for eliminating false alarms
• This combines the results of the filtering engines and determines coincident triggers
Coincidence Test
• Coincidence is one of the most powerful methods for eliminating false alarms
• This combines the results of the filtering engines and determines coincident triggers
Coincidence Test
• Coincidence is one of the most powerful methods for eliminating false alarms
• This combines the results of the filtering engines and determines coincident triggers
Coincidence Test
• Coincidence is one of the most powerful methods for eliminating false alarms
• This combines the results of the filtering engines and determines coincident triggers
Not Coincident
Coincidence Test
• Coincidence is one of the most powerful methods for eliminating false alarms
• This combines the results of the filtering engines and determines coincident triggers
Not Coincident
Coincidence Test
• Coincidence is one of the most powerful methods for eliminating false alarms
• This combines the results of the filtering engines and determines coincident triggers
Not Coincident
Coincidence Test
• Coincidence is one of the most powerful methods for eliminating false alarms
• This combines the results of the filtering engines and determines coincident triggers
Not Coincident
Coincident
Pipeline Components
• Each component of the pipeline is written as a separate program
Pipeline Components
• Each component of the pipeline is written as a separate program
• Read data in from, process it, write out to disk
Pipeline Components
• Each component of the pipeline is written as a separate program
• Read data in from, process it, write out to disk
• Command line arguments determine the specific behavior of the program (e.g. thresholds)
Pipeline Components
• Each component of the pipeline is written as a separate program
• Read data in from, process it, write out to disk
• Command line arguments determine the specific behavior of the program (e.g. thresholds)
• For the LSC-Virgo search pipeline, the programs can be found in lalsuitehttps://www.lsc-group.phys.uwm.edu/daswg/projects/lalsuite.html
Pipeline Components
Pipeline Components
Pipeline Components
lalapps_tmpltbank
Pipeline Components
lalapps_tmpltbankRead in h(t) data and compute a template bank valid for that data
Pipeline Components
lalapps_tmpltbank
lalapps_inspiral
Read in h(t) data and compute a template bank valid for that data
Pipeline Components
lalapps_tmpltbank
lalapps_inspiral
Read in h(t) data and compute a template bank valid for that data
Read in h(t) data and a template bank and generate single-detector triggers
Pipeline Components
lalapps_tmpltbank
lalapps_inspiral
lalapps_thinca
Read in h(t) data and compute a template bank valid for that data
Read in h(t) data and a template bank and generate single-detector triggers
Pipeline Components
lalapps_tmpltbank
lalapps_inspiral
lalapps_thinca
Read in h(t) data and compute a template bank valid for that data
Read in h(t) data and a template bank and generate single-detector triggers
Read in triggers from multiple detectors and determine coincidence
Pipeline Topology
Pipeline Topology
• We need to decide how to chain these (and other) executables together to make a pipeline
Pipeline Topology
• We need to decide how to chain these (and other) executables together to make a pipeline
• There is no “right” way to do this: different approaches may be better for different situations
Designing a Pipeline
Designing a Pipeline
• Things to consider:
Designing a Pipeline
• Things to consider:
• Type of source (low-mass, high-mass, GRB, etc.)
Designing a Pipeline
• Things to consider:
• Type of source (low-mass, high-mass, GRB, etc.)
• Desired latency (minutes, hours, days, etc.)
Designing a Pipeline
• Things to consider:
• Type of source (low-mass, high-mass, GRB, etc.)
• Desired latency (minutes, hours, days, etc.)
• Computational cost
Two Examples of Pipeline Topologies
inspiral_online_pipe
inspiral_online_pipe
• Sometimes called “daily ihope”
inspiral_online_pipe
• Sometimes called “daily ihope”
• Single detector pipeline used in CBC detector characterization
inspiral_online_pipe
• Sometimes called “daily ihope”
• Single detector pipeline used in CBC detector characterization
• Don’t need to search all the data: just get an idea of how the search behaves
inspiral_online_pipe
• Sometimes called “daily ihope”
• Single detector pipeline used in CBC detector characterization
• Don’t need to search all the data: just get an idea of how the search behaves
• Keep computational cost low
inspiral_online_pipe
• Sometimes called “daily ihope”
• Single detector pipeline used in CBC detector characterization
• Don’t need to search all the data: just get an idea of how the search behaves
• Keep computational cost low
• Generate summary results quickly
ligo_data_find
Locates the h(t) data on the cluster
lalapps_tmpltbank
ligo_data_find
Create a template bank for the low-mass region
lalapps_tmpltbank
lalapps_splitbank
ligo_data_find
Split the bank into pieces for faster processing
lalapps_tmpltbank
lalapps_inspiral
lalapps_splitbank
lalapps_inspiral lalapps_inspiral...
ligo_data_find
Filter the data through each template bank
lalapps_tmpltbank
lalapps_inspiral
lalapps_splitbank
lalapps_inspiral lalapps_inspiral...
ligo_data_find
ligolw_add
Combine the results of the individual filtering jobs
lalapps_tmpltbank
lalapps_inspiral
lalapps_splitbank
lalapps_inspiral lalapps_inspiral...
ligo_data_find
ligolw_add
Plot results
lalapps_inspiral_hipe
lalapps_inspiral_hipe
• hierarchal inspiral pipeline
lalapps_inspiral_hipe
• hierarchal inspiral pipeline
• Used in the LSC-Virgo low- and high-mass searches
lalapps_inspiral_hipe
• hierarchal inspiral pipeline
• Used in the LSC-Virgo low- and high-mass searches
• All-sky, all-time (i.e. search a whole ~ 1 year science run) from all detectors
lalapps_inspiral_hipe
• hierarchal inspiral pipeline
• Used in the LSC-Virgo low- and high-mass searches
• All-sky, all-time (i.e. search a whole ~ 1 year science run) from all detectors
• Demand multi-detector coincidence
lalapps_inspiral_hipe
• hierarchal inspiral pipeline
• Used in the LSC-Virgo low- and high-mass searches
• All-sky, all-time (i.e. search a whole ~ 1 year science run) from all detectors
• Demand multi-detector coincidence
• Keep computational cost “reasonable”
H1 lalapps_inspiral
H1 ligo_data_find
lalapps_tmpltbank
lalapps_tmpltbank
H1 lalapps_inspiral L1 lalapps_inspiral
L1 ligo_data_findH1 ligo_data_find
lalapps_tmpltbank
lalapps_tmpltbank
H1 lalapps_inspiral L1 lalapps_inspiral V1 lalapps_inspiral
L1 ligo_data_findH1 ligo_data_find V1 ligo_data_find
lalapps_tmpltbank lalapps_tmpltbank
lalapps_tmpltbank
H1 lalapps_inspiral L1 lalapps_inspiral V1 lalapps_inspiral
L1 ligo_data_find
lalapps_thinca
H1 ligo_data_find V1 ligo_data_find
lalapps_tmpltbank lalapps_tmpltbank
lalapps_tmpltbank
H1 lalapps_inspiral L1 lalapps_inspiral V1 lalapps_inspiral
L1 ligo_data_find
lalapps_thinca
H1 ligo_data_find V1 ligo_data_find
lalapps_tmpltbank lalapps_tmpltbank
trig_to_tmpltbanktrig_to_tmpltbank trig_to_tmpltbank
lalapps_tmpltbank
H1 lalapps_inspiral L1 lalapps_inspiral V1 lalapps_inspiral
L1 ligo_data_find
lalapps_thinca
H1 ligo_data_find V1 ligo_data_find
lalapps_tmpltbank lalapps_tmpltbank
H1 lalapps_inspiral L1 lalapps_inspiral V1 lalapps_inspiral
trig_to_tmpltbanktrig_to_tmpltbank trig_to_tmpltbank
lalapps_tmpltbank
H1 lalapps_inspiral L1 lalapps_inspiral V1 lalapps_inspiral
L1 ligo_data_find
lalapps_thinca
H1 ligo_data_find V1 ligo_data_find
lalapps_tmpltbank lalapps_tmpltbank
H1 lalapps_inspiral L1 lalapps_inspiral V1 lalapps_inspiral
lalapps_thinca
trig_to_tmpltbanktrig_to_tmpltbank trig_to_tmpltbank
• “First inspiral” only computes triggers based on SNR time series
• “First inspiral” only computes triggers based on SNR time series
• Triggers not coincident between detectors are excluded by “first thinca”
• “First inspiral” only computes triggers based on SNR time series
• Triggers not coincident between detectors are excluded by “first thinca”
• “Second inspiral” computes the chisq-veto for the surviving triggers
• “First inspiral” only computes triggers based on SNR time series
• Triggers not coincident between detectors are excluded by “first thinca”
• “Second inspiral” computes the chisq-veto for the surviving triggers
• Perform “second thinca” to include only triggers that pass both SNR and chisq
• This saves computational cost as chisq veto is expensive to compute (15 times cost of matched filtering)
• This saves computational cost as chisq veto is expensive to compute (15 times cost of matched filtering)
• But it increases the complexity of the pipeline
• This saves computational cost as chisq veto is expensive to compute (15 times cost of matched filtering)
• But it increases the complexity of the pipeline
• GPUs make chisq veto cheaper: move back to single stage pipeline?
CBC Workflows
Getting it all done...
Getting it all done...
• A LIGO-Virgo “science run” typically contains many weeks (or months!) of data
Getting it all done...
• A LIGO-Virgo “science run” typically contains many weeks (or months!) of data
• The executables process short “blocks” of data
Getting it all done...
• A LIGO-Virgo “science run” typically contains many weeks (or months!) of data
• The executables process short “blocks” of data
• Typically 2048 seconds (about 35 minutes)
Getting it all done...
• A LIGO-Virgo “science run” typically contains many weeks (or months!) of data
• The executables process short “blocks” of data
• Typically 2048 seconds (about 35 minutes)
• The pipeline needs to make sure all the data in analyzed
Getting it all done...
• A LIGO-Virgo “science run” typically contains many weeks (or months!) of data
• The executables process short “blocks” of data
• Typically 2048 seconds (about 35 minutes)
• The pipeline needs to make sure all the data in analyzed
• It does this by writing a workflow
Detector “metadata”
Detector “metadata”
Pipeline configuration file (a.k.a. “ini” file)
Detector “metadata”
Pipeline configuration file (a.k.a. “ini” file)
lalapps_inspiral_hipe
Detector “metadata”
Pipeline configuration file (a.k.a. “ini” file)
lalapps_inspiral_hipe
• A pipeline like hipe is really a script for generating workflows
• A pipeline like hipe is really a script for generating workflows
• It takes detector metadata and configuration parameters as input
• A pipeline like hipe is really a script for generating workflows
• It takes detector metadata and configuration parameters as input
• It writes out a workflow containing a list of jobs that analyze specific data from the given detectors