6dfgs data quality: comparison of pipeline and iraf redshifts
DESCRIPTION
6dFGS data quality: comparison of pipeline and IRAF redshifts. Lesa Moore Macquarie University AAO 6dF Workshop 2005. Outline. Spectral reduction and S/N Wavelength calibration Cross-correlation redshift agreement Quality measures – S/N, Q, r Repeatability Final uncertainties - PowerPoint PPT PresentationTRANSCRIPT
6dFGS data quality:6dFGS data quality:comparison of pipeline and comparison of pipeline and
IRAF redshiftsIRAF redshifts
Lesa MooreLesa Moore
Macquarie UniversityMacquarie University
AAO 6dF Workshop 2005AAO 6dF Workshop 2005
OutlineOutline Spectral reduction and S/NSpectral reduction and S/N Wavelength calibrationWavelength calibration Cross-correlation redshift agreementCross-correlation redshift agreement Quality measures – S/N, Q, rQuality measures – S/N, Q, r RepeatabilityRepeatability Final uncertaintiesFinal uncertainties Based on comparisons between Based on comparisons between
IRAF … and … 6dFDR/RUNZ processingIRAF … and … 6dFDR/RUNZ processing
DataData
Three fields studied (A, B and C) with Three fields studied (A, B and C) with repeat observations of the B and C repeat observations of the B and C fields:fields: A: Mar 16 2002, reflection gratingsA: Mar 16 2002, reflection gratings B1: Sep 9 2002, reflection gratingsB1: Sep 9 2002, reflection gratings B2: Sep 29 2003, VPH gratingsB2: Sep 29 2003, VPH gratings C1: Sep 29 2002, VPH gratingsC1: Sep 29 2002, VPH gratings C2: Sep 18 2003, VPH gratingsC2: Sep 18 2003, VPH gratings
MethodMethod
6dFDR on separate V and R spectra6dFDR on separate V and R spectra 6dFDR line lists6dFDR line lists
RUNZ on spliced spectraRUNZ on spliced spectra
IRAF dofibers on separate V and R spectraIRAF dofibers on separate V and R spectra My own line listsMy own line lists
IRAF xcsao on spliced spectraIRAF xcsao on spliced spectra
Batch mode processing, no heliocentric Batch mode processing, no heliocentric correction in either casecorrection in either case
Spectral ReductionSpectral Reduction
6dFDR reduction6dFDR reduction Sometimes requires FIT rather than Sometimes requires FIT rather than
TRAM extraction (slower)TRAM extraction (slower) Sometimes requires FLUX WEIGHTING Sometimes requires FLUX WEIGHTING
option turned offoption turned off
Signal to NoiseSignal to Noise
FieldField GratingGrating S/NS/N RemarksRemarks
B1 RefB1 Ref 316R ref316R ref 5.1245.124 Average Average for 68 for 68 galaxiesgalaxiesB2 VPHB2 VPH 425R VPH425R VPH 12.11912.119
B1 RefB1 Ref 600V ref600V ref 7.3137.313 Average Average for 69 for 69 galaxiesgalaxiesB2 VPHB2 VPH 580V VPH580V VPH 8.7958.795
SoftwareSoftware Field A, R dataField A, R data Field A, V dataField A, V data
6dFDR/splot6dFDR/splot 3.063.06 2.342.34
Dofibers/splotDofibers/splot 3.173.17 2.222.22
6dFDR/RUNZ6dFDR/RUNZ 10.3910.39 9.509.50
Dofibers/RUNZDofibers/RUNZ 10.2410.24 8.898.89
Cross-processed
Pipeline and IRAF reductions about equal
VPH data superior to reflection grating data (figures from 6dFDR/RUNZ)
Line lists andLine lists and calibration calibration Blue (V) data original line list had 14 Blue (V) data original line list had 14
lineslines 6dFDR typically locates 126dFDR typically locates 12 Software throws away 2 worst-fitting Software throws away 2 worst-fitting
(leaves 10)(leaves 10) For red (R) data For red (R) data
Line list omitted strong Ne line at 7032.41 Line list omitted strong Ne line at 7032.41 ÅÅ Arc spectra deficient of lines redward of Arc spectra deficient of lines redward of
7500 7500 Å (does not affect cross-correlation)Å (does not affect cross-correlation) Found 2Found 2ndnd order line from Hg at ~8092 Å order line from Hg at ~8092 Å
Calibration Calibration TestTest
Field C1 VPH reduced Field C1 VPH reduced without sky without sky subtractionsubtraction
Sky lines measuredSky lines measured 6dFDR results (6dFDR results (±0.4 ±0.4
Å) Å) equal to or equal to or superior to IRAFsuperior to IRAF
V 5577.34 Å Oxygen Line - 6dFDR
5576.55576.75576.95577.15577.35577.55577.75577.95578.15578.35578.5
0 50 100 150
Fibre
Wav
elen
gth
(Å)
R 5577.34 Å Oxygen Line - 6dFDR
5576.45576.65576.85577.05577.25577.45577.65577.85578.05578.25578.4
0 50 100 150
Fibre
Wav
elen
gth
(Å)
R 6300.30 Å Oxygen Line - 6dFDR
6299.36299.56299.76299.96300.16300.36300.56300.76300.96301.16301.3
0 50 100 150
Fibre
Wav
elen
gth
(Å)
R 6363.78 Å Oxygen Line - 6dFDR
6362.86363.06363.26363.46363.66363.86364.06364.26364.46364.66364.8
0 50 100 150
Fibre
Wav
elen
gth
(Å)
R 7243 Å OH Line - 6dFDR
7242.37242.57242.77242.97243.17243.37243.57243.77243.97244.17244.3
0 50 100 150
Fibre
Wav
elen
gth
(Å)
Cross-correlationCross-correlation
An earlier version of RUNZ at Epping An earlier version of RUNZ at Epping was applying the heliocentric was applying the heliocentric correction incorrectlycorrection incorrectly
RUNZ confused by noise in low S/N RUNZ confused by noise in low S/N spectraspectra
Spliced spectra much more reliable Spliced spectra much more reliable than separate R and V than separate R and V
Redshift AgreementRedshift Agreement
Field A spliced data
108 galaxies
y = 1.0016x + 1E-0592 galaxies
-0.1
0
0.1
0.2
0.3
0.4
-0.1 0.0 0.1 0.2 0.3
Field B1 REF spliced data
80 galaxies
y = 0.9993x + 0.0002 63 galaxies
-0.1
0
0.1
0.2
0.3
0.4
-0.1 0.0 0.1 0.2 0.3
Field C1 VPH spliced data
106 galaxies
y = 0.9989x + 0.000183 galaxies
-0.1
0
0.1
0.2
0.3
0.4
-0.1 0.0 0.1 0.2 0.3
IRAF redshifts
Pip
eli
ne
re
ds
hif
ts
Agreement ifAgreement if z| ≤ 0.0005z| ≤ 0.0005 cz| ≤ 150 km/scz| ≤ 150 km/s
Overall agreement 81% for Overall agreement 81% for 294 galaxies in 3 fields294 galaxies in 3 fields
Could still both be wrongCould still both be wrong need to check by eyeneed to check by eye
Quality MeasuresQuality Measures
Compared this redshift agreement Compared this redshift agreement with:with: S/NS/N RUNZ Q-ranking RUNZ Q-ranking Cross-correlation r-valuesCross-correlation r-values
as obtained from pipeline processingas obtained from pipeline processing
S/NS/N
No strong No strong correspondenccorrespondence between e between redshift redshift agreement and agreement and S/N in separate S/N in separate R and V R and V spectraspectra
Field C1 VPH R data
106 galaxies
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0
Field C1 VPH V data
103 galaxies
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0IRA
F z
- P
ipe
lin
e z
RUNZ S/N
Field B1 REF R data
78 galaxies
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0
Field B1 REF V data
81 galaxies
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0
Field A R data
106 galaxies
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0
Field A V data
107 galaxies
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0
Q-rankingsQ-rankings
Large scatter Large scatter even with high even with high QQ
Q-rankingQ-ranking 00 11 22 33 44 55 TotalTotalss
DisagreesDisagrees 66 1212 55 2121 88 44 5656
AgreesAgrees 88 33 11 3232 7676 118118 238238} 294
Delta z vs RUNZ Q-ranking
0
30000
60000
90000
0 1 2 3 4 5 6
RUNZ Q-ranking
|IR
AF
cz
- R
UN
Z c
z| (
km/s
)
Q-rankingsQ-rankings Around half the “disagrees” have Around half the “disagrees” have
Q of 3, 4, or 5 Q of 3, 4, or 5
Q-rankingQ-ranking 00 11 22 33 44 55 TotalTotalss
DisagreesDisagrees 66 1212 55 2121 88 44 5656
AgreesAgrees 88 33 11 3232 7676 118118 238238} 294
Q-rankingsQ-rankings Around half the “disagrees” have Around half the “disagrees” have
Q of 3, 4, or 5 Q of 3, 4, or 5 Q=3 meant to imply 75% confidence Q=3 meant to imply 75% confidence
(only 60% agree)(only 60% agree)Q-rankingQ-ranking 00 11 22 33 44 55 TotalTotal
ss
DisagreesDisagrees 66 1212 55 2121 88 44 5656
AgreesAgrees 88 33 11 3232 7676 118118 238238} 294
Cross-correlation Cross-correlation r-value r-value
R-value is a much moreR-value is a much more useful indicator of redshift useful indicator of redshift reliability than Q-rankingreliability than Q-ranking
84% of disagrees have r<684% of disagrees have r<6 27 of 33 disagrees with 27 of 33 disagrees with
Q=3,4,5 have r<6Q=3,4,5 have r<6 72% of agrees have r>672% of agrees have r>6
Field C1 VPH spliced data
106 galaxies
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0IRA
F z
- P
ipe
lin
e z
RUNZ r-value
Field B1 REF spliced data
80 galaxies
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0
Field A spliced data
108 galaxies
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0
SIMBAD-RUNZ difference SIMBAD-RUNZ difference vs. Q-rankingvs. Q-ranking
Q – rankings of 3, 4, 5 show large spread of Q – rankings of 3, 4, 5 show large spread of error values (4, 14, 29 data points respectively)error values (4, 14, 29 data points respectively) Have applied heliocentric correction this timeHave applied heliocentric correction this time Note bias towards high-q resultsNote bias towards high-q results
SIMBAD - RUNZ Redshifts50 Galaxies
-100000
-50000
0
50000
100000
150000
200000
250000
300000
0 1 2 3 4 5 6
RUNZ Q
SIM
BA
D -
RU
NZ
cz
(km
/s)
SIMBAD - RUNZ Redshifts
-600-500-400-300-200-100
0100200300400500600
0 1 2 3 4 5 6
RUNZ Q
SIM
BA
D -
RU
NZ
cz
(km
/s)
SIMBAD-RUNZ difference SIMBAD-RUNZ difference vs. R-valuevs. R-value
Differences scale inversely with r-valuesDifferences scale inversely with r-values St dev: St dev: z z = 0.00012, = 0.00012, km/s km/s = 52 km/s = 52 km/s
(based on 34 galaxies whose redshift agreement (based on 34 galaxies whose redshift agreement meets criterion of meets criterion of cz| ≤ 150 km/s)cz| ≤ 150 km/s)
SIMBAD - RUNZ Redshifts50 Galaxies
-100000
-50000
0
50000
100000
150000
200000
250000
300000
0 6 12 18 24 30 36
R-value
SIM
BA
D -
RU
NZ
cz
(km
/s)
SIMBAD - RUNZ Redshifts
-600-500-400-300-200-100
0100200300400500600
0 6 12 18 24 30 36
R-value
SIM
BA
D -
RU
NZ
cz
(km
/s)
RepeatabilitRepeatabilityy
End columns End columns are large are large discrepanciesdiscrepancies
All r>6 results All r>6 results lie within lie within z| z| ≤ 0.001≤ 0.001
Overall st.dev. Overall st.dev. zz = 0.00033 = 0.00033
km/skm/s = 98 = 98 km/skm/s
Redshift Agreement - Fields B1 REF & B2 VPH
02468
101214
-0.0
5
-0.0
05
-0.0
01
-0.0
005
-0.0
004
-0.0
003
-0.0
002
-0.0
001 0
0.00
01
0.00
02
0.00
03
0.00
04
0.00
05
0.00
1
0.00
5
0.05
>0.0
5
Redshift difference (z)
Nu
mb
er o
f g
alax
ies
68 common galaxies 32 galaxies with r > 6
Redshift Agreement - Fields C1 VPH and C2 VPH
02468
101214161820
-0.0
5
-0.0
05
-0.0
01
-0.0
00
5
-0.0
00
4
-0.0
00
3
-0.0
00
2
-0.0
00
1 0
0.0
00
1
0.0
00
2
0.0
00
3
0.0
00
4
0.0
00
5
0.0
01
0.0
05
0.0
5
>0
.05
Redshift difference (z)
Nu
mb
er
of
ga
lax
ies
80 common galaxies 49 galaxies with r > 6
Final uncertaintiesFinal uncertainties
NotesNotes 1. Based on 0.4 1. Based on 0.4 ÅÅ at 4000 at 4000 ÅÅ 2. Mean of “verr” from RUNZ (278 galaxies in total, 190 with r>6, possibly over-stated)2. Mean of “verr” from RUNZ (278 galaxies in total, 190 with r>6, possibly over-stated) 3. 1/√2 * st. dev.(3. 1/√2 * st. dev.(z) of repeat 6dF observations (125 galaxies in total, 81 with r>6)z) of repeat 6dF observations (125 galaxies in total, 81 with r>6) 4. Added in quadrature4. Added in quadrature kms/s = z * 300,000 assumed in all caseskms/s = z * 300,000 assumed in all cases
SourceSource General General UncertaintyUncertainty
Uncert. for r>6 Uncert. for r>6 redshiftsredshifts
±±km/skm/s ±±zz ±±km/skm/s ±±zz
calibrationcalibration11 3030 0.00010.0001 3030 0.00010.0001
Cross-correlationCross-correlation22 6363 0.000210.00021 4848 0.000160.00016
RepeatabilityRepeatability33 6969 0.000230.00023 3939 0.000130.00013
Total uncertaintyTotal uncertainty44 9898 0.000330.00033 6969 0.000230.00023
The “6dF Galaxy The “6dF Galaxy Survey” Paper Survey” Paper (2004)(2004)
82 km/s for Q = 382 km/s for Q = 3 46 km/s for Q = 446 km/s for Q = 4
Final uncertaintiesFinal uncertainties
NotesNotes 1. Based on 0.4 1. Based on 0.4 ÅÅ at 4000 at 4000 ÅÅ 2. Mean of “verr” from RUNZ (possibly over-stated)2. Mean of “verr” from RUNZ (possibly over-stated) 3. 1/√2 * st. dev.(3. 1/√2 * st. dev.(z)z) 4. Added in quadrature, final results rounded to one significant figure4. Added in quadrature, final results rounded to one significant figure kms/s = z * 300,000 assumed in all caseskms/s = z * 300,000 assumed in all cases
SourceSource General General UncertaintyUncertainty
Uncert. for r>6 Uncert. for r>6 redshiftsredshifts
±±km/skm/s ±±zz ±±km/skm/s ±±zz
calibrationcalibration11 3030 0.00010.0001 3030 0.00010.0001
Cross-correlationCross-correlation22 6363 0.000210.00021 4848 0.000160.00016
RepeatabilityRepeatability33 6969 0.000230.00023 3939 0.000130.00013
Total uncertaintyTotal uncertainty44 100100 0.00030.0003 7070 0.00020.0002
The “6dF Galaxy The “6dF Galaxy Survey” Paper Survey” Paper (2004)(2004)
82 km/s for Q = 382 km/s for Q = 3 46 km/s for Q = 446 km/s for Q = 4
SummarySummary
S/N superior with VPH gratingsS/N superior with VPH gratings RUNZ q-ranking RUNZ q-ranking notnot reliable indicator of z quality reliable indicator of z quality Redshift agreement (IRAF-RUNZ) scales strongly Redshift agreement (IRAF-RUNZ) scales strongly
with cross-correlation r-valuewith cross-correlation r-value Small uncertainty in Small uncertainty in calibration test calibration test Larger uncertainties in cross-correlation and Larger uncertainties in cross-correlation and
repeatability testsrepeatability tests Total uncertainties:Total uncertainties:
100 km/s general uncertainty100 km/s general uncertainty 70 km/s uncertainty for r>6 redshifts70 km/s uncertainty for r>6 redshifts
ThanksThanks
Macquarie UniversityMacquarie University Anglo-Australian ObservatoryAnglo-Australian Observatory Wide-Field Astronomy Unit, EdinburghWide-Field Astronomy Unit, Edinburgh
Supervisors: Quentin Parker (MU/AAO), Will Saunders Supervisors: Quentin Parker (MU/AAO), Will Saunders (AAO)(AAO)
6df Galaxy Survey Team (37 members)6df Galaxy Survey Team (37 members)
References References The 6dF Galaxy Survey: samples, observational techniques and the The 6dF Galaxy Survey: samples, observational techniques and the
first data releasefirst data release, MNRAS, , MNRAS, 355355, 747-763 (2004), 747-763 (2004) This research has made use of the SIMBAD database, operated at This research has made use of the SIMBAD database, operated at
CDS, Strasbourg, FranceCDS, Strasbourg, France