july 2006instrument scientist retreat, santiago quality control: closing the loop reinhard...

July 2006 Instrument Scientist Retreat, Santiago

Quality Control: closing the loopReinhard Hanuschik, DFO/QC

1. Principles

2. Processes

3. Critical components

4. Conclusions


• purpose of calibration:– calibrate the science observations

= remove (atm+tel)*(ins+det)– know the status of tel+ins+det

maintenance, intervention– calibrate the instrument

• purpose of QC:– ensure that calibs can do that – to a known and predictable accuracy

1. Principles


• QC group:– 7+1 scientists– data processing

and quality control– roughly 400 GB

raw data per month, 11,500 processing jobs

– 80 SM packages– 9 VLT, 2 VLTI; 2

survey telescopes


• aspects of QC:– formal compliance of file format:

PSO, DMD/DBCM, DMD/SAO– compliance with user constraints:

PSO (OB grading)– check raw files against reference files:

PSO– check pipeline products, do optimized

association, optimized processing: QC group

– compare to similar data (trending):QC group


• standard QC

Paranal SciOps

QC Garching

raw proc (on-line pipe)

raw proc (off-line pipe)

certification

trending

10 days

fits file transfer


• standard QC

• shared QC: close the loop

• feedback to enable corrections

Paranal SciOps

QC Garching

raw proc (on-line pipe)

raw proc (off-line pipe)

certification

trending

10 days

fits file transfer

HC process

HC monitor

< 1 day

op

s l

og

ftp


2. Processes

Components:

• On-site QC

• Health Check monitor

• QC processing and certification

• QC trending

10 days!


• On-site QC– day/night astronomers:

• completeness check • visual quality checks: exposure level, patterns • inspection of pipeline products: are there any?

do look reasonable?


• Reference frames– www.eso.org/qc/ALL/ref_frames/ref_frames.html

http://www.eso.org/qc/ALL/ref_frames/ref_frames.html


• Health Check monitor– www.eso.org/qc/ALL/daily_qc1.html– day astronomer + QC scientist:

• how do new calibs compare to previous ones

• do new calibs show up at all

• outliers, alerts? trigger actions

• currently being upgraded to better organize information, closeups etc.

http://www.eso.org/qc/ALL/daily_qc1.html


• upgraded: Giraffe, VIMOS


• QC process, certification– QC scientists in Garching:

• process all calibration data, SM science data• associate best-possible calibs to new data

– best: closest in time, certified, dynamically updated– correct dependencies (following the cascade)

• process with best-possible pipeline setup– optimized, customized parameters– maybe patched/improved beta versions

• measure and assess product quality– customized QC reports, experience– trending analysis

• certify products– released for archiving and delivery– released for usage in processing other data


• Trending– done and maintained by QC group

• set of current and history trending plots• customized for each product type or instrument component• access to ASCII data

– www.eso.org/qc

http://www.eso.org/qc


– useful for:• maintenance planning (“by how much do the

zeropoints degrade over a year by dust accumulation”)

• complete instrument history (“how often in the past has grating XZ shown this kind of shift”)

• InsScientists, PIs, archive users (“how does my current dispersion rms compare to earlier observations in P74”)


3. Critical components, challenges

• calibration plan– definition of calibs and frequency must be

complete– must include Health Check calibs– execution must be complete:

• daytime (calOBbuilder)• twilight/nighttime (…)


• on-site QC, Health Check monitor– define within IOT:

• regular set of HC calibs (regular spacing, daily or so)• set of key QC1 params to be included (cannot be all,

must focus on most critical ones):– DET, FILT, GRISM/GRAT, LAMPs, system throughput etc.

– don’t expect all trending info on the HC monitor• simply too much …• find under www.eso.org/qc/:• full history, all plots, all tutorial info, all text files

http://www.eso.org/qc/


• pipelines – QC1 parameters need to be pipeline-

produced and included in ops log files, to show up on the Health Check monitor

• Health Check monitor, trending:– maintenance continuous effort– complexity is a challenge!


• Data transfer– delay by 8-14 days– large impact on QC task:

• large data volume affected by any problem discovered in Garching

• delivery bursts processing delays may happen

dvdMonitor

http://www.eso.org/~qc/monitor/dvdMonitor_GIRAFFE.html

http://www.eso.org/~qc/monitor/dvdMonitor_GIRAFFE.html


• complexity– supporting 9 VLT + 2 VLTI plus 2 VST/VISTA– publish and maintain ~600 web pages– we are largely on our own with maintenance/

development of most components


• IOTs– extremely important for setting up and

maintaining the QC process– shared QC requires each side to know what

the other side is doing and why

– we visit you come and visit us!


• shared QC: – in excellent shape– but needs continuous efforts– can still be improved …

4. Conclusions

• key issues: – data delivery pattern– daytime QC/HC monitor– operational stability

july 2006instrument scientist retreat, santiago quality control: closing the loop reinhard...

Documents