xmm data reduction with sas simon vaughan (original notes by tim roberts)

XMM data reduction with SAS

Simon Vaughan

(original notes by Tim Roberts)

XMM data reduction 2

Overview

Context & basics Obtaining XMM-Newton data and identifying

useful files Setting up the analysis environment Reducing and cleaning EPIC data Producing images, light curves and spectra

using XMMSELECT


What you are aiming for…Cas A supernova remnant

Silicon Continuum IronThe nearest star in X-raysJupiter’s X-ray aurora

Relativistic Fe line emission from close to a black hole

X-rays from the Cen A radio jet

Hot gas in the Coma cluster of galaxies


Obtaining X-ray data

Astrophysical source of X-rays Intervening absorption (e.g. Galactic neutral gas) X-ray optics (e.g. grazing incidence mirrors) X-ray detectors (e.g. CCDs)


Peculiarities of X-ray data

X-ray detectors are photon counting (as opposed to measuring incoming flux) X-ray data composed of lists of events and their

attributes (time, energy etc…) X-ray data is usually photon limited –

products have few or no counts in many bins Requires specific data analysis techniques and

statistical approaches


X-ray data products

Event list – time-tagged events, with a position (detector/sky space) and energy Detector attributes e.g. CCD pixel pattern for

event – allows rejection of “poor” events Filter event list then project in 1-, 2-D to give

data products Images, energy spectra, light curves

Calibration essential in interpreting data e.g. exposure maps + PSF, response matrices


XMM-Newton

X-ray telescopes

Optical monitor

XMM-Newton instruments:

• EPIC – pn, MOS × 2

• RGS × 2

• OM

X-ray Detectors


Obtaining XMM-Newton data

Write your own proposal Unfortunately, a high risk process where success

may not be rewarded for up to 1.5 years Use the archive

XSA provides access to all datasets beyond the 1-year proprietary period

Accessed via: http://xmm.vilspa.esa.es – java interface best run via netscape


EPIC set-up


Select object of interest…

…then execute query


Select data…

..then move it to your basket (will need to log in)…

… before going to check out


Request multiple then highlight both ODF and PPS…

…then submit request, and follow instructions


Data receipt

E-mailed ftp instructions – follow, unpack tar files

Two types of data ODF – observation data files – telemetry data

reformatted to FITS files PPS – pipeline processing system – top-level

science products including event lists, images, source lists, catalogue cross-correlations TIP: load INDEX.HTM into browser – summary info


PPS files

Good for “first look” at data Specific naming convention:

Where:

PiiiiiijjkkaablllCCCCCCnmmm.zzz

aa – detector: pn, m1, m2, r1, r2, om

CCCCCC – file ID

e.g. P(M)IEVLI – pn (MOS) imaging event list, IMAGE_n – image (n gives band ID)

zzz – file type: ASC, PDF, PNG, HTM, TAR, FTZ


Should I reprocess?

i.e. are the PPS files sufficiently well formatted and calibrated?

New (proprietary) data Should be OK – just gone through most recent

version of pipeline Archival data

Whilst reprocessing of archival datasets does occur, perhaps best to adopt “better safe than sorry” approach and reprocess


A manageable directory structure /data/05/sav2/xmm/

tons180/ mrk766/ source_name/

odf/ processed/ pps/

pn/mos/rgs/ om/

…processed files……processed files……processed files……processed files…


Setting up the user environment To run on XROA system:> sas-setup-new initialises latest

version> setenv SAS_ODF (path_to_ODF_directory)

e.g. /data/05/sav2/xmm/tons180/odf> setenv SAS_CCFPATH /usr/local/ccf> cifbuild >& cifbuild.log

builds ccf.cif (= Calibration Index File)> setenv SAS_CCF (path_to_ccf.cif_file)> odfingest >& odfingest.log

builds ***SUM.SAS file in ODF directory – ODF summary file necessary for reprocessing


Pipeline processing of EPIC data Each pipeline (pn, MOS) needs one command> emchain (or emproc)

> epchain (or epproc)

NB – may need to set ftools up first> lhea-setup-new

Output is calibrated event lists (*EVLI*) Caveats – multiple event lists may be formed if

more than one exposure in dataset, can take some time to run!


What is the result?

One ‘event’ list file [*EVLI*] per exposure An ‘event’ is a detection (usually an X-ray) Each event on a CCD is tagged with:

which detector (camera/CCD) time (CCDs are ‘read out’ periodically) position (X,Y) on detector ‘pattern’ indicating how many pixels are involved energy (~amount of charge deposited in the pixels) quality ‘flag’ indicating known good/bad pixels/events

To get a ‘science product’ you need to filter this list all the unwanted times, patterns, image areas, etc…


Event patterns (grades)


The SAS GUI

Run by> sas &

Simply scroll, double-click on utility e.g. emproc

PRO: transparency

CON: only handles one dataset at a time


What next?

Clean data, and produce science products – XMMSELECT GUI

Can run this from SAS GUI… …but quicker to start from command line> xmmselect table=P0106860101PNU002PIEVLI0000.FIT%events

NB – can take some time to load large datasets esp. pn data containing considerable background flaring


Logical expression used to filter data

Filtering criteria and ranges

Product selection

Process:

• Edit ranges

• Click parameter

• Repeat…

• Select param(s). for product

• Select product


Filtering options

Main choices are: PI – output energy range for product in eV Time – portion(s) of the observation to include Pattern – charge signature on one or more pixels

pn: 0 – 4 MOS: 0 – 12

Flag – quality control for events pn: FLAG==0 MOS: #xmmea_em


Cleaning data: flare exclusion Perhaps largest problem with XMM-Newton

data: space weather Enhances background – dilutes source

signals Energy-dependent – reduces effectiveness of

spectroscopy Can remove by identifying periods when

flaring at worst and excluding them from products


Do I need to flare-filter?

YES! Examine PPS

MOS images – look for enhancement in background in region visible to sky


Filtering

Do for pn: select events above 10 keV, flag & pattern

Select “time” Extract “OGIP

Rate Curve” Set output file, bin

size (10 sec normally OK)


Excluding time intervals

For one large flare, or a small number of flares work out time intervals when you want to accumulate data (zoom in grace) xmmselect: (TIME in (xxx:yyy))&&

For noisy data, create Good Time Intervals (GTI) file

> tabgtigen table=pn_rates.fits:RATE expression=‘RATE<1.5’ gtiset=pn_gti.fits timecolumn=TIME


Total 21.9 ks data


Caveats

Energy spectra of flares can vary Conservative approach is to also check light curve

from 0.3 – 10 keV data using GTI filter MOS & pn may have different start/stop times

If need both instruments on (light curves, some spectra) add extra filter with Tstart, Tstop

OK to use pn GTI file on MOS! If treated separately – do similar filtering for MOS

Very heavy flaring – create new filtered event file before extracting products


Images

Select energy, flag, pattern, time

Indicate X, Y & extract image

Select image tab – set output name, binning size pn: x/ybinsize=80 MOS: binsize=20


Individual source products

Next step: extract spectra, light curves for individual sources

Selection of correct source, background regions very important!

Rules of thumb: Avoid: other sources(!), chip gaps, out-of-time

events, diffuse sources (if possible) Consider: distance from read-out, detector

structure, same quadrant (pn)


Al-K

Si-K


Regions

30-arcsec around NGC 1313 X-1

45-arcsec background region

Save both regions separately!

Read-out direction

Set to background using “info” then “property”


Then extract….

Light curves: Create for source, background separately Use “2-D region” button to automatically transfer region to xmmselect selected expression

Select “time”, then “OGIP rate curve” Choose “withtimeranges=yes” Set “timemin” and “timemax” same for source, background But note: using flare filter means light curves broken up

(i.e. data gaps present) Spectra:

Highlight source, background regions in ds9, then…


Especget

Select PI, time filter Choose “OGIP

spectral products” choice to optimise

region Change “stem” in

“filenames” Run (may be slow!) Spectrum appears in grace window

RMFs, ARFs produced


Resources

This talk (and others): http://www.star.le.ac.uk/~sav2/stats/

XMM-Newton SAS web pages Via http://xmm.vilspa.esa.es Particularly useful documentation:

HEASARC ABC guide SAS user’s guide

Talk to other experienced users! Once you’ve mastered the GUI, try the

command line (more power!)

xmm data reduction with sas simon vaughan (original notes by tim roberts)

Documents

data products images

ftzxmm data reductionshould

energy etcxray data

incoming fluxxray data

event lists

epic pn

source lists

lists of events