nienke van der marel - science.nrao.edu
TRANSCRIPT
![Page 1: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/1.jpg)
Atacama Large Millimeter/submillimeter ArrayExpanded Very Large Array
Robert C. Byrd Green Bank TelescopeVery Long Baseline Array
ALMA Data – what to expect after your observations are made
Nienke van der Marel
Authors: Sarah Wood, Nathan Brunetti, Devaky Kunneriath, Sabrina Stierwalt
Atacama Large Millimeter/submillimeter ArrayExpanded Very Large Array
![Page 2: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/2.jpg)
How do you get your data?
• Submit your proposal (deadline April 20th 2017)• Receive an e-mail with the TAC ranking (August 2017)• When your data is taken, the data is reduced and
quality-checked for you• Receive an e-mail saying you can download your
processed data• Once you have your data: one year proprietary time• Alternative: download ALMA data from the ALMA
archive
![Page 3: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/3.jpg)
The Condensed Version
• Data delivered after passing Quality Assurance (QA)• Download data from Archive Query and Request
Handler tools on the ALMA Science Portal• Processed data include:
• Calibration tables and diagnostics• Preliminary images (better products may be
possible with more careful continuum identification & interactive cleaning)
![Page 4: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/4.jpg)
QA2 Data Products Package: the processed dataAfter un-tarring the processed data we have a directory tree:
Project codeSciencegoal
Member OUS: may contain 12-m array, ALMA Compact Array (ACA), or Total Power observation
Group OUS: combination of member OUS’s
Data delivery products
![Page 5: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/5.jpg)
Goals of Quality Assurance (QA) Process
• Ensure reliable final data product– Desired sensitivity (as specified by PI)– Desired resolution (as specified by PI)
• Ensure calibration and QA imaging free from major artifacts
• Warning: Errors in PI-supplied parameters are outside scope of QA process, including:– Incorrect source coordinates– Inadequate frequency specification– Inadequate sensitivity limits
See ALMA Technical Handbook for details.
![Page 6: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/6.jpg)
During Observations – QA0• Monitoring of on-the-fly calibration and system performance• Rapidly-varying parameters (~SB/EB timescales)
– Atmospheric effects– Antenna issues– Front-end issues– Connectivity issues– Back-end issues
• Tolerances for each are explicitly laid out– No fewer than 40 antennas in 12m array– Bandpass calibrator is strong enough
![Page 7: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/7.jpg)
Between Observations – QA1• “Regular array maintenance” timescales• Slowly Varying Parameters (~MOUS timescales)• General array calibration
– Baseline measurements– Delays
• Antenna Calibrations– All-sky pointing– Focus curves– Beam patterns, etc.
• Observatory Calibrator Surveys– Solar-system and quasar flux monitoring
![Page 8: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/8.jpg)
After Observations – QA2• Calibration by pipeline (~70%) or DA/staff.• Final QA checks include
– RMS of complex antenna-based gains– Absolute flux calibration scale– Tsys within acceptable range– Proper phase transfer cadence– Proper bandpass corrections
• Assessment of Imaging Products– Signal-to-noise and angular resolution– No strong artifacts– Performed on the reference source/spectra
• Information about QA review is aggregated for delivery
![Page 9: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/9.jpg)
After Delivery – QA3• Additional QA stage possibly triggered by PI reporting any issues
underlying:– Data, observing procedure, calibration
• Re-evaluation of calibrated data products– Only occurs if QA0 -> QA2 miss something
• Likely results in fix being implemented and products re-ingested into ALMA archive
• Proprietary period extension (within two months of delivery)
• After two months, extension only until fix is delivered
Proprietary Period Extension
Problem Reported Fix Delivered
![Page 10: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/10.jpg)
Monitor Project Status: SnooPI https://almascience.eso.org/observing/snoopi
Be sure to check “All projects” to see PI’ed and co-I’ed
projects of all ranks
![Page 11: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/11.jpg)
Monitor Project Status: SnooPI Listing of PI’ed projects
Check observing status for all of your projects at a glance
![Page 12: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/12.jpg)
Monitor Project Status: SnooPI Listing of PI’ed projects
Approved but SBs not yet preparedSBs prepared but are not yet in the observing queueSBs are in the observing queue but not yet takenSome data has been takenAll the data has been takenCompleted and deliveredProject is timed outRejected at proposal review stageUnknown status
![Page 13: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/13.jpg)
Monitor Project Status: SnooPI Single Project View:
![Page 14: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/14.jpg)
Monitor Project Status: SnooPI Single Project View:
![Page 15: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/15.jpg)
• Subscribe to email notification for updates on changes to project status through your Science Portal user profile– …– Phase2Submitted– Running– Partially Observed– Fully Observed– Pipeline Processing– …
• With or without optional emails, PIs always receive notification when new data are available
Monitor Project Status:Optional emails
![Page 16: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/16.jpg)
Optional emails
![Page 17: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/17.jpg)
Optional emails
ALMA username and password
![Page 18: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/18.jpg)
Optional emails Click Name
Click Profile
![Page 19: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/19.jpg)
Optional emails
Click Checkbox
![Page 20: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/20.jpg)
Data Delivery Email• Sent when an individual MOUS passes QA2 • Data are ingested into the archive and made available at all
Regional Centers• Triggers Start of Proprietary Period
– Usually 12 months• Only Sent to PI• Included Metadata:
– MOUS ID, Scheduling Block (SB) name, project title• Included Instructions:
– Downloading data– Delegating access for registered ALMA users
• Included Descriptions:– Proprietary period
![Page 21: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/21.jpg)
Data Delivery Email• Included Links:
– Archive query for MOUS package– Fully-calibrated MS (North America Only)– CASA download and mailing lists
• Publication Requirements:– ALMA acknowledgement– ARC specific acknowledgement
• Additional Support:– Funded face-to-face reduction visits to your home ARC– Contact info for ARC Helpdesk
![Page 22: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/22.jpg)
Find data in archive:Archive Query
![Page 23: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/23.jpg)
Archive Query
![Page 24: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/24.jpg)
Archive Query: more columns
![Page 25: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/25.jpg)
• All data downloaded as tar files• Large data sets may be broken into several pieces
– Name is [project_code]_[OUS_ID]_m_of_n.tar– Raw data packaged as one tar file per execution block (EB)
• name is [project_code]_[EB_ID].asdm.sdm.tar• For Cycle 0-4 projects, cannot directly download individual
data products but potentially coming in Cycle 5…– FITS images– Diagnostic plots, etc.
Downloading the data:Request Handler
![Page 26: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/26.jpg)
Request Handler
![Page 27: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/27.jpg)
Request Handler
![Page 28: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/28.jpg)
Request Handler: Download options
![Page 29: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/29.jpg)
Request Handler: script
#!/bin/bash #Please use the current script to download the whole content of request 223732763
echo "Please provide a password" read -s PASSWORD
wget --auth-no-challenge --no-check-certificate --http-user="nbrunett" --http-password=$PASSWORD https://almascience.nrao.edu/dataPortal/api/requests/nbrunett/223732763/ALMA/2012.1.00090.S_uid___A002_X5eed86_X2b_001_of_001.tar/2012.1.00090.S_uid___A002_X5eed86_X2b_001_of_001.tar . . .
![Page 30: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/30.jpg)
Request Handler: Web start
![Page 31: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/31.jpg)
Request Handler: Download Manager
Using JAVA browser-based GUI
![Page 32: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/32.jpg)
Resources After Delivery• HelpDesk• Face to Face visits in Charlottesville: https://science.nrao.edu/
facilities/alma/visitors-shortterm• For Hawaii also an option to go to Victoria, but small facility
![Page 33: Nienke van der Marel - science.nrao.edu](https://reader033.vdocument.in/reader033/viewer/2022041600/6252c946d82eab662371a0b9/html5/thumbnails/33.jpg)
The Atacama Large Millimeter/submillimeter Array (ALMA), an international astronomy facility, is a partnership of Europe, North America and East Asia in cooperation with the Republic of Chile. ALMA is funded in Europe by the European Organization for Astronomical Research in the Southern Hemisphere (ESO), in North America by the U.S. National Science Foundation (NSF) in cooperation with the National Research Council of Canada (NRC) and the National Science Council of Taiwan (NSC), and in East Asia by the National Institutes of Natural Sciences (NINS) of Japan in cooperation with the Academia Sinica (AS) in Taiwan. ALMA construction and operations are led on behalf of Europe by ESO, on behalf of North America by the National Radio Astronomy Observatory (NRAO), which is managed by Associated Universities, Inc. (AUI), and on behalf of East Asia by the National Astronomical Observatory of Japan (NAOJ). The Joint ALMA Observatory (JAO) provides the unified leadership and management of the construction and operation of ALMA.
For more info: https://almascience.nrao.edu/