As telescopes have grown larger and data rates haveincreased, so have the challenges in providing reliable andaccurate calibration strategies for transforming raw datainto useful science-ready outputs. The Daniel K. Inouye SolarTelescope (DKIST) will be the largest solar telescope in theworld and will use adaptive optics to provide the highestresolution view of the Sun. Its data acquisition rates will bein the hundreds of thousands of frames per day, and it willdeliver an average of 11TB of raw solar data on a daily basis.DKIST data will enable significant and transformativediscoveries that will dramatically increase our understandingof the Sun and its effects on the Sun-Earth environment. Assuch, it is a priority of the DKIST Data Center team at theNational Solar Observatory (NSO) to be able to delivertimely and accurately calibrated data to the astronomicalcommunity for further analysis. The facility will execute avariety of investigator-driven observing programs, which willproduce day-to-day variations in the types of acquired data.In combination with large data rates and limited personnel,this will require some degree of automation to beincorporated into the calibration workflows to facilitate thegeneration of scientifically useful data. The heterogeneity ofthe data and the unpredictable variations in the seeingconditions (on timescales of seconds or minutes) introducecomplexity, which requires a self-adapting, extensiblecalibration pipeline to provide sufficient automation to theprocess. Our knowledge of the instrument performance andtelescope characteristics will grow as the telescope beginsoperations, and continuously through the facility lifetime.The automated calibration pipelines will be capable ofmodification and improvement to incorporate the newinformation about the DKIST system, as well as potentialimprovements providedby the DKIST user community.

Abstract

Calibration development strategies for the Daniel K. Inouye (DKIST) Data CenterFraser Watson, Steven Berukoff, Tony Hays, Kevin Reardon, Daniel Spiess, Scott Wiant

National Solar Observatory, Boulder CO

DKIST instrumentationTheDKISTfacilitywillhouseasuiteof5instrumentstosupportscienceobservations:

Calibrations

CRYO-NIRSP: an infrared spectropolarimeter designed tomeasure magnetic fields in the solar corona.

DL-NIRSP: an integral field unit, fiber fed,spectropolarimeter for studying solar magnetism in avariety of regimes.

VBI: a high resolution, high cadence imager which will oftenbe used to provide context for observations from otherDKIST instruments.

ViSP: an echelle spectrograph for studying polarisationeffects, magnetism via the Zeeman effect, and diagnosticsof the Hanle effect in the solar atmosphere.

VTF: a Fabry-Perot interferometer designed to allow forrapid imaging spectrometry to study dynamics andmagnetism in the photosphere and corona.

Some instrument configurations will be capable ofoperating simultaneously to provide more completeinformationon the target being observed.

Calibrations development for the DKIST is a complexproblem due to the range of scientific investigations thatwill be carried out by the various instruments. To ease thedevelopment process, the Data Center is using a modularapproach to build calibration pipelines for eachinstrument. This allows us to take advantage ofsimilarities between instruments. For example, eachinstrument will use a dark frame module, but only DL-NIRSP will have to demux the integral field unit fibers.

To simplify the creation and implementation of theseprototype modules, the Data Center is using Jupyternotebooks and Python for development. These allow thecombination of executable code, descriptive text, HTMLrendering, equation rendering, and visualisations in onesingle document.

Due to the volumes of data that will be created by theDKIST, some degree of automated calibration processing isrequired to be supported by the Data Center. However, fullautomation is likely not possible, nor desired. DKIST willobserve at spatial scales not yet explored and it is unknownwhat features may be discovered. As such, it is dangerousto implement automated calibration routines that mayremove unknown features and so a capability for manualcalibration is also required, alongside manual oversight ofautomated processes.

An example Jupyter notebook showing text, executable code, and visualisation.

The DKIST facility will differ greatly from anothercurrent generation high resolution and cadenceobservatory, the Solar Dynamics Observatory (SDO).The three SDO instruments take images of the full solardisk at regular intervals and continuously beam themback to Earth for analysis. The major advantage of theSDO is consistency. The instruments are taking imagesof the same object, at the same scale, at very regularintervals. As such, many calibration steps can beremoved. The DKIST has a far smaller field-of-view, andonly a small part of the Sun can be observed at any onetime. There are also large differences in the reductionof data that contain solar flares, data of sunspots, dataof the quiet Sun, or data of tenuous coronal magneticfields. SDO also flies in space, which removes largeaspects of calibration, as there is no atmospherepresent in any SDO observations. The calibration ofDKIST data will have to include algorithms to removeeffects from the Earth's atmosphere, and these aretraditionallydifficult and computationallyexpensive.

Developing the calibration modules for the DKISTinstruments is a process that stems from two startingpoints: a theoretical analysis of all of the possiblecalibration steps for an instrument, and a workingexemplar from a current generation instruments. All ofthe DKIST instruments have some form of currentgeneration analogue that can be used as a source ofoperational knowledge, however they are researchgrade codes that are not necessarily suitable foroperational use in the Data Center.

By taking these two approaches, the Data Center cancombine them into working prototypes of calibrationmodules and can begin to piece them together into aframework that is suitable for simulated or test data.The Data Center is also going to be in detaileddiscussion with the instrument partners as theirintimate knowledge of instrument design andcharacteristics makes them key partners in effectivecalibrationdesign.

Render of the Coudé table showing instrument and beamsplitter configuration.

DKISTinstrument Exemplar instrument

CRYO-NIRSP CYRAatBigBearSolar Observatory

DL-NIRSP SPIESattheUniversityofHawai’i

VBI ROSAattheDunnSolarTelescope

ViSP FIRSattheDunn SolarTelescope

VTF IBISattheDunnSolar Telescope