Calibration

Calibration files

Auxiliary data used in DRP algorithms are called calibration data. This includes both on-sky data (that is not of the astronomical target itself), daytime calibration frames, and other sub-component metadata. Metadata is non-image information that will typically come from the header of raw FITS files, or from IRIS, and/or the adaptive optics system via the observatory telemetry service. The NFIRAOS Science Calibration Unit (NSCU) will include a calibration system that will facilitate the taking of daytime calibration frames, such as arc lamp spectra, white light flat field images, and pinhole grids for measuring distortion. The following table summarizes the required calibration files necessary for the Data Reduction Software.

Notes about the table: Note: * = SPEC only, PTG = pointing, D-Map = Distortion Map, Env = Environmental, DTC = Daytime calibration, NTC = Nightime calibration.

Calibration frames

Name	Reference Type	Source	Algorithms
Atm. Dispersion Residual	Metadata	IRIS ADC	Atmospheric Correction
Arc lamp spectra*	CAL (2D)	IRIS DTC (NSCU)	Wavelength solution
Bad pixel map	CAL (2D)	IRIS DTC	Correction of detector artifacts
Dark Frame	CAL (2D)	IRIS DTC and NTC	Dark subtraction
Env metadata	Metadata	ESW, FITS header	All
Fiber image	CAL (2D, 3D)	IRIS DTC (NSCU)	PSF Calibration
Flux calibration star	CAL (2D, 3D)	IRIS On-sky	Extract Star, Remove Absorption Lines
Instrument config	Metadata	ESW, FITS header	All
Lenslet scan*	Rect Matrix CAL (2D)	IRIS DTC (NSCU)	Spectral Extraction
NFIRAOS config	Metadata	ESW, FITS header	All
Pinhole Grid (D-Map)	CAL (2D)	IRIS DTC (NSCU)	Field distortion correction
PSF metadata	Metadata	ESW, FITS header	PSF calibration
PSF star	CAL (2D, 3D)	IRIS on-sky	PSF calibration
Sky frame	CAL (2D, 3D)	IRIS on-sky	Sky-subtraction
Telescope config PTG	Metadata	ESW, FITS header	All

Real time Calibration frames

Name	Reference Type	Source	Algorithms
Atm. Dispersion Residual	Metadata	IRIS ADC	Atmospheric Correction
Arc lamp spectra*	CAL (2D)	IRIS DTC (NSCU)	Wavelength solution
Bad pixel map	CAL (2D)	IRIS DTC	Correction of detector artifacts
Dark Frame	CAL (2D)	IRIS DTC and NTC	Dark subtraction
Env metadata	Metadata	ESW, FITS header	All
Instrument config	Metadata	ESW, FITS header	All
NFIRAOS config	Metadata	ESW, FITS header	All
Sky frame	CAL (2D, 3D)	IRIS on-sky	Sky-subtraction
Telescope config PTG	Metadata	ESW, FITS header	All

Access calibration files via the Calibration Reference Data System (CRDS)

The Calibration Reference Data System (CRDS) is a set of tools developed by Space Telescope to organize and retrieve calibration reference files, e.g. flat frames, dark frames, for JWST and HST. When stpipe is executing a pipeline, it can automatically connect to the JWST CRDS server and get the right flat based on the metadata in the header of the data FITS files. The logic necessary to choose the right file is encoded in text files. Those configuration files and the actual calibration FITS files are also cached locally so that the CRDS client library works even without any connection to a central server.

We have created a CRDS cache folder in the Github repository https://github.com/oirlab/tmt-crds-cache, this includes in the mappings/tmt folder the metadata for IRIS and the rules to choose the right flat-field frame, for now there is only a dummy rule but this can be easily customize querying the metadata in the input file.

Currently we do not have any CRDS server running, but the users can download the CRDS cache locally and use it anyway, see the Getting started documentation.

Also, the CRDS client library needs to have minimal knowledge about metadata for TMT, therefore we maintain a fork of that library which simply adds a submodule dedicated to IRIS, https://github.com/oirlab/tmt-crds, it is quite easy to upgrade this to newer releases of CRDS by Space Telescope.

If TMT decided to use CRDS as their Data Management System, it would leverage the extensive set of tools and documentation available and would not require modifications to stpipe; otherwise, we will implement support for the DMS API into (our own fork of) stpipe.