2.2 Types of STIS Files

The naming convention for STIS files is rootname_xxx.fits, where xxx is a three-character file suffix. The suffix identifies the type of data within the file. Table 2.1 lists the file suffixes for both uncalibrated and calibrated data files. Depending on the type of observation you have obtained, and therefore on the path it has taken through the calibration pipeline, you will find an appropriate subset of these files in your particular dataset. Table 2.2 shows the datasets that are produced by the calstis pipeline for different types of datasets.

Table 2.1: Data File Naming Conventions

¹ Raw data from non-automated GO wavecals, biases, darks, and flats, as well as from ACQs and ACQ/PEAKs, also have the _raw suffix.

Table 2.2: STIS Data File by Observation Type

¹ Combined: summed if MAMA data, summed with cosmic ray rejection if CCD data.
² Note that, while the pipeline does not produce a summed sx1 file for MAMA multiple imset spectroscopic datasets, you can easily do so by running x1d on the sfl file.
³ A single raw image is generated from the tag file for TIME-TAG data. For other options, see Section 5.6.

2.2.1 Trailer Files

Each task in the calstis package creates messages during processing which describe the progress of the calibration. These messages are quite relevant to understanding how the data was calibrated, and in some of the cases, to determining the accuracy of the products.

When the data is processed in the archive pipeline, the output messages from the conversion of the telemetry data and the different calibration steps done by the calstis software are stored in a text FITS file known as the trailer file, with extension _trl. Each time the archive processes data before retrieval, the old trailer file is erased and a new one created using the results of the most recent processing performed. This is not the case when calstis is run in a user’s home environment. Calstis redirects the output of its many steps to the STDOUT, so when run on a personal machine, it will not override this _trl file but rather will direct the output to STDOUT.

The data on the telemetry conversion appears first in this table. In this section of the _trl file, there is also information relevant to the selection of the best reference files and the population of some of the header keywords. The second part of this file is the information on the calibration steps performed by the calstis pipeline, appearing in the order in which each step was performed. In this last section of the _trl file, the calstis steps are indicated by their module name (see Table 3.1 of this document).

The calstis messages provide information on the input and output files for each step, the corrections performed, information regarding the reference files used, and in the case of spectroscopic data, messages about the location of the extracted spectrum or shift correction applied to the data. Calstis also gives warnings in the event of failures of the software to locate the spectrum or when the appropriate correction to the data could not be applied. For more detailed information on the calibration steps and structure of calstis, please refer to Chapter 3.

2.2.2 Understanding Associations

An association is created when repeated exposures are obtained through CR-SPLITs or REPEATOBS, and when wavecal exposures are linked to science exposures. The repeated exposures in an association will appear in a single FITS file. You can recognize a file as part of an association because there will be a zero in the last position of the rootname (e.g., o3tt01010_raw.fits). The rootnames of the individual exposures in an association are contained in the association file, which has suffix _asn (e.g., o3tt01010_asn.fits). An association file holds a single binary table extension, which can be displayed with astropy.table. The information within an association table shows how the associated exposures are related. Table 2.3 illustrates the contents of the association table for a CRSPLIT=2 observation, with an associated wavecal.

Table 2.3: Contents of Association Table.
To display the association table for o3tt01010_asn.fits:

>>> from astropy.table import Table

>>> data_table = Table.read('o3tt01010_asn.fits', hdu='asn')
>>> print (data_table)

MEMNAME    MEMTYPE       MEMPRSNT
---------  -----------   --------
O3TT01AVQ   CRSPLIT       True
O3TT01AWQ   CRSPLIT       True
O3TT01010   PRODUCT       True

The association table above tells the user that the product, or dataset, will have the rootname o3tt01010, that there will be two science exposures contained in the o3tt01010_raw.fits file that are CR-SPLITS, and that a o3tt01010_wav.fits file should exist containing the contemporaneously obtained automatic wavecal. The o3tt01010_raw.fits file will contain six image extensions, one triplet of {SCI, ERR, DQ} for each exposure (see Section 2.3.1). The pipeline will calibrate these data as a unit, producing, in the case of CCD data, a single cosmic ray rejected image (rootname_crj.fits), its data quality and error images, and rectified spectra. Similarly, for REPEATOBS observations, in which many identical exposures are taken to obtain a time series, all the science data will be stored in sequential triplet extensions of a single FITS file (rootname_flt.fits). These will be processed through the calstis pipeline as a unit, with each image extension individually calibrated. The set of images will also be combined to produce a total time-integrated calibrated image.

Originally, only automatically inserted wavecals were associated and used for wavelength calibration in the OTFR pipeline. Although GOs could specify additional lamp exposures with the LINE, HITM1, or HITM2 lamps that could be used to supplement or replace the automatic wavecals, these were not used in the pipeline.

For a number of such spectroscopic datasets without auto-wavecals that were taken between STIS launch and the Side-2 failure in 2004, the associations were subsequently redefined to allow GO-specified wavecals to be used in place of the missing auto-wavecals. In these new associations, the GO-specified wavecal is treated in the same way as a standard auto-wavecal, although some parameters used for the exposure (e.g., aperture, exposure time, lamp, or lamp current) may differ from the default values that would have been used by the automatic wavecal.

It was also decided that it would be desirable to “associate” STIS IR fringe flats taken using the tungsten calibration lamp with the G750M or G750L science exposures for which they were taken. Instead of creating a formal association between such datasets, the name of the recommended fringe flat is placed in the FRNGFLAT header keyword in the extension header of the science data, and all tungsten lamp images taken contiguously with the science data (i.e., no MSM motion between exposures) are automatically delivered to the user whenever data are requested from the archive.

When suitable fringe flats are available, the recommended flat is chosen based on the aperture used for the fringe flat with the order of preference being 52X0.1, 52X0.05, 52X0.2, 52X0.5, 52X2, 0.3X0.09, and then any other. Should there be multiple suitable fringe flats taken using the same aperture, the one taken closest in time to the science exposure is preferred.
Chapter 3 of this document gives more information about the pipeline processing.