15.1 Pipeline Processing Overview

Here, we briefly summarize the basic reductions and calibrations that are performed in the STScI STIS pipeline, and summarize the effects that particular Phase II proposal parameter choices have on calibration. The material in this chapter is intended to provide only enough background to develop robust observing proposals. A series of STIS Instrument Science Reports (see a listing in Section 15.3) and the STIS Data Handbook provide the more detailed information needed for analyzing your data.

Data taken with STIS are initially received by the Space Telescope Data Capture Facility at NASA's Goddard Space Flight Center before being sent to STScI.  Once at STScI, the data are reformatted and repackaged into raw (uncalibrated) Flexible Image Transport System (FITS) files by the generic conversion process, which includes populating header keywords with information extracted from the telemetry stream.  All STIS data products are FITS files.  Most of them are stored as images, with separate FITS extensions holding the science, error, and data quality arrays in a single file.  Uncalibrated FITS files are the input to the calstis software package, which produces calibrated FITS files for subsequent analysis.  Consult the STIS Data Handbook for more details concerning the structure and naming conventions for the files produced by calstis.

Calstis performs the following calibrations for an input file (rootname_raw.fits):

• Basic, two-dimensional image reduction producing a flat-fielded output image (rootname_flt.fits), which, depending on whether the data are from the MAMA detectors or the CCD and whether they are imaging or spectroscopic data, includes the following: data quality initialization, dark subtraction, bias subtraction, non-linearity flagging, flat fielding, and photometric calibration.
• Two-dimensional spectral extraction producing a flux-calibrated, rectified spectroscopic image (usually rootname_x2d.fits for MAMA data, rootname_sx2.fits for CCD) with distance along the slit running linearly along the Y axis and wavelength running linearly along the X axis, for spectroscopic first-order mode data. See Table 2.2 in the STIS Data Handbook.
• One-dimensional spectral extraction producing a one-dimensional spectrum of flux versus wavelength (usually rootname_x1d.fits for MAMA data, rootname_sx1.fits for CCD), uninterpolated in wavelength space, but integrated across an extraction aperture in the spatial direction, for first-order and echelle spectroscopic data. See Table 2.2 in the STIS Data Handbook.
• Data taken in TIME-TAG mode are corrected for the Doppler shift from the spacecraft motion and output as an uncalibrated event stream by the pipeline in a FITS binary table (rootname_tag.fits). The time-tag data stream is subsequently integrated in time to produce an uncalibrated ACCUM mode image (rootname_raw.fits) which is then passed through standard calibration. To correct TIME-TAG spacecraft times to heliocentric times, refer to Section 5.6.1 in the STIS Data Handbook.

In addition, calstis performs two types of contemporaneous calibrations:

• For CCD exposures which have been CR-SPLIT or when multiple exposures have been taken, calstis combines the exposures, producing a cosmic ray rejected image (rootname_crj.fits) which is then passed through subsequent calibration steps (e.g., spectral ­extraction).
• For spectroscopic exposures, calstis processes the associated wavecal exposure (see Section 3.3 Routine Wavecals) to determine the zero point offset of the wavelength and spatial scales in the science image, thereby correcting for thermal drifts and the lack of repeatability of the mode select mechanism. Raw wavecal data are stored in the file rootname_wav.fits to distinguish them from the uncalibrated science data in rootname_raw.fits.
  

Figure 15.1 through Figure 15.3 show example outputs from the calstis pipeline. The calstis program propagates statistical errors and tracks data quality flags through the calibration process. Although calstis is written in C, convenient Python wrapper programs are available as part of the stenv environment.  These routines may be used to recreate the results of the pipeline (e.g., by re-running calstis in its entirety) or to customize the processing by running modular components such as as basic two-dimensional image reduction (basic2d), two-dimensional spectral extraction (x2d), one-dimensional spectral extraction (x1d), or cosmic ray rejection (ocrreject).  Users can manually update the reference files listed in FITS headers by using the crds.bestrefs script. The calibration steps that calstis performs depends on the type of data.  The action of each processing step is controlled by the value of a keyword flag in the FITS header.  However, data obtained by available-but-unsupported-modes are calibrated only through flat-fielding in the pipeline.

Observers can retrieve STIS data from the MAST Portal, the HST Archive, or can use the Astropy module astroquery.mast to query and download data.

Between Spring 2001 and Fall 2016, calibrated data products for STIS were available through on-the-fly-reprocessing (OTFR), which replaced on-the-fly-calibration (OTFC). The OTFR system started with raw telemetry products, converted these to FITS files, and added the latest instrument calibrations.

Currently, the MAST archive maintains a cache of all STIS data that has been uniformly processed with the current instrumental calibration files.  When new reference files become available, all affected data sets are promptly reprocessed and the cache is updated.  This approach occasionally results in a delay between the submission of revised reference files and their application to the cached data.  Users who require data calibrated with the most recently updated reference files may wish to consult with the STIS team to verify via the HST Help Desk when calibrated products made using these updated files will be available.

Calstis evolves and improves with time as we understand and characterize the on-orbit performance of STIS more fully. For details about the various changes made to the STIS calibration processes, please refer to the STIS Data Handbook.