E.1 The STScI Reduction and Calibration Pipeline

In this appendix, we summarize the basic reductions and calibrations that are performed in the STScI WFC3 pipeline. The material in this appendix is intended to provide only enough background to develop robust observing proposals. The WFC3 Data Handbook provides more detailed information needed for analyzing your data.

In February 2016, the pipeline began to process WFC3/UVIS data with calwf3 version 3.3, which incorporates two fundamental changes to the way exposures are calibrated and corrected. First, calwf3 applies pixel-based CTE (Charge Transfer Efficiency) corrections. (Calwf3 version 3.4, which added CTE corrections for subarray apertures, became operational in October 2016.) Second, photometric calibrations are determined and applied independently for each CCD chip. See WFC3 Space Telescope Analysis Newsletter issue 22 for a summary of the changes underlying and included in calwf3 version 3.3. WFC3 ISR 2016-01 and WFC3 ISR 2016-02 provide a reference guide and cookbook for calwf3 version 3.3, respectively. WFC3 ISR 2016-03 provides chip-dependent inverse sensitivities. WFC3 ISR 2016-04 describes the creation of chip-dependent flat fields, which included a correction for the effect of the crosshatch pattern in the UV filters on sensitivity calibrations, based on analysis presented in WFC3 ISR 2015-18 and described in Section 5.4.3WFC3 ISR 2016-07 documents the changes in WFC3/UVIS component files used by SYNPHOT and PYSYNPHOT to simulate HST photometric measurements.

Science data taken by WFC3 are received from the Space Telescope Data Capture Facility and sent to the STScI OPUS pipeline, where the data are unpacked, keyword values are extracted from the telemetry stream, and the science data reformatted and repackaged into raw (uncalibrated) FITS files by the generic conversion process. All WFC3 science data products are two-dimensional images that are stored in FITS image-extension files. Like ACS and STIS images, WFC3 UVIS channel exposures are stored as triplets of FITS image extensions, consisting of science (SCI), error (ERR), and data quality (DQ) arrays. There is one triplet of image extensions for each CCD chip used in an exposure. Full-frame exposures, using both chips, therefore have two triplets of SCI, ERR, and DQ extensions in a single FITS file. UVIS subarray exposures, which use only one CCD chip, have a single triplet of extensions in their FITS files.

WFC3 IR channel exposures use the NICMOS file structure, which are quintuplets of FITS image extensions, consisting of science (SCI), error (ERR), data quality (DQ), number of samples (SAMP), and integration time (TIME) arrays. There is one quintuplet of extensions for each of the non-destructive detector readouts that make up an IR exposure. Using the maximum number of readouts (16) in an IR exposure therefore results in a single FITS file containing a total of 80 image extensions.

The raw, uncalibrated FITS files are processed through calwf3, the software task that calibrates the data for individual exposures, producing calibrated FITS files. Exposures that are obtained as part of an associated set, such as dithered images, have calwf3 calibration applied to the individual exposures before being processed as a set for the purpose of image combination. All calibrated images will be processed further with the STScI AstroDrizzle software, for the purpose of removing geometric distortions from individual exposures and for combining associated exposures.

The FITS file name suffixes given to WFC3 raw and calibrated data products are described in Table E.1 and closely mimic the suffixes used by ACS and NICMOS. The initial input files to calwf3 are the raw (RAW) files from generic conversion and the association (ASN) table, if applicable, for the complete observation set. Most WFC3/UVIS RAW images first go through pixel-based CTE correction, producing a temporary, CTE-corrected RAW file with the suffix “RAC_TMP”. The RAC_TMP and original RAW files have the same calibration steps applied, producing two sets of final calibrated products (one uncorrected and one corrected for CTE). For WFC3/UVIS images, a temporary file, with the suffix “BLV_TMP” (BLC_TMP for CTE products), is created by calwf3 once bias levels have been subtracted and the overscan regions trimmed. This file is renamed using the “FLT” (“FLC” for CTE products) suffix after the remaining standard calibrations (dark subtraction, flat fielding, etc.) have been completed. For exposures taken as part of a UVIS CR-SPLIT or REPEAT-OBS set, a parallel set of processing is performed, using the BLV_TMP/BLC_TMP files as input to an image combination and cosmic ray rejection routine. The resulting CR-combined image, with a temporary file name suffix of “CRJ_TMP” (“CRC_TMP” for CTE products), then receives the remaining standard calibrations, after which it is renamed using the “CRJ” (“CRC” for CTE products) suffix.

Table E.1: WFC3 File Name Suffixes.

File Suffix




Raw data (with CTE correction)



Association file for observation set



Telemetry and engineering data



Trailer file with processing log



Bias subtracted individual UVIS exposure (with CTE correction)



Uncalibrated, CR-rejected combined UVIS image (with CTE correction)



Calibrated intermediate IR exposure



Calibrated individual exposure (with CTE correction)

e (UVIS)
e/s (IR)


Calibrated, CR-rejected, combined UVIS image (with CTE correction)



Calibrated, geometrically-corrected, dither-combined image


Processing of WFC3/IR exposures results in an intermediate MULTIACCUM (IMA) file, which is a file that has had all calibrations applied (dark subtraction, linearity correction, flat fielding, etc.) to all of the individual readouts of the IR exposure. A final step in calwf3 processing of WFC3/IR exposures produces a combined image from the individual readouts, which is stored in an FLT output product file.

The UVIS processing portion of calwf3 is based on calacs calibrations of ACS/WFC CCD images, while the calwf3 IR processing is very similar to calnica processing of NICMOS images. Calwf3 performs the following basic science data calibrations:

  • Pixel-based CTE correction (UVIS only)
  • Bad pixel flagging
  • Bias level subtraction (UVIS); Reference pixel subtraction (IR)
  • Bias image subtraction (UVIS); Zero-read subtraction (IR)
  • Dark current subtraction
  • Post-Flash subtraction
  • Non-linearity correction
  • Flat-field correction and gain calibration
  • Shutter shading correction (UVIS only)
  • Up-the-ramp fitting (IR only)
  • Photometric calibration
  • CR-SPLIT/REPEAT-OBS image combination

As noted in the list above, the details of some calibration steps differ for UVIS and IR exposures, while others do not apply at all. The process of bias subtraction, in particular, differs for UVIS and IR exposures. The UVIS channel CCDs include regions of overscan, which are used for measuring and subtracting the overall bias level from each CCD exposure. A bias reference image is also subtracted from each science exposure to remove spatial variations in the bias. For IR exposures, the reference pixels located around the perimeter of the detector are used to track and remove changes in the overall bias level between readouts, while the image from the initial (“zeroth”) readout of the exposure is subtracted from all subsequent readouts to remove spatial bias structure.

UVIS shutter shading correction is in principle only necessary for very short duration exposures. Note, however, that testing has shown that the shading amounts to only a 0.2-0.3% variation across the field and therefore this step is normally not applied.

Up-the-ramp fitting is applied to IR exposures to determine a final signal rate for each pixel in the image. This process not only determines the best-fit rate from the individual readouts of the exposure, but also detects and removes effects due to cosmic-ray hits. This process is also capable of recovering a useful signal for pixels that go into saturation during the exposure by using only the non-saturated readouts to compute the fit.

WFC3 grism observations are handled in a special way by the pipeline. Grism observations require a special flat-fielding procedure, where the flat-field value for each pixel is based on the wavelength of the detected signal. Calwf3 processing of grism images therefore uses an “identity” flat-field reference image (an image filled with values of 1.0 at each pixel), which allows for the gain calibration part of the flat-fielding step to still be applied without actually flat-fielding the science image. A separate software package, aXe, which was developed at ST-ECF and previously used for processing NICMOS and ACS spectral observations, is used to extract and calibrate one-dimensional spectra from WFC3 grism exposures (see Section 8.5). The aXe software is available in STSDAS via PyRAF. The aXe software is used to locate and extract spectra of individual sources from calibrated images and performs wavelength calibration, background subtraction, flat fielding, and absolute flux calibration for the extracted spectra.

Table E.2 shows the values assigned to pixels in the DQ arrays of calibrated images, which indicate anomalous conditions and are frequently used in downstream processes to reject a pixel value. If more than one data quality condition applies to a pixel, the sum of the values is used. Note that some flag values have different meanings for UVIS and IR images.

Table E.2: WFC3 Data Quality Flags.

FLAG ValueData Quality Condition








Reed-Solomon decoding error

Reed-Solomon decoding error


Data replaced by fill value

Data replaced by fill value


Bad detector pixel

Bad detector pixel



Unstable in zero-read


Hot pixel

Hot pixel


CTE tail

Unstable photometric response


Warm pixel

(Obsolete: Warm pixel)


Bad pixel in bias

Bad reference pixel


Full- well saturation

Full-well saturation


Bad or uncertain flat value

Bad or uncertain flat value


Charge trap



A-to-D saturation

Signal in zero-read


Cosmic ray detected by AstroDrizzle

Cosmic ray detected by AstroDrizzle


Cosmic ray detected during CR-SPLIT or REPEAT-OBS combination

Cosmic ray detected during up-the-ramp fitting


Pixel affected by ghost or crosstalk (not used)

Pixel affected by crosstalk (not used)