2.5 Error and Data Quality Array

The STIS pipeline propagates both statistical errors and data quality flags throughout the calibration process. These are then combined from both the science data and the reference file data to produce triplets of {SCI, ERR and DQ} in the calibrated direct and spectral imaging data.

Note that both the error and data quality image extensions may be represented with a null array (i.e., NAXIS=0 following STScI conventions) if all the values are identically zero (see Table 2.9 and STIS ISR 95-06).

2.5.1 The Error Array

The error array contains an estimate of the statistical error at each pixel. In the raw file, the error array is empty. The first step of calstis is to calculate the error array for the input data. This raw data error is simply given as:

\sigma_\mathrm{DN} = \frac{\sqrt{\sigma_\mathrm{c}^2 + R \cdot g}}{g}

where:

  • R is the observed data number (counts) minus the electronic bias of the pixel; note that the electronic bias is zero for the MAMA.
  • g is the gain factor; note that the gain is unity for MAMA observations.
  • \sigma_\mathrm{c} is the read noise in electrons for CCD observations (it is set to zero for MAMA observations).

The bias, gain factor, and read noise are read from the CCD parameters reference file for CCD data (a _ccd file specified by keyword CCDTAB). As the data are calibrated through the calstis pipeline, the statistical errors are propagated through, reflecting both the science and reference file errors (see STIS ISR 98-26 Section 6 for details).

2.5.2 Data Quality Flagging

Data quality flags are assigned to each pixel in the data quality extension. Each flag has a true (set) or false (unset) state. Flagged conditions are set as specific bits in a 16-bit integer word. For a single pixel, this allows for up to 15 data quality conditions to be flagged simultaneously, using the bitwise logical OR operation. Note that the data quality flags cannot be interpreted simply as integers but must be converted to base-2 and interpreted as flags. Table 2.9 gives the specific conditions that are flagged, depending on the states of different bits (i.e., being on or off).

The raw data quality files will be filled only when there are missing (data lost) or dubious (software error) data. If no such errors exist, initialization will produce an empty data quality file whose header has NAXIS=0.

These flags are set and used during the course of calibration, and may likewise be interpreted and used by downstream analysis applications.

Table 2.9: STIS Data Quality Flags

FLAG Value

Bit Setting1

Quality Condition Indicated

1

0000 0000 0000 0001

Error in the Reed-Solomon decoding (an algorithm for error correction in digital communications).

2

0000 0000 0000 0010

Lost data replaced by fill values.

4

0000 0000 0000 0100

Bad detector pixel (e.g., bad column or row, mixed science and bias for overscan, or beyond aperture).

8

0000 0000 0000 1000

Data masked by occulting bar.

16

0000 0000 0001 0000

Pixel having dark rate > 5 σ times the median dark level.

32

0000 0000 0010 0000

Large blemish, depth < 40% of the normalized p-flat (repeller wire).

64

0000 0000 0100 0000

Vignetted pixel

128

0000 0000 1000 0000

Pixel in the overscan region.

256

0000 0001 0000 0000

Saturated pixel, count rate at 90% of max possible—local non-linearity turns over and is multi-valued; pixels within 10% of turnover and all pixels within 4 pixels of that pixel are flagged.

512

0000 0010 0000 0000

Bad pixel in reference file.

1024

0000 0100 0000 0000

Small blemish, depth between 40% and 70% of the normalized flat. Applies to MAMA and CCD p-flats.

2048

0000 1000 0000 0000

>30% of background pixels rejected by sigma-clip, or flagged, during 1-D spectral extraction.

4096

0001 0000 0000 0000

Extracted flux affected by bad input data.

8192

0010 0000 0000 0000

Data rejected in input pixel during image combination for cosmic ray rejection.

16384

0100 0000 0000 0000

Extracted flux not CTI corrected because gross counts are ≤ 0.

1 The most significant bit is on the left in this representation.