2.7 Error and Data Quality Array

The COS pipeline propagates data quality flags throughout the calibration process. The error estimate is computed from the total counts detected using the Gehrels variance approximation.

2.7.1 Error Array

The error array contains an estimate of the statistical error at each pixel. In the raw file, the error array is empty, and in the calibrated files the error array is generated from Poisson statistics of the gross counts and propagation of the errors to account for those introduced by correcting the observed data for flat-field and dead-time effects. The Poisson statistics are determined from using the methodology outlined in Gehrel, N. 1986.

2.7.2 Data Quality Flags

Every photon event in a COS corrtag file has a Data Quality (DQ) flag (Table 2.18). Each flagged condition sets a specific bit in the 16-bit DQ word, thus allowing each event during an exposure to be flagged with multiple conditions using the bitwise logical OR operation. DQ flags can be divided into four types:

1. Spatial flags mark events which fall on a detector region which may be questionable. The BPIXTAB reference file marks the corners of each region on the detector which falls into each of these categories. Separate BPIXTAB files are used for the FUV and NUV detectors. These regions were determined by visual inspection of a set of science data files. For FUV data, the GSAGTAB is applied along with the BPIXTAB and SPOTTAB. The GSAGTAB is used to flag regions that are severely gain sagged. The SPOTTAB is the hot spot reference table.

The DQICORR step of calcos maps these spatial regions to the individual photon events, and the x1dcorr module uses these flags and the value of SDQFLAGS to create the DQ and DQ_WGT arrays, and ultimately to determine which events to include in the final (x1dsum) spectrum (Section 3.4.18).

The spatial flags include:

• Detector shadows (4) include the locations of the grid wires for the FUV detector, and the vignetted region on the NUV detector.
• Poorly calibrated regions (8) include areas near the edge of the detector which may be suspect.
• Very low response regions (16) are areas on the detector where the response presents a >80% depression.
• Background features (32) correspond to regions on the detector where the background count rate has been observed to be higher than the surrounding region and/or unstable.
• The pixel out-of-bounds flag (128) marks regions outside of the calibrated region of the detector.
• Low response regions (1024) are areas on the detector where the response presents a >50% depression.
• Low PHA features (4096) are regions in which unusual features have been identified in long background exposures. These features may have an effect on very low count rate observations.
• Gain-Sag holes (8192) are regions on the FUV detector where the gain is low enough that the calibration may be affected (see Section 3.7.15 describing the GSAGTAB reference file).

2. Temporal flags mark photons that occur during time spans in which the data quality are suspect. Events flagged in this way will be removed from the data products, and the exposure time will be adjusted accordingly. Two types of temporal flags are used:

• FUV event bursts (64), which are flagged by the BRSTCORR module of calcos. As of this writing, no bursts have been seen on orbit, so the BRSTCORR step is set to OMIT by default. If bursts are seen at some point, it is likely that the parameters in the BRSTTAB reference table will have to be adjusted before using BRSTCORR.
• Other Bad Time Intervals (2048) can be defined in the BADTTAB reference file, for time ranges that are known to be problematic. At present, STScI has not defined any bad time intervals, but users running calcos on their own may wish to define their own intervals in order to exclude times with high background, etc.

3. Spatial and Temporal flags mark events that fall on a specific part of the detector, but also during specific time spans in which the data quality is suspect. Currently only the hotspot flag falls into this category.

• Hotspot flag (2) only applies to FUV data. If a hotspot overlaps any of the good time intervals, the region is added to the set of regions that are applied to create the DQ mask and against which each event is tested to assign a DQ value. The hot-spot regions are flagged in the two-zone extraction module, even if they fall only in the outer zone, and they do not contribute to the summed spectra in the x1dsum file.

4. Event flags are set by calcos if a photon event falls outside defined thresholds. Currently, only the FUV Pulse Height flag (512) falls into this category. All FUV events with pulse heights falling outside the range specified in the PHATAB reference file will have this flag set, and the data will be excluded by the DQICORR module. This flag does not apply to NUV data. The default value of SDQFLAGS does not include 512, but pulse height thresholding is still conducted by default.

5. Lost Data flags occur if data are missing for some reason, such as errors in transmitting the data from the instrument to the ground. Data marked with these flags is always excluded from the final products. There are two flags in this category:

• Reed-Solomon errors (1)
• Fill Data (256)

Screening for temporal and event flags is done by turning calibration switches on or off, or by altering reference files. Once a photon has been determined to have a bad temporal or event flag, it will never appear in a final data product unless the modules which screen for it are turned off or the reference files which define them are changed. Events with a spatial DQ flag are included in the calibrated product, and flagged in the final DQ array. The screening for the spatial flags can be easily altered by changing the SDQFLAGS keyword in the header of the raw data file.

The DQ extension of raw ACCUM files will be filled only when there are missing (data lost) or dubious (software error) data. If no such errors exist, initialization will produce an empty data quality extension whose header has NAXIS=0. These flags are set and used during the course of calibration, and may likewise be interpreted and used by downstream analysis applications. See Section 3.4.16 for more information on the data quality initialization calibration module.

Table 2.18: COS Data Quality Flags