3.7 Reference Files

This section contains a description of the COS reference files. See Figure 3.1–Figure 3.6 for which modules use these files and Section 3.4 for explanations of how their contents are applied by those modules. The reference files are now described in the order they are called by the pipeline for the case of FUV TIME-TAG data (Figure 3.1).

3.7.1 BADTTAB: Bad Time Interval Table

• File Suffix: _badt

The BADTTAB reference file lists the start and end times of known bad time intervals. It is used by the BADTCORR calibration module to flag events in TIME-TAG events lists which occur during a bad time interval. In later processing the flagged events will be removed from the final calibrated data, and the exposure time header keyword, EXPTIME, updated. The bad time interval table consists of segment, start, and end columns (see Table 3.4). The segments columns can be populated with either FUVA, FUVB or ANY. The start and end columns are in Modified Julian Date.

Table 3.4: BADTTAB Table Content.

3.7.2 BRFTAB: Baseline Reference Frame Table

• File Suffix: _brf

The BRFTAB reference file is only applicable to FUV data and is used during pipeline processing in the TEMPCORR module to apply the thermal distortion correction. The FUV detector does not have physical pixels like a CCD. Instead, the x and y positions of detected photon events are obtained from analog electronics, which are susceptible to thermal changes. Electronic stim pulses are normally commanded during integration and are used as physical position reference points. To return the FUV data to a known physical space, the BRFTAB defines the stim pulse positions.

The BRFTAB file consists of a primary header extension and a binary table extension. The table lists the stim pulse locations and search regions, and the active detector areas (Table 3.5).

Table 3.5: BRFTAB Table Contents.

¹ Stim pulse 1 is located in the upper left corner.
² Stim pulse 2 is located in the lower right corner.

3.7.3 GEOFILE: Geometric Correction File

• File Suffix: _geo

This file is only used for FUV data. The GEOFILE is used by the GEOCORR calibration module to perform the geometric correction. The analog nature of the XDL detector means that the physical sizes of the pixels vary across the detector. The geometric distortion maps are used to correct for this variation and to transform the data into a constant physical pixel size early in the data reduction calibration process. After the thermal correction has been applied, the geometric correction can be applied. This implies that all the files used to determine the geometric correction were initially thermally corrected.

Each geometric correction reference file contains four IMAGE extensions. There are two for each segment, and for each segment, there is one for each axis. At a given (X,Y) location in the thermally corrected COS data, the value at that location (corrected for binning and offset) in the geometric correction image gives the distortion to be subtracted from the X or Y coordinates. The order of the extensions are: 1=X coordinate for FUVA, 2=Y coordinate for FUVA, 3=X coordinate for FUVB and 4=Y coordinate for FUVB. This information is also available in the header file, with keywords EXTVER (1 for x and 2 for y) and EXTNAME (FUVA or FUVB).

3.7.4 DGEOFILE: Delta Geometric Correction File

• File Suffix: _dgeo

The delta geometric distortion reference file is used to improve the geometric correction for the FUV detector. It is defined and applied in the same way as the geometric correction, and is only applied to data that have been geometrically corrected. At a given (X,Y) location in the geometrically corrected COS data, the value at that location (corrected for binning and offset) in the delta geometric correction image gives the distortion to be subtracted from the X or Y coordinate.

Each delta geometric correction reference file contains four IMAGE extensions. There are two for each segment, and for each segment, there is one for each axis. At a given (X,Y) location in the corrected COS data, the value at that location (corrected for binning and offset) in the delta geometric correction image gives the distortion to be subtracted from the X or Y coordinates. The order of the extensions are: 1=X coordinate for FUVA, 2=Y coordinate for FUVA, 3=X coordinate for FUVB and 4=Y coordinate for FUVB.

3.7.5 XWLKFILE, YWLKFILE: X Walk Correction, Y Walk Correction

• File Suffix: _xwalk, _ywalk

The XWLKFILE and YWLKFILE reference files are only applicable to FUV data and are used during pipeline processing in the XWLKCORR and YWLKCORR modules to correct the effects of walk. The COS FUV XDL detector is subject to gain sag; as physical locations on the detector accumulate photon events, the number of electrons in the charge cloud generated by an event becomes smaller, and as a result the coordinates of the event may be mis-registered by the electronics. These effects depend on event pulse height and the position on the detector.

The design of the XWLKFILE and YWLKFILE allow both the X and Y position to be corrected based on the geometrically corrected X location and the pulse height. Each file consists of a primary header and a two binary table extension (one for each segment). One extension (EXTNAME='FUVA') is used to correct the data on Segment A, and the other (EXTNAME='FUVB') is used for Segment B. Details of how the correction is applied are given in the XWLKCORR/YWALKCORR section (Section 3.4.6). The data in these files will be updated in the future as the walk correction is further refined. Note that these files replace the original WALKTAB, which is now deprecated.

3.7.6 DEADTAB: Deadtime Table

• File Suffix: _dead

The DEADTAB reference file is used in the DQICORR: Initialize Data Quality File module, to obtain the true number of events received compared to the number of events appearing in the raw data files.

There is one DEADTAB reference file for each of the NUV and FUV detectors. Each consists of a primary header and a binary table extension which contains the LIVETIME values for a given observed count rate (OBS_RATE) and segment. The livetime is defined as:

and can be used to calculate the true count rate.

3.7.7 PHATAB: Pulse Height Discrimination Table

• File Suffix: _pha

The PHATAB reference file is only valid for FUV data, and is applied during the PHACORR step of calcos to filter non-photon events. The file consists of two header/data units, the first being the primary header, and the second a binary table (see Table 3.6). The table lists the lower and upper thresholds for valid individual pulse heights in TIME-TAG mode. In TIME-TAG mode, each detector event has an associated pulse-height of 5 bits with values ranging from 0 to 31, The table also gives the minimum and maximum values for the location of the mean value of the pulse height distribution used in ACCUM mode. In ACCUM mode, a pulse height distribution histogram is generated for the whole exposure over the entire detector and downloaded as part of the science data file. The histogram includes all the digitized events for each segment independently of the currently defined subarrays. Note in ACCUM mode the pulse height is a 7 bit number with values ranging from 0 to 127.

Table 3.6: PHATAB Table Contents.

3.7.8 PHAFILE: Pulse Height Discrimination File

• File Suffix: _phf

This file is only used for FUV data, and is a 2D equivalent to the PHATAB. The PHAFILE is used by the PHACORR calibration module to filter non-photon events. If both a PHATAB and PHAFILE are available, the PHAFILE will be used.

Each pulse height discrimination reference file contains four IMAGE extensions. There are two for each segment, containing the lower and upper PHA limits for each pixel. At a given (X,Y) location in the uncorrected COS data, the value at that location gives the lowest and highest (respectively) pulse height that will be treated as a valid photon event at that detector location.

3.7.9 FLATFILE: Flat-field File

• File Suffix: _flat

FLATFILE provides a flat-field image which is used by the pipeline to remove the pixel-to-pixel variations in the detector. The FUV FLATFILE consists of a primary header and two 14000 × 400 IMAGE extensions, one for each segment. This file is lifetime dependent. The NUV FLATFILE consists of a primary header and a 1024 × 1024 IMAGE extension.

The FUV flat-field reference files correct for grid wire shadows and for an effect of small-scale geometric distortion. There are multiple files for different combinations of lifetime position and grating (G130M, G160M, and G140L).

The NUV flat-field is a combination of internal and external deuterium flat field lamp exposures from thermal-vacuum testing which illuminate the portion of the detector where spectra fall. The data cover the following pixel region of the detector: x (dispersion): 0 to 1023, and y (cross-dispersion): 495 to 964. The rest of the detector, where flat field data are not available, has a value of 1.0. The bottom four and top three rows of the detector do not fit well with the rest of the detector and they are flagged in the data quality table.

3.7.10 LAMPTAB: Template Calibration Lamp Spectra Table

• File Suffix: _lamp

The LAMPTAB files consist of a primary header and a binary table extension which contains an extracted 1-D spectrum from the internal PtNe calibration lamp through the WCA aperture, for each grating, central wavelength, and FP-POS setting. It is used in the calcos pipeline to determine the pixel offset of the observed data. The structure of the template calibration lamp spectra table is shown in Table 3.7. The stepper motor offsets range from –2 to +1 and correspond to FP-POS settings of 1 to 4. Note that LAMPTAB files are lifetime position specific in the FUV. LP2 currently uses the LAMPTAB file from LP1. The NUV remains at the same LP1.

Table 3.7: LAMPTAB Table Contents.

3.7.11 DISPTAB: Dispersion Coefficient Table

• File Suffix: _disp

There are multiple DISPTAB files in CRDS with similar formats, one for the NUV, and multiple for the FUV at different lifetime positions. They consist of a main header and a binary table in the second HDU. These tables provide the dispersion relations for each segment, aperture, optical element, and central wavelength. Each file has the format given in Table 3.8. The dispersion relation table gives a set of polynomial coefficients for computing wavelength from pixel number (see Oliveira et al., COS ISR 2010-05 and 06 for details).

Each row of the table gives a set of dispersion coefficients. The row to be used is selected on SEGMENT, OPT_ELEM, CENWAVE, and APERTURE. Note that DISPTAB files are lifetime position specific.

Table 3.8: DISPTAB Table Format.

For P_x = the zero-indexed Doppler corrected pixel value in the dispersion direction, let

then the corresponding wavelength in Angstroms is given by:

3.7.12 XTRACTAB: 1-D Spectral Extraction Table

• File Suffix: _1dx

There are multiple XTRACTAB files with similar formats, one for the NUV and multiple for the FUV. The FUV XTRACTAB files are lifetime dependent. They consist of a main header and a binary table in the second HDU. These tables provide the information needed to extract the spectrum from a geometrically corrected image of the detector for each optical element and central wavelength. Each file has the format given in Table 3.9.

Table 3.9: XTRACTAB Table Format.

The spectral extraction of a source is performed by collapsing the data within a parallelogram of height HEIGHT that is centered on a line whose slope and intercept are given by SLOPE and B_SPEC. Similarly, two background spectra are determined by collapsing the data within parallelograms of height B_HGT1 or B_HGT2 centered on the lines defined by SLOPE and B_BKG1, and SLOPE and B_BKG2, respectively. The background spectra are then smoothed by a boxcar of width BWIDTH. These are then scaled and subtracted from the source spectrum.

3.7.13 BRSTTAB: Burst Parameters Table

• File Suffix: _burst

The BRSTTAB file is used for FUV data only. It provides the parameters needed to identify bursts. It consists of a primary header extension and a binary table extension with the columns listed in Table 3.10. Details of the burst rejection routine are given in Section 3.4.13.

Table 3.10: BRSTTAB Table Contents.

3.7.14 BPIXTAB: Bad Pixel Table

• File Suffix: _bpix

The data quality initialization table identifies rectangular regions on the detectors that are known to be less then optimal. The feature type describes the type of detector blemish enclosed within the bounding box and DQ is the quality value assigned to all events detected within the box. The regions were identified by visual inspection of the combined flat field data for each detector (and segment). The BPIXTAB files consist of a primary header and a binary table extension which consists of the columns listed in Table 3.11.

Table 3.11: BPIXTAB Table Content.

In the BPIXTAB table, the DQ field may be a logical OR due to several different values, each associated with a unique issue (see Table 2.19).

3.7.15 GSAGTAB: Gain Sag Table

• File Suffix: _gsag

The gain sag reference table is only applicable for FUV data and it is used along with the bad pixel reference table (_bpix) in the DQICORR module. The table provides the locations of rectangular regions for portions of the FUV detector that have very low pulse height amplitude.

After the primary header, each extension of the GSAGTAB is a binary fits table of the gain sagged pixels on the detector at a given voltage. During the pipeline processing, these extensions are selected depending on the SEGMENT and HVLEVEL. Each row in the table gives the location and data quality value for one rectangular region. The DATE column is used to select rows. A row will be used to flag a gain sagged region if the value in the DATE column is less than or equal to the exposure start time. For a description on the columns contained in the binary tables see Table 3.12.

Table 3.12: GSAGTAB Table Format.

3.7.16 SPOTTAB: Hotspot Table

• File Suffix: _spot

The hotspot table is only applicable for FUV data, and is used along with the bad pixel reference table (_bpix) and gain sag table (_gsag) in the DQICORR module. The table provides the start and stop times, locations and extents of hotspots, which are transient regions of high detector background.

The hotspot table is a FITS table with a primary header and 1 extension with optional EXTNAME = HOTSPOT. Each row has 9 columns: SEGMENT is the segment name the hotspot appears in (FUVA or FUVB). START and STOP are the MJD times of the start and stop of the hotspot. LX and LY are the (XCORR, YCORR) coordinates of the lower left corner of the rectangular hotspot region. DX and DY are the extent, in pixels, of the rectangular hotspot region. DQ is the value of the DQ flag to be applied to the region (see Table 2.19), and COMMENT is a comment string.

The following table describes the column definitions.

Table 3.13: SPOTTAB Table Format.

The hotspot is selected based on the value of SEGMENT, and then the START and STOP times are compared to the start and stop times of the good time intervals of the exposure being calibrated. If a hotspot overlaps any of the good time intervals, the region is added to the set of regions that are applied to create the DQ mask and against which each event is tested to assign a DQ value. The hotspot regions are flagged in the two-zone extraction module even if they are only in the outer zone, and they do not contribute to the summed spectra in the x1dsum file.

3.7.17 WCPTAB: Wavecal Parameter Table

• File Suffix: _wcp

The WCPTAB file contains information relevant for the wavecal pipeline processing. This file is lifetime dependent. It consists of primary header and a binary table extension which is described in Table 3.14. XC_RANGE is the maximum pixel offset to use when doing a cross correlation between the observed data and the template wavecal. That is, the observed spectrum should be shifted relative to the template by a number of pixels, ranging from –XC_RANGE to +XC_RANGE inclusive. XD_RANGE is half the search range for finding the spectrum in the cross dispersion direction. The search range is from B_SPEC–XD_RANGE to B_SPEC+XD_RANGE inclusive, where B_SPEC is the nominal location of the spectrum from the XTRACTAB table discussed below. BOX is the width of the boxcar filter for smoothing the cross-dispersion profile. RESWIDTH is the number of pixels per resolution element, and is assigned a value of 6.0 for the FUV detectors and 3.0 for the NUV detector.

When applying the offsets found from the wavecals to the science data, it may happen that there was no wavecal at the same OSM position. In this case, the wavecal that was closest in time to the science observation may be used, with a correction for the difference in OSM positions. That correction is based on STEPSIZE, the number of pixels corresponding to one OSM step. There may be a check, however, to guard against using a wavecal that was taken too far away in time from the science observation. If the science observation and wavecal were taken more than MAX_TIME_DIFF apart, then the wavecal should not be used for that science observation.

Table 3.14: WCPTAB Table Contents.

¹ Only at LP4.

3.7.18 FLUXTAB: Photometric Throughput Table

• File Suffix: _phot

There are multiple FLUXTAB files with similar formats, one for the NUV, and multiple for the FUV. The FUV FLUXTAB files are lifetime dependent. They consist of a main header and a binary table in the second HDU. These tables provide the information needed to convert from corrected detector counts to flux units of erg s^–1 cm^–2 Å^–1 for each segment, optical element, aperture, and central wavelength. Each file has the format given in Table 3.15.

Table 3.15: FLUXTAB Table Format.

The units of the Sensitivity array are (count s^–1 pixel^–1)/(erg s^–1 cm^–2 Angstrom^–1). For each segment, optical element, central wavelength setting, and aperture, these files contain arrays of wavelengths and sensitivities which can be interpolated onto the observed wavelength grid. The net counts can then be divided by the sensitivity curves to produce flux calibrated spectra.

3.7.19 TDSTAB: Time Dependent Sensitivity Table

• File Suffix: _tds

There are two such files, one for the FUV and one for the NUV. They are only used for spectroscopic data. The files contain the information necessary to determine the relative sensitivity curve at any given time by interpolating between relative sensitivity curves given at fiducial times which bracket the observation, or else extrapolate the results from the last curve if the observation date is more recent than the last fiducial date. Interpolation data are provided for each segment, optical element, and aperture (see Table 3.16). Updated TDS plots are located at:
http://www.stsci.edu/hst/instrumentation/cos/performance/sensitivity.

Table 3.16: TDSTAB Table Format.

For an observation obtained at time T, which lies between TIME[j] and TIME[j+1], the sensitivity curve used to calibrate the spectrum will be corrected by the following factor:

where REF_TIME is a general reference time given in the header of the FITS extension.

3.7.20 TRACETAB: Trace Correction Table

• File Suffix: _trace

The trace table gives the variation of the centroid of the spectrum as a function of column number (XCORR) in COS FUV data. This file is lifetime dependent, but currently only provided for LP3 and greater.

The file is a FITS table with a primary header and one extension. The row to be used is selected on SEGMENT, OPT_ELEM, CENWAVE and APERTURE. Each row has 8 columns. DESCRIP supplies a short description, while TRACE_YLOC is the location of the center of the trace. TRACE is an array of 16,384 floats where the index is the value of XCORR and the value is the offset to be subtracted from each event's YFULL value. The value of XCORR for each event is interpolated onto the TRACE array to give the value of the shift to be applied to the corresponding YFULL value of the event. ERROR is an array of 16,384 floats that gives the statistical error of the TRACE measurement. Table 3.17 describes the column definitions.

Table 3.17: TRACETAB Table Format.

3.7.21 PROFTAB: Profile Table

• File Suffix: _profile

The profile table gives the profile of a point source perpendicular to the dispersion direction as a function of column number (XFULL) in COS FUV data. This file is lifetime dependent, but currently only provided for LP3 and greater.

PROFTAB is a FITS table with a primary header and one extension with optional EXTNAME = PROFILE. The row to be used is selected on SEGMENT, OPT_ELEM, CENWAVE and APERTURE. Each row has 8 columns. DESCRIP gives a short description of the row. CENTER is the measured centroid of the profile in the full-sized array in (XFULL, YFULL) coordinates. ROW_0 is the index of the first row of the profile in the full-sized array. In other words, if the profile has NROWS rows, it corresponds to rows with 0-based indices running from ROW_0 to (ROW_0 + NROWS – 1). PROFILE is the 2-d array of floats that gives the profile in the cross-dispersion direction for each column of data in (XFULL, YFULL) space (offset by ROW_0). During the ALGNCORR step, the flux-weighted centroid of the science data over 'good' rows and columns is calculated, and compared with the flux-weighted centroid of the profile contained in this reference file over the same rows and columns. The difference between these centroids is applied to the YFULL values of the events to align each set of science data to the same center. Table 3.18 describes the column definitions. The 2D spectral profiles contained in the PROFTAB for three settings (G140L/1280, G130M/1291, and G160M/1577) are given in Figure 3.17, Figure 3.18, and Figure 3.19.

Table 3.18: PROFTAB Table Format.

3.7.22 TWOZXTAB: TWOZONE Spectral Extraction Table

• File Suffix: _2zx

The TWOZONE extraction table contains the starting values for the object center and background regions, as well as the cumulative flux boundary values for the TWOZONE extraction. This file is lifetime dependent, but currently only provided for LP3 and greater.

TWOZXTAB is a FITS table with a primary header and one data extension. The row to be used is selected on SEGMENT, OPT_ELEM, CENWAVE and APERTURE. Each row has 16 columns. B_SPEC is the center of the science extraction aperture, and is used by the ALGNCORR step to get an initial guess for the location of the spectral trace. B_BKG1 and B_BKG2 are the center of the background regions, HEIGHT is the height of the target extraction region, and BHEIGHT is the height of the background extraction regions. BWIDTH is the width of the smoothing box used to smooth the background region in the extraction step.

In the TWOZONE extraction step, the spectral profile in the PROFTAB is analyzed to determine the boundaries of INNER and OUTER zones. These boundaries are specified in terms of the cumulative flux enclosed. In the INNER region, the flux is summed within the region and any DQ flags are propagated to the extracted spectrum. In the OUTER region, the flux is also summed and added to the flux in the inner region, but any DQ flags in the outer region are not propagated to the final extracted spectrum unless they are in the DQ value SDQOUTER from the primary header. The columns LOWER_OUTER, UPPER_OUTER, LOWER_INNER and UPPER_INNER give the cumulative flux boundaries to be used in the two zone extraction. Typically the outer boundaries enclose 99% of the flux, while the inner boundaries enclose 80%.

The YERRMAX column is used in the ALGNCORR step to test the statistical error in the calculation of the flux-weighted centroid of the science data. If this measurement is greater than the value of YERRMAX for that setting, the spectrum is deemed 'not found,' and the location of the center of the reference profile is used instead. The PEDIGREE column gives the pedigree of the information in the row, with values that are typically INFLIGHT, GROUND or DUMMY.

Table 3.19 describes the column definitions.

Table 3.19: TWOZXTAB Table Format.

3.7.23 SPWCSTAB: Spectroscopic WCS Parameters Table

• File Suffix: _spwcs

The spectroscopic SPWCS table gives the parameters needed to populate the world coordinate keywords in the corrtag, counts, and flt files. There are entries for each SEGMENT, OPT_ELEM, CENWAVE, and APERTURE. The columns (see Table 3.20) are interpreted as follows. The detector coordinate system has two dimensions. Let the more rapidly varying axis be X and the less rapidly varying axis Y. The world coordinate system has three dimensions, the spectral coordinate, right ascension, and declination. The reference pixel is at approximately the middle of the detector. CTYPE1 can be WAVE to indicate that the wavelength is a linear function of pixel number, or it can be WAVE-GRI to indicate that the wavelengths should be computed by using the grating ("grism") equation. In either case, the wavelengths are in vacuum. CRVAL1 is the wavelength at the reference pixel. CRPIX1 is the location of the reference pixel in the first axis (X); the location of the reference pixel in the second axis (Y) is obtained separately from the 1-D Extraction Parameters Table (XTRACTAB). CDELT1 is the dispersion in Angstroms per pixel at the reference pixel. At a single wavelength (nominally the wavelength at the reference pixel), a pixel when projected onto the sky would be approximately a rectangle. CDELT2 and CDELT3 are the sizes of that rectangle in the X and Y directions. SPECRES is the spectral resolution; this is only used for updating the archive search keyword of the same name. G is the groove density of the grating, e.g., 3.8E6 grooves per meter for G130M. SPORDER is the spectral order. This will usually be 1, but for G230L, stripe NUVC, SPORDER will be 2. ALPHA is the angle between the normal to the grating and the light that is incident onto the grating. THETA is the angle between two lines from the grating to the detector, the line to the reference pixel and the line that is perpendicular to the detector. Since the reference pixel is close to the middle of the detector, THETA will probably be close to zero.

Table 3.20: SPWCSTAB Table Format.