4.1 Error Sources Associated with Pipeline Calibration Steps

In this section, we discuss sources of error that are associated with major steps in the STIS calibration pipeline (calstis). Note that these steps themselves were already described in Chapter 3 and will not be repeated here; this section will only describe specific issues related to the error budget of the resulting data which were not described before.

4.1.1 Readout Noise and A-to-D Conversion

Readout Noise (only relevant for CCD observations)

The readout noise is an unavoidable contribution to the total error budget of CCD observations. (MAMA observations do not suffer from readout noise.) It is linked to the readout process and there are no reduction steps that can minimize or remove it. The effect of the read noise on science measurements can however be minimized by keeping the number of pixels where signal is measured small and by minimizing the number of CCD readouts (while still allowing for good cosmic ray and hot pixel removal). The read noise is independent of position on the CCD, but it does vary with the gain setting. Furthermore, the read noise of the STIS CCD has been found to vary with time. The main changes occurred after Servicing Mission 3a (January 2000) and after the switch to the Side-2 electronics (July 2001). A table of read noise values of the STIS CCD as a function of time and gain setting can be found at the following URL: http://www.stsci.edu/~STIS/monitors/readnoise/.

A-to-D Conversion (only relevant for CCD observations)

The analog information (electrons) accumulated in the CCD are converted into data numbers (DN) by an analog-to-digital converter (ADC). The STIS CCD camera employs a 16-bit ADC, which can produce a maximum of 2¹⁶ = 65,535 DN. However, this is not a limitation for STIS CCD operations, because other factors set the maximum observable DN to lower levels in both supported gain settings. (For the CCDGAIN = 1 setting, the gain amplifier already saturates at ~33,000 e¯/pixel, while the full well of the CCD varies across the detector between 108,000 and 160,000 e¯/pixel (ISR STIS 2015-06), which is before digital saturation would occur in the CCDGAIN = 4 setting.) The ADC produces only discrete output levels. This means that a range of analog inputs can produce the same digital output. This round-off error is called quantization noise (QN). It can be shown (Janesick, 2001, Scientific Charge Coupled Devices, SPIE Press) that the quantization noise is constant for a given gain setting when expressed in DN: $//<![CDATA[ \begin{array}{l}QN(\mathrm{DN}) = 12^{-1/2}\end{array} //]]>$. The quantization noise can be converted into noise electrons as follows: $//<![CDATA[ \begin{array}{l}QN(e^-) = 0.288675 \times g\end{array} //]]>$, where $//<![CDATA[ \begin{array}{l}g\end{array} //]]>$ is the ATODGAIN (from the CCDTAB). The total noise ($//<![CDATA[ \begin{array}{l}TRN\end{array} //]]>$) associated with CCD readout noise (RN) and quantization noise is obtained by adding the two figures in quadrature: $//<![CDATA[ \begin{array}{l}TRN = \sqrt{(RN^2 + (0.288675\times g)^2)}\end{array} //]]>$.

4.1.2 Bias Subtraction (only relevant for CCD observations)

CCD bias frames are acquired daily for scientific calibration purposes and for monitoring detector performance. Every week multiple CCD bias frames are combined together into a reference superbias image. The combination removes the cosmic rays accumulated during the readout time and reduces the noise associated with the bias subtraction procedure to a level that is insignificant relative to the read noise associated with any single image.

The calstis calibration pipeline performs the bias correction in two steps (see Section 3.4): BLEVCORR subtracts the bias level from the overscan regions in two steps: it first measures the level and slope in part of the 19 pixel wide leading physical overscan region and then does the same in a 20 row wide virtual overscan region. After the BLEVCORR procedure, some structure remains in the bias frames. This structure includes real bias signal (“spurious charge”) and also some dark current accumulated during the time required to read out the detector. However, this structure is stable on timescales of a few weeks. To remove this structure a superbias frame with high signal-to-noise ratio is subtracted from each science image in the BIASCORR step.

This process has been found to be highly stable and reliable. Details can be found in STIS ISR 97-09 and STIS ISR 98-31.

4.1.3 Dark Current and Hot Pixels

CCD Dark Current and Hot Pixels

Similar to the case of CCD bias frames, CCD dark frames are acquired on a daily basis, and combined together every week for use in the calstis calibration pipeline. The creation procedure of the CCD dark reference files involves two main steps:

1. Every month, a high signal-to-noise superdark frame is created from a combination of typically 40-60 “long” darks (dark frames with exposure time > 900 s). For Side-2 electronics, the "long" darks are first scaled to a reference temperature. These monthly superdark frames are not actually delivered to the calibration data base, but used as “baseline” dark for the next steps.
2. All “long” darks taken during a given week are combined together (rejecting cosmic rays in the process), and normalized to intensity units of e¯/s. After that, the hot pixels in the monthly baseline dark (created as described above) are replaced by those of the normalized weekly superdark. These new ‘hot’ pixels have a value higher than “median dark current + 5σ of the normalized weekly superdark.” Previously hot pixels that have fully annealed out between the observing dates of the baseline and the new dark are assigned a value equal to that in a median filtered version of the baseline dark. The resulting dark has the high signal-to-noise ratio of the monthly baseline superdark, updated with the hot pixels of the current week. These “weekly darks” are the STIS CCD dark reference files used in the calstis pipeline (and are retrievable from the HST Archive).

We emphasize that the use of the correct dark reference file is important for most science applications. An incorrect dark reference file will likely introduce a poor dark correction. It will either leave too many hot pixels uncorrected and unflagged, or create many negative “holes” caused by the correction of hot pixels which were not actually hot in the science data (e.g., if the CCD was annealed in the meantime).

Finally, observers have the option to create “daily darks” for their CCD observations, i.e., CCD dark files for which the hot pixels have been optimized for any given date. These “daily darks” are not created within the automatic calstis pipeline, but observers have been provided with the Python script weekdark within the refstis package as well as detailed instructions to create such files by themselves. Details are described on the following web page: https://refstis.readthedocs.io/en/latest/api/weekdark.html.

In terms of long term trends of the CCD dark current rate, it is relevant to recall that the STIS CCD temperature was stabilized at -83 °C between its installation onto HST on 1997-Feb-14 and the failure of the Side-1 electronics on 2001-May-16. During the Side-1 era, the CCD median dark rate rose from 0.0013 to 0.0043 e¯/s/pixel. CCD temperature cannot be stabilized with the Side-2 electronics, so thermal fluctuations perturbed the dark rate during the Side-2 era. These fluctuations are discussed in STIS ISR 2001-03 and 2018-05 and are monitored through daily dark exposures: https://www.stsci.edu/~STIS/monitors/darks/dark_ccd.png.

The calstis pipeline uses EPC files to scale the CCD superdark according to temperature. When recalibrating locally with calstis, it is important to run the correction from within the directory containing the EPC files so they can be found.

MAMA Dark Current

NUV-MAMA Dark Current

Most of the dark current in the NUV-MAMA detector is caused by phosphorescence of impurities in the MgF₂ detector faceplate (Kimble 1997). A simple model of the phenomenon has been developed by Jenkins and Kimble (private communication) that envisions a population of impurity sites each having three levels: (1) a ground state, (2) an excited energy level which can decay immediately to the ground state, and (3) a metastable level that is at an energy slightly below the one that can emit radiation. The metastable state can be thermally excited to the upper level, and this excitation rate is proportional to exp(-E/kT), where E is the energy difference between the levels. The behavior of the count rate versus temperature leads to an estimate of 1.1 eV for E. At a fixed detector temperature of 30 °C, the time constant for the dark current to reach an equilibrium value is about 8 days. However, since the MAMA high voltage power supplies have to be shut down during orbits impacted by the South Atlantic Anomaly (SAA), the detector temperature varies from about 27 °C to 40 °C, and the dark current never actually reaches equilibrium.

MAMA temperatures cycle on a roughly daily time scale, being lowest just after the high voltage is turned on after an SAA passage. The dark current can be predicted with about 5% to 10% accuracy using the contemporaneous temperature of the NUV-MAMA charge amplifier recorded in the OM2CAT keyword in the _raw science file that is part of the standard data products. Originally the NUV-MAMA dark current was fit with the curve $//<![CDATA[ \begin{array}{l}darkrate = 9.012 \times 1020 \times \exp{(-12710/T)}\end{array} //]]>$. This worked well for the first two years of STIS operations, but as the mean time-averaged temperature of the detector increased with time, this formula started to predict too high a dark current. Once this was realized, a more flexible fitting formula was implemented in the calstis pipeline. This updated formula for the dark current was

$$//<![CDATA[ \begin{array}{l}\displaystyle darkrate = norm \times 1.805 \times 1020 \times \exp{\left[ -12211.8 / \max(T, Tmin) \right]},\end{array} //]]>$$

where both $//<![CDATA[ \begin{array}{l}norm\end{array} //]]>$ and $//<![CDATA[ \begin{array}{l}Tmin\end{array} //]]>$ are slowly varying functions of time that are empirically adjusted to give a good match to the observed dark rate, and which are tabulated in the temperature dependent dark correction table (_tdc) reference file. The uncertainty related to the application of the updated NUV-MAMA dark rate is 5% rms. Note however that this doesn’t translate to any significant photometric uncertainty, since any residue of the dark subtraction process gets subtracted out during the extraction of imaging photometry or spectroscopic flux.

Right after SM4, the dark count rate for NUV-MAMA was found to be as high as 0.0160 counts/pix/s, significantly higher than expected from the detector’s pre-SM4 behavior. However, over a time scale of a year, the count rate decreased exponentially with time down to a level of ~0.0035 counts/pix/s, until the STIS detectors were turned off on October 2010 due to issues with HST’s control and data handling electronics. When the NUV-MAMA was turned back on a month later on Nov 2010, the dark rate was measured to be at an increased level of 0.006 counts/pix/s. As before, the dark rate followed an exponential decrease with time eventually reaching a median rate of ~0.0015 counts/pix/s and staying at that level for several years. In STIS ISR 2013-01, the dark rate formula has been updated for post-SM4 data to include both the short-term variation due to detector temperature and the long-term variation due to the exponential decay with time. Details of this formula can be found in Section 3.4.7 and in STIS ISR 2013-01.

HST entered safe mode on October 5, 2018 due to a gyro failure, and came back online on Oct 27, 2018. After this, the STIS NUV-MAMA dark count rate was measured to be as high as 0.0038 counts/pix/s, but it quickly declined back to normal levels. As of January 2019, the dark count rate is at ~0.0020 counts/pix/s. The STIS team has updated the parameters in the TDCTAB for observations obtained after October 27, 2018. The long-term time variation of NUV-MAMA dark count rate can be found at: https://www.stsci.edu/~STIS/monitors/nuv/dark_nuv.png. 

FUV-MAMA Dark Current

The FUV-MAMA dark current is very low compared to that of the NUV-MAMA. In the early years of STIS operations, dark current values as low as 7 counts/s integrated over the whole detector were routinely encountered. However, there is also an intermittent glow that covers a large fraction of the detector (see Figure 4.2). The source of the dark current is not phosphorescence but is intrinsic to the micro-channel plate array. This glow can substantially increase the dark current over a large fraction of the detector, and this leads to count rates of up to 100 counts/s integrated across the face of the detector (which is of course still very low). During the first two years of STIS operations, this glow was only present intermittently, but since mid-1999 it showed up more often than not. This extra glow appears to be a function of both the detector temperature, and the length of time the detector has been on. (The STIS MAMA detectors were turned off daily during HST’s passages through the South Atlantic Anomaly.) However, the strength and frequency with which this glow appears increased throughout the lifetime of STIS. At later epochs, only the first non-SAA-impacted orbit each day could be reliably expected to be free of the extra glow.

Examples of the FUV-MAMA dark current variation across the detector can be seen in Figure 4.2, which are the sums of a number of 1300-1380 second dark frames taken during periods of high dark current in 2010 (left) and in 2017 (right). The dark current in the lower right quadrant varies with time and temperature. The total dark current can be approximated by the sum of a constant dark current plus a “glow” image, scaled to the net rate in the upper left quadrant. No reliable method has been found to predict the brightness of the glow with sufficient accuracy to subtract it from pipeline processed FUV-MAMA images, and so FUV-MAMA dark images provided by archive contain only the base dark current plus the mean values for any pixels flagged as hot pixels. For the FUV-MAMA, hot pixels are defined to be those that show an average dark current of more than 1 × 10^-4 counts/hi-res-pixel/s as measured over several months of dark monitor observations. In 1997, only 97 out of the more than 4 million pixels had a dark rate of more than 1 × 10^-4 counts/s. By the time of the Side-2 failure, more than 3,000 pixels were at this level; by 2018, over 20,000 pixels were at this level.

Subtracting the extra glow requires identifying a region on the detector where the glow is strong while the intrinsic target flux is weak, and then using this region to scale and subtract a smoothed image of the glow. For long slit spectroscopic observations, the region of the detector occulted by one of the aperture bars may sometimes be useful for this purpose. Mean glow images for different epochs can be obtained upon request by contacting the help desk and can be used for off-line data reduction if appropriate.

However, it is good to keep in mind that the FUV-MAMA dark current is so low that a typical STIS FUV-MAMA observation will have less than one count per pixel from the dark, so that many science projects will not require special data reduction efforts in this respect. Also, various standard measures of background (the median for example) are not good estimates of the dark when the data are quantized into just a few values. The best way to estimate the background is to identify hot pixels using the dark reference files, and then use an unclipped mean for the remaining pixels in a source-free region of the image.

4.1.4 Flat Fields

The flat field reference files currently in use by the calstis calibration pipeline were derived by several different methods, depending on the detector, the observing mode (imaging vs. spectroscopy), and (for some modes) the optical elements used. For many observing modes, there are two types of flat fields involved in the calibration: p-flats (the *_pfl.fits reference files) which correct for the pixel-to-pixel sensitivity variations, and l-flats (*_lfl.fits) which model the low-frequency variations in sensitivity over the field of view of the observing mode in question. Important features regarding the accuracy of the flat fields for different observing modes are described in the following sections.

CCD Imaging Flat Fields

• p-flat: The p-flat reference file for CCD imaging was created from several exposures of the internal tungsten lamp during ground calibration. The total accumulated intensity was at least 100,000 electrons/pixel, except for the borders of the image which are partially vignetted by the mode select mechanism (MSM). After image combination, the lamp illumination function was removed through division by a median-filtered version of the image (using a 55 × 55 pixel kernel). The vignetted borders of the image were flagged during this procedure (note that due to the intrinsic non-repeatability of the MSM, the exact vignetting of the borders of the image is different every time the MSM is moved to the CCD imaging mode). The structure of the CCD p-flat is shown in Figure 4.3. The most obvious features are circular dark patterns with typical diameters of ~20 pixels. These artifacts are shadows of dust on the CCD window, often called "dust motes". They are removed by the p-flat to well within 1%. On-orbit testing of CCD imaging p-flat images has shown that the p-flat is stable to within about 0.2% after correction for Poisson noise.
  
  
• l-flat: The low-order flat-field reference file for CCD imaging was created from numerous images taken during the Hubble Deep Field South (HDF-S) campaign in October 1998. After bias subtraction and dark subtraction using dedicated high-signal-to-noise combinations of bias and dark files, the images were masked for cosmic rays and heavily masked around all sources detected in the individual images. The combined image was then fit by a 5-piece cubic spline function. The main purpose of this l-flat is a correction of the 5-10% sensitivity roll off near the edges of the 50CCD aperture which remains after application of the CCD imaging p-flat described above.

CCD Spectroscopic Flat Fields

• p-flats: P-flat reference files for the CCD spectroscopic modes were originally created from high S/N spectra of deuterium and tungsten lamps during ground calibration of STIS. They were first updated by in-flight data on 1999-Sep-20 (and several updates followed afterwards). Flat field data were taken in flight on a regular schedule and in every supported observation mode in order to enable one to identify any dependence on in-flight epoch and/or the grating used. Details of the flat field file creation procedure and analysis are published in STIS ISR 99-06. The main results of this analysis were: 
  (a) The pixel-to-pixel flat field for the CCD has been found to be independent (to within the limits set by Poisson statistics) of wavelength, CCD gain setting, and signal level. 
  (b) The structure of the L-mode flat fields is different from that of the M modes, due to the different optical magnification of the two sets of modes. Hence, flat fields have been derived separately for L- and M-modes. 
  (c) A time dependence of the structure of the flat field was identified. The systematic (non-Poisson) effect is of order 0.3% per year. To keep this effect within 1%, spectroscopic CCD flat fields have been delivered for all spectroscopic CCD modes with USEAFTER dates of April 1997, July 1999, August 2000, and October 2001.
  
  
• l-flats: To probe the sensitivity as a two-dimensional function of wavelength and position along the slit, a number of calibration programs were performed in which a bright star was dithered up and down the length of the 52X2 slit. In addition, CCD sensitivity monitor measurements made after April 2002 routinely included measurements at both the central and E1 aperture positions on the slit (see Section 7.2.7 of the STIS Instrument Handbook). The ratio of sensitivities at the E1 position to the central row is therefore very well measured, while more limited data is available at other positions along the slit. The latter measurements feature 1 to 4 arcsec spacing along the cross-dispersion direction. 
  
  L-flats were produced for the G140L, G230LB, G430L, and G750L gratings. The medium-resolution gratings lack sufficient data at most central wavelengths, while the G230L mode was found not to require a low-order flat field correction. The l-flats are defined to be unity at the row where the absolute sensitivity calibration is determined. The typical effect of applying the l-flats is of order 1-2% in sensitivity along the dispersion (depending on the grating in question) and 2-3% perpendicular to the dispersion. It should be kept in mind that the l-flat corrections are only strictly applicable to targets positioned along the center line of the slits. Targets that are significantly offset in the dispersion direction will not be corrected with the same degree of accuracy, especially near the extremes of the wavelength coverage of the grating used. More details are available in Proffitt (2006).

MAMA Flat Fields

• p-flats: The MAMA flat fields are dominated by a fixed pattern that is a combination of several effects including "beating" between the micro-channel plate array and the anode pixel array and variations in the charge cloud structure at the anode. For this reason, the flat field correction for MAMA data is applied to the data in unbinned format (2048 × 2048 pixel²). tiIntrinsic pixel-to-pixel variations are 3.9% rms for the FUV-MAMA and 2.8% rms for the NUV-MAMA, respectively, in the format delivered by the calstis pipeline ("low-resolution pixels", 1024 × 1024 pixel²). 
  
  The creation of high-signal-to-noise-ratio p-flats with the STIS MAMA detectors is done using the internal Krypton (for the FUV-MAMA) and Deuterium (for the NUV-MAMA) lamps. It is a rather long and tedious process, since individual exposures have to adhere to the global count rate linearity limit for MAMA detectors^[1] One also has to take exposures at several slit locations in order to illuminate regions of the detector that are normally occulted by the slit fiducial bars (see STIS Instrument Handbook, Chapter 13.2.6) and other scheduling limitations, it takes several HST cycles to achieve this. 
  
  The current MAMA p-flat reference files have been in use by the calstis pipeline since 2002-Nov-20, and have a signal-to-noise ratio of ~200 per pixel. A detailed analysis of the quality of the MAMA p-flats using well-illuminated spectra of flux standard stars (white dwarfs) shows that the rms of extracted spectra (summing 11 MAMA rows) is of order 1%. This is larger than the rms of the p-flats due to Poisson statistics, and it represents the current limit of the quality of flux calibration of individual targets from individual exposures (imaging or spectra). STIS users whose science requires higher signal-to-noise ratios have been directed to use multiple exposures using FP-SPLIT slits or spatial dithering along the slit (see STIS Instrument Handbook, Chapter 12.5).
  
  
• Spectroscopic l-flats: The need for l-flats was considered for the low-resolution spectral modes G140L and G230L, as described in detail above for the CCD spectral modes. It was eventually determined that an l-flat correction was only required for the G140L mode.
  
  
• Imaging l-flat: MAMA imaging l-flats were determined from observations of the Galactic globular cluster NGC 6681 obtained at a large number of offset pointing positions (144 for the NUV-MAMA, 207 for the FUV-MAMA) using the "CLEAR" (25MAMA) filter. The l-flat consists of a two-dimensional, second-order polynomial surface fit to the ratios of large-aperture fluxes obtained for the same stars, normalized to unity in the inner 400 × 400 pixel² region. The results showed that an imaging l-flat is only needed for the FUV-MAMA, which features a significant ~22% peak-to-peak variation in sensitivity over the field of view (mainly due to variations towards the upper right corner and near the left-hand edge of the detector). The l-flat corrects this to 2% rms. In contrast, the sensitivity variations of the NUV-MAMA stay below 1% rms even without the l-flat correction.

---

 ¹200,000 counts/s; given the lamp brightness, this can only be achieved with the G140M and G230M gratings and a narrow slit. Hence these p-flats are applied to both imaging and spectroscopic MAMA data.