3.2 UVIS Data Calibration Steps

As of calwf3 v3.3, the pipeline processes all UVIS data twice, once with the CTE correction applied as the first step, and a second time without the CTE correction. Figure 3.1 shows a schematic representation of all the UVIS calibration steps, which are also briefly summarized below, in the order they are performed, with the corresponding calibration switch keyword in parenthesis:

• Calculate and correct for degradation in charge transfer efficiency (CTE) in the image (PCTECORR), for UVIS only
• Calculate and record a noise model for each pixel (NOISCORR) in the error array, ERR, of the header and initialize the data quality, DQ, array of the image (DQICORR). Note the (NOISCORR) is a calibration switch inaccessible to the user i.e. it is not listed explicitly in the image header and the correction is always performed,
• Correct for Analog-to-Digital conversion errors, currently skipped (ATODCORR)
• Subtract bias level determined from overscan regions (BLEVCORR)
• Subtract the bias image (BIASCORR)
• Propagate flagged pixels (DQICORR) from bad pixel table and from SINK pixel mask into the DQ mask
• Subtract the post-flash image if applicable (FLSHCORR), for UVIS only
• Scale and subtract the dark image (DARKCORR)
• Perform flat fielding (FLATCORR)
• Perform shutter-shading correction where necessary, currently skipped (SHADCORR), for UVIS only
• Populate photometric header keywords (PHOTCORR)
• Correct chips to use the same zero point (FLUXCORR)
• Calculate basic pixel statistics for the image and store them in relevant header keywords (No switch).

3.2.1 Correction For Charge Transfer Efficiency

• Header switch: PCTECORR
• Reference file: PCTETAB

The charge transfer efficiency (CTE) of the UVIS detector has been declining over time as on-orbit radiation damage creates charge traps in the CCDs. During the charge transfer in the CCD readout process, these traps capture photo-electrons generated in other “upstream” pixels, thus lowering the detected charge. Faint sources in particular can suffer large fractional flux losses or even be lost entirely if observations are not planned and analyzed carefully. The CTE depends on the morphology of the source, the distribution of electrons in the field of view, and the population of charge traps in the detector column between the source and the transfer register. Further details of the current understanding of the state of the WFC3/UVIS CTE are presented in Chapter 6, as well as on the WFC3 CTE web page. The PCTECORR step aims to mitigate the flux loss incurred from CTE.

More information on this part of the pipeline can be found in Section 3.4.1, which details the functioning of the wf3cte routine.

3.2.2 Error Array Initialization

• Header switch: NOISCORR (not listed explicitly in image header, see text)
• Reference file: CCDTAB

In this step, the image error array is initialized. The NOISCORR switch is not user-accessible and always set to PERFORM. The function examines the ERR extension of the input data to determine the state of the array. The input _raw image contains an empty ERR array. If the ERR array has already been expanded and contains values other than zero, then this function does nothing. Otherwise it will initialize the ERR array by assigning pixel values based on a simple noise model. The noise model uses the science (SCI) array and for each pixel calculates the error value σ in units of DN:

σCCD =  

The CCDTAB reference file contains the bias, gain and readnoise values used for each CCD amplifier quadrant used in this calculation. The table contains one row for each configuration that can be used during readout, which is uniquely identified by the list of amplifiers (replicated in the CCDAMP header keyword), the particular chip being read out (CCDCHIP), the commanded gain (CCDGAIN), the commanded bias offset level (CCDOFST) and the pixel bin size (BINAXIS). These commanded values are used to find the table row that matches the characteristics of the image that is being processed and reads each amplifiers properties, including readnoise (READNSE), A-to-D gain (ATODGN) and the mean bias level (CCDBIAS).

3.2.3 Data Quality Array Initialization

• Header switch: DQICORR
• Reference files: CCDTAB, BPIXTAB,SNKCFILE

This step populates the data quality array by reading a table of known bad pixels for the detector, as stored in the Bad Pixel reference table, as well as the sink pixel mask (BPIXTAB and SNKCFILE, respectively). Descriptions of the bad pixels that can be flagged are indicated in Table 3.2

Table 3.2: Data quality array flags for UVIS files

Note: The CTETAIL (32) and CROSSTALK (16384) flags are not currently used.
 
The DQ array may already have been populated with some values to flag pixels affected by telemetry problems during downlink. Other DQ values will only be marked during further processing (such as cosmic-ray rejection). This function also checks pixel values in the SCI extension for full-well saturation, using the value of the SATURATE column in the CCD parameters table (CCDTAB). Any SCI array pixel value that is greater than the SATURATE value will be assigned the appropriate flag value in the DQ array (256). This function also checks for SCI array pixel values that have reached the limit of the detector’s 16-bit A-to-D converters, flagging any pixel with a value > 65534 DN with the ‘A-to-D saturation’ DQ value (2048), A-to-D saturated pixels are also flagged as full-well saturated, so their DQ is actually updated (bitwise) by 256+2048.

Even full-well saturated pixels (but not A-to-D saturated) could still be used for aperture photometry, since charge keeps being created in saturated pixels but it’s no longer retained in the original pixel and spills over to the neighboring ones (bleeding, see also WFC3 ISR 2010-10). Users should be aware that the apertures in this case should be defined as to include all the bled charge. Users can check Section 5.4.5 and Section 5.4.6 of the WFC3 Instrument Handbook for further details on full-well and A-to-D saturation.

DQICORRcombines the DQ flags from preprocessing, BPIXTAB, SNKCFILE, and saturation tests into a single result for the particular observation. These values are combined using a bit-wise logical OR operation for each pixel. Thus, if a single pixel is affected by two DQ flags, those flag values will be added in the final DQ array. This array then becomes a mask of all pixels that had some problem coming into the calibrations, so that the calibration processing steps can ignore bad pixels during processing. The BPIXTAB reference file maintains a record of the x, y position and DQ value for all known bad pixels in each CCD chip for a given time period.

3.2.4 Correct for Analog-to-Digital Conversion Errors

• Header Switch: ATODCORR
• Reference File: ATODTAB

An analog-to-digital conversion correction is applied if the CCD electronic circuitry, which performs the analog-to-digital conversion, is biased toward the assignment of certain DN values. WFC3 ground test results showed that this correction is not currently needed, so the ATODCORR switch is currently always set to ‘OMIT’ so that this function is not performed.

3.2.5 Overscan Bias Correction

• Header Switch: BLEVCORR
• Reference File: OSCNTAB

The overscan regions are used to monitor the instrument as well as provide a measure of the bias level at the time the detector was exposed (see Figure 3.2). BLEVCORR fits the bias level in the CCD overscan regions and subtracts it from the image data. The boundaries of the overscan regions are taken from the OSCNTAB reference file. With these regions defined, the serial and parallel virtual overscans are analyzed to produce a two-dimensional linear fit to the bias level. The overscan level for each row of the input image is measured within the serial virtual overscan region, utilizing sigma-clipping to reject anomalous values (e.g., cosmic-ray hits that occur in the overscan) and a straight line is fit as a function of image line number. The same procedure is followed for the parallel overscan, resulting in a straight line fit as a function of image column number. The parallel fit is computed in the form of a correction to be added to the serial fit result, in order to remove any gradient that may exist along the x-axis direction of the image. The serial fit and the parallel correction to it are then evaluated at the coordinates of each pixel and the computed bias value is subtracted from the pixel. This is done independently for each region of the image that was read out by one of the four CCD amplifiers. The mean bias value determined for each of the amplifier quadrants is recorded in the primary header keywords BIASLEV[ABCD] and the overall mean bias value is computed and written to the output SCI extension header as MEANBLEV.

The full bias level-subtracted image is retained in memory until the completion of all the processing steps in wf3ccd. The overscan regions will not be trimmed until the image is written to disk at the completion of wf3ccd.

The OSCNTAB reference file (Overscan Region Table) describes the overscan regions for each chip along with the regions to be used for determining the actual bias level of the observation. Each row corresponds to a specific configuration, given by the gain, amplifier used in readout, charge injection, and binning adopted. The OSCNTAB columns BIASSECTAn and BIASSECTBn give the range of image columns to be used for determining the bias level in the leading and trailing regions, respectively, of the serial physical overscan regions, while columns BIASSECTCn and BIASSECTDn give the range of columns to be used for determining the bias level from the serial virtual overscan regions. The parallel virtual overscan regions are defined in the OSCNTAB in the VXn and VYn columns. To determine which overscan regions were actually used for measuring the bias level, check the OSCNTAB reference file. Users may modify the overscan region definitions in the reference table for manual calibration, but the TRIMXn and TRIMYn values must not be changed.

3.2.6 Bias Structure Correction

• Header Switch: BIASCORR
• Reference File: BIASFILE

This step subtracts the two dimensional bias structure from the image using the superbias reference image listed in the header keyword BIASFILE. The dimensions of the image are used to distinguish between full and sub-array images. The BIASFILE has the same dimensions as a full-size science image, complete with overscan regions. Only after completion of wf3ccd are the science images trimmed to their final calibrated size. The same reference image is used for full-frame and subarray images, calwf3 will extract the matching region from the full-size bias file and apply it to the subarray image.

3.2.7 Sink Pixel Detection and Marking

• Header Switch: DQICORR
• Reference File: SNKCFILE

Sink pixels are a detector defect. These pixels contain a number of charge traps and under-report the number of electrons that were generated in them during an exposure. These pixels can have an impact on nearby upstream or downstream pixels, though they often only impact one or two pixels when the background is high, they can impact up to 10 pixels if the background is low (see Section 6.7 for more information about sink pixels detection and flagging).

Flagging of sink pixels in the DQ extension of calibrated images, controlled with the DQICORR header keyword, happens after the bias correction has been performed. When set to PERFORM, the sink pixels marked in the SNKCFILE reference image are flagged in the science image DQ. Given the SNKCFILE reference image, the procedure for flagging the sink pixel in science data involves:

• Extraction of the MJD of the science exposure
• Examination of the reference image, pixel by pixel, to find values greater than 999, pixels indicating a sink pixels. The value of the reference file pixel corresponds to the date (in MJD format) at which this pixel started exhibiting sink pixel behavior.
• If the turn-on date of the sink pixel is after the exposure date of the science image, then we ignore the sink pixel in this exposure and move on to the next pixel.
• If the turn-on date of the sink pixel is before the exposure date of the science image, then this science pixel was compromised at the time of the exposure. The corresponding pixel in the DQ extension of the science image is flagged with the "charge trap" value of 1024.
• If the pixel “downstream” of the sink pixel, along the readout direction, has a value of -1 in the SNKCFILE reference image, then it is also flagged with the “charge trap” value in the DQ extension. We then proceed vertically “upstream” from the sink pixel and compare each pixel in the reference file to the value of the sink pixel in the science exposure at hand. The reference file contains –at the positions of the upstream pixels- estimates of the minimum charge a sink pixel has to contain, in order not to affect the nearby upstream pixel. If the value of the sink pixel in the exposure is below the value of the upstream pixel in the reference image, we flag that upstream pixel with the “charge trap” value in the DQ extension. We continue to flag upstream pixels until the value of the pixel in the reference image is zero or until the value of the sink pixel in the exposure is greater than the value of the upstream pixel in the reference image.

When originally introduced in calwf3 version 3.3, the correction for sink pixels was only made available for images taken in full-frame mode. From calwf3 version 3.4 onwards, sink pixels are being flagged in images taken in both full-frame and subarray observing modes.

3.2.8 Post-flash Correction

• Header Switch: FLSHCORR
• Reference File: FLSHFILE

WFC3 has post-flash capability to provide a means of mitigating the effects of Charge Transfer Efficiency (CTE) degradation (see WFC3 ISR 2013-12). When FLSHCORR=PERFORM, this routine subtracts the post-flash reference image, FLSHFILE, from the science image. This file has the same dimensions as a full-size science image complete with overscan regions. FLSHFILE has units of electrons per second. The appropriate FLSHFILE has matching values of the following keywords from the image header: DETECTOR, CCDAMP, CCDGAIN, FLASHCUR, BINAXISi, and SHUTRPOS.

The success of the post-flash operation during the exposure is first verified by checking the keyword FLASHSTA (ABORTED, SUCCESSFUL, NOT PERFORMED). calwf3 selects the reference file which matches the science image’s binning, shutter position (SHUTRPOS, either A or B, i.e. the blade used reflect the postflash LED's light into the optical path to illuminate the detector) and flash current level settings (LOW, MED, HIGH, recorded in the FLASHCUR keyword), and scales it by the flash duration (stored in the FLASHDUR keyword). This scaled postflash image is then subtracted from the science image. The mean value of the scaled post-flash image is assigned to the MEANFLSH keyword in the output SCI extension header. As the blade used for post-flash alternates between exposures, different members of an association can have different values of SHUTRPOS(shutter blade used to ). This does not constitute a problem for calibration because the reference files are populated separately for each exposure.

3.2.9 Dark Current Subtraction

• Header Switch: DARKCORR
• Reference File: DARKFILE, DRKCFILE

The dark current correction step subtracts the estimate of the dark current from the science image. The reference file listed under the DARKFILE header keyword is used in the non-CTE corrected UVIS pipeline branch, DRKCFILE is instead used in the CTE-corrected branch. The dark image (in units of electrons/sec) is multiplied by the exposure time, and subtracted from the input image.

The mean dark value is computed from the scaled dark image and used to update the MEANDARK keyword in the SCI image header. The dark reference file is updated frequently to allow the tracking of hot pixels over time. The time by which the dark reference file gets multiplied is simply the exposure time in the science image; it does not include the idle time since the last flushing of the chip or the readout time. Any dark accumulation during readout time is included automatically in the BIASFILE.

The reference file for dark subtraction, DARKFILE (DRKCFILE), is selected based on the values of the keywords DETECTOR, CCDAMP,CHINJECT and BINAXISi in the image header. The dark correction is applied after the overscan regions are trimmed from the input science image. In a similar fashion to the BIASCORR step, calwf3 requires the binning factors of the DARKFILE (DRKCFILE) and science image to match. Sub-array UVIS science images use the same reference file as a full-sized DARKFILE (DRKCFILE), calwf3 simply extracts the appropriate region from the reference file and applies it to the sub-array input image.

3.2.10 Flat-field Correction

• Header Switch: FLATCORR
• Reference Files: PFLTFILE, LFLTFILE, DFLTFILE

This routine corrects for pixel-to-pixel and large-scale sensitivity variations across the detector by dividing the overscan-trimmed and dark-subtracted science image by a flat-field image.

Because of geometric distortion effects, the area of the sky seen by different pixels is not constant and therefore observations of a constant surface brightness object will have counts per pixel that vary over the detector, even if every pixel were to have the same intrinsic sensitivity. In order to produce images that appear uniform for uniform illumination, the same counts per pixel variation across the field is left in place in the flat-field images, so that when a science image is divided by the flat it makes an implicit correction for the distortion effects on photometry. A consequence of this procedure is that two point-source objects of equal brightness will not have the same total counts after the flat-fielding step, thus point source photometry requires the application of a pixel area map (PAM) correction.

The PAM correction is automatically included in pipeline processing by Astrodrizzle, which uses the geometric distortion solution to correct all pixels to equal areas. Photometry is therefore correct for both point and extended sources in drizzled (drz, drc) images, but incorrect for point sources for flt or flc images.

Up to three separate flat-field reference files can be used: the pixel-to-pixel flat-field file (PFLTFILE), the low-order flat-field file (LFLTFILE), and the delta flat-field file (DFLTFILE). The most recent information about the various UVIS flats currently in the pipeline can be found in  ISR 2016-05 and ISR 2016-04. The PFLTFILE is a pixel-to-pixel flat-field correction file containing the small-scale flat-field variations. Unlike the other flat fields, the PFLTFILE is always used in the calibration pipeline. The LFLTFILE is a low-order flat that corrects for any large-scale sensitivity variations across each detector. This file can be stored as a binned image, which is then expanded when being applied by calwf3. Finally, the DFLTFILE is a delta-flat containing any needed changes to the small-scale PFLTFILE. If the LFLTFILE and DFLTFILE are not specified in the SCI header, only the PFLTFILE is used for the flat-field correction. If two or more reference files are specified, they are multiplied together to form a combined flat-field correction image. Currently the standard UVIS calwf3 image processing uses a single flat field image, indicated by the PFLTFILE keyword. This image, however, does not only contain pixel-to-pixel variations but includes low order polynomial corrections as well. All flat-field reference images must have detector, amplifier, filter, and binning modes that match the observation. A UVIS sub-array science image uses the same reference file as a full-size image; calwf3 extracts the appropriate region from the reference file and applies it to the sub-array input image.

3.2.11 Shutter Shading Correction

• Header Switch: SHADCORR
• Reference Files: SHADFILE

This step corrects the science image for differential exposure time across the detector caused by the amount of time it takes for the shutter to completely open and close, which is a potentially significant effect only for images with very short exposure times (less than ~5 seconds). Pixels are corrected based on the exposure time using the relation:

corrected=uncorrected×EXPTIME÷(EXPTIME+SHADFILE)

WFC3 tests have shown that the shutter shading effect is insignificant (< 1%), even for the shortest allowed UVIS exposure time of 0.5 seconds (see WFC3 ISR 2007-17). Therefore this step is set to OMIT in calwf3 default settings.

3.2.12 Photometry Keywords Calculation

• Header Switch: PHOTCORR
• Reference Files: IMPHTTAB

In order to extract calibrated magnitudes (or equivalently fluxes), directly from wfc3 images, a transformation to absolute flux units is necessary. Users who do not wish to use this feature should set the header keyword PHOTCORR to OMIT. However, users that intend to use the FLUXCORR step (see Section 3.2.13), must also set PHOTCORR to PERFORM as well.

The PHOTCORR step is performed using tables of precomputed values instead of calling SYNPHOT to compute keyword values on-the-fly as was done prior to November 2013. Calwf3 version 3.1.6 and greater uses an Image Photometry reference table (IMPHTTAB); each WFC3 detector (IR or UVIS) uses a different photometry table. The PHOTMODE keyword string is reflects the observing configuration for the exposure (e.g., ‘WFC3, UVIS1, F814W’). Calwf3 then uses the PHOTMODE to retrieve the keyword values from the appropriate row in the reference table and update the science data headers with the keywords listed below.

• PHOTFNU: the inverse sensitivity in units of: Jansky sec electron ⁻¹
• PHOTFLAM: the inverse sensitivity in units of ergs cm⁻² A⁻¹ electron⁻¹
• PHOTPLAM: the bandpass pivot wavelength in Å
• PHOTBW: the bandpass RMS width in Å
• PHTFLAM1: the inverse sensitivity in units of ergs cm⁻² A⁻¹ electron⁻¹ for infinite aperture (chip1)
• PHTFLAM2: the inverse sensitivity in units of ergs cm⁻² A⁻¹ electron⁻¹ for infinite aperture (chip2)

For calwf3 version 3.3 and beyond, the value PHOTFNU is calculated for each UVIS chip (see also Section 3.2.13). The SCI headers for each chip contain the PHOTFNU keyword, which is valid for its respective chip, and is calculated as:

For UVIS 1:
photfnu = 3.33564e⁴ * PHTFLAM1*PHOTPLAM²

For UVIS 2:
photfnu = 3.33564e⁴ * PHTFLAM2*PHOTPLAM²
 
The IMPHTTAB file format for WFC3 UVIS is as follows:

EXT#        FITSNAME          FILENAME        EXTVE         DIMENS         BITPI OBJECT
0           z7n21066i_imp z7n21066i_imp.fits                                   16
1           BINTABLE          PHOTFLAM        1             5Fx256R
2           BINTABLE          PHOTPLAM        1             5Fx256R
3           BINTABLE          PHOTBW          1             5Fx256R
4           BINTABLE          PHTFLAM1        1             5Fx256R
5           BINTABLE          PHTFLAM2        1             5Fx256R

where each extension contains the photometry keyword information for that specific header keyword. The rows inside the tables are split on observation mode.

3.2.13 Flux normalization for UVIS1 and UVIS2

• Header Switch: FLUXCORR
• Reference Files: None

The FLUXCORR step was implemented in calwf3 v3.3 on February 23, 2016 as part of the new UVIS chip-dependent photometric calibration ( WFC3 ISR 2016-01). This step multiplies the UVIS2 image (SCI, 1 in the data file) by the ratio of inverse sensitivities PHTFLAM2/PHTFLAM1 so that the same flux correction can be used for both chips. This ratio is stored for reference in the image header keyword PHTRATIO. FLUXCORR is set to PERFORM by default during calwf3 processing, and in this case the keyword PHOTFLAM will be valid for both chips after the correction is applied. If users do not wish to perform the correction, the FLUXCORR keyword may be set to OMIT and the raw data reprocessed through calwf3. In this case, the keywords PHTFLAM1 and PHTFLAM2 may be used to convert to flux units in the respective chips.

Chip-dependent flat fields (from 2016 or later) must be used with calwf3 v3.3 onwards and should not be used with older versions of the pipeline, and vice versa, otherwise the data will be scaled incorrectly. 

3.2.14 Cosmic-ray Rejection

• Header Switch: CRCORR
• Reference Files: CCREJTAB

Associations with more than one member taken via CR-SPLIT or REPEAT-OBS will be combined using wf3rej (see Section 3.4.5 for more details). The task uses the same statistical detection algorithm developed for ACS (acsrej), STIS (ocrrj) and WFPC2(crrej), providing a well-tested and robust procedure. For all associations (including dithered observations), the DRZ and DRC products are created by Astrodrizzle which performs both cosmic ray detection (in addition to wf3rej, for CR-SPLIT or REPEAT-OBS observations) and corrects for geometric distortion.

3.2.15 UVIS Image Statistics Calculation

• Header Switch: None
• Reference File: None

This routine computes several statistics for data values that are flagged as ‘good’ in the data quality array. These quantities are updated in the SCI image header: the minimum (GOODMIN), mean (GOODMEAN), and maximum (GOODMAX) values, as well as the minimum (SNRMIN), mean (SNRMEAN), and maximum (SNRMAX) signal-to-noise ratio (the ratio of the SCI and ERR pixel values). The number of good pixel is recorded in NGOODPIX. The minimum, mean, and maximum statistics are also computed for the ERR array.