3.5.1 Requirements for Manual Recalibration

Retrieving Software and Input Data Files

Data retrieved from MAST (Barbara A. Mikulski Archive for Space Telescopes), via the Portal or the search form, will have been processed with the most up-to-date calibration reference files and software. Nevertheless, some users may wish to reprocess existing WFC3 observations with e.g. alternative reference files.  To do so, the following must be available on their system:

• Software for data processing (distributed in the stenv package)
• Reference files may be obtained from CRDS, as described in Section 3.1.4.
• Uncalibrated (raw) data files from MAST
• Any association tables that describe observation sets.

Setting the Path to Reference Files

Before any recalibration can be done, the user must define a local directory containing the calibration reference files. The directory for WFC3 is given the environment variable name ‘iref’. The raw image headers will already contain the appropriate keywords, 'iref’ variable, and the reference file names that were assigned during STScI pipeline processing. The user must simply define the ‘iref’ directory as the local directory in their Unix environment:

csh shells: setenv iref /mydisk/myiref/
sh  shells: export iref /mydisk/myiref/

If executed from the command line, this setup must be done in the same window in which Python will be started. (Setting ‘iref’ from within Python will not work, even though subsequently typing ‘show iref’ would suggest it might.) For convenience, this setup command can be added to the '.setenv' file for csh or ‘.bash_profile/.bashrc’ file for sh, so that the 'iref' environment variable will always be defined when you start a new terminal. The ‘iref’ environment variable is required whether reprocessing through Python or from the OS command line.

An alternate method of accessing WFC3 reference files is shown in a new Python tutorial in Section 3.5.2. In this example, the best WFC3 reference files for a given set of input images are retrieved directly from CRDS and placed in a local 'crds_cache/' directory.  The notebook shows how to update the header keywords to point to these new reference files and sets the 'iref' environment variable for subsequent calwf3 processing.

Selecting Calibration Switches

The MAST HST Data Processing uses the most up-to-date calibration reference files by default. In order to use non-default reference files, manual recalibration is required and, in this case, the calibration reference file keywords will need to be updated manually in the primary header of the raw data files with the desired file names before running calwf3. In addition, the user can choose to change which calibration steps are performed by calwf3 by resetting the values of the calibration switch keywords. These keywords are listed in Table 3.7 along with their default values as used in the STScI pipeline. To change the values of any of the keyword switches, a FITS keyword editor such as the Python package astropy.io.fits may be used.

from astropy.io import fits
fits.setval(‘path/to/myfile_raw.fits’, keyword= ‘DARKCORR’, value= ‘OMIT’, ext=0)

Table 3.7: Calibration switch and default settings.

FLUXCORR

3.5.2 calwf3 Examples   

Documentation for the calwf3 software is provided at readthedocs, with version history available on the STScI github repository or readthedocs software history. Users may reprocess either a single RAW image, an image association, or a list of raw files in Python:

A single RAW exposure       

from wfc3tools import calwf3
calwf3(‘ibohbfb9q_raw.fits’)

An image association (ASN)

calwf3(‘ibohbf040_asn.fits)

A list of RAW files (not part of an ASN)

import glob
for raws in glob.glob('ibohbf*_raw.fits'):
    calwf3(raws)

A new WFC3 notebook tutorial provides calwf3 reprocessing examples to improve calibrated WFC3/IR images affected by time-variable background. The notebook shows how to diagnose images with poor-quality ramp fits and rerun calwf3 with the 'CRCORR' step turned off.

Time-variable background, described in detail in Section 7.10, can be caused by either scattered Earth shine and/or helium I emission in the Earth's atmosphere at 1.083 microns (WFC3 ISR 2014-03). The latter effect can appear in F105W and F110W images as a flat excess signal which is added to the total background for a subset of reads. When strong enough, this non-linear signal may compromise the ramp fitting performed by calwf3, which is designed to flag and remove cosmic rays and saturated reads. The affected calibrated FLT data products will have much larger noise and a distinctly non-gaussian background. In the tutorials described in Section 7.10, the images are reprocessed using the 'Last-minus-first' technique described in WFC3 ISR 2016-16.  calwf3's ramp fitting step (CRCORR) is turned off and treats the IR detector like a CCD that accumulates charge and is read out only at the end of the exposure. In this case, the observed count rate is determined by simply subtracting the first from the last read of the detector and dividing by the time elapsed between the two reads.

Note: while time-variable background also impacts the IR grisms, the methods used for imaging data should not be used to correct G102 and G141 observations, which are affected by a combination of zodiacal background, helium line emission, and scattered Earth light, each of which vary spatially across the detector. More detail on background corrections for grism data is provided in WFC3 ISR 2020-04.