3.5.1 Requirements for Manual Recalibration

Retrieving Software and Input Data Files

If observers decide to recalibrate their WFC3 data on their own, the following must be available on their system:

• Software currently being used by HST Data Processing (distributed in the Astroconda package and containing the calwf3 executables and Astrodrizzle)
• Reference files obtained from the HST Data Archive
• Uncalibrated (raw) data files from the HST Data Archive
• Any association tables that describe observation sets.

Uncalibrated data and calibration reference files can be obtained from the HST Data Archive.

If updated reference files are required for local reprocessing, they can be obtained using the interface to CRDS, described in Section 3.1.4.

Setting up ‘iref’

Before any recalibration can be done, the user’s local directory containing the calibration reference files must be defined for the software tasks. For WFC3, this directory is referred to as ‘iref’. The raw image headers already contain the appropriate keywords that list the reference file names that were assigned during STScI pipeline processing. The user must simply define the location of the ‘iref’ directory in the Unix environment:

setenv iref /mydisk/myiref/

If done from the command line, this setup must be done in the same window in which python will be started. Setting ‘iref’ from within python will not work, even though subsequently typing ‘show iref’ would suggest it might. For convenience, this setup command can be added to your .setenv file, so that the iref environment variable will always be defined. The ‘iref’ environment variable is required whether you are reprocessing through python or from your OS command line.

Selecting Calibration Switches

The MAST HST Data Processing uses the most up-to-date calibration reference files by default. In order to use non-default reference files, manual recalibration is required. The calibration reference file keywords will need to be updated manually in the raw data files with the desired file names before running calwf3. In addition, the user can choose to change which calibration steps are performed by calwf3 by resetting the values of the calibration switch keywords. These keywords are listed in Table 3.7 along with their default values as used in the STScI pipeline. To change the values of any of the keyword switches, use a FITS keyword editor, such as the python package astropy.io.fits:

from astropy.io import fits
fits.setval(‘myfile_raw.fits’, keyword= ‘DARKCORR’, value= ‘OMIT’, ext=0)

Table 3.7: Calibration switch and default settings.

Set Up to Run the PHOTCORR Step

calwf3 does not alter the units of the pixels in the image when calculating photometric information. Instead it calculates and writes the inverse sensitivity conversion factors (PHOTFLAM and PHOTFNU) and the ST magnitude scale zero point (PHOTZPT) into header keywords in the calibrated data files. Refer to subsections on PHOTCORR in Section 3.2.12 (UVIS) and Section 3.2.8 (IR) for more information.

Speed of Pipeline Processing

Reprocessing WFC3 UVIS and IR datasets can stress some computing platforms because of the potentially large data volume and CPU-intensive calculations. Great care has been taken to minimize the memory requirements of the pipeline software.

Line-by-line I/O used during UVIS processing is particularly useful when more than one image is operated on at a time, such as during flat-field application or combining images. Unfortunately, this places an extra burden on the I/O capabilities of the computer. calwf3 requires up to 130MB of memory to process a full-frame UVIS image and up to 250MB for an IR exposure containing a full set of 16 non-destructive reads. Astrodrizzle requires up to 400MB. Geometric correction or dither-combining using Astrodrizzle will take extra time, because these are performed separately. WFC3 observers should keep these requirements in mind when securing computing resources for data processing.

3.5.2 calwf3 Examples

This section presents several examples of calwf3 reprocessing. The boxes show commands and output to the screen. The following examples indicate commands that are typed into python. Pure python commands may still be used in a PyRAF session.

Example 1: Reprocessing a Single Exposure

The following example uses hypothetical UVIS observations of a stellar cluster, observed with the F814W filter. The exposures are CR-SPLIT into two exposures of 20 seconds each. The association table for this observation is i8bt07020_asn.fits. In Python, typing ‘fits.getdata(‘i8bt07020_asn.fits’,1) reveals the rootnames of the individual exposures: i8bt07oyq and i8bt07ozq.

For the purposes of this first example, assume that the observer desires to reprocess only one of these exposures. This example illustrates the steps required to reprocess a single exposure after changing the bias reference file from the default value to a file specified by the user.

1. We assume here that the user has followed the instructions in the section See “Setting up ‘iref’” at the beginning of Section 3.5.1. We also assume that calibration reference files were obtained as part of the HDA request, and are stored in the local ‘iref’ directory (i.e., ‘/mydisk/myiref/’). In this example, the directory ‘/mydisk/’ contains the pipeline data.

2. To see what bias reference file is currently in use, use astropy.io.fits.getval( ) in Python. 
   
   

   from astropy.io import fits fits.getval(‘i8bt07oyq_raw.fits’, ‘BIASFILE’,0)

3. Next, edit the primary image header of the raw file to reflect the name of the new bias file. 
   
   

   fits.setval(‘i8bt07oyq_raw.fits’, ‘BIASFILE’,value='iref$new_bias_filename.fits')

4. Within the directory containing the pipeline products (‘/mydisk/’), create the subdirectory ‘recal’. Copy the raw file to this subdirectory, and ‘cd’ to it.

   
   

5. Finally, load the package and run calwf3 on the single raw exposure.
   
   

   from wfc3tools import calwf3 calwf3(‘ibt07oyq_raw.fits’)

The product will be a single calibrated image with the ‘_flt’ file name suffix.

Example 2: Reprocessing Multiple Exposures within an Association

This example uses the same data from Example 1 and illustrates the steps required to reprocess a WFC3 association after changing the bias reference file from the default value to a file specified by the user. The steps required are similar to the previous example, with a few modifications.

1. First, look at the contents of the association (asn) table. You will get a FITS_rec. 

   fits.getdata(‘ibt07020_asn.fits’,1) FITS_rec([('I8BT07OYQ', 'EXP-CRJ', 1), ('I8BT07OZQ', 'EXP-CRJ', 1), ('I8BT07021’, 'PROD-CRJ', 1)], dtype=(numpy.record, [('MEMNAME', 'S14'), ('MEMTYPE', 'S14'), ('MEMPRSNT', 'i1')]))

2. To see what bias reference file is currently in use, use fits.getval( ).

   fits.getval(‘i8bt07oyq_raw.fits’, ‘BIASFILE’,0)

3. Edit the primary image header of all the raw images to reflect the name of the
   new bias file.

   for raw in glob.glob(‘i8bt07*raw.fits’): fits.setval(raw, ‘BIASFILE’, value = ‘iref$mybias.fits’, ext=0)

   Note: If this command is executed in the same directory in which you have run the previous example, then one of the flt files will already exist and calwf3 will not overwrite existing images. Either delete the existing flt file, move it to a separate directory, or rename it.

4. Within the directory containing the pipeline products (‘/mydisk/’), create the subdirectory ‘recal’. Copy the raw file to this subdirectory, and ‘cd’ to it.

5. Finally, load the package and run calwf3 on the single raw exposure.

   from wfc3tools import calwf3 calwf3(‘ibt07020_asn.fits)

   The products will be two separate calibrated flt images (i8bt07oyq_flt.fits, i8bt07oyq_flt.fits) and a single CR-combined crj image (i8bt07021_crj.fits).

Example 3: Reprocessing Images taken as part of a Dither Pattern

When using IR images that are part of a 2-point line dither pattern the steps required to reprocess images are the same as in Example 2, but the format of the association and the data products are different. If you wish to use images that are part of a dither pattern that uses non-default dark reference files then replace step 3 from above with this step:

1. Edit the primary image header of all the raw files to reflect the name of the new dark reference file.

   for raw in glob.glob(‘i8e654*raw.fits’): fits.setval(raw, ‘DARKFILE, value = ‘iref$mydark.fits’, ext=0)

The output products from this run of calwf3 will be two separate calibrated datasets, consisting of ima and flt files for each of the input images. In subsequent processing Astrodrizzle can be used to combine the two flt files into a single drz image (i8e654010_drz.fits).