3.4 Pipeline Tasks


The following section contains information regarding the 5 high-level tasks that are called by the main program calwf3, namely wf3ctewf3ccdwf32dwf3irwf3rej. These tasks can be called individually by the users from either the command line or within a Python interpreter. They generally are “umbrella” tasks that can perform multiple data processing steps. However, the user can control the processing flow of calwf3 (and limit it to individual steps) by appropriate setting of the header keyword switches, as detailed in Sections 3.2 (UVIS) and 3.3 (IR).

3.4.1 wf3cte

This routine is used to correct for Charge Transfer Efficiency (CTE) losses due to detector defects trapping electrons during readout. The wf3cte step corrects for Y-CTE losses which result in charge trailing in the vertical direction; as of the publication of this handbook (June 2024), there is no automated correction for the substantially smaller effect of X-CTE losses. The effect of the latter is primarily astrometric, most images will not be significantly impacted. Observers performing high-precision astrometry should consider applying the stand-alone X-CTE correction available on the CTE tools page (see ISR 2024-07).

More details on the CTE phenomenon are given in Chapter 6, see also the WFC3 CTE webpage. Note that while the CTE model has recently been updated (WFC3 ISR 2021-09), the interface to the code as described below remains the same.

The PCTETAB reference file contains extensions of calibration images and tables of parameters used during the CTE correction stage. The header of this file also contains parameters for the CTE correction algorithm, which are used by wf3cte to correct the data. Although not recommended, users who wish to use other settings for the CTE correction algorithm can adjust the pertinent keywords in their dataset header (see WFC3 ISR 2016-02).

wf3cte performs the CTE correction on raw data files. If the calibration step keyword PCTECORR is set to PERFORM then the CTE correction will be applied to the dataset. Some caveats for its use:

  • CTE corrections can ONLY be performed on raw data which has not been calibrated in any way.
  • Data which have BLEVCORRBIASCORR or DARKCORR set to COMPLETE will be rejected.
  • As of calwf3 v3.4, both full frame and most subarray modes (Table 3.5) are corrected for CTE. The calwf3 version used to process any data is stored in the header keyword CAL_VER. The CTE correction is unavailable for subarray images acquired with aperture UVIS2-M1K1C-SUB or UVIS2-M512C-SUB as these images, near the middle of the chip, lack physical overscan pixels. The CTE correction code requires overscan to calculate a secondary bias subtraction for the image before the CTE is measured. 

Table 3.5: Sub-array modes for which CTE correction is enabled as of calwf3 version 3.4. For sub-arrays not listed here, a workaround is available.

UVIS1-2K2A-SUB
UVIS1-2K2B-SUB
UVIS2-2K2C-SUB
UVIS2-2K2D-SUB
UVIS2-C1K1C-SUB
UVIS2-C512C-SUB
UVIS2-C512D-SUB
UVIS1-C512A-SUB
UVIS1-C512B-SUB
UVIS1-2K4-SUB
UVIS-QUAD-SUB
UVIS2-2K4-SUB

The standalone call of wf3cte will produce a RAC fits (*_rac_tmp.fits) file by default. This contains only the CTE-corrected raw data, no other calibrations have been performed. The WFC3 CTE algorithm was updated in calwf3/wf3cte v3.6.0 (December 2020) where the pixel-based model is better constrained for both low and high pixel values. See the thorough discussion regarding the update in WFC3 ISR 2021-09

Displaying output from wf3cte in a Jupyter Notebook

When calling wf3cte from a Jupyter notebook, informational text output from the underlying wf3cte.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func keyword argument to wf3cte. As output is read from the underlying program, the wf3cte Python wrapper will call log_func with the contents of each line (this provides a way to connect wf3cte to the Python logging system by passing the logging.debug function or similar).

If log_func=None is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.

Input Parameters for the Python interface

Parameters 
    input : str or list 
        Name of input files, such as 
			- a single filename (iaa012wdq_raw.fits)
 			- a Python list of filenames,a partial filename with wildcards (*raw.fits) 
 			- an at-file (@input) 

    parallel : bool, default=True 
 			If True, run the code with OpenMP parallel processing turned on for the UVIS  CTE correction. 

    verbose: bool, optional, default=False 
 			If True, print verbose time stamps. 

    log_func : func(), default=print() 
 			 If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. 

Returns 

    None 

Usage 

from wfc3tools import wf3cte 
wf3cte('ib3805v0q_raw.fits', verbose=True) 

Command Line Options for the wf3cte executable

wf3cte.e input [-options] 
   wf3cte.e ib3805v0q_raw.fits -vt1 
   wf3cte.e ib3805v0q_raw.fits,ib3805v1q_raw.fits -vt1 

   input : str or comma-separated list of string(s) 
   		Name of input filename or list of comma-separated input filenames 
 		- single filename: ipppssoot_raw.fits 
		- multiple filenames: ipppssoot_raw.fits,ipppssoot_raw.fits (Note: Do not include any blank spaces.) 

   options 
		-r : print version number and date of software (e.g., Current version: 3.6.2 (May-27-2021)) and exit 
		-t : print a detailed time stamp 
		-v : print verbose time stamps and information 
		-1 : suppress the OpenMP parallel processing for the UVIS CTE correction 

   --help : print the syntax for executing this command and exit 

--version : print version number of software (e.g., 3.6.2) and exit 
--gitinfo : print git information (if it can be obtained) and exit 

The output CTE corrected file is the ipppssoot root of the input file with the suffix of _rac_tmp.fits (e.g., ib3805v0q_raw.fits -> ib3805v0q_rac_tmp.fits). 

Basic Steps In The CTE Correction

  • The reference bias image named in the BIACFILE header keyword is subtracted from the data.
  • Parameters from the CTE parameter table, referenced in the PCTETAB header keyword, are read and stored.
  • The data are reformatted: each quadrant is rotated such that the readout amp is located at the lower left of the array. The reoriented four quadrants are then arranged side-by-side into a single 8412 × 2070 image (including the overscan pixels) with amps CDAB in that order. In this format, the pixels are all parallel-shifted down, then serial-shifted to the left.
  • An additional bias correction is performed using the residual bias level measured for each amplifier from the steadiest pixels in the horizontal overscan, this value is then subtracted from all the pixels in each respective amp.
  • The image is multiplied by the gain to convert to electrons.
  • CTE correction v1.0 (calwf3 v3.5.2 and earlier). The bias-subtracted image contained contribution from read-noise. Those read-noise “electrons”, however, are generated at the readout and are not affected by CTE.  The smoothest possible image is constructed to be consistent with the original image plus pure readnoise (a Gaussian distribution with a mean of zero and a sigma of 3.25 e-). The CTE reconstruction algorithm then operates on the "smooth" version of the scene, determining which pre-read-out charge distribution could be CTE-trailed to result in the "smooth" scene. The difference between these two images is an estimate of how CTE likely redistributed charge in the exposure. This difference was added to the original raw, uncorrected, uncalibrated image. 
  • CTE correction v2.0 (calwf3 v3.6.0 and later). While v1.0 worked well for a number of years, once the CTE correction became larger than a perturbation, read-noise was significantly amplified and required a new approach. In v2.0, pixel-to-pixel variations smaller than the background fluctuations (or 10 electrons, whichever is smaller) are essentially untouched by the reconstruction algorithm. The penalty is that the faintest sources will not be CTE-corrected and users will need to apply other strategies to correct the faintest measurements (see WFC3 ISR 2021-09 for more details).
  • The corrected image is now ready to continue through the rest of the pipeline. When the DARKCORR header keyword is set to PERFORM, the CTE corrected image will be dark-subtracted using the dark reference file assigned in the DRKCFILE header keyword.
  • In the case of a subarray image, the same steps are performed as above after the image has been placed into the correct full-frame reference position since the correction is dependent on the distance of the pixels away from the read-out amplifier.

The PCTETAB and Algorithm Parameters

The following are new primary header keywords that will be updated in the data headers during the wf3cte step. They are also specified in the PCTETAB reference file.

KEYWORD

DESCRIPTION

CTE_NAME

name of CTE algorithm [string]

CTE_VER

version number of CTE algorithm [string]

CTEDATE0

date of WFC3/UVIS installation in HST, in modified Julian days (MJD)

CTEDATE1

reference date of CTE model pinning, in modified Julian days (MJD)

PCTETLEN

max length of CTE trail

PCTERNOI

read-noise amplitude for clipping

PCTENFOR

number of iterations used in CTE forward modeling

PCTENPAR

number of iterations used in the parallel transfer

PCTENSMD

read-noise mitigation algorithm

PCTETRSH

over-subtraction threshold

PCTEFRAC

CTE scaling fraction calculated from expstart and used in the algorithm

PCTERNOI

read-noise clipping level to use*

FIXROCR

make allowance for readout cosmic rays

*Note: The value PCTERNOI is the read-noise clipping level to use during processing. This value is no longer used from the PCTETAB file since calwf3 v3.6.0 (December 2020). If the PCTERNOI keyword value in the raw science image header is non-zero, it will be used for the CTE computations. Otherwise, the value is computed on-the-fly based upon the raw image data. 

The PCTETAB reference file has 4 extensions, two tables, and two images:

Filename: 54l1347ei_cte.fits
No. Name 		Type 		Cards	Dimensions	Format
0 	PRIMARY 	PrimaryHDU 	80		()
1 	QPROF 		BinTableHDU	18 		999R x 4C 	['I', 'J', 'E', '20A']
2 	SCLBYCOL	BinTableHDU	22 		8412R x 6C 	['I', 'E', 'E', 'E', 'E', '20A']
3 	RPROF 		ImageHDU 	33 		(999, 100) 	float32
4 	CPROF 		ImageHDU 	33 		(999, 100) 	float32

The first extension lists the charge-trap levels. The columns are respectively the trap number, the charge-packet size it applies to (in electrons), the size of the trap (in electrons), and a description.

The second extension contains the CTE scalings as a function of column number. There are 6 columns, each with 8412 elements. The first column contains the integer column number in the readout amp-aligned large array. The other columns contain the CTE scaling appropriate for that column at the 512th, 1024th, 1536th and 2048th rows respectively. The final column provides a description.

The third extension contains the differential CTE trail profile as a function of charge level in the form of an image.

The fourth extension contains the cumulative CTE trail profile as a function of charge level, also in the form of an image.

Output Files

If the user is running the separate wf3cte.e step a _rac.fits file will be produced. This is the same as a _raw.fits file except the CTE correction has been applied to the data.

If the PCTECORR step is set to PERFORM:

  • When the _raw.fits file enters calwf3, no intermediate rac_tmp.fits file will be saved unless you specify the -s flag, which instructs calwf3.e to save all intermediate files.
  • The calwf3 pipeline will produce both CTE calibrated products and non-CTE calibrated products. The CTE products have a ‘c’ at the end of their extension name, such as _blc, _rac, _crc, _flc, and the non-CTE calibrated products contain the familiar: _blv, _crj, _flt, etc.

Note: The CTE correction step uses all available CPUs (while the rest of the calibration steps do not).

3.4.2 wf3ccd

This routine performs the initial processing steps for all the WFC3 UVIS channel data. These steps are:

  • DQICORR -  initialize the data quality array with values from BPIXTAB, flag for A-to-D saturation, and potentially flag for full-well saturation using scalar value as the threshold (fall-back algorithm)
  • ATODCORR - perform the a-to-d conversion correction
  • BLEVCORR - subtract the bias level determined from the overscan regions
  • BIASCORR - subtract the superbias image
  • Flag for full-well saturation using a two-dimensional image (new algorithm) 
  • Detect and record SINK pixels in the DQ mask (performed if DQICORR is set to PERFORM
  • FLSHCORR - subtract the post-flash image


The wf3ccd routine processes everything in data numbers (DN). If a calibration reference file is in units of electrons, wf3ccd will divide it by the gain before use. The conversion to electrons is performed by the wf32d routine. 

wf3ccd first subtracts the bias and trims the overscan regions from the image. If an associated set of UVIS CR-SPLIT or REPEAT-OBS images is being processed, all of the overscan-trimmed images are sent through wf3rej to be combined and receive cosmic-ray rejection. The resulting combined image then receives final calibration with wf32d, which includes dark subtraction and flat fielding. If there are multiple subsets of CR-SPLIT or REPEAT-OBS images in an association, each set goes through the cycle of wf3ccdwf3rej and wf32d processing.

Only those calibration steps with a switch value of PERFORM in the input files will be executed, after which the switch will be set to COMPLETE in the corresponding output files.

Displaying output from wf3ccd in a Jupyter Notebook

When calling wf3ccd from a Jupyter notebook, informational text output from the underlying wf3ccd.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func keyword argument to wf3ccd. As output is read from the underlying program, the wf3ccd Python wrapper will call log_func with the contents of each line (this provides a way to connect wf3ccd to the Python logging system by passing the logging.debug function or similar).

If log_func=None is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.

Input parameters for the Python interface

Parameters 

input : str or list 
	Name of input files, such as 
		- a single filename (iaa012wdq_raw.fits) 
		- a Python list of filenames 
		- a partial filename with wildcards (\*raw.fits) 
		- filename of an ASN table (\*asn.fits) 
		- an at-file (@input) 

output : str, default=None 
	Name of the output FITS file. 

dqicorr : str, optional, default=”PERFORM” 
	Update the dq array from bad pixel table, as well as flag the A-to-D saturation. If the comparatively new FITS keyword (mid-2023) SATUFILE is missing or not populated in the input file primary header, the full-well saturation will also be flagged using a single value as the threshold. Allowed values are “PERFORM” and “OMIT”. 

atodcorr : str, optional, default=”PERFORM” 
	Analog to digital correction. Allowed values are “PERFORM” and “OMIT”. 

blevcorr : str, optional, default=”PERFORM” 
	Subtract bias level determined from overscan regions. Allowed values are “PERFORM” and “OMIT”. 

biascorr : str, optional, default=”PERFORM” 
	Subtract superbias image. Allowed values are “PERFORM” and “OMIT”. 

	NOTE: Strictly speaking, the application of the full-well saturation image is not a calibration step (i.e., there is no SATCORR), but the application of a 2D image to flag pixels versus using a single scalar value to flag saturated pixels as previously done in DQICORR will be done in doFullWellSat() after BLEVCORR and BIASCORR. This correction should only be done if both BLEVCORR and BIASCORR have been performed. This flagging is only applicable for the UVIS. 

flashcorr : str, optional, default=”PERFORM” 
	Subtract post-flash image. Allowed values are “PERFORM” and “OMIT”. 

verbose : bool, optional, default=False 
	If True, print verbose time stamps. 

quiet : bool, optional, default=True 
	If True, print messages only to trailer file. 

log_func : func(), default=print() 
	If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. 

Returns 

None 

Usage 

from wfc3tools import wf3ccd 
wf3ccd(filename) 

Command Line Options for the wf3ccd C executable

wf32ccd.e input output [-options]

Input may be a single filename and the options include:

-v: verbose
-f: print time stamps
-dqi: udpate the DQ array
-atod: perform a-to-d gain correction
-blev: subtract bias from overscan
-bias: perform bias correction
-flash: remove post-flash image

3.4.3 wf32d

This routine performs the remaining series of tasks in the UVIS pipeline. The wf32d primary functions include:

  • DARKCORR: dark current subtraction
  • FLATCORR: flat-fielding
  • SHADCORR: apply shutter shading correction (currently skipped) 
  • PHOTCORR: photometric keyword calculations

  • FLUXCORR: photometric normalization of the UVIS1 and UVIS2 chips

Only those calibration steps with a switch value of PERFORM in the input files primary headers will be executed, after which the switch will be set to COMPLETE in the primary headers of the corresponding output files.

Displaying output from wf32d in a Jupyter Notebook

When calling wf32d from a Jupyter notebook, informational text output from the underlying wf32d.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func keyword argument to wf32d. As output is read from the underlying program, the wf32d Python wrapper will call log_func with the contents of each line (this provides a way to connect wf32d to the Python logging system by passing the logging.debug function or similar).

If log_func=None is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.

Parameters

Parameters 

input : str or list 
	Name of input files, such as 
		- a single filename (iaa012wdq_raw.fits) 
		- a Python list of filenames 
		- a partial filename with wildcards (\*raw.fits) 
		- filename of an ASN table (\*asn.fits) 
		- an at-file (@input) 

output : str, default=None 
	Name of the output FITS file. 

dqicorr : str, optional, default=”PERFORM” 
	Update the dq array from bad pixel table. Allowed values are “PERFORM” and “OMIT”. 

darkcorr : str, optional, default=”PERFORM” 
	Subtract the dark image. Allowed values are “PERFORM” and “OMIT”. 

flatcorr : str, optional, default=”PERFORM” 
	Multiply by the flatfield image. Allowed values are “PERFORM” and “OMIT”. 

shadcorr : str, optional, default=”PERFORM” 
	Correct for shutter shading (CCD). Allowed values are “PERFORM” and “OMIT”. 

photcorr : str, optional, default=”PERFORM” 
	Update photometry keywords in the header. Allowed values are “PERFORM” and “OMIT”. 

verbose : bool, optional, default=False 
	If True, print verbose time stamps. 

quiet : bool, optional, default=True 
	If True, print messages only to trailer file. 

debug : bool, optional, default=False 
	If True, print debugging statements. 

log_func : func(), default=print() 
	If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. 

Returns 

None 

Usage 

from wfc3tools import wf32d 
wf32d(filename) 

Command Line Options for the wf32d executable

wf32d.e input output [-options]

Input may be a single filename, and the options include:

-v: verbose
-t: print time stamps
-d: debug
-dark: perform dark subtraction
-dqi: update the DQ array
-flat: perform flat correction
-shad: perform shading correction
-phot: perform phot correction

3.4.4 wf3ir

This routine contains all the instrumental calibration steps for WFC3 IR channel images. The steps are:

  • DQICORR - initialize the data quality array
  • ZSIGCORR - estimate the amount of signal in the zeroth-read
  • BLEVCORR - subtract the bias level from the reference pixels
  • ZOFFCORR - subtract the zeroth read image
  • NLINCORR - correct for detector non-linear response
  • DARKCORR - subtract the dark current image
  • PHOTCORR - compute the photometric keyword values
  • UNITCORR - convert to units of count rate
  • CRCORR - fit accumulating signal and identify the cr hits
  • FLATCORR - divide by the flat-field images and apply gain conversion

The output images include the calibrated image ramp (ima file) and the accumulated ramp image (flt file).

Only those calibration steps with a switch value of PERFORM in the input files primary headers will be executed, after which the switch will be set to COMPLETE in the primary headers of the corresponding output files. 

Displaying output from wf3ir in a Jupyter Notebook

When calling wf3ir from a Jupyter notebook, informational text output from the underlying wf3ir.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func keyword argument to wf3ir. As output is read from the underlying program, the wf3ir Python wrapper will call log_func with the contents of each line (this provides a way to connect wf3ir to the Python logging system by passing the logging.debug function or similar).

If log_func=None is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.

Input Parameters for the Python interface

Parameters 

input : str 
	Name of input files, such as 
		- a single filename (iaa012wdq_raw.fits) 
		- a Python list of filenames 
		- a partial filename with wildcards (\*raw.fits) 
		- filename of an ASN table (\*asn.fits) 
		- an at-file (@input) 

output : str, default=None 
	Name of the output FITS file. 

verbose : bool, optional, default=False 
	If True, print verbose time stamps. 

quiet : bool, optional, default=True 
	If True, print messages only to trailer file. 

log_func : func(), default=print() 
	If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. 

Returns 

None 

Usage 

from wfc3tools import wf3ir 
wf3ir(filename) 

Command Line Options for the wf3ir executable

wf32ir.e input output [-options]

Input may be a single filename, and the options include:

-v: verbose
-f: print time stamps

3.4.5 wf3rej

wf3rej, the cosmic-ray rejection and image combination task in calwf3, combines CR-SPLIT or REPEAT-OBS exposures into a single image, first detecting and then replacing flagged pixels. The task uses the same statistical detection algorithm developed for ACS (acsrej), STIS (ocrrej), and WFPC2 data (crrej), providing a well-tested and robust procedure.

First, wf3rej temporarily removes the sky background from each input image (specified by the SKYSUB parameter in the CRREJTAB reference file or by the skysub parameter directly passed to the Python script or C executable), usually computed using the mode of each image. Sky subtraction is performed before any statistical checks are made for cosmic rays. Next, wf3rej constructs an initial comparison image from each sky-subtracted exposure. This comparison image can either be a median- or minimum-value sky-subtracted image constructed from all the input images, and it represents the ‘initial guess’ of a cosmic-ray free image. The comparison image serves as the basis for determining the statistical deviation of each pixel within the input images.

A detection threshold is then calculated for each pixel based on the comparison image:

\tau_{n} = \sigma^2 \times (noise+\frac{value}{gain}+(scale\times value)^2)/\Tau_{n}^2

where:

  • σ is the sigma value used as the detection limit (CRSIGMAS),
  • noise is the readnoise in DN squared and gain is the e-/DN of the amplifier used to read the pixel,
  • scale is the scale factor for the noise model,
  • Tn is the exposure time for the input image, and
  •  value is the pixel value (in DN) from the median or minimum combined comparison image. 

The actual detection criterion for a cosmic ray is determined as:

\Delta= \bigg( \frac{pix_{n}-sky_{n}}{T_{n}}-median \bigg)^2

where:

  • pixeln is the pixel value from input image n,
  • skyn is the sky background of image n, and
  • median is the median or minimum pixel value from the comparison image.

If Δ > τn the pixel is flagged as a cosmic ray in the input image’s DQ array and is ignored when images are summed together. Surrounding pixels within some expansion radius (CRRADIUS) are marked as ‘SPILL’ pixels and are given less stringent detection thresholds.


When all input images have been processed, the values of the non-rejected pixels are summed over all input images. Each pixel in the summed output array is then scaled by the total exposure time:


pixout(x, y) = T \cdot \frac{\sum_{n} (pix_n(x, y) - sky_n) m_n(x, y)}{\sum_{n} T_n m_n(x, y)} + \sum_{n} sky_n

where:

  • Τn is the exposure time for image n,
  • mn(x,y) is the mask value (0 for rejected pixels, 1 for good data) for the n-th image at pixel (x, y),
  • Τ is the total exposure time (regardless of whether all input images were used for that particular pixel). This corresponds to the value recorded in the header keywords TEXPTIME and EXPTIME.

The following keywords are also derived from the variables in this equation:

  • TEXPTIME = EXPTIME = T
  • SKYSUM = ∑n skyn
  • REJ_RATE = (∑n Tnmn(x,y)/Τ) averaged over all pixels
  • NCOMBINE = n

The remaining keywords EXPSTARTEXPEND are updated based on the values corresponding to the first and last input images, respectively. In summary, the cosmic ray rejection task sums all non-rejected pixel values, computes the true exposure time for that pixel, and scales the sum to correspond to the total exposure time. The final scaled, cleaned pixel is written to the comparison image to be used for the next iteration. This process is then repeated with increasingly stringent detection thresholds, as specified by CRSIGMAS.

Cosmic Ray Rejection Table

wf3rej uses the Cosmic Ray Rejection parameter table (CRREJTAB) to determine the number of iterations for cosmic-ray rejection, the sigma levels to use for each iteration, and the spill radius to use during detection. This allows the rejection process to be tuned to each detector and observation, with suitable defaults being applied during pipeline processing. Observers may fine-tune the cosmic-ray rejection parameters when manually reprocessing data with wf3rej by editing the CRREJTAB.

The CRREJTAB reference file contains the basic parameters necessary for performing cosmic-ray rejection. The column names and default values for the CRREJTAB are given in Table 3.6. The appropriate row is selected based on the chip being processed (CCDCHIP), the number of images into which the exposure was split (CR-SPLIT), and the exposure time of each CR-SPLIT image (MEANEXP). If an exact match is not found for the exposure time, the table row with the closest value is used. If the CR-SPLIT value of the input images exceeds the values in the table, the table row with the largest CR-SPLIT value will be used. The sky fitting algorithm is controlled by the parameter SKYSUB, which can have values of ‘mode’, ‘mean’ or ‘none’. The ‘initial guess’ image is created using the median or minimum value of the input exposures, as specified by the value of INITGUES.

Cosmic-ray detection requires the specification of a threshold above which a pixel value is considered a cosmic ray. This threshold was defined above and uses sigma rejection thresholds. The sigma threshold for each iteration and number of iterations corresponds to the CRSIGMAS column values in the CRREJTAB file. SCALENSE is a multiplicative term (in percent) for the noise model and is given as scale in the threshold equation above. This term can be useful when the pointing of the telescope has changed by a small fraction of a pixel between images. Under such circumstances, the undersampling of the image by the detector will cause stars to be mistakenly rejected as cosmic rays if a scale noise term is not included. This is a crude but effective step taken to satisfy the maxim of ‘do no harm’. However, for cases in which there have been no image-to-image offsets or the image is locally well-sampled, this will unduly bias against rejecting cosmic rays.

Pixels within a given radius, CRRADIUS, of a cosmic ray will also be treated as cosmic rays. A less stringent rejection threshold, CRTHRESH, can be used for detecting pixels adjacent to a cosmic ray. As for CRSIGMASCRTHRESH is also given as a sigma value. If CRTHRESH is exceeded, pixels within the defined radius of the cosmic ray will also be flagged. All pixels determined to be affected by a cosmic ray will have their DQ values set to 8192, as described in Table 3.6.

Table 3.6: Columns in the cosmic-ray rejection parameter table.

Column Name

Default Value

Contents

CRSPLIT

-

Number of exposures into which observation was split

CCDCHIP

-

Chip to which this conversion applies

MEANEXP

-

Average exposure time (sec) for each image

SCALENSE

30.0

Multiplicative term (in percent) for the noise model

INITGUES

minimum

Method for computing initial-guess image (minimum, median)

SKYSUB

mode

Sky fitting algorithm (mode, none)

CRSIGMAS

6.5, 5.5, 4.5

Rejection thresholds (sigma)

CRRADIUS

2.1

Radius (in pixels) for propagating cosmic ray

CRTHRESH

0.5555

Propagation factor

BADINPDQ

39

Data quality file bits to reject

CRMASK

yes

Flag CR-rejected pixels in input files?


Displaying output from wf3rej in a Jupyter Notebook

When calling wf3rej from a Jupyter notebook, informational text output from the underlying wf3rej.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func keyword argument to wf3rej. As output is read from the underlying program, the wf3rej Python wrapper will call log_func with the contents of each line (this provides a way to connect wf3rej to the Python logging system by passing the logging.debug function or similar).

If log_func=None is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.

Input Parameters for the Python interface

Parameters 

input : str or list 
	Name of input files, such as:
		- comma-separated (no spaces) filenames (iaao01k8q_flc.fits,iaao01k9q_flc.fits) 
		- a Python list of filenames 
		- a partial filename with wildcards (*flt.fits) 
		- an at-file (@input) 

output : str 
	Name of the output FITS file. 

crrejtab : str, default=”” 
	Reference filename. 

scalense : float, default=0. 
	Scale factor applied to noise. 

initgues : str, default=”” 
	Initial value estimate scheme (min|med). 

skysub : str, default=”” 
	How to compute the sky (none|mode|mean). 

crsigmas : str, default=”” 
	Rejection levels in each iteration. 

crradius : float, default=0. 
	Cosmic ray expansion radius in pixels. 

crthresh : float, default=0. 
	Rejection propagation threshold. 

badinpdq : int, default=0 
	Data quality flag bits to reject. 

crmask : bool, default=Setting to be read from CRREJTAB. 
	If argument is present, write the CR flag value to the input DQ images. 

shadcorr : bool, default=Setting to be read from SHADCORR keyword value in primary header of first image to process. 
	If argument is present, perform shading shutter correction. 

verbose : bool, optional, default=False 
	If True, print timestamps and other output. 

log_func : func(), default=print() 
	If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. 

Returns 

None 

Usage 

from wfc3tools import wf3rej 
from glob import glob 
infiles = glob("*flt.fits") 
wf3rej(infiles, "output.fits", verbose=True) 
wf3rej("*flt.fits", "output.fits", verbose=True) 
wf3rej("@input.lst", "output.fits", verbose=True) 

Please see the warning at the bottom of the page regarding the parameter settings for wf3rej.e for more details as to the action taken when the parameters use their default values.

Command Line Options for the wf3rej C executable

wf3rej.e input output [-r] [-v] [-t] [-shadcorr] [-crmask] [-table <filename>] 
    [-scale <float>] [-init <med|min>] [-sky <none|mode|mean>] [-sigmas <string>] 
    [-radius <float>] [-thresh <float>] [-pdq <short>] 
Example - Process data with timestamps and a custom cosmic ray rejection table: 
wf3rej.e iaao01k8q_flc.fits,iaao01k9q_flc.fits output.fits -t 
Example - Print the code version and exit: 
wf3rej.e -r 

input : comma-separated list of strings 
    - Input filenames as a list of comma-separated input names 
	- ipppssoot_raw.fits,ipppssoot_raw.fits (Note: Do not include any blank spaces.) 

output : str 
    Name of output filename 

options 
       -r : print version number/date of software and exit (no other options selected) 
       -v : verbose mode 
       -t : print the timestamps 
-shadcorr : perform shading shutter correction    (not used)
-crmask : set CR flags in input DQ images 

-table <filename>: string, the crrejtab filename 
-scale <number>: float, scale factor for noise 
-init <med|min>: string, initial value estimate scheme 
-sky <none|mode|mean>: string, method to compute sky 

-sigmas <string of numbers>: string, rejection levels for each iteration (e.g., "3.5,4.5,5.5") 

-radius <number>: float, CR expansion radius 

-thresh <number> : float, rejection propagation threshold 

-pdq <number>: short, data quality flag bits to reject   


Observers should not invoke the shadcorr option as there is no SHADFILE.

Including the crmask option on the command line will have the CR flag values placed into the DQ extension of the input images. Omitting this option does not turn off the insertion, but rather the program will follow the default setting for the crmask option as indicated in the CRREJTAB calibration file.  

If any of the CR-rejection options are omitted from the command line, default values from CRREJTAB will be used; those options are: crmask,scale,init,sky,sigmas,radius,thresh,and pdq. In verbose mode, all of the option values are printed to the output logfile.