3.4 Pipeline Tasks
The following section contains information regarding the 5 high-level tasks that are called by the main program calwf3, namely wf3cte, wf3ccd, wf32d, wf3ir, wf3rej. These tasks can be called individually by the users from either the command line or within a Python interpreter. They generally are “umbrella” tasks that can perform multiple data processing steps. However, the user can control the processing flow of calwf3 (and limit it to individual steps) by appropriate setting of the header keyword switches, as detailed in Sections 3.2 (UVIS) and 3.3 (IR).
3.4.1 wf3cte
This routine is used to correct for Charge Transfer Efficiency (CTE) losses due to detector defects trapping electrons during readout. The wf3cte step corrects for Y-CTE losses which result in charge trailing in the vertical direction; as of the publication of this handbook (June 2024), there is no automated correction for the substantially smaller effect of X-CTE losses. The effect of the latter is primarily astrometric, most images will not be significantly impacted. Observers performing high-precision astrometry should consider applying the stand-alone X-CTE correction available on the CTE tools page (see ISR 2024-07).
More details on the CTE phenomenon are given in Chapter 6, see also the WFC3 CTE webpage. Note that while the CTE model has recently been updated (WFC3 ISR 2021-09), the interface to the code as described below remains the same.
The PCTETAB
reference file contains extensions of calibration images and tables of parameters used during the CTE correction stage. The header of this file also contains parameters for the CTE correction algorithm, which are used by wf3cte to correct the data. Although not recommended, users who wish to use other settings for the CTE correction algorithm can adjust the pertinent keywords in their dataset header (see WFC3 ISR 2016-02).
wf3cte performs the CTE correction on raw data files. If the calibration step keyword PCTECORR
is set to PERFORM
then the CTE correction will be applied to the dataset. Some caveats for its use:
- CTE corrections can ONLY be performed on raw data which has not been calibrated in any way.
- Data which have
BLEVCORR
,BIASCORR
orDARKCORR
set toCOMPLETE
will be rejected. - As of calwf3 v3.4, both full frame and most subarray modes (Table 3.5) are corrected for CTE. The calwf3 version used to process any data is stored in the header keyword
CAL_VER
. The CTE correction is unavailable for subarray images acquired with aperture UVIS2-M1K1C-SUB or UVIS2-M512C-SUB as these images, near the middle of the chip, lack physical overscan pixels. The CTE correction code requires overscan to calculate a secondary bias subtraction for the image before the CTE is measured.
Table 3.5: Sub-array modes for which CTE correction is enabled as of calwf3 version 3.4. For sub-arrays not listed here, a workaround is available.
UVIS1-2K2A-SUB UVIS1-2K2B-SUB UVIS2-2K2C-SUB UVIS2-2K2D-SUB UVIS2-C1K1C-SUB UVIS2-C512C-SUB UVIS2-C512D-SUB UVIS1-C512A-SUB UVIS1-C512B-SUB UVIS1-2K4-SUB UVIS-QUAD-SUB UVIS2-2K4-SUB
The standalone call of wf3cte will produce a RAC fits (*_rac_tmp.fits) file by default. This contains only the CTE-corrected raw data, no other calibrations have been performed. The WFC3 CTE algorithm was updated in calwf3/wf3cte v3.6.0 (December 2020) where the pixel-based model is better constrained for both low and high pixel values. See the thorough discussion regarding the update in WFC3 ISR 2021-09.
Displaying output from wf3cte in a Jupyter Notebook
When calling wf3cte from a Jupyter notebook, informational text output from the underlying wf3cte.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func
keyword argument to wf3cte. As output is read from the underlying program, the wf3cte Python wrapper will call log_func
with the contents of each line (this provides a way to connect wf3cte to the Python logging system by passing the logging.debug function or similar).
If log_func=None
is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.
Input Parameters for the Python interface
Parameters input : str or list Name of input files, such as - a single filename (iaa012wdq_raw.fits) - a Python list of filenames,a partial filename with wildcards (*raw.fits) - an at-file (@input) parallel : bool, default=True If True, run the code with OpenMP parallel processing turned on for the UVIS CTE correction. verbose: bool, optional, default=False If True, print verbose time stamps. log_func : func(), default=print() If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. Returns None Usage from wfc3tools import wf3cte wf3cte('ib3805v0q_raw.fits', verbose=True)
Command Line Options for the wf3cte executable
wf3cte.e input [-options] wf3cte.e ib3805v0q_raw.fits -vt1 wf3cte.e ib3805v0q_raw.fits,ib3805v1q_raw.fits -vt1 input : str or comma-separated list of string(s) Name of input filename or list of comma-separated input filenames - single filename: ipppssoot_raw.fits - multiple filenames: ipppssoot_raw.fits,ipppssoot_raw.fits (Note: Do not include any blank spaces.) options -r : print version number and date of software (e.g., Current version: 3.6.2 (May-27-2021)) and exit -t : print a detailed time stamp -v : print verbose time stamps and information -1 : suppress the OpenMP parallel processing for the UVIS CTE correction --help : print the syntax for executing this command and exit --version : print version number of software (e.g., 3.6.2) and exit --gitinfo : print git information (if it can be obtained) and exit The output CTE corrected file is the ipppssoot root of the input file with the suffix of _rac_tmp.fits (e.g., ib3805v0q_raw.fits -> ib3805v0q_rac_tmp.fits).
Basic Steps In The CTE Correction
- The reference bias image named in the
BIACFILE
header keyword is subtracted from the data. - Parameters from the CTE parameter table, referenced in the
PCTETAB
header keyword, are read and stored. - The data are reformatted: each quadrant is rotated such that the readout amp is located at the lower left of the array. The reoriented four quadrants are then arranged side-by-side into a single 8412 × 2070 image (including the overscan pixels) with amps CDAB in that order. In this format, the pixels are all parallel-shifted down, then serial-shifted to the left.
- An additional bias correction is performed using the residual bias level measured for each amplifier from the steadiest pixels in the horizontal overscan, this value is then subtracted from all the pixels in each respective amp.
- The image is multiplied by the gain to convert to electrons.
- CTE correction v1.0 (calwf3 v3.5.2 and earlier). The bias-subtracted image contained contribution from read-noise. Those read-noise “electrons”, however, are generated at the readout and are not affected by CTE. The smoothest possible image is constructed to be consistent with the original image plus pure readnoise (a Gaussian distribution with a mean of zero and a sigma of 3.25 e-). The CTE reconstruction algorithm then operates on the "smooth" version of the scene, determining which pre-read-out charge distribution could be CTE-trailed to result in the "smooth" scene. The difference between these two images is an estimate of how CTE likely redistributed charge in the exposure. This difference was added to the original raw, uncorrected, uncalibrated image.
- CTE correction v2.0 (calwf3 v3.6.0 and later). While v1.0 worked well for a number of years, once the CTE correction became larger than a perturbation, read-noise was significantly amplified and required a new approach. In v2.0, pixel-to-pixel variations smaller than the background fluctuations (or 10 electrons, whichever is smaller) are essentially untouched by the reconstruction algorithm. The penalty is that the faintest sources will not be CTE-corrected and users will need to apply other strategies to correct the faintest measurements (see WFC3 ISR 2021-09 for more details).
- The corrected image is now ready to continue through the rest of the pipeline. When the
DARKCORR
header keyword is set toPERFORM
, the CTE corrected image will be dark-subtracted using the dark reference file assigned in theDRKCFILE
header keyword. - In the case of a subarray image, the same steps are performed as above after the image has been placed into the correct full-frame reference position since the correction is dependent on the distance of the pixels away from the read-out amplifier.
The PCTETAB
and Algorithm Parameters
The following are new primary header keywords that will be updated in the data headers during the wf3cte step. They are also specified in the PCTETAB
reference file.
KEYWORD | DESCRIPTION |
---|---|
CTE_NAME | name of CTE algorithm [string] |
CTE_VER | version number of CTE algorithm [string] |
CTEDATE0 | date of WFC3/UVIS installation in HST, in modified Julian days (MJD) |
CTEDATE1 | reference date of CTE model pinning, in modified Julian days (MJD) |
PCTETLEN | max length of CTE trail |
PCTERNOI | read-noise amplitude for clipping |
PCTENFOR | number of iterations used in CTE forward modeling |
PCTENPAR | number of iterations used in the parallel transfer |
PCTENSMD | read-noise mitigation algorithm |
PCTETRSH | over-subtraction threshold |
PCTEFRAC | CTE scaling fraction calculated from expstart and used in the algorithm |
PCTERNOI | read-noise clipping level to use* |
FIXROCR | make allowance for readout cosmic rays |
*Note: The value PCTERNOI
is the read-noise clipping level to use during processing. This value is no longer used from the PCTETAB
file since calwf3 v3.6.0 (December 2020). If the PCTERNOI
keyword value in the raw science image header is non-zero, it will be used for the CTE computations. Otherwise, the value is computed on-the-fly based upon the raw image data.
The PCTETAB
reference file has 4 extensions, two tables, and two images:
Filename: 54l1347ei_cte.fits No. Name Type Cards Dimensions Format 0 PRIMARY PrimaryHDU 80 () 1 QPROF BinTableHDU 18 999R x 4C ['I', 'J', 'E', '20A'] 2 SCLBYCOL BinTableHDU 22 8412R x 6C ['I', 'E', 'E', 'E', 'E', '20A'] 3 RPROF ImageHDU 33 (999, 100) float32 4 CPROF ImageHDU 33 (999, 100) float32
The first extension lists the charge-trap levels. The columns are respectively the trap number, the charge-packet size it applies to (in electrons), the size of the trap (in electrons), and a description.
The second extension contains the CTE scalings as a function of column number. There are 6 columns, each with 8412 elements. The first column contains the integer column number in the readout amp-aligned large array. The other columns contain the CTE scaling appropriate for that column at the 512th, 1024th, 1536th and 2048th rows respectively. The final column provides a description.
The third extension contains the differential CTE trail profile as a function of charge level in the form of an image.
The fourth extension contains the cumulative CTE trail profile as a function of charge level, also in the form of an image.
Output Files
If the user is running the separate wf3cte.e step a _rac.fits file will be produced. This is the same as a _raw.fits file except the CTE correction has been applied to the data.
If the PCTECORR
step is set to PERFORM
:
- When the _raw.fits file enters calwf3, no intermediate rac_tmp.fits file will be saved unless you specify the -s flag, which instructs calwf3.e to save all intermediate files.
- The calwf3 pipeline will produce both CTE calibrated products and non-CTE calibrated products. The CTE products have a ‘c’ at the end of their extension name, such as _blc, _rac, _crc, _flc, and the non-CTE calibrated products contain the familiar: _blv, _crj, _flt, etc.
Note: The CTE correction step uses all available CPUs (while the rest of the calibration steps do not).
3.4.2 wf3ccd
This routine performs the initial processing steps for all the WFC3 UVIS channel data. These steps are:
DQICORR
- initialize the data quality array with values fromBPIXTAB
, flag for A-to-D saturation, and potentially flag for full-well saturation using scalar value as the threshold (fall-back algorithm)ATODCORR
- perform the a-to-d conversion correctionBLEVCORR
- subtract the bias level determined from the overscan regionsBIASCORR
- subtract the superbias image- Flag for full-well saturation using a two-dimensional image (new algorithm)
- Detect and record SINK pixels in the DQ mask (performed if
DQICORR
is set toPERFORM
) FLSHCORR
- subtract the post-flash image
The wf3ccd routine processes everything in data numbers (DN). If a calibration reference file is in units of electrons, wf3ccd will divide it by the gain before use. The conversion to electrons is performed by the wf32d routine.
wf3ccd first subtracts the bias and trims the overscan regions from the image. If an associated set of UVIS CR-SPLIT
or REPEAT-OBS
images is being processed, all of the overscan-trimmed images are sent through wf3rej to be combined and receive cosmic-ray rejection. The resulting combined image then receives final calibration with wf32d, which includes dark subtraction and flat fielding. If there are multiple subsets of CR-SPLIT
or REPEAT-OBS
images in an association, each set goes through the cycle of wf3ccd, wf3rej and wf32d processing.
Only those calibration steps with a switch value of PERFORM
in the input files will be executed, after which the switch will be set to COMPLETE
in the corresponding output files.
Displaying output from wf3ccd in a Jupyter Notebook
When calling wf3ccd from a Jupyter notebook, informational text output from the underlying wf3ccd.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func
keyword argument to wf3ccd. As output is read from the underlying program, the wf3ccd Python wrapper will call log_func
with the contents of each line (this provides a way to connect wf3ccd to the Python logging system by passing the logging.debug
function or similar).
If log_func=None
is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.
Input parameters for the Python interface
Parameters input : str or list Name of input files, such as - a single filename (iaa012wdq_raw.fits) - a Python list of filenames - a partial filename with wildcards (\*raw.fits) - filename of an ASN table (\*asn.fits) - an at-file (@input) output : str, default=None Name of the output FITS file. dqicorr : str, optional, default=”PERFORM” Update the dq array from bad pixel table, as well as flag the A-to-D saturation. If the comparatively new FITS keyword (mid-2023) SATUFILE is missing or not populated in the input file primary header, the full-well saturation will also be flagged using a single value as the threshold. Allowed values are “PERFORM” and “OMIT”. atodcorr : str, optional, default=”PERFORM” Analog to digital correction. Allowed values are “PERFORM” and “OMIT”. blevcorr : str, optional, default=”PERFORM” Subtract bias level determined from overscan regions. Allowed values are “PERFORM” and “OMIT”. biascorr : str, optional, default=”PERFORM” Subtract superbias image. Allowed values are “PERFORM” and “OMIT”. NOTE: Strictly speaking, the application of the full-well saturation image is not a calibration step (i.e., there is no SATCORR), but the application of a 2D image to flag pixels versus using a single scalar value to flag saturated pixels as previously done in DQICORR will be done in doFullWellSat() after BLEVCORR and BIASCORR. This correction should only be done if both BLEVCORR and BIASCORR have been performed. This flagging is only applicable for the UVIS. flashcorr : str, optional, default=”PERFORM” Subtract post-flash image. Allowed values are “PERFORM” and “OMIT”. verbose : bool, optional, default=False If True, print verbose time stamps. quiet : bool, optional, default=True If True, print messages only to trailer file. log_func : func(), default=print() If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. Returns None Usage from wfc3tools import wf3ccd wf3ccd(filename)
Command Line Options for the wf3ccd C executable
wf32ccd.e input output [-options] |
Input may be a single filename and the options include:
-v: verbose |
3.4.3 wf32d
This routine performs the remaining series of tasks in the UVIS pipeline. The wf32d primary functions include:
DARKCORR
: dark current subtractionFLATCORR
: flat-fieldingSHADCORR
: apply shutter shading correction (currently skipped)PHOTCORR
: photometric keyword calculationsFLUXCORR
: photometric normalization of the UVIS1 and UVIS2 chips
Only those calibration steps with a switch value of PERFORM
in the input files primary headers will be executed, after which the switch will be set to COMPLETE
in the primary headers of the corresponding output files.
Displaying output from wf32d in a Jupyter Notebook
When calling wf32d from a Jupyter notebook, informational text output from the underlying wf32d.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func
keyword argument to wf32d. As output is read from the underlying program, the wf32d Python wrapper will call log_func
with the contents of each line (this provides a way to connect wf32d to the Python logging system by passing the logging.debug function or similar).
If log_func=None
is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.
Parameters
Parameters input : str or list Name of input files, such as - a single filename (iaa012wdq_raw.fits) - a Python list of filenames - a partial filename with wildcards (\*raw.fits) - filename of an ASN table (\*asn.fits) - an at-file (@input) output : str, default=None Name of the output FITS file. dqicorr : str, optional, default=”PERFORM” Update the dq array from bad pixel table. Allowed values are “PERFORM” and “OMIT”. darkcorr : str, optional, default=”PERFORM” Subtract the dark image. Allowed values are “PERFORM” and “OMIT”. flatcorr : str, optional, default=”PERFORM” Multiply by the flatfield image. Allowed values are “PERFORM” and “OMIT”. shadcorr : str, optional, default=”PERFORM” Correct for shutter shading (CCD). Allowed values are “PERFORM” and “OMIT”. photcorr : str, optional, default=”PERFORM” Update photometry keywords in the header. Allowed values are “PERFORM” and “OMIT”. verbose : bool, optional, default=False If True, print verbose time stamps. quiet : bool, optional, default=True If True, print messages only to trailer file. debug : bool, optional, default=False If True, print debugging statements. log_func : func(), default=print() If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. Returns None Usage from wfc3tools import wf32d wf32d(filename)
Command Line Options for the wf32d executable
wf32d.e input output [-options] |
Input may be a single filename, and the options include:
|
3.4.4 wf3ir
This routine contains all the instrumental calibration steps for WFC3 IR channel images. The steps are:
DQICORR
- initialize the data quality arrayZSIGCORR
- estimate the amount of signal in the zeroth-readBLEVCORR
- subtract the bias level from the reference pixelsZOFFCORR
- subtract the zeroth read imageNLINCORR
- correct for detector non-linear responseDARKCORR
- subtract the dark current imagePHOTCORR
- compute the photometric keyword valuesUNITCORR
- convert to units of count rateCRCORR
- fit accumulating signal and identify the cr hitsFLATCORR
- divide by the flat-field images and apply gain conversion
The output images include the calibrated image ramp (ima file) and the accumulated ramp image (flt file).
Only those calibration steps with a switch value of PERFORM
in the input files primary headers will be executed, after which the switch will be set to COMPLETE
in the primary headers of the corresponding output files.
Displaying output from wf3ir in a Jupyter Notebook
When calling wf3ir from a Jupyter notebook, informational text output from the underlying wf3ir.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func
keyword argument to wf3ir. As output is read from the underlying program, the wf3ir Python wrapper will call log_func
with the contents of each line (this provides a way to connect wf3ir to the Python logging system by passing the logging.debug function or similar).
If log_func=None
is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.
Input Parameters for the Python interface
Parameters input : str Name of input files, such as - a single filename (iaa012wdq_raw.fits) - a Python list of filenames - a partial filename with wildcards (\*raw.fits) - filename of an ASN table (\*asn.fits) - an at-file (@input) output : str, default=None Name of the output FITS file. verbose : bool, optional, default=False If True, print verbose time stamps. quiet : bool, optional, default=True If True, print messages only to trailer file. log_func : func(), default=print() If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. Returns None Usage from wfc3tools import wf3ir wf3ir(filename)
Command Line Options for the wf3ir executable
wf32ir.e input output [-options] |
Input may be a single filename, and the options include:
-v: verbose |
3.4.5 wf3rej
wf3rej, the cosmic-ray rejection and image combination task in calwf3, combines CR-SPLIT
or REPEAT-OBS
exposures into a single image, first detecting and then replacing flagged pixels. The task uses the same statistical detection algorithm developed for ACS (acsrej), STIS (ocrrej), and WFPC2 data (crrej), providing a well-tested and robust procedure.
First, wf3rej temporarily removes the sky background from each input image (specified by the SKYSUB
parameter in the CRREJTAB
reference file or by the skysub
parameter directly passed to the Python script or C executable), usually computed using the mode of each image. Sky subtraction is performed before any statistical checks are made for cosmic rays. Next, wf3rej constructs an initial comparison image from each sky-subtracted exposure. This comparison image can either be a median- or minimum-value sky-subtracted image constructed from all the input images, and it represents the ‘initial guess’ of a cosmic-ray free image. The comparison image serves as the basis for determining the statistical deviation of each pixel within the input images.
A detection threshold is then calculated for each pixel based on the comparison image:
\tau_{n} = \sigma^2 \times (noise+\frac{value}{gain}+(scale\times value)^2)/\Tau_{n}^2 |
where:
- σ is the sigma value used as the detection limit (
CRSIGMAS
), - noise is the readnoise in DN squared and gain is the e-/DN of the amplifier used to read the pixel,
- scale is the scale factor for the noise model,
- Tn is the exposure time for the input image, and
- value is the pixel value (in DN) from the median or minimum combined comparison image.
The actual detection criterion for a cosmic ray is determined as:
\Delta= \bigg( \frac{pix_{n}-sky_{n}}{T_{n}}-median \bigg)^2 |
where:
- pixeln is the pixel value from input image n,
- skyn is the sky background of image n, and
- median is the median or minimum pixel value from the comparison image.
If Δ > τn the pixel is flagged as a cosmic ray in the input image’s DQ array and is ignored when images are summed together. Surrounding pixels within some expansion radius (CRRADIUS
) are marked as ‘SPILL
’ pixels and are given less stringent detection thresholds.
When all input images have been processed, the values of the non-rejected pixels are summed over all input images. Each pixel in the summed output array is then scaled by the total exposure time:
pixout(x, y) = T \cdot \frac{\sum_{n} (pix_n(x, y) - sky_n) m_n(x, y)}{\sum_{n} T_n m_n(x, y)} + \sum_{n} sky_n |
where:
- Τn is the exposure time for image n,
- mn(x,y) is the mask value (0 for rejected pixels, 1 for good data) for the n-th image at pixel (x, y),
- Τ is the total exposure time (regardless of whether all input images were used for that particular pixel). This corresponds to the value recorded in the header keywords
TEXPTIME
andEXPTIME
.
The following keywords are also derived from the variables in this equation:
TEXPTIME
=EXPTIME
= TSKYSUM
= ∑n skynREJ_RATE
= (∑n Tnmn(x,y)/Τ) averaged over all pixelsNCOMBINE
= n
The remaining keywords EXPSTART
, EXPEND
are updated based on the values corresponding to the first and last input images, respectively. In summary, the cosmic ray rejection task sums all non-rejected pixel values, computes the true exposure time for that pixel, and scales the sum to correspond to the total exposure time. The final scaled, cleaned pixel is written to the comparison image to be used for the next iteration. This process is then repeated with increasingly stringent detection thresholds, as specified by CRSIGMAS
.
Cosmic Ray Rejection Table
wf3rej uses the Cosmic Ray Rejection parameter table (CRREJTAB
) to determine the number of iterations for cosmic-ray rejection, the sigma levels to use for each iteration, and the spill radius to use during detection. This allows the rejection process to be tuned to each detector and observation, with suitable defaults being applied during pipeline processing. Observers may fine-tune the cosmic-ray rejection parameters when manually reprocessing data with wf3rej by editing the CRREJTAB
.
The CRREJTAB
reference file contains the basic parameters necessary for performing cosmic-ray rejection. The column names and default values for the CRREJTAB
are given in Table 3.6. The appropriate row is selected based on the chip being processed (CCDCHIP
), the number of images into which the exposure was split (CR-SPLIT
), and the exposure time of each CR-SPLIT
image (MEANEXP
). If an exact match is not found for the exposure time, the table row with the closest value is used. If the CR-SPLIT
value of the input images exceeds the values in the table, the table row with the largest CR-SPLIT
value will be used. The sky fitting algorithm is controlled by the parameter SKYSUB
, which can have values of ‘mode’, ‘mean’ or ‘none’. The ‘initial guess’ image is created using the median or minimum value of the input exposures, as specified by the value of INITGUES
.
Cosmic-ray detection requires the specification of a threshold above which a pixel value is considered a cosmic ray. This threshold was defined above and uses sigma rejection thresholds. The sigma threshold for each iteration and number of iterations corresponds to the CRSIGMAS
column values in the CRREJTAB
file. SCALENSE
is a multiplicative term (in percent) for the noise model and is given as scale in the threshold equation above. This term can be useful when the pointing of the telescope has changed by a small fraction of a pixel between images. Under such circumstances, the undersampling of the image by the detector will cause stars to be mistakenly rejected as cosmic rays if a scale noise term is not included. This is a crude but effective step taken to satisfy the maxim of ‘do no harm’. However, for cases in which there have been no image-to-image offsets or the image is locally well-sampled, this will unduly bias against rejecting cosmic rays.
Pixels within a given radius, CRRADIUS
, of a cosmic ray will also be treated as cosmic rays. A less stringent rejection threshold, CRTHRESH
, can be used for detecting pixels adjacent to a cosmic ray. As for CRSIGMAS
, CRTHRESH
is also given as a sigma value. If CRTHRESH
is exceeded, pixels within the defined radius of the cosmic ray will also be flagged. All pixels determined to be affected by a cosmic ray will have their DQ values set to 8192, as described in Table 3.6.
Table 3.6: Columns in the cosmic-ray rejection parameter table.
Column Name | Default Value | Contents |
---|---|---|
CRSPLIT | - | Number of exposures into which observation was split |
CCDCHIP | - | Chip to which this conversion applies |
MEANEXP | - | Average exposure time (sec) for each image |
SCALENSE | 30.0 | Multiplicative term (in percent) for the noise model |
INITGUES | minimum | Method for computing initial-guess image (minimum, median) |
SKYSUB | mode | Sky fitting algorithm (mode, none) |
CRSIGMAS | 6.5, 5.5, 4.5 | Rejection thresholds (sigma) |
CRRADIUS | 2.1 | Radius (in pixels) for propagating cosmic ray |
CRTHRESH | 0.5555 | Propagation factor |
BADINPDQ | 39 | Data quality file bits to reject |
CRMASK | yes | Flag CR-rejected pixels in input files? |
Displaying output from wf3rej in a Jupyter Notebook
When calling wf3rej from a Jupyter notebook, informational text output from the underlying wf3rej.e program will be passed through print as the calibration runs by default, and show up in the user’s cell output. This behavior can be customized by passing your own function as the log_func
keyword argument to wf3rej. As output is read from the underlying program, the wf3rej Python wrapper will call log_func
with the contents of each line (this provides a way to connect wf3rej to the Python logging system by passing the logging.debug function or similar).
If log_func=None
is passed, informational text output from the underlying program will be ignored, but the program’s exit code will still be checked for successful completion.
Input Parameters for the Python interface
Parameters input : str or list Name of input files, such as: - comma-separated (no spaces) filenames (iaao01k8q_flc.fits,iaao01k9q_flc.fits) - a Python list of filenames - a partial filename with wildcards (*flt.fits) - an at-file (@input) output : str Name of the output FITS file. crrejtab : str, default=”” Reference filename. scalense : float, default=0. Scale factor applied to noise. initgues : str, default=”” Initial value estimate scheme (min|med). skysub : str, default=”” How to compute the sky (none|mode|mean). crsigmas : str, default=”” Rejection levels in each iteration. crradius : float, default=0. Cosmic ray expansion radius in pixels. crthresh : float, default=0. Rejection propagation threshold. badinpdq : int, default=0 Data quality flag bits to reject. crmask : bool, default=Setting to be read from CRREJTAB. If argument is present, write the CR flag value to the input DQ images. shadcorr : bool, default=Setting to be read from SHADCORR keyword value in primary header of first image to process. If argument is present, perform shading shutter correction. verbose : bool, optional, default=False If True, print timestamps and other output. log_func : func(), default=print() If not specified, the print function is used for logging to facilitate use in the Jupyter notebook. Returns None Usage from wfc3tools import wf3rej from glob import glob infiles = glob("*flt.fits") wf3rej(infiles, "output.fits", verbose=True) wf3rej("*flt.fits", "output.fits", verbose=True) wf3rej("@input.lst", "output.fits", verbose=True)
Please see the warning at the bottom of the page regarding the parameter settings for wf3rej.e for more details as to the action taken when the parameters use their default values.
Command Line Options for the wf3rej C executable
wf3rej.e input output [-r] [-v] [-t] [-shadcorr] [-crmask] [-table <filename>] [-scale <float>] [-init <med|min>] [-sky <none|mode|mean>] [-sigmas <string>] [-radius <float>] [-thresh <float>] [-pdq <short>]
Example - Process data with timestamps and a custom cosmic ray rejection table:
wf3rej.e iaao01k8q_flc.fits,iaao01k9q_flc.fits output.fits -t
Example - Print the code version and exit:
wf3rej.e -r input : comma-separated list of strings - Input filenames as a list of comma-separated input names - ipppssoot_raw.fits,ipppssoot_raw.fits (Note: Do not include any blank spaces.) output : str Name of output filename options -r : print version number/date of software and exit (no other options selected) -v : verbose mode -t : print the timestamps -shadcorr : perform shading shutter correction (not used) -crmask : set CR flags in input DQ images -table <filename>: string, the crrejtab filename -scale <number>: float, scale factor for noise -init <med|min>: string, initial value estimate scheme -sky <none|mode|mean>: string, method to compute sky -sigmas <string of numbers>: string, rejection levels for each iteration (e.g., "3.5,4.5,5.5") -radius <number>: float, CR expansion radius -thresh <number> : float, rejection propagation threshold -pdq <number>: short, data quality flag bits to reject
Observers should not invoke the shadcorr
option as there is no SHADFILE.
Including the crmask
option on the command line will have the CR flag values placed into the DQ extension of the input images. Omitting this option does not turn off the insertion, but rather the program will follow the default setting for the crmask
option as indicated in the CRREJTAB
calibration file.
If any of the CR-rejection options are omitted from the command line, default values from CRREJTAB
will be used; those options are: crmask,scale,init,sky,sigmas,radius,thresh,and pdq
. In verbose mode, all of the option values are printed to the output logfile.
-
WFC3 Data Handbook
- • Acknowledgments
- • What's New in This Revision
- Preface
- Chapter 1: WFC3 Instruments
- Chapter 2: WFC3 Data Structure
- Chapter 3: WFC3 Data Calibration
- Chapter 4: WFC3 Images: Distortion Correction and AstroDrizzle
- Chapter 5: WFC3 UVIS Sources of Error
- Chapter 6: WFC3 UVIS Charge Transfer Efficiency - CTE
-
Chapter 7: WFC3 IR Sources of Error
- • 7.1 WFC3 IR Error Source Overview
- • 7.2 Gain
- • 7.3 WFC3 IR Bias Correction
- • 7.4 WFC3 Dark Current and Banding
- • 7.5 Blobs
- • 7.6 Detector Nonlinearity Issues
- • 7.7 Count Rate Non-Linearity
- • 7.8 IR Flat Fields
- • 7.9 Pixel Defects and Bad Imaging Regions
- • 7.10 Time-Variable Background
- • 7.11 IR Photometry Errors
- • 7.12 References
- Chapter 8: Persistence in WFC3 IR
- Chapter 9: WFC3 Data Analysis
- Chapter 10: WFC3 Spatial Scan Data