6.3 Running AstroDrizzle
Reprocessing images with AstroDrizzle requires consideration of the science to be performed on the images, as well as the field of view in the images, in order to determine the optimal set of parameters. This section provides some considerations on key aspects of the AstroDrizzle processing that can help guide the user in deciding on the best way to reprocess their images.
Sky Subtraction Considerations
Many astronomical fields of view cover parts of the sky devoid of any large objects and as a result the default sky subtraction performed by AstroDrizzle will generally work well enough without much modification needed. Incorrect sky subtraction by AstroDrizzle can slightly bias the cosmic ray identification, but more noticeably result in each input image showing up as a tile with distinct edges in the final combined output image. Sky subtraction will be turned off in the pipeline when processing any narrow-band exposures as they typically do not detect enough background to affect the final drizzle product.
The sky subtraction step can also be biased by the presence of large extended sources in the field of view. If the extended source does not cover the entire field of view, it may be possible to simply change the sky computation to use the mode instead of the default clipped median as specified by the skystat parameter. Any observation where no true background (sky) pixels have been observed due to the presence of an extended source filling the field of view will almost certainly require that the sky subtraction step be turned off.
AstroDrizzle does support a mode where the user can either compute a custom value for the sky or perform sky subtraction prior to running AstroDrizzle and still account for an average sky value. A custom sky value can be computed and added to each image's science (SCI) header as a new keyword of the user's choosing. The name of this keyword can then be provided to AstroDrizzle through use of the skyuser parameter with sky subtraction turned on. This value will then be used by AstroDrizzle when performing cosmic ray identification but will not be applied during drizzling as is usually done with sky subtraction.
These custom values will then be copied into the MDRIZSKY keyword as a record to indicate it was used during processing. This allows the user to apply their own custom sky-subtraction. The user may find they want a record of the custom sky subtraction values applied to each image, in which case, a list with a custom sky value for each chip could be written to a file. This file, tagged by the skyfile parameter, would also allow the user to specify whether or not these values have been used to sky-subtract the input images prior to processing by AstroDrizzle. If they were not applied, AstroDrizzle would use the values from the file for sky subtraction when drizzling the data.
Further information on sky matching can be found in the Jupyter Notebook on Sky Matching.
The precise value chosen for the sky is rarely a concern; in general one will subtract the sky from around an object when doing photometry on it. What is crucial in Drizzle is that the sky from the various images be the same. This is because data from the images being combined goes into different pixels with different weights. Therefore, if the sky values of the input images are not the same, the drizzling can create artificial sky noise. If a user does the sky subtraction by themself, then they must be sure the images are left with sky values that match.
Cosmic Ray Rejection
AstroDrizzle processes each of the input images separately onto the same output frame and then creates a median of these images to produce an image largely free of bad pixels. This median image is "blotted" or interpolated back to the frame of each of the input images and compared with the inputs to determine the locations of bad pixels which are not included in the pixel masks created by the instrument groups.
By default, the pipeline drizzles both the intermediate images and the final image onto an output frame with pixels that are the same size as the input pixels. In most images, the vast majority of these bad pixels are caused by cosmic rays. Using a finer output pixel scale than the default can often given a better interpolation, and thus a more accurate cosmic ray removal. Therefore the same rules that are discussed in the next section for choosing a final pixel scale should be considered for choosing the pixel scale in the cosmic ray removal step. Choosing a small pixfrac, however, is less crucial, and indeed less desirable, as a larger pixfrac gives more images from which to determine a median. So in general, the user may wish to keep driz_sep_pixfrac = 1.0.
Cosmic ray rejection in the pipeline also relies on the original header astrometry. If that astrometry is not accurate (to about 0.1 pixels or better) then the cosmic ray rejection will be compromised and, in particular, pixels in stars may be incorrectly marked as cosmic rays. One way to check to see if this has occurred is to compare the data quality (DQ) and image (SCI) extensions from the same flt.fits file. If pixels on stars (usually the brightest pixel) are frequently marked as bad, then the pipeline had a problem with the dataset.
Correctly aligning the images and using a smaller output scale will usually solve this problem. In some cases the user may find it necessary to adjust the driz_cr_scale and driz_cr_snr parameters. These adjust how sensitive the cosmic ray rejection algorithm is to differences between the blotted median and the image. In particular, driz_cr_scale is a fudge factor that multiplies the local derivative of the blotted image. This is added to the calculated statistical noise in the pixel to create a new (larger) estimate of the noise, making it less likely that a pixel will be marked as bad.
Selecting the Optimal Scale and Pixfrac
In combining images, the Drizzle algorithm maps pixels from input images into pixels in a sub-sampled output image, taking into account shifts and rotations between input images, as well as the geometric distortion of the images. However, to avoid re-convolving the output image with the large pixel "footprint" of the input images, Astrodrizzle allows users to "shrink" the input pixel before it is assigned to a location in the output image.
The shrunken pixels, called "drops," can be imagined as raining down on the sub-sampled output image. The input pixel's value is applied to an output pixel with a weight proportional to the area of overlap between the "drop" and the output pixel. Sub-sampling in the output image is set in AstroDrizzle using the final_scale parameter, which is in units of arcseconds. The drop size is controlled in AstroDrizzle by the parameter final_pixfrac, which is the ratio of the drop's linear size to the input pixel's linear size before geometric distortion corrections.
There is no single setting of these parameters that is optimal for all observations and scientific goals. With only a few images, a large pixfrac will make sure that all of the output image is well-covered. Even in this case, however, an output final_scale which is smaller than the input pixel is usually preferable. A common exception is the single image case, where the goal is to primarily remove distortion from the image. In this case one may want to set both final_pixfrac and final_scale to the original image size and use the Lanczos kernel. This kernel is very good in preserving the PSF; however it introduces strong artifacts around bad pixels and cosmic rays.
One way to test for good final coverage of the output pixels is to examine the final weight map and make sure that the ratio of the standard deviation of the weights in a typical area of the image does not go significantly below 0.2. As users generally do not employ the weight map in final photometry, this insures that ignoring the weight map will not significantly increase the noise of the final result.
Using an even finer scale or pixfrac may be advantageous with large numbers of images to obtain the best possible SNR on point sources. The PSF is convolved in quadrature with both the final scale and pixfrac. Thus, the Hubble Ultra Deep Field (Beckwith et al. 2006) used a scale equal to 0.4 of the original pixel and a pixfrac of zero! Since the addition of the effect is in quadrature and the original pixel and PSF are substantially larger than the pixfrac, the main effect of using such a small pixfrac is to entirely eliminate correlated noise. However, using a smaller pixfrac also reduces the ability of the eye to detect low surface brightness features in the image. In practice many users may find convenience outweighs perfection. In particular, using an output pixel size of 0.03333 arcseconds for ACS and WFC3 is generally sufficiently small to give fine resolution but has the benefit that three pixels are just about 0.1 arcseconds across.
A final output pixel size of 0.06666 arcseconds for the WFC3/IR is closer to one half of an original pixel and has the advantage that three pixels are 0.2 arcseconds across. Again users with four or fewer dithers will probably want to keep their pixfrac >~0.7, but are free to experiment. To summarize, when experimenting with pixfrac and scale users should keep these points in mind:
- For sub-pixel dithered data, select an output scale that's smaller than the native scale. It will even help in the cosmic ray rejection step.
- A smaller final_pixfrac gives higher resolution and lower correlated noise, but also reduces sensitivity to low-surface brightness features (though it is possible to convolve a high resolution image later to go after low surface brightness features).
- Keep the standard deviation of the weight map over the main part of the image to above ~0.3 of the mean to insure that one does not lose significant signal-to-noise in ignoring the weight map in final photometry.
Controlling the Bit Mask
Data Quality Flags
Data quality flags which were set during image calibration can be used as bit masks when drizzling. These specific DQ flag values are unique for each detector and are defined in the Instrument Data Handbooks.
AstroDrizzle will mask all pixels with non-zero DQ flag values, unless otherwise specified. When a pixel is flagged in one image but has a corresponding un-flagged (good) pixel in any other image in the stack, that pixel will be replaced with the "good" value during image combination. Otherwise, the pixel will be replaced with a "fill value" specified by the user. If the "fill value" is set to INDEF, and if there are no "good" pixels in the stack, the pixel will retain its original value but show zero weight in the drizzled weight image.
While the calibration pipeline assigns DQ flags to a large fraction of pixels, the science quality of those pixels may not be compromised. Choice of which pixels to treat as "good" or "bad" depends largely on the number of images being combined and the dithering strategy. The user may override flags in the DQ array by specifying which bit values should be considered as "good" for inclusion in the final image combination. Control over these DQ flags is specified by parameter values in both the "single-drizzle" step and in the "final drizzle" step. These parameters tell AstroDrizzle which "suspect" pixels in the DQ array to keep. This is particularly useful when only a handful of images are combined and excessive flagging compromises the final product. The drizzled weight image should be carefully examined to get an idea of how many input pixels contributed to each output pixel. When the weight image has a significant number of "holes" where no valid input pixel was available, the user may need to re-evaluate the stringency of selecting which DQ bits should be considered as "good" and adjust the parameters accordingly.
During pipeline processing, values for driz_sep_bits and final_bits are specified in the MDRIZTAB reference file.
In early 2017, the ACS instrument team changed the definition of data quality (DQ) flags populated in the calibrated FLT/FLC files. New calibration techniques now make it possible discern between unstable and stable hot pixels, the later of which are corrected by ‘calacs’ when subtracting the dark. Thus, pixels identified as hot and stable (DQ flag=16) may now be treated as 'good' data when drizzling, and those identified as unstable (DQ flag=32) should be treated as 'bad'. A new MDRIZTAB reference table (16r12191j_mdz.fits) was delivered in June 2017 and contains a set of default parameters for combining exposures with AstroDrizzle. With changes to the DQ flag definitions, the parameters driz_sep_bits and final_bits, which define DQ flags for drizzle to ignore (e.g. to treat as good), are now set to a value of 336 (the sum of 16+64+256) so that stable hot pixels, warm pixels, and full-well saturated pixels will not be rejected when combining exposures. For details, see ACS ISR 2017-05.
The WFC3 instrument team implemented a similar change to the DQ flag definitions in December 2018, and an updated MDRIZTAB reference file (2ck18260i_mdz.fits) reflects the new recommended drizzle parameter settings such that DQ flag values 16, 64, and 256 are treated as good pixels. These new flags are valid for UVIS observations obtained after Nov 08 2012, when the dark calibration program began using post-flash to mitigate hot pixel trailing due to poor charge transfer efficiency at low background levels. A description of the new UVIS bad pixel tables is described in WFC3 ISR 2018-15.
AstroDrizzle was designed for use with multiple instruments, so the default value for the driz_sep_bits and final_bits parameters are set to zero for offline reprocessing. This assumes that all pixels flagged in the DQ array are "bad" pixels and will be masked. When only a few images are being combined, this may be an overly aggressive approach. The user can decide the best strategy for assigning these parameters based on the number of input frames and the dither pattern used. Ultimately, one wants to avoid having too many pixels with no "good" input pixel in the stack of images used to create the final product.
Note that in AstroDrizzle these parameters may be given as either the sum of the DQ flags or as a comma-separated list.
Drizzled Masks
While running AstroDrizzle, several sets of masks are created for each image during processing.
The first is called the "single" mask, where every pixel flagged in the input image DQ array is assigned a value of zero. All other pixels are assumed to be good and will be assigned a value of 1. The single mask image is used in the "driz_separate step" to create the single_sci.fits images which are used as input for creating the median image. The user may tell AstroDrizzle to ignore specific input image DQ flag values by specifying them in the parameter driz_sep_bits.
The second mask image is called the "final" mask. The final mask is used in the "driz_combine" step to create the final drizzled product. The parameter final_bits allows the user to tell AstroDrizzle which input image DQ flags to ignore for the final image combination. When this parameter is left to the default value of zero, every pixel flagged in the input image DQ array is assigned a value of zero in the mask.
These two mask files are created during software initialization and are subsequently updated when running AstroDrizzle steps to compute the static mask and the cosmic ray masks. The static mask is computed to correct unusually low pixels which were over-subtracted when applying the dark image calibration. Note that this mask is not written to disk but retained in memory only. The static mask is combined with the original single and final masks to create updated versions of these images.
The cosmic ray mask is computed when the parameter driz_cr_corr is True, and is written to a file called *_crmask.fits. The cosmic ray mask should be inspected and blinked with the original input image to verify the quality of the rejection. Note that during pipeline processing AstroDrizzle flags potential cosmic rays with a value of 4096 in the input image DQ array when combining data which is part of an association. During reprocessing, the parameter resetbits may be used to reset the input image DQ bit value from 4096 to zero (designating them as good pixels), so that cosmic rays can be re-identified using user-specified parameter values in AstroDrizzle. The specific value of the flag may be defined via the crbit parameter, if desired. These new optimized cosmic ray flags will be used to update the DQ array of the original input image (flt.fits or flc.fits).
During the final drizzle combination the static mask and the cosmic ray mask are combined with pixels selected as "bad" in the input flt.fits DQ arrays (the final_bits parameter specifies which input image DQ pixels should be treated as "good.") The resulting "master" mask is then used during the final image drizzle-combination step to create the drizzled product *_drz.fits.