8.3 Mitigating the Effects of Persistence

Two possible ways to mitigate persistence are to 1) exclude the affected pixels from the analysis, or 2) subtract an estimate of the persistence signal from the data and use the modified data for the analysis.

Simple procedures can be used to mark compromised pixels and modify the data quality extensions of the flt files. Typically one would use the persist.fits products (described in the previous section) to define an acceptable level of persistence, e.g. 0.01 e¯/sec, based upon a visual comparison of the data and the corresponding persistence images. The choice of level should be based on the science objective and the fraction of pixels impacted. The selected level can be used to flag all pixels in the persist.fits image that exceed predicted persistence values. 

This method is typically very effective at flagging persistence from previous observations in staring mode. However, the “A-gamma” model predictions can differ somewhat from the observed amount of persistence for images taken shortly before the image of interest (short-term persistence) causing a mismatch between the estimated internal persistence and what is observed. In a similar fashion, when persistence is from prior spatially scanned observations, the model significantly underestimates the level of persistence observed in the data. The “A-gamma” model relies on fluence estimates from the flt images of prior observations, i.e. implicitly assumes that the estimated flux in the flt accurately reflects the fluence absorbed by specific pixels, which is not the case for scanned observations. The CRCORR step of calwf3 works by fitting a line to the accumulating charge between reads, but for scan-mode observations, the flux is only accumulating in a very short interval, thus the pipeline is unable to report the true flux (Section 10.2). This is true not only for the scans but also for other very rapidly moving targets such as like asteroid trails (WFC3 ISR 2017-18) or bright light sources (e.g. cities) that streak across some of the dark-Earth IR flats used to monitor the IR blobs (Section 7.5). Persistence in those cases is very poorly estimated and can be inaccurate by orders of magnitude.

Note, however, that for both internal persistence and for persistence from moving targets, the “A-gamma” model does predict the location of persistence on the detector very well.  Therefore, the user is advised to tune the tolerance level to account for possible mismatches if moving targets are the suspected causes of persistence.

Once the pixels are flagged, down-stream analysis proceeds as it normally would, assuming the tools that are used take the data quality flags into account. Persistence is a property of the pixel: as long as the observer who planned the original observations created a set of appropriately-dithered images, using this procedure to flag bad-persistence pixels should not significantly impact the science.

For example, using AstroDrizzle to exclude those flagged pixels would result in a combined image that is largely cleansed of the effects of persistence. In some cases, the threshold defined from inspection of the flt.fits may not be low enough to detect lower-level persistence, so the deeper, combined drz.fits images may be used to verify whether persistence was fully removed.

An alternative method is to use the persistence-subtracted files in down-stream analysis. If one adopts this approach, one should carefully inspect the _flt_cor.fits corrected data products (produced by simply subtracting the persist.fits model directly from the flt.fits image) to evaluate how well persistence has been subtracted from the image. Observers should inspect all of the images in a particular science observation as the persistence signal will decline with time. Typically, about 90% of the persistence signature is removed by the model. Because there are variations in persistence which are still being studied, the corrected data products can be either under- or over-subtracted, particularly in the case of internal persistence where the power law model is not optimal for estimating the short term decay (WFC3 ISR 2019-02).

The persistence-subtracted image of our example is shown in Figure 8.5. The resulting image is much cleaner, especially at the center of the image where the γ-ray burst had occurred. However, a careful inspection shows some residual signatures of the persistence, particularly due to the set of early images that produced the horizontally-stepped faint features in the original image.

If persistence remains a significant problem in the image analysis in spite of the approaches suggested above, please contact the help desk (http://hsthelp.stsci.edu). It is possible that by changing some of the parameters in the model from their nominal values, better estimates of the persistence in an flt file can be obtained, particularly in situations where several earlier visits have affected a science image.

In all cases when dealing with persistence, as is true of most problems associated with data analysis, first assess the severity of the problem and then choose a method of handling the issue that is consistent with the science being carried out. Users concerned about the effects of persistence on their data should check for updates on the WFC3 Persistence pages as well as the main WFC3 web page.