8.2 Evaluating the Amount of Persistence in Science Images

As described above, persistence in a WFC3 IR image depends on images that have been obtained previously. However, when an observer retrieves data, the observations that came before will most likely be proprietary and inaccessible to the observer. To address this difficulty, the WFC3 team has developed software that estimates the persistence in an observer’s image (WFC3 ISR 2015-15 and https://github.com/kslong/Persistence). Although not currently part of the standard calibration pipeline, the WFC3 team runs these persistence estimation tools on all WFC3 IR images. The tools produce an HTML file summarizing the amount of persistence in each image as well as several FITS files that can be used by observers to analyze the impact of persistence in their own images. Observers can retrieve the output products via MAST using a special screen shown in Figure 8.3 and located at http://archive.stsci.edu/prepds/persist/search.php. As an exercise to illustrate the tool's functionality, the reader can type "ia21h2e9q" into the box under "Dataset" then click "Search": a new browser tab will appear with a list; clicking on the hyperlink "ia21h2e9q" in the left column will open another HTML file with the available persistence information, including a graphic for each instance of persistence predicted in the image.

Normally, these output data products are available within few weeks after an IR observation has taken place.

There are two types of persistence that generally affect data analysis. We define external persistence as persistence that is generated by an earlier visit, and internal persistence as persistence generated within the same visit as the image in question. External persistence is not within the control of the observer, and can appear anywhere in an IR image. Internal persistence is usually less important, since at least in the case of small dithers, it affects regions of the detector which are in the wings of the point spread function of bright sources. The results of a query from the persistence query screen for Visit H of program ID 11189 are shown in Figure 8.4. The columns EXT1, EXT2 and EXT3 indicate that 0.3%, 1.89% and 6.83% of the pixels were predicted to have external persistence exceeding 0.1, 0.03, and 0.01 e¯/sec. The columns labeled TOT1, TOT2, and TOT3 provide the percentages for the sum of external and internal persistence. If all of the estimates of persistence for a dataset in the results screen, particularly those having to do with external persistence, are zero then one should not need to investigate further.

Observers should check for persistence as part of their evaluation of the science data quality of their observations and do so early enough that they are able to submit a Hubble Observation Problem Report (HOPR) if their data are so compromised by external persistence that the proposed science cannot be achieved. Filing a HOPR for persistence is rare; more often, investigators must determine whether they need to take persistence into account during their analysis.

If the summary estimates in the results screen indicate there is persistence, then more information about the amount of persistence and its location in the images can be obtained by clicking on the links associated with data set column in the results screen (if the data are still proprietary, you will be asked for your MAST login name and password at this point). This will bring up a page that contains additional text as well as a number of images showing where persistence is predicted. There are also various thumbnail images, showing how well the persistence is removed by the model in selected areas of an image. The utility of these thumbnails varies, depending upon the crowding of the image with real sources. An algorithm is used to try to select regions of the image which have the largest amounts of persistence but which are not very crowded, so that it is easy to establish the sky level and automatically generate thumbnails. But this algorithm is not very sophisticated, and in many cases, all of the regions with significant persistence lie in regions of the image that contain brighter *real* objects. Nevertheless, inspecting the images is recommended, and should help one to determine, qualitatively, whether persistence is likely to affect the science goals.

The persistence products software also produces several FITS files (summarized in Table 8.1) for each flt file in a dataset. The *_extper.fits and *_persist.fits files contain predictions of the external and total persistence in a dataset. The *_flt_cor.fits file is an flt file with the total persistence subtracted. These files can be retrieved from MAST as a tar.gz file by clicking one of the Visit links (see Visit column in Figure 8.4); each Visit must be retrieved separately. In situations where there is some persistence predicted in a science image, comparing these persistence product FITS files to the original data in DS9 or some other image display is ultimately the best way to determine whether persistence needs to be taken into account in the analysis of the data.

Table 8.1: Files Associated with Persistence Correction