8.2 Evaluating the Amount of Persistence in Science Images
As described above, persistence in an image depends on images that have been obtained previously. However, when an observer retrieves his/her data, the observations that came before will most likely be proprietary and inaccessible to the observer. To address this difficulty, the WFC3 team has developed a set of prototype software that estimates the persistence in an observer’s image. Although not currently part of the standard calibration pipeline, the WFC3 team runs these persistence estimation tools on all WFC3 IR images. The tools produce an html file summarizing the amount of persistence in the image as well a several FITS files that can be used in downstream analysis of the HST data.
The output products, which consist of a combination of html files and a number of FITS files, can be retrieved by observes through MAST using a special screen located at:
The query screen is shown in Figure 8.3.
Normally, these output data products are available a few weeks after an IR observation has taken place.
There are two types of persistence that generally affect data analysis differently. We define external persistence as persistence that is generated by an earlier visit, and internal persistence as persistence generated within the same visit as the image in question. External persistence is not within the control of the observer, and can appear anywhere in an IR image. Internal persistence is usually less important, since at least in the case of small dithers, it affects regions of the detector which are in the wings of the point spread function of bright sources. The results of a query from the persistence query screen for Visit H of program ID 11189 are shown in Figure 8.4. The columns EXT1, EXT2 and EXT3 indicate that 0.3%, 1.89% and 6.83% of the pixels were predicted to have external persistence exceeding 0.1, 0.03, and 0.01 e¯/sec. The columns labeled TOT1, TOT2, and TOT3 provide the percentages for the sum of external and internal persistence. If all of the estimates of persistence for a dataset in the results screen, particularly those having to do with external persistence, are zero then one should not need to investigate further.
Observers should check for persistence as part of their evaluation of the science data quality of their observations. Observers should do so early enough that they have an opportunity to submit a Hubble Observation Problem Report (HOPR) if the data are so compromised by the existence of external persistence in their images that the science they proposed cannot be carried out. Data analyzers should also check to determine whether they need to take persistence into account in analysis.
If the summary estimates in the results screen indicate there is persistence, then more information about the amount of persistence and its location in the images can be obtained by clicking on the links associated with data set column in the results screen. (If the data are still proprietary, you will be asked for your MAST login name and password at this point). This will bring up a page that contains additional text as well as a number of images showing where persistence is predicted. There are also various thumbnail images, showing how well the persistence is removed by the model in selected areas of an image. The utility of these thumbnails varies, depending upon the crowding of the image with real sources. An algorithm is used to try to select regions of the image which have the largest amounts of persistence but which are not very crowded, so that it is easy to establish the sky level and automatically generate thumbnails. But this algorithm is not very sophisticated, and in many cases, all of the regions with significant persistence lie in regions of the image that contain brighter *real* objects. Nevertheless, inspecting the images is recommended, and should help one to determine, qualitatively, whether persistence is likely to affect the science one is attempting to carry out.
Table 8.1: Files Associated with Persistence Correction
Prediction of external persistence
Prediction of total persistence
flt file with persistence model subtracted
WFC3 Data Handbook
- • Acknowledgments
- Chapter 1: WFC3 Instruments
- Chapter 2: WFC3 Data Structure
- Chapter 3: WFC3 Data Calibration
- Chapter 4: WFC3 Images: Distortion Correction and AstroDrizzle
- Chapter 5: WFC3-UVIS Sources of Error
- Chapter 6: WFC3 UVIS Charge Transfer Efficiency - CTE
Chapter 7: WFC3 IR Sources of Error
- • 7.1 WFC3 IR Error Source Overview
- • 7.2 Gain
- • 7.3 WFC3 IR Bias Correction
- • 7.4 WFC3 Dark Current and Banding
- • 7.5 Blobs
- • 7.6 Detector Nonlinearity Issues
- • 7.7 Count Rate Non-Linearity
- • 7.8 IR Flat Fields
- • 7.9 Pixel Defects and Bad Imaging Regions
- • 7.10 Time Variable Background Contamination
- • 7.11 IR Photometry Errors
- • 7.12 References
- Chapter 8: Persistence in WFC3 IR
- Chapter 9: WFC3 Data Analysis
- Chapter 10: WFC3 Spatial Scan Data