2.1 Types of ACS Files
2.1.1 Data Files and Suffixes
File suffixes for ACS data products are given in Table 2.1, and are described below.
- Initial input files to the calibration pipeline:
- Raw (raw.fits) files from Generic Conversion.
- If applicable, an association table (
asn.fits) for a complete "observation set."1
- A single fully-calibrated MAMA image is given the suffix
flt.fits. (MAMA images do not have an overscan region, and they are not affected by cosmic rays and CTE.)
- Calibration of a single CCD image:
- After the bias image and bias level are subtracted, and the overscan regions trimmed, a temporary file, with the suffix
blv_tmp.fits, is created. (By default, CTE-corrected images are also created for WFC data so calacs creates a temporary file with suffix
- Upon completion of additional calibration steps (dark subtraction, flat-fielding, etc.), the temporary file is renamed with the suffix
flt.fits. For WFC data, CTE-corrected images, with suffix
flc.fits, are also created. The
flc.fits files will later serve as input for AstroDrizzle.
- A "CR-SPLIT" CCD observation:
- Raw images from a "CR-SPLIT" observation undergo bias image and bias level subtraction. These images are then combined at the cosmic ray rejection step in calacs to create a temporary image with suffix
crj_tmp.fits. For WFC images with CTE corrections, a file with suffix is
crc_tmp.fits is also created.
- Other basic calibrations are performed on the temporary combined image. It is then renamed with the
crj.fits suffix, and
crc.fits suffix for WFC CTE-corrected images.
- Individual calibrated images (
flt.fits/flc.fits) are also created for each exposure in the "CR-SPLIT" observation.
- Multiple CCD exposures (e.g., CR-SPLIT):
- Follows the same procedure described for "CR-SPLIT" CCD observations.
- A MAMA observation consisting of several repeated sub-exposures:
flt.fits images are created for each sub-exposure.
- A summed flat-fielded image is created, with suffix
Table 2.1: ACS File Suffixes
Raw uncalibrated image from a single exposure.
Association table for an observation set.
Telemetry and engineering data.
Trailer file containing calacs processing comments. (This is the same as
Overscan-trimmed individual exposure (these will be renamed
CR-rejected combined image created using blv_tmp.fits and
Calibrated, flat-fielded individual exposure. CTE-corrected images with suffix
Calibrated and combined image, with CR rejection (using images from "CR-SPLIT" sub-exposures). The CTE-corrected version, for WFC images, has suffix
Calibrated and summed MAMA image (no CR rejection needed) created from sub-exposures in an observation.
Calibrated, geometrically-corrected, dither-combined image (created by AstroDrizzle, which is not a part of calacs). The CTE-corrected version, for WFC images, has suffix
Intermediate calibrated products created by calacs, such as
sfl.fits, blv_tmp.fits, blc_tmp.fits, crj_tmp.fits, crc_tmp.fits, crj.fits and
crc.fits, are, by default, not delivered by the Archive.
Standard calibrated files delivered from the Archive contains these extensions:
asn.fits, spt.fits, trl.fits, flt.fits, and
drz.fits. For WFC images, flc.fits and drc.fits are also delivered.
To obtain intermediate calacs products from the Archive, enter specific extensions (e.g.,
crj, crj_tmp) in a field titled " File Extensions Requested" near the bottom of the Archive data request Web page.
2.1.2 Association Tables
Association tables describe and track the relationship or "associations" between data products for a set of observations. Such relationships include repeated exposures in observations, "CR-SPLIT" observations, and dithered observations. ACS association tables can be used to instruct calacs to create different levels of calibration products. These tables are particularly useful for keeping track of complex observations, for example, an observation at a specific dither position that may be additionally split into multiple exposures. Edited association tables can also be used with calacs to create non-default calibration products (see Example 3 in Section 3.5).
ACS data files are given the following definitions:
- A single image from an exposure or sub-exposure is the "atomic unit" of HST data.
- A dataset is a collection of files having a common rootname (first nine characters of the image name).
- A sub-product is created by combining a subset of the exposures in an association.
- A product is created by combining sub-products, or in some cases, individual exposures (before they were incorporated into a sub-product), of an association
An ACS association table has three primary columns: MEMNAME, MEMTYPE, and MEMPRSNT. (An example of an association table is shown in Table 2.3.)
- The column MEMNAME lists the name of each exposure in the association and names of calacsoutput products.
- The column MEMTYPE describes the role of a file in the association. A unique set of MEMTYPES, specific to ACS, were adopted to provide descriptions for multiple products. These types are summarized in Table 2.2.
- The MEMPRSNT column indicates the calibration status of each product.
Table 2.2: Exposure Types, or MEMTYPEs, in ACS Associations
An image that is part of a "CR-SPLIT" observation. ("EXP-CRJ" is used when there is only one "CR-SPLIT" observation in an association.)
Same as "EXP-CRJ," but used when there are multiple "CR-SPLIT" observations in an association. n is a numerical identification for each "CR-SPLIT" set (e.g., "EXP-CR1," "EXP-CR2").
A calibrated and CR-rejected combined image created from a "CR-SPLIT" observation. ("PROD-CRJ" is used when there is only one "CR-SPLIT" observation in an association.)
Same as "PROD-CRJ" but used when there are multiple "CR-SPLIT" observations in an association. n is a numerical identification for each "CR-SPLIT" set and its corresponding PROD-CRn combined image (e.g., "PROD-CR1," "PROD-CR2").
An image that is part of an observation consisting of several repeated sub-exposures. ("EXP-RPT" is used when there is only one such observation in an association.)
Same as "EXP-RPT," but used when there are multiple observations in an association, each containing a series of repeated sub-exposures. n is a numerical identification for each such observation (e.g., "EXP-RP1," "EXP-RP2").
A calibrated summed MAMA image, created from an observation containing a series of repeated sub-exposures. ("PROD-RPT" is used when there is only one such observation in an association.)
Same as "PROD-RPT," but used when there are multiple observations in an association, each containing a series of repeated sub-exposures. n is a numerical identification for each such observation and its corresponding summed PROD-RPTn image (e.g., "PROD-RPT1," "PROD-RPT-2").
An image from an observation that is part of a dither pattern.
A dither-combined output product.
An example of an association table is shown in Table 2.3. But first, to trace back its origins, begin by looking at the Phase II proposal commands that created the data. This example came from proposal 10605, visit 1, exposure log sheet number 1. The observations, using ACS/WFC, were taken as a two-point dither with a "CR-SPLIT=2" at each dither point. The proposal's dither pattern specification looks like this:
The exposure log sheet commands used to execute these observations were as follows:
A standard calibrated data retrieval from the Archive for images taken by Visit 1, exposure 1, are listed below. (Intermediate calacs products from the Archive have to be specifically requested, therefore, the
crc.fits files are not included in the delivery.)
There are two pairs of 4 single calibrated images, one pair with suffix
flt.fits, and the other, corrected for CTE, with suffix
- From the "CR-SPLIT" sub-exposures at dither point 1:
j9cm01jwq_flt.fits and j9cm01jwq_flc.fits
- From the "CR-SPLIT" sub-exposures at dither point 2:
j9cm01k2q_flt.fits and j9cm01k2q_flc.fits
j9cm01k4q_flt.fits and j9cm01k4q_flc.fits
- The combined image created by
AstroDrizzle, in the pipeline, from four
flc.fitsimages, respectively, are:
Table 2.3: Contents of Association Table, j9cm01010_asn.fits
- Rows 1 and 2 describe the association's first "CR-SPLIT" observation at dither point 1.
- "MEMNAME" shows the image rootnames for two sub-exposures in the first "CR-SPLIT" observation.
- "MEMTYPE" of "EXP-CR1" means that the images came from the association's first "CR-SPLIT" observation.
- "MEMPRSNT" set to "yes" indicates that those images underwent standard calibrations in calacs.
- Rows 3 and 4 describe the association's second "CR-SPLIT" observation at dither point 2. The two images have rootnames "J9CM01K2Q" and "J9CM01K4Q." "EXP-CR2" means they came from the second CR-SPLIT observation, and "yes" indicates that those images underwent standard calibrations in calacs.
- In row 5, "PROD-DTH" and "yes" indicate that the single exposure images in the association were drizzle-combined in the pipeline by AstroDrizzle, and named with the rootname specified in the MEMNAME column. In this example, four
flt.fitsimages were drizzled-combined to produce
j9cm01010_drz.fits, and four
flc.fitsimages were drizzle-combined to create
j9cm01010_drc.fits. (If "MEMPRSNT" had been set to "no," the
AstroDrizzlestep would have been omitted in the pipeline.)
- In Row 6, "PROD-CR1" and "yes" indicate that the two images from the first "CR-SPLIT" (rows 1 and 2) were combined with cosmic ray rejection by calacs to create an image with the rootname specified in the MEMNAME column. This was done for
flc.fitsimages to create the files
- In Row 7, "PROD-CR2" and "yes" indicate that the two images from the second "CR-SPLIT" (rows 3 and 4) were combined with cosmic ray rejection by calacs to create an image with the rootname specified in the MEMNAME column. This was done for
flc.fits images to create the files
2.1.3 Trailer Files
Each task in the calacs package creates messages that describe the progress of the calibration; these messages are directed to STDOUT (STanDard OUTput), which simply means that processing messages appear on the screen during the calacs run.
In pipeline processing for first and second generation HST instruments, where data files were calibrated one at a time, trailer files were created by simply redirecting the STDOUT contents to a file. However, the ACS pipeline was designed to calibrate several images at a time (like those described in Section 2.1.2), and create different types of output files. Therefore, each task within the calacs package must decide how to populate the trailer files associated with each product.
calacs will always overwrite information in existing trailer files from previous runs of calacs while preserving any comments generated by Generic Conversion. This ensures that the trailer files accurately reflect the most recent processing performed by calacs. After the Generic Conversion entries, the string "CALACSBEG" marks the first comment added to a trailer file by calacs. If the trailer file already exists, calacs searches for this string, then begins to write new processing comments from that point onwards, over-writing previous calacs comments. If "CALACSBEG" is not found in an existing trailer file, calacs will write that string at the end of the trailer file, then continue populating the trailer file with calibration processing comments.
As each image is reprocessed, an accompanying trailer text file with the suffix "
tra" (without the
.fits extension) is created. (The
trl.fits file from the Archive has the same content, in FITS table format.)
Following the processing hierarchy specified in the association table, information in trailer files belonging to images used for creating a higher level product will be included in the trailer file of that higher level product. In other words, the trailer file for any product processed by the pipeline will contain processing comments from trailers belonging to each input file.
Linking trailer files together can result in multiple occurrences of the "CALACSBEG" string. Only the first, however, determines where calacs will begin overwriting comments if an observation is reprocessed.
1 An "observation" set is a group of exposures under the umbrella of one unique ID. Each set has the same target, instrument configuration, operating mode, aperture, and spectral elements. An observation set created by the pipeline usually contains a set of dithered, repeated, and/or "CR-SPLIT" exposures taken within a visit. The data products in each calibrated observation set are described in its association table.
ACS Data Handbook
- • Acknowledgments
- Chapter 1: ACS Overview
- Chapter 2: ACS Data Structure
- Chapter 3: ACS Calibration Pipeline
- Chapter 4: ACS Data Processing Considerations
- Chapter 5: ACS Data Analysis