5.1 Data Reduction and Analysis Applications
Most of the software tools for operating on COS FITS files are contained in three Python packages that come with the AstroConda python distribution, which is available for download from STScI:
The three Python packages used with COS are:
- calcos: Contains the calcos package, used for processing data through the calibration pipeline. Section 3.6 describes how to run calcos.
- costools: Contains COS-specific tasks to calibrate and interpret data. Many of these tasks are described in Chapter 3. A complete listing is presented in Section 5.1.1.
- astropy: Contains packages and modules to read and operate on FITS tables, which is the format used for many COS products including 1-D spectra and most COS reference files. We provide specific examples of their use here in Section 5.1.2.
We also present a brief summary of spectral display and analysis tasks in Section 5.1.3. To run the examples in Sections 5.1.1 to 5.4, one must activate the AstroConda environment before starting Python. This can usually be done by typing "conda activate astroconda" on a terminal command line.
In addition to what is below, users are encouraged to use the Jupyter Notebooks for COS. Those available include:
|Setup||Setting up your computer environment to work with COS data|
|DataDI||Downloading COS Data from the archive|
|ViewData||Beginning to work with COS data in Python: plotting, binning, calculating SNR, & evaluating a spectrum|
|AsnFile||Modifying or creating an association file|
|CalCOS||Running the COS pipeline|
|DayNight||Filtering out COS data taken during the day or night|
|LSF||Working with the COS Line Spread Function (LSF)|
|Extract||Editing the extraction boxes in a BOXCAR-method spectral extraction file (XTRACTAB)|
5.1.1 COS-Specific AstroConda Tasks
In Chapter 3, we gave detailed discussions of the use of the data reduction pipeline calcos. This task is contained in the AstroConda package, calcos. Other modules useful for reducing and analyzing COS data are contained in a separate package called costools. A complete listing and brief description of these modules is given in Table 5.1. All of these tasks are run in Python. Consult the online help for each module for more information. Some of these modules will be discussed in greater detail in the remainder of this chapter.
Table 5.1: COS-Specific Modules.
Process COS data through the calibration pipeline
Get vertices for SAA model by model number
Filter a corrtag table based on the TIMELINE extension
5.1.2 FITS Table Tasks
COS 1-D spectra and most COS reference files are stored in FITS tables (see Section 2.4.1 for a description of the structure of the table extension files for 1-D spectra and
TIME-TAG data). Tasks designed to handle this format can be used to examine and extract information from the tables. Here we give specific examples of the use of routines in astropy.table.Table and astropy.io.fits to help you get started. A sample output is given after each command.
Use the Table module in the astropy.table package to find out what information is given in the columns of a FITS table.
Use print() to look at the contents of the table:
Note that the number of columns displayed is limited by the width of the window that you are working in. To see more columns, you can adjust your window to be wider and re-type the command:
The table output indicates that some of the columns contain arrays of 16384 elements rather than a single value. For those columns, printing the table element t displays the value of the first and last elements in the array.
The user may wish to look at values in a COS reference file
FITS table, which generally have many rows. Each row typically characterizes a specific operating mode, location on the detector, a value of a parameter to be used in the reduction, etc.
In Python, there are often many methods to accomplish the same goal. For instance, in the above example, one could have used fits.getdata() instead of fits.open() (as shown in examples below). When using fits.open(), it is important to close the fits file afterwards as shown in the last line of this example.
5.1.3 General Spectral Display and Analysis Tasks
Table 5.2 lists several useful Python packages for displaying and analyzing COS spectral data.
Table 5.2: Spectral Analysis Tasks.
1-D fits or ASCII files
Generating 1-D plots, fitting curves to spectral features, and much more with a graphical user interface
Generating 1-D plots from columns in tables
2-D image arrays
Generating 2-D plots from 2-D image arrays
1-D and 2-D arrays
Fitting curves to spectral features
Fitting curves to spectral features
Specvis is a powerful spectral plotting and fitting package that provides the user with many options for modeling emission and absorption features, continuum, and much more. It is controlled via a graphical user interface, and has documentation and examples available: https://specviz.readthedocs.io/. Please note that this is a new package which has not been fully tested and verified as of May 2018.
Plotting COS Spectral Images (
The COS FUV data consist of separate files for each of the two FUV detector segments, segment A and segment B. In general, each segment contains one target spectrum and one wavecal. On the other hand, for the NUV data there are three disjointed portions of the spectrum present on the image for both the target and the wavecal. The following examples illustrate the use of the imshow() function in matplotlib.pyplot module to plot COS FUV and NUV spectra from the 2-D
flt files. The parameters in plt.ylim() refer to the Y location of the spectrum for each particular grating/segment combination. One may need to adjust the scaling factors, vmin and vmax, to suit the data range of any particular dataset. A range of different colormaps may be used, and can be found here:
The parameters in plt.ylim() refer to the Y location of the spectrum for each particular grating/segment combination. For the FUV detector, the Y location also depends on lifetime position (LP). The example below shows both segments of a lifetime position 4, G160M spectrum.
Plotting COS Tabular Spectra (
The following example illustrates the use of the Python package matplotlib.pyplot to plot COS tabular data. See Section 5.3.1 for more details and examples.
The above example selects the portion of the data that corresponds to the segment we want to plot. The segment selector can be either 'FUVA' or 'FUVB' for FUV data and 'NUVA', 'NUVB', or 'NUVC' for NUV data.
The next example shows how one would plot both FUV segments by using the concatenate() module in numpy.
We note that instead of using concatenate(), one could also use the flatten() module as shown in the example in Section 5.3.1.
COS Data Handbook
- • List of Figures
- • List of Tables
- Chapter 1: COS Overview
- Chapter 2: COS Data Files
- Chapter 3: COS Calibration
- Chapter 4: COS Error Sources
- Chapter 5: COS Data Analysis
- Appendix A: COS Lifetime Positions
- Appendix B: COS High Voltage History