A PDF of the contents of this page, including the grading rubric and examples, is linked here for convenience. Note that the online version of this page will always be the most up-to-date.

Selection Criteria

Proposals reviewed by external panelists are subject to a single-phase review; proposals reviewed by the virtual topical panels are subject to a two-stage review process: 1) preliminary grading and triage; and 2) the review meeting. In all cases, panelists use the same scoring system.

Each topical panel covers a very broad science category, and each science category contains a number of narrower sub-fields. Ideally a proposal will be impactful to both the narrow sub-field of the proposal and to other sub-fields within the science category or in other science categories. Proposals will be assessed on an absolute scale against three primary criteria described in the Call for Proposals with a separate grade given for each.

Impact within the sub-field:
- The scientific merit of the program and its contribution to advancement of knowledge.
- Will the proposed program improve our understanding of the objects, classes of object, or specialist topics under study in the proposal? By how much? How relevant is the proposed work to the immediate sub-field of the proposal? Is the science within the sub-field timely?
- The immediate sub-field of the proposal is the niche area of the program, not the whole broad science area of the topical panel to which it was assigned. The evaluation should be based on what is written in the proposal, not on the reviewer's broader knowledge, even if the reviewer is an expert in the sub-field. Though, in most cases, the reviewer will not be an expert in the sub-field of the proposal, and the proposal should have been written accordingly.
Out of field impact:
- The program’s impact for astronomy in general.
- Are there implications for other science areas and/or insights into larger-scale questions? Will the proposed program improve our understanding of science areas beyond the immediate sub-field of the proposal? How broad and how significant is this new understanding? Is the out-of-field impact timely?
- The proposal does not have to impact all of astronomy, but should ideally impact a number of other sub-fields or provide significant impacts in at least one other sub-field. The out-of-field impacts could be in other areas within the topical science panel of the proposal, or in other topical science areas. This evaluation should be based on what is written in the proposal, not on the reviewer's broader knowledge.
Suitability:
- The necessity for HST observations, including special requirements, or relevance to HST science. The feasibility of the science program.
- For GO programs: Has the proposed program demonstrated that the unique capabilities of HST are required to achieve the science goals? How much of an advantage does HST data offer over other facilities? Has the time request been well justified? Have any special requirements been well justified? Have parallel observations been well justified? Have any duplications been well justified? If joint time was requested, has those additional observations been well justified? Does the observing plan make efficient use of resources? Is there a clear path to science?
- For AR programs: Has the proposed program demonstrated that the unique capabilities of HST are required to achieve the science goals? How much of an advantage does HST data offer over other facilities? Is it clear how the analysis plan will lead to the completion of the science goals? Is there a clear path to science?
- For Theory programs: Has the proposed program demonstrated a broad applicability to HST observational programs? Is it clear how the analysis plan will lead to the completion of the science goals? Is there a clear path to science?

The final grade is the straight average of these values.

AR and GO calibration proposals are required to provide an analysis plan; reviewers should also consider the strength of the analysis plan in assessing the first two criteria.

Descriptions of additional criteria by type of proposal are given in the Proposal Selection Procedures section of the Call for Proposals.

While reviewing the proposals if you notice and/or identify any issues with the proposal template formatting, page limit violations or resource request issues, please contact SPG to discuss before downgrading that proposal.

Scoring System

Preliminary Grades and External Grading

The full set of criteria to apply in assessing different types of proposals are described in the Proposal Selection Procedures section of the Call for Proposals. Those criteria should be taken into account when grading each proposal.

The preliminary scoring and the grades from external panelists should be on an absolute scale with the framework set by the following criteria. Reviewers may submit grades in decimal form, but please limit to one decimal place.

Grade

Impact within the sub-field

Out-of-field impact

Suitability

1

Transformative advancement in the sub-field.

The proposed program has the potential for transformative results in the immediate sub-field of the proposal. The program will transform understanding of the objects, class of objects, or specialist topics under study.

Transformative implications beyond the sub-field.

The proposed program has transformative implications for one or more other sub-fields of astronomy. The impacts of the program are extremely broad and/or extremely significant.

Extremely well-justified with a very clear path to science.

GO: The program's science goals can only be achieved with the proposed observations. Only HST is capable of collecting the required observations. Archival data may supplement the analysis, but is insufficient or does not exist in key areas, so new observations are needed. All aspects of the requested observations have been well justified and are necessary for the success of the science program. The observing plan uses resources very efficiently and provides a very clear path to science.

AR: Analysis of archival HST data is critical to reach the stated science goals. The analysis may be supplemented by data from other observatories but the majority of the data will come from HST. The analysis plan is comprehensive and provides a very clear path to science.

Theory: The proposed theoretical or computational work will be transformative for planning future HST observing programs, for future analysis of HST data, or interpreting results obtained from HST data. The analysis plan is comprehensive and provides a very clear path to science.

2

Major advancement in the sub-field.

The proposed program has the potential for major advancement in the immediate sub-field of the proposal. The program will provide major advances in understanding of the objects, class of objects, or specialist topics under study.

Major implications beyond the sub-field.

The proposed program has major implications for one or more other sub-fields of astronomy. The impacts of the program are broad and/or significant.

Well-justified with a clear path to science.

GO: The program's science goals will certainly be achieved with the proposed observations. HST offers major advantages over other facilities for collecting the required observations. The need for new data instead of or to supplement archival data is clear. Most aspects of the requested observations have been reasonably well justified and they will contribute to the success of the science program. The observing plan uses resources efficiently and provides a clear path to science.

AR: Analysis of archival HST data offers major advantages over data from other facilities to reach the stated science goals. The analysis may be supplemented by data from other observatories but a significant fraction of the data will come from HST. The analysis plan is adequate and provides a reasonably clear path to science.

Theory: The proposed theoretical or computational work will offer major advances for planning future HST observing programs, for future analysis of HST data, or interpreting results obtained from HST data. The analysis plan is adequate and provides a reasonably clear path to science.

3

Moderate advancement in the sub-field.

The proposed program has the potential for moderate advancement in the immediate sub-field of the proposal. The program will provide moderate advances in understanding of the objects, class of objects, or specialist topics under study.

Moderate implications beyond the sub-field.

The proposed program has moderate implications for one or more other sub-fields of astronomy. The impacts of the program are moderate in scope and significance.

Some justification lacking but path to science likely achievable.

GO: The program's science goals will be achieved with the proposed observations. HST offers some advantages over other facilities for collecting the required observations. The need for new data over archival data is discussed but not compelling. Some aspects of the requested observations have been justified but other aspects are lacking adequate justification and it is not clear that all are needed. The observing plan could use resources more efficiently but will likely lead to science.

AR: Analysis of archival HST data offers some advantages over data from other facilities to reach the stated science goals. The analysis may be supplemented by data from other observatories but some of the data will come from HST. The analysis plan is lacking in some areas but the provided information gives confidence that the science program can be completed.

Theory: The proposed theoretical or computational work will offer some advances for planning future HST observing programs, for future analysis of HST data, or interpreting results obtained from HST data. The analysis plan is lacking in some areas but the provided information gives confidence that the science program can be completed.

4

Minor advancement in the sub-field.

The proposed program has the potential for minor advancement in the immediate sub-field of the proposal. The program will provide minor advances in understanding of the objects, class of objects, or specialist topics under study.

Minor implications beyond the sub-field.

The proposed program has minor implications for other sub-fields of astronomy. The impacts of the program are limited in scope and significance.

Many aspects poorly justified and path to science uncertain.

GO: The program's science goals will probably be achieved with the proposed observations. HST offers minor advantages over other facilities for collecting the required observations. The need for new data over archival data is not discussed in depth or is not clear. Many aspects of the proposed observations have been poorly justified. There are questions about many aspects of the observing set up. The observing plan uses resources inefficiently and the path to science in uncertain.

AR: Analysis of archival HST data offers minor advantages over data from other facilities to reach the stated science goals. The analysis will mostly use data from other observatories but small fraction of the data will come from HST. The analysis plan leaves many unanswered questions and the path to science is uncertain.

Theory: The proposed theoretical or computational work will offer minor advances for planning future HST observing programs, for future analysis of HST data, or interpreting results obtained from HST data. The analysis plan leaves many unanswered questions and the path to science is uncertain.

5

Limited advancement in the sub-field.

The proposed program has limited or no potential for advancement in the immediate sub-field of the proposal. The program will not advance understanding of the objects, class of objects, or specialist topics under study.

Limited implications beyond the sub-field.

The proposed program has little or no implications for other sub-fields of astronomy. The impacts of the program are extremely limited in scope and significance.

Very poorly justified with no clear path to science.

GO: The program's science goals will not be achieved with the proposed observations or HST offers little to no advantage over other facilities for collecting the required observations. The need for new data over archival data is not discussed. The proposed observations are poorly justified or not explained at all. The observing set up is unlikely to be successful. The observing plan is very inefficient and a path to science is lacking.

AR: Analysis of archival HST data offers little to no advantage over data from other facilities to reach the stated science goals. The analysis will predominantly use data from other observatories with only very little or no data coming from HST. The analysis plan is extremely weak or missing altogether and leaves no confidence that the work can be completed successfully.

Theory: The proposed theoretical or computational work will offer little or no advance in planning future HST observing programs, for future analysis of HST data, or interpreting results obtained from HST data. The analysis plan is extremely weak or missing altogether and leaves no confidence that the work can be completed successfully.

Examples

The following examples aim to give guidance in applying these rubrics to grading proposals; reviewers should use their best judgement.

Case 1: UV observations of gas in young stars

In field	Highly significant improvement in our understanding of gas flow in young stars.	1-2
Out of field	Potential for significant changes in our understanding of gas flows in a wide range of other environments.	1-2
Suitability	UV observations are essential to achieve the science goals and can only be acquired through HST observations.	1

Case 2: Analysis of archival near-IR imaging of a nearby galaxy for stellar population investigations

In field	Major advance in understanding stellar populations in that galaxy.	2
Out of field	Some implications for stellar populations and stellar evolution in other galaxies.	3
Suitability	The increased spatial resolution offered by HST provides some advantages over other facilities in addressing the science goals. The analysis offers significant improvements and/or additional value with respect to the original use of the data.	2-3

Case 3: Optical/near-IR spectroscopy of an emission-line galaxy

In field	Moderate increase in understanding of prevalence of star formation in that galaxy.	3
Out of field	Minor implications for the properties of other galactic systems, but no wider impact.	4
Suitability	Only optical data are required for the science case; limited gains in performance at near-IR wavelengths as compared with larger ground-based facilities	4-5

Case 4: Developing theoretical tools to characterize gas and dust in Galactic star-forming regions

In field	Potential significant increase in understanding of chemical composition in dusty environments.	1-2
Out of field	Results have significant implications for interpreting dust composition in other galaxies.	2
Suitability	The theoretical analysis will enable and support additional HST observational programs.	2

The grades for each virtual panelist are normalized by SPIRIT to a mean value of 3.0 and a standard deviation of 1.0. The reviewer's grade for that proposal is the average of the three normalized grades.

The preliminary grade for each proposal is determined by averaging the overall grade from each reviewer. The preliminary grades are used to create a rank order list for each panel and the lowest-ranked proposals (typically ~40%) are triaged from further discussion.

Next: Proposal Feedback Comments

Selection Criteria and Scoring System