All proposers receive feedback.

• For proposals under Discussion Review, Primary Reviewers are responsible for compiling feedback for their assigned proposals. These feedback comments should take into account the panel discussion for proposals discussed at the in-person meetings. For triaged proposals, consensus reports should be based on the comments submitted by reviewers during preliminary grading.
• For proposals under External Review, a brief report is provided by each External Panelist. These reports are appended together and sent verbatim. Panelists must ensure that their comments are appropriate.
• For all proposals, the reports are anonymous.

Contents of the Feedback Comments

Discussion and External Reviews

The feedback comments are broken down into different subsections. The first two are compulsory, the rest are optional (and will likely not be required for most proposals):

• Strengths: self-explanatory. Compulsory.
• Weaknesses: self-explanatory. Compulsory.
• Resources: notes about the Orbits or Targets or Funding levels, or instrument/configuration/filters, only needed if something is out of the ordinary. Optional.
• Comments: anything else that you want to pass on that doesn’t fit under Strengths, Weaknesses, or Resources. Optional.
• Technical Notes: updated based on technical reviews and for most cases not used by the reviewer, but by STScI staff. These notes are usually relevant for Joint Observatory programs or if reviewers have concerns about instruments. In the latter case the reviewer must request a consultation from an instrument expert via the Panel Support Scientist (PSS), and the notes from that consultation will be entered in this section by the PSS.  (This section will not be sent to the Proposers, it is for the review only; anything relevant for the proposers to see should be copied into the Resources section). Optional.
• Instructions: Additional instructions for the proposers to help them prepare their submission for implementation/scheduling should their proposal be approved. This can include mandatory recommendation for a change in the orbit allocation or the targets. Targets or orbits may be changed if two non-identical approved proposals contain duplicate observations; one of the proposals would be instructed to remove the duplicated observations, while preserving the unique elements of the proposal.  (These recommendations will be made only by the Discussion Panel.) Optional.

Additionally, any concerns abut DAPR, page limits, or formatting violations should be sent via email to the PSS and SPD manager rather than including them in the feedback comments.

Guidelines for Writing Feedback Comments

Discussion and External Reviews

Proposal feedback comments should be concise. They should focus on the evaluation of the proposal by the reviewers. Avoid “questions” in the comments. For example, “why have 6 targets instead of 5?” should be re-phrased to “the proposal did not sufficiently motivate the number of requested targets”. The reports should focus on the scientific content and not the team, so please refer to what the proposal says rather than what the proposers say. Where possible, comments should avoid referencing the reviewer or the panel as this creates undue focus on the reviewers rather than the integrity of the process. For example, "The proposal did not sufficiently explain why these targets were chosen" is preferred over "It is not clear to me why these targets were chosen", and "The analysis plan lacked a description of the spectral data reduction" is preferred over "The panel had some concerns about the spectral data reduction".

Comments that may be perceived as derogatory or insulting must be avoided. The Panel Chairs are responsible for checking for any such improprieties in their review of the comments.

Except for triaged proposals (relevant for Discussion Panelists only), reviewers cannot be sure at the time of writing feedback comments whether the proposed observations will be scheduled for execution. The comments should be phrased in such a way that they are sensible and meaningful regardless of the final outcome.

Reviewers should avoid statements that create the impression that the low ranking of a proposal is due to a minor mistake. Many proposals do not have obvious weaknesses but are just less compelling than others: in such a case, acknowledge that the considered proposal is good but that there were others that were more compelling. However, never include in the report an explicit reference to another proposal, such as the proposal ID. Whenever possible, make suggestions for possible improvements, but avoid giving the impression that following those suggestions guarantees that the proposal will be more successful in next cycle.

Do not use generative language AI (e.g. ChatGPT) to write or finesse your comments.

Process & Workflow for Finalizing Feedback Comments

Discussion Reviews Only

The Primary Reviewer is responsible for submitting a feedback report that is as specific, accurate, clear and informative as possible. The Panel Chairs shall in the week following the TAC meeting review the panel reports of all the proposals that have been reviewed in their panel, with a view to ensuring that all of them achieve the necessary standard of quality in providing the PIs with meaningful feedback. The STScI Science Policies Division will undertake a final review of all comments before they are distributed to the community.

Some guidelines on workflow for writing reviews prior to and during the Virtual Discussion Meeting:

• Please do not discuss individual proposals in the Slack panel channel. It is OK to mention proposals such as “I think I’ve incorporated your comments into the feedback for proposal #123, let me know if it’s right” or “I could use some extra input on proposal #321”, but do not discuss the merits or the science on the Slack channel.
• You can use the “Remarks” section in SPIRIT to collaborate on writing the reviews and contributing comments to the primary reviewer.
• Direct Messages (DMs) on Slack are allowed. Please use the Slack workspace over emails and we remind you that the proposals and reviews are proprietary information. Use common sense here.

General Procedure for Reviewers:

1. Both Primary and Secondary Reviewer add their comments. This can be done when the status of the proposal is in “Ready” or “Reviewing” status. (Saving often!)
2. The Secondary Reviewer (typically) will 'Mark Completed' their comments.
3. The Primary Reviewer merges Secondary’s comments into their own, as well as any Remarks that have been left by other panelists. The Primary can hit the 'Mark Completed', to indicate they are done with making updates to their comments.
4. The Primary or Secondary both have the ability to Reopen the Review to make edits to their comments.
5. Once the Discussion review has begun, either Reviewer, but it should generally be the Primary, can 'Submit as Final' to indicate the comments are finalized. A Panel Admin or the Chair can also 'Submit as Final', if necessary. Whichever reviewer hits the 'Submit as Final' button will have their comments pushed through as the “Submitted Review."
6. Once the 'Submit as Final' button has been pressed, two things happen:
   1. The review status for the proposal changes to “Submitted.”
   2. The tab “Submitted Review” can now be edited by the Chair/PSS to make edits for continuity (they should also Save often!).

Both the Chair and the SPD Manager will proofread the reviews. The they will look for the following, among other things:

• grammatical errors and typos
• anything contradicting the previously posted guidelines
• personal remarks about proposers
• explicit identifications of other The page .Hubble Space Telescope Call for Proposals for Cycle 32 v32 was not found  -- Please check/update the page name used in the MultiExcerpt-Include macro proposals or proposers, reviewers, or STScI staff
• comments that contain only insubstantial or superficial remarks
• remarks that do not make sense because the reviewers expected the proposal to be approved, while in the end it was not

Once they are happy with the final review they should click the Review Signoff button. This will change the review status of the proposal to “Complete.”

Next: Reviews, Grades & Comments