Guidelines for Panel Support Scientists

Introduction

The Hubble Space Telescope science peer review process is well established. Please review the documentation carefully. The Panel members have their own guidelines that will also be useful for your review. To minimize duplication, these guidelines describe only those issues unique to the panel support duties.

Proposals will be reviewed by discussion during (virtual) review panels. Panel Support staff will be responsible for supporting the virtual discussion review.

The Cycle 29 review will comprise eight topical panels:

  • Solar System Astronomy,
  • Exoplanets and Exoplanet Formation,
  • Stellar Physics and Stellar Types,
  • Stellar Populations and the ISM,
  • Galaxies
  • The Intergalactic and Circumgalactic Medium,
  • Supermassive Black Holes and Active Galaxies,
  • and Large-Scale Structure of the Universe.

Panelists are chosen based on their expertise in one or more of the areas under review by the panels. The discussion panels will assess and grade all of the proposals submitted in response to the Call for Proposals. Each panel will be managed by a panel chair and a vice chair.

Confidentiality and Conflicts of Interest

The proprietary nature of the proposals and related documents is taken extremely seriously by STScI. Proposals are the intellectual property of the PIs, and the sensitivity of the process is enhanced by the heavy over-subscription of HST time and the significant funding implications for US proposers. Review-related materials should not be left in the open when not in use, and should be locked away overnight or when otherwise inactive. Please purge any review files from your computer after the review.

Panel membership remains confidential except within a panel. The full TAC membership is published after the results have been announced, but without panel attributions.

It is critically important that Conflicts of Interest are avoided during the selection process. In general, Panel Support Scientists are not assigned to panels where there is the possibility of their having a conflict of interest. Please inform a member of the Science Policies Group (SPG) if such an issue arises.

Role of the Panel Support Staff (PSS)

The PSSs are intermediaries between the reviewers and STScI. They represent STScI and provide continuity through their insight into the operations of HST, STScI and the review process. Their main tasks are to answer questions, to ensure that the panel meetings proceed smoothly, to record the results of the panel discussions, and to bring any critical issues promptly to the attention of SPG and/or relevant STScI technical staff. The PSSs do not take part in the scientific discussions of the panel and do not grade.

  • Reviewer assignments:  Proposals are assigned to reviewers by SPG based on the information submitted by each panelist. If a panelist identifies a conflict after they receive their assignments, they should contact the PSS and the PSS should re-assign that proposal to another unconflicted reviewer. The PSS may consult with SPG if necessary. The PSS should inform SPG of the re-assignment to ensure that the appropriate information is updated in the database. The Panel Chair should also be informed about reassignments; the PSS should ascertain from the Chair whether they prefer to be updated with each reassignment or to receive the final assignment list after all swaps are made (just before the review).
  • Proposal content: The PSS should read (at a minimum) each proposal's abstract and observation summary in order to acquire an overall familiarity with the proposal content of their panel, to assist the Chairs in organizing the material and running the meetings.
  • Panelist questions prior to the meeting: Panelists should be directed to the HST Help Desk (https://hsthelp.stsci.edu) for all questions about science policy or technical performance. In the correspondence, panelist should identify themselves as a TAC member. If the question is about a specific proposal, the proposal ID should be stated in the question.

During the meetings, the PSS functions as an executive assistant to the Chair. They organize the materials, provide input on any technical issues noted before the meeting, coordinate real-time consultation with STScI specialists, monitor the grading and final ranking process as well as the reviewer comments status. More specifically, the principal activities during the meetings are as follows.

  1. Assist the Chair in the organization of the materials and discussion, and provide any other assistance requested by the Chair.
  2. Coordinate requests for consultations with STScI experts. A list of contacts and phone numbers for on-call experts will be available in each panel. An STScI specialist should be consulted on every significant feasibility or policy issue, and no proposal may be rejected on feasibility or policy grounds without such consultation. There have been problems on other missions with peer reviewers making their own judgments that later turned out to be wrong. Any consultation must be reported in SPIRIT.
  3. Panelists should record mandatory comments from the panel: duplications specifically allowed by the panel should be noted in the 'notification comments' of the database as being approved; verified technical issues should be entered into the 'Technical Comments' section of the Web-Reviewer tool.
  4. The PSS should keep careful notes on any controversies, conflicts, or other exceptional issues, and the resolutions of them recommended by the panels, for subsequent consideration during the Director's Review. These notes should be of a quality such that they would be intelligible to someone else (e.g., SPG staff responding months later to appeals from rejected proposers). An example of such information is keeping track of who leaves the room for each proposal, due to a conflict of interest. Another example would be noting where concerns were raised about the technical feasibility of a proposal, and how those concerns were addressed.
  5. Although it may be difficult, the PSS should completely refrain from making any scientific, technical, scheduling or policy comments, even if asked to do so by the Chair.
  6. Monitor comments (see below) progress via the Web-based system. Ensure that all comments are finished and signed off by the primary reviewers before they leave the meeting. Ensure that the Panel Chair is aware of the comments status and proof-read the submitted comments as time permits.
  7. Each panel member gives a numerical grade for each of the 3 grading criteria to the proposal in the web-based review system SPIRIT. (We strongly encourage panelists to review proposals as submitted, however, in exceptional cases, the panel may wish to change the time allocation, this should be voted on before grading.) The PSS must verify the orbit allocations are the same as entered in the Web Tool. The importance of care and accuracy in this activity cannot be overemphasized, since it is the only record of the panel's intent.
  8. After the grading stage follows a ranking stage. The final product of each panel will be a ranked list for all proposals. Ranks should be modified in the Web-Reviewer tool, as agreed upon by the panel. Grades can not be changed during the ranking stage.

Proposal Comments

Before the end of the Panel meetings, the PSS will be asked to proof-read and edit the proposal comments to make them suitable for dissemination to the proposers. Panel Chairs are also required to proof-read the final comments. Use the following guidelines:

  1. All rejected proposals (those below the Cutoff line), should have a clear statement on why the proposal was "rejected".
  2. Correct grammatical and typographical errors.
  3. Eliminate all offensive, gratuitous or personal remarks.
  4. Remove any explicit identifications of other Cycle 29 proposals or proposers, reviewers, or STScI staff.
  5. Comments that contain only insubstantial or superficial remarks should be removed, whether for approved or unapproved proposals.
  6. Remove remarks that do not make sense because the reviewers expected the proposal to be approved, while in the end it was not. However, it is okay if a rejected proposal has positive remarks that are useful; that possibility is covered in the notification letter as a consequence of the heavy over-subscription.
  7. If any comments mention that the proposal exceed the page limits, make sure that you verify this personally.
  8. Make sure you identify any proposals to SPG that violate the Anonymous Review Guidelines.
  9. Consult with SPG on any questionable cases.
  10. Inform SPG when a complete set of the final, edited comments are available.

Specific PSS Duties

  • Attend Panel Support Meetings as required.
  • Familiarize yourself with the proposals before your panel.
  • Identify any conflicts that you might have.
  • Handle any review conflict reassignments.
  • Experiment with the Web-Based Reviewer System.
  • Answer and resolve any questions brought up by the reviewers as needed.
  • Assist Panel Chair as required prior to the actual meetings.
  • Monitor the voting process, document any issues about grading, conflicts of interest, etc., into the Web-Reviewer tool during the panel meetings.
  • Summon assistance, as needed, from SPG (policy questions), Instrument Division (INS, instrument questions) or the Observation Planning Branch (OPB, implementation questions).
  • Monitor adherence to all conflict-of-interest rules.
  • Inform someone from SPG that your panel is finished at the conclusion of your panel meeting.
  • Review final Comments.

As a valuable member of the Cycle 29 Peer Review process, you are a critical part of the system and we value your comments and suggestions, and welcome feedback throughout the process.



Next: Blue Jeans and Slack Guidelines