Role of Panel Support Scientists
The PSSs are intermediaries between the reviewers and STScI. They represent STScI and provide continuity through their insight into the operations of HST, STScI and the review process. Their main tasks are to answer questions, to ensure that the panel meetings proceed smoothly, to record the results of the panel discussions, and to bring any critical issues promptly to the attention of SPD and/or relevant STScI technical staff. The PSSs do not take part in the scientific discussions of the panel and do not grade.
Before the Meeting
- Reviewer assignments: Proposals are assigned to reviewers by SPD based on the information submitted by each panelist. If a panelist identifies a conflict after they receive their assignments, they should contact the PSS and the PSS should re-assign that proposal to another unconflicted reviewer. The PSS may consult with SPD if necessary. The PSS should inform SPD of the re-assignment to ensure that the appropriate information is updated in the database. The Panel Chair should also be informed about reassignments; the PSS should ascertain from the Chair whether they prefer to be updated with each reassignment or to receive the final assignment list after all swaps are made (just before the review).
- Proposal content: The PSS should read (at a minimum) each proposal's abstract and observation summary in order to acquire an overall familiarity with the proposal content of their panel, to assist the Chairs in organizing the material and running the meetings.
- Panelist questions prior to the meeting: PSSs should answer or re-direct panelists questions to the appropriate people (Who to Contact for Help). Questions about technical performance should be sent to the TAC TMs and questions about science policy to the SPD manager. If unsure, the PSS can ask their SPD manager.
- Regular email reminders: PSSs should send their panelists emails with reminders about upcoming deadlines, important information, and links to guidelines, including reminders about conflicts and potential DAPR, page limits, and formatting violations.
During the Meeting
The PSS functions as an executive assistant to the Chair and Vice Chair. They organize the materials, provide input on any technical issues noted before the meeting, coordinate real-time consultation with STScI specialists, monitor the grading and final ranking process as well as the reviewer comments status. More specifically, the principal activities during the meetings are as follows.
- Assist the Chair in the organization of the materials and discussion, and provide any other assistance requested by the Chair.
- Coordinate requests for consultations with STScI experts. A list of contacts and phone numbers for on-call experts will be available in each panel. An STScI specialist should be consulted on every significant feasibility or policy issue, and no proposal may be rejected on feasibility or policy grounds without such consultation. There have been problems on other missions with peer reviewers making their own judgments that later turned out to be wrong. Any consultation must be reported in SPIRIT.
- Panelists should record mandatory comments from the panel: duplications specifically allowed by the panel should be noted in the 'notification comments' of the database as being approved; verified technical issues should be entered into the 'Technical Comments' section of the Web-Reviewer tool.
- The PSS should keep careful notes on any controversies, conflicts, or other exceptional issues, and the resolutions of them recommended by the panels, for subsequent consideration during the Director's Review. These notes should be of a quality such that they would be intelligible to someone else (e.g., SPD staff responding months later to appeals from rejected proposers). An example of such information is keeping track of who leaves the room for each proposal, due to a conflict of interest. Another example would be noting where concerns were raised about the technical feasibility of a proposal, and how those concerns were addressed.
- The PSS should refocus the discussion if the discussion is tending towards comments about the proposing team, their past work, their validity, or their identities. The PSS has the authority to stop the discussion on a proposal. If DAPR concerns are raised during the meeting, the PSS can help the panel determine the appropriate course of action. If the mistake is slight, the PSS will help refocus the discussion on the science case. However, if an investigator’s self-revealed identify becomes impossible to ignore, and that identity has a clear impact on the discussion, the proposal should be flagged for disqualification, and the PSS should add a note in SPIRIT and notify the SPD about the issue.
- Although it may be difficult, the PSS should completely refrain from making any scientific, technical, scheduling or policy comments, even if asked to do so by the Chair. The PSS should instead ask for an STScI expert, if needed.
- Monitor comments (see below) progress via the Web-based system. Ensure that all comments are finished and signed off by the primary reviewers before they leave the meeting. Ensure that the Panel Chair is aware of the comments status and proof-read the submitted comments as time permits.
- Each panel member gives a numerical grade for each of the 3 grading criteria to the proposal in the web-based review system SPIRIT. (We strongly encourage panelists to review proposals as submitted, however, in exceptional cases, the panel may wish to change the time allocation, this should be voted on before grading.) The PSS must verify the orbit allocations are the same as entered in the Web Tool. The importance of care and accuracy in this activity cannot be overemphasized, since it is the only record of the panel's intent.
- After the grading stage follows a ranking stage. The final product of each panel will be a ranked list for all proposals. Ranks should be modified in the Web-Reviewer tool, as agreed upon by the panel. Grades can not be changed during the ranking stage.
- After the ranking is finalized and locked, the PSS will de-anonymize the proposals up to 1N+1 (one proposal past the 1N line) for the team expertise review. Once the panel is done verifying the team expertise, the proposals are re-anonymized, so all proposal should be anonymous again.