CIHR Instructions for Committee Members - Structured, Remote Review

[ PDF (25 KB) ]

In Structured review, the entire review process is done electronically, hence, the common reference to virtual review. Reviewers complete their assessments remotely and submit their evaluations on ResearchNet. Applications are assigned to two reviewers who provide individual ratings which are given equal weight and averaged. In instances where there is a discrepancy1 between the assessments, a re-review process takes place. Both reviewers of an application communicate with one another and are asked to reconsider their initial assessment and resubmit their scores. If the discrepancy between the two reviewers cannot be resolved with a re-review, a third reviewer will be asked to evaluate the application. The final score will be determined by taking the average of the three scores.

Structured review is used primarily for Meetings, Planning and Dissemination (MPD) Grants and for Training Awards programs. Reviewers of MPD grants are to provide an assessment on the project's potential communication of health research while reviewers for Training Awards programs are to provide CIHR with an assessment of each applicant. Furthermore, the evaluation of their potential to develop, the proposed training environment, their aptitudes, skills and experience necessary for future research achievement and productivity are taken into consideration.

Before evaluating your assigned funding applications:

  • Reviewers must abide by CIHR's Policy on Conflict of Interest and Confidentiality through ResearchNet;
  • After assignments, reviewers must declare their conflicts of interest with applications on ResearchNet. This task should be completed in a timely manner. If you are uncertain regarding a potential conflict or your level of comfort for an application, please discuss this with the Program Delivery Coordinator as soon as possible;
  • Depending on the number of conflicts declared by members, reviewers may be asked to review a few additional applications;
  • The primary and most secure tool for conducting peer review is ResearchNet. Paper copies of applications are no longer provided;
  • Applications must be treated as protected information. You must not divulge their contents to others, or use the information provided for any purpose other than peer review;
  • Refer to the "Guide on Handling Documents used in Peer Review" for the safe handling and disposing of documents;
  • Be familiar with CIHR policies and procedures posted on ResearchNet and on the Internet. It is also important to be familiar with the objectives of the funding opportunity (see Find Funding);
  • Reviewers must rate applications according to the set criteria provided in the Review Process and Evaluation section specific to the Funding Opportunity and are encouraged to provide comments as part of their Reviewer Report (as described below);
  • Reviewers must submit their scores and reviews on ResearchNet according to the deadlines provided. Detailed instructions for posting your reviews can be found on the ResearchNet site on the Conduct Reviews page entitled "Get to know the Conduct Reviews task for Committee Members".

Reviewer Report

It is important to note that, while this is not mandatory, we do strongly recommend that you take the time to provide comments to the applicant. These are welcomed by candidates as they are helpful in improving their applications for subsequent competitions and reduce applicant complaints.

  • Please refer to the Guide for Reviewers, specific to your peer review committee, before you write the report (e.g., Reviewers' Guide for CIHR Doctoral Research Awards, for CIHR Fellowships, etc.):
    • Purpose: to provide critical assessment of the application and constructive feedback to the applicant based on the objectives and evaluation criteria described in the funding opportunity;
    • The review should be clear and concise;
    • The applicant will receive the review as you submit it. Do not include your score or identify yourself in the text of your review;
    • Keep in mind that there can be sensitivity to tone, etc.;
    • Use caution when referring to the candidate personally (e.g., use of he/she).

Issues for CIHR's Attention

Issues related to ethics, eligibility, human pluripotent stem cell research and other concerns need to be flagged to be brought to CIHR's attention. The rating should not be influenced by such factors. Applications which have been flagged for CIHR's attention will be withheld as "pending" until the applicant is contacted for further information or the application is reviewed by CIHR's appropriate committee. Please complete the Issues for CIHR Attention form, available at the link found at the end of the Reviewer Guide in ResearchNet, and communicate with your Committee Coordinator.

Following the Review Process

  1. Provide feedback to the Program Delivery Coordinator regarding the effectiveness and functioning of the peer review process and policy issues;
  2. CIHR will identify applications which are at risk of an unfair decision because of a wide spread between the two reviewers' ratings. In cases where narrowing the gap between the two reviewers' ratings may affect the funding decision, CIHR will ask both reviewers to reconsider their initial assessment and resubmit scores;
  3. If the discrepancy persists following re-review, you may be asked to act as third reviewer on applications which were assigned to other committee members;
  4. Keep the applications and your working notes on file until one month after competition results have been announced in case applicants have any questions regarding the notes provided;
  5. One month following the competition results announcements, please destroy all peer review materials, including applications copies, CD and electronic copies on your computer.

References

  1. When all initial scores are received, CIHR will calculate an average for each applicant based on both reviewers weighted scores and not on each specific criterion. CIHR will then identify applications which are at risk of an unfair decision because of a wide spread between the two reviewers' ratings (the "acceptable" range is determined based on the level of variation of all scores received). In cases where narrowing the gap between the two reviewers' ratings may affect the funding decision, CIHR will ask both reviewers to reconsider their initial assessment and resubmit scores.