Peer Review Manual - Foundation

Notice

As announced by CIHR's Acting President, Roderick R. McInnes, on July 17, CIHR is considering changes and improvements to the peer review process for the Foundation Grant program. It has been determined, however, that changes to the review process for the 2017-18 competition are not feasible given the existing timelines. Stages 1, 2, and 3 of peer review for the 2017-18 competition will therefore continue to use the processes developed for previous Foundation Grant competitions.

Discussions about the best way forward for the 2018-2019 competition are currently under way.

Table of contents

  1. Introduction
  2. Policies Impacting Peer Review
  3. Principles of Peer Review
  4. Peer Review Roles
  5. Equity in the Peer Review Process
  6. Foundation Peer Review Process
  7. Annexes
    Interpretation guidelines compendium: sample indicators of Foundation Grant adjudication criteria
    1. Leadership Sample Indicators
    2. Significant Contribution Sample Indicators
    3. Productivity Sample Indicators
    4. Foundation grant peer review process diagram

Please note that in the context of this document, “the applicant” may refer to an individual Program Leader or multiple Program co-Leaders.

1. Introduction

The purpose of this manual is to provide highlights on the policies and information on procedures for peer review of applications in the Foundation Grant competition, as well as to outline the roles and responsibilities of peer review committee members.

The mandate of the Canadian Institutes of Health Research (CIHR) is to "To excel, according to internationally accepted standards of scientific excellence, in the creation of new knowledge and its translation into improved health for Canadians, more effective health services and products and a strengthened health care system."

The Foundation Grant Program is designed to contribute to a sustainable foundation of health research leaders by providing long-term support for the pursuit of innovative and high-impact programs of research. Programs of research are expected to include integrated, thematically-linked research, knowledge translation, and mentoring/training components.

Foundation grants are designed to support a broad base of health research leaders to build and conduct programs of health research across CIHR's mandate. Eligible applicants include mid career and senior investigators, all of whom are independent researchers with a demonstrable track record of excellence and impact in their field of study.

While the application form and adjudication criteria are the same for all applicants, the criteria must be applied in the context of the applicant's career stage.

The Foundation Grant Program is supported by a three-stage competition and review process that focuses reviewer attention on specific structured review criteria at Stage 1 and at Stage 2.

This process involves the evaluation of applications by reviewers, who have the required experience and expertise to assess the application, within the context of the program’s objectives and the review criteria for each stage of the peer review process.

Peer Reviewers and Competition Chairs should review the material in this manual as they prepare for and conduct evaluations of applications. In addition to reviewing the material in this manual, it is essential that participants read and become familiar with the Funding Opportunity. Furthermore, all CIHR peer reviewers are also expected to:

  • Become familiar with CIHR funding policies and guidelines as outlined in CIHR's Funding Policies.
  • Formally agree to abide by CIHR's Confidentiality and Conflict of Interest Policy through ResearchNet.
  • Ensure that all review materials are handled according to CIHR’s Guide on Handling Documents used in Peer Review.
  • Provide feedback on the adjudication process to CIHR (for detailed regulations concerning all aspects of CIHR funding programs, please refer to the Grants and Awards Guide).
  • Complete training on the Foundation review process through online learning modules and attendance to webinars.

2. Policies Impacting Peer Review

2.1 Global Health Research and International Collaborations

As outlined in the CIHR Grants and Awards Guide Global Health Research Section and the International Collaborations Section, applicants who are eligible for CIHR funding may apply for funds to support research to be carried out in, or in direct collaboration with researchers and/or knowledge users based in other countries. CIHR supports international research projects and collaborations to address a range of areas, including but not limited to, established priorities in global health research and contributes to the development of health-research capacity both internationally and nationally.

2.2 Knowledge Translation (KT)

Knowledge translation is integral to CIHR's mandate and falls into two main categories: end of grant KT and integrated KT. With both categories of knowledge translation, CIHR expects researchers to disseminate their findings and facilitate their translation into improved health, more effective products or services, and/or a strengthened healthcare system. Note that the costs of dissemination are eligible expenditures in all CIHR grants.

For end of grant KT, many means of dissemination exist and the onus is on the researcher to select the most appropriate vehicle for the intended knowledge-user audience to ensure maximum impact. When the primary knowledge users are researchers, dissemination of results through the publication of articles in high quality and accessible journals is appropriate, although other strategies that increase awareness of the results and facilitate their application may also be appropriate. When knowledge-user audiences outside the research community should be informed of specific research findings, dissemination plans with more ambitious goals and comprehensive strategies are expected. With integrated KT, stakeholders or potential research knowledge users are engaged in the entire research process and the research is directed at producing solutions to issues or problems the stakeholders/knowledge users have identified.

2.3 Tri-Agency Open Access Policy on Publications

CIHR believes that greater access to research publications and data will promote the ability of researchers and knowledge users in Canada and abroad to use and build on the knowledge needed to address significant health challenges. Open access will promote accessibility to CIHR-funded research and will serve to increase the international visibility of Canadian research. CIHR grant recipients are reminded to adhere to the responsibilities outlined in the Tri-Agency Open Access Policy on Publications. Under this policy, grant recipients must ensure that research papers and bio-molecular data generated from CIHR funding are freely accessible online.

2.4 Gender, Sex and Health Research

Applicants are encouraged to demonstrate the use of Gender and Sex-Based Analysis in their applications. Gender and sex-based analysis is an approach to research which systematically inquires about biological (sex-based) and sociocultural (gender-based) differences between women and men, boys and girls, without presuming that any such differences exist. The purpose of this line of inquiry is to promote rigorous health research which expands the understanding of health determinants in both sexes and results in improvements in health and health care. One helpful reference document for applicants and peer reviewers is the Sex, Gender and Health Research Guide: A Tool for CIHR Applicants. This document includes CIHR's definitions for gender and sex-based analysis and information on applying gender and sex-based analysis to research proposals.

2.5 Official Languages

In order to ensure that positive measures are undertaken within CIHR's mandate to contribute to the vitality of English and French minority communities in Canada, CIHR:

  • encourages the appropriate inclusion of Official Language considerations and of English and French speaking Canadians, including those living in linguistic minority communities, in health research design, conduct and application to improve health outcomes, and
  • facilitates equitable access to its programs and services for health research stakeholders in official language minority communities.

3. Principles of Peer Review

The integrity of the review process relies on well-established principles and policies that:

  • ensure fair and effective evaluation; and,
  • support CIHR’s mandate and objectives.

CIHR’s principles of review are:

3.1 Confidentiality

Confidentiality is information about a person that will not be disclosed directly or indirectly to anyone else without that person's prior expressed consent. The information provided by applicants in their applications is protected by the Privacy Act and is made available to external assessors for reviewing purposes only. The use of this information for any other purpose is a breach of the Privacy Act and could result in a CIHR investigation and/or report to the federal Privacy Commissioner's Office.

CIHR instructs peer reviewers to not approach or communicate in any way with applicants, or anyone outside of the peer review process, any information relating to the review of a specific application, or offer opinions on the applicant’s chances of success or failure. In turn, applicants are not to contact peer review participants, including the Competition Chair and the Chair and Scientific Officers for the Final Assessment Stage meeting, regarding the status of their applications.

By law, applicants have access to their own application files. Therefore, all written materials used in evaluating an application are made available to the applicants when they are notified of CIHR's funding decision for a competition.

The identity of the reviewers will not be revealed to the applicants. However, a list of review committee members is typically published on the CIHR website approximately 60 days after the approval of funding of a competition.

3.2 Conflict of Interest

CIHR makes every effort to ensure that its decisions are fair and objective by identifying and addressing any conflict of interest between an applicant and a reviewer. According to the Conflict of Interest and Confidentiality Policy of the Federal Research Funding Organizations (COIC), a conflict of interest means a conflict between a participant's (e.g., reviewers and observers) duties and responsibilities with regard to the review process, and a participant's private, professional, business or public interests. There may be a real, perceived or potential conflict of interest when the participant:

  • would receive professional or personal benefit resulting from the funding opportunity or application being reviewed;
  • has a professional or personal relationship with an applicant or the applicant's institution; or,
  • has a direct or indirect financial interest in a funding opportunity or application being reviewed.

A conflict of interest may be deemed to exist or be perceived as such when committee members and observers:

  • are a relative or close friend, or have a personal relationship with the applicants;
  • are in a position to gain or lose financially/materially from the funding of the application;
  • have had long-standing scientific or personal differences with the applicants;
  • are currently affiliated with the applicants' institutions, organizations or companies—including research hospitals and research institutes*;
  • are closely professionally affiliated with the applicants, as a result of having in the last six years:
    • frequent and regular interactions with the applicants in the course of their duties at their department, institution, organization or company;
    • been a supervisor or a trainee of the applicants;
    • collaborated, published or shared funding with the applicants, or have plans to do so in the immediate future; or,
    • been employed by the institution, when an institution is the applicant; and/or
  • feel for any reason unable to provide an impartial review of the application.

All peer review participants (Reviewers, Competition Chair and Final Assessment Stage (FAS) Chair, Scientific Officer) and observers are subject to the same conflict of interest guidelines. CIHR staff is responsible for resolving areas of uncertainty. All peer review participants must read and agree to abide by the COIC policy prior to viewing any application information.

*It is important to manage both real and perceived conflicts of interest, but at the same time the review process should not suffer because of an overly strict application of the conflict of interest guidelines. For instance, a reviewer is not automatically in conflict with an application if he/she is from the same institution as the applicant, as he/she may not know or work with the applicant.

3.3 Fairness

Success of the review system is critically dependent upon the willingness and ability of all reviewers to:

  • exercise rigorous judgement;
  • be impartial and reasonable;
  • understand and take into account in a balanced way the particular context of each application; and,
  • provide constructive, quality review which helps the applicant by pointing out strengths and weaknesses that contributed to the application rating.

3.4 Transparency

CIHR ensures transparency in the review process through a number of different mechanisms. All applications submitted to CIHR are evaluated by reviewers who provide an overall assessment of the application. This assessment is performed using the application requirements and review criteria established for each individual competition, as seen in the relevant funding opportunity. In addition, the Scientific Officer (SO) notes capture the discussion during the peer review deliberations if the application is discussed at the Final Assessment Stage. All reviews and SO notes, as applicable, are shared with applicants.

On its website, CIHR publishes the Peer Review Committee Membership Lists and lists all CIHR successful applications by competition through its Funding Decisions Database.

4. Peer Review Roles

4.1 Reviewers (Stage 1 and Stage 2)

The majority of participants in the peer review process are reviewers who conduct the evaluation of the applications submitted to the competition. Reviewers are responsible for providing high quality reviews by reading and assessing the applications based on the evaluation criteria and participating in online discussion for applications where there are discrepancies among reviewers. Once the discussion period is finished, reviewers confirm the ranking of their applications and submit their final reviews to CIHR.

Principal tasks:

  • complete Conflict of Interest and Ability to review tasks;
  • evaluate/rate/rank applications;
  • submit preliminary reviews;
  • participate in online discussion of applications, as required;
  • submit final reviews;
  • provide feedback to CIHR regarding the review process.

Success of the peer review system is critically dependent upon the willingness and ability of all reviewers to exercise rigorous scientific judgment, provide fair and reasonable feedback, and understand and take into account each application’s specific context. The opinions expressed in the reviews are those of the reviewers. In circumstances where the Competition Chair or CIHR staff deems that the review contains comments that could be construed as sarcastic, flippant, arrogant, inappropriate in any way, or if the comments contain factually incorrect information regarding CIHR's policies, CIHR reserves the right to remove such information from the review.

A high quality review will:

  • be appropriate with no bias towards the applicant/research field/Institution;
  • have a tone, which is respectful and professional;
  • respect CIHR’s Conflict and Confidentiality policy;
  • have clear strengths and weaknesses;
  • have constructive and respectful justifications for each rating;
  • inspire confidence in the reviewer’s ability to fairly assess the application.

4.2 Competition Chairs (Stage 1 and Stage 2)

Competition Chairs are responsible for overseeing and supporting the Stage 1 and Stage 2 remote review process. Chairs will:

  • assist with reviewer recruitment and identify reviewers to fill expertise gaps;
  • validate the assignment of applications to reviewers;
  • identify applications that should be discussed based on discrepant reviews (Stages 1 and 2) and budget recommendations (Stage 2);
  • oversee and moderate the online discussions;
  • ensure quality reviews by identifying reviews that require revision and communicate with CIHR staff;
  • provide feedback to CIHR regarding the review process.

Competition Chairs are able to:

  • Flag applications for discussion in ResearchNet when there are discrepancies in the reviewers’ assessment of an application. At Stage 2, Competition Chairs will also be expected to identify applications with discrepant budget recommendations and guide the online discussion to reach consensus on budget recommendations, where possible. If an application is flagged, reviewers will receive an email indicating that the application must be discussed.
  • Send private messages directly to reviewers, using the private messaging function in ResearchNet. This approach may be used to address reviews with insufficient/inappropriate content.

One of the most important roles of the Competition Chair is to monitor the quality of reviews.

Stage 1 reviews will be provided to all applicants. Stage 2 reviews will be provided to the applicants as well as to the Final Assessment Stage committee members who will rely on these reviews to make funding recommendations. The following information should be included in Stage 1 and Stage 2 reviews:

  • Context: reviewers are asked to provide context in their assessment of each adjudication criterion.
  • Sufficient information: reviewers are asked to provide relevant and substantial information to inform decision making and recommendation for funding.
  • Justification of rating: reviewers are asked to provide sufficient information in the body of their review that would explain the choice of rating.

In monitoring the online discussions to ensure that all comments are appropriate, the Competition Chairs will also look for instances of:

  • information identifying the reviewer;
  • use of outside information;
  • mismatch between the comments and the rating being provided.

During online discussion, the time requirements for ensuring quality reviews will vary depending on:

  • total applications assigned to a Competition Chair;
  • number of applications that require discussion.

4.3 Reviewers (Final Assessment Stage - FAS)

Competition Chairs who participated in Stage 2 will be asked to participate as members in the Final Assessment Stage (FAS) face-to-face committee meeting. This multidisciplinary committee covers the full spectrum of health research. At this stage, respecting conflicts of interest, the reviewers discuss "grey zone" applications, with particular emphasis on applications with variances in independent reviewer rankings. This final stage therefore builds on Stage 2 where applications will have been thoroughly and expertly assessed. FAS members will be provided with the Stage 2 reviews and access to the full application. The Final Assessment Stage committee's recommendations will be summarized and submitted to CIHR for the final funding decision. The FAS Reviewer's duties include:

  • complete Conflict of Interest and Ability to review tasks;
  • assess Stage 2 reviews;
  • assign applications into “Yes” bin (considered for funding) and “No” bin (not to be considered for funded);
  • prepare for the FAS meeting;
  • attend the FAS meeting;
  • provide feedback to CIHR regarding the review process.

For each assigned application, members will have access to the following:

  • written reviews of each of the Stage 2 reviewers;
  • individual reviewer ratings and rakings from Stage 2;
  • stage 2 Consolidated Ranking; and
  • full application.

4.4 Chair and Scientific Officer (Final Assessment Stage – FAS)

The Final Assessment Stage committee also includes a committee Chair and a Scientific Officer (SO).

The Chair ensures that the peer review committee functions smoothly, effectively and objectively, in accordance with CIHR's policies and procedures. The Chair's duties include:

  • act as a moderator to establish a positive, constructive, fair-minded environment in which research proposals are discussed;
  • fulfill an oversight role ; and
  • work with the SO to summarize the discussion of each application before the voting, when required.

The Scientific Officer's duties include:

  • take notes of the discussion of each application, which will be sent to applicants with their review documents;
  • ensure that issues of ethics and other concerns that arise during discussion are flagged for the attention of CIHR are recorded for each application; and
  • support the Chair in his/her role during the peer review committee meeting.

The Chair and the Scientific Officer do not rate applications or vote during the committee meeting.

The Chair and Scientific Officer participate in the Budget Subcommittee meeting following the FAS meeting.

5. Equity in the Peer Review Process

Peer review by nature is a subjective process. Bias may manifest in several ways and could be based on a school of thought, fundamental versus applied or translational research, areas of research or approaches (including emerging ones), size or reputation of a participating institution, age, language, personal factors or gender of the applicant. To sensitize reviewers to unconscious biases they may hold, they will be cautioned against any judgment of an application based on such factors, and asked to constantly guard against the possibility of implicit bias influencing the decision-making process. CIHR has developed several learning modules to support reviewers. All reviewers must view the modules in the Excellence in Peer Review section, including Conducting Quality Reviews and Unconscious Bias in Peer Review. All peer review participants are encouraged to complete these learning modules and participate in webinars.

5.1 Addressing gender equity and reducing gender bias

CIHR is actively engaged in increasing gender equity in their review processes to address some of the gender inequities that we have observed from recent funding competitions.

The following are some tips for reviewers.

  1. Before engaging in peer review, reflect on the susceptibility of all humans to bias in judgment. The following resources can help:
  2. Allow sufficient time and avoid multi-tasking when reviewing applications to allow for self-correction of bias related tendencies.
  3. Review the assessment criteria before evaluating applications.
  4. Question whether your evaluation would change if the investigator were of a different gender?
  5. Guard against over-reliance on one piece of information or “first-impression”.
  6. When writing or presenting your review, please:
    • Avoid using stereotypical or interpersonal attributes when describing character and skills, (e.g., words like nice, hardworking, conscientious, dependable, diligent, kind, agreeable, sympathetic, compassionate, selfless, giving, caring, warm, nurturing, maternal, etc.);
    • Instead, focus on research skills/achievements using words that describe the candidates research excellence (e.g., thought provoking, innovative, novel, thorough, detailed, impactful);
    • Consider using 'stand-out' adjectives for both men and women, where appropriate (e.g., superb, excellent, outstanding, confident, successful, ambitious, knowledgeable, intellectual etc.);
    • Use the nominee's formal title and last name instead of their first name. Avoid attributing the contribution of an applicant's work to the order of authors, as not all disciplines follow a single convention;
    • Consider whether your comment unintentionally includes 'doubt raisers' (negative language, hedges, unexplained comments, faint praise and irrelevancies (e.g., 'might make an excellent leader' versus 'is an established leader').

6. Foundation Peer Review Process

This manual is divided into three sections which correspond to the three stages of the adjudication process as illustrated in Figure 1.

This figure illustrates the three-stage adjudication process for the Foundation.

  • Stage 1 assesses the Caliber of the Applicant(s) and Vision and Program Direction. Applicants submit their Stage 1 application, which is matched to up to five expert reviewers. Reviewers complete the Stage 1 remote review, participate in an online discussion and submit their results to CIHR. Successful applicants are invited to submit a Stage 2 application.
  • Stage 2 assesses the Quality of the Program and Quality of the Expertise, Experience and Resources. Reviewers complete the Stage 2 remote review, participate in an online discussion and submit their results to CIHR.
  • The Final Assessment Stage involves a face-to-face multidisciplinary committee meeting. This committee will make recommendations on which applications to fund.

Figure 1. Overview of the Foundation Grant Adjudication Process

Figure 1 – Long Description

This figure illustrates the three-stage adjudication process for the Foundation.

  • Stage 1 assesses the Caliber of the Applicant(s) and Vision and Program Direction. Applicants submit their Stage 1 application, which is matched to up to five expert reviewers. Reviewers complete the Stage 1 remote review and submit their results to CIHR. Successful applicants are invited to submit a Stage 2 application.
  • Stage 2 assesses the Quality of the Program and Quality of the Expertise, Experience and Resources. Reviewers complete the Stage 2 remote review and submit their results to CIHR.
  • The Final Assessment Stage involves a face-to-face multidisciplinary committee meeting. This committee will make recommendations on which applications to fund.

Section 1 outlines the adjudication process for Stage 1 of the competition. It describes the responsibilities of reviewers participating in this stage, which includes the following: 1) preliminary review of the applications, 2) online discussion, and 3) submission of the final rank list to CIHR.

Section 2 outlines the adjudication process for Stage 2 of the competition. It describes the responsibilities of reviewers participating in this stage, which includes the following: 1) preliminary review of the applications, 2) budget assessment, 3) online discussion, and 4) submission of the final rank list to CIHR.

Section 3 outlines the adjudication process for the final stage of the competition. It describes the responsibilities of reviewers participating in this stage, which includes the following: 1) pre-meeting activities, and 2) a face-to-face committee meeting.

Some reviewers may be invited to participate in both Stage 1 and Stage 2 if required. The Final Assessment Stage face-to-face committee will be composed of the Stage 2 Competition Chairs.

CIHR offers a number of learning modules to help peer reviewers gain in-depth knowledge about the program and processes.

6.1 Section 1 – Adjudication Process for Stage 1

Stage 1 focuses on the Caliber of the Applicant and the Vision and Program Direction.

6.1.1 Preliminary Reviews of Applications

Each application will be assigned to up to five reviewers, based on optimal matching between the application content and reviewer expertise. Each reviewer will receive between 8 and 12 applications. Reviewers will access their assigned applications through ResearchNet and complete their reviews remotely. It is expected that it will take approximately 1-3 hours to complete each review.

The application will be presented in a structured format to align with the adjudication criteria, outlined in this manual. Reviewers must consult section 6.1.2 - Stage 1 Adjudication Criteria Descriptors and Interpretation Guidelines for guidance on how to apply the adjudication criteria. Reviewers will also be asked to review the applicant CV(s). Through their CV(s), applicants will highlight their recognitions, funding history, activities and contributions that best demonstrate their leadership, significant contributions, and productivity in the context of their research field(s).

For each application, reviewers will be asked to provide a rating and justify the rating by stating the strengths and weaknesses for each sub-criterion outlined in the table below. The fields to outline strengths and weaknesses for each application in ResearchNet are mandatory fields. It is important for reviewers to clearly articulate the strengths and weaknesses as they will be used to: 1) provide applicants with feedback; and 2) provide other reviewers who were assigned to the same application with justification regarding the rating and ranking of that application.

In some cases, there may be multiple Program Leaders for a single Foundation grant. These applicants will need to demonstrate synergy and a history of co-leading programs of research. In these cases, the assessment of each adjudication criterion should consider both the individual and joint research contributions of the Program co-Leaders.

When assessing each criterion, reviewers must take into consideration the career stage, research field, and institution setting of all applicants. Mid career investigators may show evidence of leadership, contributions, and productivity in different ways as compared to senior investigators; the evidence should be compared to peers in similar fields and career stage.

To ensure consistency, reviewers must adhere to a common adjudication scale. Note that granularity has been built into the top descriptors of the scale acknowledging that many of the ratings are likely to fall into this range. Reviewers are encouraged to use the full breadth of the scale, as appropriate, and should use the increased granularity within the top descriptors to express differences within these categories. To facilitate this, the following scale including descriptors, definitions and numerical values is provided:

Descriptor Abbreviation Definition ResearchNet Value

Outstanding

O++

For this sub-criterion, the application excels in most or all relevant aspects. Any short-comings are minimal.

28

O+

26

O

24

Excellent

E++

For this sub-criterion, the application excels in many relevant aspects, and reasonably addresses all others. Certain improvements are possible.

22

E+

20

E

18

Good

G

For this sub-criterion, the application excels in some relevant aspects, and reasonably addresses all others. Some improvements are necessary.

12

Fair

F

For this sub-criterion, the application broadly addresses all relevant aspects. Major revisions are required.

6

Poor

P

For this sub-criterion, the application fails to provide convincing information and/or has serious inherent flaws or gaps.

0

Adjudication Criteria Stage 1:
  • Caliber of the Applicant(s) – 75%     
    • Leadership (25%)
    • Significance of Contributions (25%)
    • Productivity (25%)
  • Vision and Program Direction – 25%

The adjudication scale has 5 descriptors and each descriptor has a corresponding abbreviation and definition. The abbreviation (also known as the alpha rating) has been assigned a numerical value so that ResearchNet is able convert the alpha ratings into a rank order. The numerical values are multiplied by their respective weight, for each sub-criterion, and then totaled to calculate the overall rating. This rating will then be used to rank reviewer’s assigned applications. Please note that ResearchNet does all these calculations automatically resulting in a ranked list of the individual reviewer's set of assigned applications based on the ratings (highest to lowest). Reviewers may make adjustments to their ratings until they are comfortable with the rankings, however, there will be no requirement to break ties at this stage of the adjudication process. Ultimately, it is the application's ranking that will determine its standing within the competition and will be used to make funding decisions.

The reviewers will then be asked to submit their preliminary reviews to CIHR in order to proceed to the next step.

6.1.2 Stage 1 Adjudication Criteria Descriptors and Interpretation Guidelines

Stage 1 focuses on the caliber of the applicant in the context of the qualities deemed essential to meet the objectives of the Foundation grant. These qualities are: leadership, ability to deliver significant contributions, and productivity. Stage 1 also assesses the ability to define and articulate a clear and compelling vision and program direction. Note that reviewers will be selected based on their expertise in the applicant's field(s) of research.

In some cases, there may be multiple Program Leaders for a single Foundation grant. These applicants will need to demonstrate synergy and a history of co-leading programs of research. In these cases, the assessment of each adjudication criterion should consider both the individual and joint research contributions of the Program co-Leaders.

In the next sections, each of the sub-criteria relating to the Caliber of the Applicant and the Vision and Program Direction will be described in more detail. For each sub-criterion, reviewers will be reminded of: (1) the requirements the applicants were asked to include in the application; and (2) the specific review questions that have been defined for each sub-criterion. A set of interpretation guidelines and considerations have also been summarized for each sub-criterion. These are intended to provide guidance to reviewers as they assess each section of the application.

6.1.3 Stage 1 - Criterion 1: Caliber of the Applicant

Please note that for each sub-criterion under Caliber of the Applicant, reviewers must consider:

  • The applicant's recent experiences and contributions, taking into account accomplishments over the applicant's entire career;
  • The nature, breadth and depth of the applicant's experiences and contributions, in the context of the applicant's career stage;
  • The nature, breadth and depth of an applicant's experiences and contributions, in the context of the applicant's research field;
  • Personal circumstances that result in legitimate career interruptions, such as extended leaves of absence or decreased productivity for health, family or other reasons. Note that applicants are encouraged to explain any personal circumstances to ensure a fair assessment of their achievements. Furthermore, the application may include a PDF document to supplement the publication information included in the applicant’s CCV.

Sub-criteria to be assessed under this criterion include:

6.1.3.1 Leadership (25%)

This sub-criterion is intended to assess the extent to which the applicant has influenced and inspired others in their field of research, as well as the applicant’s ability to effectively direct a program of research.

  • What are applicants asked to provide?

    The applicants are asked to highlight their leadership experience. This should include: (1) a description that demonstrates leadership in the context of their career stage and field(s) of research; and (2) relevant and recent examples of major projects or research programs that they have led where effective leadership was evident. The applicant is asked to point to relevant, recent examples in their CV as appropriate.

  • Items for reviewers’consideration:

    • Is the applicant recognized in their field, demonstrating a history of holding influential roles, inspiring others, mobilizing communities and advancing the direction of a field?
    • Has the applicant demonstrated the ability to successfully establish, resource, and direct programs of research, which should include: securing the required resources; ensuring effective collaboration, and/or incorporating knowledge translation strategies?
  • Interpretation guidelines:

    • The applicant should be recognized and respected in their research field. This can be demonstrated through descriptions of the sphere of influence the applicant has achieved to date in their research field, their institution, within their research community, and any influence they have had in society at large. The applicant should also have experience in building and/or mobilizing networks and/or communities.
    • The applicant should demonstrate the ability to effectively direct a program of research. A Program Leader should demonstrate the ability to:
      • Address compelling research questions;
      • Develop and direct major projects and/or programs of research, which may include national or international networks (where appropriate);
      • Resource a program of research - including securing funding for research projects, attracting and retaining research personnel, acquiring appropriate infrastructure, etc.;
      • Build and manage collaborations; and/or,
      • Incorporate knowledge translation strategies and activities into a program of research, including the dissemination, application, and/or uptake of research findings.
6.1.3.2 Significance of Contributions (25%)

This sub-criterion is intended to assess the extent to which the applicant's contributions to date have had an impact (a tangible benefit, or positive influence) on health, health care, health systems and/or health research. These contributions could include advancements in different research and health-related areas, as well as contributions to maintaining a sustainable foundation of health researchers. The nature and quantity of these advancements may depend on the applicant's field of research and career stage.

  • What are applicants asked to provide?

    The applicants are asked to highlight the impact of their contributions on health and health research, describing tangible benefits and/or positive influences of their contributions on health, health care, health systems, and/or health research. The applicant is asked to point to relevant, recent examples in their CV(s), as appropriate.

  • Items for reviewers’ consideration:

    • Has the applicant significantly advanced knowledge and/or its translation into improved health care, health systems, and/or health outcomes?
    • Has the applicant engaged, trained, and/or launched the careers paths of promising individuals in research and/or other health-related non-academic fields?
  • Interpretation guidelines:

    • The applicant should demonstrate the impact they have had on advancing research knowledge and/or improving research practice in their field. Additionally, the applicant should demonstrate how they have contributed to the advancement of health care, health systems, health outcomes, and/or economic prosperity (as applicable). When assessing contributions, consider the following:
      • Contributions to the advancement of knowledge (health or other fields) include advances or improvements to the current thinking in the field of research that have been sustained.
      • Contributions to research (health or other fields) include broad-scale or transformative changes to current research practices and approaches.
      • Contributions to health care include improvements in diagnosis, treatment, and prevention of disease, conditions, disability, injury, and other physical and cognitive impairments.
      • Contributions to health systems include improvements in the organization of people, institutions, health information, health policies, and the human and financial resources to deliver health care services.
      • Contributions to health outcomes include improvements to the health and/or quality of life of individuals or populations.
      • Contributions to economic prosperity include the development of economic opportunities stemming from the application or commercialization of health research.
    • Applicants should demonstrate a history of contributions to the advancement of research capacity.
      • Contributions to a sustainable foundation for health research and knowledge translation (health or other fields) include supporting the mentoring/training, career development, and recruitment and/or retention of highly qualified personnel in academic and non-academic health-related fields.
      • Contributions to a sustainable foundation of health research from mid career investigators may vary as compared to senior investigators and might focus more on efforts to build capacity (e.g., ability to attract students/trainees/emerging scholars, developing and implementing innovative mentoring/training opportunities, etc.).
6.1.3.3 Productivity (25%)

This sub-criterion is intended to assess both the quality and quantity of research outputs generated by the applicant.

  • What are applicants asked to provide?

    The applicants are asked to highlight both their career and recent productivity, focusing on both the quality and quantity of research outputs. The applicant is asked to point to relevant, recent examples in their CV(s), as appropriate.

  • Items for reviewers’ consideration:

    • Has the applicant demonstrated an outstanding level (quantity) of research outputs in their field based on prior work?
    • Has the applicant’s previous work generated high-quality research outputs?
  • Interpretation guidelines:

    • The applicant should demonstrate an exceptionally high level of research outputs compared to investigators in a similar field and at a similar career stage.
    • The applicant’s previous work should have produced research contributions that are recognized to be of high-quality within their research field.

6.1.4 Stage 1 - Criterion 2: Vision and Program Direction

6.1.4.1 Vision and Program Direction (25%)

This sub-criterion is intended to assess the applicant’s ability to define and articulate a clear and compelling high level vision of their program of research. The assessment of the quality of the vision should consider the following characteristics:

  • What are applicants asked to provide?

    The applicant will be asked to articulate a compelling high level vision and direction for their research program. This will require the applicant to highlight the goal, overall objective(s), expected outputs/ contribution(s), and the significance of the proposed program if the objectives are met.

    Foundation grants are meant to be flexible in order to allow the Program Leader(s) an opportunity to innovate and explore new lines of inquiry. The information requested in this section is not intended to include details of each thematically-linked project within the program of research.

    If successful at Stage 1, applicants will be asked to expand on their program of research at Stage 2.

  • Items for reviewers’ consideration:

    • Are the vision, goal, overall objective(s), and potential contributions of the proposed research program well-defined and well-articulated in the context of a logical career progression for the Program Leader(s)?
    • Is the vision forward-looking, creative, and appropriately ambitious?
    • Does the vision aim to significantly advance knowledge and/or its translation to improved health care, health systems, and/or health outcomes?
  • Interpretation guidelines:

    • The vision should outline the program direction envisioned for the duration of the grant in a coherent and clear fashion, highlighting the key synergistic components of the program of research.
    • A forward-looking vision is future-oriented, and describes the proposed contribution(s) of the program of research (i.e., what the program intends to achieve in the short- and long-term). A creative vision describes the program's potential to foster innovative approaches, and/or innovative outputs, outcomes, and impacts. An appropriately ambitious vision is believable and realistic (e.g., sets an appropriate scope for the program of research). It considers key elements required for the success of the program, and sets the direction for program planning through a clearly articulated research program goal and supporting objectives.
    • The vision of the research program should aim to achieve significant advancement in health-related knowledge, health research, health care, health systems, and/or health outcomes. This could be defined by the importance of the program (e.g., based on an assessment of the declared benefits, and their relation to any current issues, gaps and opportunities), as well as the significance of short-term and long-term research outcomes.

CIHR offers a number of learning modules to help peer reviewers gain in-depth knowledge about the program and processes.

6.1.5 Online Discussion

Once a reviewer submits the preliminary reviews for a given application via ResearchNet, the reviews from other reviewer's for that application will be made available; reviewers will then be able to discuss each of their assigned applications with the other reviewers who were assigned to the same application. The online discussion tool has been developed to allow reviewers to communicate with each other to share expert perspectives and to discuss any discrepancies.

The objective of the discussion will be to discuss and understand discrepancies in reviews. There will be no requirement to reach consensus. Each reviewer's name and their preliminary review (ratings, written comments, and ranking) will be visible for each application. If there are limited differences in opinions among reviews for a given application, it is possible that a discussion may not be required.

The discussion period will last 72 hours and will be moderated by a Competition Chair. Reviewers will be made aware of the planned discussion period for the competition well in advance of the deadlines. Reviewers will not be expected to participate in online simultaneously.

Reviewers may log in at any time once the discussion period opens. However, to facilitate a productive exchange of opinions, reviewers are highly encouraged to log into ResearchNet frequently during the online discussion period. To maximize the exchange among reviewers, we ask that reviewers also participate in the online discussion as early in the 72-hours period as possible. It is important to know that applicants benefit from a thorough discussion of their applications and so it is important that reviewers make themselves available to log into ResearchNet frequently to exchange opinions on their applications throughout the online discussion period.

All reviewers assigned to an application will be able to participate in the discussion as soon as all of the reviews for that application have been submitted. The discussion for all applications will open on the established deadline and it is therefore important to respect the established deadlines. Reviewers who have not submitted their preliminary reviews will not be able to participate in the discussion and this will be flagged in their online status.

A Competition Chair will also be assigned to the application to moderate the discussion. The Competition Chair will be tasked with ensuring that discussions take place if warranted, and may prompt reviewers to discuss. CIHR staff may also initiate a discussion, or provide input where necessary, to ensure that CIHR policies and procedures are followed.

Once the discussion for each application is complete, reviewers will be given the opportunity to make adjustments to their reviews as required. This may include editing their comments, changing their ratings and/or rankings.

It should be noted:

  1. Every reviewer will have the ability to initiate a discussion thread. Every comment will be shared with every reviewer assigned to that application. Once comments are posted, reviewers will not be able to delete or edit them.
  2. Reviewers will be able to post comments for the attention of CIHR staff; however they will be visible to the other reviewers.
  3. A notification email will be sent daily to advise reviewers of new posts.
  4. As applications are discussed, reviewers may feel the need to update their preliminary ratings and reviews. This can happen at any time both during and after the discussion period. These modifications will be dynamically updated and visible to other reviewers.
  5. A transcript of the online discussions will not be given to the applicants. However, discussion thread content will be retained by CIHR as part of the application record and this record is subject to the Government of Canada’s Access to Information Policy and Privacy Acts.

CIHR offers a number of learning modules to help peer reviewers gain in-depth knowledge about the program and processes.

6.1.6 Submit the Final Rank List to CIHR

Once the discussions are complete, reviewers will be asked to finalize their reviews and their individual ranked list. At this point, reviewers must break any ties in their individual ranked list. Once ties are eliminated, reviewers may modify the rank order of their assigned applications by moving the applications up or down the rank list. The original rank order will remain visible as a reference point. Once satisfied with the rank order of applications that they have been assigned, the reviewers will be asked to submit the final rankings to CIHR. Note that reviewers will not see the final ranked lists of other reviewers. At this point, the adjudication process for Stage 1 is complete.

Once all rankings have been received, CIHR will calculate a consolidated ranking for each application and compile a list of all applications in the competition ranked from highest to lowest. Each application will have up to five reviews, a ranking from each reviewer, a consolidated ranking and a standard deviation. CIHR will determine which applicants will be invited to complete a Stage 2 application based on the number of applications, consolidated ranking, standard deviation and funds available for the competition.

CIHR offers a number of learning modules to help peer reviewers gain in-depth knowledge about the program and processes.

6.2 Section 2 – Adjudication Process for Stage 2

Stage 2 focuses on the Quality of the Program and Quality of the Expertise, Experience and Resources and is distinct from the Stage 1 application.

6.2.1 Preliminary Reviews of Applications

Each application will be assigned to up to five reviewers, based on optimal matching between the application content and reviewer expertise. Matching and assignments will be done on the basis of reviewer declarations of their ability to review and with the assistance and oversight of Competition Chairs. Reviewers will access their assigned applications through ResearchNet and complete their reviews remotely. Reviewers at Stage 2 will not have access to the materials and reviews from Stage 1.

The application will be presented in a structured format to align with the adjudication criteria, outlined in this manual. Reviewers must consult the Stage 2 Adjudication Criteria Descriptors and Interpretation Guidelines for guidance on how to apply the adjudication criteria.

Reviewers will be asked to provide a rating and to justify the rating by stating the strengths and weaknesses for each sub-criterion outlined in the table below. The fields to outline strengths and weaknesses for each application in ResearchNet are mandatory fields. It is important for reviewers to clearly articulate the strengths and weaknesses as they will be used for three purposes: 1) to provide the other reviewers assigned to the application with a justification for the ratings and rankings given to the application, 2) to provide reviewers in the final stage with a justification for the ratings and rankings given to the application, and 3) to provide applicants with feedback.

Reviewers must take into consideration the career stage, research field and institution setting of all applicants when assessing each criterion.

To ensure consistency, reviewers must adhere to a common adjudication scale. Note that granularity has been built into the top descriptors of the scale acknowledging that many of the ratings are likely to fall into this range. Reviewers are encouraged to use the full breadth of the scale as appropriate and should use the increased granularity within the top descriptors to express differences within these categories. To facilitate this, the following scale including descriptors definitions and numerical values is provided:

Descriptor Abbreviation Definition ResearchNet Value

Outstanding

O++

For this sub-criterion, the application excels in most or all relevant aspects. Any short-comings are minimal.

28

O+

26

O

24

Excellent

E++

For this sub-criterion, the application excels in many relevant aspects, and reasonably addresses all others. Certain improvements are possible.

22

E+

20

E

18

Good

G

For this sub-criterion, the application excels in some relevant aspects, and reasonably addresses all others. Some improvements are necessary.

12

Fair

F

For this sub-criterion, the application broadly addresses all relevant aspects. Major revisions are required.

6

Poor

P

For this sub-criterion, the application fails to provide convincing information and/or has serious inherent flaws or gaps.

0

Adjudication Criteria Stage 2:
  • Quality of the Program - 50%
    • Research Concept (25%)
    • Research Approach (25%)
  • Quality of the Expertise, Experience, and Resources - 50% 
    • Expertise (20%)
    • Mentorship and Training (20%)
    • Quality of Support Environment (10%)

The adjudication scale has 5 descriptors and each descriptor has a corresponding abbreviation and definition. The abbreviation (also known as the alpha rating) has been assigned a numerical value so that ResearchNet is able convert the alpha ratings into a rank order. The numerical values are multiplied by their respective weight, for each sub-criterion, and then totaled to calculate the overall rating. This rating will then be used to rank reviewer’s assigned applications. Please note that ResearchNet does all these calculations automatically resulting in a ranked list of the individual reviewer's set of assigned applications based on the ratings (highest to lowest). Reviewers may make adjustments to their ratings until they are comfortable with the rankings, however, there will be no requirement to break ties at this stage of the adjudication process.

Reviewers will also be asked to flag applications in ResearchNet that involve any of the issues outlined below. These issues are not to be considered as criteria for adjudication unless they impact the scientific quality of the application. For more information concerning these issues, please refer to the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS-2).

  1. Ethical and other policy requirements: Responsibility for ensuring that all research meets ethical standards is delegated to the local institution by CIHR. Ethics forms are not required as part of the application. However, the reviewer may flag specific issues, such as the use of human participants, animals, human tissues or hazardous material, or research that appears to involve Aboriginal people, if they feel they have not been adequately addressed.
  2. Human pluripotent stem cell research (PSCC): Applications involving the use of human stem cells which will be funded will also be reviewed by the Stem Cell Oversight Committee (SCOC). If the box is not checked by the applicant and the proposed work utilizes HPSC, then it is essential that this is flagged by reviewers.

Once the budget recommendation has been completed, the reviewers will then be asked to submit their preliminary reviews to CIHR in order to proceed to the next step.

6.2.2 Stage 2 Adjudication Criteria Descriptors and Interpretation Guidelines

Stage 2 focuses on the quality of the proposed program of research, including the research concept, research approach, expertise and capacity building, as well as the supporting environment for the proposed research program.

Reviewers are therefore selected on the basis of expertise to review the research concept and approach for each application. The applicants invited to submit an application in Stage 2 of the Foundation competition process have been selected based on their demonstrated past success in leading programs of research. They have proven themselves able to develop and implement detailed methodologies, techniques, etc., to bring research projects to fruition; and, capable of mobilizing and securing the appropriate expertise to deliver significant contributions. Therefore, the level of information provided by the applicants to address their proposed program of research at Stage 2 is expected to be at a high-level, and will not include details of each specific project within the program of research. Foundation grants are meant to be flexible in order to allow Program Leader(s) an opportunity to innovate, or explore new lines of inquiry. Reviewers will be selected based on their declared expertise regarding the applicant's field of research.

In the next sections, each of the sub-criterion relating to the Quality of the Program and the Quality of the Expertise, Experience and Resources, will be described in more detail. Reviewers will be reminded of what applicants were asked to provide in the application for the sub-criterion as well as the specific review questions that have been defined for each sub-criterion. A set of interpretation guidelines and considerations have also been summarized for each sub-criterion. These are intended to provide guidance to reviewers as they assess each section of the application.

6.2.3 Stage 2 - Criterion 1: Quality of the Program

Sub-criteria to be assessed under this criterion include:

6.2.3.1 Research Concept (25%)

This sub-criterion is intended to assess the research component of the program, as well as the significance of the anticipated outcomes.

  • What applicants have been asked to provide?

    The applicant will be asked to articulate the program of research by outlining the goal, objective(s), and, at a high level, the components of the program of research and potential short-term and long-term impact(s) of the proposed program.

  • Items for reviewers to consider:

    • Are the goal and objectives of the proposed program well-defined and well-articulated?
    • Is there conceptual coherence within the program of research?
    • Are the potential short-term and long-term program outputs significant? Are they likely to significantly advance health-related knowledge and/or its translation into improved health care, health systems, and/or health outcomes?
  • Interpretation guidelines:

    • The goal and objectives of the program should be well-defined, and appropriate given the nature and scope of the program of research. There should be a sound, conceptual justification for the component parts and should be grounded in the relevant literature and/or previous results (as appropriate).
    • The components of the program of research (including any relevant knowledge translation components) should be aligned with the overall goal and objectives of the program, and demonstrate an integration of concepts among components.
    • The significance and innovation of the proposed program of research should be assessed in relation to other research in the field, and the relative need for the outputs and outcomes of the program of research. Reviewers should also assess the importance of the short-term and long-term anticipated outcomes, as it relates to the likelihood of significantly advancing health-related knowledge, approaches/methodologies, health care, health systems, and/or health outcomes.
  • Other Considerations (if applicable)

  • International Considerations:

    Note that CIHR contributes to, and supports, international research projects and international collaborations to address a range of research areas such as global health research issues, and to contribute to the development of health-research capacity both internationally and domestically. For these types of programs, demonstration of significance should focus on showing that the international component(s) is relevant to the goals and objectives of the program of research, and that the collaboration has the potential to improve health outcomes in the broader global community.

6.2.3.2 Research Approach (25%)

This sub-criterion is intended to assess the quality of the approach of the proposed program of research.

  • What applicants have been asked to provide?

    The applicant will be asked to articulate their research approach (i.e., how they will deliver on the objectives of the program of research) and include:

    • Potential challenges to the approach;
    • Mitigation strategies that will be employed to overcome those challenges;
    • How progress and success of their proposed research program will be measured against key achievements.

    The applicant will not be expected to provide extensive details on established methodologies; however, they should appropriately describe novel/innovative approaches.

    Foundation grants are meant to be flexible in order to allow the Program Leader(s) an opportunity to innovate, and explore new lines of inquiry. A detailed research design and project-by-project plan for each thematically-linked project in the program of research is not expected.

  • Items for reviewers to consider:

    • Is the research approach appropriate to deliver on the proposed program objectives?
    • Does the approach allow for flexibility in direction as the program evolves?
    • Does the approach include a high-level description of how progress and success will be measured?
    • Does the approach include a plan for identifying potential challenges and applying appropriate mitigation strategies?
  • Interpretation guidelines:

    • The research approach encompasses a number of supporting elements. These elements should be well-planned, appropriate, and poised to deliver on the objectives of the program of research. A detailed research design and project-by-project plan for each thematically-linked project in the program of research is not expected.
    • The approach should be:
      • Flexible enough to explore new avenues of research; and,
      • Poised to maximize contributions through appropriate collaboration and knowledge translation.
    • There should be an appropriate plan to measure progress against key milestones.
    • There should be a high-level plan to identify and address any critical challenges or risks related to the key research and any knowledge translation components of the program. An exhaustive list is not required.
  • Considerations:

    The appropriateness of knowledge translation strategies will vary by research field. All knowledge translation strategies should be relevant to the context of the proposed program of research.

  • Mandatory Requirements (where appropriate)

    Evidence demonstrates that biological, economic, and social differences between women and men contribute to differences in health risks, health services use, health system interaction, and health outcomes. Therefore, all applicants to CIHR are expected to integrate gender and sex considerations into their research design, where appropriate, in order to maximize the relevance and applicability of health research findings to both men and women.

6.2.4 Stage 2 - Criterion 2: Quality of the Expertise, Experience, and Resources

6.2.4.1 Expertise (20%)

This sub-criterion is intended to assess the expertise and experience of the Program Leader(s), as well as the proposed Program Expert(s) (see definition below) to collectively deliver on the objectives of the proposed program. It is the responsibility of the Program Leader(s) to ensure that the proposed research program is poised for success.

  • What applicants have been asked to provide?

    The applicant will be asked to outline, at a high-level, the collective expertise, experience and resources being assembled, and how it is appropriate to ensure the delivery of the objectives of the proposed research program. This includes:

    • The expertise and experience (disciplinary, professional, or methodological) of the proposed Program Leader(s), as well as Program Expert(s) (e.g., researchers, technicians, knowledge-users, partners, patients and trainees, etc.);
    • The level of engagement (e.g., time commitment and contribution) of Program Leader(s);
    • As applicable, the commitment (cash or in-kind) from interested or engaged knowledge user(s) and/or applicant partners;
    • The coordinated roles of the Program Leader(s) and any Program Expert(s) in the oversight and management of the program of research;
    • A plan to seek out expertise (new Program Expert(s)) based on the anticipated future needs of the program of research, as it is expected that the Program Experts may evolve over the duration of the grant, based on the needs of the proposed program.
  • Items for reviewers to consider:

    • Does the applicant have the appropriate expertise and relevant experience to lead and manage the proposed program of research, considering its objectives and scope?
    • Is there an appropriate complement and level of engagement and/or commitment from key Program Expert(s) and (as applicable) applicant partners?
  • Interpretation guidelines:

    • The roles and responsibilities the Program Leader(s) should be clearly described, and linked to the objectives of the research program, referencing the CVs of the proposed Program Leader(s) as required. Please note that if the Program Leader(s) had a career interruption, their application may include a PDF document to supplement the roles and responsibilities information included in their CCV.
      • An overview of expertise assembled to undertake the program of research (i.e., Program Expert(s)); as well as a plan to seek out expertise (new Program Expert(s)) based on the anticipated future needs of the program of research should be outlined. Note that Program Expert(s) will not be submitting a CV with the Stage 2 application.
      • The level of engagement (e.g., time commitment and contribution) of the Program Leader(s) should be appropriate based on the roles and responsibilities described. The coordination of the Program Leader(s) and any Program Expert(s) in the oversight and management of the program of research should be clearly described. There should be no gaps in expertise or ability to lead and deliver on the objectives of the proposed program.
    • The Program Leader(s) should demonstrate an ability to plan for the future by securing a time commitment (e.g., participation in the described research program), a resource commitment (e.g., access to equipment) or a financial commitment (cash or in-kind) from identified Program Expert(s) and/or applicant partners.
  • Considerations

    For any knowledge translation approach, the applicant should have the relevant knowledge user(s) identified. In some cases, the appropriate knowledge user(s) is critical to achieving the desired impact of the research program, and, its ultimate success; hence, an integrated knowledge translation approach may be appropriate. Such an approach would require a commitment to a genuine relationship with the relevant knowledge user(s). Knowledge user(s) may be responsible and accountable for the application/uptake of the program outputs.

    A commitment (cash or in-kind) from interested or engaged knowledge user(s) and/or applicant partners is not an absolute requirement, but can be reasonably expected depending on the nature and type of knowledge user. In some cases, the financial commitment may support a critical step towards the application of the program outputs.

6.2.4.2 Mentorship and Training (20%)

This sub-criterion is intended to assess the quality of the mentorship/training plan, through the demonstrated commitment to/level of engagement in shaping the future of the applicant’s students, trainees, emerging scholars, and Early Career Investigators, as well as other individuals in non-academic, health-related fields.

  • What applicants have been asked to provide?

    The applicant will be asked to articulate a mentorship and training plan including:

    • An outline of training goals, learning opportunities, and key activities;
    • Appropriate mentoring/training approaches in relation to the proposed program of research and research field;
    • The rationale for the proposed training approach;
    • Potential challenges of the mentorship and training plan and a strategy for identifying and mitigating these challenges;
    • How progress and success will be measured.
  • Items for reviewers to consider:

    • Does the research program include a comprehensive mentorship and training plan for building capacity and positioning students, trainees, knowledge users, emerging scholars, and Early Career Investigators for successful research careers and/or other career paths in non-academic health-related fields?
    • Does the proposed plan demonstrate an appropriate approach for meeting its objectives in relation to the program of research and the research field?
    • Does the plan include a strategy for identifying and mitigating potential challenges?
  • Interpretation guidelines:

    • A comprehensive mentoring and training plan includes a combination of formal education, and informal mentoring, and training approaches. The plan should be coherently presented with training goals, learning opportunities, and key activities clearly articulated. Mentoring/training activities should focus on:
      • Skill development (e.g., technical/methodological, oral and written, teaching, grants management, budgeting, research values and ethics, and, if appropriate, lab management);
      • Career development (e.g., preparation for research or non-academic careers).
    • The mentorship and training plan should be appropriate to the field and career stage of the Program Leader(s), as well as the nature and scope of the program of research, and the specific training level(s) (e.g., graduate students, trainees, emerging scholars, Early Career Investigators, and/or health-related professionals). The benefits of the training and mentoring activities to the target audience should be clear. Program Leader(s) are expected to show direct involvement in all mentoring/training activities.
    • The risk management strategy for the mentoring/training component of the program should describe any potential challenges to recruiting, retaining, or mentoring/training that demonstrate a sensitivity to challenges that may be faced by graduate students, trainees, emerging scholars, Early Career Investigators, and/or health-related professionals (where applicable). There should also be an appropriate high-level plan to address any identified challenges.
  • Considerations:

    Activities and collaboration with individuals from non-academic spheres is an important consideration. Non-academic professionals benefit from both interactions with and access to research resources, and training on use of these resources. The reverse is also true.

6.2.4.3 Quality of Support Environment (10%)

This criterion is intended to assess whether the applicant has the resources necessary in order to successfully deliver on the objectives of the research program in both the short- and long-term.

  • What applicants have been asked to provide?

    The applicant will be asked to outline the resources that they currently have in place to ensure the successful delivery of the research program objectives, including:

    • physical infrastructure (and/or other types of infrastructure such as consortia, professional networks, etc.)
    • support personnel
    • equipment
    • specialized facilities
    • supplies
  • Items for reviewers to consider:

    • Is the described environment(s) appropriate to enable the conduct of the program of research, and to manage and deliver on the objectives and key components of the proposed research program (e.g., research, knowledge translation, mentoring/training) through the provision of, or access to, the required infrastructure?
  • Interpretation guidelines:

    • Depending on the nature of the program, access to environments in other organizations may be essential to deliver on the objectives of the program. Reviewers should consider that appropriate support environments may be found within an institution, or may be built through "networked" environments or platforms outside of the host institution.

    Examples of other environments include, but are not limited to: hospitals, long-term care facilities, experimental facilities (e.g., Canadian Light Source, Conseil Européen pour la Recherche Nucléaire (CERN), etc.), schools, First Nations communities, industry laboratories, prisons, foreign jurisdictions/locations, community populations, and specialized databases.

CIHR offers a number of learning modules to help peer reviewers gain in-depth knowledge about the program and processes.

6.2.5 Budget Assessment

Reviewers will be required to review the requested budget and justification. Applicants will be asked to submit their budget using a modular template with defined budget increments. Reviewers will be provided with the applicants’ baseline funding amount as calculated by CIHR and as provided to applicants in advance of submitting their Stage 2 applications.

Partner contributions, in cash or in-kind, if applicable, should be considered when reviewing support for the proposed program of research. Partner contributions will be outlined in letters of support and in a Partnership budget module through ResearchNet.

Please note that the budget assessment must not be factored into the scientific assessment.

During the online discussion, Stage 2 reviewers will be required to come to a consensus on their budget recommendations. The Competition Chair may communicate with Stage 2 reviewers if there is a need for clarification. In cases where a discrepancy still exists, the budget will be reconciled at the end of the Final Assessment Stage by the Budget subcommittee.

6.2.5.1 Stage 2 Reviewer Budget Recommendations

The budget recommendation, including any written comments, is intended to assist CIHR in determining the appropriate funding levels for individual programs of research.

  • What applicants have been asked to provide

    The applicants are asked to provide information regarding the budget request to support the core activities of the proposed program of research, including the total amount requested and the amount requested per annum.

    They need to demonstrate that the amount requested is appropriate to support the core activities of the proposed program of research, given the context of their baseline funding amount. Any increase above the calculated baseline amount will need to be justified by the applicant.

  • Items for reviewers to consider

    • Is the requested budget appropriate in order to support the core proposed program of research? Is it realistic and well-justified, in the context of the applicants' baseline funding amount?
    • If the request is significantly higher than the applicant's historical grant levels, is it appropriately justified?
  • Interpretation guidelines

    • Applicants are required to justify that their request is appropriate in the context of their proposed program of research by providing their total financial requirement for the duration of their grant (7 years) and an amount for each of the identified categories defined by CIHR with a concise justification for their allocation. A precise valuation of each line item within the budget justification is not expected nor will it be provided by the applicants. Additionally, a yearly breakdown of funds will not be provided.
    • Applicants with funding from CIHR must provide a robust justification for requests that are significantly higher than their historical grant levels.
      • Applicants without a CIHR funding history should justify the appropriateness of their request in the context of their past funding history.
      • Justifications for CIHR funding to replace other on-going sources of funding (e.g., health charity, provincial funding agency) are not acceptable.
  • Considerations

    Reviewers are asked to consider the proposed budget of the program, and provide a justified recommendation as to whether they:

    • recommend the proposed budget, as described by the applicant(s); or,
    • recommend a total reduction in the proposed budget of: $ _______.

    Applicants were provided their baseline amount in their application, as calculated by CIHR:

    • For applicants with prior CIHR funding history, this baseline amount was calculated based on identified eligible funding history.
    • For applicants with no prior CIHR funding history, applicants were provided instructions on what to submit in order for their baseline amount to be calculated.

For more information about how the baseline budgets are calculated, please refer to the ”Enter Budget Information” section of the Foundation Grant - Stage 2 Application and to the Budget Baseline Calculation instructions.

6.2.6 Online Discussion

The online discussion tool has been developed to allow reviewers to communicate with each other to share expert perspectives and to discuss any discrepancies. The objective of the online discussion period is to discuss and understand differences in reviews and budget recommendations.

The discussion period will last 72 hours and will be moderated by a Competition Chair. Reviewers will be made aware of the planned discussion period for the competition well in advance of the deadlines. Reviewers will not be expected to participate in online discussions simultaneously.

Reviewers may log in at any time once the discussion period opens. To maximize the exchange among reviewers, we ask that you participate in the online discussion as early in the 72-hour period as possible. It is important to know that applicants benefit from a thorough discussion of their applications and so it is important that reviewers make themselves available to log into ResearchNet frequently to exchange opinions on applications throughout the online discussion period. However, to facilitate a productive exchange of opinions, reviewers are highly encouraged to log into ResearchNet frequently during the online discussion period.

Once a reviewer submits the preliminary reviews for a given application in ResearchNet, reviews from other reviewers for that application will be made available. Reviewers will then be able to discuss each of their assigned applications with the other reviewers who were assigned to the same application. Reviewers who have not submitted their preliminary reviews will not be permitted to participate in the discussion and this will be flagged in their online status. The discussion for all applications will open on the established date. There will be no requirement to reach consensus for the scientific discussion but, if possible, reviewers are asked to reach a consensus regarding recommendations for the proposed budget. Each reviewer's name and preliminary review (ratings, written comments, budget recommendation and ranking) will be visible for each application. At this point, if any reviewer has suggested a change in the budget, this should also be discussed. If there are limited differences in opinions among reviews for a given application, it is possible that a discussion may not be required.

A Competition Chair will be assigned to each application to moderate the discussion. The Competition Chair will be tasked with ensuring that discussions take place if warranted, and may prompt reviewers to discuss the application. CIHR staff may also initiate a discussion, or provide input where necessary, to ensure that CIHR policies and procedures are followed.

Once the discussion for each application is complete, reviewers will be given the opportunity to make adjustments to their reviews as required. This may include editing their comments, changing their ratings and/or rankings.

It should be noted:

  1. Every reviewer will have the ability to initiate a discussion thread. Every comment will be shared with every reviewer assigned to that application. Once comments are posted, reviewers will not be able to delete or edit them.
  2. Reviewers will be able to post comments for the attention of CIHR staff; however they will be visible to the other reviewers.
  3. A notification email will be sent daily to advise reviewers of new posts.
  4. As applications are discussed, reviewers may feel the need to update their preliminary ratings and reviews. This can happen at any time both during and after the discussion period. These modifications will be dynamically updated and visible to other reviewers.
  5. A transcript of the online discussions will not be given to the applicants. However, discussion thread content will be retained by CIHR as part of the application record and this record is subject to the Government of Canada’s Access to Information Policy and Privacy Acts.

CIHR offers a number of learning modules to help peer reviewers gain in-depth knowledge about the program and processes.

6.2.7 Submit the Final Rank List to CIHR

Once the online discussions are complete, reviewers will be asked to finalize their reviews and their individual ranked list. At this point, reviewers must break any ties in their individual ranked list. Once ties are eliminated, reviewers may modify the rank order of their assigned applications by moving the applications up or down the rank list. The original rank order will remain visible as a reference point. Once satisfied with the rank order of applications that they have been assigned, the reviewers will be asked to submit the final rankings to CIHR. Note that reviewers will see the final rank lists of other reviewers. At this point, the adjudication process for Stage 2 is complete.

Once all rankings have been received, CIHR will calculate a consolidated ranking for each application and compile a list of all applications in the competition ranked from highest to lowest. Each application will receive up to five reviews, a consolidated ranking and a standard deviation for Stage 2.

CIHR will determine which applicants will be discussed at the Final Assessment Stage based on the number of applications, consolidated ranking, standard deviation and funds available for the competition. Based on the results from Stage 2, CIHR will identify the highest ranked applications that will be considered for funding at this stage. These applications will constitute the "green zone" and do not require further discussion.

A second set of applications will move to the Final Assessment Stage. These applications will constitute the "grey zone". The "grey zone" applications are those applications that were highly ranked and have a large standard deviation due to discrepancies in reviewer ranking.

The bottom-ranked applications will be deemed unsuccessful, and will not be considered for funding. After the Final Assessment Stage, a notice of decision will be provided to all applicants.

6.3 Section 3 – Adjudication Process for the Final Assessment Stage

In this stage, the Competition chairs who participated in Stage 2 will be asked to participate as members in the multidisciplinary committee covering the full spectrum of health research. The committee will include a Chair and a Scientific Officer (SO). This committee will be responsible for integrating the results of the Stage 2 reviews and making recommendations on which "grey zone" applications should be funded.

6.3.1 Pre-Meeting Activities

Prior to the face-to-face meeting, each committee member will first complete their conflicts and ability to review and will then be assigned a subset of applications. Each application will be assigned to three committee members. For each application, the committee member will have access to information from Stage 2, including the reviews, the consolidated rankings, standard deviations and the full applications.

  • Consolidated Ranking

    Applications are attributed a percent (%) rank from each reviewer, based on the ranking of their assigned list of applications. An application that is ranked first out of a list of applications for that reviewer is attributed a percent (%) rank of 100%. Conversely, an application that is ranked last is attributed a percent rank of 0%.

    The consolidated ranking (or Consolidated % Rank) is the arithmetic average (or mean) of all reviewers' individual rankings of the application expressed as a percent rank (i.e., the sum of the reviewer’s % ranks divided by the number of reviewers). In other words, the consolidated ranking represents the average of all reviewers’ percent (%) ranks for a given application.

  • Standard Deviation

    The standard deviation indicates the variability among reviewer rankings. Each application receives a % rank from each reviewer. Depending on the number of reviewers assigned to the application, an application may have up to 5 reviewer % ranks that are factored into the Consolidated % Rank.

    To contextualize the average (or Consolidated % Rank), it is important to also provide the standard deviation for this dataset, which is a statistical measure of how close the data points are to the mean. If the standard deviation is calculated at 0, this indicated that all data points are the same (i.e., 100% reviewer consensus). Therefore, the higher standard deviation would indicate that the data points are spread over a wider range of values.

In order to help committee members differentiate between applications, a binning system will be used. Reviewers will be asked to assign applications to a "Yes" bin (to be considered for funding) or a "No" bin (not to be considered for funding). Each reviewer will be allocated a minimum number of applications that may be placed in the "Yes" and "No" bins. This number will depend on the funds available for a given competition. Committee members will be asked to submit their recommendations to CIHR prior to the meeting.

Based on the "yes/no" binning recommendations reviewers made as part of the pre-meeting activities, CIHR will rank all the Final Assessment Stage applications in order from the highest ranked to lowest ranked. The primary sort criteria will be the number of "Yes" votes received, and the secondary sort criteria will be based on the Stage 2 results.

The Chair of the committee will have an opportunity to review the recommendations from the committee members prior to the meeting. CIHR staff will prepare a schedule for the face-to-face committee meeting in consultation with the Chair and SO. An illustration of the process is provided in Figure 4.

Figure 4. Pre-Meeting Activities Overview

Reviewer assigned a subset of applications.

Reviewer reads the Stage 2 reviews of their assigned applications and the Stage 2 application, if required.

Based on the reviews, the Final Assessment Stage reviewer must choose which applications should be considered for funding and which ones should not. The reviewer will receive a minimum number of applications that they may place in the "Yes" Bin (to be considered for funding), and the rest of the applications will be placed in the "No" Bin (not to be considered for funding).

Based on the "yes/no" binning recommendations reviewers made as part of the pre-meeting activities, CIHR will rank all Final Assessment Stage applications in order from the highest ranked to lowest ranked. The primary sort criteria will be the number of "Yes" votes received, and the secondary sort criteria is based on ranking after Stage 2 results.

6.3.2 Face-to-Face Committee Meeting

The multidisciplinary committee will be asked to make recommendations on which applications in the "grey zone" should be considered for funding.

At the meeting, committee members will see the grant applications ordered from highest-rated to lowest-rated according to a combination of the pre-meeting "yes/no" binning recommendations and Stage 2 results. Based on these results, CIHR will place the applications into one of three groups:

  1. Group A (applications that the assigned committee members recommended should be funded);
  2. Group B (applications for discussion at the meeting);
  3. Group C (applications that the assigned committee members recommended should not be funded).

The meeting will start with the committee Chair asking the committee members to validate the groupings of applications. To validate, the committee Chair will ask if any committee member would like to move any of the proposed Group A or Group C applications into Group B so that they may be discussed by the committee. Once the committee is satisfied with the groupings, the Group A and Group C applications will be locked in and will not be discussed any further at the meeting.

The Chair will moderate the discussion of all Group B applications, and the SO will prepare the notes that summarize the key points discussed for each application.

For each application that is discussed, the following process will be used:

  1. Any committee member in conflict with the application will be asked to leave the room.
  2. The Chair will ask the three reviewers who reviewed the application to indicate whether they voted "Yes" or "No" during the pre-meeting activities.
  3. The Chair will ask the assigned member who was the Competition Chair at Stage 2 to start the discussion and then the other members to continue the discussion. The discussion will focus on the key points that led to their "yes/no" assessments.
  4. Committee members not assigned to the application will be asked to contribute to the conversation by asking questions and adding their opinions/advice/expertise to clarify points of disagreement.
  5. The Scientific Officer will read the SO notes, which will capture the key elements of the discussion, to the committee for approval.

Once all of the applications in Group B have been discussed, members will be asked to vote on the Group B applications. Members will be provided with a maximum number of "Yes" votes (to be considered for funding) before the discussions begins. This maximum will be based on both the number of applications in Group B and the estimated funding cut-off line.

The committee Chair will go through the Group B applications one at a time, calling for a vote on each application. Using the electronic voting tool, committee members will vote either "Yes" or "No" for any application with which they are not in conflict. The Chair and SO will not be permitted to vote.

Committee members will be expected to adhere to the Conflict of Interest and Confidentiality Policy of the Federal Research Funding Organizations. While committee members will not be in the room during the discussion of the applications with which they are in conflict, they will not be asked to leave during the voting process. They will not be permitted to vote on any in-conflict applications.

The committee will conclude its work after all the members have reviewed the applications they recommend for funding. These recommendations will be summarized and submitted to CIHR for the final review of budgets.

An important component of any peer review committee meeting is the final review of the committee's effectiveness and functioning, as well as a discussion of policy issues that may have arisen in the course of its deliberations. This discussion is important as it provides an opportunity for CIHR staff to address any concerns of the committee members and for staff to record feedback on the adjudication process.

CIHR will make the final funding decision. Applicants will be informed of the results of the competition once CIHR's President has approved the grants to be funded. Applicants can view their notice of decision, reviews and Scientific Officer Notes (if applicable) on ResearchNet. Funded applications that have been flagged for special attention by CIHR staff are withheld as “Pending”. Applicants are notified if further information is required by CIHR.

Applicants, research administrators and reviewers will be asked to complete surveys after each stage to provide their feedback. Feedback received through the surveys will help the continuous improvement of the Foundation Grant Program.

Annexes

Interpretation guidelines compendium: sample indicators of Foundation Grant adjudication criteria

In the process of working towards interpretation guidelines of Foundation Grant adjudication criteria, CIHR consulted extensively with researchers from across all of CIHR’s research pillars. Working groups from each pillar contributed to the development of sample indicators of each of the following adjudication criteria: Leadership, Significant Contributions, and Productivity.

The list of sample indicators is provided below. The sample indicators are meant to be illustrative, and in no way represent a comprehensive list. Please also note the following:

  1. The list of sample indicators describes a variety of examples from all of CIHR’s pillars. Therefore, not all of the items in the list will be relevant to each applicant.
  2. The list may be updated over time based on feedback from the research community.
  3. Reviewers will be selected based on their expertise in the research field of the applicant, and CIHR is depending on the reviewers’ expertise to determine whether the applicant has included the relevant indicators for each adjudication criterion, while also taking into consideration the applicant's career stage, field of research, and other relevant conceptual factors.

Annex A - Leadership Sample Indicators

  • Advisor in interdisciplinary or inter-organizational collaborations
  • Editorial experience
  • Innovator in existing or emergent areas of research
  • Leader of research endeavours with major impact
  • Leader of provincial, national and international collaborations, societies, associations, etc. within and outside the research community
  • Leader of community-based research and/or knowledge translation initiatives
  • Number and type of awards, honours, distinctions (note that these vary by research field)
  • Number and type of leadership roles/appointments (e.g., key note speaker, committee, advisory group, network, expert witness, appeals counsel, etc.)
  • Number, type and value of previous grants awarded
  • Number and value of salary awards received
  • Peer review committee service

Annex B - Significant Contribution Sample Indicators

Note that the type of contributions listed may vary by research field. Note that for Stage 1 applications, the applicant has been asked to describe his/her contributions according to the following headings:

Contributions to the Advancement of Knowledge (Health or other fields)

  • Important advances in understanding (e.g., informing controversies, examining assumptions, advancing education theories, improving theoretical understanding in integrative research);  
  • Significant advances in knowledge (e.g., pathogenesis, cell differentiation, biological pathway, medical imaging, evaluation of cognitive functioning, gene-environment interactions, barriers to medication adherence, synthesis);  
  • Significant impact on a current area of research (e.g., paradigm shifts; challenges to current thinking).

And/or

Contributions to Research (Health or other fields)

  • Development of new areas of research
  • Development of theoretical frameworks/models
  • Development of research methods/techniques
  • Benefit of sustained change in research practice (e.g., ethics considerations)
  • Impact of new research approaches (e.g., multi-disciplinary approaches, horizontal (system-wide) research focus, etc.)

And/or

Contributions to the Foundation of Researchers (Health or other fields)

  • Number and types of domestic and international trainees mentored
  • Increased ability to attract students/trainees/emerging scholars
  • Implementation or broader uptake of innovative mentoring/training approaches
  • Career success of mentees/trainees
  • Success of mentees/trainees in working within both academic and non-academic spheres

And/or

Contributions to Health Care

  • Evidence-informed health care practices
  • Evidence-informed health promotion and prevention programs
  • Evidence-informed treatments, therapies, interventions or disease management approaches
  • Evidence-informed health technologies, tools, diagnostics and devices
  • Increased patient satisfaction
  • Reduced burden of illness

And/or

Contributions to the Health System

  • Evidence-informed policies (health or other) that positively impact health
  • Efficiencies from improved resource allocations or practices (e.g., cost savings, reduced wait times)
  • Improvements in health care data, information management and infrastructure
  • Reduced health inequalities in a specific population
  • Effectiveness of new or improved health care services, practices and regulations
  • Improved health professional education policies or education system
  • Cultural, organizational or pedagogical changes that have improved health professional competency

And/or

Contributions to Health Outcomes

  • Improvements to the health status or quality of life of populations
  • Clinical outcomes of health research interventions
  • Incidence and prevalence of chronic and acute illnesses
  • Changes in culture, attitudes and behaviour that improve health
  • Public health awareness and education (documentaries, classroom/course material, and films)
  • De-bunking stereotypes/myths

And/or

Contributions to Economic Opportunities

  • Business opportunities created
  • Creation of jobs from spin-off companies or from expansion of existing company
  • Sales/revenues generated or licensing returns
  • New products, processes or services commercialized
  • License or royalty agreements

Annex C - Productivity Sample Indicators

  • Publications, peer-reviewed or other (e.g., theses, journal articles, books, book chapters, workshop reports, synthesis reports, dissemination report)
  • Collaborations or networks
  • Communication and knowledge translation products or resources
  • Competency frameworks (e.g., health professionals)
  • Conferences and conference proceedings
  • Consultations
  • Databases and evidence repositories
  • Devices (e.g., medical, robotic)
  • Editorial contributions
  • Intellectual property claims
  • Invention disclosures
  • License agreements
  • Media interviews (e.g., television, radio, print)
  • Networking activities (e.g., workshops)
  • Organization of meetings
  • Patents (filed or obtained)
  • Policy briefs
  • Presentations at public forums
  • Prevention or intervention programs
  • Products
  • Professional practices
  • Programs or services
  • Prototypes
  • Public information, resources, and tools
  • Software and hardware
  • Spin-off companies
  • Standards and guidelines
  • Systems
  • Theories, models or frameworks
  • Tools, techniques, instruments, procedures or methods
  • Training approaches/curricula

The competition process begins at Stage 1 with the applicant submitting a registration followed by a structured application. CIHR conducts an eligibility check and matches applications to reviewers. The reviewers identify their conflicts and their ability to review for a subset of applications. CIHR establishes the assignment of applications to reviewers; approval of the assignments is done by the Competition Chairs. The Competition Chairs assist CIHR in filling expertise gaps by recommending additional reviewers. The reviewers conduct the preliminary reviews of their assigned applications; the review process is monitored by the Competition Chairs via the online discussions. The reviewers complete their final reviews and determine the final rank list of their assigned applications. CIHR then establishes the consolidated rank of all the applications and determines the subset of applications that move to Stage 2.

Applicants who move on to Stage 2 are invited to submit a Stage 2 structured application. CIHR will provide a calculated baseline budget amount that will be included in the Stage 2 application. This process triggers the same competition process as outlined above. CIHR once again calculates a consolidated ranking for each application; CIHR then compiles a list of all applications in the competition ranking from highest (“green zone”) to lowest (“red zone”) to determine which of the Stage 2 “grey zone” applications (i.e., those applications that are close to the funding cut-off and that demonstrate a high degree of variability in reviewer assessments) should be discussed at the Final Assessment Stage (FAS or Stage 3). The FAS review committee is composed of the Stage 2 Competition Chairs.

As part of the FAS, prior to the face-to-face meeting, reviewers identify their conflicts and ability to review for a subset of applications. FAS reviewers then have access to information on the applications assigned to them, including Stage 2 reviews, consolidated rankings, standard deviations, and the full applications. FAS reviewers use a binning system to differentiate between applications by assigning them to a “Yes” bin (to be considered for funding) or “No” bin (not to be considered for funding), and submit their recommendations to CIHR prior to the meeting.

At the face-to-face meeting, the multidisciplinary committee identifies, discusses, and votes on the “grey zone” (also known as Group B) applications they recommend for funding. In cases where a discrepancy in the budget still exists, the budget is reconciled by the Foundation Budget Subcommittee. After the Final Assessment Stage, CIHR reviews the recommendations from the FAS and the Foundation Budget Subcommittee to make its final funding decisions; a notice of decision is provided to all applicants.

An illustration of the peer review process is shown in Figure 5.

Figure 5. Foundation Grant Peer Review Process

Date modified: