CIHR Peer Review Manual for Clinician Scientist Award Applications

Revised: December 2009

Table of Contents

1. Purpose of the Manual

The purpose of this manual is to provide information on CIHR's objectives, governance and policies; to outline the roles and responsibilities of peer review committee members evaluating Clinician Scientist Award applications; and to define the policies and procedures for the peer review of the applications.

This manual is addressed primarily to committee members, but is also of use to applicants in explaining the peer review process for the Clinician Scientist Award applications from submission to final judgment. For detailed regulations concerning all aspects of CIHR funding programs, please see the Grants and Awards Guide.


Top of Page

2. Peer Review at CIHR

The mandate of the Canadian Institutes of Health Research (CIHR) is as follows:

"To excel, according to internationally accepted standards of scientific excellence, in the creation of new knowledge and its translation into improved health for Canadians, more effective health services and products and a strengthened health care system."

The purpose of peer review is to ensure excellence in the research funded by CIHR. The peer review system also ensures accountability, not only to the Government of Canada and the Canadian taxpayer - the source of CIHR funding - but to the research community at large. Peer review is carried out by committees of experts (peer review committees) that encompass all four pillars of health research (Biomedical, Clinical, Health Systems and Services, and Population and Public Health).

CIHR funds research through both open and strategic competitions. Open competitions accept proposals in any area of health research. Strategic competitions are sponsored by one or more of CIHR's Institutes or Branches and applications are solicited in specified areas of health research. In general, the same policies and procedures are followed for both types of competitions, unless otherwise specified in the funding opportunity. A list of current and archived CIHR funding opportunities can be found on the CIHR Funding Opportunities Database.

Peer review is overseen by CIHR's Scientific Council (SC), which governs all aspects of research-related decision making. SC develops, implements and reports on CIHR's research and knowledge translation strategies, in accordance with the CIHR Act and the overarching strategic directions set out by the Governing Council. The approval of funding for all research and knowledge translation initiatives is an integral part of SC's responsibilities.


Top of Page

3. Principles of Peer Review

3.1 Confidentiality

Integrity of the peer review process depends on well established principles of confidentiality. All information contained in applications, reviewer reports, Scientific Officer notes and committee discussions is strictly confidential. The applications and any discussions thereof may not be used for any purpose beyond that for which they were originally intended. Committee members must not discuss with applicants or reviewers any information relating to the review of a specific application outside of the committee meeting. Applicants must not contact committee members, including the Chair and Scientific Officer, regarding the status of their applications (ratings, rank within committee, etc.). All requests for information on an application or a reviewer report should be referred to CIHR Program Delivery staff responsible for the committee in question. Please see the website for CIHR's policy on Conflict of Interest and Confidentiality in the context of Merit, Relevance and Peer review (CCIP).

By law, applicants have access to their own application files. Therefore, all written material used in evaluating an application is made available to the applicants when they are notified of CIHR's decision. The identity of the reviewers will not be revealed to the applicants under any circumstances. However, a list of peer review committee members will be published on the CIHR website 60 days after the Scientific Council approves funding for a competition cycle.

All materials related to the review process provided to peer review committee members must be stored in a secure manner to prevent unauthorized access. They must be transmitted using secure carriers and technologies. When they are no longer required, all material related to peer review must be destroyed using a secure method or returned to CIHR for destruction. Any loss or theft of materials related to peer review must be reported to your committee coordinator immediately.

3.2 Conflict of Interest

CIHR must make every effort to ensure not only that its decisions are fair and objective, but also that they are seen to be so. According to CIHR's CCIP policy, any otherwise eligible reviewer may be considered for membership on a review committee unless he/she:

  1. has disclosed a potential conflict of interest in regard to the competition to be reviewed, and has been determined to be in material conflict of interest in regard to the competition, by CIHR's Chief Financial Officer (CFO) or his/her delegate,
  2. has funding-decision authority for the competition, or
  3. is a Principal Applicant on an application to be reviewed by that committee.

A committee member is considered to have a conflict of interest with an application if he/she:

  • is from the same immediate department, institution, organization or company as the applicant, and who interacts with the applicant in the course of his/her duties at the department, institution, organization or company;
  • has collaborated, been a co-applicant or published with the applicant, within the last five years (exception will be made for CIHR funded networks designed to increase partnerships among disciplines, institutions and thematic research);
  • has been a student or supervisor of the applicant within the last ten years;
  • is a close personal friend or relative of the applicant;
  • has had long-standing scientific or personal differences with the applicant;
  • is in a position to gain or lose financially from the outcome of the application (e.g., holds stock in the company of an industry partner or a competitor); or
  • for some other reason feels that he/she cannot provide an objective review of the application.

All committee members (Chair, Scientific Officer, reviewers, etc.) are subject to the same conflict of interest guidelines. CIHR staff and the Chair are responsible for resolving areas of uncertainty during the committee meeting.

All committee members must read and agree to abide by the CCIP policy prior to viewing any application information. This task is performed electronically (on ResearchNet) or by using a form provided by CIHR.

3.3 Fairness

Success of the peer review system is critically dependent upon the willingness and ability of all committee members to be fair and reasonable; to exercise rigorous scientific judgment; and to understand, and take into account in a balanced way, the particular context of each application. Reviews are provided to the applicant without prior editing by CIHR staff, and CIHR does not take responsibility for their content. An applicant will not accept that your review is fair if it contains comments that could be construed as sarcastic, flippant, arrogant, or inappropriate in any way. Conversely, a constructive review, which includes helping the applicant by pointing out deficiencies that could be repaired in a resubmission, will help to convince a disappointed applicant that you provided a fair assessment of the proposal.


Top of Page

4. Policies Impacting Peer Review

4.1 International Collaborations

As stated in the CIHR Act, one of the ways CIHR fulfills its mandate is by "pursuing opportunities and providing support for the participation of Canadian scientists in international collaborations and partnerships in health research." As a result, CIHR accepts applications for research to be carried out in, or in collaboration with applicants based in, other countries. The international nature of the research should not be a factor in the scientific assessment of the proposal, beyond how it relates to the feasibility of the proposed research and the quality of the research question. For detailed information on applying for funding with an international partnership component, please see the subsection titled "Global Health Research" in the Grants and Awards Guide.

4.2 Knowledge Translation

Knowledge translation is integral to CIHR's mandate and falls into two main categories, end of support KT and integrated KT. With both categories of knowledge translation CIHR expects researchers to disseminate their findings and facilitate their translation into improved health, more effective products or services, and/or a strengthened healthcare system. Note that the costs of dissemination are eligible expenditures in all CIHR grants.

For end of support KT, many means of dissemination exist and the onus is on the researcher to select the most appropriate vehicle for the intended knowledge-user audience to ensure maximum impact. When the primary knowledge users are researchers, dissemination of results through the publication of articles in high quality and accessible journals is appropriate, although other strategies that increase awareness of the results and facilitate their application may also be appropriate. When knowledge-user audiences outside the research community should be informed of specific research findings, dissemination plans with more ambitious goals and comprehensive strategies are expected. With integrated KT, stakeholders or potential research knowledge users are engaged in the entire research process and the research is directed at producing solutions to issues or problems the stakeholders/knowledge users have identified. Further information is available on the KT Portfolio website.

4.3 Access to Research Outputs

Applicants and peer reviewers are reminded that the CIHR Policy on Access to Research Outputs applies to all funding awarded after January 1, 2008. Award recipients must ensure that all published peer-reviewed articles are freely available online within six months, where possible under publisher agreements. Authors can adhere with the policy by archiving peer-reviewed manuscripts in an open access repository (e.g. institutional repository) or by publishing in an open access journal. Please consult the policy web site for more detail.

4.4 Guidelines for Research Involving Aboriginal People

The Ethics Office of the CIHR, in conjunction with the Institute of Aboriginal Peoples' Health, has prepared guidelines to assist researchers and institutions in carrying out ethical and culturally competent research involving Aboriginal people. The intent is to promote health through research that is in keeping with Aboriginal values and traditions. These guidelines will assist in developing research partnerships that will facilitate and encourage mutually beneficial and culturally competent research and will promote ethics review that enables and facilitates rather than suppresses or obstructs research. Further information can be found on the CIHR Guidelines for Health Research Involving Aboriginal People webpage.

4.5 Gender and Sex-Based Analysis

Applicants are encouraged to demonstrate the use of gender and sex-based analysis in applications. Gender and sex-based analysis is an approach to research which systematically inquires about biological (sex-based) and sociocultural (gender-based) differences between women and men, boys and girls, without presuming that any such differences exist. The purpose of this line of inquiry is to promote rigorous health research which expands understanding of health determinants in both sexes and results in improvements in health and health care. For further information, please see the Gender and Sex-Based Analysis in Health Research: A Guide for CIHR Researchers and Reviewers webpage.

4.6 Official Language Minority Communities

Federal agencies are required to take positive measures to ensure the support and recognition of minority language communities in Canada. For CIHR, this means an obligation to promote health research in these communities. For further information, please see the Official Language Minority Communities (OLMC) webpage. Research proposals in these areas should still be subject to the same rigorous peer review process as any other application. However, the justification for promoting health research in minority language communities should not be a factor in the assessment.

4.7 Publications and Productivity

An important evaluation criterion in all Clinician Scientist Award programs is the excellence of the applicant(s). A key factor in assessing this criterion is the productivity of the applicant(s), as determined by the quality and impact of contributions to the field. When assessing the quality of publications, peer review committees should focus on the quality of a publication's content and not simply the number of publications nor the quality or impact factor of journals. In the case of multi-authored publications or other collaborative work, applicants are advised to describe their contribution and reviewers should assess the specific contribution of the applicant to the work.

CIHR funds researchers in many health-related areas, and the forms of research publications can vary greatly among disciplines. In addition to the more traditional peer-reviewed journals, health researchers also publish in books, monographs, memoirs or special papers, review articles, conference/symposia proceedings and abstracts, government publications, etc. Some fast-moving research fields, such as some areas of computing science, genetics or microelectronics, use special means to reach the target audience quickly. Communications, quick-print reports, letters and electronic distribution of pre-prints are important vehicles for disseminating research results. All such contributions should be treated equally when assessing quality and impact, and reviewers should not regard certain types as "second class" or "grey literature."

When assessing productivity, reviewers should also be sensitive to legitimate delays in research and dissemination of research results. Some circumstances make it impossible or undesirable for researchers to publish important results of their research prior to applying for CIHR support. For instance, the time required to complete a monograph may exceed the time available between consecutive applications, or the protection of intellectual property may require a delay in publication. Research productivity may also vary as a result of personal circumstances, such as pregnancy or early child care, administrative leave, disability, elder care, etc., whether or not a formal leave of absence is taken. Applicants are advised to clearly and fully describe any circumstances that affect the dissemination of research results in their application. Peer review committees must be sensitive to the impact of these circumstances on the level of productivity, while ensuring that the quality of the research remains competitive.


Top of Page

5. Clinician Scientist Award Peer Review Committee Members

A typical CIHR Clinician Scientist Awards peer review committee consists of a Chair, Scientific Officer, peer reviewers and CIHR staff. Individual committee members are selected for their research excellence, as reflected by their ability to obtain continued extramural peer-reviewed grant support, and for their breadth of knowledge and maturity of judgment. Please see the website for peer review membership guidelines. Committees as a whole should also satisfy the need to cover the range of research areas for which the committee is responsible, to appropriately represent the Canadian health research community as a whole, to review in both official languages, and to allow for the logistics of conflict of interest and turnover of committee members. Please see the website for the procedure for selection of peer review committee members.

CIHR salary and training award competitions can be held on a recurring or an ad hoc basis. For recurring competitions, standing peer review committees are formed, and committee members are recruited for a term of service (typically three years) in order to ensure consistency and continuity in the review process. Standing committee membership may be supplemented by additional members as required for a specific competition, based on the applications received and expertise needed for their review. For ad hoc competitions, committees are formed to review applications for that particular competition and then disbanded.

5.1 CIHR Staff

CIHR staff are typically represented by a Deputy Director and a Program Delivery Coordinator, who are responsible for ensuring the integrity and quality of the peer review process. CIHR staff:

  • are involved in the assignment of applications to peer reviewers;
  • provide advice and guidance to the committee on CIHR policies;
  • keep notes on procedural aspects of the committee's functions;
  • record the consensus rating for each application;
  • record concerns raised by the committee on issues requiring later attention by staff, for example, ethics, eligibility, etc.

5.2 Chair

The committee Chair is directly responsible to CIHR for ensuring that the peer review committee functions smoothly, effectively and objectively, according to CIHR's policies. He/she establishes a positive, constructive, fair-minded environment in which the applications are to be evaluated. The Chair (and the Scientific Officer) fulfills an oversight role and does not rate applications before the committee. His/her responsibilities include:

  • working with the Scientific Officer and CIHR staff during the assignment of applications to specific peer review committees and during the selection of reviewers for each application;
  • working with CIHR staff to manage conflicts of interest;
  • appointing a delegate as Chair or Scientific Officer when either leaves the committee room due to conflict of interest. Whenever possible, the same individual should not occupy both Chair and SO roles;
  • ensuring the involvement of the entire committee with recommendations for each application;
  • working with the Scientific Officer to summarize the discussion of each application before the rating;
  • guiding the committee to a consensus rating;
  • ensuring that specific concerns of ethics and other CIHR requirements are addressed.

5.3 Scientific Officer

In addition to the duties shared with the Chair, as described above, the Scientific Officer:

  • supports the Chair in his/her role during the peer review committee meeting;
  • takes notes of the discussion as it is proceeding ("SO notes"), which will be sent to applicants;
  • ensures that issues of ethics, eligibility, and other concerns that have been flagged for the attention of CIHR are recorded for the applicant.

Under exceptional circumstances, CIHR staff may assume the role of Scientific Officer during the committee meeting.

5.4 Internal Reviewers

Internal reviewers are committee members who attend the peer review committee meeting, normally in person but occasionally by teleconference. Applications are assigned to a minimum of two internal reviewers for assessment, who review them in depth and submit a written review that is provided to the applicant after the committee meeting. Internal reviewers present their review at the peer review committee meeting and lead the review of applications assigned to them (see also Section 6.2.3). They also participate in the discussion and rating of all other applications before the committee for which they are not in conflict.

On occasion, a reviewer with a very specific expertise may be called on to review a small number of applications, typically by teleconference. These reviewers only take part in the discussion of the application(s) they have been assigned, and they rate the application(s) by e-mail to maintain confidentiality.

5.5 Observers

Occasionally, individuals are permitted to observe peer review committee meetings. Observers are typically CIHR Institute staff or representatives from partner organizations who have no funding decision-making authority for that competition. Observers must adhere to the same CCIP Policy as all committee members, and they do not contribute in any way to the review process or discussions surrounding the applications (including any discussions that arise during breaks). Observers may not remove any notes or other information related to the review of applications they observe from the meeting room.


Top of Page

6. Clinician Scientist Awards Application Review Process

6.1 Before the Meeting

6.1.1 Assigning Applications

All eligible applications received by the appropriate deadline date (posted in the Funding Opportunity) are entered into the competition. Applications must be complete at the time of submission, otherwise they are withdrawn from the competition. Specific exceptions to this rule can be found in the funding opportunity descriptions.

The Chairs and Scientific Officers of the peer review committees together with CIHR staff review the applications assigned to their committee. Together, they are responsible for ensuring their committees are equipped with the appropriate expertise and, upon accepting an application for review by their committee, accept responsibility for ensuring that the committee performs a fair review.

After the list of applications is compiled, committee members are given access to specific sections of the applications to declare any conflicts of interest and indicate their level of expertise on ResearchNet.

Chairs, Scientific Officers and CIHR staff then assign the applications to two internal reviewers and possibly a reader. All committee members are then given access to the full applications assigned to their committee four to six weeks before the peer review committee meeting.

6.1.2 Reviewing Applications

Read all of your assigned applications before rating any of them. Note that you are reviewing two different types of applications: Training Awards, these are the Phase One applications; and Salary Awards, these are the Phase Two applications. As such, ensure that you rate each application using the appropriate selection criteria described in Section 8 of this guide. It is recommended that reviewers separate their Phase One and Phase Two applications and devote their attention to one type of application at a time.

As you examine each application, jot down notes to capture your impressions. The Reviewer Worksheets, located in Annex II (for Phase One applications) [ PDF (127 KB) | Help ] and Annex III (for Phase Two applications) [ PDF (59 KB) | Help ] of this guide, provide templates that you could use. Note that the worksheets will not be filed with CIHR.

Be alert to unconscious bias related to gender, discipline or geographic location. Remember that:

  • Career interruptions for child bearing and raising can influence opportunity for knowledge production, publications and related variables;
  • Different disciplines and environments offer different opportunity for publication; and,
  • The reputation of institutions should not affect your view of candidates or their research training environment.

You are expected to prepare a written internal reviewer' report for all the applications for which you are assigned as either the first or second reviewer. Your internal reviewers' reports should include a brief summary of the qualities of the candidate and their research objectives and areas to be improved. The Internal Reviewer Report should include comments on the below mentioned evaluation criteria. The report should be clear and concise, using objective and non-inflammatory language, and include justification. Constructive advice to the applicant will allow him/her to improve the quality and efficiency of the proposed research. The applicant will receive the review as it is submitted by the reviewer. For this reason, please do not identify yourself in order to ensure the confidentiality of the review process. Note that it is still the responsibility of all peer review committee members to familiarize themselves in advance of the meeting with all applications to be assessed by their committee.

In advance of the meeting, reviewers are required to complete the following tasks:

  1. finalize reviews;
  2. provide an initial rating for each application (note that reviewers are not bound by this initial rating and can change it at the peer review committee meeting).
  3. divide the applications reviewed into a top and bottom group based on their overall quality. The top group should include the applications considered to be highly competitive and most deserving of being funded. This assessment is to be based on the reviewers' experience and will be used during the streamlining phase of the review (Section 6.2.2). The proportion of applications in each group may vary depending on the overall quality of the pool of applications reviewed relative to the reviewers' experience.

6.2 During the Meeting

The prime responsibilities of a peer review committee are to evaluate applications submitted for a particular competition and to rate them so that they may be ranked in order of priority and excellence. It is important that committees follow defined procedures in order to function in a consistent manner. For a summary of the review procedure, please see Appendix I.

Any committee member who has a conflict of interest with an application (as defined in Section 3.2) must not take part in the discussion of that application. Committee members in conflict must leave the room before the application is discussed. The Chair and CIHR staff are responsible for monitoring conflicts and for resolving areas of uncertainty.

6.2.1 Attendance at the Committee Meeting

Committee meetings are held usually within six months of the application deadline date and last not more than three days. The effective and fair review of applications depends on all committee members participating for the full duration of the committee meeting. Therefore, if your meeting starts in the morning, please arrive the night before instead of traveling the morning of the meeting, to avoid travel delays. In addition, please avoid planning an early departure to return home at the end of the meeting, in case the meeting runs late on the last day. If the meeting ends early, CIHR will cover certain costs to change your travel plans if necessary. Your MKI Travel Agent can provide further details and help you plan your itinerary accordingly.

6.2.2 Streamlining

In order to allow reviewers to devote more time to the consideration of applications that have the highest probability of being funded, streamlining may be applied to restrict discussion of non-competitive applications. Assessment of each application at peer review committee meetings begins with both internal reviewers announcing their initial ratings, to one decimal place.

An application is streamlined if it meets the following three conditions:

  • both reviewers placed the application in their bottom group (Section 6.1.2);
  • the average of the internal reviewers' initial ratings is <3.50;
  • there is no objection from the other committee members that the application not be discussed.

If an application is not discussed, the applicant will receive a copy of all internal reviewers' reports but there will be no Scientific Officer notes. Committee members do not vote on the rating; it is calculated as the mean of the initial ratings of the two internal reviewers.

6.2.3 Rating of Applications

If an application is not streamlined, the discussion proceeds as follows:

  • The primary reviewer presents his/her assessment, describing strengths and weaknesses of the application;
  • The secondary reviewer follows, concentrating on points of agreement or disagreement and elaborating points not addressed by the first reviewer;
  • The Chair leads the discussion of the application by all committee members;
  • The Scientific Officer reads back the Scientific Officer notes, capturing the key elements of the discussion to be considered when rating the application;
  • The Chair seeks a "consensus rating" from the two internal reviewers. The internal reviewers may revise their initial ratings as they see fit. If a consensus cannot be reached, the mean value of the ratings of the two internal reviewers is used (round up, if necessary, to obtain a single decimal point);
  • All committee members, including the two internal reviewers but excluding the Chair and Scientific Officer, then cast individual confidential votes within ±0.5 of the consensus rating. The internal reviewers are not bound to the consensus rating. The rating assigned to the application is the average of these confidential votes. A vote is taken even if the consensus rating is <3.5 (i.e., not in the fundable range).

6.2.4 Term

The appropriateness of the requested term of support is discussed, and recommendations are made.

6.2.5 Flagging of Applications for Special Attention

Any concerns in the following areas should be discussed and, if necessary, flagged for CIHR staff to address. These issues are not to be considered as criteria for evaluation, except as they may impact on the scientific quality of the application. For detailed regulations concerning these issues, please see the Grants and Awards Guide.

  1. Eligibility: Reviewers should raise any concerns with respect to whether the Principal Applicant(s) and their affiliated institutions meet the criteria to receive CIHR funding.
  2. Ethics: Responsibility for ensuring that all research meets ethical standards is delegated to the local institution by CIHR. Ethics forms are not required as part of the application. However, the reviewer may comment on specific issues, such as the use of human subjects, animals, human tissues or hazardous material, or research that appears to involve Aboriginal people, if they feel they have not been adequately addressed.
  3. Human pluripotent stem cell research: Applications involving the use of human stem cells and likely to be funded will also be reviewed by the Stem Cell Oversight Committee (SCOC). Applicants are instructed to check the relevant box in the section entitled "Certification Requirements", but it is essential that this be verified by committee members.

6.2.6 End of Meeting Review

Once all applications have been reviewed, if the peer review committee feels that any application(s) has been treated inconsistently, re-review of one or a small number of applications is permitted. Any committee member with a conflict of interest must again leave the room. Following discussion, a consensus rating is determined by the two internal reviewers and voting proceeds as before. The committee does not review the overall rankings of all applications at the end of the meeting as individuals with conflicts of interest would inevitably be present.

An important component of any peer review committee meeting is the final review of the committee's effectiveness and functioning, and a discussion of policy issues that may have arisen in the course of its deliberations. This discussion provides an opportunity for CIHR staff to address any concerns of the committee members and for staff to record feedback on the peer review process as part of CIHR's ongoing efforts to maintain an effective and high quality peer review system.

6.3 After the Meeting

Applicants are informed of the results of the competition once the SC has approved the awards to be funded. All applicants are sent a Notice of Decision, indicating whether or not their application was approved. The Notice of Decision will normally be released (either on ResearchNet or by mail) within three weeks following the SC meeting. A list of successful applicants is posted on the CIHR Funding Decisions Notifications webpage as soon as it is available.

Applications which have been flagged for Special Attention (see Section 6.2.5) are withheld as "pending". The applicant will be notified if further information is required. The additional information may be discussed by CIHR staff and peer review committee members if necessary prior to a final decision regarding funding.


Top of Page

7. Types of Applications

Clinician Scientist Award applications may be new applications, resubmissions of a previously unsuccessful applications, or renewal applications. All application types are evaluated together "on a level playing field" and the same criteria and funding cut-offs are applied to all, though peer review committee members are reminded to take the stage of career and previous progress made into account and to vary the emphasis placed on track record appropriately. Note that Phase One and Phase Two applications are evaluated independently using separate evaluation criteria and cut-offs.


Top of Page

8. Evaluation Criteria

8.1 Evaluation Criteria for Clinician Scientist Training Awards – Phase 1

Clinician Scientist Training Awards - Phase One selection criteria are based on studies of the predictors of successful post-training research activity. The following criteria are used to review these applications:

  1. Achievements and Activities of the Candidate
    1. Candidate's plans (Training expectations)
    2. Proposed research project
    3. Honours, awards and academic distinction
    4. Publications and related research achievements
  2. Characteristics and Abilities of the Candidate
  3. Training Environment
    1. Research activity, resources and mentorship

A.1. Candidate's Plans

Working Definition: A description of the candidate's career intentions and proposal for achieving them.

What to Look For: Clarity and logic in the explanation of the candidate's plans for a research career and the relevance of the proposed training.

Rating Range Benchmark
4.5 to 4.9 Faultless depiction of research career intentions and relevance of the proposed training.
Ideal career path.
4.0 to 4.4 Clear, convincing depiction of research career intentions and relevance of the proposed training.
Highly appropriate career path.
3.5 to 3.9 Very good depiction of research career intentions and relevance of the proposed training.
Logical career path.
3.0 to 3.4 Reasonable depiction of research career intentions and relevance of the proposed training.
Suitable career path.
2.0 to 2.9 Mediocre
1.0 to 1.9 Weak
0 to 0.9 Not acceptable

A.2. Proposed Research Project

Working Definition: A carefully planned, systematic study aimed at clearly answering a question in health research.

What to Look For: The ideal project is one that is best for the candidate given their education, experience and interests. It is the right balance of challenge, importance of the research question and feasibility in relation to the candidate's experience and training.
Bear in mind that it is not the project per se that is being assessed. It is the project as an integral part of the candidate's development as a researcher.

Rating Range Benchmark
4.5 to 4.9 Extraordinary optimization of: challenge to the candidate, scientific importance and feasibility of completion during the training period.
An ideal project that is faultlessly outlined
4.0 to 4.4 Excellent optimization of: challenge, importance and feasibility.
A highly suitable project that was superbly outlined.
3.5 to 3.9 Strong optimization of: challenge, importance, and feasibility.
A very suitable project that was very clearly outlined.
3.0 to 3.4 Good optimization of challenge, scientific importance and feasibility.
A suitable project that was well outlined.
2.0 to 2.9 Mediocre
1.0 to 1.9 Below average
0 to 0.9 Not acceptable

A.3. Honours, Awards and Academic Distinction of the Candidate

Working Definition: Official recognition or prizes signifying special qualities of the recipient. Includes accomplishments in terms of formal education and scholarship.

What to Look For: In assessing this variable and other achievements of the candidate, it is essential that you take into consideration the career path that they have followed to date. Assess the number, importance and breadth of the candidate's special distinctions relative to their education, training and work experience. Note relevance to research and whether the recognition is regional, national or international. Note the length of time required to complete academic programs and any indications of special academic distinction.

Rating Range Benchmark
4.5 to 4.9 Extraordinary
All aspects of the candidate's distinctions (their number, importance and breadth) indicate recognition of a very rarely encountered level of talent.
4.0 to 4.4 Excellent
Several aspects of the candidate's distinctions (their number, importance or depth) indicate recognition of superb talent.
3.5 to 3.9 Very Good
At least one aspect of the candidate's distinctions (their number, importance or depth) indicates recognition of a talent.
3.0 to 3.4 Good
The candidate's distinctions indicate an above-average performance.
2.0 to 2.9 Mediocre
1.0 to 1.9 Below Average
0 to 0.9 Not acceptable

A.4. Publications and Related Research Achievements of the Candidate

Working Definition: Articles, chapters or books published (particularly peer-reviewed) as well as conference presentations, abstracts and evidence of practical impact such as patents or copyrights.

What to Look For: Evidence of achievement in research relative to opportunity to date. Bear in mind that opportunity to publish may vary according to research discipline and life course (e.g., time spent raising children).
For publications, observe the number of co-authors and the position of the candidate's name in the authors list. Note the candidate's role in publications and their estimated percent contribution to the work.
Try to get a sense of the entire body of work and its likely impact. Note the publication dates and relate them to the candidate's education and training. Consider the list of abstracts as an indication of conference presentation activity. Note the candidate's other professional activities. Consider any patents or copyrights to which the candidate contributed.

Rating Range Benchmark
4.5 to 4.9 Brilliant
All aspects of the candidate's publications and related research achievements (number, likely impact and breadth) indicate an extraordinarily productive and creative individual.
4.0 to 4.4 Excellent
Several aspects of the candidate's publications and related research achievements (number, likely impact or breadth) indicate excellent productivity and creativity.
3.5 to 3.9 Strong
At least one aspect of the candidate's publications and related achievements (number, likely impact or breadth) indicate very good productivity or creativity.
3.0 to 3.4 Good
There is evidence of greater than expected involvement in publication and related research activities.
2.0 to 2.9 Mediocre
1.0 to 1.9 Below Average
0 to 0.9 Not acceptable.

B. Characteristics and Abilities of the Candidate

Working Definition: A perspective on the candidate provided by persons who are familiar with her/his characteristics and abilities.

What to Look For: Evidence from the sponsors that the candidate exhibits the characteristics and skills that correlate with career research achievement. Examine the sponsor's scores, recognizing that high scores are common while low scores are not. Read the supporting text carefully, taking note of the extent to which they justify the scores.
Look particularly for indications that the sponsors perceive the candidate as an investigative type, that is, someone whose thinking is critical, questioning, original and independent.
Look also for indications that the sponsors perceive the candidate as both energetic and capable of being highly focused.
If the candidate has had an opportunity to conduct research, look for mention of creativity in setting research goals, designing experiments, developing new methodologies, interpreting findings and presenting results in writing.

Rating Range Benchmark
4.5 to 4.9 Extraordinary
An outstandingly critical, original and independent thinker.
Exceptionally focused, energetic and creative.
An ideal role model for others.
4.0 to 4.4 First-Rate
A highly critical, original and independent thinker.
Very focused, energetic and creative.
Excellent potential for future research leadership.
3.5 to 3.9 Strong
Clearly a critical, original and independent thinker.
Definitely focused, energetic and creative.
Very good potential for success as an independent researcher.
3.0 to 3.4 Good
Appears to be a critical, original and independent thinker.
Seems to be focused, energetic and creative.
Above average potential for a productive career in research.
2.0 to 2.9 Mediocre
1.0 to 1.9 Weak
0 to 0.9 Not acceptable.

C. Research Activity, Resources and Mentorship in the Training Environment

Working Definition: Elements of the research milieu that will contribute directly or indirectly to the quality of the candidate's research training experience.

What to Look For: Review information on the education, research experience, qualifications, honours and awards of the supervisor. Examine the supervisor's publication record to get a sense of productivity, impact and collaboration.
Determine the space, facilities and personnel support available. Review the information on grants currently held, noting the extent to which the supervisor was "principal or co-applicant" for the funds. Get a sense of the resources available and the overall level of activity.
Review the supervisor's training record. Note for each person listed: the level of training, length of time with the supervisor, degree received (if applicable) and current position. Your assessment should take into consideration the career stage and discipline of the supervisor. Your expectations of mentoring by a recently-established investigator should differ from your expectations of mentoring by a long-established researcher.

Rating Range Benchmark
4.5 to 4.9 Exceptional
A vibrant, world-class research environment.
Outstanding availability of research resources.
Superb mentorship.
4.0 to 4.4 Excellent
A highly active research environment.
Excellent availability of research resources
First-rate mentorship.
3.5 to 3.9 Strong
A very active research environment.
Very good availability of research resources.
Strong mentorship.
3.0 to 3.4 Good
An active research environment.
Sufficient research resources available.
Appropriate mentorship.
2.0 to 2.9 Mediocre
1.0 to 1.9 Below Average
0 to 0.9 Not acceptable.

Please refer to the Reviewer Worksheet available in Appendix II [ PDF (127 KB) | Help ] to assist in preparing your review. This document is for your use only and therefore will not be forwarded to applicants.

8.2 Evaluation Criteria for Clinician Scientist Salary Awards – Phase 2

The evaluation of Clinician Scientist Salary Award – Phase 2 applications is based on the following three criteria:

  1. Environment and Support
  2. Research Proposal
  3. Track Record of the Candidate

According to the objectives of the Clinician Scientist Salary Award funding program, the evaluation of these applications should be focused on the applicant, rather than the proposed research project/program. Although the research proposal is to be evaluated, its review should be directed at the overall quality of the presentation and its reflection on the candidate's ability to pursue independent research. Reviewers should bear in mind that it is not the project per se that is being assessed, but how it is represented as an integral part of the candidate's development as a independent researcher.

To assess the candidate's environment and support, reviewers should evaluate:

  • What is (or will be) the applicant's position within the institution? Note that it is not necessary for New Investigator candidates to hold a faculty appointment at the time of application.
  • What space, operating funds and/or infrastructure will be provided to the candidate?
  • Has the institution demonstrated a commitment to protect the candidate's research time?
  • Does the institution or organization demonstrate leadership in the candidate's chosen field?
  • Comment on the applicant's letters of support.

To assess the research proposal, reviewers should evaluate:

  • Are the ideas put forward innovative and/or original?
  • Is the proposal well written and focused on the problem to be solved? Has a reasonable hypothesis been generated?
  • Could the project or program of research extend significantly our understanding of the area?
  • The quality and extent of proposed dissemination and outreach activities within and beyond the academic community.
  • Is the project feasible, given the resources and support available to the investigator?

To assess the applicant's track record, reviewers should evaluate:

  • The academic and research training received by the applicant.
  • What prestigious awards or acknowledgements of academic achievement has the applicant received?
  • Does the applicant have a good publication record in peer-reviewed journals? To what extent does he/she appear to have contributed to the work published? Consider patents and other significant contributions as well.
  • What research has been accomplished to date and has the applicant clearly demonstrated independence and originality? Significant contributions to team research should also be considered.
  • Does the applicant demonstrate leadership in the field? What is the quality of the applicant's training, mentorship and supervisory activities?
  • What peer-reviewed grant funding has been obtained or applied for? Note that it is not necessary for Clinician Scientist Salary Award candidates to hold an operating grant at the time of application.

This last criteria requires special note: by holding peer reviewed operating funds, the candidate demonstrates the ability to elaborate competitive research proposals and has been judged by their peers as having the potential to conduct high quality research. This information also allows the review committee to gain a broader idea of the candidate's research program and to judge whether the candidate holds the necessary funds to make pursuit of the proposed research feasible. However, note that it is not necessary for Clinician Scientist Salary Award candidates to hold an operating grant at the time of application.

Please refer to the Reviewer Worksheet available in Appendix III [ PDF (59 KB) | Help ] to assist in preparing your review. This document is for your use only and therefore will not be forwarded to applicants.


Top of Page

9. Rating Scale

The criteria to assess the scientific merit of applications are as described in Section 8. The relative weighting of these criteria depends on the program objectives as described in the funding opportunity description; if in doubt, please contact the Program Delivery Coordinator.

To ensure consistency, all reviewers must adhere to a common scale. It is particularly important that the full scale be used and the same conventions applied to assign ratings. To facilitate this, the following scale and descriptors should be used:

Descriptor Range Outcome
Outstanding 4.5 - 4.9 May be Funded -
Will be Discussed by the Committee
Excellent 4.0 - 4.4
Very good 3.5 - 3.9
Good* 3.0 - 3.4 Not Fundable -
May or may not be Discussed by the Committee
Average 2.0 - 2.9
Below average 1.0 - 1.9
Not acceptable 0 - 0.9

* Only applications rated 3.5 or higher are eligible for CIHR funding. The range 3.0 to 3.4 should be used for applications which, while rated as good, are not considered to be a high priority for CIHR funding. Please note that applications rated 3.0 to 3.4 are not eligible for CIHR funds, including those from partnership programs, and may or may not be discussed by the committee.


Top of Page

Appendix I: Sequence of Steps for Peer Review of a Clinician Scientist Award Application

  1. Members in conflict leave the room. The two internal reviewers announce their initial rating.
    Note: ratings can be different from those previously posted on ResearchNet
  2. Assessment of overall quality:
    Review is terminated if the following conditions are met:
    1. application is not considered competitive by both reviewers (placed into bottom group by both reviewers)
    2. the mean of the rating of the two internal reviewers is <3.50
    3. there is no objection from other committee members
    Committee members will not vote. The rating is calculated as the mean of the rating of the two internal reviewers. No SO notes will be taken.
  3. Internal Reviewers:
    The primary reviewer presents his/her assessment, describing strengths and weaknesses of the application followed by the secondary reviewer, who should concentrate on points of agreement or disagreement and elaborate on points not addressed by the first reviewer.
  4. Reader raises additional issues (if applicable)
  5. Discussion of application should focus on:
    • factors important in rating
    • differences of view between reviewers
  6. Scientific Officer reads SO notes to the committee:
    • summary of discussions
    • strength and weaknesses of application
  7. Consensus rating by internal reviewers:
    • use full scale
    • check consistency with previous applications
    If a consensus cannot be reached, use mean of internal reviewers' ratings.
  8. Individual ratings:
    • ± 0.5 of consensus rating
    • confidential vote
    • internal reviewers are not bound to consensus rating
  9. Term of Award
  10. Issues to be flagged:
    • ethics
    • eligibility
    • human stem cells
  11. Scientific Officer reads final notes for review / modifications / additions by committee