Update on the Project Grant and Foundation Grant competitions

January 19, 2017

Dr. Jeff Latimer answers questions from the research community regarding the Project Grant and Foundation Grant competitions

Dr. Jeff Latimer, CIHR's Associate Vice-President, answers questions regarding peer review quality, engaging early career investigators (ECIs), competition schedules, and gender bias.

Additional details about the Project Grant competition and the Foundation Grant competition are available on CIHR's website.

Transcript

Jeff Latimer: Hi, I’m Jeff Latimer, the Associate Vice-President of Program Operations here at CIHR.

As you know, we’re in the middle of delivering our two biggest programs—the Project Grant competition and the Foundation Grant competition—and we wanted to take the time to answer a few of the questions that we’ve been receiving recently.

Question 1: How will CIHR ensure that the peer review quality meets high standards for the Project Grant competition?

CIHR is going to ensure the quality of our peer review process using four key actions. The first is that we’ve made significant improvements to the matching process that allows us to select the most appropriate reviewers for the applications that we receive.

But beyond the matching, what we’re doing is we’re having each of our Competition Chairs go through every single match that we’ve produced to validate that these reviewers are the most appropriate to review those applications. We’re in the middle of this process right now and the Chairs are doing an amazing job of improving further the reviewer assignments that we’ve created.

The third thing that we’re doing is that the reviews will be assessed by the Competition Chairs and the Scientific Officers, so that each review will be read by a scientist and they will provide advice to CIHR on whether or not these reviews meet a sufficient quality standard.

Even if all of these things are done, it is possible that reviewers might drop out, they may not complete their reviews on time, or their review quality still might not be where one might hope.

So, we have a fourth strategy in place to mitigate against this possibility. The Competition Chairs and the Scientific Officers, in partnership with CIHR, have the ability to move any application forward from the original first phase to the final face-to-face assessment stage.

We believe that these four mitigation strategies will greatly increase the quality of peer review in the next Project Grant competition.

Question 2: What will CIHR do to make sure that early career investigators (ECIs) get the experience that they need to be eligible peer reviewers for future competitions?

To ensure that the early career investigators have the expertise and experience to become peer reviewers for CIHR, we’re going to introduce a new program with three key aspects.

First and foremost, if you want to be a reviewer for CIHR going forward in the future, you’ll have to meet strict criteria. Those criteria are that you have to have held a CIHR grant or an equivalent grant. And you’ve also had to have done reviews in the past--and so how does an early career investigator become eligible to do reviews in the future?

First, each Chair and Scientific Officer we’re hoping will partner with an early career investigator. So this early career investigator will help the Chair and Scientific Officers do their assessments of the review quality. That will be really important for each early career investigator to read each and every one of those reviews and to work with a Chair and a Scientific Officer in a mentorship role.

The second thing we’ll do is well invite those early career investigators to the final face-to-face meeting with their Chair and Scientific Officers. So they’ll be able to see the process from the first stage right through to the final face-to-face stage, and I think those two steps will be really important. Lastly, under our College of Reviewers, we’re developing a formal mentoring program that we think will really help early career investigators learn how to do high quality peer review.

Question 3: Will there be two Project Grant competitions in 2017?

There will be two Project Grant competitions each and every calendar year, including 2017, which does mean two application deadlines per year. At CIHR, we tend to talk about the funding start dates, and to be clear – there are two of those each calendar year, as well.

So right now, we’re in the midst of our Fall 2016 competition. The results will be released on May 15th, 2017. The funding start date is April 1st, so we have the ability to back date those grants a little bit.

We’ve recently released the dates for the next competition. As you’ll see, it’s taking place immediately after this current competition, with an application deadline of June 13th and a notice of decision in November, with another backdated funding start date of October 1st.

There will be a second application deadline in late 2017, and we will share those dates well in advance.

So, ideally we would like to see a spring competition and a fall competition. The issue that we’ve run into is that this summer when we were meeting with the research community, we had to make a number of changes (as you know) to our design, and so it took us a few extra months to launch the second Project Grant competition. That does throw our pattern off a little bit, so we’re not quite at the point where we have stable, predictable deadlines and dates. We are working toward that steady state – but we don’t want to risk the quality of the review process so we’re ensuring that we give ourselves enough time to conduct each competition properly.

We’re hoping that the 2018 competitions will be on track to have that spring and fall pattern again, but we’ll keep you posted about dates and timelines as we get a bit closer.

Question 4: How is CIHR addressing bias in the Foundation Grant competition?

In order to reduce bias in our Foundation Grant competitions, CIHR has introduced mandatory unconscious bias training for all reviewers, and we’ve developed a process to ensure the appropriate number of female applicants move forward in the next stage of the competition.

So, to remind you, in the Foundation competition, there are three stages. In Stage 1, we review the leadership and quality of the individual, and not necessarily the quality of their ideas. And in that stage we’ve noticed that female applicants have not necessarily done as well as we would have hoped.

When we look at the data in Stage 2, we see that female applicants do quite well and there’s no difference in their success rates. So, why would that be? Well, in Stage 2, we review the science – not the person or their leadership. This is important to underscore – women do fabulous research, and when evaluated on the merit of their research proposals, they do very well. It’s when the applicant and their leadership qualities get evaluated that bias may be present.

So, in Stage 1, we’re concerned that the number of female applicants is not moving forward at the proportion that we would expect. If the new mandatory unconscious bias training is not effective, then what we’ll do is we’ll ensure that the proportion of female applicants moving to Stage 2 is representative of the competition overall. And what that means basically is that if 30% of applications that come in are submitted by female applicants, we would expect that approximately 30% of those applications moving from Stage 1 to Stage 2 would be from female program leaders.

Review quality assurance

As part of CIHR’s efforts to ensure quality peer review for the Project Grant: Fall 2016 competition, the Chairs and Scientific Officers (SOs) will lead a review quality assurance exercise during Stage 1 of the peer review process. For this exercise, the Chairs and SOs will read each review submitted for each application in Stage 1 and will assess each review for appropriateness and robustness, which are two key indicators identified by the Chairs of the College of Reviewers. Additional information will be provided to reviewers, but it is important to note that reviews will not be assessed for the validity of the scientific opinion expressed therein. (Discussions about the scientific opinions will take place during the face-to-face meetings in Stage 2.)

It is also important to note that if a Competition Chair or a Scientific Officer concludes that a review is inappropriate, inadequate, and affects the ranking, then they can move the application forward to the face-to-face meeting (Stage 2).

Early career investigators (ECIs)

CIHR has developed an opportunity for ECIs to observe the peer review process for the Project Grant: Fall 2016 competition. In addition to observing the face-to-face meeting during Stage 2 of the review process, eligible ECIs will assist Competition Chairs with the review quality assurance exercise during Stage 1. More details about this opportunity and how to apply are now available.

Details about the full peer review process are available in the Project Grant Peer Review Manual.

Date modified: