Design Discussion Document - Town hall presentation

March 2012

Table of Contents


Setting the Context

  • As part of the commitments made in CIHR's second strategic plan, Health Research Roadmap: Creating innovative research for better health and health care, CIHR began designing a new Open Suite of Programs and peer review system.
  • In its report, the IRP made a series of observations and recommendations that relate to the Open Suite of programs.
  • We are engaging the research community throughout this process, and have used what we've heard from various consultations to inform the work on the new design.
  • The design is still evolving, and we need your help to advance the design and move forward with its implementation.

The objective is to continue the discussion on the proposed design for the new Open Suite of Programs and Peer Review Enhancements

The Rationale

As the major federal funder of health research in Canada, CIHR must:

  1. Meet the requirements of its broad mandate to support the creation and translation of health research across all domains;
  2. Ensure the long-term sustainability of its contributions to the health research enterprise; and
  3. Maintain Canada's competitiveness in today's knowledge-based economy.

To meet these requirements, CIHR must have programs and a peer review system that are capable of identifying and supporting research excellence across the entirety of its mandate.

The Canadian health research community has also told us that they are looking for change.

Common challenges in the Open Suite of Programs that have been experienced and/or identified by various CIHR stakeholders

Figure 1 long description

CIHR believes that re-designing the Open Suite of Programs and Peer Review System will help address these challenges

The Architecture

New Open Suite of Programs will be structured into two separate, complementary funding schemes:

  • Foundation/Programmatic Research Scheme
  • Project Scheme

CIHR's direct training programs will continue to be a part of its new investigator-initiated funding strategy.

CIHR believes these schemes will contribute to a sustainable foundation of excellence for the Canadian health research enterprise by supporting world-class researchers in the conduct of research and its translation across the spectrum of health.

The Foundation/Programmatic Research Scheme is about funding people, and will include a separate stream for new/early career investigators.

Today, there is a cadre of researchers who are consistently successful - we want them to spend less time writing grants and spend more time doing research and translating their results.

This Scheme will provide longer-term support to investigators with a demonstrated track record of success. We want provide freedom to create, change, and redirect research efforts as required.

The assessment criteria in this scheme would be based on the caliber of the applicant and their vision for their program of research.

The Project Scheme is about funding ideas, and will provide support for projects with a defined beginning, middle and end.

Today, there are both real and perceived barriers in CIHR's programming which limit certain types of ideas.

We want to reduce and remove these barriers and to ensure that there are opportunities for all types of researchers to bring forward proposals from all areas of health research.

The assessment criteria in this scheme would be based on the originality of the idea and quality of the project.

CIHR is in the process of modeling different scenarios.

Our principle is to maintain the number of unique NPIs funded by Open programs.

Amount of in-year Funding from the Open Suite of Programs held by Nominated Principal Investigators in 2010-2011

Figure 2 long description

This funding profile illustrates that the majority of grantees held less than $150K of in-year funding in 2010-11. The average value of an individual grant is approximately $123K per year.

The Mechanics

Competition processes

Proposed Multi-phased Competition Process: Foundation/Programmatic Research Scheme

Figure 3 long description

For example:

  • Approximately 750 applications are received.
  • They are reviewed by 250 – 400 reviewers.
  • Approximately 250 apps progress to stage 2.
  • Those are reviewed by 200 reviewers.
  • Approx. 114 program grants are awarded.

Proposed Multi-phased Competition Process: Project Scheme

Figure 4 long description

For example:

  • Approximately 5000 applications are received.
  • They are reviewed by 500-800 reviewers.
  • Approximately 2500 applications progress to stage 2.
  • Those are reviewed by approximately 500-800 reviewers
  • Approximately 955 project grants are awarded.

College of Reviewers

The College of Reviewers will support peer review activities for both the current system and the new Open Suite of Programs.

  • A more robust registry of potential reviewers:
    • grant holders
    • international reviewers
    • knowledge users
    • other disciplines
  • lay reviewers with the breath and depth of expertise required to review health-related applications
  • Modular/personalized training provided strategically
  • Mentoring program for new reviewers
  • Strengthened recognition of reviewer service to the research enterprise by CIHR and institutions

CIHR needs the support from the entire research community to develop a truly multi-disciplinary registry of potential reviewers to support the review of applications received by CIHR each year.

What We've Heard

What We've Heard

CIHR is creating hard funding caps of $125,000 per year per grant for the Project Scheme and $300,000 per year per grant for the Foundation/Programmatic Scheme.

Clarification

The grant values in the Design Discussion Document are averages based on the level of funding that researchers currently receive. The expectation is that grant values in both schemes will vary and be commensurate with the needs of each individual program or project. Modeling predicts a distribution of program and project grant values as shown in the figure below.

Projected Program and Project Counts (steady state) in new Open Schemes

Figure 5 long description

What We've Heard

Only people who hold more than one CIHR Operating Grant will be eligible for the Foundation/Programmatic Scheme.

Clarification

In our proposed design, applicants do not need to hold multiple grants to be eligible, or competitive, in the Foundation/Programmatic Scheme. There are currently many researchers who sustain successful programs of research supported by one CIHR grant. These individuals will be eligible and should continue to be successful under the new scheme.

What We've Heard

CIHR's Foundation/Programmatic Scheme is intended only to support senior investigators with long and established track records. Mid-career investigators will not be successful in this scheme.

Clarification

The Foundation/Programmatic Scheme is intended to support excellent researchers with excellent research programs at all stages of the career pipeline. The proposal is to have an individual's track record assessed relative to his/her career stage and in the context of his/her area of research.

The mid-career stage is clearly an area of concern for the research community and CIHR, and we are committed to monitoring their success and that of those at other career stages.

What We've Heard

Institutional support in the Foundation/Programmatic Scheme will result in a loss of academic and scientific freedom to pursue innovative and impactful research.

Institutional support in the Foundation/Programmatic Scheme will disadvantage researchers in smaller institutions.

Clarification

The requirement for institutional support in the Foundation/Programmatic Scheme is not intended to stifle innovation or scientific freedom. On the contrary, providing longer-term, flexible funding is expected to increase scientific freedom to explore innovative lines of research.

Institutional support is not about matching dollars, but is meant to ensure that the researchers funded through the Foundation/Programmatic Scheme have the necessary resources, tools, and time to be successful.

What We've Heard

CIHR's consultation is not real and everything has already been decided.

CIHR will be implementing all of the proposed changes effective March 31, 2012 and the research community will not have time to transition.

Clarification

CIHR is fully engaged in its consultation of the proposed changes to the Open Suite of Programs. This engagement began last fall with a targeted set of researchers and other stakeholders and has grown in both scope and reach to include the entire research community. We are listening, and using the input to inform program and peer review design details and how we transition and phase in changes. In June, we will provide the community with a summary of the feedback received through the engagement process, including how we are planning to use the feedback to inform the design. No firm design decisions will be announced until the fall of this year.

Transitioning to the New Schemes

CIHR's transition plan for phasing in the new Open Suite of Programs and Peer Review enhancements is currently in its early stages.

Current thinking suggests a gradual phase-in strategy will be used to implement the new design in small, progressive steps.

CIHR is considering piloting some elements of the new Open Suite of Programs design.

Current considerations for transition

  • Education, training and support for applicants and reviewers.
  • Developing a thorough understanding of system-wide impacts of changes to CIHR's programming.
  • Working with institutions and partners to ensure smooth transition.
  • Development of a monitoring and evaluation system to ensure continuous quality improvement of the new system.

The target is to announce some decisions about the design of the new Open Suite of Programs in the fall of 2012.

Applicants and reviewers would be provided with at least one year to allow time to prepare.

The first funded researchers under a new set of funding schemes announced in 2014-15.

What other transitional considerations should CIHR be aware of?

Feedback

The Design Discussion Document was posted to the CIHR website on February 9, 2012.

Feedback on CIHR's proposed changes can be submitted through:

  • On-line discussion forum - February to March 2012
  • Direct e-mail address - February to April 2012
  • Feedback form - February to April 2012
  • Town Halls/Institution Discussion Forums - February to April 2012

We invite you to join us in an on-going, active and production discussion about the proposed new Open Suite of Programs and peer review enhancements over the coming weeks.

Appendix

The Mechanics - These proposed design elements could simultaneously address multiple challenges.

Figure 6 long description

The Mechanics - Changes to the competition process and enhancements to peer review will address the current challenge
Change Result
Multi-phase competition process Reduces the number of applicants that complete full applications and the length of time required to review applications at each stage
Application-focused review Improves the way applications are matched to reviewers to ensure that the appropriate expertise is assigned to each application
Mainstream integrated knowledge translation
Structured review criteria
Recognizes the importance of knowledge users, and supports collaborative, application-based research
Structured review criteria Focuses reviewer attention on specific criteria
Emphasizes remote (virtual) screening process Supports matching for application-focused review and increases access to peer reviewers
College of Reviewers Facilitates access to appropriate expertise, and provides the framework for mechanisms to recruit, train and reward reviewers

Facilitates access to appropriate expertise, and provides the framework for mechanisms to recruit, train and reward reviewers.