Results from Phase I of CIHR’s Strategic Planning Engagement

Please note: This report reflects a summary of responses received through surveys and meetings. It is not a CIHR policy document and should not be interpreted to explicitly or implicitly represent CIHR’s opinions or intentions concerning any of the subject areas discussed. The purpose of this document is to present the responses received to date in order to support discussion at the September Workshop.

September 2019

Table of contents

Executive summary

In spring 2019, the Canadian Institutes of Health Research (CIHR) began a national dialogue on the future of health research in Canada, with the goal of developing a 30-year vision for the health of Canadians, as well as a five-year CIHR Strategic Plan (to be launched in 2020). CIHR initiated this process in order to better align its vision with other health research funders in Canada, as well as to seek comprehensive input from stakeholders.

The results of our engagements to date (including surveys and roundtable discussions) are summarized in this report. These findings will be provided to approximately 150 stakeholders who will participate in a Consensus Building Workshop on September 10 and 11, 2019. This group will be asked to further refine and expand on the ideas within this report and to seek common ground that CIHR will consider as one input, amongst many, in the development of its new Strategic Plan.

Key highlights of this report:

Respondents identified several high-level potential health research priorities:

As well, respondents identified several values that they believed should underlie CIHR’s operations and decision-making:

CIHR is committed to sharing this report with participants at the September Workshop, as part of a continued effort to seek clarity and develop consensus around a vision for the future of health research in Canada.

Purpose of this report

This report provides a summary of the input that CIHR received during the two-month engagement period (April-June 2019). The report (which is embargoed and confidential) will be provided to the individuals attending the September Workshop. This forum will provide participants with opportunities to discuss the report as they deliberate on the future of health research in Canada. These deliberations will contribute to our ongoing efforts to develop a new strategic plan for CIHR and will also help inform the vision of a health research ecosystem that will shape the health of Canadians.

The September workshop will provide further opportunities to deliberate on this report, to share additional ideas, and to refine and grapple with ideas that have emerged from the early stages of the consultation. As well, workshop participants will be invited to engage in discussions about the parameters that will guide the next steps in the development of a Strategic Plan for CIHR – a plan that will seek to accelerate the delivery of CIHR’s mandate and improve the health of Canadians.

In addition to the September Workshop deliberations, CIHR will review additional information, including international best practices in science funding policy, government priorities, foresight scans, Institute Strategic Plans, existing CIHR strategies and action plans, expert reports, implications for funding partners, an assessment of the operational feasibility of options, and budget implications. Together, these inputs will be reviewed through the lens of the CIHR Act and prioritized to ensure that CIHR delivers on its legislated mandate. These efforts will inform the creation of a short list of potential priorities for CIHR, which will subsequently inform the development of a preliminary version of the Strategic Plan (to be discussed at a Health Research Summit in December 2019). It is intended that following those deliberations, the Strategic Plan will be further refined and approved by CIHR Governing Council, before being finally launched in June 2020.

This report reflects a summary of “what we heard” during the initial consultation phase and does not reflect the final priorities nor the finalized Strategic Plan – there is much work yet to be done and many additional consultations to be conducted before the finalized plan is presented to Governing Council.

Figure 1: CIHR Strategic Planning Project

Figure 1 long description
Steakholder engagement May-June 2019
Consensus Building Workshop September 2019
Analysis of all inputs
  • September Workshop outcomes
  • CIHR Act
  • Current CIHR strategies/action plans
  • Government priorities
  • National survey of Canadians
  • International best practices
  • Expert reports
  • Foresight scans
  • Institute strategic plans
  • Feasibility assessments
  • Implications for funding partners
  • Budget implications
October-November 2019
Health Research Summit December 2019
Preparation of Strategic Plan January-May 2020
Release of Strategic Plan June 2020

How did CIHR seek input?

CIHR embarked on a national engagement project to inform the development of its 2020–2025 Strategic Plan. The purpose of the engagement was to seek input from stakeholder groups on the agency’s potential direction for the short-term (i.e., five years) and to develop a longer-term (i.e., 30 years) vision for health research in Canada. In order to gather input, CIHR developed a National Engagement Guide, which was launched on the Let’s Talk – CIHR website on May 6, 2019. Individuals and organizations were encouraged to complete the online guide and submit results by June 28, 2019. The National Engagement Guide was organized across six lines of inquiry developed internally at CIHR, in consultation with Governing Council and the Scientific Directors of the 13 Institutes, and based on the mandate of the agency under the CIHR Act.

Please note: These lines of inquiry were designed to elicit responses from stakeholders and do not signal the priorities of CIHR. As well, it should be noted that the responses were collected through online surveys and meetings; no probability sampling method was used (i.e., respondents volunteered to participate).

Invitations to participate in the online platform were distributed to more than 150 organizations in Canada. The link was also shared with the more than 30,000 recipients of CIHR’s ACCESS e-newsletter and promoted on CIHR’s website and social media platforms. Stakeholders were able to provide input through the website during the engagement period. The surveys were available in both official languages, as was all correspondence promoting the surveys.

Over 100 organizations were directly contacted by CIHR to encourage participation in the online survey or to submit feedback directly. In addition, roundtable meetings were held with the following national stakeholder groups, using the same six lines of inquiry to seek their input:

A series of engagement events with Indigenous communities were also organized by the CIHR Institute of Indigenous Peoples’ Health in Alberta, Quebec, and Nova Scotia during the summer of 2019. These events were designed to complement the recent engagement work conducted by the Institute in the development of its own Strategic Plan, as well as the work of the Canadian Research Coordinating Committee, which also recently embarked on a significant Indigenous community consultation. Given the importance of Indigenous Peoples’ health as a research priority for CIHR, as well as the existing CIHR Action Plan: Building a healthier future for First Nations, Inuit, and Métis peoples, this consultation offered an opportunity to supplement the existing work and fill in any potential gaps.

Additional meetings were held with CIHR Institute Advisory Boards (approximately 12 to 14 members on each board) during the engagement period.

CIHR also recently completed a comprehensive engagement of stakeholders in the area of global health. The results of this engagement will also be considered in the development of the Strategic Plan.

Please see Appendix A for details of the analysis method and Appendix B for further information regarding the online survey component of the engagement strategy.

What did respondents say?

The following section highlights the findings that emerged from the online survey, the stakeholder roundtable meetings, and the Institute Advisory Board meetings. The findings can be categorized across seven major areas:

What did respondents say about strategic priorities?

Strategic Priority Areas

There were a large number of suggestions put forward by respondents for both specific and more general strategic priority areas. Through the analysis, six key findings emerged.

Principles to Guide Prioritization

Consistently, respondents indicated that the views of patients, citizens, and governments are key to informing CIHR priorities. When asked to provide views on the principles that a health research funder should consider when identifying research priorities, respondents provided a range of responses and identified a number of potentially competing principles.

“More flexibility in funding high-risk research would be warranted. I have noted that in decision-making perhaps too much emphasis is placed on the experience of the research team and perhaps not enough on novel and innovative research. [T]he tendency is to become more conservative [in times of underfunding].The consequences are retrenchment, limited vision, and reluctance to fund high-risk or novel research approaches that fall ‘outside the box.’ CIHR needs to find a way out of this mindset.”

Researcher

Research Pillars and Approaches

A number of respondents emphasized the importance of the four pillars of research at CIHR. While responses varied in terms of prioritizing the needs of any one pillar, there were many respondents who emphasized the importance of supporting research that would address improvements in the delivery of health care across Canada. The responses dovetail with the responses summarized within the Knowledge Translation and Institute sections of this report.

A number of respondents also argued for more funding for interdisciplinary research, which they suggest is undervalued by CIHR. Collaborative science was also identified by some respondents as an area that CIHR should prioritize. Many respondents indicated that science is becoming more team-oriented and international in scope and that researchers must broaden their collaborations to be more successful.

CIHR’s Role in Funding National Platforms

Respondents were asked to consider CIHR’s role in funding national platforms, which are essentially non-research grants that fund organizations to provide direct services to the research community. A number of examples were included in the survey to provide context, such as the Canadian Council on Animal Care and the Canadian Light Source. There were significant tensions in the data with regard to this subject, ranging from those who asserted that platforms are an efficient/effective use of CIHR funds that should be further prioritized, to those who suggested that platforms should be deprioritized by CIHR. Some respondents also offered suggestions for new platforms and three emerged as common ideas:

Respondents also suggested that CIHR could play a more robust oversight role in ensuring that platforms are delivering as designed, particularly as these grants are often considered“directed grants” (i.e., there is only one application accepted and submitted for peer review). Finally, respondents suggested that CIHR should focus more on ensuring sustainability plans, so that CIHR is not responsible for funding such platforms in perpetuity.

Respondents also highlighted that CIHR could better align itself across the health research ecosystem when it comes to funding national platforms, such as by working more closely with the Canadian Foundation for Innovation (CFI).

Balancing Priority Driven and Investigator-Initiated Funding

Respondents’ responses varied widely on the question of how best to balance CIHR’s investments in its two major programs (i.e., investigator-initiated research or priority driven research). Many felt that CIHR needed to prioritize investigator-initiated research over priority driven research, in line with the recommendation of the Fundamental Science Review (that at least 70% of funds should be invested  in investigator-initiated research).

Many respondents also recommended that CIHR investments in priority driven research should be increased to ensure that research is linked to the priorities of patients, citizens, and governments, which are responsible for managing health care delivery. These respondents argued that more investment is needed to direct researchers towards such priorities.

What did respondents say about science policies and values?

Responses to the questions related to science policies and CIHR’s role in this area were focused mostly on three main findings: (1) equity, diversity, and inclusion (EDI); (2) open science; and (3) administrative burden.

Equity, Diversity, and Inclusion

Respondents put forward a number of suggestions for CIHR to consider within its strategic planning process with respect to equity, diversity, and inclusion:

Open Science

Open science also emerged as a key finding. Concepts such as open access, open data, and open research were highlighted by some respondents, including the need for CIHR to encourage the publishing of null results, develop a position statement on predatory journals, and mandate plain language summaries of all funded research for dissemination to the public.

Data management, alongside rigour and reproducibility, also emerged as important findings.

Within the data management policy area, a number of related ideas were raised, including creating more common data standards, improving data access and sharing, and developing better data storage principles and practices (e.g., data curation practices to preserve long-term storage and access).

With regard to rigour and reproducibility, the central tenet was the need to improve the efficiency of research across the many domains wherein research cannot be replicated. One of the issues flagged was the culture in science that does not reward or incentivize the sharing of null results. In terms of rigour, respondents suggested that CIHR could play a role by improving the peer review of experimental designs and by enhancing training standards so that researchers are well versed in rigorous methodological designs.

“CIHR has an opportunity to embrace the core principles of open science and data, and work with other Canadian funding agencies to move Canada to the lead in the international community when it comes to changing the culture of research. CIHR’s support for data management is getting stronger, and is much appreciated, but the agency can do more. Creating policies that mandate the use of best practices in open science can be a challenge in the Canadian context, but achieving a greater balance between traditional modes of knowledge translation (e.g., various forms of IP protection, scholarly publication via journals), and the approaches of open science (e.g., open lab books, immediate deposit of data), is critical to moving Canada’s health research and innovation to a new level.”

Not-for-profit organization

Administrative Burden

Many respondents expressed the perspective that there is a significant administrative burden in research and that CIHR could play a role to reduce this burden. The four predominant issues identified were:

Other Policy Issues

Some respondents also suggested that CIHR could work with institutions across Canada to better recognize the service element of a researcher’s role (e.g., their work on peer review, advisory committees, community engagement, and other related activities). This was also flagged as an equity issue, as some respondents suggested that female researchers are more likely to engage in these kinds of activities or that some areas of science that require much more engagement (e.g., community-based research, patient-oriented research) are not being recognized or valued for this additional engagement work.

Responses also pointed to the importance of supports to help researchers work effectively with knowledge users (e.g., those who participate in research as a research partner, advisor, or reviewer). This included the need to compensate and better recognize the important contributions of research participants (such as people with lived experience) for their time and expertise.

What Values Should Guide CIHR Decision-Making?

Several values were highlighted by respondents, particularly within the science policy area:

“Inclusion of patients, families, and the public that represent diversity and inclusiveness and the broad spectrum of Canadians on CIHR working groups that advise and create [priorities] will be of critical importance.”

Researcher

What did respondents say about partnerships and collaboration?

Principles and Approaches to Partnerships

Respondents were asked what approaches and principles they thought should guide CIHR’s research partnership activities. In their responses, they mainly focused on partnerships with organizations (convened by CIHR through the arrangement of joint funding opportunities), and partnerships developed by researchers in support of individual projects. In general, respondents were supportive  of such partnership activities, recognizing the benefits that they can bring in terms of experience, expertise, and funding.

“Given the need to seek partnered funding, scientists are often at a loss when it comes to initiating discussions with potential partners. CIHR could be play a more proactive role as a connector between scientists and potential partners (both for-profit and not-for-profit). CIHR already does this fairly well when funding networks via an iterative approach which involve ’strengthening exercises.’ Building on this by augmenting the support for scientists who are keen to seek help would yield a good return on investment in my view.”

Researcher

Themes that emerged from the respondents’ feedback include:

International Research Collaborations

When asked about the balance of investing in international collaborations versus research partnerships within Canada only, many respondents saw benefits to international collaboration.

Stated benefits included:

To ensure that these benefits are achieved, however, respondents noted the need to evaluate the impact and effectiveness of these collaborations to ensure that they represent value for money. Further, respondents suggested that CIHR should better communicate the benefits of international collaborations to Canadians to justify the investments.

Concerns were also flagged by some respondents who suggested that:

Suggestions for research areas that would specifically benefit from international collaborations were diverse, but comments about the international relevance of rural and northern health issues were especially common among respondents. Global health was also mentioned often,  and  many  of those respondents suggested that such research would particularly benefit from collaborations with partners from low- and middle-income countries (LMICs) who, in turn, could benefit from access to Canadian resources.

Health Research Funding Ecosystem

When discussing where to focus partnership activities, the concept of a coordinated national health research funding ecosystem was often raised. While responses varied in terms of the strategies to achieve funding coherence, the common sentiment was that CIHR should assume a leadership position to coordinate and align the various health research funding partners, given its national presence and convening power.  Respondents also identified various groups that must be engaged in order to realize the full potential of Canada’s health research funding ecosystem, including provincial health research funders, charities, philanthropic organizations, other federal government departments (e.g., Health Canada, Public Health Agency of Canada), provincial and territorial ministries of health, and the private sector.

Some respondents highlighted the benefits of a coordinated and aligned research funding ecosystem, suggesting that this would allow CIHR to focus its resources on areas of research that are not otherwise covered by partnering organizations, and vice-versa. The general thrust was that the current system could benefit from more cohesion and that CIHR could play an important role in leading a conversation with other funders to achieve this.

“CIHR can play a leadership role with other stakeholders (e.g., provincial funding agencies) and pan-Canadian organizations supporting data (e.g., Canadian Institute for Health Information)... and innovation in Canada. There are multiple players working without a common structure or goal, which results in redundancy and even working at odds [with] each other. A more coordinated approach will have a synergistic effect on meeting the overall goals of CIHR.”

Researcher

Respondents suggested several ways in which the Canadian health research funding ecosystem could work more collaboratively, and how CIHR could provide leadership in that regard:

What did respondents say about knowledge translation?

Importance of Knowledge Translation Mandate

There was a common belief expressed by respondents that the knowledge translation (KT) aspect of CIHR’s mandate (as delineated within the CIHR Act) was currently under-emphasized. Some respondents felt that there was an abundance of knowledge that has not yet been successfully integrated into practice and that much more of CIHR’s budget should be dedicated to translation  (as opposed to creation) in order to address this issue. Some suggested that this lack of translation places an enormous burden on the health care system and places patients at an unnecessary risk.

“The Act actually specifies BOTH discovery and translation of research to start with. Unless we focus aggressively on translation as well, we will not be much further ahead in 30 years.”

Researcher

Often, respondents argued that presenting knowledge creation and translation as separate (and possibly competing) concepts creates a false dichotomy. Some respondents suggested that CIHR could do more to promote an integrated approach, which includes both knowledge creation and translation.

The importance of integrated knowledge translation (wherein patients, clinicians, and policy-makers are an integral part of a research team from the very beginning) was a prevailing theme across many responses, and was flagged as the key to effective, scalable translation of knowledge.

Knowledge Translation Grant Programs

Many respondents also suggested that new knowledge translation grant programs should be launched by CIHR. Currently, CIHR does not have a dedicated program designed specifically to fund KT science or KT activities specifically (although it was acknowledged that these types of grants are eligible to be funded through CIHR’s Project Grant competition). Many respondents argued that these types of programs should be made available (and some suggested that CIHR consider examples of KT funding programs that had previously been managed by CIHR). Amongst those who recommended additional KT-specific grant programs, four main types were identified:

Balancing Knowledge Translation and Knowledge Creation

While some respondents argued for an enhanced focus on knowledge translation within CIHR, others maintained that CIHR should not focus efforts on this aspect of its mandate. The overall issue expressed by those respondents who cautioned CIHR in this regard was the belief that a heavy focus on knowledge translation would signal an undervaluing of curiosity-driven, fundamental research  that does not necessarily have a specific translational aspect at this time.

“Care needs to be taken not to let the pendulum shift too far away from fundamental research. The fact that CIHR has dedicated an entire survey to [knowledge translation] reinforces the feeling that CIHR does not value fundamental research across the pillars.”

Researcher

Another common view was that CIHR should focus its efforts on improving success rates in the Project Grant competition before any specialized KT programs are considered. These respondents argued that not enough creation is occurring to justify a stronger focus on translation. The number of high quality (yet unfunded) research ideas was felt to be too high to warrant such a shift for CIHR right now.

Another belief was that CIHR should focus on knowledge creation, and that any emerging ideas that were highly valuable and relevant would intrinsically translate into improved health care delivery and health outcomes without CIHR. There was also a sense that excellent ideas (i.e., discoveries) would be picked up by knowledge users (such as clinicians and policy-makers) or be commercialized without significant intervention by CIHR.

Internal Knowledge Translation Capacity at CIHR

Beyond grant funding, a substantial number of respondents recommended that CIHR should enhance its own internal knowledge translation capacity. There were comments related to the need elevate the importance of KT through organizational changes within CIHR, such as the addition of a senior executive responsible for KT and/or more dedicated professional staff with expertise in KT. Some respondents also suggested a new CIHR Institute of Knowledge Translation.

There were four suggestions put forth by respondents pertaining to how CIHR could use this increased KT capacity:

Peer Review and Knowledge Translation

There was a collection of comments surrounding the relationship between knowledge translation and peer review. Some respondents suggested that it should not be an expectation among reviewers that all research projects have a mature and organized KT plan built into the proposal. The belief was that CIHR should allow more flexibility for researchers (and thus reviewers) to use discretion and judgement with regard to whether a KT plan is required for a given project proposal. Some respondents suggested that discovery-based research should not be held to the same standard as more applied research in terms of KT. Some respondents provided concrete examples wherein knowledge translation plans were not appropriate and where the lack of such a focus should not be considered a flaw of the project design.

There was substantial support for incentivizing knowledge translation (where appropriate), primarily by developing explicit peer-review criteria that would substantially weigh KT in the overall scoring. Training was frequently mentioned as a way to ensure that peer reviewers better understand the science of KT, appreciate the review criteria, and can properly assess the KT elements of an application. Some respondents felt that review panels should always include knowledge users as reviewers, with a particular emphasis on patients. Finally, some respondents encouraged CIHR to mandate researchers to report back (either to the peer review committee adjudicating their next grant or to CIHR itself ) on their knowledge translation activities within the funded research.

What did respondents say about capacity development?

There were a number of questions within the surveys that focused on capacity development and respondents focused their answers in three main areas: indirect training support, direct training support, and gaps in the research enterprise.

Indirect Training Support

One of the prevailing points that was raised during the engagement process was the manner in which CIHR represents investments in capacity development and, consequently, structures its interventions. Many respondents believed that CIHR focuses only on direct training (i.e., specific grant funding directly provided to graduate students and post-doctoral fellows) without considering the substantial amount of indirect training that occurs across most CIHR funded research projects (i.e., graduate students and post-doctoral fellows paid through research grants). There was a belief that CIHR should consider both and that there has been an undervaluing (or a lack of recognition) of indirect training. Many responses also highlighted the perceived need to prepare research trainees for career success (recognizing that many trainees will pursue careers outside of academic institutions).

The main suggestion put forward was to develop a minimum training standard from which all trainees (both direct and indirect) could benefit. Respondents highlighted the following for such a standard:

Direct Training Support

Responses in this area can be captured under four key findings:

Capacity Development Gaps

Respondents identified a number of critical gaps within the research enterprise that they felt CIHR should consider addressing through policies, programs, or other activities:

What did respondents say about the CIHR institute model?

Respondents expressed a diverse set of views related to the current slate of CIHR Institutes. Many suggested that CIHR should maintain the current slate of Institutes, while some recommended abandoning the Institute model altogether. Most respondents, however, offered suggestions and recommendations for changes to the existing CIHR Institute model.

Perspectives on Changing the Slate of Institutes

The main issue flagged with the current slate was a perceived lack of a coherent framework or conceptual model for the Institutes. Some Institutes were described as being based on pillars (i.e., health services or population health), while others were described as being based on diseases (e.g., cancer, diabetes, arthritis), anatomical systems (e.g., circulatory, respiratory, musculoskeletal), populations (e.g., Indigenous, children), and some are based on cross-cutting issues (e.g., gender, genetics, aging). In addition, there were a number of comments about a perceived silo effect, which some respondents argued encourages Institutes to focus too narrowly on a prescribed mandate.

Respondents also flagged concerns that the process to make changes to the slate of Institutes would be too political and/or too difficult to justify the effort. They also indicated that the grant budgets managed by the Institutes are too small to warrant the time and energy required to make such changes. A relatively common suggestion was to simplify the slate by structuring the Institutes along the four pillars of health research.

Perspectives on Creating Additional Institutes

Many respondents recommended the creation of additional Institutes in areas such as: primary  care (with an emphasis on patient engagement); environment and health; and, technology and health (with an emphasis on artificial intelligence). Some respondents suggested adding Institutes that would focus on: vulnerable populations; rural and remote health; knowledge transfer (with an emphasis on commercialization); oral health; and health and wellness (with a focus on prevention and social determinants of health).

Institute Activities and Responsibilities

Many respondents indicated that the activities of current Institutes were both appropriate and valuable. Some respondents cited the positive focus on capacity development and training, particularly in gap areas in Canada, while others lauded the work of Institutes in building partnerships and networks.

“There are many positive aspects of the Institute model – the scientific independence and proximity to the research community are assets, as are Institutes’ capacity to do innovative, responsive work in their fields.”

Not-for-profit organization

There were four key findings that emerged from respondents’ input on suggestions for improving Institute activities:

What did respondents say about the project grant competition?

Overall, there was a degree of satisfaction expressed with the processes and peer review structure of the Project Grant competition across stakeholder groups. Often, positive comments were related to the CIHR decision to return to face-to-face standing panels as opposed to the virtual model that was developed under the CIHR Reforms. There also were a number of suggestions for change that were commonly expressed.

Balancing Priority Driven Research and Investigator-Initiated Research

When asked about CIHR’s need to balance funding for both investigator-initiated research and priority driven research, two divergent views emerged, although one was more dominant.

Some respondents felt that CIHR should increase the proportion of its budget allocated to priority areas that have been identified by policy-makers, patients, and other groups. The more prevalent view, however, was that CIHR should increase its investments in investigator-initiated research. The central rationale put forward by proponents of the latter was that the success rates within the existing Project Grant competition are far too low and that a substantial amount of high quality research remains unfunded. Some respondents also expressed the view that the quality of research funded through open competitions (e.g., Project Grants) exceeds the quality of research funded via priority driven competitions. An additional argument that was often put forward was that researchers are best positioned to identify emerging areas that need more research (as opposed to priorities that are identified through other means).

Project Grant Competition Success Rates

As indicated several times within this report, there was a clear and prevailing position amongst many respondents that success rates within the Project Grant competition must increase. Many argued that CIHR should primarily focus its efforts and resources on attaining a success rate within each competition of approximately 20-25%.

Respondents put forward a number of suggestions to attain a higher success rate:

Project Grant Budgets

Another major issue highlighted by respondents was the “across-the-board” cut that CIHR imposes  on all successful grants within the Project Grant competition. This cut was introduced by CIHR in an effort to ensure that a baseline number of researchers are funded. There was substantial agreement amongst respondents, however, that the ensuing consequences were both negative and significant. There was a sense that this cut is a blunt instrument – that it is not a strategic way to fund more grants, given that it applies to all grants.

There was also a belief that the cut either has created a culture of “padding” budgets to accommodate the anticipated cut, or unfairly reduces budgets for those who do not “pad” their budgets. Another perceived consequence expressed by some respondents was a shift in the focus of peer review panels away from carefully reviewing budgets, given that there would automatically be a large cut imposed on every successful grant. The main recommendation that emerged was to eliminate the “across-the-board” cut and introduce changes to the way in which budgets are submitted and reviewed.

The introduction of modular budgets was the most common suggestion to address this issue. Modular budgets (which are used by the National Institutes of Health in the United States) allow researchers to plan their expenses in modules of $25,000, with the number of modules selected by the applicants. Many respondents argued that this approach would allow CIHR to better manage budget reviews and reduce administrative burden for applicants and reviewers as a detailed budget is only required upon a successful review and funding decision. In addition, it was recommended that peer reviewers should be better trained and provided with more autonomy to make recommendations related to an application’s budget.

Project Grant Peer Review

There were a number of responses related to the current slate of peer review panels based on the 47 areas of science. Respondents pointed out that CIHR should have some type of formal process in place to review the slate of panels regularly. However, there was no consensus on how to do this or what a reconfigured slate of review panels would look like.

The need to ensure that multi-disciplinary (or cross-disciplinary or trans-disciplinary) research applications are appropriately reviewed was frequently cited by respondents. There were, however, very few concrete suggestions put forth on how to achieve this objective.

Respondents also described the perceived difficulty in obtaining funding for more risky research (e.g., exploratory, high/risk, new methods). Some respondents felt that CIHR should ensure that panels are open to accepting more risky research proposals, regardless of the area of science.

There were some clear ideas suggested related to the peer review process in general:

Appendix A: How did cihr synthesize the input?

Analysis Approach

For the analysis, CIHR used an approach called “framework analysis” to analyze the inputs received through the engagement activities. This is a qualitative analysis method related to content analysis, but adapted for the purposes of applied policy research. The analysis approach was discovery-focused as opposed to hypothesis-based. This means that there were no predetermined themes – the key findings emerge out of the data analysis.

For the analysis, CIHR used NVivo 12, a qualitative analysis software package. Given the highly collaborative nature of qualitative analysis, it was coupled with NVivo Server, an application that allows people to work on NVivo analysis projects concurrently. Further, a project room was made available where analysts could meet to code concurrently and discuss issues and observations. In addition, weekly group meetings were held to discuss each coder’s approach to coding and general codebook updates, to ensure general conformity to the analysis approach. Inter-rater reliability was assessed by asking analysts to double-code a sample of the data. The agreement levels were consistently higher than 80% and in many cases close to 100%.

External Expert Validation Panel

An External Expert Validation Panel was created to provide advice to CIHR. It consisted of four researchers from outside CIHR who have specialized expertise in qualitative analysis and who bring various perspectives (e.g., research pillar, career stage, gender, region, and institution). The Panel’s purpose was to advise on the methodology, analysis techniques, and data interpretation. During the period of data analysis and report writing, the Panel held weekly teleconference meetings with CIHR. Panel members have also been invited to participate in the future stakeholder meetings related to Strategic Planning (i.e., the Consensus Workshop and the Health Research Summit). Its members were:

The Panel received various materials, such as documentation on CIHR’s strategic planning process, updated analysis codebooks, visualization concepts, a full codebook excerpt (including coded responses), data audit trails, and draft versions of this report. Throughout the process, the Panel provided comments on the analysis approach and the rigour of the work.

Appendix B: Online survey demographics

The Let’s Talk – CIHR platform was visited more than 7,500 times during the two month engagement period and 1,551 individuals or organizations registered on the site (registration was required to participate in the engagement to avoid spamming and trolling). Of those who registered, 785 responded to at least one of the six surveys. In total, CIHR received 2,168 completed submissions across the six lines of inquiry. The tables below contain information on those who completed the online surveys and do not reflect all of the other individuals and organizations consulted through stakeholder meetings and IAB meetings.

Table 1: Total number of submissions by survey

Strategic Research Priorities 621 (28.6%)
Capacity Development 392 (18.1%)
Investment Strategy 345 (15.9%)
Project Grant Competition 276 (12.7%)
Institute Model 269 (12.4%)
Knowledge Translation 265 (12.2%)
Total 2,168 (100%)

Table 2: Respondents by province/territory

British Columbia 112 (14.3%)
Alberta 82 (10.4%)
Saskatchewan 20 (2.5%)
Manitoba 31 (3.9%)
Ontario 338 (43.1%)
Quebec 131 (16.7%)
New Brunswick 8 (1.0%)
Prince Edward Island 1 (0.1%)
Nova Scotia 40 (5.1%)
Newfoundland and Labrador 10 (1.3%)
Yukon 1 (0.1%)
Northwest Territories 1 (0.1%)
Nunavut 1 (0.1%)
Outside Canada 9 (1.1%)
Total 785 (100%)

Table 3: Respondent demographics (self-identified)

Gender
Woman 401 (58.0%)
Man 259 (37.5%)
Gender-fluid, non-binary, and/or Two-Spirit 8 (1.2%)
I prefer not to answer 23 (3.3%)
Total 691 (100%)
Indigenous
Yes 21 (3.0%)
No 642 (92.9%)
I prefer not to answer 28 (4.1%)
Total 691 (100%)
Visible minority
Yes 106 (15.3%)
No 547 (79.2%)
I prefer not to answer 38 (5.5%)
Total 691 (100%)
Disability
Yes 50 (7.2%)
No 602 (87.1%)
I prefer not to answer 39 (5.6%)
Total 691 (100%)

* The total is 691 because CIHR did not collect demographic information from the 94 organizations who completed a survey.

Table 4: Respondents by sector

Researcher 399 (57.7%)
Health care professional 97 (14.0%)
Student or trainee 60 (8.7%)
Member of the general public 36 (5.2%)
Academic employee (e.g., research administrator) 31 (4.5%)
Government employee 21 (3.0%)
Not-for-profit sector employee 21 (3.0%)
Private sector employee 5 (0.7%)
Other 21 (3.0%)
Total 691 (100%)

* The total is 691 because CIHR did not collect demographic information from the 94 organizations who completed a survey.

Table 5: Researcher respondents by pillar/career stage

Pillar
Pillar 1: Biomedical research 178 (44.6%)
Pillar 2: Clinical research 49 (12.3%)
Pillar 3: Health services research 95 (23.8%)
Pillar 4: Population health research 77 (19.3%)
Total 399 (100%)
Career stage
Early career researcher (less than 5 years) 99 (24.8%)
Mid career researcher (5 to 15 years) 132 (33.1%)
Senior career researcher (more than 15 years) 168 (42.1%)
Total 399 (100%)

* The total is 399 because CIHR only collected this information from researchers.

Date modified: