Appendix A: The Business of Strategic Initiatives

[ Table of Contents ]

"CIHR doesn't take the second part of its mandate as seriously as the first. This is a fundamental issue." ICR Institute Advisory Board (IAB) member

The experience of the Palliative and End-of-life Care Initiative highlights some missing tools from CIHR's tool box of funding schemes, which are required to sustain new capacity and integrated KT. The PELC Initiative also helps identify some of the ways in which CIHR could better deliver on its new strategic plan (Chapter 4). While lessons learned from the PELC1 initiative are particularly important to applied health research, they can also enhance the outcomes of biomedical research initiatives.

Planning integrated initiatives

Partners and knowledge users – including other researchers – must be engaged at the earliest possible stage in developing a new initiative to harness the full range of available resources and expertise and produce a feasible agenda and useable outputs. Discussion about any proposed strategic initiative should start with definition of the problem: who needs what type of evidence to solve what problem?

If one of the goals of the initiative is to create increased research capacity - and most initiatives address perceived gaps - it's unrealistic to expect any field of research to move from small and fragmented to world-class in just five years. Nor is a single five-year infusion of funding to a single cadre of investigators likely to result in a sustainable, self-renewing community. Although CIHR has taken many creative approaches to addressing research challenges since its inception, there is still a strong tendency to equate "initiative" with "an RFA". As the first set of big, one-time five-year funding efforts come to a close, Institutes are increasingly realizing there is value yet to be obtained from their investment, and responsibilities not yet discharged. "Fund and forget" won't get the results they wanted.

Instead, Institutes are discovering that comparatively small investments in research-enabling activities add huge value. Some help new investigators to become more competitive. Others use multiple rounds of different kinds of RFAs, each feeding better-trained investigators up to the next level of excellence and productivity. For many, activities to enhance team effectiveness, KT and networking are becoming more common. Increasingly, initiatives are seen as packages of activities delivered over time, including but not limited to RFAs, and individually tailored to respond to the identified needs and the available expertise, capacity and resources to address them.

"If you don't sunset, you create a sense of entitlement. But you don't want to be premature, to turn the tap off and kill what you've created". Funding partner

Once key players have identified what needs to be done, they can see who can do what to help achieve common goals. What resources, expertise, open doors, and infrastructure can various stakeholders bring to the initiative? What are their needs, and their long term interests in this research community? CIHR and partners' roles should not be binary: funding Y/N? Instead, roles should evolve as the research community matures, from large initial funding to smaller strategic investments, to finally more of an influencing than a supporting role. CIHR should develop exit strategies for its funding initiatives in close coordination with partners: if CIHR is going to hand-off the responsibility for developing a particular field of research, it must ensure there is someone to hand-off to.

Many of these important in-program activities that help to extract full value from strategic initiatives have been formalized by the Heart and Stroke Foundation of Canada into a model it calls the "Managed Research Cycle"2. This facilitates the definition of the objectives, or deliverables, of an initiative at the planning stage, helps to keep the funded research teams focused on the objectives while the research is ongoing, and requires an end-of-initiative process to evaluate the degree of success of the overall research programs. This knowledge is then fed back into the planning process for the next strategic initiative. The PELC initiative followed most of the early steps of this cycle, up to stage 8, and has done some work in stages 9 and 10.

The Managed Research Cycle

Adapted from the Heart and Stroke Foundation of Canada

The Managed Research Cycle, Adapted from the Heart and Stroke Foundation of Canada

Defining outcome measures and an evaluation plan

"You can't develop a new area of science in a couple of years." Partner

Without clear success measures, it is impossible to define an exit strategy: "You have to figure out at the beginning what success is going to look like. When do you plant the flag, declare success and move on?" But while old definitions of success are increasingly coming under fire, new ones have not yet gained legitimacy.

Academic definitions of worthiness, excellence and success continue to dominate program design and peer review, and these are often incompatible with excellence in KT and knowledge implementation in the real world. For example, the most urgently-needed and implementable health research often lacks the conceptual novelty required to succeed in peer review. Research whose goal is to achieve health impacts must be judged by its ability to produce health impacts – not by its ability to produce academic outputs.

As the goals of strategic research increasingly diverge from those of the open grants competition, it becomes commensurately more important to ensure that follow-on funding mechanisms judge applicants based on the success measures of the initiative, not those of open grants. As noted in Chapter 4, applicants must be rewarded, not penalized, for integrating KT, investing time in forming partnerships and dissemination, and implementing research in practice.

If CIHR seeks to encourage interdisciplinary research with integrated KT, then participation in such activities must be rewarding to both researchers and knowledge users. CIHR has a great opportunity to exert its influence with academia and advise those universities/departments whose promotion policies are mis-aligned with the directions of CIHR's strategic plan that they risk reduced competitiveness for CIHR funds. CIHR could equally exert huge influence on health care organizations by providing release-time for research clinicians and for knowledge users (a.k.a. decision-makers) in teams, thereby proving its commitment to improving health care, and challenging the health system to respond in kind.

Making team funding work better

Training teams

Given the large investments teams entail, it's in CIHR's interest to help build better-functioning teams. Potential applicants need to know the implications of the role and the skills it requires to lead a large multidisciplinary team, as "a good PI is not necessarily a good NET leader." The NETs' lessons learned in creating and sustaining multidisciplinary, KT integrated teams, should be captured in a workshop, casebook and/or training module bringing together recent research findings with the practical experiences of team participants. Key elements might include:

  • Basic leadership requirements and skills, and where to learn more;
  • How to identify the strengths and weakness of (potential) team members; interpersonal issues, enhancing team dynamics and practical problem solving;
  • Best practices in making teams work – e.g. project charters, time charts, milestones, and self-appraisal;
  • Communications strategies and useful infrastructure;
  • Best practices in mentoring and training in a team environment;
  • The art and science of integrated KT.

Evaluating teams

A lot of a team's funding and time is used to build and sustain the team, especially over the first two years. A formative mid-term evaluation should look closely at how well the team is working, and provide constructive feedback to enhance the team's effectiveness. Evaluation at renewal (if available) should equally consider these issues. Key questions at mid-term and renewal ask about the extent to which the team:

  • Demonstrates effective leadership, with positive impact on the research accomplishments of team members.
  • Functions as a whole, interacting and collaborating-with all team members feeling that they have a voice in the scientific and practical direction of the team.
  • Attracts trainees, new investigators and collaborators.
  • Undertakes research which could not be performed by the team members as individuals.
  • Develops and implements an appropriate KT plan, including the building of effective relationships with knowledge users.

Team-based knowledge translation

Outside of the cancelled team grants program, CIHR has a number of other vehicles which support at least some aspects of collaborative research or integrated knowledge translation. For example, Partnerships in Health Improvement and Knowledge to Action Grants both incorporate knowledge users as meaningful partners. However, existing CIHR vehicles for research support tend to be project-oriented, and it is difficult to apply them to teams, which should undertake long-term, complex programs of research and relationship building.

For KT, what teams need most is the ability to bring together their wide-ranging work into a coherent package for dissemination, preferably set in the context of what was already known and is being done elsewhere. It is well recognized in KT circles that a single project is rarely an appropriate unit of knowledge transfer. But while this is a form of synthesis, it is not the kind of formal research activity supported by CIHR's synthesis programs. CIHR's end-of-grant KT funding is designed to fund a process to communicate a single research result: it isn't meant to support integrated KT in large teams. None of CIHR's current KT funding seems to support the extensive work required to roll-up the whole body of a team's results under common themes and corroborating messages to transform it into a more useable and meaningful package. This step is a critical pre-requisite to any further activities, including initiative-wide activities, discussed below.

Funding teams versus operating grants: a false dichotomy

We have been convinced by the evidence and the many teams we've spoken with that overall, CIHR is receiving good value from its investments in teams: they are an effective approach to addressing complex research challenges. Nonetheless, some remain skeptical of the added-value of expensive teams, a view presumably shared by CIHR's leadership, given the canceling of its open team grants competition in order to preserve its operating grants budget. Given that both teams and individual grants are the right way to support research in different circumstances, the question is one of balance of investment in the two types of research support, and choosing the right tool for the job. CIHR might consider asking those who have developed significant expertise in team research3 to help it make an evidence-informed decision on this point.

Initiative-wide knowledge translation

"Essentially, if CIHR is going to be the body we need… it will need to be doing quite different things from what it is doing now. The excellence of CIHR's work is compromised if it doesn't lead to change in the field." Partner

We believe large strategic initiatives need an explicit up-front KT strategy and dedicated KT resources. A knowledge broker assigned to a large initiative could expand the reach and enhance the outcomes of an initiative by working across individual teams, partners and KT staff to:

  • Lead and support communications among the funded researchers and/or teams, as well as with non-funded investigators in the area; help create/support an initiative-based network and on-going community dialogue, activities and interactions;
  • Exchange experiences, lessons learned and best practices among teams;
  • Help the researchers collectively seek out and interact with key partners;
  • Gather and synthesize research outcomes across and among teams;
  • Disseminate results and best practices from researchers and user partners to other teams and stakeholders;
  • Play a lead role in bringing all the results of an initiative together and planning appropriate mechanisms to showcase and get them implemented.

This KT resource should be funded through the initiative and strategically located where partners agree it can be most effective. For example, in the palliative care community the Canadian Hospice Palliative Care Association already acts as a knowledge broker, and might be ideally suited to housing and supporting someone in this role. A well-developed knowledge broker role should be able to attract investment from knowledge users as well as research funders. The value placed on this role by knowledge users initially and in the long-term will provide important measures of the successful design and then implementation of a knowledge broker function.

Capturing additional lessons learned

CIHR has been innovative in trying out new tools or models of funding, in new areas of research, with new kinds of investigators and partners, and it is recognized internationally for its pioneering approaches, a number of which have been emulated by other agencies. Institutes themselves are learning a great deal about what works best under what circumstances, and could benefit from a more systematic approach to collecting, sharing, and updating these experiences, particularly when combined with best practices from the research literature and other funders. CIHR's community (staff and researchers together) is developing a broad range of experience and expertise in doing research differently (and hopefully more effectively), and there is enormous scope to share these experiences and best practices in more systematic ways.

References

  1. While the conclusions drawn here arise from interviews and data analysis in the PELC community, they are also consistent with our previous assessment of strategic initiatives in the infection and immunity community.
  2. The Heart and Stroke Foundation Research Fund
  3. For example: Daniel Stokols, University of California Irvine; the Evaluation of Large Initiatives (ELI) team at the National Cancer Institute. For an overview, see “The Science of Team Science - Assessing the Value of Transdisciplinary Research”. The American Journal of Preventive Medicine (AJPM) Supplement, August 2008 (Volume 35, issue 2)