Year 5 - International Review Panel Report, 2000-2005

June 2006

Canadian Institutes of Health Research
160 Elgin Street, 9th Floor
Address Locator 4809A
Ottawa, Ontario K1A 0W9 Canada
www.cihr-irsc.gc.ca/

Other format

PDF version
(800 KB)

Acknowledgements
Executive Summary
1 Introduction
2 Accomplishments
3 Opportunities
4 Risks
5 The Institutes and Allied Research Themes
6 Governance and Management
7 Communications
8 Evaluation
9 Summary
Appendix 1 Reports on Individual Institutes and Their Related Open Competition Research
Appendix 2 International Review Panel Members

Acknowledgements

The peer review process remains an essential part of maintaining excellence in all fields of scientific endeavour. It is rarely applied to the funding structures that underpin health research but can play a pivotal role in ensuring that the structures and mechanisms used for allocating resources are both appropriate and effectively applied. The Canadian Institutes of Health Research (CIHR) deserves much credit for undertaking an international review and putting itself under external, independent scrutiny at this important stage in its evolution. It may provide a model for other funding agencies to evaluate their internal structures and performance.

The review has benefited greatly from the very substantial effort made by the members of the review panel who have contributed greatly to its success. Their contributions before, during and after the February 2006 site visit have brought to bear an enormous wealth of experience from the global health research community. I would very much like to thank them for their patience and good humour throughout this very laborious process. In addition, this review could not have taken place without the full commitment of CIHR, in particular the support of the Governing Council, the President and his senior management team, the Scientific Directors and the staff of CIHR. The International Review Panel (IRP) had available to it very significant amounts of supporting information for the review, and our deliberations in Ottawa were greatly facilitated by face-to-face contact with many of CIHR's key stakeholders. I hope that the efforts of all these people have helped to deliver a report which correctly identifies the very considerable strengths that have developed around the CIHR concept over the past five years and also appropriately identifies aspects of CIHR structure and function that now should change to ensure that it successfully completes the next phase of its evolution. There remain many opportunities for Canada to provide international leadership in the area of health research.

John I. Bell
Regius Professor of Medicine, Oxford University
Chair, International Review Panel

Executive Summary

Canada has embarked on a bold journey to change its approach to the funding of health research and ultimately to influence its breadth, its quality and its impact on the health of Canadians and the delivery of health care. The creation of the Canadian Institutes of Health Research (CIHR) six years ago was an experiment to establish whether a single agency for health research funding and leadership in Canada could sustain the excellence in Canadian basic and clinical biomedical research but also expand more effectively into areas such as population-based research, health policy research, health services research and research using social sciences and humanities to address important health issues. A successful outcome would be indicated by an expanded capacity and increased output in research reaching into previously undeveloped areas, improved interactions between disciplines undertaking health research, effective knowledge transfer and, most importantly, a retention of excellence in all areas of health research.

The International Review Panel (IRP) was impressed by the progress made in developing a more unified model of health research funding. The capacity to fund research across all health related disciplines has clearly been enhanced and new strategic initiatives have strengthened multidisciplinary research and training. Together, these changes have all occurred in a remarkably short timeframe, evidence of the commitment and success of the management team. The IRP feels now that CIHR should take stock and ensure that it has the necessary governance and management structures in place to further progress its mandate and vision.

A key component of the CIHR vision was a set of virtual Institutes, funded modestly but with responsibility for providing leadership and focus to a range of subjects at a national level. The selection of these Institutes and their leadership has been an important part of the first five years of CIHR development and was a major focus of our review. Also important, however, are the panels responsible for allocating much of the response mode research funding. We have therefore also attempted to consider the current state of the panel system in CIHR.

Our impressions of the CIHR at this moment of its evolution are intended to provide insights into its structure and function that will allow the successful development of this novel vision. The challenges in achieving objective evaluation of the outputs of the CIHR so early in its expansion and transition has led us to make comments on some of the key components of a successful organization, particularly sound governance and management. The success of the CIHR in rapidly implementing many aspects of the new mandate over the last five years has led to a much larger and more diverse organization. The speed at which this transformation has occurred is remarkable and clearly has required extraordinary effort and energy. The CIHR is in our view now at an important inflexion point in its development. New structures need now to be imbedded, transparency in decision making and process is crucial and sound governance becomes increasingly important. We believe that this represents a natural progression in the growth of this new entity but nevertheless a crucial one for the long-term viability of the organization.

We have intentionally avoided interventions around the detailed operational management of the CIHR. We do not believe it is our role to judge the balance of funding priorities, the distribution or the appropriate size of the CIHR budget. The comments we make provide some reassurance that the CIHR model is one well worth pursuing but that crucial aspects of its structure and governance need now to be reconsidered in light of its recent growth and expansion. If addressed, these suggestions will ensure that internal structures in CIHR are accountable and that CIHR remains responsive to the key stakeholders, including the policy-makers and scientific community.

Our review has resulted in the following observations:

The CIHR is clearly in a state of rapid evolution and has already transformed the face of health research in Canada. Evidence of the benefits of a more strategic approach to health research is apparent and multidisciplinary activity is widespread. Canada is beginning to enter into health research activities that have been previously inadequately resourced and underdeveloped.
The CIHR has clearly made progress in delivering leadership in the Canadian research community fostering collaborations, creating an environment for capacity building, forging an integrated health agenda, creating multidisciplinary Health Research Institutes, developing Knowledge Transfer and a sound ethical environment for research.
It is too early to make conclusive judgments as to the effectiveness of this model of health research funding based on currently available objective outputs.
All 13 Institutes appear to be functioning well, providing leadership in their fields and providing a focus for strategic activities. The Scientific Directors are all clearly contributing significantly to the success of the Institutes which are now fulfilling a crucial function in the health research community in Canada.
Rapid growth, particularly of new strategic initiatives and panels, has led to excessive complexity. This complexity needs now to be reduced to enable opportunities and activities to be both focused and manageable.
The panel system that is responsible for handling most of the research funding is currently under strain. It requires more academic leadership, and a review of process and structure for this system is necessary.
A major outstanding challenge for the CIHR and health research in Canada is the apparent lack of coordination at the federal and provincial levels of the many different types and sources for funding for different aspects of health research. Support for infrastructure and research posts are welcome but must be aligned with the operating grants that are necessary to keep the research enterprise running.
Governance will be crucial in the next phase of CIHR evolution. Accountability and transparency need to be reinforced at all levels of the organization. Governing Council should consider its position as a main board of the organization and the executive team needs expanding and strengthening. A single research committee should be established to account for all research expenditure in the CIHR, and to agree on the balance of strategic and response mode funding in each research area.
To formally provide scientific leadership across their disciplines, Scientific Directors should now be given oversight of their Institutes and their associated panels. Each should be responsible for ensuring the balance of strategic and response mode funding is appropriate and that the panel system functions more efficiently. It is likely that Scientific Directors (SDs) would participate in a central research committee.
Rapid growth and the challenges associated with matrix management across the Institutes and Ottawa have created management challenges within the CIHR. The most appropriate structure for handling these issues should be considered after an organizational review.
Knowledge Transfer (KT) has been accepted as an important part of the CIHR mandate. Progress has already been made in some areas of knowledge translation particularly in infectious disease, public health and some areas of health services research. There remains lack of clarity about the definition of KT across the organization. More attention should be directed at providing leadership in the area of technology commercialization.
CIHR should increase its emphasis on research in ethics as well as its governance responsibilities to ensure that the research it funds meets the highest ethical standards. Leadership in this area across Canada could come from the CIHR.
It is important to determine and to collect the objective data for each research activity that needs to be collected to allow an effective review of CIHR activities in the future. In particular, terminal reports from all grant holders would be an essential part of this process. This will be crucial if the next review committee is to provide a judgment on the success of the CIHR model.
Communication remains an important and challenging activity for the CIHR. The range of potential audiences, including funding partners, provincial and federal governments, universities, health researchers, international agencies and the citizens of Canada, make this particularly challenging. CIHR management needs to consider creative approaches to the utilization of a wide range of communication sources and resources including effective use of electronic and web-based dissemination, and should continue to improve its communication with key stakeholders.

1 Introduction

In 2000, the Canadian Parliament enacted legislation that created a new structure for the support of health research in Canada. The Canadian Institutes of Health Research (CIHR) was established with a mandate to support health research in its broadest definition, incorporating the strong biomedical and clinical research activities historically funded by Canada's Medical Research Council (MRC) and also supporting research relevant to population and public health, health services research, and social science or humanities research relevant to health. This new structure incorporated research areas previously funded by the MRC, the National Health Research and Development Program (NHRDP) and the Social Sciences and Humanities Research Council (SSHRC). A research funding agency with this broad inclusive vision of health research is, to our knowledge, unique worldwide and represents a novel experiment in research funding for health. The new CIHR was to emphasize the importance of scientific excellence and also provide support to facilitate and accelerate the translation of knowledge as it might apply to patients and health systems, and to extend the knowledge-based economy in Canada. Significantly, emphasis was placed on the importance of Ethics, both in the governance of the best research and also as a research activity in its own right.

With such a bold and broad vision, requiring the creation of a whole set of new structures and programs, the CIHR must still today be seen as an organization in evolution. The past five years have been spent establishing the Institutes, developing a range of initiatives directed at improving capacity in areas where Canada had little previous research record and conveying the vision behind this project to the research community in Canada and abroad. These achievements will provide the platform necessary for the future development of the organization.

As the CIHR begins the next phase of its development, it will likely need to consolidate some of its activities, embedding new ones in an organization that can ensure that the vision behind CIHR is sustained. Our comments must be seen in this light and may be both important and timely if the organization is to continue to mature successfully.

The International Review Panel (IRP) was made up of 27 scientists and health care professionals. All but one of these individuals was based outside Canada and their expertise covered the full range of activities encompassed by the CIHR mandate, including research activities in all four pillars, as well as expertise in knowledge transfer and ethics. Although the IRP spent a considerable time evaluating the 13 Institutes established within CIHR, it also took the opportunity to attempt to evaluate the current activity funded through the panels within CIHR and the organizational and management structures in place to ensure that its mandate was fulfilled. The IRP met for three days in February 2006 and was informed by significant documentation on the activity of the CIHR and its health research Institutes, feedback obtained from the health research community and direct contact with a range of stakeholders who contributed significant time and effort to come to Ottawa during the review to talk to the IRP. Full biographic details of IRP members are available in Appendix 2.

The IRP has had an opportunity to assess the progress made in establishing the CIHR over the past five years. This assessment cannot, however, accurately evaluate the scientific output of this new institution or effectively compare its success relative to its immediate predecessor, the MRC. Much of the last five years have been dedicated to establishing the structures and programs of the CIHR and the time taken between research funding, discovery or observation and the realization of those research studies in terms of publications, implementation in clinical practice, health service reform, policy recommendations or product development does not permit a realistic evaluation of this new funding model at this time.

The IRP has recognized, however, that while we are not yet able to thoroughly evaluate the success of this new model of research, as judged by the scientific output and impact of the Canadian health research community, we are nonetheless able to provide important insights into the effectiveness of the CIHR in managing its part of the health research agenda in Canada. We have achieved this by four different but complementary approaches through:

review of materials and progress reports provided to us by CIHR and their respective Institutes;
feedback from a diverse range of stakeholders involved in health research in Canada;
face-to-face meetings with Institute Directors and researchers;
assessment of the management structures and governance of the current CIHR to establish if these appear appropriate for creating and sustaining an environment likely to yield a productive health research base.

As we outline below, the product of our assessment will principally be our impression of the structure and function of the organization, followed by recommendations as to how this structure might be modified to cope with its rapid evolution and to ensure its future success.

The IRP felt that despite its short life, CIHR has achieved a great deal, particularly in refocusing and energizing the health research community in Canada, as well as facilitating and promoting interactions between groups of researchers who had not previously worked together. It has also developed and facilitated a range of programs in population health and health services research that had not been supported by previous activities. Our impression was that much of the success in establishing the CIHR as it currently exists can be attributed to its current President, Dr. Alan Bernstein. The view widely held in the community and endorsed by us is that his vision and energy have been instrumental in the creation of the CIHR. Importantly, however, it is now essential to ensure that an appropriate structure, necessary to sustain this vision, is firmly in place.

2 Accomplishments

The CIHR mandate is "to excel, according to internationally acceptable standards of scientific excellence in the creation of new knowledge and its translation into improved health for Canadians, more effective health services and products and a strengthened Canadian health care system". Although CIHR is still in a state of evolution and objective outputs are difficult to assess, at this point there is no doubt that the organization has made considerable progress in implementing its mandate. In particular, CIHR has provided leadership within the Canadian research community and has fostered collaboration with the provinces and with individuals and organizations in Canada and internationally that have an interest in health research. Collaborative programs abound within CIHR and there is clear evidence that the organization has been working effectively to bridge its activities to other health research funding organizations, with health care providers and provincial research agencies, with the Public Health Agency of Canada and with a wide range of international organizations. For example, the reach of its activities in areas such as Aboriginal health has extended to research councils in New Zealand and Australia. In total, partnerships have contributed substantially to the success of CIHR programs. These have grown steadily since 2000, almost doubling over the past five years.

Also in line with its mandate, CIHR has focused on creating a robust research environment, particularly in developing capacity in areas of health research in Canada and providing the necessary support to retain excellent research where it already existed. Programs such as the New Emerging Teams program provide a format through which research capacity can be developed in areas where it is required. A crucial accomplishment of the CIHR has been to support research across not just biomedical and clinical research, but also research relating to health systems, health services, health of populations, environmental influences on health, and social and cultural dimensions of health. Substantial activity in areas that have previously received little or no research support in Canada provides clear evidence that the CIHR has been delivering on its mandate across all four pillars of activity. A focus on developing multidisciplinary research was also a component of the CIHR mandate and this too has been effectively addressed by the CIHR. Examples of multidisciplinary programs incorporating multiple pillars and bridging Institutes within the CIHR include human embryonic stem cell research, consideration of issues relating to privacy and confidentiality, wait times research or the evaluation of challenges associated with translating genetics into health care. These have provided information to support health policy decision-making and illustrate the importance of multidisciplinary research in the health arena.

In the area of ethics and knowledge transfer, progress is clearly being made in advancing these agendas at a national level. Particularly in the area of infectious disease, knowledge translation has been delivered in a rapid and effective timeframe; similarly in key issues relating to health service functions such as waiting times. Progress has also been made in the area of commercialization through the development of the proof-of-principle program and a technology commercialization program, another important aspect of the CIHR mandate.

The creation of health research Institutes was a major component of the mandate of CIHR when it was established and the selection of topics to be covered by health research Institutes and the establishment of 13 such Institutes must be seen as a major accomplishment of the organization. These research Institutes together cover all aspects of health and incorporate activities relevant to all of the four pillars of research. They have emerged as powerful components of the new vision of health research in Canada and, despite the limited resource available to them, have catalyzed a range of new initiatives in the health research arena in the country. To have achieved the successful delivery of 13 such organizations must be seen as one of the most significant accomplishments of the CIHR.

The accomplishments of the CIHR to date are many and virtually every aspect of the mandate has been developed significantly over the past five years. This expansion of activity has been greatly facilitated by the more than doubling of the budget of the CIHR compared to its predecessor, the MRC, providing the necessary resource to build and develop in new areas. Although the IRP was not in a position to judge whether or not this new approach to health research funding can actually deliver more outputs, it is clear that the CIHR has been active in pursuing all aspects of the mandate set within the CIHR Act and that many of these activities provide novel and potentially very interesting and productive approaches towards making health research both more effective and more relevant.

3 Opportunities

a) Canada could achieve excellence across many areas of health research

Enormous opportunities exist for Canada to achieve international excellence in all fields of health research. The country has a long and distinguished tradition in the fields of basic biomedical and clinical research and more recently has spearheaded innovative collaborative approaches in developmental and population health research. These fields are poised internationally for significant advances over the next twenty years. Few would question the view that biomedical research is one of the most exciting and potentially transforming areas of scientific endeavour thanks to a range of new technological innovations that allow cell and molecular biology, structural biology and genetics to provide crucial information into basic biological process and into the fundamental causes of disease. In addition, there are tremendous opportunities in population-based health research that will have a critical impact on the health of population and, similarly, many of the new insights into disease at a cellular or systems level can now be moved effectively into the clinical research arena. It is also clear that effort is needed to understand better the social factors associated with disease and its consequences and to better characterize, refine and optimize the delivery of health care based on research evidence.

Together, these pillars of health research create an imperative to rigorously pursue a broad and robust research agenda in Canada, with excellence at its core. The ability to manage and support all of these diverse activities through one agency provides many novel opportunities. In particular, it allows research to be more multidisciplinary and the results of research to be translated more effectively into practice. Properly managed, these advantages create more productive research interfaces that may ultimately lead to more innovation.

b) Internationally, the CIHR is a new model for supporting health research

The novelty of the inclusive model of health research funding as prosecuted by the CIHR is clear. Through this process, Canada has become an international leader in bringing different components of health research together. Although significant cultural differences still remain between research areas, it is likely that this model will provide important new research outputs relevant to human health. Its success has clearly been noted in other countries. For example, a similar model has recently been proposed in the United Kingdom and other funding agencies are considering expanding their remit along similar lines. This inclusive model may prove to be one of Canada's most important gifts to the international health research community.

c) Canada could benefit from the many outputs of successful health research

Canada and Canadians will be the major beneficiaries of the outputs likely to emerge from a thriving health research community. A better understanding of the biological processes that underlie disease and the application of this knowledge in a clinical setting will ultimately lead to better medicines and diagnostics for patients. Canadian scientists have historically made major contributions to basic and clinical science that have had profound effects on patient well-being. From the pioneering work on insulin by Banting and Best through the fundamental insights into stem cells by Till and McCulloch, these important scientific insights have continued to provide benefits to millions of patients in Canada and around the world. Ongoing investment through the CIHR is likely to produce many more such examples in the future.

At a population health level, there are remarkable opportunities to be gained utilizing the comprehensive system of health care found in Canada and some of its unique record linkage systems. Large-scale population studies are likely to reveal much about the major environmental factors and genetic factors that contribute to disease pathogenesis and will provide significant opportunities to adapt and target public health policies to reduce ill health and premature death in the population at large. Effective population health strategies to avoid disease processes before they are irreversible and costly have to become the mainstay of containing health care costs. These are likely to be effective as they result from collaborations across both basic and social sciences. As we become increasingly aware of the considerable impact that social factors have in disease pathogenesis, input from the social sciences and humanities in the health research arena are crucial to unpacking why social inequalities are so pervasive and powerful, and how best to reduce their impacts. As with all Western nations, the Canadian health care system is under continual pressure to become more efficient at delivering health services within a structure that has limited resources. Understanding through health services research how best to apply health care in the community and in the hospital setting is likely to prove increasingly important. A strong evidence base from health services research will be a major asset to the Canadian health care system and will ensure that the limited resources are spent most effectively for Canadians.

The funding and support of outstanding health research in all of its settings can provide Canada with a well-trained, dynamic workforce that will ultimately have an important effect on other aspects of the economy including the biotechnology and pharmaceutical sectors. The creation of new knowledge and the exploitation of this knowledge to create better medicines and diagnostics potentially could also generate a thriving, knowledge-based economy founded on health research. Although this is not a simple relationship, experience elsewhere has clearly demonstrated that investment in health research by government can have an important effect in attracting and creating commercial activity that can significantly drive economic growth as part of a knowledge-based economy. The creation of intellectual property from the biomedical science base is a crucial driver of this process.

All these opportunities could be realized if the CIHR were to successfully deliver on its mandate and if a focus is retained on internationally recognized excellence in its research endeavour. The IRP believes that Canada is particularly well positioned to exploit a coalition of health research communities as directed by the CIHR mandate. Few places in the world have the quality of health researchers, the universal health care system, the university structure and the mandate to put together a program such as that being developed by the CIHR.

4 Risks

a) Rapid expansion produces management and governance challenges

The CIHR concept is a novel one and its creation, followed by rapid growth, generates significant management and governance issues. For example, the establishment of "virtual" Institutes —and of so many at one time— is a new (and bold) idea. To have established thirteen of these in parallel over eighteen months is a considerable achievement. Similarly, there has been a dramatic expansion in research review panels, funding instruments and strategic initiatives over a very short timeframe. The growth in resources available to the CIHR over the first five years of its existence has allowed the organization to meet some of the expectations of the different constituencies brought together under its funding umbrella. Important new initiatives have been launched, knowledge translation programs have begun to make a difference to both the health care and public health system, and it is clear that many research groups are now working across disciplines and generating very considerable benefits. All these very recent developments need careful management, particularly now as the organization stabilizes after its rapid expansion. It is important now to ensure that the CIHR has in place both governance and management structures appropriate for its next five years.

b) Getting the balance right between strategic and investigator-led research

One opportunity created by the CIHR is the possibility of more strategic research than was historically supported through previous mechanisms. This strategic activity takes a number of forms, both generated through the Institutes and much initiated elsewhere both within and without the organization. Some areas of research are highly dependent on strategic initiatives, in particular areas of research activity where human capital is inadequate and the history of research endeavour in these areas in Canada is limited. In both population health and health services research, for example, strategic initiatives are crucial for developing the necessary capacity to broaden and expand Canadian health research. In other areas of research, however, there is also a continuing need to provide operating support for investigator-led research. Basic biomedical, clinical, epidemiological, and social scientists that have existing strong, investigator-led programs simply require basic grant support to carry out their research programs. The tensions that currently exist between those requiring operating grant support and the need for strategic initiatives are likely to continue. The appropriate balance between these various forms of research funding, as with the balance between funding between disciplines, is a critical determinant of the future success of Canadian health research. This balance is likely to vary in different areas of health research so that no single formula can be applied across the organization. Both processes and structures must be established for the research community to productively and collaboratively participate in these crucial decisions. These deliberations must be transparent.

c) CIHR's broad mandate across many disciplines requires a diversity of approaches to achieving multidisciplinary research

One of the most important features of the CIHR has been its commitment to encourage research across different disciplines in the health research arena. This is likely to considerably increase opportunities for new and novel insights into disease and its major causes, particularly at subject interfaces where much innovation occurs. Multidisciplinarity, however, can be achieved in many different ways and, amongst the CIHR constituency, there should be different approaches to encourage this activity. The mandate of the Institutes was to promote multidisciplinary research, and many of the new strategic initiatives have encouraged applications that include investigators from different health research backgrounds. Initiatives such as the Large Team Grants clearly promote these interactions. This approach to encouraging multidisciplinarity is valuable in many settings but not all. Basic biomedical and clinical investigators often develop multidisciplinary programs, but they do so in a bottom-up fashion designed to solve particular problems that they encounter as they undertake their individual research programs. Collaborations and interactions are made to solve specific scientific problems that are encountered along the way and cannot be predicted in advance. This can be as valuable as predetermining multidisciplinary groupings in a strategic way. Both approaches to multidisciplinarity need to be valued and encouraged. In order to ensure this, the organization needs to be flexible, responsive and intelligent.

d) Has rapid growth led to too much complexity?

The rapid growth in the CIHR, including the establishment of 13 Institutes, eighteen new panels, a range of new strategic initiatives and the ongoing support of four pillars of research activity, has led to an organization that is vastly more complex than its predecessor. This is to be expected after a period of such intense development, but such rapid growth may lead to a lack of research focus and can distract from the primary objective of research excellence. An excess of review panels, different funding opportunities, strategic initiatives and programs can be as bewildering to the scientific community as it can to the organization itself. Complexity is an expected consequence of rapid expansion and growth but it needs to be continuously managed and limited. In research funding, "simple" can often be the best approach.

But simplicity is a value not just because it makes the system easier to manage, since even simple organizations can be poorly managed. The risks arising from the rapid growth of the organization include:

The growth in the number of panels may lead to inexperienced panel members, reviewer fatigue and possibly inferior reviews;
The growth in the number of initiatives (on its face, a welcome sign of healthy funding), when coupled with a relatively new area of study, may lead to insufficient applications of quality. Moreover, the sheer complexity may cause investigators (especially first time applicants) to become disillusioned while experienced researchers may become frustrated spending more time working through the system than conducting research;
Confusion can arise if the communication of the changes and their rationale have not been adequately communicated to the stakeholders.

The IRP appreciated that the growth in the number of initiatives was perhaps an inevitable consequence of the new broadened mandate of the CIHR and the need to build capacity in underdeveloped areas of research for Canada. Now, five years later, it is an appropriate time for CIHR to step back and both reduce and manage this complexity.

e) Sustainability and succession is likely to be challenging for the organization

A crucial component of the CIHR structure must be that its momentum and success can survive the change of leadership at all levels. The planned changes in Institute Directors will, in our view, be challenging if each Institute is to maintain the corporate memory that has made them successful to date. Transitioning the centers for these Institutes geographically as well as changing individual leaders may create significant difficulties. In addition, changes in the Executive Team now or in the future will require a management structure that is not individual-specific, a management structure that is stable with clear roles and responsibilities. These elements will be essential if the success achieved by the CIHR to date is to be sustained.

f) Financial risks are significant

Important financial risks are associated with the current funding structure for CIHR. As with many government funded agencies, there is little opportunity for CIHR to carry resources over from one funding year to the next. This provides very little headroom should the budget for the CIHR stop growing or even shrink, given the long-term commitments that are necessary to ensure successful health research activities. Managing a year-on-year funding model is extremely difficult for the Executive Team and is fundamentally at odds with the nature of research, little or none of which operates within a one-year timeframe. Such a funding structure is unlikely to produce the best resource allocation decisions.

g) Other national and provincial funding initiatives have significantly increased the demand for operating support from CIHR

The IRP was surprised by the extent, diversity and complexity of the research funding environment in Canada for health research. Canada appears to have more potential sources of such research funding than many nations. In addition to charities and foundations, Provincial governments appear to be playing an ever-increasing role in this funding landscape with significant funding initiatives in Alberta (Alberta Heritage Foundation for Medical Research), British Columbia (Michael Smith Foundation), and Quebec (Fonds de recherche en santé du Québec). In addition, the Federal Government in Canada has rightly recognized the importance of supporting several different components of health research. Although operating support is primarily provided through CIHR, significant amounts of Federal resource has allowed Canada to compete internationally in major projects in the areas of genomics (Genome Canada) and in research areas considered ripe for commercialization (Networks of Centers of Excellence). Importantly, very significant investments in infrastructure (Canadian Foundation for Innovation-CFI) and personnel support (Canada Research Chairs Program-CRC) have also been made by the Federal Government in recent years. Each of these activities represents an important investment in the field of health research. Crucially, however, each of these strands of research funding has important consequences for other funding streams. For example, significant support for new buildings and infrastructure through the Canadian Foundation for Innovation (CFI) would not be as productive were there also not essential investment in personnel through numerous Provincial programs and through the Canada Research Chairs Program. These investments show a strong and impressive commitment by the Canadian Government to strengthening all aspects of the health research portfolio.

The single health research funding agency most affected by investment made through many of these various funding streams is the CIHR. This is because the CIHR remains the single, major source of federal funds for the operating grants necessary to ensure that the investments made in personnel and infrastructure succeed. Each investment in personnel or building infrastructure inevitably puts further demands on the CIHR operating grant budget. If all such streams of funding were coordinated this would provide a powerful expansion in capacity across the research sector in Canada. When not well matched, however, significant new demands on grant support cannot be met. Failure to align these funding streams at a federal level creates a serious risk that supply and demand in health research becomes dangerously unbalanced.

Although complete alignment with various Provincial funding streams may prove to be difficult to achieve, rationalization of Federal sources of support for health research is essential and should be possible. The IRP believes that some mechanism needs to be identified to better orchestrate and coordinate the various sources of Federal research funding in the health arena to ensure that the most effective use is made of the significant investments currently being made in personnel and infrastructure. Without the necessary balance between operating grants and posts or infrastructure, much of the benefit associated with these other investments may be lost and some mechanism to ensure better coordination of these various funding streams is, in our view, crucially important. Various indicators suggest that there is not currently a balance between these investments. For example, the fall in success rates in CIHR's competitions, despite the increase in the number of successful applicants, suggests a mismatch between growth in capacity and operational support.

5 The Institutes and Allied Research Themes

The creation of a set of virtual Institutes was one of the major changes in health research funding introduced by the CIHR and hence a review of the Institutes was a major focus of the IRP. We also, however attempted to look at the state of science funded through the panels associated with the scientific themes of the Institutes as this is where the majority of research funding of the CIHR is allocated. Although considerable efforts had been made in the year preceding our review to accumulate feedback on the success and utility of the Institutes, we felt that the information available to us was insufficient to make clear scientific judgments about the achievements of the Institutes. Information about publications was not confined to work performed on CIHR grants and it was unclear which of the work that might have related to CIHR funding may have passed through the review panels with no input from the Institutes. Finally it is clear that most of the programs that could be ascribed to CIHR rather than its predecessor had not yet had sufficient time to deliver outputs that could be readily measured.

Despite these challenges, we felt comfortable that we could provide input on the direction that Institutes had set for themselves and could also comment on their apparent impact on the research community and their structure and leadership. We could also provide information on the general state of research activities funded by the panels in areas related to each Institute. In doing so we believe our assessment provides crucial interim information that could guide CIHR in its continued development.

Our review of the Institutes and their allied panel-based research activities was undertaken primarily through a set of interviews with leading scientists associated with the Institutes (including all Scientific Directors of Institutes) and with health researchers associated with panel activity. Extensive discussion at these interviews allowed us to explore the apparent success or failure of these activities. A summary of these deliberations for each of the Institutes and its related area of panel-based research activity is found in Appendix 1. These reviews indicate that the Institute structures within CIHR have brought an important new dimension to health research in Canada.

a) The Institutes have, without exception, made significant progress in fulfilling their mandates

The Institutes have all achieved considerable progress in developing and implementing an agenda to coordinate and develop their research area. The IRP was particularly impressed by the quality of the Scientific Directors of all the Institutes whose energy and enthusiasm have clearly been central to the success of these new organizations. Despite resource constraints, these Institutes have operated extremely effectively. Each has developed a specific set of strategic initiatives as well as provided leadership through training initiatives, mentoring and workshops.

The Institutes were particularly successful in expanding their research activities beyond the traditional research pillars and it was clear that the Scientific Directors operated extremely effectively as a group to encourage and develop multidisciplinary and cross-Institute initiatives. Because of the success of these virtual Institutes, what had emerged in the CIHR was a very substantial "strategy generator". We learned of one Institute that alone has developed sixteen new strategic initiatives. Although it is very encouraging to see the development of these strategic plans, it may prove difficult if not impossible for the CIHR to either manage or deliver on such a large number of initiatives and some approach to managing both the number and expectations that might arise around these initiatives should be considered. These and other strategic initiatives provide a challenge within the current funding envelope to ensure that the balance between strategic and response mode science funding remains appropriate for each subject area.

The IRP noted that initiatives unlikely to be funded in the previous funding system were now beginning to be discussed and realized. For example, prospective cohorts that provide a crucial dimension to population health research had been initiated or were being discussed by several Institutes and several public health research initiatives had been successfully delivered. The creation of Institutes had also led to a detailed evaluation of research strengths and weaknesses in each research area. Strategic initiatives often were targeted at building capacity in areas not previously well served with research scientists (e.g. the Centres for Aboriginal Health Research Capacity Building). There were many examples of novel and interesting new schemes for training and networking within scientific areas that are clearly having impact. For example, a new PI meeting within one Institute provides a model example of mentoring and networking within and between scientific areas.

The IRP identified strong and effective leadership in all the Institutes and attributed much of the success of the Institutes to these individuals. It is clear that the Advisory Board system is also operating well. The Institute Directors were clearly committed to the CIHR mandate and were particularly effective in developing interdisciplinary research both within and between Institutes. We were also told that many scientists have strongly identified themselves with particular Institutes. This is true in both the traditional disease-based Institutes with a historical biomedical focus (i.e., biomedical research focus, such as Genetics or Infection and Immunity), as well as Institutes where pillars three and four have emerged as dominant themes (Gender and Health or Health Services and Policy Research). Each Institute clearly had specific and unique approaches to developing their field and this diversity of approaches, associated with the flexibility of the Institutes, is clearly one of the very considerable strengths of the CIHR. In many cases, considerable leverage had been gained by collaborating with other funding agencies.

The IRP believes that all Institutes are now operating effectively and that it is too early to consider changes in the number or scale of the Institutes. Without robust and objective data about outputs that can clearly be ascribed to Institutes, it is inappropriate for the IRP to recommend such changes. We believe that future flexibility is important and that a change in Institute number and structure should evolve, but would be much better judged after the Institutes had a longer period to develop their programs and when objective data are available on which to base these decisions.

In summary, as vehicles to develop the mandate of the CIHR, the Institutes are undoubtedly having a very significant impact and the Scientific Directors deserve considerable credit for making the Institute concept viable and potentially extremely successful.

b) The affiliated research programs managed by the panels require leadership and more active management

The IRP also had an opportunity to consider the status of research funded entirely through the panels. Although the research funded through the panels was judged by those we met to be of a high standard, there was concern expressed that the number of panels and levels of funding available to the panels were creating significant problems for investigator-led research activities. Because of the number of mechanisms available for strategic initiatives, it was difficult to be clear about the relative distribution of strategic versus investigator-led research funding. As a result, the IRP was not positioned to make a judgment about what this ratio was or ought to be in each research area. It is clear that although Institutes and centrally driven strategic initiatives have the benefit of advocacy and leadership to ensure their agenda will happen, the panels do not benefit from such support. An explosion of new initiatives, new panels and new funding streams has also put a very considerable pressure on the peer review system and we were told that researchers are now suffering from significant review fatigue. Ensuring that panels are supplied with high quality and senior scientists is apparently proving difficult and the changing of panels due to potential conflicts of interest makes these problems even more difficult. The small size and short duration of some grants, the establishment of a large number of new grants committees and the presence of committees that see few proposals suggests that the peer review system is perhaps not being optimally managed. There appears to be no open and transparent process for the establishment of new panels, nor does there appear to be clear criteria or process for their evaluation and, in the event that a particular panel is no longer needed, how this decision is to be reached. There have been many new panels established and none eliminated in the past six years.

These challenges associated with the peer review panel system illustrate some of the current risks associated with rapid growth and increased complexity. It would be timely for this important activity within the CIHR to be reviewed and possibly restructured. Crucially, the governance of scientific decision-making and resource allocation across all subject areas needs to be reviewed alongside consideration of the optimal structure to support the process. Ideally, a single accountable committee should have responsibility for this activity and should properly represent the views of the scientific constituency and stakeholders. This committee could consider and agree on the balance in strategic versus response mode funding in each scientific area.

Simplification of the number and responsibilities of the panels could be orchestrated by this committee, leading to a considerable reduction in the complexity of the organization. Importantly, this committee should be responsible for the distribution of the entire research budget of the CIHR (including panels, strategic initiatives and Institutes) and the implementation at a research level of the agreed strategy approved by Governing Council.

c) Knowledge Transfer and Ethics provide important and novel components of the CIHR vision

The IRP noted the important attention paid to both Knowledge Transfer and Ethics in the mandate of the CIHR and hence considered the development of these aspects of the CIHR function in all our discussions with Institute leaders and the scientists associated with the panels. The inclusion of KT in the CIHR mandate is an important distinguishing feature of the organization as it recognizes the imperative to make the research undertaken by the CIHR relevant to the health or health care systems of Canadians. We were told of several impressive examples of KT applied to major public health issues. These were particularly evident, for example, in the Institute of Infection and Immunity where a rapid scientific response to the SARS outbreak was formulated and where work on prion mediated disease, avian influenza and water safety may have an impact on public health. The recent work on waiting list times across Canada, led by the Institute of Health Services and Policy Research, is another example of KT relevant to the health care system. Despite these notable examples, however, most Institutes are just beginning to contribute to the KT agenda, largely because there remains uncertainty as to what constitutes KT in different scientific areas. A clear and uniform understanding of the meaning of KT was evidently lacking in large parts of the CIHR. Most successful KT also requires partners wishing to utilize information obtained for this purpose and the fragmented nature of health care provision by the provinces could be a significant obstacle to this process. Nevertheless, we felt that some significant progress was being made in this area and that the CIHR was on track to make a significant difference to the public health and health care system if this commitment to KT was extended across all Institutes.

Interestingly, while much of the discussion on KT focused on knowledge translation, considerably less focused on commercialization as an important part of the CIHR mission. In other countries, commercialization is a major focus for those responsible for funding health research as it potentially underpins a knowledge base for the economy and creates a workforce capable of supporting such economic growth. Again partners are crucial for this activity to occur successfully. Universities are particularly important for this aspect of KT as this is where most research activity takes place. The IRP noted the very mixed success of Canadian Universities in technology commercialization with some Institutions having a world-class reputation in this area while other major Universities have a much less impressive track record. Given the importance of this aspect of KT to government in other countries the IRP felt that this activity should be embraced more openly and effectively even if the role played by CIHR is one of facilitation to ensure that the benefits of the research funded include commercialization where relevant.

Ethics is also a crucial component of the CIHR mandate, and it is evident that CIHR believes that ethics and science are jointly necessary for successful research. The CIHR commitment to Ethics is reflected in a number of ways including the existence of a central Ethics Office whose Director reports to the President, the presence of ethics delegates on Institute Advisory Boards, and the many relationships that CIHR has with other bodies that have ethics mandates/responsibilities. Despite difficulty in finding a permanent Ethics Director, CIHR's ethics portfolio has a number of accomplishments to its credit including: the facilitation of several public policy initiatives that had federal impact (e.g., stem cell research, privacy, Aboriginal health research); funding for 22 ethics research projects; an effective transition from MRC to CIHR Standing Committee on Ethics, and the development of standards for Research Integrity.

Opportunities exist not only to provide support for CIHR-funded research from an ethical perspective but also there exist remarkable opportunities to develop research programs exploring bioethics in a range of research settings. The IRP identified four key opportunities:

To position Canada as an international leader in all aspects of ethics in health research, including best practices, innovative mechanisms for oversight, governance, review and related policy topics.
To become an international laboratory for developing effective methods for integrating ethics, science and policy.
To support innovative ways of integrating ethics into all aspects of CIHR activities (i.e., within/across Institutes; from CIHR central to Institutes; and from CIHR to external partners and collaborators).
To develop ethics capacity in young Canadian researchers in a way that includes broadening the scope of field beyond ethics and law to include other areas/disciplines.

As with other aspects of CIHR, the Ethics portfolio also faces certain challenges. The IRP identified the following four: (1) Leadership. The IRP recognized that the lack of consistent ethics leadership over time had prevented the Ethics portfolio from blossoming, and further prevented it from using its considerable convening power; (2) Focus. The IRP noted that the Ethics portfolio is somewhat blurred between facilitation and leadership. The former is a service function and is necessary for good governance of research in CIHR; the latter is both a scholarly and management function and is vital if CIHR is to take advantage of the many opportunities described above; (3) Capacity. The IRP noted that until the recent hiring of a permanent Ethics Director, there was relatively little ethics/policy expertise within CIHR central offices. But this addresses only one aspect of the issue. It is equally important for CIHR to appreciate that there remains a small ethics research community in Canada. It is a well-respected and in many instances internationally recognized group of scholars and experts. But for CIHR to thrive, it will need to continue to build capacity to train the next generation of researchers and scholars; (4) The Research Ethics Board (REB) system and related governance issues. While Health Canada rather than CIHR is responsible for the governance system in Canada for the protection of human subjects, the IRP noted with interest that CIHR is in a unique position to promote its continued development and (if necessary) reform.

We appreciate that CIHR is but one of many parties engaged in both KT and Ethics. It was evident, however, that the potential leadership role of Canada's main funding agency in both these areas could be very powerful. In many issues in KT and Ethics, we repeatedly recognized that, if CIHR had not provided the lead in Canada, many of these activities might not have occurred at all. We would also add that if CIHR does not provide leadership in the future, who will?

d) Scientific Directors now play a crucial role in the success of CIHR

Our review of the Institutes and their associated panels indicated that much of the academic leadership for health research was now coming from the Scientific Directors of Institutes. Despite the limited amount of resource being spent by the Institutes, their leadership provides a focal point for research activity generally in the areas covered by the Institutes and these individuals have become important and vocal spokespersons within the CIHR structure. In this regard, their views of the future of the CIHR are important.

The IRP met the Scientific Directors who expressed the view that the CIHR and the Institutes needed to help reduce the complexity of the organization. In their view, large initiatives have not always proved better than fewer small, focused initiatives. We would support this sentiment. The Scientific Directors believed that they were optimally positioned to take responsibility and defend the panels, both for strategic initiatives and for the open competition. This view aligned strongly with the views of the IRP. The Institute Directors also felt they could be more involved in the evaluation and structuring of the peer review system. The recognition that reviewer fatigue is widespread and that insufficient senior investigators are involved in the panels was suggested by the Scientific Directors. They emphasized the complexity of different funding streams in the Canadian system and indicated that this was a particular challenge in making the CIHR resources operate effectively. Again, this resonated with the view of the IRP.

As one Scientific Director said to us, "The Institutes are now the CIHR". Although not initially empowered to dominate the CIHR agenda, we believe that through the considerable efforts of the Scientific Directors and their Advisory Boards, the Institute agenda has truly become a central feature of the CIHR. Our view is that the Governing Council should recognize this and use this structure as a mechanism of managing the entire CIHR portfolio at a scientific level, to some extent decentralizing control of the panels to Institutes and Institute Directors and using as a group the Institute Directors as scientific adjudicators of funding balance and decisions in the organization. This would ensure accountability and responsibility within the science community for crucial funding decisions and would also ensure that the Governing Council would be considering for approval decisions about funding taken by full-time, active scientists fully in touch with all the relevant communities (see Section 6 b)).

The crucial leadership role played by the Scientific Directors led the IRP to consider the succession challenges associated with moving Institutes every five to seven years. Although the Institute Directors were less concerned about the impact of these transitions, we believed that this is a significant issue for the CIHR. Institutional memory based around existing Institute Directors will be lost every five to seven years as new Directors establish themselves in other parts of the country. It is likely that administrative staff associated with the Institute Directors will not move and hence there is a significant risk that the continuity necessary for the sustained success of these Institutes will be thwarted. One solution to this problem will be to ensure that each Institute has some key activity based in Ottawa. This would be necessary if Institutes took on some responsibility for their associated panels (see below) and hence an administrative Associate Director for the Institute, based in Ottawa and reporting to each Scientific Director, could provide the essential administrative support to ensure that panels were properly managed and also contribute to the continuity as Scientific Directors changed. Another suggestion would be that a Deputy Director for the Institute be identified three years before the retirement of a Scientific Director and that this Deputy Director would succeed the existing Scientific Director on his or her retirement. Although longer term tenure for Scientific Directors has been suggested, we do not believe a ten-year tenure would be helpful for the organization. Our belief is that a five- to seven-year arrangement is appropriate. We believe that the Governing Council should consider options for ensuring the smooth transition of these Institutes at the time that Scientific Directors change.

6 Governance and Management

a) Executive and non-executive functions

The remarkable growth in programs supported by the CIHR has been greatly facilitated by a highly flexible approach to funding, a dynamic and energetic President and a decision-making structure that has allowed new strategic initiatives to emerge from the Institutes and the management team to be implemented relatively quickly and efficiently. As the organization evolves, however, we believe that its governance and management must also evolve to ensure clear lines of responsibility and accountability. The size and complexity of the organization demands that it now applies well-established rules of corporate governance and management. We believe that a re-evaluation of these issues needs to take place at all levels of the organization to accommodate the significant change in size and complexity seen over recent years.

It was unclear to the IRP whether the Governing Council (GC) was operating as an Advisory Committee, a Committee with executive functions or as a main Board of the CIHR. We believe its role should be the latter and clarity on this point is urgently needed. The Governing Council should undertake a review of its own position with regard to the sound corporate governance of the CIHR based on standards widely applied in the private sector. As the main Board of the CIHR, it may wish to have the ability to appoint the CEO and needs to consider the corporate governance issues associated with having a unified Chairman and Chief Executive role. As the Governing Board, the GC should not be involved in subcommittees with executive functions or in directly managing key parts of the CIHR activity such as decisions about funding approvals or funding distributions. It should agree on a strategy for the organization with the President and then allow executive functions and key committees to implement that strategy. This very important aspect of governance now needs attention.

The role of the executive team also now needs review. The President needs to take the responsibility of chairing the committee responsible for funding structures and decisions, and should provide the link between the GC and the management team. The executive team in Ottawa is now changing with the retirement of the VP, Research, and consideration should be given to how this crucial management structure should be shaped for the future, particularly in the light of the significant changes in breadth of activity and staffing seen in the organization in recent years.

b) Research funding structures and the role of Scientific Directors

We expect that an early priority as the organization evolves from this point will be to reconsider the structures in place for evaluating and reviewing scientific research proposals. There has been a dramatic growth in the complexity of these structures over the lifetime of CIHR. We understand that there are fifty-five standing panels and over fifty other additional panels used to evaluate research activities. Two thousand five hundred reviewers are now required to support this system, compared to eight hundred when CIHR was created. Although the majority of grant funding flows through these panels, they have not been seen as the responsibility of any senior scientific figure associated with the CIHR, but have been managed administratively by the Vice-President for Research. Twenty-six new committees have been created and none have been eliminated. The IRP was concerned about the lack of transparent process in determining which committees exist, what criteria were used to determine the need for new committees and which body was responsible for making these important decisions. This expansion of committees amounts to strategic expansion that brings additional resource to particular disciplines. As such, transparent processes are essential to ensure these strategic changes are equitably agreed and that new committees cannot be used to create strategy.

Our panel was also interested by the comparative success of the virtual Institute concept that underpins the CIHR. We felt that the Institute definitions were creative and unique, not being structured entirely in and around disease or therapeutic areas. The IRP was surprised to see how much of the activity in each Institute research area was funded through the open panels over which Institute Directors and Institutes had no direct or indirect responsibility. This, we believe, is now a fundamental issue that needs to be addressed. Indeed, when questioned, the Institute Directors unanimously and enthusiastically felt it was their responsibility to support the open competitions and the panels. It is our belief that this relationship should now be formalized. This would have many benefits. For example, those scientists applying entirely through the open competition with no involvement in strategic initiatives would nevertheless feel that the relevant Institute and Institute Director were responsible for ensuring an appropriate distribution of funds. This accountability is crucial in the process as at the present time it is not clear who is responsible for looking after the interests of those applying only to the open competition.

No-one is in a better position than the Scientific Directors (SDs) and their Advisory Boards for deciding the appropriate allocation of resources between operating and strategic initiatives. This ratio needs to vary significantly in different Institutes - old, more established disciplines may choose to resource their constituency through more operating grants while disciplines where capacity building and development is important may rely more heavily on strategic initiatives. Indeed, the IRP noted that this is de facto often the situation. For example, about 10% of CIHR's overall investments in neuroscience is strategic, reflecting that maturity and strength of that community while over 50% of the investment in Aboriginal health is strategic. Similarly, the Institutes should be in a strong position to oversee the committees that fall within their research domain. The difficulty in identifying panel members will be partly resolved by engagement of the Scientific Directors and their Advisory Boards in the selection and recruitment of panel members. Importantly, the Scientific Directors could, with the President, establish specific criteria for the introduction of new panels, make more efficient use of the existing panels and establish criteria and a process for eliminating panels that were no longer necessary. These lines of accountability and process are crucial and separating the Scientific Directors and their Advisory Boards from the major flow of resources from the CIHR is unhelpful and, in our view, unnecessary.

It seems inescapable that, given this important role played by SDs in creating a strategic vision for CIHR and their emerging role as advocates for both Institutes and related panels, SDs should now be given further responsibility to oversee the panel activity in their scientific area. It would also seem reasonable that a future role for SDs might be to form the core of the central committee replacing the Research Priorities and Planning Committee (RPPC) responsible for the allocation of the whole research budget. Such a committee would resolve the important outstanding issue of accountability and responsibility for funding decisions and its alignment with the strategy endorsed by the GC.

c) Organizational structure and management

As with all rapidly expanding organizations, the CIHR is facing challenges in managing its administrative workforce. The administrative staff in the CIHR has grown to three hundred and forty in a very short timeframe. Our Panel was unable to do a full review of this structure, but our discussions with employees, management and stakeholders suggested that there is an evolving need to ensure that with such a large administrative workforce, the roles and responsibilities for each individual in the organization are clearly defined and that accountability is clearly understood. Each individual should be aware of those responsible for key decisions and should be clear what their own responsibilities are in that process. This internal management is made particularly challenging by the matrix of relationships between the management team responsible for core functions in Ottawa and the Scientific Directors and their teams responsible for the Institutes.

We believe that achieving clarity of management structures should now be an important goal. This, we believe, is likely to be best resolved by initiating an organizational review. This should help establish an appropriate level of senior management support for the CIHR and should help clarify the roles and responsibilities of the administrative staff of the organization. This review should re-examine the role of additional Vice-Presidents and should consider a potential role for a chief operating officer. Attention should be paid to ensuring that lines of accountability are clear and, in particular, that the staff in the department responsible for managing the panels has clear leadership. Given the considerable expansion in activity and scope of the CIHR, it is not surprising that a review of this kind is necessary. The organization has to date relied very heavily on a small number of very capable senior managers who have carried the very significant burden of expansion. Going forward, its mission will demand more such senior support and clarity of responsibilities throughout the organization.

7 Communications

As the CIHR evolves, communication becomes an increasingly important function, both for achieving adequate communication within the organization and with its funded health researchers and also with the range of stakeholders associated with the CIHR including, importantly, policy officials and the public. This task has many dimensions and will continue to demand significant resources and energy. The IRP has not, as a committee, assessed in detail this function but our impression is that the CIHR communicates major health research findings to the public effectively through its central communications function in Ottawa. Communications with stakeholders, particularly Provincial and Federal Governments, universities and funding partners, is clearly a crucial function, again best served by the central communication function. This we were told has not received the attention it deserves and concern was expressed over the difficulties some important stakeholders have had in communicating effectively with CIHR. Communication with the public relating to public education and improving the public understanding of science, however, is a function best devolved to scientists, ideally through the Institutes. Similarly, the Institutes require significant capacity to communicate effectively between them and with the scientists who ally themselves with individual scientific areas. Therefore, we believe that, as the organization grows, communication is likely to become increasingly important. In this phase of consolidation, more effort needs to be made in communicating with stakeholders. As with other aspects of governance, some devolution to scientific Institutes would be merited.

8 Evaluation

The experiment that CIHR has undertaken to develop a new approach to health research is a crucially important one. It will be difficult, however, to evaluate whether or not it has truly been successful if processes are not in place to facilitate evaluation now and in the future. We are aware that a team exists with responsibilities for evaluation of various CIHR functions and, indeed, the IRP benefited from the work of this team during our visit to Ottawa. Nevertheless, it is clear that the necessary objective data to properly evaluate CIHR's impact and success is not being collected. The extensive review of the Institutes done last year was based largely on feedback and interviews and simple data such as publications emerging from CIHR-funded grants are not yet available. The IRP was surprised that end-of-grant reports are only a requirement for applicants seeking a subsequent grant. We believe that end-of-grant reports provide an important mechanism in accumulating data on achievements that could be used for future evaluations. Distinguishing between the contributions of the CIHR and those from other sources requires thorough documentation of grant outputs. We are conscious of the fact that the mandate of CIHR calls for activities that are particularly difficult to evaluate. Knowledge Transfer, Ethics and the activities particularly in pillars three and four do not lend themselves easily to conventional evaluative methodology used for pillars one and two. Nevertheless, there are standard metrics in all these settings (e.g. evidence of the research being used in policy development) and more effort needs to be invested in ensuring that these are collected and analyzed to plot the relative success of the organization. We believe that this process needs to be addressed immediately so that adequate information is available to assess CIHR objectively on its performance at the time of the next review and, importantly, to quantitatively measure some aspects of its activities against the MRC's productivity.

9 Summary

We hope that this review will provide important signposts to the CIHR as it continues to develop its program in health research in Canada. Establishing new and novel structures such as the CIHR is never easy and is a challenging task that inevitably will be met with resistance from within the research community. Thus, it is unlikely that it could be achieved without problems. Given that the new structure is only six years old, the accomplishments of the CIHR leadership and, indeed, the health research community in Canada generally, in delivering the mandate are clearly substantial. Like most new organizations in both the public and private sectors, creating new structures and new programs requires rapid and effective decision-making and the willingness to take risks and make mistakes. Without this approach, it is likely that this new structure for health research funding in Canada would never have emerged. There also comes a time, particularly after periods of rapid growth and development, to consolidate and bed down the activities which have clearly been successful, to reduce complexity and to ensure that the organization has the governance and management structures to move through its next phase of evolution. We believe that the CIHR has now reached that point where attention must be paid to issues of process and transparency that will allow CIHR to continue to thrive and be sustained into its next phase of development. As an experiment in new ways to approach health care research, there are many signs to suggest that the CIHR will ultimately be an important new model for health research funding, particularly if time is taken now to reconsider how many of the important innovations that have been introduced in the past six years can be established for the future. Simplification and reorganization now is likely to assure the success of this important experiment and guarantee the excellence and breadth of Canadian health research for the future.

Breadcrumb trail