Knowledge Translation Guidance Document for DSEN Researchers and Stakeholders

November 2014

 

List of acronyms used in the document

AMSTAR A Measurement Tool to Assess Systematic Reviews
CADTH Canadian Agency for Drugs and Technologies in Health
CIHI Canadian Institute for Health Information
CIHR Canadian Institutes of Health Research
CMA Canadian Medical Association
CNA Canadian Nurses Association
CNODES Canadian Network for Observational Drug Effect Studies
CPhA Canadian Pharmacists Association
CPSI Canadian Patient Safety Institute
DSEN Drug Safety and Effectiveness Network
DSEN CO Drug Safety and Effectiveness Network Coordinating Office
EoPKT End of Project Knowledge Translation
F/P/T Federal/Provincial/Territorial
HC Health Canada
iKT Integrated Knowledge Translation
INESSS Institut national d'excellence en santé et services sociaux
KT Knowledge Translation
Magic Methods and Applications Group for Indirect Comparison
Prevent Pharmacogenomics of Adverse Events National Team
Sac Science Advisory Committee
Search DSEN active Surveillance and Evaluation of Adverse Reactions in Canadian Healthcare team
SSHRC Social Sciences and Humanities Research Council
CFHI Canadian Foundation for Healthcare Improvement
PMAP Project Management Action Plan

Table of Contents


Introduction

Objectives

This document has been prepared to assist the DSEN community (researchers and stakeholders alike) in understanding the translation and dissemination of research results to Query Submitters and other audiences. It is intended to provide clarity and transparency on the process and make tools available in the form of adaptive procedures and templates. This, in turn, is expected to promote the dissemination, application, and impact of DSEN research results.

Background

Knowledge translation (KT) is inherent in the DSEN Program design which incorporates a variety of mechanisms to exchange research knowledge. Regular interactions among the DSEN Steering Committee, the DSEN Coordinating Office (DSEN CO), and the research teams (the Network) serve to bring researchers and users of information together to ensure that the research conducted will address identified information gaps. Research findings are shared with the Steering Committee, the centres and teams participating in the Network, and DSEN stakeholders. In many cases, research results are published by the researchers in peer-reviewed journals and/or disseminated through scientific fora, and the DSEN CO publishes results on its CIHR website and makes them available in a public repository.

As the research project is completed, the research teams bring the new research knowledge back to the stakeholder group that submitted the Query (Query Submitter) for consideration as part of their decision-making process, and when relevant, for broader dissemination through their own established communication channels.

Leveraging existing investments in KT activities in other organizations maximizes DSEN funds available for research. For example, connecting to CIHR's existing KT activities and partnering with other organizations that facilitate KT such as CADTH, with whom DSEN developed a collaborative agreement, also enhance the DSEN's KT capacity. The Network also routinely engages, through semi-annual Network meetings, workshops, DSEN Steering Committee meetings, biannual Newsletter, a broader stakeholder audience including health care providers and patients to share learning from the Network's research results.

This document serves to guide DSEN researchers and stakeholders towards further development of materials for various audiences through tailoring key messages and suggesting tools for dissemination. This includes guidance for the implementation of transparent plans/methods to communicate research results regarding benefits, harms, and uncertainties of medications studied, tailored to the federal, provincial and territorial (F/P/T) governments and other relevant organizations.Footnote 1

The proposed activities are based upon the DSEN program design as well as on the proposed KT activities of the funded teams comprising the Network. The KT functions outlined are not the sole responsibility of any one individual network component. Rather, responsibility is distributed across the stakeholders and managed in as collaborative a fashion as possible.

This document:

  1. presents the CIHR and DSEN's approach to knowledge translation;
  2. defines relevant terms and concepts;
  3. lists the KT principles under which the Network operates;
  4. describes some modes of effective translation of research results to specific audiences;
  5. and offers some adaptive KT tools and a standardized reporting/disseminating template.

Part I - Definitions

Knowledge Translation

DSEN has adopted the CIHR definition of KT: "At CIHR, KT is defined as a dynamic and iterative process that includes synthesis, dissemination, exchange and ethical application of knowledge to improve the health of Canadians, provide more effective health services and products and strengthen the health care system."

About knowledge translation & commercialization

"This process takes place within a complex system of interactions between researchers and knowledge users which may vary in intensity, complexity and level of engagement depending on the nature of the research and the results, as well as on the needs of the particular knowledge user."

KT is important to CIHR and DSEN because:

  1. "The creation of new knowledge often does not automatically lead to widespread implementation or impacts on health.
  2. With the increased focus on research governance and accountability from the federal, provincial, and territorial governments, as well as from the public, it becomes increasingly important to demonstrate the benefits of investment of taxpayer dollars in health research by moving research into policy, programs and practice."
More About Knowledge Translation at CIHR

Integrated KT (iKT)

Integrated KT means the involvement of the knowledge user right at the start of the project. Within the Network, Query Submitters are involved in nearly all aspects of DSEN governance and advisory committees as well as many stages of the research process. They pose the research question, work with researchers to refine the question; provide needed direction and clarification about the question during the research process, if requested; assist in interpreting the results and crafting the message; identify potential audiences for dissemination of the results, if needed; and consider results in their decision making as appropriate (See Appendix 1: KT in the DSEN Query Process)

For example, the Network is designed to foster equal engagement by researchers and decision-makers by regularly bringing them to ensure that the research conducted addresses identified information gaps relevant to decision-making. In keeping with this, DSEN implemented a Science Advisory Committee (SAC) to:

  • facilitate and improve Query submission by increasing interaction between researchers and Query Submitters i.e., during the identification, development and formulation of new Queries;
  • build tangible relationships between researchers and the Query Submitters by allowing them to meet face to face and keep a line of communication open between them for the duration of the research project, if needed.

To this end, it is important that the points of contact between the research team and the Query Submitter's group are well identified. The DSEN CO can facilitate the communication upon request and should be kept informed in case of changes to the persons responsible for the Query both on the Query Submitter's and the research team's ends.

End of Project KT – Dissemination

End of project KT is the dissemination of the research results and it has a very important role to play for DSEN. Activities that focus on intensifying and accelerating the impact of DSEN's research results are essential to its mandate to "increase the evidence on drug safety and effectiveness available to regulators, policy-makers, health care providers and patients."

End of Project KT can involve three different levels of dissemination:

  • translation of the research results by way of presentation, research report, teleconference, or other means, to the Query Submitter;
  • translation of the research results to the Query Submitter, followed by publication in a scientific journal / presentations at scientific fora;
  • more intensive dissemination activities that tailor the message and medium to a specific audience (e.g. clinicians, patients groups, professional associations) through traditional media, social media, research abstracts, videos, etc.. This more intensive dissemination of the results should happen after the results have been transmitted to the Query Submitter.

For its part, the DSEN CO will post on its CIHR website a one-pager describing the summary of the research results (see example in Appendix 3) for all funded projects whether they are the object of scientific publications / presentation or not. More extensive products (3-pager, presentations, full research results reports) will be made available on a public repository as soon as possible after the reporting to the Query Submitter and eventual scientific publication / presentation.

The DSEN CO can also facilitate the dissemination of research abstracts and publications to external stakeholders beyond the posting of the one-pager research abstract (and other documents as available) on the CIHR website and DSEN KT repository. Different approaches can be considered on a case by case basis with regards to the level of confidence in the findings and impact of the research results, and the audience to be reached.

Part II – The Practice of KT in DSEN

DSEN Knowledge Users and Stakeholders

When planning for KT, it is important to know that DSEN has two distinct categories of knowledge users and stakeholders: those "internal" and those "external" to the Network.

Internal stakeholders include:

  • DSEN Funded collaborating teams (i.e., CNODES, SEARCH, PREVENT, CAN-AIM, MAGIC)
  • Program partners (i.e., HC, CADTH, INESSS)
  • Steering Committee members

External stakeholders include:

  • Citizens (e.g., patient groups, individuals receiving care, caregivers responsible for delivery of care)
  • Voluntary Health Organizations (e.g., Heart and Stroke, Canadian Cancer Society)
  • Prescribers/clinicians/pharmacists
  • Governmental agencies (e.g., CPSI, CIHI)
  • Professional health organizations (e.g., CMA, CNA, CPhA)
  • Academic educators (Schools of Nursing, Medicine, Pharmacy)
  • Administrators (including regional and district health authorities)
  • Research Funders (e.g., SSHRC, CHSRF)
  • Researchers outside the Network

At this time, only the F/P/T representatives are allowed to submit Queries to DSEN.

DSEN KT Principles

A number of KT principles have been endorsed by the Network to guide its KT activities and approaches:

  • The primary audience for DSEN KT is the F/P/T Query Submitter as well as other policy audiences (other F/P/T decision-makers, through CADTH and INESSS);
  • Query Submitters require results (as soon as available during the course of the research, and final results) in advance of scientific publication / presentation;
  • DSEN does not make recommendations; thus KT products should not prescribe actions to decision makers, but can offer evidence-based clinical implications;
  • Researchers' Intellectual Property will be safeguarded within the Network until such time as researchers publish results;
  • There should be no surprises for the query submitters about the results (i.e. results will be communicated and made clear to the Query Submitter before they are more largely disseminated);
  • DSEN-funded research teams must be represented in the Network KT activities (including Science Advisory Committee meetings and semi-annual Network Meetings)

Knowledge Translation – Some approaches

Multiple strategies could be used to get the message to the internal and external stakeholders. These include (i) traditional KT approaches such as publication in peer reviewed and open access journals, depositing results in open access research repositories, conference presentations, education sessions, visual lectures, briefings, and webinars; (ii) use of traditional and social media; (iii) outreach facilitation; (iv) use of champions and knowledge brokers; and (v) arts-based KT activities. Decisions about the intensity of KT strategies should be judged against the imperative to ensure the message is received by a specific audience (for more information about possible KT strategies and activities, see "Supplementary information and tools" at the end of this document).

With an iKT approach to dissemination, KT planning consultations may be held with the Query Submitter (as appropriate) in order to ensure that the best approach is used. The DSEN CO can facilitate those planning consultations upon request. This may include:

  • reviewing the results
  • determining the necessity of KT beyond the Query Submitter and internal stakeholders
  • identifying appropriate audiences
  • suggesting strategies to reach these audiences
  • reviewing messages to make sure they are in a language and format suitable for specific stakeholders
  • identifying potential champions for dissemination
  • Leveraging existing communication channels to reach identified stakeholders

Of note: The timing of these larger KT activities with stakeholders should be considered in light of the need to keep the study results embargoed until publication.

Reporting Results to Query Submitters and Other Internal Decision Makers

In the context of DSEN, the first audience in the chain of dissemination is always the Query Submitter. All DSEN study results should be presented to the Query Submitter in a format suitable for their use, and be shared with the DSEN Coordinating Office. DSEN KT products should be held in an open access Research Repository. The Repository should meet open access criteria and CIHR's policy on open access (CIHR Open Access Policy) by making accessible all study results and KT tools that are developed for various audiences. The results that are published in the peer reviewed literature should be open access whenever possible to comply with CIHR's policy.

There are many steps in the life cycle of a DSEN Query where information exchange regarding a Query is essential. A Table of Reporting and Dissemination (Appendix 2) has been prepared to help stakeholders identify the KT product, parties responsible for its creation, target recipient(s) and the recommended level of dissemination. When there are no concerns about the need to keep the study methodology confidential, the study protocol should also be published.

It is recommended that DSEN research teams use a graded entry format (1-3-25) for reporting research results. These 1-3-25 reports will be the foundation for all dissemination activities and will facilitate development of derivative KT tools. '1' refers to one page of key messages that will be available to all audiences through the DSEN repository and CIHR website, '3' refers to a three page high-level summary of the research or executive summary, and '25' refers to a research report of approximately 25 pages presenting the full results in a format agreeable to the Query Submitter (e.g. the full publication, Power Point, Policy Paper, etc.). In every case, the DSEN Coordinating Office should receive copies of the (see Appendices 2, 3, and 4 for the one-pager template and suggestions of items to be included in each research report format).

Reporting Interim Results

Decision makers are interested in knowing that projects are on track and want to receive results as they become available. Interim reports allow for the sharing of important information on the status of projects and may include identification of challenges requiring alteration of project milestones and / or research protocols as well as provision of early results from sub-samples or initial or preliminary analyses. This type of reporting is considered internal and is not viewed as a DSEN knowledge translation activity per se. Interim results and reports will not be made publicly available. In general, reporting to DSEN and the Query Submitter during the course of the research project is considered an internal process and, as such, is addressed by the Project Management Action Plan (PMAP) framework.

Reporting and Disseminating Final Results

For final results, it is recommended that research teams generate all three (1-3-25) documents. This makes available graded reports, meaning that readers can choose the level of information they would like, with each component providing greater detail than the component before. Each component of the 1-3-25 style report is structured, headings are consistent across reports, and each component can stand on its own.

  • 1 page Report. This is a high level summary focusing on key findings. One of its uses will be to disseminate results via the published research agenda posted on the DSEN CIHR website and it will be deposited in the DSEN public KT repository
  • 3 page Report. This is an executive summary, or structured report of approximately 3 pages. It should also be used for reporting preliminary or interim results (when available), and can take the form of a Power Point
  • 25 page Full Report or academic publication. In the case of Queries not intended to be published in academic journals, the DSEN Coordinating Office will expect to receive a full research project report.

Depending on the nature of the study results and the interest in the results by other policy makers/knowledge users, dissemination to provincial and territorial authorities will be done through CADTH and its Optimal Use Working Group under the Agreement for a Joint Project developed between CADTH and DSEN. Sharing the study results with other audiences would likely only occur after the study results are published and no longer considered confidential.

DSEN expects:

  • Ongoing interactions between researchers and the Query Submitter throughout the research project (through the SAC meetings and other means);
  • A focus on timely sharing of relevant information;
  • Graded entry reports (e.g. 1 page, 3 page, 25 page versions or equivalent );
  • Structured and standardized one-pagers for public repository and CIHR website (other more extensive reports will be posted to the public repository, as they become available).

Reporting Results to External Stakeholders

In the Network, the decision to disseminate research results to external stakeholders should be made judiciously and left to the discretion of the individual teams, in consultation with the Query Submitter. Beyond making the research report (and summary products) freely available, decisions about the extent and aggressiveness of KT plans to actively disseminate results beyond the Query Submitters should be guided by the reliability, validity, and strength of the generated results (i.e., when the level of certainty in the results is high) and by the receptivity (i.e. the "pull") from external stakeholders. As a result, a more extensive KT strategy should be developed for each project based on the stakeholders, pertinent findings, and actionable items.

End of project KT plans should be discussed with the Query Submitter at the beginning of the query process and may reviewed or revised once the interim results and/or final results are available (as necessary). The message for dissemination to external stakeholders should be adapted for the audience to be reached and different KT strategies should be considered.

Conclusion

Evidence generated via DSEN provides decision-makers with an important additional source of information about drug products' therapeutic benefits relative to their harms. DSEN evidence also supports decision-making on public reimbursement, as well as safe and optimal prescribing and use of drugs within Canada.

  • DSEN is committed to operating in a transparent manner.
  • Submitted Queries will be treated as work in progress during the feasibility and prioritization review phases.
  • Prioritized Queries will be posted on the CIHR/DSEN website. Posted Queries will have passed through feasibility and prioritization processes and be identified by the DSEN Steering Committee as forming part of the DSEN prioritized research agenda.
  • A dialogue between researchers and Query Submitters should be maintained throughout the entire Query process.
  • One-pagers will be posted in a publicly accessible repository and on the DSEN CIHR website; other documents will be posted to the public repository as they become available.

DSEN welcomes the inclusion of data from many sources within the studies conducted by the Network, provided they adhere to DSEN's transparency principle. All data analyses would need to be available outside any confidentiality agreements so that third parties can replicate/validate/assess DSEN generated evidenceFootnote 2. Ultimately, all evidence produced through DSEN research will become publicly accessible.

Appendix 1: KT in the DSEN Query Process

DSEN Query Process Map

Appendix 1 figure: long description

Appendix 2: Table of Reporting and Dissemination to Stakeholders

Network (internal stakeholders) = DSEN CO, DSEN Funded collaborating teams (CNODES, SEARCH, PREVENT, CAN-AIM, MAGIC), Program partners (HC, CADTH, INESSS) and DSEN Steering Committee.
What Prepared by Audiences Status
Internal Informal or pre-Query Decision Maker Research Teams, SAC members, FPT Drug Plans via CADTH, INESSS, DSEN CO Sharable within the Network.
Formal Query Decision Maker in consult with Research Teams Research teams to go forward with feasible protocols, FPT Drug Plans via CADTH, INESSS, DSEN CO Research Question posted to DSEN website, Complete Query form available upon request to DSEN CO.
Project timelines (P-MAP) Research Teams Query Submitter, HC, FPT Drug Plans via CADTH, INESSS, DSEN CO Status updates on project milestones sharable within the Network. Not a KT product, for internal use only.
Protocols Research Teams Query Submitter Considered as work in progress until publication.
Interim results/report(s) Research Teams Query Submitter Interim results to be shared only with Query Submitter. Part of the PMAP process, not a KT product.
Final report(s) Research Teams Query Submitter, CADTH, INESSS, DSEN CO Publicly available unless embargoed pending academic publication.
External Publications Research Teams Public Scientific publications by Research Teams (copy to DSEN CO). Full research reports deposited in DSEN repository.
Further dissemination to F/P/T (through CADTH) as it relates to decision makers accountabilities.
DSEN 1-pager Research Abstract Research Teams Public One page Research Abstract posted to DSEN website.
Other KT products Decision Makers Research Teams
DSEN CO
Dissemination to targeted audiences (e.g. prescribers, patients, pharmacists, etc.) based upon discussions between Research Teams and Query Submitters Researcher IP to be respected through appropriate referencing, agreements, and licensing as appropriate.

Appendix 3: Suggested format for one-pager Research Abstract

Appendix 3 figure: long description

Appendix 4: Suggested Check list for Structured Reporting Template

1 page Query Submitter Report (high level summary)

  • Title: reiterates the Query question
    • Authors and their affiliations
    • Knowledge users involved in project
  • Key messages (results), use bullets and tables as appropriate
  • The issue: background – why the Query is important
  • Interpretation of significance of results
  • Methods
  • Cautionary notes

Appendix 5: Suggested Check list for Structured Reporting Template

3 page Query Submitter Report (executive summary)

  • Title: reiterates the Query question
    • Authors and their affiliations
    • Knowledge users involved in project
  • The issue: background – why the Query is important to Query Submitter (why the question is a priority and for whom – this should be discussed with the Query Submitter)
  • PICOST (depending on the methodology)
    • Population
    • Intervention
    • Comparators/Context
    • Study Design
    • Time period
  • Key results: main results including estimates of benefits, harms, costs where relevant and quality of the evidence, use bullets and tables as appropriate
  • Degree of uncertainty related to key results: differential effects by subgroups: equity considerations, sex and gender based analysis; generalizability of results, most important study limitations that might influence interpretation/applicability of results
  • Links to additional information (where applicable)
  • Statement about whether document has been peer-reviewed or not, and if so by who: policy makers, researchers, stakeholders
  • Funding
  • Statement about conflict of interests

Appendix 6: Suggested check list for Structured Reporting Template

25 page Query Submitter full report

In some cases the researchers' peer reviewed publication might mirror the full DSEN report, however in other cases the peer reviewed publication might focus on methodological issues or analyses beyond the original query question. As the full report will eventually be made publicly available, it must focus exclusively on reporting the results to the Query question. The same should apply in the cases where no academic publication is considered. The full report is needed as part of DSEN's accountability, transparency, and performance measurement requirements.

  • Title: reiterates the Query question
    • Authors and their affiliations
    • Knowledge users involved in project
  • The issue: background – why is the query important to the Query Submitter (why is the question a priority and for whom?) Much of the information for this item can be obtained from the Query
  • PICOST (if applicable)
    • Population (what is the study population?; age, sex, geographic location, health condition of interest)
    • Intervention (what is the drug(s) of interest?)
    • Comparators/Context (when the study does not involve comparators, describe the context of the study participants)
    • Study Design (what is the study design?)
    • Time period (what is the time period encompassed by the study?)
  • Details of the outcome measures of interest (what are they, how were they calculated?)
  • Methods: provide a detailed description of methods used to identify, select, access, synthesize the research evidence
  • Results: where relevant include estimates of benefits, harms, costs of the intervention. Where relevant provide an assessment of the quality of the evidence, use bullets and tables as appropriate
  • Differential effects by subgroups: present equity considerations, sex and gender based analysis, etc.
  • Degree of uncertainty related to key results: describe the reliability, validity, generalizability of results and discuss study limitations that should be considered when interpreting or assessing applicability of results
  • Interpretation of significance of results for (as appropriate):
    • Query Submitter, internal policy makers
    • external policy makers
    • prescribers/ clinicians
    • patients/public
    • other relevant stakeholders
    Including resource implications if relevant
  • Links to additional information (where applicable)
  • Statement about whether document was peer-reviewed or not, and if so by who: policy makers, researchers, stakeholders (the reviewers could also appraise the report using appropriate tools (e.g. AMSTAR for systematic reviews) and their appraisal could be appended to the report.
  • Funding source
  • Statement about conflict of interests

Appendix 7: Frequently Asked Questions

Who should disseminate results to "external stakeholders"?

The DSEN Coordinating Office can act as facilitator for studies whose results are likely to have importance beyond the focus of the Query Submitter. The basic approach for all DSEN-funded research will be to disseminate the study results (one-pager and other materials, if available) through the DSEN CIHR website and the DSEN KT repository. More extensive and focused KT can be envisioned for which the DSEN CO can facilitate contacts between research teams and external stakeholders. This should be evaluated on a case by case basis in association with the Query Submitter.

How can the DSEN CO facilitate contacts between researchers and Query Submitters?

In cases where there is an obvious need for more communication between the research teams and the Query Submitters, over and above the initial Science Advisory Committee meeting, the DSEN CO should be informed of the situation and can facilitate subsequent teleconferences or provide names of contacts.

The DSEN CO is also implementing the Project Management Action Plan (PMAP) for each study undertaken by the Network teams. The PMAP will provide both the DSEN CO and the Query Submitter with a pre-determined timeline for each Query right at the beginning of the research project. The PMAP will serve to document the research progress and to report unanticipated changes/delays to the project timeline.

What is a DSEN KT product?

A DSEN KT product is a DSEN-funded research report, disseminated by DSEN. This does not negate the Intellectual Property of authors. DSEN-funded researchers are engaged in the Query process from the beginning to the end, along with the Query Submitters, and are recognized as the authors of the DSEN KT product.

When should the DSEN template be used versus the individual Network Teams's own branding templates?

All research reports produced for DSEN and to be posted to the DSEN CIHR website or the DSEN KT repository should be on the DSEN reporting template. Individual teams are welcome to use their own template when posting on their website or their affiliated institution's website.

Supplementary information and tools

KT considerations

KT principles Considerations
Integrated KT approach Who are knowledge users?

How will they be engaged?

When will they be engaged?
End of project KT Who is the audience?

What is the message?

What is the medium to reach the audience?

Who are credible messengers?

How will end of project KT be evaluated?
Balancing knowledge users and researchers need for information and publishing What do knowledge users need to know?

When?

Are there issues around publishing results?

What is the process for resolving differences?
"Judicious" KT Is the evidence sufficiently rigorous and robust to justify KT activities beyond standard diffusion and dissemination (publication, report etc.)?

What is the process for determining when extraordinary KT efforts are required?
Open access How will CIHR's open access policy be met?
Building capacity for KT practice and science What are the learning opportunities for DSEN researchers and trainees to increase their knowledge of the practice and science of KT?
Advancing KT science How can DSEN studies and their dissemination be leveraged to advance the science of KT (i.e. studying the best ways to present information to different knowledge users groups, studying how to engage in integrated KT, etc.)

Other tools and references

For more information, references, and tools regarding Knowledge Translation, please see the Guide to Knowledge Translation Planning at CIHR: Integrated and End-of-Grant Approaches.

Footnotes

Footnote 1

This document draws heavily on the DSEN KT workshops (March 2013 and February 2014) documents, extensive consultations with affiliated DSEN federal and provincial decision makers and DSEN affiliated researchers, which provided an opportunity to gather feedback that is reflected in the present document.

1

Footnote 2

In the case of administrative health data, individual level data are not available for general analysis without project-specific ethics and/or government approval for access, but the results derived from meta-analysis will be made available.

2

Date modified: