Appendix 4: Instruments

Midterm Evaluation of the Pandemic Preparedness Strategic Research Initiative

[ Table of contents ]

Key informant interview guide, v4 (19.12.08)

Interviewee: _______________________________________________

Interviewer: _______________________________________________

Date: _______________________________________________

Thank you for agreeing to participate in this evaluation of the Pandemic Preparedness Strategic Research Initiative. Its objective is to assess the overall design and implementation of the Initiative, in order inform future developments in the PPSRI, as well as similar strategic initiatives to be undertaken by III.

Note that interviewees will be asked to respond to only those questions for which they judge they can contribute an informed opinion.

1. Introduction

1.1 To begin, could you please describe how familiar are you with the Initiative? What has been your involvement with the PPSRI?

2. Effectiveness of priority setting

2.1 What is your overall opinion on the research priority setting processes implemented under the PPSRI? Which aspects seemed to work well? Which aspects seemed to not work so well?

Follow up if not addressed in response provided above - To what extent were the PPSRI research priority setting processes appropriate?

Follow up if not addressed in response provided above - How would you assess the processes in terms of:

  • Timeliness?
  • The consultation mechanisms used?
  • Who was and was not involved?

a) What alternatives do you consider might have been more appropriate? What changes would make improvements?

2.2 What is your overall opinion on the actual set of priorities that have been selected for Canadian research in pandemic preparedness?

Follow up if not addressed in response provided above - To what extent did the process arrive at the most appropriate set of priorities for Canadian research in pandemic preparedness?

a) When you consider the actual set of priorities selected, is there any national and/or international duplication of effort with this set of priorities?

  • If yes, in which areas? What changes to the process of priority setting would have assisted in avoiding duplication?
  • If no, how was this avoided?

b) When you consider the actual set of priorities selected, is there sufficient room for complementarity with other initiatives?

  • If yes, how was this achieved? With which other initiatives?
  • If no, what changes to the process of priority setting would have assisted in ensuring or enhancing complimentarity with other initiatives?

2.3 Were you involved with the CIHR Pandemic Preparedness Task Group? How effective was it as a priority setting mechanism for the PPSRI?

Follow up if not addressed in response provided above - what are your observations and opinions with respect to:

  • How necessary it was to convene such a group
  • Task Group's role
  • Membership

a) What alternatives do you consider might have been more appropriate or effective? What changes would make improvements?

3. Effectiveness of partnership development

3.1 To what extent has the PPSRI had an influence on national and international partnerships? (Describe observed impacts)

Follow up if not addressed in response provided above - what are your observations and opinions with respect to:

  • Which partnerships have been most and least successful?
  • Partnerships that should be developed but have not been - which ones, and why?
  • What alternatives do you consider might have been more successful?

3.2 As far as can be seen at this point, what have been the impacts of partnerships on:

  • coordination and integration of national and international research programming?
  • resource leveraging?
  • research duplication and complementarity?

a) What alternatives do you consider might have generated more impact? What changes would make improvements?

3.3 What are your opinions and observations on the outcome(s) that the partnership structure under the PPSRI has had on ongoing initiatives of partners? On the PPSRI itself? (Describe observed outcomes)

a) What alternative partnership structures do you consider might have been more effective? What changes would make improvements?

3.4 What are your opinions and observations on the outcomes(s) that the reporting strategy of the PPSRI has had on ongoing initiatives of partners? On the PPSRI itself? (Describe observed outcomes)

a) What alternative reporting strategies do you consider might have been more effective? What changes would make improvements?

4. Appropriateness of program design

4.1 What is your perspective on whether or not the set of funding opportunities offered is assisting the PPSRI in achievement of the Initiative's objectives?

Follow up if not addressed in response provided above -

  • Which of the funding opportunities do you consider most and least successful?
  • Are there gaps in the funding opportunities made available so far? If so, which?
  • Is the overall approach of strategic competition effective in ensuring the most promising research is funded? What would have been alternatives? How would they have been more effective?
  • Is this the most effective approach to ensure both short and long term capacity development? What would have been alternatives? How would they have been more effective?

a) What alternative funding opportunities do you consider might have been more appropriate? What changes would make improvements?

4.2 What are your main observations on the communication strategy used to launch the funding opportunities?

Follow up if not addressed in response provided above -
Which aspects of the communications do you consider most and least successful?

a) What alternatives do you consider might have been more effective with respect to the communication strategy? What changes would make improvements?

5. Achievement of desired funding opportunity outputs

5.1 What is your perspective on the uptake of the funding programs?

Follow up if not addressed in response provided above - what are your observations and opinions with respect to:

  • Whether or not the relevant research communities responded to the RFAs
  • Have there been gaps in response? If so, which areas? Which teams? Which key players?
  • The quality of submissions (e.g., teams, proposed projects)

a) What alternatives do you consider might have generated more satisfactory uptake? What changes would make improvements?

5.2 What are your thoughts with respect to the level of coverage across the intended fields of research priorities?

Follow up if not addressed in response provided above - what are your observations and opinions with respect to:

  • Which priorities are more and less well represented across funded projects
    • Vaccines and immunization
    • Virus biology and diagnostics
    • Prevention and treatment
    • Ethics, legal and social contract

a) What alternatives do you consider might have generated more satisfactory coverage? What changes would make improvements?

5.3 What is your opinion with respect to whether or not the funded projects will result in research training in areas related to pandemic preparedness?

Follow up if not addressed in response provided above -
Would these training opportunities likely have existed without PPSRI funding?

a) What alternatives do you consider might have generated more training opportunities?

5.4 What is your opinion on the likelihood that there will be sufficient knowledge translation of the results from the funded projects?

Follow up if not addressed in response provided above -
Do project proposals include realistic, relevant plans for knowledge translation?
Are there the appropriate end-users involved in project teams for this to occur?

a) What alternatives do you consider might ensure appropriate knowledge translation occurs for projects?

6. Success of PPSRI networking activities

6.1 From your observations, what impact has the PPSRI had on the facilitation of communication and networking among researchers involved in pandemic preparedness research?

Follow up if not addressed in response provided above -

  • How has the PPSRI facilitated communication and networking? (examples)
  • What are the more formal vs. more informal opportunities provided?
  • Have all relevant teams and individuals been provided with networking opportunities?
  • How satisfactory has been the uptake of these opportunities?

a) What alternatives do you consider might have been more successful? What changes would make improvements?

6.2 Have you observed whether or not the communication and networking activities had any impacts on enhanced collaboration? Increased research capacity?

Follow up if not addressed in response provided above -

  • If yes, how was this achieved? Are these more formal or informal activities?
  • If no, what changes to these activities would have assisted in ensuring or enhancing collaboration? Increasing research capacity?

7. Final comments

7.1 Do you have any other comments on the PPSRI and how it, for future strategic initiatives, might be improved?

Thank you for your collaboration!

E-survey of researchers

Cover E-mail

Thank you for your interest in the evaluation of the Pandemic Preparedness Strategic Research Initiative (PPSRI).

Purpose: This survey is part of the midterm evaluation of the PPSRI, being conducted by the Institute of Infection and Immunity (III) of the Canadian Institutes of Health Research (CIHR). The evaluation will assess the overall design and implementation of the PPSRI, in order to inform future developments in this initiative, as well as similar strategic initiatives to be undertaken by III.

Deadline: Please complete this survey by February 28, 2009.

Questions: If you have any questions about the evaluation or the survey, please do not hesitate to contact Celine Pinsent, Project Manager, at 613-230-5577 X 226 or cpinsent@ggi.ca, or Carol Richardson, at 519-661-3228 or carol.richardson@schulich.uwo.ca.

Web-based Survey

A. Awareness

1. I have heard of the Pandemic Preparedness Strategic Research Initiative.
Yes
No
Don't know
2. My research work is relevant to pandemic preparedness and influenza research in the following areas:
- Vaccines and immunization
Yes
No
Don't know
- Virus biology and diagnostics
Yes
No
Don't know
- Prevention and treatment
Yes
No
Don't know
- Ethical, legal or social aspects
Yes
No
Don't know
- Other area related to pandemic preparedness and influenza research:
Yes
No
Don't know
If yes to above: Please specify which other area:

B. Decision to apply
Applicants to the PPSRI funding opportunities

3. Were any of the following a factor in your decision to submit your proposal to the PPSRI?

Yes No Degree of fit of my research with the themes
Yes No Qualifications of me and my co-applicants
Yes No Size of the grants offered
Yes No Type of grants offered (e.g., team grants)
Yes No Duration of the grants offered
Yes No Relative prestige compared to other programs available
Yes No Program seemed to be targeting specific groups of researchers
Yes No Expected involvement of students in the project
Yes No Needed more research funding
Yes No Timing of RFA was good
Yes No Past experience with applications to CIHR
Yes No Thought my/our chances of success were good
Yes No Other reasons: ______________________________________

Non-applicants to the PPSRI funding opportunities

3. Were any of the following a factor in your decision NOT to submit a proposal to the PPSRI?

Yes No Was not aware it existed
Yes No Degree of fit of my research with the themes
Yes No Qualifications of me and my co-applicants
Yes No Size of the grants offered
Yes No Type of grants offered (e.g., team grants)
Yes No Duration of the grants offered
Yes No Relative prestige compared to other programs available
Yes No Program seemed to be targeting specific groups of researchers
Yes No Expected involvement of students in the project
Yes No Already had enough research funding
Yes No Timing of RFA was poor
Yes No Past experience with applications to CIHR
Yes No Thought my/our chances of success were not good enough
Yes No Other reasons: ______________________________________

All successful and unsuccessful applicants to the PPSRI funding opportunities
C. Views of the design and competition process

To what extent do you agree or disagree with the following statements?

Strongly Disagree Disagree Neutral Agree Strongly Agree Don't Know
4. I felt there was conflict of interest in the way the PPSRI was designed.
5. The PPSRI's competitions' peer review committees had adequate expertise.
6. I was aware of the criteria used to judge the PPSRI funding applications.
7. The PPSRI competitions' funding decisions were fair.
8. I was provided with sufficient information with respect to the reason(s) my PPSRI funding application was successful or unsuccessful.
9. I felt there was conflict of interest in the way the PPSRI applications were reviewed.

Successful applicants to the ppsri funding opportunities
D. Communication, networking, impacts on research and knowledge transfer

Please note: if you have received more than one grant through the PPSRI, please answer in terms of the most recent.
To what extent do you agree or disagree with the following statements?

Strongly Disagree Disagree Neutral Agree Strongly Agree Don't Know
10. Communication about the PPSRI has been adequate.
11. The PPSRI has helped me/my team network with other researchers involved in influenza and pandemic preparedness research.
12. The PPSRI has helped me/my team collaborate with other researchers involved in influenza and pandemic preparedness research.
13. Because of the PPSRI, my research work involves researchers from areas or disciplines that were not involved with my work before.
14. The PPSRI has contributed to a reorientation of my research focus.
15. The PPSRI has helped me/my team involve international research collaborators.
15a. (If "agree" or "strongly agree" to question 15)
What are the main roles of your international collaborators? Please check all that apply;

Collaborators working jointly on the same problems
Collaborators working on different aspects of the same problem
Collaborators working on problems pertinent to their setting
Other: please specify: _______________________________
15b. (If "agree" or "strongly agree" to question 15)
Where are your international collaborators located? Please check all that apply;

U.S.
Latin America
Europe
Africa
China
Other Asia
Other: please specify: ________________________________
16. Research users are currently involved in my PPSRI project.

16a. (If "agree" or "strongly agree" to question 16)
What types of research users are involved with your project? Please check all that apply.

Pharmaceutical company
Other type of private sector firm
Government
Not-for-profit, NGO organization
Other: please specify: ________________________________
16b. (If "agree" or "strongly agree" to question 16)
How are the research users involved in your project? Please check all that apply;

Development of research questions
Carrying out some parts of the research
Interpretation of the results
Application or use of the results
Other: please specify: ________________________________
16c. (If "strongly disagree", "disagree" or "neutral" to Q16. Skip if "agree' or 'strongly agree" to question 16 )
Research users will eventually become involved in my PPSRI project.
17. My PPSRI project has a knowledge translation plan in place.
18. My PPSRI project has included interdisciplinary training for students/fellows.
19. My PPSRI project has included mentoring of students/fellows in influenza and pandemic preparedness research.
20. My PPSRI project has increased the number of trainees in influenza and pandemic preparedness research.

Only if role is Nominated Principal Investigator
21. As of December 31, 2008, how many trainees at each of the following levels have been involved in your PPSRI project? (If none, please enter "0")

____ Undergraduate students
____ Master's students
____ PhD students
____ Post-doctoral fellows
____ Other types of trainees (specify)

22. (if the total of Q21 is greater than 0):
Have any of these trainees been involved in international research collaborations?

Yes
No
Don't know

All
E. Continued relevance of the strategies and priorities

In designing strategic research initiatives such as the PPSRI, funding agencies may use different strategies to maximize the chances of significant advancement in knowledge. With respect to the PPSRI, to what extent to do you agree with the following strategies?

Strongly Disagree Disagree Neutral Agree Strongly Agree Don't Know
23. Funding should ensure that recognized leaders in the targeted research areas can continue to pursue relevant research.
24. Funding should ensure that all researchers who can possibly make a contribution have the opportunity to compete for funds.
25. Funding should concentrate on increasing research capacity by targeting researchers and their trainees who are new to the field.

26. In your view, were there important research priorities NOT included in the PPSRI?

Yes
No
Don't know
If yes: Please specify which areas were not included:

27. In your view, what are the most important research priority areas for pandemic preparedness research over the next five years? Please rate the existing priorities and add any others you wish to rate.

Very low priority Low Moderate High Very high priority Don't know
Vaccines and immunization
Virus biology and diagnostics
Prevention and treatment
Ethics, legal and social aspects
Others:
1.
2.
3.
4.
5.

F. Background

The following information will help us analyze survey responses according to different types of research and settings.

28. In which of the following disciplines is your main research experience? (Select all that apply)
Biomedical sciences
Clinical health sciences
Health systems and services
Population and public health
29. With which type of institution or organization are you associated? (Select all that apply)
University
College
University-affiliated hospital
Community-based hospital
Community-based/volunteer agency or organization
Government-funded research agency
Private research organization
Other (specify)

G. Suggestions for improvement

30. Do you have any suggestions for improving the PPSRI?

31. Do you have any suggestions for improving other future Institute of Infection and Immunity strategic research initiatives?

Thank you for your collaboration!

[ next section ]

Date modified: