Evaluation Report - Consultations on CIHR Draft Privacy Best Practice Guidelines (2004)
Prepared for: The Ethics Office of the Canadian Institutes of Health Research (CIHR)
Prepared by: One World Inc. (November 2004)
The Ethics Office of the Canadian Institutes of Health Research (CIHR), with the advice of its Privacy Advisory Committee, has developed draft privacy best practice guidelines for addressing privacy, confidentiality and security concerns in the design, conduct and evaluation of health research. A public consultation process to obtain feedback on the draft guidelines was conducted from March to September, 2004. It was designed to test out the guidelines with a wide variety of stakeholders involved in various capacities in health research. It also included a limited consultation with citizens.
The consultation process had three complementary components:
- Three multi-stakeholder sessions, each emphasizing a different research theme;
- Small group dialogues with citizens; and
- Written submissions, including an on-line questionnaire.
This process evaluation was conducted based on an evaluation framework agreed upon before the consultation process began. There were three key sources of data: participant evaluation forms, interviews with members of the Privacy Advisory Committee and the involved CIHR staff, and documentation.
Overall, the evaluation concludes that the consultation process was very successful as assessed through the four areas of inquiry.
- The process did largely fulfill the guiding principles of transparency, inclusiveness and critical dialogue. A contributing factor to the consultation’s success was flexibility and CIHR’s willingness to revise supporting documents and strategies as required.
- The process was very effective in getting feedback on the guidelines. It should be noted that the complementarity of the three process components was important in achieving comprehensive feedback. Certainly, the multi-stakeholder sessions coupled with the opportunity for written feedback enhanced the possibilities for considered reflection and feedback.
- The perspective of staff and PAC members, based on anecdotal information from others, is that visibility and interest in the guidelines was enhanced. Over 500 direct invitations were sent out to stakeholders. These were complemented by the website, the use of list-serves, announcements at conferences, etc.
- From the comments of participants and the perspectives of PAC members, CIHR has clearly demonstrated the kind of national leadership participants expect from it. The inclusive and yet pro-active style of leadership being demonstrated by CIHR was highly commended as were the staff involved in this process.
This consultation process did most things right, and certainly features of it should be replicated in future consultations when appropriate. These features include:
- A process design that encouraged small group work and productive dialogue;
- A document or discussion paper that was clear and met the challenge of presenting complex and dense material in a format that could be readily used in consultation;
- Good, professional facilitation that created an environment conducive to a respectful but frank exchange of ideas;
- A proactive outreach strategy that helped ensure the desired diversity and calibre of participants; and
- A multi-faceted consultation process with complementary strategies –– in particular, in this instance, the combination of the multi-stakeholder sessions with the opportunity to make written submissions.
Possible improvements for future consultations are:
- Ensure that key documents are easily accessible on-line before the first invitations are sent out.
- Allow more time between invitations and the deadlines or sessions. For example, the 2.5 weeks for the May session was very tight and did not allow time for word-of-mouth recruitment.
- Allow more time when the invitation process is coordinated with partners.
- Send out invitations for written submissions earlier in the process and provide a longer period for feedback. Avoid having the summer period as the only time for written submissions.
- Use an on-line feedback format that permits people to save their feedback and return to it at a later point in time.
- Have a note-taker at each small break-out group, when overall resources permit.
- Use the same key consultation document for all multi-stakeholder sessions. This is a general best practice to better ensure comparability of results.
- Pilot-test consultation materials, such as the scenarios used in the small group dialogues.
- Ensure that statistics are collected on the number of guidelines/invitations sent out.