Moving Population and Public Health Knowledge Into Action
Child and youth health
- Knowledge translation in the community: The Early Child Development Mapping Project
- Joint policy making in early childhood development
- The Canadian Adolescents at Risk Research Network: Research for and with youth
- Should we teach harm minimization to teenagers in school? The production and translation of controversial new knowledge in addictions
Janet N. Mort, Doctoral Candidate, University of Victoria
The Early Child Development Mapping Project is a population-wide developmental assessment of kindergarten children in B.C. Results of the assessment, which evaluates physical, social, emotional, cognitive and language development, are mapped by neighbourhood and reported back to local communities. Hundreds of community-driven early childhood and family projects have resulted. The project's knowledge translation success is attributed to the researchers' personal commitment to the cause and the resulting community involvement from the earliest stages; the presentation of research findings in a way that supports community uptake; and the development of provincial networks to highlight and communicate successful research use.
British Columbia's kindergarten teachers know that as many as 25% of children arrive on the first day of school challenged in fundamental aspects of cognitive, language, social, physical and emotional development. The experiences in the first six years of childhood have a significant impact on learning, behaviour, health and well-being throughout a person's life.
Since 2000, the Early Child Development (ECD) Mapping Project, under the direction of Dr. Clyde Hertzman of the Human Early Learning Partnership (HELP)* at the University of British Columbia, has been conducting a population-wide developmental assessment of kindergarten children in B.C. using the Early Development Instrument (EDI), a questionnaire completed by kindergarten teachers for each child in their class. The EDI measures a child's development in physical health and well-being, social competence, emotional maturity, language and cognitive development and communication skills.
The EDI results are mapped on a neighbourhood basis to gain a greater understanding of the role of community factors in supporting early child development. These results are then reported back to school districts in public meetings to provide them with information about the school readiness of their preschool population and to help communities assess local ECD programs. Smaller meetings for intersectoral groups are also held in most communities to develop more detailed interpretations.
Partners in the initiative include HELP, the B.C. Ministry of Children and Family Development, the B.C. Minister of State for Early Childhood Development, the six major provincial universities and sixty school districts who are responsible for administering the instrument to kindergarten children. The project has significant support and community involvement in B.C., and has helped policy makers, researchers, service providers, and community members monitor, understand and plan for support for early child development.
As a former school superintendent, my interest in the ECD Mapping Project evolved from my initial scepticism about this unusual union between a health researcher and hundreds of kindergarten teachers. What possible difference could this data make in the lives of young children? My research, the EDI Impact Study, focused on what school districts and their communities did with the EDI data and maps when they were returned to them, to determine if programs or policies changed for the better as a result. In effect, I wanted to assess the impact of what appeared to be a model, community-based knowledge translation (KT) effort.
The KT initiative
To assess the success of the project's KT activities, I interviewed members of the participating school districts who had supervised the assessments of the children and actively participated in all stages of the mapping project. Forty-one districts agreed to participate in the study over a period of six months. Interviewees included professionals from health, social services, ECD and education.
My study aimed to address four major questions:
- What processes were used to disseminate the project results in the community and what was the community's initial response to the results?
- Were the process, data and mapping valued?
- What issues or problems arose or were perceived?
- What initiatives or projects have resulted from the experience?
The resulting report1 was published; presented to all participating school districts, members of B.C.'s Legislative Assembly and all deputy ministers; and posted on the HELP website.
Results of the KT experience
Of all the districts interviewed, only one saw no further need for EDI activity. In contrast, most of the other districts are keen to be involved in continuing efforts to refine the tool and to move forward with other developments, such as linking the EDI data with other measurements. A small number of districts reported that, although there had been initial resistance from some kindergarten teachers to administering the EDI, most felt validated by the process.
There was unanimity on the part of those who attended the community reporting presentations that the personal presence of Dr. Hertzman had a pivotal effect. Dr. Hertzman and HELP staff personally visited all regions and most communities in B.C., initially to persuade teachers, school administrators and community representatives about the importance of the initiative, and then returning to present and explain the project results. Almost all contacts commented on the importance of Dr. Hertzman's contribution and his knowledge, passion and commitment to the EDI project.
Respondents valued the experience for different reasons. For some, the data resulted in intersectoral representatives coming together for the first time to discuss common issues and concerns and to begin planning for action. Some school districts felt that it was the first time they had been presented with hard data indicating why they should become involved in addressing the preschool agenda and used the results to confidently reallocate funding to areas of greatest need in their schools. Kindergarten teachers particularly valued the release time provided by HELP for the assessment task and appreciated the additional information it gave them about their students' developmental issues.
The most frequently raised issue was the complexity of the maps and complications that arose as a result of confusion over overlapping school and community boundaries. A few districts questioned the credibility of the assessment tool and, therefore, the data and resulting interpretation. School districts expressed frustration that, even though the mapping data had raised their awareness of the issues, they were still prevented from taking appropriate action by a lack of funding and by a mandate that restricts their responsibility to the five- to eighteen-year-old population.
The result is hundreds of new community-driven early childhood and family projects focused on strengthening social capital and family capacity.
The most significant result of the project was the coalescing and strengthening of intersectoral community coalitions that used the mapping data to develop community action plans and services in the geographical areas of greatest need. The result is hundreds of new community-driven early childhood and family projects focused on strengthening social capital and family capacity. They include parenting programs and resource centres; intersectoral centres based on school sites; hearing, sight and dental screening programs; social development programs; nutrition studies; full-day kindergartens; special programs for Aboriginal preschoolers; First Nations reciprocity agreements; relocation of recreational services; preschool community coordinators; literacy intervention programs; family literacy projects; and numerous forms of new early childhood networks.
How do you galvanize an entire province to engage in meaningful change based on data-driven research? How do you bring diverse professionals with different mandates together to share resources for a common goal? How do you inspire grassroots, community-based change?
The ECD Mapping Project has spurred communities in all areas of the province to engage in change on behalf of young children and provides at least three powerful lessons:
1. The researcher's passion for the cause under investigation and the results of the research must be communicated in person, in powerful ways, to those who share the passion for the common cause.
The researchers—Dr. Hertzman and the HELP staff—played a vitally important role in the KT success of this project. The researchers presented compelling arguments for communities to engage in the project and proved the project's worth by returning to present the research data in public meetings, using visually striking, locally oriented maps. This demonstration of personal commitment and communication was identified as a key reason for the project's success.
2. If the results of the research are presented in compelling ways to key community leaders with a commitment and a responsibility to the cause, it is far more likely that the community will assume leadership for the implementation of change.
External support in the form of hard data and maps provided local leaders with the incentive to form and/or strengthen intersectoral coalitions to plan for shared resources and improved services. These leaders were most often self-selected and came from any one of the participating agencies. The intersectoral coalitions have become leading forces in B.C. communities and have implemented hundreds of projects as a direct result of data provided through the ECD Mapping Project.
3. Provincial networks must be created to highlight and communicate successful models that inspire others and help to build momentum.
The formation of provincial networks reinforced community action planning. Pre-existing government agencies shifted their agendas to support the intersectoral coalitions (and, in some cases, were pressured to do so). New provincial networks and new linkages between old networks are emerging to support the coalitions in the form of presentations at regional conferences, newsletters, university-based gatherings* and a new HELP website* designed specifically to support the intersectoral coalitions. Each had its genesis in the ECD Mapping Project.
Conclusions and implications
Many innovative initiatives continue to unfold in B.C.'s communities on behalf of young children. Inspired by ongoing health research, embraced by intersectoral coalitions and galvanized by community resources, these changes mean the future is getting brighter for our youngest learners. While the ECD Mapping Project continues to establish longitudinal data and to sponsor other ECD-related research, the leadership of the implementation phase has shifted from government to a community base. The collaborative development and practical use of population and public health research has resulted in powerful strategies that support all young children—an excellent example of KT in action.
* For more information about HELP, visit their website.
* As a result of the publication of the EDI Impact Study, the University of Victoria sponsored a colloquium, with HELP funding, to bring together parties to discuss the wealth of initiatives being developed in communities across the province.
* For more information, visit their website.
1. Mort, J. N. 2004. The EDI (Early Development Instrument) Impact Study. Human Early Learning Partnership.
Nazeem Muhajarine, Saskatchewan Population Health and Evaluation Research Unit, University of Saskatchewan
Thomas McIntosh, Saskatchewan Population Health and Evaluation Research Unit, University of Saskatchewan
Ronald Labonte*, Saskatchewan Population Health and Evaluation Research Unit, University of Saskatchewan
Bryan Klatt, University of Regina
Lan Vu, University of Saskatchewan
Fleur Macqueen Smith, Saskatchewan Population Health and Evaluation Research Unit, University of Saskatchewan
A Saskatchewan research project into the impact families' economic circumstances can have on early childhood development generated a number of complex findings. Rather than speculating on the implications, the researchers convened a roundtable for policy makers and program personnel to jointly discuss policy recommendations. The roundtable session and associated knowledge translation activities sparked significant interest in the research project and resulted in a greater understanding of the research findings. It also set the stage for an ongoing dialogue between policy makers and researchers, and may lead to collaborative research in the future.
We know that children who grow up in unsafe, unstimulating or dysfunctional circumstances—in their families, schools or neighbourhoods—are more likely to have problems, as children and as adults. However, we are less certain how multiple circumstances in a changing childhood environment determine health outcomes. With this Canadian Population Health Initiative-funded research project, we attempted to understand the seemingly intuitive question of how families and neighbourhoods help or hinder children in the earliest years of their lives.
Instead of speculating on the policy and practice implications of our findings, we identified a group of decision makers working in early childhood development, and met with them to discuss policy recommendations.
We examined the impact that families' ongoing economic circumstances have on their children's health outcomes, from birth to age eight, using the population of all singleton children born in Saskatoon and Regina between 1992 and 1995 (a total of 17,544 children). Our investigations focused on the continuing health impact of living in low-income families, but also considered how factors related to families and neighbourhoods contributed to adverse birth and health outcomes.
The Birth Cohort Study generated a large number of complex findings, not easily rendered to simple one-sentence messages. Instead of speculating on the policy and practice implications of our findings, we identified a group of decision makers working in early childhood development, and met with them to discuss policy recommendations.
The KT initiative
Our knowledge translation (KT) goals were to give policy makers a more complete picture of early childhood development in the province of Saskatchewan; to begin a process of using the study's results to assess the effectiveness of specific policy interventions (e.g. by highlighting areas or neighbourhoods that might be underserved by specific programs); and to set the stage for ongoing dialogue between policy makers and Saskatchewan researchers in this area.
To formulate a more complete picture of early childhood development in Saskatchewan, we first conducted an environmental scan of the relevant policy landscape in Saskatchewan and Canada.
We then assembled and met with a group of twenty-five decision makers from the provincial government and representatives of community-based organizations in Saskatchewan and Canada. The attendees included policy personnel from the departments of Health, Learning and Community Resources and Employment (formerly Social Services); people responsible for the actual delivery of programs such as KidsFirst (a Saskatchewan early intervention program targeted at high-risk families and areas); individuals involved in community-based research initiatives; and university-based researchers.
We opened our roundtable session with an overview of the results of the Birth Cohort Study, highlighting a few key findings. First, specific neighbourhood characteristics in Saskatoon and Regina appeared to be linked with early childhood health outcomes. Second, some neighbourhoods not currently served by programs such as KidsFirst appeared to be in need of targeted services. Third, cycling on and off of social assistance appeared to have a stronger negative effect on children's use of health care services than did long-term receipt of benefits. Prior to this session, we had distributed a four-page research summary, describing the research questions and key findings. We followed with a presentation of the policy landscape from our environmental scan, and then discussed a series of questions we had posed on the best policy mix for children and how to target services and assess programs.
We subsequently presented the results of both the Birth Cohort Study and the roundtable at the annual general meeting of the Canadian Public Health Association in June 2004. In the fall we produced the report Understanding the Policy Landscape of Early Childhood Development in Saskatchewan,1 which reported both on the environmental scan and on the policy roundtable, and published it in conjunction with a detailed research report2 on the Birth Cohort Study. We distributed these reports to our decision-maker group, associate deputy ministers in relevant government ministries and our funding agencies.
Results of the KT experience
Presenting the results of our research at the roundtable session and through associated activities successfully met our KT goals. The policy and program personnel who participated were very interested in pursuing a more detailed and ongoing dialogue with researchers and were eager to explore the analysis presented in both the Birth Cohort Study, and our scan of the policy landscape.
The roundtable session also gave us a better understanding of some of our research findings. While it appeared from our environmental scan that early childhood development policy in Saskatchewan is designed, funded and delivered in a coordinated manner, the actual situation "on the ground" is less coherent, with little coordination between departments and limited ways for community-level program people to participate in the policy process.
This lack of coherence between policy development at senior government levels and program implementation at the community level results in significant gaps or overlaps in service, especially for Aboriginal peoples. It is further complicated by intergovernmental conflict at the federal-provincial level over responsibility and financing of services for Aboriginal peoples, as well as the duplication of services for some targeted populations (e.g. single mothers with young children). In parts of the province, some groups have access to multiple programs delivered by different provincial or local agencies, while other identifiable groups suffer limited access.
Despite these frustrations, there was a clear sense from the participants that many initiatives underway in the province were making important differences in the lives of children. Some programs, particularly KidsFirst and Aboriginal Head Start, were seen as moving in the right direction and deserving of the increased support that had recently been announced in the provincial budget. In particular, participants felt that these kinds of programs were successful not only because of their specific content or mix of services, but because of the way in which communities were actively engaged in their design and implementation.
Our discussions also shed light on findings we thought were counterintuitive: that children's health appears to be worse in families receiving intermittent income assistance (off and on over eight years) than in those families receiving long-term assistance (three or more consecutive years). To the decision makers, such results made sense, as the long-term receipt of social assistance provided a stable, albeit meagre, level of support around which families could develop coping strategies. Moving in and out of the labour market made for increased income instability and higher levels of family stress, in turn leading to poor child health outcomes.
While we had hoped to develop a set of policy recommendations at the roundtable meeting, we realized that this was too ambitious a goal for the initial meeting. Instead, a more realistic goal is to maintain "space" for creative and ongoing dialogue between researchers and policy makers, so that policy recommendations are developed methodically, and at a time to coincide with the policy makers' readiness for uptake.
Research results, as complex and counterintuitive as they can sometimes be, are understood by a diverse audience and can be the impetus to engage a variety of people at a high level of discussion.
When engaging potential users of research findings, we learned that research results, as complex and counterintuitive as they can sometimes be, are understood by a diverse audience and can be the impetus to engage a variety of people at a high level of discussion. For useful dialogue to occur, however, there are a few prerequisites.
First, research results must be clear, timely and well-presented. We developed and circulated our research summary to the roundtable participants well in advance of the meeting. We took great care to highlight the questions that the research was addressing and used a variety of graphs and geo-coded maps to summarize and present our findings. Our audience found maps of the study areas presenting neighbourhood-level results particularly compelling.
The composition of the discussion group is a key determinant of its success.
Second, you need to carefully select participants, as the composition of the discussion group is a key determinant of its success. We strived to assemble a balanced mix of senior level policy analysts, program managers from various provincial departments, community-based advocates and university researchers. Many of the participants were familiar with the principal investigator and research team, and also knew each other. This familiarity creates a level of trust and comfort that encourages honest and deep discussion.
Third, you need to animate the dialogue with a few compelling discussion questions. Our questions were informed not only by our research results, but also by our review of early childhood development policy, providing good context for our results and ensuing discussion.
Conclusions and implications
This experience has confirmed to us the utility of stronger researcher-policy maker interactions. The nature of the study made it of particularly high interest to the policy makers and program personnel, many of whom requested to be both made aware of future work in this area, and to participate in future studies.
But a great deal of work remains to be done in terms of how we build and sustain linkages between researchers and policy makers. Sometimes the researcher-policy maker linkages need to be built on a case-by-case basis, as we did in this project. Other times, these linkages need to be channelled through existing advisory committees for policy making, such as Saskatchewan's intergovernmental committee on Human Services Integration Forum. In any event, we need to focus on making linkages earlier in the research process (so they can influence the nature of the research questions) and on using them to validate research findings in a manner that leads to both deepening existing analysis and pointing to directions for further research.
1. McIntosh, T., N. Muhajarine, and B. Klatt. 2004. Understanding the Policy Landscape of Early Childhood Development in Saskatchewan. Policy Report. Saskatchewan Population Health and Evaluation Research Unit.
2. Muhajarine, N., L. Vu, R. Labonte, L. Dodds, D. Fell, and G. Kephart. 2004. Community and Family Characteristics, Income Dynamics and Child Health Outcomes: Researching across the boundaries. Technical Report. Saskatchewan Population Health and Evaluation Research Unit.
Diane Davies, MSc, Social Program Evaluation Group, Queen's University
Will Boyce, PhD, Social Program Evaluation Group, Queen's University
The Canadian Adolescents at Risk Research Network (CAARRN) is an interdisciplinary research program focused on adolescent health. CAARRN's comprehensive knowledge translation strategy targets multiple audiences in both the health and education sectors, including researchers, policy makers, program developers, clinicians, community agencies, school boards, educators and young people themselves. CAARRN also incorporates a youth engagement component that enables research by and for youth. CAARRN's extensive knowledge translation activities have resulted in the uptake of research findings by a number of diverse organizations and the development of a national network of youth researchers and associated personnel.
Adolescence is a time of transition, involving physical, psychological, social and vocational changes. But very little attention has been paid to developing health policy based on current Canadian data for this important population group. In particular, research that illustrates the influence of school and neighbourhood settings, family and peer groups, coping skills and socio-demographic factors on health behaviours and outcomes has been underutilized in policy formulation.
The Canadian Adolescents at Risk Research Network (CAARRN)* is an interdisciplinary research program focused on adolescent health. It facilitates knowledge exchange, communication and collaboration among youth, researchers, policy makers and programmers to improve the quality of life of young people. Through studies of national and international data in seven key areas—bullying, sexual health, injuries, school culture, disability and chronic conditions, social capital and obesity and physical activity—CAARRN aims to better understand the complex interrelationships that have an impact on the health status and daily lives of adolescents, and to facilitate policy development in these areas. The knowledge developed through CAARRN, apart from its value as basic research, is enhancing our understanding of appropriate foci and timing for preventive interventions and the extent to which interventions must be specifically tailored to adolescent sub-populations.1
CAARRN involves researchers, policy experts and young people from Queen's University, the University of Toronto, Bloorview MacMillan Children's Centre, the Public Health Agency of Canada, the Canadian Education Association and the Centre of Excellence for Youth Engagement. The program is funded (2001-2006) by the Canadian Population Health Initiative (CPHI).
The KT initiative
CAARRN, in conjunction with CPHI, has developed a comprehensive knowledge translation (KT) strategy. The broad relevance of the adolescent health topics addressed in the CAARRN program means that many audiences in both the health and education sectors are targeted, including researchers, policy makers, program developers, clinicians, public health agencies, community agencies, government agencies, non-government organizations, school boards, educators and young people themselves. We view any meeting as a KT opportunity, and take dissemination products (usually fact sheets) to distribute among participants. We also engage our stakeholders to help us develop and disseminate our KT products.
Our targeted KT activities include:
- Peer-reviewed publications and academic conferences: We present our research findings in the traditional forums of academic journals and presentations at provincial, national and international academic conferences.
- Other publications: We produce research reports, an electronic bi-annual newsletter, research fact sheets and plain-language articles in local, provincial and national agency publications.
- Media relations: We have developed relationships with Queen's University and Canadian Institute for Health Information media representatives to facilitate press coverage of our research findings. For example, two recent publications—comparing overweight and obesity prevalence in school-aged youth from thirty-four countries, and the associations between being overweight or obese with bullying behaviours in school-aged children—received a great deal of media interest.
- Website: We develop and maintain a website containing program information, results and downloadable dissemination products.
- Listserv: We use a listserv to distribute findings, dissemination products and announcements pertaining to adolescent health research and policy to our array of stakeholders.
- Rapid response policy team: We have a rapid response team that responds quickly to issues that arise in the media or in the policy domain. For example, we provided advice to Health Canada on the Canadian Youth, Sexual Health and HIV/AIDS Study, and advice to local school boards on policy development regarding bullying.
- Network meetings: CAARRN network meetings establish research objectives and provide opportunities for collaboration, data sharing and identification of research gaps.
- Regional policy workshops: We host regional workshops to share our research findings with policy makers and researchers around the country and to develop the national network of youth health researchers.
- National policy conference: We host a national policy conference targeting local, provincial and federal decision makers in adolescent health, as well as youth organizations/advocates and researchers.
- Student educational activities: We have developed student activities for use in school settings based on adolescent health risk behaviour data and findings.
- Regional outreach/local engagement: We present our research findings to local schools and school boards, and develop targeted dissemination items upon request.
CAARRN also has a youth engagement component, which is separately funded by the Public Health Agency of Canada's Population Health Fund. It aims not only to facilitate research by youth and through youth organizations, but also to learn from youth and to model methods of working with youth. The Youth Engagement in Health Research and Policy project provides an opportunity for in-depth interaction with young people that involves skill development, basic research training and mutual learning in self-directed health research projects on topics of importance to youth themselves.
Each KT activity has an evaluation component. Meetings and workshops are evaluated with questionnaires and telephone interviews have been conducted with youth researchers to gather feedback on their experiences. Listserv member counts and the dissemination of CAARRN products are also monitored.
Results of the KT experience
CAARRN research findings have been used by a number of organizations, including the Canadian Mental Health Association for bullying prevention program development.
CAARRN research findings have been used by a number of organizations, including the Canadian Mental Health Association for bullying prevention program development, local school boards for bullying prevention policy development and the National Children's Alliance for a position paper on the need for a youth policy agenda in Canada. As well, Statistics Canada is using CAARRN data in its E-STAT program, an interactive teaching and learning tool for the education community, and the Public Health Agency of Canada's Division of Childhood and Adolescence utilizes CAARRN findings to inform program and policy development, as well as to monitor adolescent health behaviours.
Several of the Youth Engagement in Health Research and Policy projects, all of which are now complete, produced meaningful research that has been disseminated and utilized across the country. Research on street youth engagement and stereotypes has been well received by both peers and researchers, and has been requested by those working on the front lines with street youth in communities across Canada.
A national network, including existing regional networks in the Atlantic provinces, Ontario and British Columbia, has now been established, and data sharing agreements have been developed with academics and community organizations. In addition, stakeholders are now contacting CAARRN with their information needs, including requests for data files and tables. This is, in part, attributable to the long-term presence of CAARRN, which has established relationships and developed rapport with various communities and sectors.
Although our KT activities are still underway, several key lessons can be shared from our experiences to date.
KT products need to be developed with audience representatives to be most useful. Concise, reader-friendly materials work best. By far the most popular CAARRN products are research fact sheets. These targeted, one-page summaries of research findings on a particular topic appeal to all audiences—researchers, policy makers, program developers and young people. We have created a template from which researchers can quickly draft fact sheets (which are similar to structured abstracts, but more user-friendly and policy-relevant) from a finished journal article.
Promoting equal partnerships with adolescent health stakeholders has been an important aspect of CAARRN. The most effective and comprehensive KT events have been our regional policy workshops, which bring together researchers, local policy makers and program developers to discuss the implications of our research findings. We have often collaborated with other organizations and researchers to ensure a more successful and high profile event. For example, CAARRN has teamed up with researchers at Dalhousie University, the University of Waterloo and the McCreary Youth Foundation to host regional policy workshops across the country. Our evaluations indicate that participants find short presentations (fifteen minutes) and opportunities for group discussions to be the most useful aspects.
Engaging young people in their own research—from design, implementation and analysis, to KT, advocacy and participation in policy development—is a challenging and rewarding experience. A key indicator of success is the capacity building within young people themselves, with the lessons learned flowing into other aspects of their lives—school and work particularly.
We also learnt a lot about the evolutionary nature of youth participatory research, as opposed to traditional linear academic approaches. Research with youth offers many lessons and challenges, most of which derive from the complex nature of youth lives. The success of participatory action research has to be judged on more than the traditional markers of academic achievement. Youth empowerment is one of these new indicators:
"I like the process itself, you know, giving young people the skills they needed to do research on the issues we wanted. Having the opportunity to create your own project and do the research on something that you are interested in, and having the support for doing that, is awesome."
Conclusions and implications
While funding for CAARRN ceases in March 2006, it is hoped the network will continue, with established relationships and new collaboration between youth, researchers, policy makers and programmers aiming to improve the quality of life of young people. CAARRN will continue to support the initiative to develop a national youth policy agenda as a member of the National Children's Alliance.
* For more information about CAARRN, visit their website.
1. Boyce, W. 2004. Young people in Canada: Their health and well-being. Ottawa, ON: Health Canada.
Should we teach harm minimization to teenagers in school? The production and translation of controversial new knowledge in addictions
Christiane Poulin, MSc, FRCPC, Canada Research Chair in Population Health and Addictions, Dalhousie University
Jocelyn Nicholson, BEd, BScH, SCIDUA coordinator, Dalhousie University
Researchers from Dalhousie University partnered with the provincial Department of Health, regional Addictions Services, school boards and schools in Nova Scotia in a cooperative participatory research project (SCIDUA) to determine if harm minimization should be adopted as the basis of drug education for junior and senior high school students. Knowledge translation was an integral aspect of SCIDUA and appeared to result in policy change in two partner institutions. However, the success of the knowledge translation may have contributed to the early adoption of harm minimization policies, which failed to reflect some important but unexpected results of the completed research.
While harm minimization is increasingly being advocated as the basis of school-based drug education, there has been little empirical evidence as to its acceptability and effectiveness. We conducted an Integrated School- and Community-Based Demonstration Intervention Addressing Drug Use Among Adolescents (SCIDUA) to determine if harm minimization should be adopted as the basis of drug education for junior and senior high school students in Nova Scotia.1 SCIDUA was a partnership project funded by CIHR, the Nova Scotia Department of Health, Addictions Services in District Health Authorities 1, 2, 3 and 9, the Halifax Regional School Board, the Annapolis Valley Regional School Board, four schools and Dalhousie University. The main goal of SCIDUA was to decrease harmful involvement with, and harmful consequences of, alcohol, tobacco and other drugs. The SCIDUA fieldwork took place from 1998 to 2002.
The KT initiative
The need to allow and encourage open dialogue among students, teachers, administrators and parents about this controversial and untested new policy in drug education led us to use cooperative participatory research.2 In cooperative inquiry, people participating in the research are acknowledged as self-determining, and what they do and experience as part of the research must be determined by them to some significant degree.
School Community Steering Committees (SCSCs), comprising students, teachers, the school counsellor, an Addiction Services staff member and a SCIDUA project coordinator, met every two weeks during the school year. Several SCSCs also included parents, law enforcement professionals, local government and community groups. The SCSCs were responsible for creating and implementing drug education initiatives as they deemed fit in their respective schools. Interventions included activities such as public service announcements, student presentations, parent information sessions and teaching videos. A memorandum of understanding and advisory committees allowed for collaboration among the partner institutions, with the advisory committees meeting monthly during the school year. Thus, the overarching process by which the four schools arrived at their interventions was the same, but the actual interventions and mix of interventions were unique to each school.
Our formal evaluation of the acceptability of harm minimization drug education was based on an analysis of 491 documents generated by about 150 informants engaged in the SCSCs or arising from their activities, from 1998 to 2002. Our formal evaluation of effectiveness was based on a survey of 1,117 and 849 students in the intervention schools, compared with 3,755 and 4,247 students in the rest of the province, in 1998 and 2002 respectively.
These evaluations revealed different results among senior and junior high school students. In senior high school, there was decreased prevalence in specific risks and negative consequences of alcohol and cannabis use, and qualitative evidence that harm minimization was acceptable. In junior high school, however, SCIDUA revealed that harm minimization was not acceptable and had not been implemented to any extent, and that there was a failure to demonstrate improvement in the prevalence of specific risks and harms of substance use. We concluded that school-based harm minimization may be acceptable and effective in senior high schools, but may not be acceptable or effective in junior high schools.
Results of the KT experience
Due to its cooperative participatory research design, knowledge translation (KT) was an integral aspect of SCIDUA from its inception. KT activities were extensive and occurred both during and after the fieldwork. Representatives of the institutional partners actively participated and received ongoing documentation through minutes, presentations, updates, summaries of activities, formal reports and local workshops. Based on the high level of engagement of many stakeholders at many levels throughout four years of fieldwork, resulting in a total of 3,000 files of qualitative data, it is clear that the participatory research process of SCIDUA was successful.
However, how do we know that the KT activities of SCIDUA, which were an integral part of the participatory research, actually resulted in policy development in health or education?
Two policy documents developed by partner institutions during their participation in SCIDUA suggest that our KT activities were successful in influencing policy. The first document, which pertains to student substance use policy, was published in November 2002 by the Nova Scotia Department of Health and indicates that their "advocated approach" to drug education and policy is harm minimization, and that the "traditional approach" of zero tolerance is "not a good idea".3 To our knowledge, this is the first document from the provincial Addictions Services that explicitly identifies harm minimization as the preferred option for drug education. The provincial Addictions Services was an early partner in SCIDUA for the express purpose of answering this policy question, and references in the document point to individuals and agencies involved in SCIDUA.
The second document is a school board substance use policy which adopts the language of harm minimization.4 The policy was created during the last two years of fieldwork by one of the school boards involved in SCIDUA, through a committee whose membership comprised, among others, several members of a SCIDUA advisory committee.
While these policies may be viewed at least in part as products of successful KT activities, they were developed before definitive conclusions could be drawn from SCIDUA as a research project. In fact, they were articulated before the sharing of preliminary findings in October 2002 about the doubtful effectiveness of harm minimization drug education targeting younger adolescents.
It appears that the success of SCIDUA as a participatory project may have been misconstrued by some participants as acceptability of the policy direction itself; that is, as acceptability of harm minimization drug education.
So why did two partner institutions involved in SCIDUA seemingly adopt harm minimization in the absence of actual evidence about its effectiveness as a basis for drug education? Here, we distinguish between support for the participatory process of SCIDUA versus support for the policy being researched through the participatory process. Based on our comprehensive analysis of the qualitative data collected during SCIDUA, it appears that the success of SCIDUA as a participatory project may have been misconstrued by some participants as acceptability of the policy direction itself; that is, as acceptability of harm minimization drug education.
A second reason that harm minimization may have been endorsed early is that the rich ongoing reporting and evaluation activities of SCIDUA could have been interpreted by some partners to be conclusive evidence supporting harm minimization. For example, midway through the fieldwork, an interim process evaluation conducted by an external team indicated that progress was being made towards the main goal of SCIDUA. The external team concluded that "a number of adolescents report less frequent and/or less risky use of substances. ... Some stakeholders are committed to the concept and implementation of a harm minimization approach." Such statements may have been viewed by some SCIDUA partners as proof that harm minimization drug education was acceptable and effective. In reality, the external team's conclusions were based only on a site visit and on the opinions of a small number of people at one point in time, without the benefit of the pre/post outcomes evaluation, which was a key feature of the formal evaluation. Similarly, two workshops conducted just before the end of the fieldwork appear to have left some participants with the impression that harm minimization was conclusively known to be the correct approach to school-based drug education. In fact, the presenters at those workshops emphasized that the outcomes evaluation had yet to be done and that preliminary information was being shared at that time because the school year and fieldwork were coming to a close.
A final reason that harm minimization may have been endorsed early is that empirical evidence is not in fact a prerequisite for policy making. In the case of SCIDUA, our conclusion that one size does not fit all for junior and senior students was new and unexpected: the international literature supporting harm minimization drug education has tended to not distinguish between older and younger adolescents. Clearly, this particular finding of SCIDUA presents a challenge from the perspective of institutions and individuals in either health or education who wish to opt for a single drug education and policy appropriate for all students, spanning primary to grade twelve.
Conclusions and implications
Early or ongoing KT can pre-empt two essential steps of quality control in science.
We have come to understand that a novel approach like harm minimization can be very seductive to health and education policy makers, particularly if it seems to be a panacea to as difficult a problem as adolescent substance use. Thus, an important lesson learned from SCIDUA is that the participatory process, while helpful in creating a groundswell of commitment to a research process and actively engaging stakeholders in ongoing KT, also risks the premature adoption of inconclusive research findings, some of which may ultimately prove to be wrong-headed. In effect, early or ongoing KT can pre-empt two essential steps of quality control in science: the use of comprehensive rather than selective information and the performance of a dispassionate peer review of the research findings. KT must, therefore, incorporate mechanisms for the users of research to be invited to re-visit findings and issues. In the case of SCIDUA, KT activities have been planned past the end of the analysis phase, with the dissemination of the final results targeting SCIDUA partners as well as a broader audience.
KT must incorporate mechanisms for the users of research to be invited to re-visit findings and issues.
1. Poulin, C., and J. Nicholson. Forthcoming. Should harm minimization as an approach to adolescent substance use be embraced by junior and senior high schools? Empirical evidence from "An Integrated School- and Community-based Demonstration Intervention Addressing Drug Use Among Adolescents (SCIDUA)". International Journal of Drug Policy.
2. Reason, P. 1994. Three approaches to participative inquiry. In Handbook of qualitative research, ed. N. K. Denzin and Y. S. Lincoln, 324-39. Thousand Oaks, CA: Sage Publications.
3. Nova Scotia Department of Health, Integrated Primary/Population Health Branch, Addictions Services. 2002. When drugs come to school: A resource manual for student substance use and school-based policy development. Halifax, NS: Nova Scotia Department of Health.
4. Annapolis Valley Regional School Board. Board Policy - BP 402.9: Substance Abuse.
- Date modified: