This section reports on the findings of the document review, key informant interviews, file review, case studies, and stakeholder consultation. The findings are organized according to the issues raised by key informants.
Through the incorporation of community-based HIV/AIDS education, program funding for CBOs, and community-based research programming, the FI emphasizes community action to address the HIV/AIDS epidemic, which remains a major public health issue. CIHR's HIV/AIDS CBR Program is one of the component programs of the FI. Its two funding streams and six funding tools support the development of partnerships, research capacity, and research proposals; the completion of research projects; and the dissemination of research results. The federal government remains committed to addressing HIV/AIDS in Canada, and, through the FI, increased HIV/AIDS funding from $42.2 million annually in 2003-2004 to $84.4 million annually in 2008-2009.
Key informants believe that the HIV/AIDS CBR Program is a flagship program that incorporates the policy direction of the FI. It encourages partnerships between communities, academia, and policymakers; involves people living with HIV and their communities in responding to the epidemic; builds research capacity in communities; and gives communities the evidence they need to address the issues they are facing. Key informants and participants in the stakeholder consultation indicated that the program supports community involvement in all stages of research, from defining the research question, designing the methodology, collecting and analyzing the data, and disseminating the results. They went on to explain that once the research is complete, communities can use the results to address their needs, make improvements to the services they deliver, advocate for policy changes and program development, and lobby for HIV/AIDS treatment and prevention funding. Some Aboriginal stream participants also spoke about how research has the capacity to renew spirituality and bring healing to community. Although key informants and participants in the stakeholder consultation support the program, several indicated that additional funding resources are needed to build the bridges and develop the partnerships required to influence policy change in government.
To ensure the research conducted through the program is applicable to community needs, RFAs do not specify criteria for research topics; they simply require the proposed research to address issues related to HIV/AIDS and clearly demonstrate the relevance to and anticipated impact on communities. Given the flexibility in eligible research topics and the involvement of communities in defining and conducting the research, key informants believe that, in principle, the program is relevant to the needs of communities. However, they cautioned that some of the program requirements (as discussed in Sections 3.4 and 3.5) can make it difficult for communities to access the program or have a sense of ownership over it.
Key informants and participants in the stakeholder consultation believe that academic institutions and large, established AIDS service organizations (ASOs) are well aware of CIHR's HIV/AIDS CBR Program, but they also commented that the program's reach is limited. They pointed to a lack of awareness about the program among small and rural organizations; organizations whose primary mandate is not HIV/AIDS, but who address some HIV/AIDS needs (e.g., organizations addressing co-infection and determinants of health); and academic departments engaged in cross-disciplines. Additionally, Aboriginal stream participants in the stakeholder consultation indicated that a broader range of community-based organizations (CBOs) may apply to the program if CIHR made the application process less onerous and provided applicants with more technical assistance (See section 3.5.1).
Current CIHR promotion efforts tend to reinforce awareness among the existing HIV/AIDS CBR community. For example, CIHR posts program information and RFAs on its website; emails communiqués and RFAs to previous and potential applicants, AIDS service organizations (ASOs), and academia; and attends and makes presentations at HIV/AIDS conferences. It also provides funding to CBRF's, who promote CBR (through word of mouth, emails, newsletters, workshops, etc.) and are available to help organizations apply for CIHR and other funding.
Key informants and participants in the stakeholder consultation support CIHR's current promotional activities and would like these to continue. However, they also offered several suggestions for supplemental CIHR promotional activities:
Regardless of the program promotion methods used, key informants stressed that it is imperative to have continuous and multiple exposure as there is high staff turnover in community organizations.
Some key informants and general stream participants in the stakeholder consultation cautioned against over-promoting the program. Key informants suggested that heightened awareness of the program could increase application pressure and drive down the application success rate. General stream participants indicated that prior to expanding its promotional efforts, CIHR should survey CBOs to determine their interest in participating in research; CIHR could then use the resulting information to target its promotional activities.
While promotion can work to expand the range of program applicants, Aboriginal stream participants in the stakeholder consultation cautioned that even if organizations are aware of the program and are interested in engaging in research, resource limitations may prevent them from becoming involved. Participants also commented that there is a need for capacity-building on both the community and academic side as CBOs have limited time to engage in research because their first priority is service delivery and there are not enough academics to respond to community need for research.
Key informants raised another issue related to the reach of the program. Currently, there is the perception that some organizations dominate a large portion of the available funding and that a relatively small number of organizations are participating in the program. Some suggested that this may prevent other organizations from applying5 to the program because they see a single organization as the primary researcher for their community. Key informants asserted that this effectively diminishes the competitive aspect of program.
This section reviews the effectiveness of the six funding tools: Community-Based Research Facilitators, capacity-building workshops, Masters Awards, Doctoral Awards, catalyst grants, and operating grants. This suite of funding tools supports a natural progression of funding from capacity- and partnership-building to proposal development to full research projects; however, there is minimal evidence suggesting that the majority of applicants navigate through the complete continuum of funding sources. Some first-time applicants may be sophisticated enough to apply for an operating grant. Other applicants may be just beginning their foray into research and therefore may require some time to continue to build their research capacity before engaging in a research project that could benefit from CIHR funding. While building their research capacity, they may engage in small internal projects that do not require an external funding source. Therefore, sufficient time may not have passed for them to have progressed through the suite of funding tools.
3.3.1 Catalyst grants and operating grants
Catalyst grants and operating grants are traditional funding tools, which are well-known, especially in academic circles.
Generally, key informants did not have any concerns with these funding tools, aside from some comments that these tools seem to be directed to academia. This may be the result of misperception among a few key informants, as the projects implemented through these tools should involve partnerships between communities and academics. Nonetheless, Aboriginal stream participants in the stakeholder consultation said that the funding caps for operating grants need to be increased to accommodate cultural traditions and protocols. These resources may be used for travelling to communities, holding feasts, and recognizing the time and voice that community members give to the research. Participants also mentioned it would be helpful if they could hire someone to do the research or get a work release stipend for an existing staff person.6
As discussed in Sections 3.5.1 and 3.5.2, some key informants suggested that the application and merit review process for catalyst grants is too complex.
3.3.2 Capacity-building workshops
Funding for capacity-building workshops is available to community and academic researchers to increase basic understanding of CBR principles and practices. While key informants did not have any concerns about the design of this funding tool, some did note that it is not well-subscribed to. Key informants and participants in the stakeholder consultation were not able to identify what factors may be contributing to low uptake levels, although some speculated that it might reflect the high demands of the application and merit review processes. Some thought that CBOs may not be using the tool because they already have research experience. Others questioned whether the mandate of this tool was broad enough and suggested that it should include provisions to cover the costs of attending conferences. Key informants and consultation participants agreed that the usefulness of this tool should be further investigated.
3.3.3 Masters and Doctoral awards
Masters Awards and Doctoral Awards are intended to build CBR capacity in academic institutions, which will increase the availability of academic partners to work with community organizations on research projects. However, some key informants do not recognize this as an intended benefit of this funding tool; a few key informants referred to it as a scholarship program for students that has no benefits for or accountability to the community.
Participants in the stakeholder consultation indicated that the three-year funding period for Doctoral Awards is too short as it takes most students five or six years to complete their degrees. Additionally, some participants believe that a lower amount of funding is available to doctoral students through the HIV/AIDS CBR Program than other CIHR programs.7
A minority of key informants expressed interest in similar awards for summer students, undergraduate students, honours students, or community investigators.
3.3.4 Community-based Research Facilitators
Objective
The Community-based Research Facilitators (CBRFs, formerly named Research Technical Assistants - RTAs) are a resource for communities and academics interested in engaging in and conducting CBR. According to CBRFs and other key informants, CBRF's primary role is not to promote CIHR's HIV/AIDS CBR Program. Rather, it is to promote CBR in general and support the preparation of applications to CIHR and other funding sources. They also help groups understand the role they can play in research, identifying research opportunities, developing the partnerships and capacity needed to conduct research projects, assisting with the development of proposals and projects, and supporting the dissemination and application of research results. In essence, they provide stakeholders with whatever assistance they require to engage in HIV/AIDS CBR.
Operating structure
CBRFs are funded through a grant program wherein CIHR provides operating funds for a variety of CBOs to employ a CBRF resource for a given geographic region. This arrangement is not well-known or understood by stakeholders. Participants in the stakeholder consultation identified the need for more clarification about the role of CBRFs and for greater communication between CIHR, the host organization, and the CBRF.
In interviews, CBRFs reported that the structure within which they operate can make it difficult for them to perform their job. Because a Principal Investigator (PI) in their host organization holds the grant, the CBRFs do not feel that they have autonomy. As a result, it can be difficult for them to set priorities and develop work plans that are relevant to the communities they are serving, particularly if the PI supervising them does not understand research. They also reported that conflicts between CIHR's mandate and their host organization's mandate can interfere with the range of activities they can undertake as well as the type of research projects they can support. In some cases, CBRFs feel as if the host organization dominates their time and resources. Participants in the stakeholder consultation echoed these concerns and suggested that when a CBRF is funded, an MOU should be signed between CIHR and the host organization, explaining the relationship between CIHR and the organization, and outlining the roles and responsibilities of the CBRF.
Monitoring the performance of CBRFs can be difficult for CIHR since the CBRFs are housed at and supervised by CBOs. Currently, CBRFs provide CIHR with a work plan and annual progress reports. Some key informants suggested evaluating the performance of CBRFs with a 360o review, which involves the community in assessing a CBRF's performance.
Interaction with CIHR and other CBRFs
Due to the nature of the granting process, CBRFs have fairly limited interaction with CIHR. Prior to beginning their role as a CBRF, they do not receive any training from CIHR; however, no key informants identified this as a concern. Aside from submitting the required reports to CIHR, they attend an annual meeting in Ottawa. Key informants noted that CIHR sets the agenda for this meeting, which mainly serves to update CIHR on CBRF activities. While this meeting does enable CBRFs to provide feedback on the program, they would like to have more influence over the design of the program.
CBRFs expressed the desire to expand the scope of the annual meeting. Prior to meeting with CIHR, they would like the opportunity to have a face-to-face peer meeting to discuss issues they might want to raise with CIHR, debrief about the work they are doing in the community, and share best practices and lessons learned. Some also suggested meeting biannually and perhaps holding one of those meeting outside of Ottawa so CIHR can see for themselves the work that is being done in the community.
CBRFs said that they are supportive of each other. They hold bimonthly conference calls and email regularly. Some have also worked collaboratively on conference presentations and other projects.
Activities
CBRFs reported that since CIHR does not provide them with guidelines on how much time to spend on various activities, this gives them the freedom and flexibility needed to ensure that they provide communities with supports that are relevant to their needs. CBRFs engage in a wide range of activities such as:
CBRFs reported using a variety of methods and/or media to undertake these activities. For example, they may conduct community presentations, develop project websites or place postings on agency websites, prepare articles for newsletters or newspapers, or host podcasts/interactive cafés. CBRFs tailor their assistance to the level of the particular community; some communities may require assistance with basic needs assessment whereas others may need assistance with large, complex projects involving multiple partners.
CBRFs have different ways of connecting with their local communities. Some of the CBRFs said that they actively pursue contact with organizations while others said they wait for organizations to contact them. Additionally, some of CBRFs reported working with any organization in their community, whereas others noted only working with organizations who are members of their host organization's network of partners (although they will work with external organizations that are collaborating with member organizations). Some CBRFs primarily focus on communities and others work with both the community and academia. For example, from the case study with the CBRF for Quebec, it was estimated that 60% of resources were used to develop partnerships with university researchers and the remaining 40% were used to work with members of the host organization.
Participants in the stakeholder consultation stressed that the roles and responsibilities of CBRFs cannot be expanded unless the total number of CBRFs available across the country is increased.
Effectiveness of the CBRF tool
Key informants and participants in the stakeholder consultation have the perception that community and academia have limited awareness of CBRFs. Moreover, some key informants suggested that CIHR should offer a mentorship program to help communities engage in CBR, which is the precisely what the CBRF tool is intended to do. Nonetheless, several program applicants mentioned working with a CBRF on their application or to build research capacity. Additionally, participants in the stakeholder consultation suggested that all RFAs should introduce the CBRFs, describe their role, and provide information on how to contact them.
Many key informants and participants in the CBRF case study indicated that the effectiveness of CBRFs relates directly to the capacity of the individual in the position. They explained that effective CBRFs have strong connections to and are well-respected by communities and academia. Some key informants suggested that CBRFs need to do a better job of fostering partnerships and acting as a liaison between communities, academia, and CIHR. They also said that CBRFs need to provide organizations with more assistance with proposal writing, help them understand the needs in their community and formulate research questions, and facilitate practice improvement based on research results.
Key informants acknowledged that the capacity and effectiveness of CBRFs is also related to the characteristics of the province and/or region they are serving; some rural and/or remote communities may be difficult to reach and there may be a limited pool of researchers for communities to partner with. CBRFs noted that it can be hard for them to provide complete coverage of their province because many of their clients prefer to meet in person; this is especially true in Aboriginal communities where it is important to respect oral traditions.
The extent to which CBRFs support the development of CIHR proposals is unclear. Part of the reason for this is that applicants are not required to inform CIHR of whether they worked with a CBRF, and the application forms do not specifically request this information. The extent to which CBRF's capacity-building activities lead to CBR is also unclear as they do not systematically collect information about or report on the research activities that communities engage in regardless of whether it is internal research, CIHR-funded research, or research funded through another program.
Currently funding is available for five general stream CBRFs and two Aboriginal stream CBRFs. Aboriginal stream participants in the stakeholder consultation strongly believe that, to properly serve the Aboriginal community, the Aboriginal stream needs at least four CBRFs. The groups also identified a need to promote CBRFs so that more CBOs are aware of the services they can provide.
Challenges
Key informants reported that there has been a lot of turnover in CBRFs. They explained that this reduces the effectiveness of the funding tool because each time a new CBRF starts, they must gain the community's trust and rebuild connections with potential research partners.
CBRFs suggested that the staff turnover results from structural problems with the funding tool. One problem they mentioned is that CBRFs cannot act as co-investigators on CIHR applications and, therefore, do not receive formal recognition for the work they do. CBRFs explained that this can hinder an individual's career development. Another problem they identified is that the funding period is only two years. They said that it takes a few months for CBRFs to determine the needs of their community, develop a work strategy, and build relationships. They then have about one year to work with communities. After that they have to begin reapplying for CIHR funding and/or start looking for another position for themselves. CBRFs suggested that a five-year funding period would be adequate; if this funding extension was granted, they would be willing to prepare more detailed work plans and progress reports.
Other challenges that CBRFs mentioned include not having access to research databases and libraries and only being able to assist a limited number of organizations with funding applications around program deadlines.
Some key informants and general stream participants in the stakeholder consultation indicated that the following adjustments to CBRF funding could reduce turnover rates:
3.3.5 General comments
To help communities address the needs they are facing, some participants in the stakeholder consultation said that CIHR needs to raise awareness of the available suite of funding tools. While they agreed that the current suite of tools is appropriate, they are concerned that some of the tools are underused. For example, participants suggested that the capacity-building workshop tool is the least intimidating and therefore is the one that CBOs with little research experience should be encouraged to apply to first.
Participants cautioned that prior to revising and/or expanding the suite of available tools, attempts should be made to raise awareness of the existing tools, and their usefulness should be assessed. They also stressed that any modifications to the funding tools should not add to the complexity of the program.
Some of the additional types of funding that participants said would be useful are:
Aboriginal stream participants in the stakeholder consultation also expressed concerned about the lapsed funding. Additionally, some of the funding for the general stream has also lapsed. Given the immediacy of the HIV/AIDS problem and the research needs in the communities, it was considered imperative that CIHR explore ways to ensure that funds do not lapse.
Projects funded through the HIV/AIDS CBR must demonstrate strong scientific merit and potential impact on the community. To balance these two aspects, the CBR teams receiving funding through this program involve partnerships between communities and academia. For a partnership to be successful, each party must benefit from and be committed to the relationship. Key informants said that communities benefit from partnerships with academic researchers by receiving advice on design and methodology and obtaining access to research ethics boards (REBs). Academic researchers benefit from the partnerships by obtaining insight from communities into the issues they are facing, the questions that need to be asked, and how to reach the individuals that will be impacted by the research.
CBR principles encourage equal participation of community and academia throughout the research project. While key informants agree that it is desirable to have equal partnerships, they suggested that it is more appropriate to have equitable partnerships as each partner brings different strengths, knowledge, and experience to the project. In an equitable partnership, everyone gives what they are able and takes what they need.
Key informants said that prior to beginning a project, it is important to determine how involved each partner wants to be. They suggested that sometimes communities may not want to be involved throughout the process if it will be burdensome. Therefore, each partner's level of effort can vary by project; for some projects there will be complete cooperation, for others the community will define the research question, the academic partner will conduct research, and both will contribute to disseminating the results. Key informants indicated that it can be helpful to establish a memorandum of understanding at the start of the project, which defines the roles and responsibilities of each partner.
Key informants cautioned that it can be difficult for communities and academia to establish effective partnerships. One reason for this is that each party may bring different priorities to the partnership. Key informants explained that often the community wants to gather information at the community-level and tends to want to conduct a small project. Their priority is service delivery; they are interested in research when it can assist their service delivery role. Conversely, academics tend to want to engage in publishable research and, therefore, are often interested in large-scale projects. Another reason that community and academic partnerships can be difficult to manage is that communities sometimes operate under different timelines than their academic partners because of the competing pressures on their time and resources. This can create tensions with academic partners and can reduce the community's interest in the project.
Key informants noted that the community can sometimes resent having to partner with an academic researcher. They can feel coerced to partner with an academic researcher because they need a PI who meets CIHR criteria, they need the academic institution to hold CIHR funds (because the community finds the CIHR process to become an eligible institution difficult), and they need to access the institution's REB. These concerns can make the community feel as if they are not an equal partner in the research. Key informants explained that this can be viewed by communities as a loss of control over the research and can raise questions as to who owns the intellectual property associated with the research.
The contributions and participation of both academia and communities throughout the research process helps ensure that funded projects incorporate traditional and community-based knowledge. Recognizing the value of traditional and community-based knowledge, Aboriginal stream participants in the stakeholder consultation emphasized the need to strengthen the statement about partnerships in the RFA. They also suggested including a requirement in the RFA for a community engagement plan, which would explain how the applicants will engage the community throughout the project.
3.4.1 Challenges to community involvement in CBR
Key informants identified several challenges to community involvement in CBR and CIHR's HIV/AIDS CBR Program. CIHR can address some of these challenges through its promotional activities.
Although not enforced prior to the evaluation, CIHR has enforced its Policy on Complete Applications. Under this policy, incomplete applications are subject to rejection. Key informants who were aware of this policy suggested that if it is enforced, organizations whose applications are rejected for reasons such as missing signatures may be reluctant to resubmit their application in the next funding cycle.
This section presents evaluation findings on the administrative processes associated with the program, including the application process, merit review, ethics review, and holding CIHR funds.
3.5.1 Application process
Applying to the HIV/AIDS CBR Program is a two-stage process. Applicants must first register with CIHR and then they must submit an application for funding.8 Key informants do not understand why both steps are needed and said that the application process is lengthy; they reported that it can take three to four months to complete an application.
Generally, applicants and other key informants believe that the application process is difficult and onerous, especially for communities. They said that the forms are difficult to complete and the web forms are difficult to navigate and require a high level of computer literacy. They indicated that the web forms involve multiple layers, ask for similar information in different locations, and are unclear about when attachments are needed. Although they acknowledged that a PDF checklist of the attachments required is available, it is not easy to find. Key informants also mentioned that collecting all of the signatures required for the application is logistically challenging and that communities do not always understand whose signatures are required. The file review found that four of the 20 funded applicant files reviewed had missing or incorrect signatures.9 Some key informants mentioned needing to discuss numerous questions about the application with CIHR. They reported that CIHR was helpful; however, the need to contact CIHR illustrates that the process is difficult.
In addition to seeking assistance with the application process from CIHR, some applicants will consult with CBRFs. For example, Pauktuutit Inuit Women of Canada worked with a CBRF to complete its application for a catalyst grant. Specifically, the CBRF acted as a liaison between the community organization and the academic institution and assisted with writing the application.
Participants in the stakeholder consultation recommended the following improvements to simplify and streamline the application process:
The most common complaint key informants had about the application process relates to the CVs. Seven of the 20 funded applicant files reviewed as part of the file review were missing CVs or had CVs that were in the incorrect format. While key informants recognized that CIHR has created a community CV in an attempt to make the format more relevant to and simpler for communities to complete they said that further modifications to the form are needed. Key informants noted that the community CV is still too academically-oriented and asks for qualifications that communities do not have (e.g., degrees, publications, students supervised).10 11 Consequently, community CVs often have several blank sections, which key informants said makes communities feel unqualified to apply for the program. They also said that it makes communities think that CIHR does not value their knowledge and experience.
Participants in the stakeholder consultation offered several suggestions for modifications to the common and/or community CV, which may help applicants demonstrate their full range of experience, respect community experience, and ensure balance between community and academic knowledge.
Key informants suggested that the community CV simply ask for a biography of the applicants, which would allow them to describe their experience in the community. They also suggested asking communities what steps they have taken to prepare for the research such as development work undertaken and community readiness for intervention.
Key informants believe that communities are at a disadvantage compared to academics when applying to the program. They mentioned that academics have more research experience and can write better proposals than communities. They said that applications do not use language that communities understand and noted that communities often do not possess the vocabulary required to write a strong methodology. For these reasons, some key informants believe that applications with PIs who are academics are more likely to be funded than those with PIs from the community who are not academics. From the file review, the proposed PI for 11 of the 20 funded files and 6 of the 10 non-funded files reviewed was an academic. It appears that academic PIs are more likely to lead catalyst grants, operating grants, and student awards, whereas community PIs are more likely to lead capacity-building workshops and applications for CBRFs.
It is not uncommon for applicants to be denied the first time they apply to the program. Key informants reported that applicants would appreciate the opportunity to have discussions with CIHR around any omissions on their applications. They suggested that this would make the application process less daunting for communities and may lead to more reapplications from those who were initially denied funding. Aboriginal stream participants in the stakeholder consultation also suggested that the CIHR should have a mentoring process for preparing an application and provide an internal review of applications that would allow for feedback and revisions before sending the applications to the Merit Review Committee.
A minority of key informants thought the application process should not be changed, but instead should be better explained to communities. They suggested that CIHR provide a clear set of guidelines for applicants and grant holders that identifies all of the required steps and suggested timelines for completion. Some also suggested posting an example of a completed application on CIHR's website. A few community applicants and consultation participants also expressed the desire to have the opportunity to verbally present their proposal to CIHR, which would give them the opportunity to explain the community- and/or Aboriginal-based realities associated with the proposal.
General stream participants in the stakeholder consultations also indicated that including the following information in all RFAs will help clarify what types of research are eligible for funding and how the research results can be used:
Key informants agreed that the application process is time-consuming, regardless of what funding tool is being applied for. One complaint was that even the catalyst grants, which provide a relatively small amount of funding for the development of a research proposal, are difficult to apply for. Key informants suggested reformatting the application for catalyst grants into a one-page application that identifies the target community and the research question being addressed, and introduces the research team.
Some key informants reported that the program timelines can prevent communities from applying. They said that:
3.5.2 Merit Review
Key informants appreciate that there is a separate Merit Review Committee for each funding stream. They also commended CIHR for including a mix of community and academic members on the committees. Although the Merit Review Committees comprise an equal number of community and academic members, some general stream participants in the stakeholder consultation were concerned that the committee does not have appropriate community representation.
Overall, members of the Merit Review Committees and other key informants believe that the community has clear voice in the review of applications. Nonetheless general stream participants in the stakeholder consultation noted that committee meetings often use academic language, which can intimidate some community representatives and diminish the strength of their voice. They suggested that additional training about merit review processes, especially for community members, will help ensure that community representation is not lost.
Aboriginal stream participants in the stakeholder consultation and some key informants suggested that the Aboriginal Merit Review Committee should have First Nations, Métis, and Inuit members as well as Elders. They specified that Elders should be additional members of the Committee and should not be seen as replacing community members. Elder input into the merit review process is critical, but participants stressed that the Elder should determine the level and type of participation - from full review to offering ceremonies, guidance, and advice. Some participants commented that the Elder should be familiar with the community and that Aboriginal applicants should be asked if they want an Elder to participate the proceedings. Following Aboriginal traditions, Elders should be presented with tobacco, cloth, and honoraria for their contributions.
Aboriginal stream participants in the stakeholder consultation also commented that there is not a lot of capacity on the Merit Review Committee to understand the Aboriginal community. Although Elders on the Committee may offer cultural ceremonies and assist with understanding the historical and cultural contexts, participants stressed that the Committee should not just depend on the Elder to provide this understanding. Instead, they said there should be capacity-building for the Committee such as having an orientation before each meeting that would include interactive discussions about Aboriginal history and culture.
General stream participants in the stakeholder consultation had limited information about how members of the Merit Review Committee value and assess the work, volunteer, and life experiences that community members bring to CBR teams. Participants expressed a desire for CIHR to provide the Merit Review Committee with additional guidance and training on how to value and assess the various components of applicants' experience. Currently, participants do not believe that volunteer and life experience are valued as highly as work experience.
Participants in the stakeholder consultation believe that the Merit Review Committee should be assessing the strength of the CBR team as a whole, including the CBO, community members, and academic members.
Key informants also identified the following challenges with the Merit Review process.
Key informants noted that it can be difficult to recruit members for the committees. They mentioned that there is a small pool of researchers to recruit from. They said it can be especially challenging to find researchers who are of Aboriginal descent and can speak French. Key informants also reported that it can be difficult to find community members who have CBR experience. To address these challenges, key informants suggested developing a mentorship and training program. They indicated that the program could move members through different stages of the review process; for example, they could move through the following progression: basic training, observer, reviewer, reader style commenter, full commenter, and full reviewer.
Most key informants believe that the Merit Review Committee provides applicants with helpful feedback. From the file review of non-funded applications, reviewers made comments about the relevance of the proposed research and linkages to the community, the proposed methods of recruiting participants, the quality of partnerships, the extent to which the project incorporates capacity-building, the proposed approach to data analysis, and the plans for disseminating the results of the research. Key informants noted some occasions of poor feedback, such as a criticism that has not helped applicants understand the committee's decision, and comments directed at factors beyond researchers' control. One key informant reported that the committees do not provide feedback on applicants that score below 3; this lack of feedback may discourage applicants from reapplying.12
Participants in the stakeholder consultation stressed the importance of having consistency within the Merit Review Committee. They indicated that turnover in committee members can be problematic, particularly when applicants re-apply to the program. Because new members are not familiar with previous applications, they do not know the reasons why these applications were rejected and are unfamiliar with any suggestions for improvements provided by former members of the Committee. For this reason, participants would like to see committee members serve three-year terms.
3.5.3 Meeting conditions for release of funds
Two of the key requirements that approved applicants must meet prior to the release of the funding are to obtain ethics certification and to demonstrate eligibility to hold CIHR funds.
Ethics certification
Most commonly, projects obtain ethics certification from a university-based research ethics board. In most cases, for communities to access these REBs, they have to partner with a university. Key informants identified several concerns with using university-based REBs to review CBR projects.
Key informants noted that few organizations outside of universities offer ethics certification; those who do may only make it available to a limited client base. Some of these REBs are only available to applicants to a particular funding source or organizations located within the same province. While some REBs may make their services available to anyone, some of the ones identified through interviews operate on a volunteer basis and therefore cannot accommodate a large number of applications.
Key informants agreed that they would like to see a community-based REB for CBR and suggested that CIHR could create its own REB. They noted that CIHR is involved in ongoing consultations around basic principles of ethics review and is trying to establish more standardized criteria and forms.
Holding CIHR funds
CIHR encourages CBOs interested in holding funds to call CIHR to discuss their eligibility before applying to the program. Key informants noted that it is difficult for communities to meet the eligibility requirements to hold CIHR funds. If CBO wants to hold the funds, it needs to demonstrate to CIHR that it has structures in place to manage the funds and that is has research as a mandate. Key informants reported that CBOs who have successfully gone through the process spent more than one year demonstrating that they meet all of the requirements. They asserted that making it difficult for CBOs to hold funds contradicts CBR principles.
Key informants noted that some organizations have long history of holding federal funds but still have difficulty getting approved to hold CIHR funds. They reported that sometimes organizations have held much larger amounts of contribution funding from the federal government than the amount of grant funding they are seeking from CIHR. Key informants suggested that instead of having to apply to hold CIHR funds, perhaps the organizations could give CIHR references who can speak to their ability to manage funds.
3.5.4 Reporting
Applicants must submit a final report within six months of the conclusion of their project. CIHR does not specify a format for these reports. Submitted reports can range in length from 3 pages to 150 pages. The file review illustrated that, because of the variation in the reports, it can be difficult to determine whether the projects have met their stated objectives and what outcomes they have achieved. Often this information is not explicitly stated, but rather is buried in descriptions of project activities. Additionally, the projects may have not achieved longer-term outcomes at the time the reports are due.
CIHR has not determined how it intends to use the information included in final reports. Key informants suggested that the Steering Committee can help create a performance measurement strategy and determine how CIHR can best use the information it receives.
Key informants did not have any concerns with the reporting requirements.
CIHR's HIV/AIDS CBR Program is intended to develop the capacity of communities and academia to conduct CBR research, establish effective and long-standing partnerships between communities and academia, and produce research results that communities can use to address issues they are facing. For example, the catalyst grant issued to Pauktuutit Inuit Women of Canada resulted in the development of a strong network of community members, Inuit organizations, groups, and academics. It also raised awareness of HIV in Inuit communities and has encouraged community members to consider the linkages between the work they are doing in areas such as education, housing, health, and HIV.
Key informants indicated that communities have a sense of ownership over the program and feel empowered to make a meaningful contribution to HIV/AIDS CBR. They said that the existence of program raises the profile of CBR, creates opportunities to conduct CBR, and encourages the formation of the relationships needed to conduct the research.
Key informants reported that the program has increased community literacy in research design, methodologies, and analysis and has successfully engaged more communities in conducting research, whether internally, through CIHR, or through other funding sources. Key informants also said that because of this program, communities are becoming more comfortable working with researchers and, as a result, the program has fostered the development of strong relationships between communities and academia. Some key informants mentioned that they could not think of any partnerships that had not continued after the CBR funding ended. Other key informants said that the program brought together partners that used to be at odds and helped them develop a productive working relationship; these partners are now cross-referring clients to each other's organizations. From the file review, some applicants noted that their partnerships led to subsequent research projects. Several key informants expressed interest in seeing conference presentations on best practices for building capacity and establishing relationships.
For communities to use research results to address the HIV epidemic at the local level, they need to be aware of the results. Therefore, knowledge dissemination is an important aspect of CIHR's HIV/AIDS CBR Program. Key informants reported that both communities and academics are involved in dissemination activities. Examples of knowledge dissemination methods that funded applicants have used include preparing two versions of research reports (one for academia and one for communities); conducting presentations at conferences (including some international conferences); posting results on the Internet; publishing journal articles; preparing factsheets and newsletter articles; summarizing research results; and organizing community gatherings, roundtables, and forums. Other dissemination activities mentioned in the project files include conducting radio interviews and sharing results with international government departments. As appropriate, these dissemination activities also include a discussion of how the research results are applicable to the community. Much of this knowledge dissemination has focused on stakeholders within the HIV/AIDS circles. Key informants suggested that it is important to extend the reach of the knowledge dissemination activities to include stakeholders in other sectors (e.g., housing, addictions, mental health).
Although there is evidence that knowledge dissemination activities are occurring, some key informants question their effectiveness. They noted that there are numerous organizations that are not aware of the research and/or the results, and indicated that the dissemination activities need to extend beyond the groups involved in the research project.
Aboriginal stream participants in the stakeholder consultation mentioned that in Aboriginal communities, it is important to share results with the communities first. Participants reported that sharing results with communities can be a lengthy and involved process. Often individual meetings are held with the communities involved in research before a joint meeting with all communities is held. Additionally, meetings are held with a variety of community groups such as Chiefs, community centres and schools. Participants mentioned that this work needs to be done in a way that acknowledges or shows respect for the community; for example, meetings may be held in conjunction with feasts. Participants stressed the importance of empowering communities to use the research results to help them address the issues they are facing.
Aboriginal stream participants in the stakeholder consultation the also discussed the need to share research results with communities in creative ways. While some knowledge transfer and dissemination will be in written form (e.g., reports, publications, newsletter articles, web postings), it also needs to be shared orally and visually. Some alternate ways of sharing results are through drama, art projects, and videos.13
While Aboriginal stream participants in the stakeholder consultation acknowledged that CIHR provides support for knowledge translation (KT) as part of CBR grants and through the Meeting, Planning and Dissemination tool that it launches three times a year, they do not believe that these tools are well known and therefore think CIHR should market them more. These participants also noted that CBOs do not have the resources to put together yet another proposal for KT funds and as a result believe that KT funding should always be part of the initial research grant offered through funders, including CIHR.
General stream participants in the stakeholder consultation indicated that knowledge translation is integral to achieving community impact, but also recognized that it is expensive. They explained that although successful applicants are required to conduct knowledge translation activities, they do not receive sufficient funding to conduct the research and carry out dissemination activities. They also indicated that, aside from publishing journal articles, academics are not recognized for their knowledge translation work.
Some key informants and participants in the stakeholder consultation suggested that there is a need for a funding tool aimed at knowledge dissemination. This funding could be used to publish journal articles, post information on the Internet, place articles in local newspapers, conduct demonstration projects, hold community meetings, hold workshops to link stakeholders with research results and discuss how research results can be used in the community, or train peers on how to convey the results of research to local communities. Some participants in the consultation suggested that this tool should include a work release stipend for dissemination activities and funding for translating materials into different languages.
In addition to funding a dissemination/knowledge translation tool, stakeholder consultation participants and key informants made the following suggestions for other forms of assistance that CIHR could provide:
CIHR's HIV/AIDS CBR Program is helping communities address the issues they are facing. Key informants reported that projects funded through the program are raising awareness of HIV/AIDS in the community, helping reduce stigma around the disease, and stimulating discussions about HIV/AIDS. They also said that through participating in these research projects, communities' awareness of the specific problems they face is increasing, they are recognizing that they have the capacity to address these problems, and they are taking steps to address them. The case study of a CBRF demonstrated that one of the roles of a CBRF is to help organizations analyze the outcomes of various studies and determine how they can use the information to address the issues their community is facing.
Communities are using research findings in proposals for program funding, to make changes to their programs and services, and to advocate for policy change. Applicants noted using research findings to develop intervention programs and programming for small groups, inform the development of new data collection instruments, define valid research methodologies in additional research projects, and create databases of contacts and potential partners. However, despite this anecdotal evidence, it is difficult to determine the extent to which the research results are being applied to the community due to the weaknesses of the program's reporting requirements.
Aboriginal stream participants in the stakeholder consultation said the CBR should lead to policy changes; however, they do not believe that the projects conducted through the HIV/AIDS CBR Program have had this effect yet. It was noted that the CBR Program could provide an opportunity to do participatory action research or intervention research where academics are involved in implementing any changes or new policies that result from the research findings.
Aboriginal stream participants in the stakeholder consultation stressed that a coordinated approach to KT and community engagement is needed to affect policy change. They reported that gathering and integrating the results of HIV/AIDS CBR would allow creators and users of the research to build the argument for systemic change. These participants do not believe that individual CBOs can take on this responsibility and said that a centralized body needs to collect the information and make it available to academics, communities, and other stakeholders. They indicated that CIHR could assist organizations in their efforts to advocate for policy change by making CBR results and information readily available. For example, CIHR could create a web portal that would serve as a library with links to CBR results/reports and, in that way, lead the effort of gathering HIV/AIDS CBR findings.
Key informants believe that this is a unique and essential program for the HIV/AIDS community. There is a long history of community action addressing the epidemic. The community is best-positioned to identify the research questions, define appropriate approaches for reaching stakeholders, and use the research results to implement change. Key informants asserted that the CIHR HIV/AIDS CBR Program raises the profile of and legitimizes CBR, which is particularly important for engaging academics in the research process. It also develops community capacity to engage in research, which is an ongoing need given the large number of community-based organizations in Canada and the high level of staff turnover in these organizations.
Key informants emphasized that the research conducted through CIHR's HIV/AIDS CBR Program is valuable to communities and simply would not happen if the program did not exist. Part of the reason for this is that few other funding sources are available to communities and academia for HIV/AIDS CBR. Although key informants identified a handful of other programs that provide funding for HIV/AIDS CBR, the funding is often only available to a limited set of applicants. Examples of other funding sources include CIHR (general funding), Social Sciences and Health Research Council (SSHRC), Ontario HIV Treatment Network, Wellsley Institute, the Vancouver Foundation, the AIDS Community Action Plan, and the Atlantic Interdisciplinary Research Network. Some funding may also be available through universities and hospitals.
Key informants believe that the HIV/AIDS CBR Program keeps communities engaged in CBR. They said that without the program, it would be harder for communities to get involved in research. They also suggested that any research that communities undertook would be small in scale and scope and would not involve robust partnerships with academia. They also indicated that communities would depend more on academia than they currently do to conduct HIV/AIDS research.
While one of the objectives of the program is to help communities address the HIV/AIDS issues they are facing, Aboriginal stream participants in the stakeholder consultation cautioned that not all CBOs can or want to be major partners in CBR projects. As they explained, the mandate of Aboriginal service organizations is to provide service, and these organizations are stretched thin meeting this responsibility. At the same time, they recognized that research is vital because of the crisis in the Aboriginal community where people are dying of the disease. Due to resource shortages, participants suggested that the program should provide the community with a researcher to conduct CBR instead of asking the community to build its own capacity.
[ Previous Section | Next Section ]
2. This comment refers to CIHR's old website. In December 2008, CIHR updated the architecture of its website; however, the content relating to CBR remains unchanged.
3. The "additional assistance" section of each RFAs includes a link to a listing of the current CBRFs and their contact information. Nevertheless, many stakeholders did not appear to know that this information was available.
4. CIHR has conducted these in the past and participants found them useful. Theoretically, CBRFs are offering workshops of this nature to local communities. However, community members may not be aware of the CBRFs or there may not be a CBRF available in their area (if the position is not filled or if an application has not been submitted or approved for a particular geographic area). Additionally, some CBRFs only offer workshops at the request of the community and the community may not be aware that it can request this type of assistance.
5. Section 3.5.1 offers additional comments on the factors that may prevent organizations from applying to the program.
6. The HIV/AIDS CBR Program includes provisions for work release stipends for staff who are not conducting research activities as part of their regular employment; however, funding cannot be used to hire an additional staff person to conduct research activities.
7. The funding amount and term for Doctoral Awards available through the HIV/AIDS CBR Program and the general CIHR Program are equal. However, greater awards, based on the excellence of candidates, are available through the Canada Graduate Scholarship and Vanier Scholarship programs.
8. This step helps CIHR determine how many and what type of applications it can expect to receive as part of a competition, which enables it to begin preparations for the merit review process.
9. CIHR followed up with these applicants to obtain the required signatures.
10. Some community members have asked their academic partners to complete their CVs for them.
11. Students applying to the Masters Award made similar comments.
12. To streamline the review process, applications receiving initial scores for scientific merit or potential impact of less than or equal to 2.9 (on the 4.9 CIHR rating scale), are not further discussed at the committee meeting. All applicants, regardless of score, receive reviewer feedback outlining the strengths and areas for improvement in the application.
13. These forms of dissemination are suitable for use in both the general and Aboriginal streams.