Evaluation of CIHR's HIV/AIDS Community-Based Research Program - Final Report
4.0 Conclusion and recommendations
CIHR's HIV/AIDS CBR Program is helping communities and academia respond to the HIV/AIDS epidemic. The program is building research capacity at the community level through workshops. It is also building research capacity in academic circles through the Masters Awards and Doctoral Awards. Through catalyst grants and operating grants, the program financially supports community-based research that involves partnerships between communities and academia; this active involvement in research also contributes to capacity-building. In addition to providing financial support for capacity-building and research, CIHR funds CBRFs, which are available to communities and academia to help them build research capacity and partnerships as well as to provide technical assistance for research proposals and ongoing projects.
This program helps communities obtain the information and evidence they need to develop strategies to address HIV/AIDS issues. The program has successfully engaged a variety of community organizations and academic researchers across the country in HIV/AIDS CBR. It has raised community awareness of HIV/AIDS and increased the profile of CBR as a valid research tool. Communities are developing research capacity and are beginning to use research findings to refine their service delivery approaches, develop new programs, and advocate for policy change. Since few funding sources for HIV/AIDS CBR exist, it is unlikely that this research would be conducted outside of this program.
While communities value this program, it can be difficult for them to access. Small organizations, organizations in rural areas, and organizations tangentially addressing HIV/AIDS may have limited or no awareness of the program. Some organizations may be reluctant to apply to the program because they fear research, are reluctant to partner with academics, or do not have the capacity or resources to engage in research. Additionally, some of the programs' administrative processes can lead communities to believe that the program is oriented toward academia. Community organizations find the application process difficult, can feel forced to partner with universities even if they do not believe the research warrants it, and may not be eligible to hold CIHR funds (or are required to go through extensive approval processes to demonstrate their eligibility).
4.1 Recommendations
Based on the evaluation findings, the feedback received at the consultation, and discussions with CIHR and the CIHR CBR Steering Committee, PRA prepared a set of recommendations for the program. The evaluation resulted in 14 recommendations, which are presented below according to the following thematic areas:
- Awareness and promotion
- Application process
- Merit review
- Funding tools
- Requirements for the release of funding
- Administration
- Expand efforts to raise awareness of CBR and the HIV/AIDS CBR Program by identifying the full range of HIV/AIDS CBR stakeholders and tailoring promotion, education, and capacity-building activities to meet their specific needs.
Identifying stakeholders
CIHR's database of HIV/AIDS stakeholders and potential applicants comprises previous applicants, a CIHR-developed CBR listserve, and CIHR and Institute of Infection and Immunity (III) researcher databases. CIHR also takes advantage of some of its partners' contact lists by placing information about the HIV/AIDS CBR Program in their newsletters. CIHR should continue to work with its partners to expand its list of HIV/AIDS stakeholders.
The range of applicants to the HIV/AIDS CBR Program tends to be limited to a small pool of large AIDS Service Organizations (ASOs) and academic departments directly involved in HIV/AIDS research. CIHR's promotional efforts typically focus on maintaining previous applicants' awareness of the program. CIHR should continue to keep these stakeholders informed about the program.
The program has received few applications from small or rural community-based organizations; organizations whose primary mandate is not HIV/AIDS, but who address some HIV/AIDS needs such as co-infection and determinants of health; or academic departments engaged in cross-disciplines. CIHR should expand its contact list to include more small and rural CBOs as well as non-HIV/AIDS CBOs.
Tailoring activities
HIV/AIDS stakeholders have a wide range of knowledge and experience with CBR and the HIV/AIDS CBR Program. While some may have sophisticated knowledge about research processes and how to complete funding applications, others may have had little to no exposure to research. Therefore, promotional efforts should involve a range of activities including distributing email notifications of funding opportunities, describing to stakeholders how CBR can benefit communities, and discussing with them their interest in engaging in research projects. Promotion, education, and capacity-building activities should be provided using various mediums such as written information, face-to-face meetings, conference presentations, and workshops or seminars.
The distribution of this information to stakeholders is essential to empowering communities and academics to respond to the HIV/AIDS epidemic. Additionally, telling stakeholders about the benefits of the program and how it has helped communities will encourage other organizations addressing HIV/AIDS issues to apply for funding. This information should be made available to stakeholders through publications, presentations at conferences, on CIHR's website, and in a database of "grey literature" sources on CBR and HIV/AIDS.
- Clarify program eligibility requirements. This includes defining the range of projects eligible for funding as well as eligible expenses.
The community response to the HIV/AIDS epidemic involves a wide range of stakeholders including organizations that provide HIV/AIDS services and individuals engaged in conducting HIV/AIDS research. Other organizations addressing factors that place people at increased risk for HIV/AIDS also have a significant role to play. While some of these organizations may include research as part of their mandate, many focus primarily on service delivery and therefore have limited resources to devote to research.
Clarifying the range of research issues and expenses that qualify for funding will help organizations understand how this program can assist them in addressing HIV/AIDS issues in their community. For example, more information is needed about whether funding can be used for research relating to issues such as needle exchange programs and housing. Additional information is also needed about the kind of expenses funding can be used for (e.g., work releases, hiring research coordinators, honouring cultural protocols and traditions, and knowledge translation.) This information should be available in requests for applications (RFAs) and on the CBR website (see recommendation 3).
- Update, restructure, and expand the CIHR CBR website to include complete program information that users can easily access.
The evaluation found that it can be difficult for stakeholders to access complete information about the HIV/AIDS CBR Program on CIHR's website. In fact, many of the applicants were not aware of the full set of funding tools and/or had limited understanding of the program's eligibility criteria, application process, merit review process (see recommendation 7), and the conditions that have to be met prior to the release of funding. Therefore, a clear and concise description of the program needs to be available to HIV/AIDS stakeholders.
The new website should include:
- A program profile with:
- An explanation of what CBR is and how it is conducted
- An overview of the suite of funding tools
- A description of the types of projects and expenses that are eligible for funding
- An overview of the merit review process, including a listing of the committee members, an explanation of the review process, and a listing of the review criteria (this should include clear definitions of what scientific merit and community impact constitutes)
- A listing of the conditions that applicants have to meet to be eligible to hold CIHR funds
- Up-to-date and complete contact information for program officers and CBRFs.
- Examples of successful applications to help those with limited grant-writing experience to understand the requirements.
- Information on program successes and the results of completed research. For example, the website could include factsheets about funded projects, final reports, and stories about the impact that funded projects have had on communities, programs, and policies.
In December 2008, following the data collection phase of the evaluation, CIHR launched a new website. Although the modifications to the CIHR website related to its general structure, they may provide simplified navigation to meet some of the above needs. CIHR should seek feedback from stakeholders on the navigability of the new site.
- A program profile with:
- Simplify the language used in the application forms and improve the usability of web forms.
CIHR requires applicants to use web forms to apply for funding. The evaluation found that these forms are hard for applicants to complete because the forms contain academic-oriented language and are difficult to use. The web forms should use plain language that is accessible to academics and community members. Additionally, given CIHR's enforcement of its Policy on Complete Applications, a checklist of the items that need to be included with the application should be easily accessible.
- Create a CV form (or forms) that captures relevant academic and community experience.
CBR teams need to include a balance of academic and community-based knowledge and experience. The evaluation determined that the common CV and community CV do not provide ample opportunity for applicants to describe their community-based qualifications. In part, this reflects the academic-orientation of the language used in the forms.
The Common CV should require academic applicants to demonstrate their experience conducting CBR and sharing results with communities. The Community CV should place less emphasis on academic qualifications and focus on community involvement. Neither the Common CV nor the Community CV is relevant for applicants who have a combination of academic and community experience.
- Give applicants the opportunity to have their application pre-screened for missing or incomplete information.
Under CIHR's Policy on Complete Applications, applications that are missing required information can automatically be denied funding. Since some applicants may have limited familiarity or experience with grant-writing processes, pre-screening applications may minimize the number of applications that are rejected for administrative reasons.
Giving applicants the opportunity to have their application pre-screened may also help ensure that all available research funding in each stream is awarded to projects. While CIHR has informally worked with applicants to ensure their applications include the required signatures, CVs, and budget modules, pre-screening should become a formalized part of the process.
- Examine the merit review process for each funding tool to determine where the process can be simplified and streamlined. Additionally, opportunities for providing a more interactive process should be explored.
Merit review processes
The same merit review process is used to assess applications for each funding tool. However, the evaluation found that an assessment of scientific merit and/or community impact may not be warranted for some funding tools such as the capacity-building workshops, catalyst grants, and community-based research facilitators. The process should be reviewed to identify opportunities to simplify and streamline.
Interactive review process
Some applications are rejected because the Merit Review Committee does not see how the project fits with CBR principles or questions the scientific merit or community impact of the project. Giving applicants the option to verbally present their proposals and ask questions to the Committee would enable them to more fully explain their proposed research project. This provision respects Aboriginal traditions of oral communication. It may also reduce the number of applications that are rejected, which in turn may help ensure all of the available funding in each stream is allocated to projects for that stream.
- Create a mentorship program for the Merit Review Committees to assist in developing the capacity of current members and recruiting new members.
The HIV/AIDS community is small, which can make it difficult to recruit members for the Merit Review Committees. Additionally, some members do not have previous experience serving in this capacity. A mentorship program would help potential and existing members to further develop their capacity to participate in the review process. It may also help retain existing members for increased lengths of time.
The mentorship program should include specific and detailed guidelines for the review of applications, as the evaluation noted concerns about how the qualifications of academics and community members are assessed and valued and how some of the information included in CVs is used in the review. The evaluation also found that it can be difficult for members of Merit Review Committees to separate the concepts of scientific merit and community impact. Thus, more specific guidelines for the review of applications would help to ensure that all applications are assessed using the same criteria and that equal weight is applied to academic and community-based qualifications.
- Ensure the Merit Review Committee for the Aboriginal stream comprises Aboriginal people as well as regionally-appropriate Elder representation.
Having Aboriginal people and Elders as members of the Merit Review Committee for the Aboriginal stream is important. Their knowledge and experience will ensure the review of applications for this stream considers whether the proposed research is socially and culturally appropriate.
To balance the various cultural and regional beliefs and traditions, the Committee membership should include First Nations, Métis, and Inuit representation. Elders included on the Committee should be familiar with the communities where the proposed research would be conducted.
Additionally, CIHR should ensure that, as needed, any non-Aboriginal members of the committee for the Aboriginal stream are provided with opportunities to improve their understanding of Aboriginal history and culture.
- Review and revise the suite of funding tools, including:
- the structure of and processes associated with existing tools. This includes, but is not limited to, the application and review processes, the frequency of funding cycles, and the funding caps.
- the range of funding tools available to applicants. This includes expanding the suite of tools to include opportunities for students at all stages of their education and increasing opportunities for end of grant knowledge translation.
Various opportunities exist to improve the efficiency of the CBR program and the effectiveness of its funding tools. In addition to the improvements described below, other modifications may be made to the suite of funding tools to prevent unnecessary overlap or duplication.
Application process
Although the maximum funding available for a single catalyst grant or capacity-building workshop is $25,000, the evaluation found the application process for these tools just as time-consuming and difficult as it is for operating grants, which provide up to $100,000 in funding for three years. Stakeholders apply for capacity-building workshops and catalyst grants when they have an issue that needs immediate attention; they may be reluctant to apply for this funding if they have to wait several months for a decision.
Additionally, the review process for these tools includes an assessment of the scientific merit of the project and accepted catalyst grant applications may be required to obtain ethics approval. Simplifying the application and review processes for these tools will increase their accessibility to CBOs and academics.
Frequency of funding cycles
As mentioned above, stakeholders apply for capacity-building workshops and catalyst grants when they have an issue that needs immediate attention. Often applicants cannot afford to wait for up to a year to apply for funding, especially if their first application was denied. Increasing the frequency of the funding cycles for these tools may encourage more applications.
Funding caps
Successful applicants use available funding for numerous activities relating to their research project. Funding may be used for gaining community support for a project, developing and evaluating the strength of partnerships, gathering information from communities, and disseminating research results to communities and other stakeholders. Sometimes projects involve the hiring of research assistants, traveling to rural or remote areas, and/or meeting cultural protocols or traditions. A review of the funding caps for each of the tools is necessary to ensure adequate resources are available to cover the various aspects of research projects.
New funding tools
- Summer internships and honours students. Summer internships were formerly available through this program when it was housed at Health Canada. Early intervention with students is essential to engaging them in the fight against HIV/AIDS. Reinstating this tool, and introducing funding for honours students, will help build academic capacity for CBR.
- Knowledge translation. Knowledge transfer, exchange, and translation is vital to empowering academics and communities to use research results to address the HIV/AIDS issues. KT occurs throughout the research process and continues past the conclusion of the project. CBR is a model of integrated KT, where communities participate in all stages of the research project and thereby share and influence the direction of the research and its outcomes.
End-of-grant KT can be conducted using various mediums, which may include written documents such as journal articles, newspaper articles, community booklets, or briefing documents; verbal information sharing such as conference presentations or community meetings; or other creative means such as art projects, dramas, or videos. Given the need to provide information to various stakeholders in a variety of formats, KT is a time-consuming and costly venture. There is often insufficient funding available within an operating grant to accommodate effective KT.
Although existing tools incorporate funding for KT activities, a separate funding tool is needed for end-of-grant KT. This funding will help ensure that the results of CBR projects can be used to influence programming and policy changes.
- Revise the structure of the CBRF funding tool to improve its effectiveness and reduce staff turnover. In particular, increase the length of the funding cycle, institute an MOU between CIHR and the host organization that outlines the roles and responsibilities of and the relationship between CBRF and the host organization, and increase the number of Aboriginal CBRFs.
CBRFs are funded through CIHR and housed at CBOs. Funding is issued for up to two years. Since 2004, the program has experienced high turnover in CBRFs. High turnover rates reduce the effectiveness of this funding tool because each time a new CBRF starts, they must gain the community's trust and rebuild connections with potential research partners. Extending the funding period for this tool may help reduce turnover.
Additionally, the evaluation found that the role of CBRFs and their relationship to CBOs is not always clear, which affects CBRFs autonomy and therefore effectiveness. An MOU outlining the relationship between CIHR and the host organization would help to reduce misunderstandings.
Almost half of the funding available to the Aboriginal stream between 2004 and 2007 was not allocated to projects. This may partially relate to the fact that only two CBRFs are available to assist Aboriginal stream applicants whereas five CBRFs are available to assist general stream applicants. An equal number of CBRFs should be available for each funding stream.
- Explore opportunities to create Review Ethics Boards (REBs) for CBR.
Projects funded through this program must obtain certification through REBs. Most commonly, applicants receive this certification through university-based REBs. Some of these boards will not review projects if the funding is to be held outside of the university and some have limited knowledge of or experience with CBR. The creation of specialized REBs that are not affiliated with academic institutions will increase the accessibility of the program for CBOs.
- Simplify the conditions that CBOs have to meet for the release of funding and develop an MOU for community organizations to hold CIHR funds.
It can be difficult for CBOs to meet the conditions for the release of program funding, which contradicts the principles of CBR. While many of these organizations may have held other sources of federal funding, it can take some of them up to a year to demonstrate to CIHR that it has the required structures and processes in place to manage the funds. Making it easier for CBOs to hold CIHR funds may encourage more organizations to apply for the program.
- Develop a performance measurement strategy for HIV/AIDS CBR and determine how project final reports can be used to monitor the performance of the program.
CIHR has not defined a performance measurement strategy for this program. The development of this strategy is essential for monitoring and communicating the success of the program. CIHR needs to finalize the program logic model and develop a template for the preparation of final project reports, which includes clear requirements for demonstrating if projects achieved their intended outputs and short-term and long-term outcomes.
[ Previous Section | Next Section ]
