Results chain framework
As funds awarded by CCSRI are raised almost entirely through public donations, the institute recognizes that it is accountable to the public for the value it creates through research. As such, CCSRI engages in ongoing progress monitoring and periodic evaluation studies to ensure that its programs and initiatives are relevant and aligned with the Canadian Cancer Society mission, as well as being successful, cost effective and sustainable. In other words, CCSRI strives to ensure that the research it funds has the most impact in the fight against cancers. It is important to note that in this context, research impacts refer not only to the impacts on cancer patients and the population at large (i.e. health and socio-economic impacts), but also the advancement of knowledge, capacity building and informing decision-making.
The Canadian Cancer Society has implemented an approach to results planning, measurement, evaluation and reporting which has been based on what is known as a ‘Bennett hierarchy’ 1. The approach has informed planning activities and the reporting of progress for accountability, and is being used as a framework to monitor and evaluate CCSRI research impacts.
The ultimate goals of CCSRI research initiatives and programs are often ambitious and long term, such as having an impact on cancer incidence rates, cancer mortality rates or the quality of life of people living with cancer. As such, it is imperative to develop strong program descriptions providing details not only on the intended long-term outcomes but the short-term and intermediate outcomes that precede it and the sequence in which they are likely to take place. The results chain hierarchy provides a simplified description of a program and is organized according to seven levels of results. It shows the logical relationships among the resources that are invested, the activities that take place, and the sequence of changes that result.
Underpinning the results chain hierarchy is the idea that CCSRI programs are taking place within circles or spheres of influence 2:
- In sphere 1 – the sphere of control – resources are spent, activities are conducted and outputs are delivered. In this sphere, choices are made about how much to invest in activities and services and their respective intensity. Levels 1 – 3 of the results chain hierarchy are understood to be operating in this sphere of control.
- In sphere 2 – the sphere of direct influence – clients, intermediaries, partners or stakeholders are directly reached by a program’s activities or services. The groups directly reached, and their reactions, use, knowledge, aspiration or behaviour changes represent the sphere of direct influence. Levels 3 – 6 of the results chain hierarchy are understood to be operating in this sphere of direct influence.
- In sphere 3 – the sphere of indirect influence – change is sought in broad communities. The results in this sphere tend to be related to an organization’s mission or program’s ultimate goals. Levels 6 – 7 of the results chain hierarchy are understood to be operating in this sphere of indirect influence.
This figure shows the results chain hierarchy with the spheres of influence and the seven levels of events/results on the left-hand side along with the matching levels of evidence on the right-hand side.
The results chain hierarchy has been adapted by CCSRI to demonstrate the many ways in which research activities can have impacts. As shown in this example, key activities and results can be mapped vertically, according to the results chain hierarchy levels, and horizontally, across time and the research spectrum. The example should be read from left to right and from the bottom up. The results chain approach attempts to build in an appreciation of the multi-year build up of research results and knowledge, while showing both the short-term and intermediate outcomes that come before the long-term impacts and the sequence in which they are likely to occur. As such, it recognizes research activity as part of a science and innovation pathway of connections and shows the cause and effect relationships of research activity. This framework also shows how research impacts can feed back into the lower levels of the results chain potentially creating inputs for future research and influencing the diffusion and impacts of other research. Ultimately, these results categories can be tracked over time to tell a performance story.
The use of the results chain hierarchy to monitor and evaluate research impacts is still in its early days of implementation; however, the framework was recently applied to develop an evaluative performance story for a Canadian Cancer Society funded NCIC CTG clinical trial for letrozole, an aromatase inhibitor used as a late adjuvant therapy for hormone receptor positive breast cancer. CCSRI also published an article on the use of the results chain hierarchy in evaluating research impacts.
1 Bennett, Claude and Kay Rockwell 1995. Targeting Outcomes of Programs (TOP): an Integrated Approach to Planning and Evaluation. Nebraska, USA: Nebraska Cooperative Extension Service.
2 Montague, S 2000. Circles of Influence: an Approach to Structured, Succinct Strategy available at http://www.pmn.net/library/Circles_of_Influence_An_Approach.htm
Last modified on: October 30, 2013