Indicator and measurement menu

The indicator and measurement menu was developed as part of CCSRI’s implementation of the results chain hierarchy to monitor and evaluate CCSRI research impacts. The CCSRI indicator and measurement menu shows a series of performance indicators classified according to the results chain hierarchy. The indicator and measurement menu can be used retrospectively (for evaluating) and prospectively (for planning and monitoring). Indicators are selected in sets based on their relevance to the program being assessed or the evaluation questions being posed, and can be reworded and refined to be made program specific, as necessary. Ultimately, this tool facilitates consistent monitoring and evaluation by providing a common framework by which to monitor and assess CCSRI programs and initiatives.

The indicator and measurement menu should be read from the bottom up and from left to right. The first column identifies the levels of results (1 to 7) according to the results chain; the second provides a description of the levels of results; the third provides a set of performance indicators for each of the levels of results; and the fourth lists typical sources of data and methods of evaluation.

Chain of results	Evaluation criteria	Indicators	Typical sources/methods
7. End Results	Measures of impact on overall problem, ultimate goals, side effects, social and economic consequences	Rate or incidence of cancer (incidence, mortality, morbidity) Level of quality of life (index TBD) Level of advances in cancer science/research	Specialized analyses/ evaluations Statistical agency data Canadian cancer statistics Analytical and policy groups
6. Practice & Behaviour Change	Measures of adoption of new practices and behaviour over time	Level of research used (knowledge transfer, practice adoption) by scientists/policy makers/institutions/ health care practitioners/consumers Level of research used in curricula for new researchers (citation in text books and reading lists) Level of research cited in ongoing health professional education material (incl. clinical guidelines) Level of research cited in public policy documents Level of research cited in advocacy publications Level of research cited in scientific literature Level of research commercialization	Physical observation Inspections, reviews Surveys Evaluation studies Content analysis
5. Knowledge, Attitude, Skills Aspirations	Measures of individual and group changes in knowledge, abilities, skills and aspirations	Level of understanding of key related science information generated through research by scientists/policy makers/institutions/ health care practitioners/consumers Level of self-expressed commitment to specific areas of science/research or practice/protocol/policy change by scientists/policy makers/institutions/health care practitioners/consumers Level of development of new knowledge in cancer research Level of development of new methods in cancer research Level of published research findings in a timely manner and in peer-reviewed journals with high “impact factors”	Independent review of target groups Content analysis Survey, group self-assessment Testing/Certification Bibliometrics
4. Reactions	What participants and other stakeholders say about the program; satisfaction; interest, strengths and weaknesses	Level of program recognition and support from key stakeholders/target groups/participants Level (volume, accuracy and ‘tone’) of media coverage of research and program activities	Usage/participation tracking Correspondence content analysis Surveys Media content analysis
3. Engagement & Participation	The characteristics of program participants and other stakeholders; number, nature of involvement, and background	Level of engagement with other research centres, networks, academic institutions, government agencies, etc. Level of engagement by stakeholders/target groups/ participants Level of multidisciplinary and/or multisectorial research activities Level of recruitment and retention of stakeholders/target groups/participants (e.g. junior investigators, researchers, review panellists, etc.) Level of established external scientific advisory board(s)	Web use tracking Correspondence content analysis Observation of meetings/events Meeting attendance records Stakeholder relationship management/tracking (e.g. contracts and agreements) Surveys
2. Activities & Outputs	Implementation data on what the program actually offers	Level of research as per internal review guidelines Extent to which plans, strategies, frameworks, etc. are delivered as per expectations (expected timelines, resource usage and quality levels) Extent to which governance structure adheres to internal guidelines Extent to which policy and financial decisions are made according to Board/Senior Management/Expert Advisory Committee(s) accepted guidelines and standards Extent to which internal and external communication strategies adhere to internal standards and protocols/policies (standard operating procedures)	Project/Initiative tracking Project reports Content analysis or records Peer review Operating reviews Risk analysis
1. Inputs	Resources expended; number and types of staff involved; time expended	Level of human resources (staffing) at all levels Level of financial resources (budgets vs. actuals) at all levels	Budget analysis Time, reporting and budget/plan review Activity-based costing

Last modified on: October 30, 2013