Measuring Change

The purpose of knowledge translation activities is to support the use of research evidence in practice. To encourage this, many projects now look at communication and implementation strategies. To determine whether implementation activities have been successful, it is important to be able to measure changes. Tracking changes can also enable plans to be modified or adapted if circumstances change or if elements are not working as intended.

There are many measures that can be used as part of the evaluation of a knowledge translation activity. Measures can look at how the process is being implemented and delivered. They can focus on whether people’s knowledge or attitudes change. Measures can also be used to monitor whether behaviour, such as prescribing habits, change. Change measures can be applied to individual clinician or to an organisational group such as a multidisciplinary team or a health service. [1]

Evaluation plans

When planning how to implement a change, it is also worthwhile to look at an evaluation strategy. Evaluation involves collecting and analysing information and data to measure the impact and effectiveness of the intervention or activity against the purpose of the activity. It helps to focus what you are expecting to happen and how you can determine the results of what you do by looking at appropriate measures. 

Evaluation generally reflects a series of familiar questions and then looks at how to find the answers:

1. Did we do what we said we would do? (Accountability reporting)
2. As we go along, are there things we could change to make it more useful? (Formative evaluation)
3. What happened as a result of what we did? (Summative evaluation. Can also include impact and outcome assessment)
4. What would we do differently next time? (Feedback loops in quality improvement)
5. How can these findings be used to further develop what we are trying to achieve? (Ongoing). [2]

The type of evaluation approach and measures used will be determined by the specific project or activity. 

Data sources

When planning the measures to show changes, it can be useful to identify what data are collected as part of normal practices. For example, many services are involved with the Palliative Care Outcomes Collaboration (PCOC). The PCOC Data Set includes information about, the person receiving care, administrative episode details about the period of care, and clinical characteristics. Many hospitals collect data as part of quality assurance processes. These could be quite relevant in measuring change in a project to reduce evidence-practice gaps.