Measuring improvement

Measuring needs to take place before, during and after the project. Beforehand, it provides a baseline against which any improvements can be measured. During the project, it enables you to chart progress and adjust approaches if necessary. After the project, it shows what has been achieved and, if it is successful, can be used as a basis for rolling out the approach more widely.

As well as generic goals to improve patients’ experience, your PFCC project needs to have clear goals from the start and associated measures. Common aims set by teams include:

• consistency of clinical care
• efficiency, including discharge processes
• improving communication among staff, and with patients and families
• improving relationships with families (for example, improving access for families wanting to talk to clinical staff or opening up visiting)
• building staff confidence (often related to communication)
• improving staff experience
• making environmental improvements, either to the physical environment, or by changing activities or ward routines.

To gauge whether your goals are specific enough, check if they are SMART: Specific, Measurable, Achievable, Realistic and Timely. For example, to make the aim ‘to improve patient and family experience when attending with acute abdominal pain’ SMART, you need to identify a goal and an intervention related to each driver. So the first measure would be overall experience (for example, 90 per cent of patients saying that care met their expectations, by [target date]).

Then think about each of the drivers. If you are focusing on pain management, the goal and measure might be ‘to assess levels of pain and offer analgesia within 15 minutes for 90 per cent of patients by [target date]’. The purpose of measurement is to use it to ask critical questions and guide intelligent action.

In order to track progress towards your goal, include a good balance of the following three types of measures.

• Outcome measures – How is the system performing? What is the result? (Was the patient’s experience better?).
• Process measures – Are the parts or steps in the system performing as planned? (Was the care better?).
• Balancing measures – What happened to the wider system as we improved the outcome and process measures? (Were there unintended consequences or impacts on outcomes elsewhere?).

Key points

• Measure little and often: measurement for improvement does not require large datasets. It is better to start with one measure, and add more, than to be ambitious about the number of measures to be collected and feel defeated by the scale of it.
• If you do not gather strong baseline data, you will never know exactly how much you have achieved.
• For the PFCC project, your measures should focus on patient experience and staff experience, as this is the focus of the overall programme. Ultimately, these factors will show whether you have met your aim.
• Data for improvement is different from data for research. It is messier and less accurate, but highly relevant to the daily work of clinicians. Sampling is often appropriate – for example, asking 10 patients per month, as opposed to all patients. In measuring for improvement, it is rapid, small-scale feedback (through PDSA cycles) that will help you assess the impact of your changes.
• Monitor your progress through a dashboard. This must include the main types of measure (process, outcomes and balancing measures). It should also make clear what the goal is (how much to achieve and by when), how progress will be calculated, and where the data will come from. All these are essential questions to answer when developing your measures. (See PFCC sample measurement dashboard).
• Make sure your measures relate directly to the factors that you are changing. For example, if your goal is ‘to improve discharge processes’, and you plan to do this by improving documentation for the staff, then measuring patient satisfaction is too broad a measure. Instead, you need to measure the extent to which staff use the documentation, and the staff’s opinions of the documentation. Again, sampling can be used here – for example, looking at 10 medical records per week or month, or asking five staff per week or month.
• Driver diagrams play a useful role in this activity as these help pin down what is important to improving the patient experience, before identifying the interventions and measures that relate to these drivers.
• Make sure you are clear about what you plan to accomplish, how you will know that this change will improve patients’ experience or outcomes, and precisely what activities you will put in place to effect this change.
• Use the expertise in quality improvement within your organisation to support you. Techniques such as ‘run charts’ (see PFCC further reading), which can track progress over time can be very useful in providing a persuasive picture of your progress. Above all, remember that the purpose of measurement for improvement is to support you to achieve your aims. The data must therefore be of value to you – not for reporting elsewhere.