How to measure patient experience for specific health conditions

Patient experience is widely recognised as one of the central elements of quality care in the NHS, alongside patient safety and clinical effectiveness. Various UK national patient surveys gather data about healthcare experiences, but most focus on specific services rather than conditions or pathways. The Cancer Patient Experience Survey is the only locally representative national programme focusing on a specific collection of related diseases.

At Picker, we work with organisations that specialise in different health conditions and procedures, such as the British Liver Transplant Group (BLTG). Our work with the BLTG involved developing a survey to measure the experiences of adult patients who have received a liver transplant in the UK. We designed a Patient Reported Experience Measure (PREM), which is a survey that aims to measure people’s experiences of care and treatment, to allow for comparison, planning, and improvement of services.

Our process to create the liver transplant PREM can be used to develop similar patient experience surveys for other groups. Here is how we approached the project:

1. Define the aims

Be clear about what you are trying to achieve, involve relevant stakeholders (eg clinicians, patients, carers), and define what the survey needs to measure. We aimed to develop a survey to facilitate understanding of people’s experiences of liver transplant care in the UK.

2. Understand your patient population

Get a firm understanding of the care pathways associated with the patient population (including access to care, common symptoms, treatment and medications, surgery and other interventions), along with knowledge about what matters to patients. How do you go about obtaining this?

• Consultation – include stakeholders at all stages of the survey design.
• Literature review – understand what information and research is already out there.
• Qualitative research – speak to patients to gather new information about what matters to them – for example by conducting in-depth interviews or focus groups.

The stakeholder advisory group, comprising patients and professionals, was consulted throughout the liver transplant PREM development. We carried out a scoping review to understand existing literature, the liver transplant care pathways and what themes might arise. Finally, we created a topic guide and conducted patient interviews to explore their experiences and priorities. These stages defined the themes that formed the basis of the survey.

3. Choose a methodology

Decide how you will reach the patient population to invite them to complete a survey: for example, by posting a survey to them, inviting them to complete an online questionnaire, or by asking patients the questions at the point of receiving care. They all have different strengths and weaknesses. For example, feedback can be collected more rapidly at the point of receiving care, but people will generally answer more positively to a survey in these cases and may be unable to describe their experiences of recovery or rehabilitation.

Other considerations to bear in mind are:

• The budget – a postal paper-based survey will be more expensive than an online survey due to the costs associated with printing, postage and data entry.
• The personnel – who do you need to involve?
• Sample size – how big is your patient population? How many people do you want to reach?

We felt the most appropriate approach for the liver transplant survey was to use a methodology similar to the NHS Patient Survey Programme. These are postal surveys, with up to two reminders, a cover letter detailing the survey and how to opt-out, and a Freepost return envelope.

4. Develop the survey

The qualitative and scoping stages and the Picker Principles of Person Centred Care informed the development of the patient experience questions. In this case, the questions covered pre-transplant care, transplant care, inpatient care, post-transplant care, and overall experience.

5. Cognitive testing

Any new survey needs to be tested with the patient population to check that respondents can understand the questions, remember the things they are being asked about, and give an appropriate answer that corresponds to their experiences. Cognitive testing happens across several rounds with changes made between each round to refine and improve the survey until no further problems arise.

The liver transplant survey was cognitively tested with 23 patients, following which it was refined and finalised.

6. Piloting

A pilot aims to trial the data collection to explore how a survey functions.

For our liver transplant survey pilot, seven transplant centres across England and Scotland sampled eligible patients. From this, 2006 patients were invited to participate using a postal methodology and were sent one or two reminders. The response rate was 60%, similar to the 61% response rate seen in the latest 2019 National Cancer Patient Experience Survey, and higher than the latest National Inpatient Survey 2019 (45%).

7. Explore the validity of the survey

In addition to cognitive testing, the validity of a survey can be improved by exploring the pilot data set. For example, high drop out and missing data can indicate problems with questions, whereas questions that show a relationship with one another may be measuring the same aspect of the patient experience.

Response frequencies, provider-level reliability and comparability, and structural analysis on the liver transplant survey showed that it functioned well in measuring care experiences.

Limitations

Investing time in testing a survey will increase the chances of the resulting data being reliable and actionable. It is important to remember that a survey will not capture everything: nor should it try to. In addition, respondents are self-selecting, ie patients may be invited, but choose whether to reply. Qualitative methods such as interviews or focus groups are best suited to smaller patient populations (eg rare conditions) and where sensitive issues might wish to be explored.

If you’d like more advice about survey design, please get in touch by emailing info@pickereurope.ac.uk

The research into the development of a Patient Reported Experience Measure for liver transplant care in the UK, is being presented at BTS NHSBT Live, 24th – 25th February 2021 – O31.