The person-based development and realist evaluation of a summary report for GP consultations

1.1 Rationale for study

The Calgary-Cambridge guide, which is used as a basis for training medical students and doctors, identifies six steps to conducting a GP consultation: initiating, information gathering, providing structure, relationship building, explanation/planning and closing¹. Opportunities to address patients problems are commonly missed at consultation initiation (when the GP should elicit the patient’s reason for attendance)². Problems can remain unaddressed at consultation closure, if advice given is unclear, particularly with regards advising patients what to do if the problem does not resolve, or gets worse, which GPs refer to as “safety-netting”³. Research suggests that interventions at each end of the consultation can help to address patient concerns. At consultation initiation, sharing the results from patient-reported outcome measures (PROMs) with clinicians can help to elicit concerns⁴. At consultation closure, providing the patient with written as well as spoken information can improve recall and adherence⁵.

To help with this problem, we designed an intervention: the Consultation Open and Close (COAC) intervention which used a pre-consultation form at consultation opening and a summary report at consultation closure. We then tested these interventions in a feasibility study.

1.2 Provision of written information at consultation closure

The closing steps of a consultation are when clinicians summarise, make a plan with the patient, safety-net and check the patient’s understanding⁶. Patients often raise last-minute concerns at this point, particularly if all concerns have not been elicited early on⁷. Patients’ memory for advice on treatment and follow-up after the consultation tends to be worse with older people; if the information given contradicts existing beliefs; or if potentially life-altering diagnostic information is given⁵.

Written advice can be provided at any point in the consultation but is most often provided at consultation closure. This may be general information on a specific condition, healthy lifestyle advice or safety-netting advice. Provision of written information can improve patient understanding, memory of the consultation and subsequent adherence^5,8. Patients remember specific advice which is individually tailored to them more easily than generic advice provided in patient information leaflets⁵. Where patients are routinely provided with information on their medication and consultation, through direct patient record access, this has improved patient safety and adherence⁹.

The aim of this study was to develop and test an intervention to more comprehensively address patients’ concerns in general practice through use of a pre-consultation form at consultation opening and a summary report on consultation closure.

This paper describes the development of the summary report and a realist evaluation of its use from the feasibility study of the COAC intervention. The development and testing of the pre-consultation form and the full feasibility study results are reported separately in two papers published alongside this one.

2.1 Study setting

This study was based in primary care involving general practices serving different patient populations in Bristol, North Somerset and South Gloucestershire (BNSSG). Practices were selected from areas within a range of socioeconomic deprivation levels as well as urban, suburban and rural areas.

The COVID-19 pandemic occurred six months into this study. Under an NIHR directive, the study was paused in March 2020 and restarted in September 2020. Research protocols were updated so the intervention and research did not require face-to-face contact.

2.2 The Consultation Open and Close (COAC) Study

The COAC Study involved the development and testing of an intervention, incorporating use of an individual-level PROM at consultation opening and written information at consultation closure. The primary aim of the COAC study was to develop and test the feasibility of a complex intervention designed to more comprehensively address patients’ concerns in general practice, thereby reducing re-consultation rates, improving patients’ well-being and health knowledge, reducing health concerns and increasing patients’ confidence in their health provision and health plan.

The COAC study incorporated two phases: an Intervention Development study (Study 1) and a feasibility study (Study 2) as follows:

Intervention Development Study: This involved design of a complex intervention to improve the ability of GPs or nurse practitioners to address patients’ concerns by a) incorporating the use of an electronic patient questionnaire consultation opening and b) providing a summary report at consultation closure, which is either printed or texted to the patient or is accessible from the patient record. These were designed and evaluated separately, in accordance with MRC guidance for design of complex interventions¹⁰.

Feasibility Study: In this study, the COAC intervention was tested in a cluster-randomised framework to establish the feasibility of both a randomised-control trial of the intervention and the intervention itself.

The sequential nature of the studies is shown in Figure 1.

Figure 1. The Intervention Development and Feasibility Studies.

2.3 Recruitment

2.3.1 Practice recruitment

Practices were approached by the NIHR Clinical Research Network for the West of England (hereafter referred to as the CRN) with the information on the study. Practices were recruited to the two phases separately, with practices who participated in the Intervention Development study actively encouraged to continue their participation in the Feasibility study.

For each study, the CRN shared the Research Information Sheet for Practices (RISP) which was developed for the study with a range of practices meeting the inclusion criteria. Interested practices then contacted the study chief investigator (CI) who arranged a meeting(s) with the practice manager or GP research lead.

Practice representatives were asked to sign a practice agreement consenting to the practice taking part in the study. Practices were approached for Study 1 in November 2019 (three practices were required for Study 1); and for Study 2 in May 2021 (six practices were required for Study 2).

All selected practices already used SMS software (MJOG and accuRx) and the patient record system EMIS. Administrators were expected to be familiar with the process of sending batch texts using practices SMS software (e.g. MJOG) and in uploading reports to and setting alerts in EMIS.

2.3.2 Eligibility criteria

For the Intervention Development study (three practices) we purposively selected: one practice in the top deprivation quartile, one at the median, and one in the lower quartile. For the feasibility study (six practices), we selected three practices in the top two deprivation quartiles and three practices in the bottom two.

Patients in the intervention development study were recruited to receive a summary report by GPs if they were:

▪      Aged 17 or over (on date of SMS invitation to participate)

▪      Had an appointment with that GP and the GP thought a summary report would be useful

▪      The patient agreed a summary report would be useful

In the feasibility study patients were recruited using these three same criteria, but additionally only if they had already completed a pre-consultation form.

The remainder of the methods are described separately for the Intervention Development study and feasibility study in the respective two sections that follow.

2.4 Intervention development study methods

2.4.1 Approach

The Intervention Development Study was carried out in two distinct parts, one for development of the online pre-consultation form and one for development of the summary report provided at consultation closure. Development of the summary report is described in this section. A prototype was developed based on the research literature and a series of patient and public involvement (PPI) group consultations. This was then tested with actual patients using a person-based approach, which involves using mixed-methods research to systematically investigate the needs, attitudes and situation of the people who will be using the intervention¹¹. Through the person-based approach, each step of the intervention was tested in rounds and adjusted after each round according to the feedback given from patients and clinicians. This iterative approach is shown in Figure 2.

Figure 2. Person-based approach taken to develop the summary report.

2.4.2 Prototype development

Starting position

Based on consultation with our PPI group we designed a report with four sub-headings as follows:

One Care, the Federation for GP Practices in BNSSG, developed an EMIS protocol, clinical template and document template that GPs could use to generate this report. EMIS is the GP patient record system used in BNSSG. The protocol allowed GPs to load a clinical template (which is a structured form) to allow them to input information under the four headings above. When the GP saved the template, a Microsoft Word report was automatically loaded that could be saved to the record and printed or sent to patients.

Initial programme theory

An initial programme theory of how the COAC intervention was intended to produce outcomes was designed. This was drafted by the study CI and reviewed by the study co-applicants and PPI group before finalisation. This is shown in Figure 3.

Figure 3. Proposed initial programme theory of COAC.

2.4.3 Patient recruitment

Patients were recruited to the intervention development study by GPs at the end of the consultation. The recruiting GPs explained to the patients that they were trying out providing a summary report of the consultation and asked the patient if they would like to receive this report. Along with the summary report, the clinician gave the patient an information leaflet about the study, with the researcher contact details. Patients were asked to contact the researcher if they were willing to be interviewed. In case the patients did not contact the researcher, a follow-up text was sent to all participating patients who received the patient information leaflet, reminding patients of the request to interview and asking them to respond “OK” if they were happy for their contact details to be shared with the researcher. The researcher then contacted these patients for interview.

2.4.4 Data collection / measures

Data collected in the intervention development study included clinician and administrator questionnaire data and qualitative interviews. Interviews were carried out by MM and AS and audio recorded. We aimed for 20 patient interviews, 6 GP interviews and 3 administrator interviews.

The purpose of these interviews was to inform development of the intervention through a person-based approach (which takes place in rounds, with the intervention changed at the end of each round). Topic guides therefore focussed on the feasibility and perceived usefulness of summary report, and on the proposed design of the intervention. Patient and GP topic guides are available as open access data (see data availability statement).

Prior to the COVID-19 pandemic, some interviews were conducted face-to-face in the patients’ own homes or other location of their choice, and GP/administrator interviews in the health centre. After March 2020 interviews were conducted by telephone or video link.

2.4.5 Analysis

Interviews were transcribed and analysed at the end of each round. The analysis focussed on establishing what changes were required to the summary report in that round before testing again in the next practice.

To do this, “guiding principles” were established. These are fundamental to the person-based approach, and highlight the objectives of the intervention and the key features that will address context-specific behavioural issues in support of these objectives¹¹. The guiding principles were drafted by the CI, adjusted by the PPI group and agreed by the study co-investigators. A coding framework for changes identified was then established (see Table 1). This framework contained codes to identify the reason for making each change, with reference to the guiding principles and the initial programme theory. After each interview was transcribed, one of two researchers (MM or AS) listed the possible changes arising from that interview and assigned a code to it. The other researcher then checked this and, if necessary, added new changes to the list or modified existing ones. The two researchers then discussed any areas of disagreement.

At the end of each round, the co-applicants all reviewed the table of changes and a final list for the round was agreed. The changes were implemented and the revised summary report was taken forward to the next round.

This continued for three rounds until a final version of the summary report was agreed.

2.5 Patient and Public Involvement

This research was informed by Patient and Public Involvement (PPI) both before the study commenced and during the study. PPI contributors received expenses and reimbursement in line with INVOLVE guidance¹².

The PPI group helped draft the person-based guiding principles, agreed the initial programme theory and made significant contributions to the structure, wording and look of both the pre-consultation questionnaire and the closure summary report. The group then met five times throughout the study. Two of these meetings were specifically to design the summary report as follows:

The PPI group also met at the end of the feasibility study to comment on the overall interpretation of the data and discussed how the results could be used in the future, either for additional research or how to benefit patients and clinicians in the future. The group assisted with drafting and approving the Plain English Summary for this paper and other publications.

2.6 Feasibility study methods

To provide some context for the realist evaluation, brief details are provided in this paper on the randomisation, recruitment and consent and data collection / measures of the feasibility study. This section mainly focusses on describing the data and analysis used for the realist evaluation embedded within the feasibility study¹³.

2.6.1 Randomisation and recruitment

In the feasibility study, both the pre-consultation form and summary report were used together in six practices, four randomised to intervention and two to control. Practices who had participated in the Intervention Development study were approached by the CI and new practices were approached by the CRN. Each practice was asked to recruit 30 patients (see Table 2) but GPs were asked to complete a summary report for patients only if it was useful. Because of the workload pressure GPs were under due to the COVID-19 pandemic, they were not given a specific target but were told that this could be more than 50% of patients or as few as 20%. Based on the intervention development study, the researcher expected an estimated 30% to receive a closure report.

Patients were recruited to the COAC intervention via an SMS message which contained a link for them to access the pre-consultation form. Patients who completed this form had it uploaded to the record by administrators so GPs could see it at the start of the consultation. During the consultation, GPs decided whether to give each patient a summary report. GPs were not given prescriptive criteria but advised that the form might be most useful when:

However, GPs were advised that the decision on whether to offer a report was an individual decision to be made by the GP on consultation with the patient.

2.6.2 Data collection / measures

Feasibility study data included clinician and administrator questionnaire data, interview data, and quantitative patient data. The quantitative patient data is described in the feasibility study linked paper¹³. Interview and questionnaire data were collected as follows:

Clinician questionnaire data: GPs were asked to complete a questionnaire row for each recruited patient. Within this questionnaire there was a question asking if clinicians had used a summary report and, if so, the reasons for this.

Interview data: Interviews in the feasibility study were conducted by MM and AS. Topic guides were designed to inform a realist evaluation and therefore focussed on the outcomes that patients/GPs perceived, the mechanisms by which these were achieved and the contexts. We aimed to interview patients and practitioners to the point of achieving “theoretical sufficiency”, i.e. when the data analysis has yielded one or more coherent theories which are relevant to the study aims¹⁴. Interviews were conducted by phone and audio-recorded.

2.6.3 Analysis

Realist evaluation seeks to explain the complex relationship between context, mechanisms and outcome. The explanatory proposition of realist evaluation is that interventions work (i.e. have successful outcomes) only in so far as the individuals involved take up ideas and opportunities (mechanisms) within the social and practical conditions in which they are operating (contexts)¹⁵. This is then reported in terms of contextual factors (What elements of the intervention work, for whom, in what consultations?) and content-mechanism-outcome configurations (CMOCs): a CMOC is a hypothesis that the program works to produce an outcome (O) because of the action of some underlying mechanism (M), which only comes into operation in particular contexts. (C)

The realist evaluation used the interview data collected in the feasibility study, supplemented with the interviews from the intervention development study. To carry out the realist evaluation, the CI (MM) read and re-read the initial interview transcripts from both patients and practitioners, in order to gain an overall view of the accounts given and to identify patterns in the data. She then revised the programme theory and devised an initial set of CMOCs. The research associate (AS) independently developed three lists of context, mechanisms and outcomes. These were cross-checked against the CI’s CMOCs which were then revised and detailed evidence presented against each of them. An experienced realist evaluator (GW), then read through the detailed evidence and the final CMOCs were agreed in collaboration. Four researchers (MM/AS/GW/CS) reviewed the realist evaluations findings and programme theory before finalising.

2.7 Consent

For the intervention development study patients gave consent to the GP to receive the intervention, and these patients contacted the researcher if they were happy to be interviewed. For the feasibility study, return of the questionnaire indicated consent to participate in the study. Patients were explicitly asked to consent to their contact phone number being shared with the University of Bristol for the purposes of sending a follow-up questionnaire. Further consent for use of that phone number to contact the patient for interview and for access to the patient’s record for demographics and re-consultation rates was requested in the follow-up questionnaire¹⁶. Patients were contacted for interview using the phone number they provided

For both the intervention development and feasibility study the researcher took informed consent for recording the interviews and use of anonymised quotations in publications prior to the interview itself. This consent was written for face-to-face interviews and audio-recorded for telephone interviews. Before the start of the COVID-19 pandemic, all consent was written. The ethics committee approved an amendment to collect audio-recorded consent for patients interviewed during the pandemic.

2.8 Sponsorship, funding and ethical arrangements

This study was sponsored by the University of Bristol. Ethics approval was granted by Frenchay Research Ethics committee¹⁷ and the Heath Research Authority (HRA). BNSSG Clinical Commissioning Group Research and Evidence Team provided research and development approval. The study was NIHR funded and supported by the NIHR Clinical Research Network who liaised with centres on the researchers’ behalf.

Insurance was provided by the University of Bristol as research sponsor. The study sponsor and funders did not have any role in study design; data collection, management, analysis, and interpretation of data; writing of the report; or the decision to submit the report for publication.

The Feasibility study was registered in the ISCTRN registry (ISRCTN13471877) and on the CRN portfolio (42005). The study protocol was published before recruitment completed¹⁸.

3.1 Intervention development

3.1.1 Participants

Table 3 shows the number of recruits to the Intervention Development study. Three practices were recruited from the top, middle and bottom of the index of deprivation (IMD) score, where 1 indicates a high level of deprivation and 10 a high level of affluence. Each practice had two participating GPs. One of these was an advanced nurse practitioner, but for simplicity and anonymity has been referred to throughout as a GP. 43 out of a target of 45 patients were recruited by GPs. We had intended to interview 20 patients from the 43 recruited but were only able to interview eight, as most patients did not contact the researchers for interview or respond “OK” to the text message asking if their details could be shared with the researcher. We therefore carried out an extended PPI consultation after the second round to augment the patient view.

The patient identifiers used herein are consistent with the feasibility study linked paper¹³, which has 50 unique patient identifiers. This is why some patient identifiers are higher in number than the overall number of patients we have data for (e.g. Patient 40).

3.1.2 Summary of changes

The person-based approach relies on a set of guiding principles. These were agreed in advance and informed the intervention development by highlighting the objectives of the intervention and the key features that will address context-specific behavioural issues in support of these objectives¹¹. (see Table 4)

Table 5 summarises the key elements of the consultation summary report that were changed over the 3 rounds. The full table of 18 changes, including verbatim quotes and coded rationale for making each change open access data (see data availability statement). As shown in Table 5, early versions of the summary report were presented in paragraph text. This was changed to a bulleted list which was easier for patients to read. The scope of the template was extended and it was made simpler for GPs to complete, with more prefilled tick-box text, which was customised for each practice. A structured PPI process was carried out to agree the details of the wording.

The process for generating the summary report changed substantially during the intervention development study. In the first version, GPs were provided with three different templates and selected one of these depending on whether they were ordering a diagnostic test, more than one test, or no tests. In contrast, the final version used a single template. GPs found this easier to use, because they only had get used to the look and feel of one form, and because they did not have to decide how many tests to order before opening that form. The first version also required a lot of free text entry, particularly for the “more than one test” template. In the final version, much of the report was generated by pre-filled tick-boxes, rather than free-text entry. This was quicker for GPs to complete.

A GP from site 1 (who were the first site to participate in the Intervention Development study and randomised to intervention in the Feasibility Study) commented that these changes were a substantial improvement.

This GP had felt during round 1 of the intervention development study that the pilot version of the summary report was too difficult to complete and she would not be happy to use it regularly. However, the final version used in the Feasibility Study was “much easier” to complete and she would be interested in using it with a subset of her patients.

Examples of the original and the final versions of the consultation summary report is shown in Figure 4 and Figure 5 respectively.

Figure 4. Consultation summary report: Pilot version (start of intervention development)*.

Figure 5. Consultation summary report: Final version (end of intervention development) *.

Figure 6 to Figure 10 show the five sections of the final EMIS form from the GP point of view, including where the form is pre-populated with standard adjustable text. Once the GP has decided to give the patient a summary report they launch the EMIS template from a shortcut key on their desktop. The EMIS template then loads on a single scrollable screen. The first section of this screen (Figure 6: advice given to patient) has three boxes, each for the GP to add a separate piece of advice. This first of these boxes is the only mandatory section in the template. The next section (Figure 7: medication changes) is for complex changes in medication and has two free text boxes, followed by a tick box containing the text “Continue with the rest of your medication as currently prescribed”. The third, and longest, section is for tests ordered (Figure 8: Tests ordered). In this section the GP can tick the names of the tests ordered, how the patient should arrange to get the tests done (pre-populated with the standard procedure for that GP practice), when and how the patient should expect to get the results and what to do if the results are abnormal or normal. The fourth section (Figure 9: safety-netting) is a single box for safety netting, populated with a standard statement that the GP can add more detail to if required. The final section (Figure 10: Next Steps) is for any other action, for example if the patient needs to book an appointment or a referral has been made.

Figure 6. Summary report EMIS template – general advice section (1).

Figure 7. Summary report EMIS template – medication changes section (2).

Figure 8. Summary report EMIS template – test requested section (3).

Figure 9. Summary report EMIS template – safety netting section (4).

Figure 10. Summary report EMIS template – next steps section (5).

The final agreed process of generating the consultation summary report and providing it to the patient is shown in Figure 11. The steps shown in this figure are as follows:

Figure 11. Consultation summary report generation: process diagram.

1.  The GP launches the COAC protocol from EMIS (This is done by pressing function->F12 and clicking “COAC protocol” in the window that appears.)

2.  The template shown in section 3.2.5 is loaded onto the GP screen. The GP completes the five sections. On saving, the codes get written to the patient record and the summary report is generated.

3.  The GP clicks on the summary report, which will open as a word document within the EMIS window. The GP presses Alt->G on the keyboard to run the macro to format the summary report. They make any other changes if they are required and save the report.

4.  Now the GP has three options:

5.  The patient receives the report. Depending on how the report was sent the patient correspondingly receives it via EITHER:

3.2 Realist evaluation

3.2.1 Participants

Thirty-two interviews were analysed. These interviews came from the following sources:

▪      Intervention development study: Eight patient interviews and six GP interviews.

▪      Feasibility study: Eleven patient and seven GP interviews.

Although we did not interview as many patients and practitioners as we had anticipated, the view of interviewees converged substantially, so we had sufficient “information power”¹⁹ to derive the new programme theory (see section 3.2.3).

The administrator interviews were not relevant for the summary report so were not included in the analysis. Interviews at each site in each phase are shown in Table 6.

In the analysis which follows, Patients 1 to 20 are from the intervention development study and patients 30 to 50 from the Feasibility study. So that the evolution of their views can be compared, the same identifier is used across the studies for GPs who were in both studies.

3.2.2 Summary findings from the process evaluation

The process evaluation of the feasibility study is presented in the linked paper¹³. A key finding of this evaluation was that the pre-consultation form and summary report are useful for different types of patients and consultation and each intervention results in different outcomes, triggered via separate mechanisms. It was therefore more appropriate to carry out a separate realist evaluation for the pre-consultation form and the summary report respectively than to update the initial joint programme theory which was shown in Figure 3.

3.2.3 Revised programme theory

Our analysis of the data from 32 interviews enabled us to revise the programme theory for the summary report. This is shown in Figure 12. This presents nine context-mechanism-outcome configurations (CMOCs) identified in the data in a single diagram showing interlinked context (what works), mechanisms and outcomes. Context is characterised differently across realist evaluations²⁰. We have interpreted context as anything in the environmental which interacts with (or ’triggers’) mechanisms that cause an outcome of interest that we have identified in our programme theory. Interventions are deliberately designed so that they change those aspects of context that are modifiable which in turn will ‘trigger’ the relevant mechanism(s) needed to produce a desired outcome. Examples of context captured in the programme theory include “what works” (how the interventional subcomponent needs to be implemented to alter context and thereby trigger mechanisms to achieve desired outcomes), “for whom” (contextual aspects of the patient that interact with mechanisms) and “in what circumstances” (contextual aspects of the presenting problem or consultation that interact with mechanisms). The mechanisms are processes that are ‘triggered’ by the context to cause outcomes. Some mechanisms are only activated for certain types of patients and consultations. This information on which types of patients and consultations are shown by the numbers in brackets in the green mechanism boxes. One of the outcomes is more distal than the others and for this, the context in which they are achieved is represented by another outcome in the programme theory (functioning as a context for that CMOC).)

Figure 12. summary report, revised programme theory.

3.2.4 Context (what works, for whom and what circumstances)

This section focuses on providing an overview of the key contexts which are needed for mechanisms to be activated. Details about the CMOCs in which these contexts function may be found in below in section 3.2.5.

3.2.4.1 What works

Key elements of the summary report part of the COAC intervention that worked well were:

1) The GP writing tailored advice in patient language 2) the GP discussing / co-creating the report with the patient 3) the bulleted format sent to patients soon after the consultation 4) the report being saved directly to the patient record.

The summary report worked best when GPs provided tailored advice to patients. The quotes below are from two patients who received the report from two different GPs in the same practice. GP1 tended to write more specific advice and GP2 briefer advice:

Patient 14 found her report clear. The GP had ticked the generic box “The surgery will contact you if your blood tests are abnormal” but added tailored safety-netting advice. Patient 15 found the advice unclear. His GP had provided only brief advice. This was one of only two patients who did not find the summary report helpful; he was recruited in round 1 of intervention development before multiple improvements were made to the template.

The other three elements that worked well are discussed within the individual CMOCs they apply to.

3.2.4.2 For what type of patient

GPs selected patients who might have problems remembering medical advice, where English was not their first language, people with health anxiety and people who shared medical advice with a relative.

One GP explained how she thought the report worked well on patients with health anxiety by removing uncertainty:

Patients who received the report explained why they felt it was useful for them:

These patients confirmed, without prompting, that the GPs had been correct in selecting them based on their own personal characteristics.

3.2.4.3 In what types of consultations

The summary report was most useful when the consultation was complex and the follow-up required multiple steps.

This GP used the form when there were multiple steps, especially if there was bad news which the patient might find it difficult to process or remember. Patients confirmed that the reports were most useful in complex consultations and would not be useful for simple consultations:

GPs and patients also felt the report was useful when the complex consultation was over the telephone:

However, GPs pointed out that, since COVID-19, they increasingly text patients. For a simple instruction it is much easier to use an SMS message rather than the summary report template.

The other two elements that worked well are discussed within the individual CMOCs they apply to.

3.2.5 Context-Mechanism-Outcome configurations (CMOCs)

Patient and family clear on safety-netting and follow-up required

CMOC 1:

When GPs are aware of the need to clearly communicate complicated follow-up to patients and write a tailored report in patient language (C) the patient and their family are clearer on the follow-up required (O) because the GP reflects more on how to plan and communicate this follow-up while they are completing the report (M). (Box 1)

Some GPs said the structured template made them reflect more on planning and communicating the follow-up:

The first GP felt that completing the report made them more careful about how they communicated within the consultation. The second GP said that, knowing the report might be referred to and shared with relatives, made her more specific about her safety netting advice, listing specific signs and symptoms to look out for.

Provided the report was written in clear language, it gave patients and their family clarity on the follow-up required. Some patients said that without the summary report they would not have been aware (or would not have remembered) this follow-up:

Other patients also described how they were clear on what to do next and what to expect from the GP.

CMOC2:

When a patient who has poor memory or likes to share medical information with family has had a consultation which is complex or involves multiple follow up steps receives a bulleted easy-read report soon after the consultation (C) the patient and their family are clearer on the follow-up required (O) because the patient has a memory aid to refer to or to share with their family (M) (Box 2)

The majority of interviewed patients said that the report functioned as a memory aid:

The quotes show different types of patient using the report as a memory aid. Patient 24 was an older patient whose memory was failing. Patient 17 had received complex information over the phone the summary helped her remember it. Patient 33 referred back to the specific medication advice a month later to verify it. Patient 40 was a younger patient without memory problems but felt that the context and intonation used by the GP in the consultation sometimes made her mishear or misinterpret what was said. Receiving a written report clarified the content of the conversation for her.

For the summary to serve as a memory aid, the immediacy of the report was important to patients:

GPs felt the report would help patients remember when a lot of information was provided:

One GP explained that her first line of management is often a “wait and see” approach where the patient is advised to reconsult if symptoms persist after a specified duration. This GP said patients often perceive this as no action taken, and either forget or do not process the advice. The summary report reminds these patients of the safety-netting plan and timings.

Some patients found it beneficial to share the report with family members. One interviewee attended the appointment with her father and received the summary report on his behalf.

The interviewee found it easier to discuss the information with her father when he was more relaxed and ready to process the information. This was particularly relevant during the UK COVID-19 lockdowns when families were unable to meet face-to-face. The bulleted formatting enabled a point-by-point discussion facilitating this conversation between patients and their relatives / carers. One GP explained that it helped her engage more with a patient when she previously might have had to engage with his spouse.

Without the summary report, this GP might have dealt directly with the spouse to ensure that patient’s safety. However, despite having a poor memory, the patient was able to understand health information if given space to process at his own speed, so the summary report was the GPs way of enabling him to take some charge of his own health.

Patient reassured

CMOC3:

When a bulleted easy-read report is given or sent to patient soon after consultation and this report reflects the patient’s recollection of the consultation (C) the patient is reassured (O) because the information provided is the same as what they thought would happen (M) (Box 3)

Patients found the consultation summary reassuring when it concorded with their recollection of the consultation. Two patients described this as follows:

GPs also felt that some of their patients found the summary report reassuring and had selected some patients on this basis.

This same GP said patients often called the reception for test results before they were ready. She felt having the summary in writing would reassure patients about the plan that had been agreed and they would be more likely to wait the specified time before following up. Another GP agreed with this and felt that this reassurance was particularly useful when the timeframe was weeks or longer. One GP described a three-week timeframe:

This GP had seen a patient with a fungal nail infection who needed to have his nail clippings sent for investigation. The GP believed that having the timescales in writing would reassure the patient about the expected timescales and lead to less phone calls to the practice.

Increased patient knowledge and empowerment

CMOC 4:

When patients are clear on the follow-up and safety netting outlined in the written report (Outcomes -> Contexts), the patient is empowered and their knowledge is increased (O) because the patient knows what they and the health care professional are responsible for and need to do (M) (Box 4)

Some GPs felt they were able to empower patients by providing them with an agreed written plan which the patient could then execute, particularly when this plan was largely the patient’s responsibility. One GP described a patient who had been advised to make lifestyle changes.

This GP felt that, having a bulleted list of six actions the patient needed to take would empower him to action this much more than if the information had been conveyed verbally. The GP described this as the patient “taking it more seriously.” Patients also felt that having the information in writing enabled them to execute it more easily. One patient described the report as creating a “target”:

This patient saw the document as a contract her and her GP, which she felt empowered to follow-up and challenge.

CMOC5:

When GPs discusses / co-creates the report with the patient (C) the patient is empowered and their knowledge is increased (O) because the GP and patient have agreed the shared plan together (M) (Box 5)

Some GPs discussed the report with the patient as they generated it in the consultation. One GP explained that she purposely used the language of shared decision-making in the summary in order to emphasise that it was a jointly agreed plan:

This GP often read the report to patients in the consultation to empower the patient to take ownership of the plan. The GP suggested that this might even improve her communication when she was not completing the report:

The COAC report gave this GP a mechanism for emphasising that the plan was a shared one they had made together, and she could potentially employ some of the same techniques even when not generating the report.

One patient described how the GP went through the report with her beforehand:

This patient valued that the GP reviewed the report with her beforehand and referred to the GPs advice as “recommendations” and “suggestions”.

CMOC6:

When GPs complete the EMIS template with sufficient detail in patient language and the report that is shared with the patient contains new or missing information or something they did not remember from the consultation (C), the patient is empowered and their knowledge is increased (O) because the patient can look up medical terms or seek clarification from the GP (M) (Box 6)

On some occasions the report either had something missing, or an additional element that the patient did not recall from the consultation. This meant that the patient could follow-this up, which increased their knowledge and empowerment. One patient had attended a consultation with her father. The report contained information about the tests and referrals but did not mention a form she had been expecting. This flagged that the GP had probably forgotten so she followed this up:

This patient felt that, had she not received the consultation summary a few hours after the consultation, when it was still fresh in her mind, she would have forgotten about the GPs promise to send a DNR form, but because she noticed the gap in the summary it reminded her to follow this up.

Some patients were able to look up information online as a result of receiving the report. One patient explained how he had a greater understanding of his cholesterol levels:

This patient had assumed that because he was prescribed medication that his cholesterol level was very high. When he received the consultation summary he realised that the GP had only written “slightly high”. He then looked up the levels on the internet which confirmed this and he was able to use this information to inform his decision about taking medication for his cholesterol.

Audit Trail

CMOC 7:

When GPs write the report with sufficient detail and it is saved to the medical record (C) this provides an audit-trail for medicolegal purposes (O) because the report can be referred to in case of a legal dispute (M) (Box 7)

Some GPs thought that the summary report might be useful for future for medico-legal purposes:

This GP felt that having the summary report saved to the patient record provided demonstrable proof of the advice given to the patient. In counterpoint to this, another GP suggested that if not well completed, the summary report could expose the practice medico-legally.

This GP was referring to the safety-netting advice, where GPs can tick the generic statement “Contact the surgery if your symptoms get worse or you develop any other symptoms of concern as we discussed. When we are closed you can call 111 to speak to an Out of Hours GP.” GPs were encouraged in the training to add more specific advice to this for example by listing the potential symptoms of concern. This GP was concerned that by adding these symptoms of concern the GP would be exposed if one symptom was missing.

More appropriate care pathway for patient

CMOC8:

When GPs are aware of the need to clearly communicate complicated follow-up to patients and write a tailored report in patient language (C) this can lead to a more appropriate care pathway for the patient (O) because the GP reflects more on the follow-up while they are completing the report (M) (Box 8)

Some GPs explained that, as well as helping the patients, the process helped their own thinking and made them clarify why they were taking particular actions.

These GPs explained that the summary report made them reflect on this in advance. The second GP considered the reasons for ordering the tests more and the likely next steps for the patient.

Some patients also felt that the summary report had led their GP to reflect more on the follow-up which led to a more “joined-up” care pathway.

The patient attributed the appointments being booked at the same time to planning and communication which was necessary to create the summary report. This may not be the case, as since the COVID-19 pandemic some GP practices are ensuring appointments coincide to reduce footfall in the practice²¹.

CMOC9:

When GPs write the report clearly, with sufficient detail, in patient language and the report and codes are saved to EMIS (C), this can lead to a more appropriate care pathway for the patient (O) because other GPs can refer to the report to see what was agreed (M) (Box 9)

Some GPs felt that the summary report could improve communication between GPs by providing a record of what was communicated to the patient.

This GP felt that the summary report would improve management continuity of care in practices where relationship continuity was difficult to maintain. One GP said she sometimes wrote paper notes for patients. While this might help the patient, it did not improve their management continuity:

Although the summary report could improve management continuity, it could have the opposite effect in the short-term if all GPs were not familiar with it:

This GP’s reflection highlights that the summary report would be best implemented as a practice-wide intervention if it is to improve communication between GPs.

4.1 Main findings

This paper reports on the person-based development of a consultation summary report and on a realist evaluation of this which was embedded within a feasibility study. The person-based development was highly successful. Numerous improvements were made to the summary report and GPs and patients agreed the final version was much improved on the initial version.

In the feasibility study the summary report was tested in a single intervention with a pre-consultation form. Through the embedded realist evaluation, we found that these were useful for different types of patient. The summary report is most useful for patients who have been given complicated follow-up and might not remember this follow up. It was only useful to patients if GPs completed it with enough detail. Although many patients felt that it would be useful to receive the report routinely, GPs felt it wouldn’t be sufficiently useful to patients to justify the time completing it in situations where patients did not have complex follow-up. Patients felt it was less useful when it was not competed with enough detail.

We identified five possible outcomes of the summary report which are captured in our finalised programme theory. The three outcomes with the most qualitative evidence were: 1) patients and family clear on follow up provided; 2) greater patient knowledge and empowerment and; 3) patient reassurance. There was also evidence that the summary report might help medico-legally and lead to more appropriate care pathways for patients because all practice staff can refer to the summary report to see what was agreed, although there were also concerns that the report could expose practices medico-legally.

4.2 Strengths and limitations

The person-based approach was an effective method of developing the consultation summary report. The PPI group were actively engaged and an important part of designing of the intervention. Practices were effectively re-engaged following the COVID-19 related study pause. An effective collaboration was developed with the GP Federation for BNSSG CCG (One Care) who were able to publish the required EMIS resources to the GP practices.

The realist evaluation is a well-established theory-based approach for making sense of why, when and for whom context-sensitive outcomes occur in complex interventions, such as the summary report. We analysed data across 32 interviews and used rigourous methods to analyse this data within a 3-person team. We had fewer patient interviews than planned across each phase. However, there was a high degree of concordance among patients on the use of the summary report and therefore we did have enough information power to develop the programme theory and have included additional quotes to demonstrate that the findings are grounded in the data. We also carried out more extensive PPI in the intervention development phase of the summary report to compensate for the lack of patient interviews.

4.3 Extended use of patient and public involvement (PPI)

Patient and public involvement was an essential part of developing the summary report. The PPI group shaped and refined the design of the pilot report in two meetings. Further to this, because of the difficulty we had eliciting wording preferences from our interviewees, we carried out an extended PPI consultation exercise on the wording of the standard text used in the template.

PPI differs from qualitative research in that the latter seeks to address questions relating to “why?”, “how?” and, “for whom?²² through collection and analysis of qualitative data, whereas PPI is used to improve the design and conduct of research, rather than providing data to answer research questions. PPI in research is carried out with or by members of the public and is a key part of the research process^23,24.

In the case of our wording-preference elicitation, the input of the PPI group overlapped into qualitative research. Unlike in pure qualitative research, our PPI group were engaged in the analysis and interpretation of data which they themselves had provided. The suggested structure, ‘look and feel’ of the report and much of the wording was generated or refined through PPI meetings. The PPI group generated data (preferred wording); this was analysed by the researcher and the outputs (wording choices) further considered and tested by the same group in later meetings.

This approach worked well for our study design. The critical importance of using PPI in all elements of the process of developing, testing, evaluating and implementing complex interventions is explained by Richards et al.²⁵. The developers of the person-based approach advocate combining PPI with qualitative research to generate a diversity of feedback, which can create more engaging interventions than would have been possible to achieve through PPI or qualitative approaches alone²⁶.

Other studies have usefully combined PPI when developing complex interventions, for example Best et al used PPI contributors to identify themes arising from qualitative data collected during complex interventions²⁷ and Morgan et al found combining PPI with qualitative research allowed inclusion of a wider range of views than would have been possible with qualitative research alone²⁸. Mann et al found that PPI positively affected a the design and implementation of a large randomised controlled trial, including changes to documents used in the trial and advice on qualitative data collection methods and analysis²⁹.

4.4 Summary report comparison with literature

There is little literature on producing a summary report for patients in UK primary care. A 2020 realist evaluation of discharge letters in a secondary care setting showed that, if patients understand their letters this can lead to patient empowerment and improved patient knowledge and recall³⁰. However, hospital discharge letters are primarily for the GP, although the patient may be sent a copy. They therefore contain medical jargon and acronyms. Some patients and GPs agreed that the patient would have benefitted from use of lay terms in the letter³⁰.

An important feature of the summary report is the documentation of safety-netting advice. A 2021 study of 295 GP consultations showed that over two-thirds involved spoken safety-netting advice, but that this advice was documented in only one third. The practice of GPs varied widely, from those that did not document their safety-netting advice to those that nearly always did so. The authors suggested this lack of documentation could have medico-legal complications in the event of an untoward incident³¹. Use of the COAC summary report would improve this documentation in the case of complex patients and consultations and the interviewed GPs suggested this might have medico-legal uses, although one GP pointed out it could have adverse medico-legal consequences if safety-netting advice was missed. In the US, failure to notify patients of test results has been identified as a relatively common occurrence, particularly when there is not a systematic process identified for doing this^32,33. GPs commented that the COAC summary report helped them to reflect on the follow-up, and patients that it gave them an agreed record of this follow up, which should help with future notification of test results. The literature shows that review templates for long-term conditions can improve documentation and act as a reminder but have sometimes acted as a barrier to provision of patient centred care³⁴. There was no evidence that this happened in this case; indeed both patients and GPs thought the template helped make the consultation more patient-centred, perhaps because it was selected for patients with complex consultations who were most likely to benefit from it. Some GPs changed their consultation style to write the report with the patient, sometimes asking the patient to repeat back to them. This kind of “teach-back” has been found to be effective across a wide range of settings, populations and outcome measures³⁵. The pandemic has shown the value of GPs working together within primary care networks (PCNs). Practices are merging and the average practice size increasing; patients both seeing and having a preferred GP has reduced over recent years as relational continuity has consequently declined³⁶. Putting what was said to the patient on a record which can be shared among GPs may be one simple way to improve informational continuity within general practice.

Sharing information through the medical record

Some patients thought the summary report should be made available through the medical record via the “patient access” portal. UK primary care patients have, in theory, had access to their medical record through this portal since 2015. The government committed that all patients would have access by 2018³⁷, but in a 2020 study on patient access to the medical record, only 18.3% of patients used this access to view their medical record. 75% of these users placed a high value on the access. The study authors noted that the low levels of access may be due to the permissions of the GP practice rather than the preferences of patients³⁸. GPs have previously expressed concerns that patient access to records could increase risks of litigation and require them to change the way they write on the records so that patients can understand them³⁹. Patients have expressed a need for support and training in using and understanding the online record⁴⁰. A systematic review of patient record sharing in UK secondary care found that the approach of giving information to patients almost exclusively verbally was insufficient; and patients should have access to notes, but that simply allowing full access, without explanation or summary, is also insufficient⁴¹. The COAC summary report may facilitate this shift in recording information in the medical record as it allows a patient-friendly report to be generated and stored alongside more technical notes. As the RCGP guidance on medical record sharing states, writing certain sections of the medical record more clearly and in lay terms will be of benefit to both patients and clinicians, but technical medical information is often necessary for patient safety⁴².

There is evidence that sharing information with patients through their medical record can lead to some of the outcomes we identified in our programme theory. For example, a US-based study found that sharing notes with patients helped engage them to improve record accuracy and health care safety together with practitioners⁴³. We similarly noted that sharing the summary report led to conversations with the patient and GP following the consultation to clarify the advice. Educating patients with timely medical information through their smartphones or tablets has been shown to improve their levels of knowledge, medication or treatment adherence, satisfaction, and clinical outcomes⁴⁴. This resonates with our qualitative findings that the summary report improves knowledge and patient empowerment. Although we did not have sufficient qualitative evidence to suggest clinical outcomes would be improved, it is entirely possible that they would be over a period of time.

Underlying data

Researchers can apply for this data via a form on the repository:

University of Bristol: COAC Study Qualitative Dataset, https://doi.org/10.5523/bris.1ljvagu1sigje2duqj3ube527y (restricted access)⁴⁵.

This project contains the qualitative data transcripts for the COAC Feasibility Study, where participants agreed that these could be shared with bona fide researchers outside the Bristol research team. Information about each transcript is listed below, as follows:

Transcript ID: The name of the transcript in the folder. The name consists of:

* a participant identifier

* the type of participant (patient, clinician or administrator)

* the site (1 to 4 – this was not reported in the paper for reasons of anonymity)

* The date of the interview

Participant identifier used in papers: This is the identifier used in this paper.

The folder also contains the consent form. All patients in this study consented to point 7 in this form: "I understand that after the study my anonymised data will be made available to bona fide researchers for future research studies, and it will not be possible to identify me from these data. If I agree to this, my data will be held for twenty years."

This dataset has an access level Restricted, which means it is not available via direct download but must be requested. Research participants did not give explicit consent to share this data as open data but agreed that it should be made available to approved bona fide researchers only, after their host institution has signed a Data Access Agreement. In order to request access to this data please complete the data request form available from the link above. We will consider any application from any organisation where an established research governance process is in place.

Data are available under a Non-Commercial Government Licence for public sector information.

Extended data

University of Bristol: COAC Study Extended Dataset, https://doi.org/10.5523/bris.386dsq2e4iii225ms7du8pd5jq⁴⁶.

This project contains the following extended data:

This file contains screenshots of the pre-consultation form which patients responded to in the COAC Study.

This file contains a detailed table of changes made to the pre-consultation form in the COAC Intervention Study. Patients who are quoted in this table all consented to the first six points in the consent form included in this folder.

This file contains a detailed table of changes made to the summary report in the COAC Intervention Study. Patients who are quoted in this table all consented to the first six points in the consent form included in this folder.

This file contains the interview topics guides for the COAC Study.

This is the patient consent form used for the COAC Study

This is the patient information leaflet given to patients interviewed for the COAC Study

This is a checklist for the COREQ reporting guidelines which demonstrates how they were following in collecting and analysing data about the pre-consultation form

This is a checklist for the COREQ reporting guidelines which demonstrates how they were following in collecting and analysing data about the summary report.

Reporting guidelines

University of Bristol: COAC Study Extended Dataset, https://doi.org/10.5523/bris.386dsq2e4iii225ms7du8pd5jq⁴⁶.

This paper has followed the Consolidated criteria for reporting qualitative studies (COREQ) checklist⁴⁷.

The realist evaluation also followed the RAMESES II reporting standards for realist evaluations⁴⁸. See extended data⁴⁶ for checklists.

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).