Protocol: Remote care as the ‘new normal’? Multi-site case study in UK general practice

Background Following a pandemic-driven shift to remote service provision, UK general practices offer telephone, video or online consultation options alongside face-to-face. This study explores practices’ varied experiences over time as they seek to establish remote forms of accessing and delivering care. Methods This protocol is for a mixed-methods multi-site case study with co-design and national stakeholder engagement. 11 general practices were selected for diversity in geographical location, size, demographics, ethos, and digital maturity. Each practice has a researcher-in-residence whose role is to become familiar with its context and activity, follow it longitudinally for two years using interviews, public-domain documents and ethnography, and support improvement efforts. Research team members meet regularly to compare and contrast across cases. Practice staff are invited to join online learning events. Patient representatives work locally within their practice patient involvement groups as well as joining an online patient learning set or linking via a non-digital buddy system. NHS Research Ethics Approval has been granted. Governance includes a diverse independent advisory group with lay chair. We also have policy in-reach (national stakeholders sit on our advisory group) and outreach (research team members sit on national policy working groups). Results (anticipated) We expect to produce rich narratives of contingent change over time, addressing cross-cutting themes including access, triage and capacity; digital and wider inequities; quality and safety of care (e.g. continuity, long-term condition management, timely diagnosis, complex needs); workforce and staff wellbeing (including non-clinical staff, students and trainees); technologies and digital infrastructure; patient perspectives; and sustainability (e.g. carbon footprint). Conclusion By using case study methods focusing on depth and detail, we hope to explain why digital solutions that work well in one practice do not work at all in another. We plan to inform policy and service development through inter-sectoral network-building, stakeholder workshops and topic-focused policy briefings.

compare and contrast across cases.Practice staff are invited to join online learning events.Patient representatives work locally within their practice patient involvement groups as well as joining an online patient learning set or linking via a non-digital buddy system.NHS Research Ethics Approval has been granted.Governance includes a diverse independent advisory group with lay chair.We also have policy in-reach (national stakeholders sit on our advisory group) and outreach (research team members sit on national policy working groups).

Results (anticipated)
We expect to produce rich narratives of contingent change over time, addressing cross-cutting themes including access, triage and capacity; digital and wider inequities; quality and safety of care (e.g.continuity, long-term condition management, timely diagnosis, complex needs); workforce and staff wellbeing (including non-clinical staff, students and trainees); technologies and digital infrastructure; patient perspectives; and sustainability (e.g.carbon footprint).

Conclusion
By using case study methods focusing on depth and detail, we hope to explain why digital solutions that work well in one practice do not work at all in another.We plan to inform policy and service development through inter-sectoral network-building, stakeholder workshops and topic-focused policy briefings.

Plain English summary
The pandemic required general practices to introduce remote (phone, video and email) consultations.That policy undoubtedly saved lives at the time but there are also clear benefits of face-to-face consultations in some circumstances, and the exact role of remote care still needs to be worked out.Despite best efforts, remote care tends to worsen health inequities (people who were poor or less well educated are less able to access and navigate the system and secure the type of appointment they need or prefer).
Workstream 1: We will look at 11 GP surgeries across England, Scotland and Wales.We have selected a variety of sites: urban and rural, serving a range of different communities.Each surgery has a different approach to technology.A researcher from our team will work alongside surgery staff to learn what methods and technologies each practice uses to deliver care.They will gather information (mostly qualitative) about how different technological solutions are playing out over time.
Workstream 2: Many people experience barriers to accessing care when it is done through technology.This could be because they lack understanding of how to do it, don't have the right equipment, can't

Plain english summary
The pandemic required general practices to introduce remote (phone, video and email) consultations.That policy undoubtedly saved lives at the time but there are also clear benefits of face-to-face consultations in some circumstances, and the exact role of remote care still needs to be worked out.Despite best efforts, remote care tends to worsen health inequities (people who were poor or less well educated are less able to access and navigate the system and secure the type of appointment they need or prefer).
Workstream 1: We will look at 11 GP surgeries across England, Scotland and Wales.We have selected a variety of sites: urban and rural, serving a range of different communities.Each surgery has a different approach to technology.A researcher from our team will work alongside surgery staff to learn what methods and technologies each practice uses to deliver care.They will gather information (mostly qualitative) about how different technological solutions are playing out over time.
Workstream 2: Many people experience barriers to accessing care when it is done through technology.This could be because they lack understanding of how to do it, don't have the right equipment, can't afford data, or other reasons.We will ask patients about their experiences and work with them and staff to develop ideas about how to overcome barriers.Workstream 3: We will take what we have learnt in Workstreams 1 and 2 to make suggestions to inform national stakeholders and to influence policymakers.
Patients and members of the public helped shape the research design.They continue to help guide our research by reading our reports, giving us their opinions and advising on how best to share our research so everyone can benefit from what we have learnt.Our governance panel is chaired by a member of the public.

Remote general practice during the pandemic
The COVID-19 pandemic has been a crisis opportunity for digital innovation 1 .In early 2020, remote-telephone, video and electronic-consultations were quickly introduced into UK general practice 2,3 .For various reasons, video was little used even at the height of the pandemic 4,5 .Practices introduced telephone and in some cases online consultations (patients complete a web template and receive email reply or call-back 6 ).Implementation challenges were common 5,[7][8][9][10] , and practices worked hard to retain a face-to-face service for vulnerable and complex patients 11 .In July 2020, the UK's Secretary of State for Health declared-prematurely as it turned out-that remote would be the default option for the indefinite future 12 .
The shift to remote general practice was initially supported as a 'heroic' response to COVID-19 but later questioned as unsatisfactory and potentially unsafe 13,14 .Commentators raised concerns about access, continuity of care, diagnostic errors, loss of the 'doorknob consultation' (in which a patient raises a serious concern only as they are leaving 15 ), safeguarding challenges, and unsafe prescribing [16][17][18][19] 20 .
Whilst our own in-pandemic research found examples of high-quality remote care [8][9][10] , we affirmed these concerns and identified six new kinds of risk: a) practice organisation and set-up (digital inequities which restricted access, technologies that were unreliable and unfit for purpose, and reduced service efficiency); b) communication and the therapeutic relationship (a shift to more transactional consultations); c) quality of clinical care (including missed diagnoses, safeguarding challenges, over-investigation and over-treatment); d) increased burden on the patient (e.g. to self-examine and navigate between services); e) fewer opportunities for screening and managing the social determinants of health; and f) adverse impact on workforce (clinician and staff stress and compromised learning) 21,22 .
Remote general practice before the pandemic Until early 2020, remote general practice consultations had been technically possible but (with the exception of telephone triage 23 ) not widely used in UK [24][25][26] .Pre-pandemic research on telephone 23,[27][28][29] , video 26,30,31 and online consultations [32][33][34][35] was typically couched in an efficiency narrative and dominated by randomised controlled trials or quasi-experimental designs in which success was measured in economic metrics such as consultation length, number of problems raised, number and type of follow-up encounters, and by 'non-inferiority' in clinical outcomes and patient and staff satisfaction 23,26,27,31,36 .
Efficiency and satisfaction are important concerns, but this early literature focused more on remote consultations under controlled conditions than on the wider question of introducing remote care as a service.It showed, broadly speaking, that remote modalities were acceptable, safe and cost-effective in the circumstances studied.But this research rarely demonstrated the hoped-for improvements in service efficiency-indeed, they often showed that remote modalities reduced efficiency as a result of double-handling or more service contacts 23,[27][28][29]32,37,38 . Some prepandemic studies had revealed remote-associated compromises to quality of care such as increased antibiotic prescribing 39 .A sparse and somewhat speculative literature promotes remote services as a means of reducing greenhouse gas emissions (e.g. from traveling to appointments) [40][41][42] , though this literature rarely considers the unintended environmental effects if remote services over-diagnose, over-investigate, over-prescribe, over-refer or result in missed diagnoses and future emergency admissions.
The strengths and limitations of large-scale quantitative studies for evaluating remote service models were illustrated by a study of 'telephone first' (in which all patients first speak to a clinician and some are invited to attend in person) using mainly quantitative methods with a small qualitative component 43 .Whilst, on average, telephone first led to an 8% increase in clinician workload with similar patient satisfaction and service usage to traditional models, there was huge diversity-with some practices reporting improved efficiency and access and others reporting the opposite.The authors commented that their methodology was not designed to explore how or why multiple interacting factors played out differently in different settings.Similarly, a rapid evidence synthesis of 'digital first' studies found that most had "very narrowly evaluate [d] the introduction or use of a class of technology (e.g.internet video consultation), rather than the integration of such technologies as part of a broader reorganisation or reimagining of services" (page 7), and that despite extensive primary research, "little evidence exists on outcomes related to quality of care, service delivery, benefits or harms for patients, or on financial costs/cost-effectiveness." 35 With few exceptions, then, pre-pandemic studies comparing remote with conventional appointments lacked descriptive detail and nuance.A sparse literature of qualitative and mixed-method case studies had begun to document technical, logistical and regulatory hurdles to digital general practice 25,32,35,[44][45][46] .The pandemic provided impetus for widespread organisational change at pace and scale, supported by dedicated funding and relaxing of red tape 1,2,5,[8][9][10]47 . Butwhilst these are excellent preconditions for innovation, sustaining such innovations long-term raises new challenges and is considerably more difficult 9,48 .

Digital inequity-a new component of inverse care
Digital inequity means unequal access to healthcare resulting from poor digital access, digital literacy or both 49 .It tends to affect those with multiple other kinds of disadvantage such as poverty, low health literacy, poor housing, weak social networks, psychological stress (e.g. from fear of crime) and-for some-language and cultural discordance, which together may increase their vulnerability to illness, disease and disability 50 .Tudor Hart's inverse care law (people most in need of health care are least likely to seek it or receive it) reflects two mutually-reinforcing phenomena: worse health in deprived communities and also barriers to their access to healthcare 51 ; such inequities have worsened recently 52,53 .SARS-CoV-2 produced a syndemic as well as a pandemic -i.e. it exacerbated, and was exacerbated by, social and economic inequities 54 .
The proportion of the public classed as "internet non-users" has fallen but there remain substantial inequities by social determinants such as geographical location, age, ethnicity and gender 50,55,56 .The digital divide operates not just in terms of basic internet access but in terms of how much bandwidth, data bytes, connectivity, compatibility, confidence, skills, power (e.g. over who in the household has use of a computer or smartphone) people have, and the size and nature of the social networks they can draw on for assistance 57 .Even basic technologies such as the telephone can exclude some individuals (e.g.hard of hearing, geographically off grid).
The NHS Widening Digital Participation programme 2017-20 aimed to ensure equity in access and care regardless of digital preferences 58 .An independent evaluation 59 inspired various proposed solutions including raising awareness, digital skills training, digital champions, intergenerational mentoring, free public Wi-Fi, assistive technology and social prescribing 60 .A qualitative systematic review recommended using diverse ways of raising awareness and inviting (e.g.online, paper, word of mouth); proactive outreach (e.g.working through agencies); partnering trusted professionals (e.g.GPs); and checking that digital interventions meet people's needs 61 .A paper on digital inclusion in the homeless talked of "assertive outreach" partnering with public and third-sector agencies 62 .
These recommendations have influenced our study design.But we believe a 'deficiency' framing (patients depicted as lacking devices, data, connectivity, awareness, skills, confidence and support, all assumed rectifiable by interventions) overlooks the pervasive impact of multiple interacting social determinants 50,63 .We hypothesise that non-digital options, easily accessible in traditional ways, will be needed for some patients.Such options are often offered on an ad hoc basis at the discretion of individual staff rather than as agreed policy.Access arrangements for some groups (e.g.disability, pregnancy) are protected under the UK's Equality Act (2010) which requires "reasonable adjustments", but people who are-for example-just poor or with complex needs (such as drug or alcohol problems or victims of domestic violence) do not have the same level of legal protection.
The need for detailed, in-depth case studies In sum, whilst remote consultations have clinical potential in general practice, remote services are difficult to set up, technically challenging, may increase workload at a stressful time, and could worsen health inequities.Despite much research, remarkably little is known about the fine-grained detail of implementing and sustaining remote services in different general practice contexts.As Flyvbjerg has put it, "a scientific discipline without a large number of thoroughly executed case studies is a discipline without systematic production of exemplars, and ... a discipline without exemplars is an ineffective one" (page 219) 64 .
Mindful of this gap in the literature, we sought to study a small but diverse sample of cases in depth to produce rich explanations of complex phenomena and generate lessons from the similarities and contrasts between them.

Aims, objectives and research questions
These are summarised in the flowchart in Figure 1.

Study design
Mixed-methods multi-site case study with co-design and national stakeholder engagement.The study has three workstreams.Workstream 1 will use an embedded researcher-in-residence model to develop a multi-site longitudinal case study of general practices.Workstream 2 will capture patient experiences and use co-design with patients and staff to re-imagine service models and address digital inequities.Workstream 3 will engage national-level stakeholders and build networks for disseminating outputs.

Workstream 1: Case studies in general practice
The goal is to produce detailed exemplars of complex change at practice level.We have recruited a sample of 11 general practices (Table 1) for maximum variety in geographical location (six in England, two in Scotland and two in Wales), list size (from 2,800 to 33,500), practice demographics (extremely deprived to moderately affluent, skewed towards the former), ethos (varying, for example, in the value placed on own-doctor appointments), and digital maturity.
Our digital maturity scale (Table 2) is simple and pragmatic.It draws on earlier work including a 136-item digital maturity self-assessment survey 65 which informed the NHS Five-year Forward View 66 but was abandoned soon after; a digital maturity matrix for electronic records 67 ; and the IDEAL framework for surgical innovations (typically technologies) 68 .Our scale incorporates practices' readiness (strategic alignment, leadership and resources), capability (remote services up and running) and infrastructure (the underpinning material, regulatory and human resources needed to accommodate new technologies and work routines).As Table 1 shows, two practices currently self-classify as 'traditional' (lowest level of digital maturity) and one is already 'system-oriented' (highest level).Each practice has a researcher-in-residence, tasked with becoming familiar with its context and activity over two years (see Figure 1 for data sources).Beginning with informal interviews with a lead clinician or practice manager, they will arrange further interviews with practice staff and invite them to provide documents (e.g.practice leaflets, annual reports, audits).Case studies will be built iteratively and adaptively, depending on experiences and priorities salient locally.The researcher-in-residence will maintain a single point of contact for the practice, engage and work with their patient representatives, and keep the practice informed of activities (such as virtual workshops) and new resources (e.g.guidance) as these appear.Where appropriate and invited, researchers-in-residence will attend practice meetings (either face-to-face or via video link).
Staff interviews will combine a few basic semi-structured prompts (including "tell me about your job", "what is your experience of remote access and remote consultations in the practice?")with open-ended narrative probes (e.g."can you tell me more about that?", "what happened next and how did everyone react?" and "could you give me a story to illustrate that?").Narrative interviews are conversational in format, seeking context and descriptive richness through examples; they are particularly useful in identifying emotive touchpoints and going beyond superficial description to capture meaning and significance 69 .
Subject to pandemic restrictions, researchers-in-residence will undertake ethnography, focusing on what Star called the 'ethnography of infrastructure' 70 .We will use strategic ethnography: a focused and historically-informed approach to data collection and analysis that considers the biography of artefacts in organisations-that is, how they emerged, their interrelationships and inter-dependencies, and what they are becoming 71 .Artefacts (most obviously, the hardware and software for phone, video and online consultations and triage, along with what some authors have called 'in-between' artefacts such as sticky notes, whiteboards and informal note systems 72 ) are considered as evolving components of a complex, dynamic system, with multi-sited ethnography providing "robust, contexted understandings of complex objects" (page 527) 71 .
Limited quantitative data (e.g.staffing levels, uptake data on different kinds of appointment) will be incorporated where relevant as part of an evolving longitudinal story of a changing practice in a changing context.
Research team members will meet every 2-4 weeks to compare and contrast across evolving cases, focusing on the key cross-cutting themes shown in Table 3 (which have emerged from the literature as well as from our early familiarisation interviews) and connect with other work packages.
We will apply the principles of action research 73 (taking an iterative and collaborative approach with practice members; establishing locally-appropriate ways to rapidly evaluate and feed into learning; and seeking participation and buy-in from staff and patients) to support each practice in its efforts to learn and develop around four key goals: a) optimising quality and safety of care; b) ensuring digital inclusion and providing equitable alternatives for the digitally excluded; c) addressing In a sub-sample of three practices (Camp St, Fernleigh and Towerhill), we will undertake detailed ethnography of front-desk and back-office work on tasks such as appointment booking, call handling and triage, with a view to teasing out key workplace routines and exploring their interdependencies and implications.We will supplement this with analysis of routine practice data on use of different consulting modalities over 24 months to appreciate service use and changes over time.
In another sub-sample (yet to be identified), we will calculate the carbon footprint of a sample of consultations and linked patient pathways (e.g.referrals, investigations), mindful that these pathways may differ in remote versus face-to-face consultations.We will explore how practices are incorporating

Theme Rationale Approach
Access and triage 'Total triage' (all initial contact to be by telephone and electronic form), introduced in March 2020 3 , has evolved in various forms since.In many practices, current systems are experienced as inefficient and hard to navigate 5,9 .
Using interviews, patient information resources and digital walk-throughs, we will map the patient pathway through the ' digital front door' and follow each practice's efforts to improve accessibility and efficiency.

Digital and wider inequities
Quantitative studies have shown inequities by age, gender and ethnicity in remote consultation uptake 74 , but those designs did not allow in-depth analysis of intersectionality-how different social determinants (e.g.being elderly and poor and chronically sick) combine and interact.
Using narrative interviews and ethnography, we will capture the complexities of disadvantage and exclusion for particular groups.Using co-design, we will develop digital personas to serve tools for re-imagining service provision.

Quality and safety of care
Remote care may compromise the therapeutic relationship and continuity of care 19,20 , lead to more transactional forms of clinical interaction 22 , fewer ' doorknob consultations' 15 , and delayed diagnosis of serious illness 75,76 ; it may be unsuitable for those with complex needs [20][21][22] .Remote reviews may be convenient and safe for patients with stable long-term conditions 77 .
Using staff and patient interviews, practice documents, ethnography, and video/audio of consultations (if feasible), we will explore how quality goals are achieved (or why they are not achieved) for different conditions and patient groups-including long-term conditions, multi-morbidity, early diagnosis of cancer, and vulnerable patients.
Workload, workforce and staff wellbeing UK general practice is under system stress 78 , with high and rising workload 79 , task shifting from other sectors 80 , a retention crisis 81,82 , and high levels of stress and burnout among clinicians, trainees and administrative staff [83][84][85] .Remote consulting is cognitively demanding 86 and may reduce opportunities for learning 87 .Trainees report low confidence in assessing patients by telephone 88 .
Workload and wellbeing will be a key focus of staff interviews and learning sets.Sub-studies will explore front-desk and back-office work routines ethnographically; study experiences of under-researched lower-grade staff (receptionists, administrators, cleaners) and those of trainers and trainees.

Technologies and their associated infrastructure
There is a longstanding policy push to strengthen NHS digital infrastructure [89][90][91][92] .Some technologies developed during the pandemic bypassed regulatory approvals 1,93 .
In some cases, products approved at speed at the height of the pandemic subsequently proved unfit for purpose.The procurement process for new technologies was sometimes poorly aligned with business cycles.
Using interviews, ethnography, digital walk-throughs and analysis of relevant national and local IT policies, we will study both the novel technologies and the material and digital infrastructure, human resources, technical expertise and business decisions needed to support and troubleshoot technology adoption and use.

Patient input to practice improvement
Drawing on patients' experience (and trying to improve it) is a well-established method for service improvement 70 .with strong theoretical grounding in phenomenology 94 .
Many but not all general practices have established patient participation groups.
Researchers-in-residence will work flexibly with each practice and (where established) local patient involvement groups, incorporating additional insights from patient interviews, our patient learning set, and lay input to stakeholder workshops.

Planetary health
Travel to healthcare appointments (e.g. by car) generates greenhouse gases 95,96 .Remote service provision could potentially reduce this, though carbon savings in primary care may be modest, and could be achieved at the expense of waste (e.g.over-diagnosis, over-treatment or over-referral).Local savings (of various kinds) may come at the expense of 'hidden' environmental waste (e.g.data warehousing).
We will calculate the carbon footprint of a sample of consultations and linked patient pathways in some practices.We will explore critical events with potential for adverse carbon impact (e.g. when patients are sent for tests rather than being examined face to face).We will explore the extent to which environmental sustainability is (or could be) built into practice business cases.
(or why they are not incorporating) sustainability considerations into their strategies and business plans, and in decisions about how and why different consultation modalities are (or aren't) being used.
Mindful of the established value of inter-organisational networking and support in complex change 48 , we will offer practices a series of webinars, link them to a range of resources (e.g.clinical standards and guidance, patient resources) as we develop these, and set up an e-mail discussion list for key practice contacts.

Workstream 2: The patient perspective and co-design
The goal is to support inclusion of patient and carer perspectives in the design and redesign of remote services.Sampling 40 participants, we will seek to maximise diversity in age, socio-demographic background, ethnicity, housing status (e.g.homeless or 'sofa-surfing', privately rented, owner-occupied), digital literacy, confidence, and nature of illness or condition(s).
We will ask practice staff to nominate patients and work with practice patient involvement groups and patient advocacy groups external to the practice (who often have strong online presence but also well-established ways of reaching less digitally confident members).We will also snowball from participants, asking them to nominate and 'buddy' a friend or relative (e.g. a young second-or third-generation immigrant from a minority ethnic group may be able to connect us to a grandparent who speaks limited English and limited digital experience or access).
Working both locally and at the level of national advocacy groups will allow us to include a perspective on what it is like being cared for in participating practices as well as a more generic patient voice for certain conditions.We will ask practices, patient groups and snowball contacts to identify people who they think may have found it challenging to consult remotely, as well as those who are keen and confident to help advise and support others.Carers of people unable to give a full account of their own experience (e.g.cognitive impairment) will also be included in the sample.We will note advice given by one of our patient advisers that people may be very digitally literate on certain platforms (e.g.Facebook) but less so on others (e.g.online consultation forms).
Potential patient participants will first be approached by someone outside the research team (practice staff or fellow patient).Participation is voluntary; they can withdraw at any time and personal details will be anonymised.They may choose video, telephone or face-to-face format (e.g.homeless people will be interviewed at lunch clubs in a private space).Interviews will combine basic semi-structured prompts (e.g."how long have you been a patient in the practice?","what illnesses or conditions do you receive care for?", "what is your experience of booking and having consultations remotely?")with and narrative probes (conversational, seeking examples and depth for whatever the patient chooses to talk about).
Interviews aim to capture the patient and carer experience of remote services across four key quality and safety areas (long term condition monitoring, getting an appointment with own clinician and maintaining continuity of a therapeutic relationship over time, presenting with symptoms that could indicate early cancer, and care for multimorbidity and other complex needs).
Findings will be fed into digital inclusion co-design workshops-two with patients and carers, and two with practice staff (along with patient representatives), described in Box 1.

Box 1. Inclusive digital transformation
A non-profit digital co-design agency (Thrive by Design), with an interest in digital inclusion 59 will use a validated action research methodology for supporting inclusive digital transformation.
Thrive by Design will begin by working with three participating practices to run digital inclusion co-design workshops using the guiding question: "How can we best provide safe and effective care through remote consultations, and what measures do we need to put in place for people for whom standard remote consultations are unsuitable or unacceptable?".
One output of such workshops will be a range of digital inclusion personas.These are fictional characters who encompass features we need to think about when selecting technologies and designing and embedding technology-aided services (e.g.Fred is a 35-yr-old heroin addict living in cardboard city who gets his methadone from an NHS general practice) 97,98 .
Working across the three initial practices, the first co-design workshop will be held with patients and carers, either virtually or in-person.People less comfortable with the virtual format will be supported to contribute using telephone in an asynchronous format (i.e.building a picture over several days/weeks).The personas and wider insights generated by these patient and carer workshops will be used to inform and enliven two additional workshops for practice staff (including clinicians, managers, administrators and patient and lay representatives across the three practices).Preparatory briefing materials will be sent out beforehand.Participants will work partly in virtual breakout rooms to think creatively about meeting the needs of the different digital inclusion personas.
The outputs of these workshops are unlikely to be simple or universal solutions.We anticipate they will generate ideas for how (and for whom) to deploy existing remote technologies, additional off-the-shelf or bespoke products which could enhance provision, and novel service models.The format will be extended to other participating practices as the study unfolds.

Workstream 3: National stakeholder engagement and dissemination work
We use 'élite' national stakeholder interviews for two purposes: to gather data on the macro-level policy, infrastructural and regulatory context including public-private partnerships, financing and reimbursement and so on; and to build strategic links for future dissemination.To sample participants, we will draw on our diverse external advisory group and our established links with Digital First Primary Care Team at NHS England and the TEC (Technology Enabled Care) teams in Scottish and Welsh governments, NHS leaders (including clinical directors, chief clinical information officers and informal digital champions), those in industry (both large technology providers and start-ups, many of whom developed new products during the pandemic and made these available free or at low cost to the NHS), professional bodies (including Royal Colleges) and advisors (e.g.defence societies), regulators (such as National Institute for Health and Clinical Excellence, General Medical Council and Medicines and Healthcare Devices Regulatory Agency), and third-sector groups including patient advocacy groups.
We use a combination of about 20 initial quick, informal interviews (often very helpful to glean over-arching themes and issues) and at least 20 more formal semi-structured and narrative interviews.The former will not be audiotaped (and hence may provide opportunity for candid insights) but we will take contemporaneous notes.The latter will be recorded and professionally transcribed.We will invite elite interviewees to recommend key documents that are guiding their field (e.g.policies, regulation, guidance) and 'follow the trail' of these documents.Where appropriate, we will snowball (i.e.ask interviewees to nominate another senior stakeholder and introduce us by email).
As our study progresses, we will hold four cross-sector stakeholder events using a method developed by our partner the Nuffield Trust.These will begin virtually but may revert to a face-to-face format.We anticipate that workshops will cover the priority topics listed in Table 3.
For each workshop, we will identify a wide mix of stakeholders (including patient groups) whose perspectives are relevant to the chosen theme, make personal contact to invite and engage them, and prepare and circulate a preliminary resource pack (with key materials such as an agenda and objectives, a lay summary of our research, digital inclusion personas, an anonymised and fictionalised significant event).The workshop will begin with a short plenary before participants discuss topics in breakout groups.A final plenary will bring groups together to report back, continue discussion and identify specific steps which need to be taken.
Follow-up activities will include meetings with particular stakeholders, convening smaller task and finish groups (e.g. to prepare a policy briefing), or planning a new stream of research.

Data management and analysis
All formal interviews and ethnographic field notes will be transcribed, de-identified and stored on an encrypted server at the University of Oxford, which will also be used to store research diary notes, key emails and correspondence, facilitator notes, chat comments and reports from online workshops, and public-domain local and national documents.We will use NVIVO, which allows for easy storage, indexing, coding and cross-linking.We will code data thematically to gain familiarity and also analyse relevant segments narratively by asking questions about characters, emplotment and emotional touchpoints 69 .
To initiate and build on practice-based case studies and cross-case comparisons, will use hermeneutic methods, in particular the constant comparative method described by Glaser 99,100 , in which each new data item is added to a progressively richer picture of the whole.For each practice case study, we will combine the various data sources (interviews, ethnographic observations, documents, quantitative data) to build a rich narrative of the local emergence, current use and intended evolution (or replacement) of these artefacts over both short and long temporal scales, attending in particular (but not exclusively) to the priority themes in Table 3.
Each researcher-in-residence has drawn together early interviews and data sources to prepare an initial practice familiarisation document.These summarise the background and context for the 11 participating practices and the issues and challenges each currently faces.These interim summaries are being compared and contrasted in cross-case review meetings, leading to refinement of the cross-case themes.Narrative methods will be crucial for drawing out understanding of micro-level causal pathways which explain (e.g.) why something that 'succeeded' in one setting 'failed' in another.Narrative richness will also allow us to identify and test demi-regularities (things that tend to be the case in particular circumstances) and candidate explanatory theories.Key to cross-case analysis is reflection and discussion among the embedded researchers, and also among patient representatives in the different practice settings.As the study progresses, we will add detail to individual practice summaries and the over-arching summary of cross-case themes.We will seek disconfirming data (qualitative or quantitative data which would lead us to question our current understanding) and use these to amend or refine our understanding.
The same approach will be taken for patient interviews, material from patient workshops, and national stakeholder interviews.In each case, an initial summary document will be prepared through thematic and narrative analysis of the first few interviews, and this summary will be progressively refined as each additional interview is added 100 .We will use member checking to clarify accuracy and interpretation of interview data.

Linked PhD projects
A linked PhD by EL (funded by the NIHR School of Primary Care Research) will track consulting activity for 30 patients with complex needs 101 (10 in each of three practices) over a two-year period; detailed methodology for this study is under development.Two additional PhDs are based in primary care settings outside our sample of 11 practices so as not to overload them.FD (funded by NIHR School of Primary Care Research) will explore the experiences of under-researched lower-grade staff such as receptionists, administrators and cleaners as practices move towards remote care as business as usual.LH (funded by THIS Institute) will study the patient experience of accessing remote care in patients with multiple disadvantage (elderly, lower socio-economic groups, limited English speakers).AB's PhD (funded by Rhodes Trust) is exploring aspects of sustainability and carbon-reduction policies relating to pharmaceutical supply and provision, including how decisions about sustainable prescribing are influenced by the shift to remote assessment and monitoring.

Patient and public involvement
There is extensive lay representation on the external advisory group (see below) including a lay chair (AAN, co-author).
We have strong links to local patient involvement groups in participating practices where these exist.AAN has established a patient / lay involvement virtual group with representation across participating practices and an arrangement where those in the group commit to buddying others who are not online (or less confident online).Patients and lay people have been formatively and iteratively involved in designing the study; their input has been crucial to shaping the original bid (especially the kinds of remote consultations they are most concerned about) and in responding to changes as the study unfolds.All inter-sectoral workshops include patient and lay participants.

Ethics and dissemination Governance
The study has an independently chaired external advisory group with diverse representation from policy, clinical care, the commercial sector, people with lived experience, and members of patient advocacy groups and regulatory bodies.It receives a three-monthly written progress report before an advisory group meeting with the research team.The advisory group's comments are summarised in writing and taken forward by the core research team.

Ethical approval and consent
Approval has been granted from East Midlands-Leicester South Research Ethics Committee and UK Health Research Authority (September 2021, 21/EM/0170) and subsequent amendments.All patients and staff interviewed gave written informed consent in accordance with our ethics protocol.The ethics committee have approved easy-read versions of the information sheets and consent forms for low-literacy participants.

Study status
We have collected and analysed baseline data on all practices, which we have presented in a separate paper 102 .
Selected additional data on this ongoing, mainly qualitative study will be made available to researchers on reasonable request to the lead author (TG).We anticipate that data collection for this study will be complete by August 2023 and analysis complete by end November 2023.

Other planned outputs
Future academic outputs will include empirical studies describing anonymised case studies and cross-case analyses, and-we anticipate-demonstrating the links between digital inequities and the wider social determinants of health.We also plan theoretical and methodological outputs-covering (for example) the challenges and contribution of small-scale in-depth case studies for addressing complex change in the digital world and understanding causality.We will use the co-production aspects of this study design to generate guidance and tools (including digital personas) on developing effective remote services and also patient-facing resources (including an animation) on securing and navigating one's digital appointment.

Conclusion
This study does not promise easy or universal answers to the question of how remote modalities can be maintained as part of a mixed-modality general practice service, nor how they impact on digital equality.However, our focus on depth and detail in a small sample of practices with different histories, geographies and current challenges will illuminate the complexity of the "new normal" and provide the case exemplars which are crucial to understanding social phenomena and supporting service improvement.

Reporting guidelines
We have followed published guidance for case study research 64 .Formal, structured protocols akin to CONSORT for randomised controlled trials do not exist for this kind of research.

Elaine C. Khoong
University of California San Francisco, San Francisco, California, USA Thank you for the opportunity to review the protocol for this large study that includes three main workstreams: (1) an in-depth study of 11 general practices across the UK and their approach to using remote health tools; (2) working with patients to co-design an ideal approach to provision of remote health care; and (3) focused interviews and dissemination to leaders and policy-makers to develop strategies that promote the best practices from workstreams 1 and 2.
My comments are minor with some suggestions for ways to increase the impact of the study: # Terminology: I think if this protocol is written for just a UK audience, then this is probably fine, but if not, there are some terminologies that may be less familiar to other audiences (e.g., GP surgeries, online consultations -which is defined in the background but was initially confusing in the plain language summary to me).

PLAIN LANGUAGE SUMMARY
# Workstream 2: "Many people experience barriers to accessing care when it is done through technology.This could be because they lack understanding of how to do it, don't have the right equipment, can't afford data, or…" -I bristled a bit at this because it was very deficiency focused.It seemed to blame patients for their lack of understanding rather than stating that perhaps clinics were not inclusive in their design.The authors themselves later on note that a "deficiency framing overlooks the pervasive impact of multiple interacting social determinants."Suggest reframing this in the plain language summary.# Workstream 2: In this summary and in the work itself, it could be beneficial to focus activities on how patients can achieve digital inclusion.(I like the definition from NDIA, based in the US: https://www.digitalinclusion.org/definitions/)Happy for authors to use another definition, but I think it helps provide a framing mechanisms for what domains need to be addressed to ensure digital inclusion.
# Workstream 3: The authors do not include this in their work right now and it may not be a good fit given funding source, but as a US-based reviewer, I'd be interested in knowing enough about the context of different findings to understand how the extensive work the authors will do can be applied to different US contexts.If authors have the bandwidth and expertise to be explicit about how to apply these findings to other high-income countries with similar challenges, I suspect there would be great interest.

WORKSTREAM 1 PROTOCOL
# Table 1: In addition to qualitative description of practice demographics, it may be valuable to have a quantitative measure of an area-level SDOH indicator for patients (such as area deprivation index, which is used in the US) "assigned" to the practice (if that occurs in UK).It might be helpful to describe more generically how disadvantaged vs advantaged the patient population is.
# Quality and safety of care approach: consider adding chart reviews or conducting root cause analyses of poor outcomes as a way to add some additional insight to any concerning outcomes.I noted the authors describe some intersecting work at three practices that focuses on consulting activity over a two-year period.I think identifying 5-10 poor outcomes in each practice and conducting a root cause analyses or chart review could provide some additional useful insights.
# Workload, staff wellbeing approach: it wasn't clear, but just wanted to make sure authors consider use of some of the existing validated burnout surveys to measure this (in addition the proposed interviews).
# Sub-sample studies: the authors note a focus on three practices to evaluate the the front-desk and back-office work.Some rationale about why these three practices were chosen would be helpful.Same thing for the practices to be chosen for the climate impact evaluation.
# General analytic approach: these detailed case studies would seem to be well-suited for coincidence analysis.The authors could take these detailed qualitative descriptions about practices at several points in team and identify its association with for example % patients that receive telephone triage, rates of patient satisfaction, or any other outcome.It may be helpful for informing understanding of what factors seemed to matter most in specific contexts.
Is the rationale for, and objectives of, the study clearly described?Yes

Are sufficient details of the methods provided to allow replication by others? Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: No competing interests were disclosed.
The study rationale and objectives are clearly described.Specifically, the manuscript successfully articulates the value of understanding in rich detail how individual practices are successfully (and unsuccessfully) using one or more modalities of remote care by investigating a diverse set of general practices across the UK.
The use of mixed-methods, multi-site case studies is appropriate and adequately detailed.The study will triangulate data from researchers-in-residence embedded in each practice, along with perspectives from patients, staff, and national-level stakeholders.The protocol also embraces an action research lens, thereby seeking to support practices as they continue to evaluate and innovate in their use of remote care.
The protocol includes details to intentionally include individuals often marginalized by health care systems and often excluded from research.Specific details note strategies to recruit and interview individuals who may speak limited English, have lower digital literacy, experience substance use, or be unhoused.One detail that was not clear was the capacity of the research team to conduct interviews in languages other than English -are there specific other languages that individuals with limited English proficiency will be able to be interviewed in (and perhaps other languages where participation will not be able to be accommodated?).
Strong patient and lay representation on the external advisory board as well as through workshops throughout the study period.
Conclusion statement appropriately addresses the anticipated value and limitations of the proposed study.
Is the rationale for, and objectives of, the study clearly described?Yes

Are sufficient details of the methods provided to allow replication by others? Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Primary care researcher focused on impact of technology and telehealth on health care access, outcomes, and equity through quantitative and qualitative methods.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Thank you for the invitation to review this excellent protocol.
The protocol describes a mixed-methods study exploring what the role of remote care (telephone, online, video) ought to be in UK general practice.
The rationale for the research is clear and the design and methods are entirely appropriate.The protocol makes a strong case for the value of detailed practice level case-studies (workstream 1) alongside patient (workstream 2), and national stakeholder (workstream 3) perspectives to address the research questions.
My only comments are all very minor: The plain English summary includes different words and phrases for the same thing: "general practices", "GP surgeries", "surgeries" and "practices".In the rest of the article, the sites are mostly called 'practices'.I suggest sticking to general practices (and, or, practices) all the way through.

○
The description of Workstream 3 in the plain English summary reads as if it doesn't include any data collection, but in the rest of the protocol you explain that you intend to interview national stakeholders.I think this should be clarified in the summary.

○
The diversity of recruited practices is good and characterising their digital maturity gives important context.A clarification question about digital maturity:-Given the pressures on general practice at the moment, I imagine there might be practices that are currently unable to do the things in levels 2 -5 (described in Table 2), even though they want to and ordinarily would.I wasn't sure whether the capacity/headroom practices have to think about or change their services right now gets considered as part of their digital maturity (is it part of capability, readiness, or infrastructure?), or whether that's a different issue.Does being at a higher level entail that practices have the capacity to innovate at the moment?
○ Is the rationale for, and objectives of, the study clearly described?Yes

Are sufficient details of the methods provided to allow replication by others? Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable
Competing Interests: I am currently the study coordinator of a study that includes Richard Byng as a co-investigator alongside around 20 other co-investigators/collaborators.The study is led by Bristol where I and the chief investigators are based.Richard, based in Plymouth, is a coinvestigator so has some strategic input into the project, although I am not a close collaborator with Richard.I can confirm I was able to review the article impartially.
. As one commentary put it, "[we should] not assume that what has been necessary in a crisis represents what patients or clinicians want or need beyond" (page 345)

Table 2 . Digital maturity scale for general practices.
9dapted from Greenhalgh et al.9.Limited leadership or vision for developing remote services (perhaps for strategic reasons).Telephone is used for triage and call-backs.Little or no online access for patients; video and telehealth unavailable.Key infrastructure is probably absent.Digital inequities are addressed by focusing on faceto-face services.The practice is traditional but one staff member is enthusiastic about remote care.They attempt to use novel technologies and engage others but have not yet succeeded in getting others to share the vision, influence practice strategy or change practice routines or policies.Infrastructure may be inadequate.Digital inclusion is not yet a priority issue.The practice has a vision and plans for providing remote care.Traditional and new technologies are used creatively, and adjusted iteratively, to try to improve an aspect of care within the practice.Attempts are made to overcome digital inequities.Focus is on technical details and feasibility (i.e.making something work).Infrastructure is adequate but has some limitations.The practice uses traditional and new technologies creatively and strategically, and evaluates benefits and disbenefits with the aim of improving care in all relevant areas, including efforts to meet the needs of digitally excluded groups.Digital capability is high (i.e.many services are successfully delivered remotely).Focus is on quality improvement and organisational learning.Work practices and routines are continuously adapted.Material and technical infrastructure is good as a result of strategic investment.The practice has a clear vision and strategy for an effective, efficient, equitable remote service.Digital capability is high.Staff are actively involved in developing, evaluating and improving remote services both within and beyond the practice -e.g.through inter-organizational benchmarking, quality improvement collaboratives, locality-wide planning, research, or national guideline development.

factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies
. BMC Med Inform Decis Mak.2016; 16(1): 120.PubMed Abstract | Publisher Full Text | Free Full Text 62. Williams H, Whelan A: An

investigation into access to digital inclusion for healthcare for the homeless population
. Hastings: Seaview.2017; Accessed 7th March 2021.Reference Source 63.Marmot M, Allen J, Goldblatt P, et al.: Build back fairer: the COVID-19 Marmot review.The pandemic, socioeconomic and health inequalities in England.London Inst Heal Equity.2020.