The development of patient-focused diagnostic target product profiles for detection of infection and exacerbation in people with cystic fibrosis

Patient and Public Involvement (PPIE)

PPIE activities were incorporated throughout this project, from project planning through to delivery and included both public contributors and people with Cystic Fibrosis (pwCF). Initial project design was reviewed by the NIHR HRC DTE ‘Insight Panel’ of public contributors prior to initiation of the work. The ‘Insight Panel’ were approached at strategic milestones in the project to review and advise on next steps including at each phase of the project (see Figure 1 below).

An independent external advisory group (EAG) was brought together for this project which included pwCF and public representatives as well as clinical, academic, laboratory, and methodology experts. The EAG were consulted at the drafting stage, at each phase, and during the elicitation and dissemination stages to provide advise and feedback on progress and direction.

Informed consent forms, project outlines, and participant information sheets were drafted jointly by the CF AMR Syndicate and NIHR HRC DTE with some revisions following consultation with a patient and public involvement and engagement (PPIE) manager and PPIE engagement group. The virtual symposium held after the focus groups had a pwCF as a keynote speaker and pwCF in all breakout room and sessions.

Expert advisory group

Our expert advisory panel consisted of eight experts: a person with lived experience of CF, a paediatric and an adult CF physician, a methodology expert in TPP development, a representative from a diagnostics trade association body, and two academics with an interest in CF and microbiology. This panel met formally seven times over the project lifecycle, supplemented by ad-hoc meetings and online document review.

Literature reviews

Methodologists experienced in guideline review searched and summarised NICE guidelines [NG78, QS168, NG237] relating to CF infection and exacerbations to build an understanding of the care pathway and current practice in the UK NHS.

A scoping review was conducted to identify tests (both currently in use or available on the market, and those still in development, published in the last 10 years) that diagnose and predict respiratory infections and exacerbations. In doing so, we could assess the current landscape of diagnostic tests available for use, identify diagnostic gaps, and identify major developers working in this space. This analysis would inform the future development of novel diagnostic tests in this area with further information on the methodology and output of this report noted in the publication⁸.

Diagnostic test characteristics and input values were extracted and collated from relevant existing and published TPP documents hosted by established TPP developers [Appendix 1]. TPPs act as blueprints and targets for developers to build against, ensuring that products are designed to provide clinical utility or address an unmet need. These were used to underpin and categorise sections of our draft diagnostic TPP document and used as the basis of prioritisation discussions during early expert elicitation stages.

Key CF literature including standard of care guidelines and laboratory guidance documents for processing microbiological samples were used to build and guide the clinical context as well as the key considerations conveyed throughout the document.

Focus groups and interviews

Informed consent forms, project outlines, and participant information sheets were drafted by all participating organisations, in collaboration with the PPIE team.

Separate draft focus group schedules were designed for clinical staff and CF patients by members of the Newcastle NIHR HRC DTE and revised iteratively through consultation with the wider project team. Three focus groups were conducted via Microsoft Teams totalling 12 clinical participants and six people with lived experience, lasting between 90 and 120 minutes. Recorded sessions were transcribed via otter.ai before being cleaned and checked for accuracy by an experienced methodologist. Notes were taken by supporting researchers throughout the session and used to assist the transcription and analysis. Accuracy of the interview content was confirmed by the methodologists, and any outstanding issues or clarifications were resolved with the focus group interviewees.

Additional interviews were conducted with individual stakeholders from within and outside of the UK. These included methodologists in TPP development, members of the world health organisation, innovators from industry, academics with expertise in CF microbiology, microbiologists, paediatric and adult CF physicians, laboratory staff, were conducted following the methods outlined above with a stronger focus on eliciting specific information relevant to the TPP development process.

Following data collection, rapid thematic analysis was performed based on the recordings, transcripts and notes taken during the interviews and focus groups. A thematic summary document was produced, and its contents were checked for accuracy and completeness by the methodologists and wider project team involved.

Surveys

A series of surveys were used to frame and inform the TPP development process.

Rank prioritisation

The project team disseminated a survey to rank prioritise 10 options for potential TPP development based on the unmet needs identified in phase 1 focus group interviews. The survey, written in SurveyMonkey, was open for 2 weeks, from Friday 12^th May 2023 – Friday 26^th May 2023. It was sent to selected individuals including members of the TPP expert advisory group project, clinicians, industry liaison, researchers, people with CF and their loved ones. Participants provided their ranking according to ‘priority’ and ‘feasibility’.

For the rank prioritisation exercises, a 10-point Likert scale was used to weight the responses, ranging from 1 (lowest priority) to 10 (highest priority). The rating average was a mean, calculated as follows, where w = weight of answer choice, x = response count for answer choice:

(x1w1 +x2w2 + x3w….xnwn)/total responses

‘High level’ TPP refinement survey (utilised for TPP1 – TPP3)

The project team disseminated a survey to gather opinions and formulate consensus on TPP topic selection, validation of the prioritised unmet needs, key considerations, and value propositions collated from phase 1 and 2, and to determine the level of agreement in the CF community with regard to the proposed TPP content. The survey, written in SurveyMonkey, and composed of 12 questions, was open for 3 weeks from 3rd October 2023 to 23rd October 2023, resulting in 102 total responses (note that some responses were partially complete - total number of responses therefore varied by question). The survey link was distributed across the globe through established networks of the NIHR HRC DTE, clinical care teams (covering physiotherapy, nursing, physician, and pharmacy networks), pwCF, industry, microbiology and academia. It utilised a 5-point scale from ‘strongly disagree’ to ‘strongly agree’ with an N/A option and compulsory free-text box to explain the reasoning for the selection. A thematic analysis was performed on the qualitative responses to the survey and themes agreed by consensus amongst the project team’s qualitative expert methodologists. [Appendix 1]

Two-round Delphi (Utilised for TPP4)

We used a 2-round modified Delphi approach with an 80% consensus threshold to determine consensus on non-tuberculous mycobacterium pulmonary disease (NTM-PD) amongst NTM experts, on the NTM TPP. Thirty-five NTM-PD experts including healthcare professionals and academics participated, recruited from the NTM Network UK [https://www.ntmnetworkuk.com/]. Round 1 comprised of 28 questions and was open to input for 3 weeks, between 25^th October 2023 and 12^th November 2023. Round 2 comprised of 22 questions, and was open for input for 2 weeks between 13th November 2023 and 1^st December 2023.

A thematic analysis was performed on the qualitative responses to the Delphi, and themes agreed by consensus amongst the project team’s qualitative expert methodologists. Wording suggestions and changes were incorporated between Round 1 and Round 2 of the Delphi exercise. [Appendix 2] [Appendix 3]

Virtual symposium

The CF AMR Syndicate hosted a virtual symposium on 6th of December 2023. Convening diverse, international expertise from the clinic, academia, industry, and people with CF, the event provided a platform to discuss key challenges in current infection and exacerbation diagnostics in CF and identify opportunities to address them. More information on the virtual symposium and outputs can be found here: https://cfamr.org.uk/report/virtual-symposium-on-diagnostics-meeting-report/

Phase 1: Landscape analysis

Literature reviews from Medline and Embase databases identified 2133 references, of which 1184 were screened after de-duplication. 261 papers met the inclusion criteria based on title and abstracts, and 200 were included after full text screening⁸. Approximately 100 different tests and modalities were identified and grouped according to use-case and sample type. This review provided the base knowledge of available diagnostics tests for infections and exacerbations in CF, supporting phase 2 TPP drafting.

Key themes from phase 1 focus groups included the need for more acceptable sampling methods (including non-sputum sampling), greater patient acceptability - particularly for sampling and convenience of testing, a need for faster time to results to inform treatment and reduce the reliance on empirical antibiotic use, and a need for novel tests to have demonstrable clinical relevancy, with a preference for accuracy over speed in non-acute settings.

Approximately 10 unmet diagnostics needs were identified from initial clinical and pwCF focus groups, under three primary categories: Baseline monitoring and detection of exacerbations, pathogen identification, and treatment monitoring. These unmet needs were prioritised by 15 clinical, laboratory, academic, and industry contributors, and 9 people with lived experience of CF based on the urgency of the unmet need, and the feasibility in finding a solution. The rankings (Table 1) were used to support prioritisation of TPPs for phase 2.

The study team, guided by the expert advisory group, and the results of the prioritisation exercise, selected the TPP topics outlined in Figure 2. The chosen TPP topic areas included generic testing needs for people with CF, and specific TPPs for 1) monitoring and predicting pulmonary exacerbation, 2) rapid pathogen detection and identification, 3) antimicrobial susceptibility testing, and 4) rapid diagnosis of NTM pulmonary infections.

Figure 2. TPP document topics.

Figure 2 shows an overview of the suite of TPPs developed in this project. 4 TPPs were generated along with additional generic content describing unmet diagnostic needs and priorities of people with Cystic Fibrosis. TPP1-3 were ‘high-level’ TPPs aiming to provide an overview of characteristics to address – TPP1 – baseline monitoring, prediction, and management of pulmonary exacerbations; TPP2 – Rapid pathogen detection and identification; and TPP3 – Antimicrobial susceptibility testing. A more in-depth ‘full TPP’ was developed for TPP4 – in vitro diagnostic tests for rapid detection of NTM pulmonary infection.

Phase 2: TPP drafting

A review of typical test characteristics in established TPPs was performed. 57 TPPs were screened [Appendix 4], and their characteristics were extracted to inform the study team and expert advisory group’s decision making.

Each TPP produced in this study included:

TPP 1-3 do not contain all characteristics as the aspirational nature of these TPPs precluded detailed descriptive requirements.

The TPP content, including the minimal and optimal requirements and potential value propositions, were derived from focus groups, 1-to-1 clinical, industry, and lived-experience interviews, utilising language and content from previously published TPPs identified in Phase 1.

‘Minimal’ requirements were defined as the lowest possible threshold to provide clinical utility, based on the value propositions highlighted during by expert interviews with key opinion leaders. ‘Optimal’ requirements were defined as values offering the best clinical utility based on the value propositions highlighted in the TPP as determined by expert interviews with key opinion leaders (Table 2).

Phase 3: TPP refinement & validation

The TPP content was refined through discussion with key stakeholders, a two-round modified Delphi consensus survey, and a virtual symposium that featured ‘lived experience’ perspectives, industry, academia and clinical discussions, and breakout rooms answering key questions.

All of the TPP content for the high level TPPs (TPP1-TPP3), including the elicited value propositions, key considerations, and characteristic values were incorporated into a refinement survey to assess community agreement and elicit feedback. The survey was completed by 102 individuals, including 21% CF physicians, 10% physiotherapist in CF, 12% CF nurse specialists, 15% CF academics, and 11% people with lived experience of CF. Over 85% of all responses were ‘Agree’ or ‘Strongly Agree’ across each of the TPPs regarding the value propositions and TPP characteristics (Figure 3a and Figure 3b). Respondents were invited to provide commentary on the rationale of their response which was used to refine the TPP document further.

Figure 3a. Consensus agreement of value propositions defined in TPP1-3.

Figure 3.a displays the overall responses and level of agreement for each of the TPPs covered in the high-level TPP survey. This question covered the value propositions outlined in the TPP document. The Value propositions for all three TPPs included in the survey were positively received, indicating strong agreement with the TPP content. The N/A option was used in the survey for both 'I don't feel qualified to answer' and 'I do not wish to answer' responses.

Figure 3b. Consensus agreement of TPP characteristics defined in TPP1-3.

Figure 3.b displays the overall responses and level of agreement for each of the TPPs covered in the high-level TPP survey. This question covered table of characteristics outlined in the TPP document. The tables of characteristics for all three TPPs included in the survey were positively received, indicating strong agreement with the TPP content. The N/A option was used in the survey for both 'I don't feel qualified to answer' and 'I do not wish to answer' responses.

Based on the initial focus groups and expert interviews in phase 1 and 2, additional sections were added to inform users of the TPP about CF-specific issues. These included the impact of modulator therapies, sample considerations due to reduced sputum production, the needs and priorities of people with CF, and considerations for diagnostic developers, including the need for robust analytical validity studies and health economic analysis to drive adoption and implementation.

For TPP4, a consensus study was designed using a modified 2-Round Delphi approach. Round-1 of the Delphi survey achieved above 80% consensus, with an average of 95% positive agreement. Open-text responses indicated a need for wording clarifications and highlighted additional functionality of the tests.

Round-2, completed by 26 participants, sought consensus agreement on the revised wording from round-1. Round-2 also achieved above 80% consensus, with an average of 92%. No additional rounds were required.

The final method of expert input and consultation was a virtual symposium. Outputs from this symposium were incorporated into the revised TPP document before final confirmation by the project team and advisory committee, and subsequent publication.

The final TPP document included one general table defining the diagnostic testing preferences and considerations of pwCF, three TPPs to address aspirational requirements (TPP1-3), and one detailed TPP defining a rapid test for NTM-pulmonary disease (NTM-PD) (TPP4).