Skip to content
ALL Metrics
-
Views
60
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

ROSSINI-Platform - Reduction of surgical site infection with a platform trial utilising a 'Basket-MAMS' design - Application acceleration award

[version 1; peer review: 1 approved with reservations]
PUBLISHED 12 Dec 2024
OPEN PEER REVIEW
REVIEWER STATUS

Abstract

Background

Surgical site infection (SSI) is the most common surgical complication, resulting in significant morbidity, mortality, and major costs to health service providers. Multi-arm, multi-stage (MAMS) trials allow multiple individual interventions to be evaluated simultaneously, rendering them highly efficient at both speed and cost. Our group is running the successful ROSSINI 2 MAMS trial in abdominal SSI reduction, which was funded by HTA and proved that such trials are deliverable and effective in a modern surgical context.

Our proposed platform study will run multiple parallel RCTs in different surgical cohorts, utilising a ‘Basket-MAMS’ design to simultaneously assess multiple interventions and will share learning between cohorts to take successful arms from one area to another for rapid testing. This would provide robust and context-specific evidence for rapidly improving patient care.

Methods

We identified a list of key methodological, logistical, health economic, PPI-related, and clinical design challenges. A large study team was assembled, consisting of both core experts and national clinical research leaders, including robust plans for capacity building, Chief Investigator (CI) development, and multi-level PPI engagement. The working groups designed a series of parallel work packages to address the identified challenges, both centrally and in clinical networks, convening regularly to discuss and plan the full application. Iterative modifications and improvements to the overall proposal were allowed and encouraged.

Results

We report the progress, activities undertaken, and decisions made throughout the application acceleration award project. This information may be beneficial to planning or preparing a similar large-scale and multi-specialty platform trial.

Conclusions

The acceleration grant allowed us to design a fundable and deliverable clinical trial for the HTA platform trial call in late 2023. The full trial has the potential to significantly impact the rates of this highly impactful complication, thereby benefitting both patients and health services worldwide.

Plain Language Summary

Up to a third of patients undergoing major surgery develop an infection of the wound (called a surgical site infection (SSI)). SSI is a major problem and costs NHS at least £700 million per year.

There are many ways in which surgeons can try to prevent SSI, but very few have been properly tested to determine how well they work. Our group is currently running a trial called ROSSINI 2 to look at reducing SSI after abdominal (tummy) surgery. ROSSINI 2 is a multi-arm, multi-stage trial. This means that at the beginning of the trial, there was a set number of treatments used during the operation to reduce infection, which were reviewed at set times during the trial. Treatments that have not performed well are dropped, and this continues until the end of the trial, when only the treatment(s) that work best remain.

The proposed ROSSINI-Platform Trial builds and expands upon ROSSINI 2, creating a trial across six surgical specialties or ‘pillars’: breast surgery, vascular groin surgery, cardiac (heart) surgery, neurosurgery (brain), leg amputations, and thoracic (lung) surgery.

By running these six otherwise separate trials together within one ‘platform’ of a single overarching design structure, we can make the management of trials simple, faster and cheaper. We will also be able to have an overview across all pillars and move the treatments proven to work in one part of the body into other groups for rapid testing. This is a very efficient design and will result in improved patient outcomes much quicker than traditional trials. Both patients and the NHS will benefit.

Before starting the full-platform trial, we need to address several design and logistical challenges. This award allowed us to bring our team together and undertake the necessary planning work.

Keywords

Surgical Site Infection, Randomised Controlled Trial, Adaptive designs, MAMS Platform Trial, Acceleration Award, Surgery, Research Methodology

Background

The strategic importance of Surgical Site Infection (SSI) as an area requiring research attention

In November 2022, there were 7.07 million patients waiting for planned hospital treatment within the NHS, the majority of which were elective surgical operations1. The rising backlog and ongoing significant pressures on the service mean that there has never been a more important time for preventable postoperative complications to be avoided.

Surgical Site Infection (SSI) is now the most common healthcare-associated infection, overtaking MRSA, Clostridium difficile, hospital-acquired pneumonia, and catheter-associated infections after significant and sustained investment in these other areas in recent years2.

An unacceptably high proportion of patients undergoing surgery develop an SSI and, as such, are exposed to preventable pain, suffering, delayed further treatment, and even a risk of death. The complication also costs NHS at least £700 million per year35.

Current practice in the prevention of SSI

A key shortcoming of all current national and international guidelines on the reduction of SSI is their highly generic nature: the recommendations made apply to all types of surgery and are not speciality-specific or surgical site/target organ-specific68. This is largely a result of the small number of high-quality trials making up the evidence base across surgery, meaning that research findings from one surgical indication are inevitably extrapolated to other settings. This can lead to both overpromotion of interventions that have shown clinical effectiveness within a discrete area of surgery and the discarding of others that might confer benefit when utilised in a different setting.

Recommendations are often made based on the contamination level of the wound. This primary classification system for wounds by contamination (clean, clean-contaminated, contaminated, and dirty) has been in place since 19649. Although the system can be useful to demonstrate the range of operation types and subsequent highly variable risk of SSI from <2% to >40%, it is overly simplistic. Within this classification, an elective wrist operation, open abdominal aortic aneurysm repair, and leg amputation are all pooled together as ‘clean’ operations. Similarly, elective operations for both lung cancer and rectal cancer can be ‘clean-contaminated’. While the etiology of SSI is multifactorial, it is increasingly clear that mechanisms of SSI development, including the source and nature of the pathogenic organisms responsible, are not the same across the body.

We believe that more accurate testing of SSI prevention measures would allow evidence-driven stratification according to the specific procedure a patient is undergoing. This would confer significant benefits for both patients and the health service due to a reduction in SSI across the panoply of surgery.

Does the solution lie in bundle trials?

It is increasingly clear that a single ‘magic bullet’ to prevent SSI does not exist; rather a cumulative package of measures for a particular operation type or surgical setting are likely to be most effective. However, most ‘bundle’ trials of interventions to reduce SSI have failed to produce convincing or reliable results.

Some years ago, our group realized that the multi-arm, multi-stage (MAMS) platform design allows multiple individual interventions to be evaluated simultaneously, utilising factorial design but a nonfactorial analysis in which interactions between interventions can be allowed and formally explored. This ‘unbundling’ design produces evidence-based combination(s) of interventions identified as the optimised preventative strategy for SSI in that setting. The simultaneous nature of multi-arm assessment renders it highly efficient in terms of both speed and cost. The methodology has been previously proven and highly successful in the STAMPEDE suite of trials in prostate cancer1013 but had not been used in a surgical context until that time. We established a collaboration with the pioneer of MAMS design, Professor Mahesh Parmar at the Medical Research Council Clinical Trials Unit (MRC CTU), and his team to create the ROSSINI 2 trial.

The ROSSINI 2 trial

ROSSINI 2 is a major multicentre RCT exploring the effectiveness of three intraoperative interventions for the reduction of SSI in patients undergoing abdominal surgery, utilising a MAMS selection design14,15. Within it, the three interventions are assessed both individually and in combination, resulting in seven treatment arms which are compared against a single (double-recruited) control arm in a 2:1:1:1:1:1:1:1 ratio. The least effective arms are dropped at prespecified interim analysis stages, leaving a final confirmatory phase where the best arms (which may be individual interventions or combinations of interventions) are compared against control. The original ROSSINI 2 trial schema is shown in Figure 1.

de3df01e-1110-43e9-8440-6a6942b16ba7_figure1.gif

Figure 1. Original ROSSINI 2 schema.

ROSSINI 2 was funded by NIHR HTA in 2018, and commenced recruitment in March 2019. By the time of our application for the accelerator award, ROSSINI 2 had recruited >3700 patients from 52 sites across the UK and proved the principle that MAMS RCTs are both deliverable and efficient in a modern surgical context. The first planned interim analysis was performed in early 2022, at which point four treatment arms were dropped owing to a lack of benefit.

ROSSINI 2 was and remains the highest-recruiting surgical trial in the NIHR portfolio. Given the ongoing success of the trial, despite the pandemic, we communicated with HTA regarding the potential cost and practicalities of adding new interventions into the existing trial structure. This was subsequently awarded, and the extension phase of the trial went live in April 2024).

While ROSSINI 2 will deliver robust information on interventions to reduce SSI in abdominal surgery, the findings will not necessarily be translatable to other types of surgery for the reasons outlined earlier.

The ROSSINI-Platform trial

In mid-2022, we created the outline of a proposal for a new platform trial that will build upon and extend the principles established in ROSSINI 2 across a wide variety of surgical settings. The proposed platform trial would utilize a bespoke ‘Basket-MAMS’ methodology in which multiple full-scale MAMS RCTs will be run in parallel within different surgical cohorts (‘pillars’), organized by wound type/site, under one over-arching protocol and governance structure. These MAMS trials will simultaneously assess multiple carefully-selected specific interventions alone or in combination and will crucially share learning, both clinical and statistical, between pillars to take effective arms (individual interventions or combinations of interventions) from one surgical cohort to the others for rapid re-testing. This iterative approach would be doubly efficient because the MAMS design allows within-trial interim analyses and dropping of ineffective arms, and then the prespecified transference of successful arms into vacated slots in other pillar RCTs within the lifetime of the overall planned protocol. This will provide robust and context-specific evidence in a highly efficient manner, in terms of both time and cost. Overall learning and efficiencies may be further enhanced by utilising the over-arching and standardized protocol, outcome measures, and homogenized datasets to synergize new data in two additional ways: i) considering borrowing information for patients across different pillars who receive the same or similar interventions for combined analysis and ii) undertaking prespecified individual patient data meta-analyses (IPDMA) of outcomes, including those in other (external) trials using the FAME structure16. This has the potential to further shorten the timelines to patient benefit.

The platform element of running these parallel RCTs under one overall organizational structure would yield further benefits from a governance, oversight, delivery, and analysis point of view. We anticipated a ‘master protocol’ which would cover the generic elements of trial procedures and governance, and then individual sub-protocols for the separate cohorts. Hospital sites would open for the overall trial and then select which of the constituent studies to participate in after appropriate capability and capacity assessments and site initiation/opening procedures, which will depend on local facilities, interest, and equipoise.

Methods

Patient and Public Involvement (PPI)

The reduction of surgical site infection has been a clear area of research need for our patients for many years, as witnessed by the repeated identification of this topic across a range of priority setting partnerships in several different surgical specialities.

Significant PPI work was done during the development of ROSSINI 2, and this was taken into account in the development of the scaled-up platform trial proposal. The project has been further developed in conjunction with an experienced patient representative, who is the patient representative sitting on the independent Trial Steering Committee of the current ROSSINI 2 trial. She has therefore already been involved already in the concept and design of the project and understands the overall premise and the MAMS design. Similarly, her direct personal experiences of undergoing multiple major operations has been invaluable.

We have also invited an experienced PPI manager with many years’ experience of leading PPI activity and working with patients and the public on health-related research to be a key part of the core management team. Both individuals will continue to be involved going into the full project, if funded, and have agreed to sit on the executive trial management group for the project.

The scale of the project has led us to involve two core patient and public representatives in this planning stage, with the expectation that each clinical pillar will subsequently also appoint multiple specific PPI representatives going forwards.

It is clear that significant PPI work will need to be undertaken both during the design phase and then as an active and ongoing key component of all of the parallel trials as well as the overarching management group if the platform trial is to be successful. We will build upon and scale up the positive models created within the ROSSINI 2 trial to deliver this – including the concept that a large MAMS trial is more complex, rapid and iterative compared to a normal RCT and as such a minimum of three PPI co-applicants should sit on each pillar’s trial management group. In addition to bringing a wealth of viewpoints, this system will limit the onus of responsibility on one individual going into the future.

Within this Application Accelerator Award, a key aspect for the individual clinical groups will be ensuring PPI remains central in each of their pillar trial designs and delivery proposals. An early activity (see Gantt chart) will be to identify suitable patient groups and/or individuals who can help in these roles In a similar manner to how we will define the key design elements for selection of interventions and other protocol factors within this initial study design award, we will also create a minimum set of standards for PPI engagement and activities which each speciality pillar will adhere to.

The opportunity to undertake an application acceleration award

There were multiple factors within the complex fledgling proposal which needed careful and structured planning if a credible and fundable bid for the 2023 HTA Platform Studies call was to be created. These comprised logistical and organizational challenges of a scale not previously seen in surgical research. We were therefore very grateful for the opportunity to apply for the Application Acceleration Award to bring the team together to undertake this necessary prework.

Multiple over-arching elements of overall design, high-level methodology, analysis strategy, oversight, governance, health economics, and core PPI elements all needed unpicking – in addition to the equally important clinically driven aspects, including evidence reviews to select interventions, exploring community and individual surgeon equipoise, delivery network development, patient pathway work, speciality-specific PPI engagement, and input. The management structure for the accelerator project, therefore, intentionally mirrored that of the proposed full platform trial in that the necessary specialist groups were key partners from the outset, working alongside the central team members with regular meetings and interactions. The initial organogram of the platform is shown in Figure 2, and the two main teams are described below.

de3df01e-1110-43e9-8440-6a6942b16ba7_figure2.gif

Figure 2. Proposed organogram for ROSSINI-Platform (initial draft from accelerator application).

Team and project management structure – CORE TEAM

We assembled a core team with internationally recognized expertise in methodology, statistics, trial delivery, health economics, and PPI. Most of the core team members already had a track record of successful collaboration in the ROSSINI 2 trial. This team will work up the proposal into a fundable HTA application and then go on to form the central trial management team with oversight and responsibility for the core elements of the Basket-MAMS platform if the platform is funded.

The application acceleration award contained costs for a dedicated Project Manager to coordinate all activities and meetings, ensure progression against the proposed timelines, and maintain and develop logs of discussions and outcomes to feed into the full application.

Team and project management structure – CLINICAL PILLARS

Individual pillar RCTs were proposed to be run by separate but linked teams of specialist surgeons and collaborative surgical research groups within each surgical area, along with a dedicated trial delivery team. A similar model of having separate specialist PIs for constituent trials within a larger trial structure has been successfully used by others, including within STAMPEDE trials.

The UK surgical community came together behind the platform accelerator bid with high-level engagement from experienced research leaders in each of the specialties involved. We worked with the Royal College of Surgeons of England Clinical Research Surgical Speciality Leads (SSLs), who are charged with developing and delivering clinical research networks across the country; the SSLs have often led HTA trials or major research programs previously themselves. Therefore, we created a model where each will mentor a relatively novice or early career researcher to run their individual pillar RCT, thereby increasing research capacity and expertise within the relatively protected environment of the overall platform trial. A similar structure of mentorship and training is currently being successfully used in the MRC CTU ACORD Fellows Academy, which we will emulate17. The clinicians involved in the original application, and their backgrounds, are shown in Table 1 a.

Table 1a. Clinical Pillar teams and members (original).

SpecialtyNameAffiliation/job description
Vascular GroinProf lan Chetter, HullRCS Surgical Speciality Lead; NIHR Senior Investigator; Chair of research committee of Vascular Society of GB&I
Mr Mike Wall, DudleyConsultant vascular surgeon (2015), NIHR CRN(WM) Clinical Research Scholar
BreastMr Stuart McIntosh, BelfastRCS Surgical Speciality Lead; experienced triallist; CI of SMALL trial (NIHR HTA)
Miss Shelley Potter, BristolRCS Surgical Speciality Lead; NIHR Clinician Scientist, Consultant breast surgeon (2017)
ThoracicMr Babu Naidu, BirminghamConsultant Thoracic Surgeon (2009); experienced triallist; Cl of Fit4Surgery 2 trial (NIHR HTA)
Miss Veena Surendrakumar, BirminghamVascular Specialist Trainee; RCS research fellowship; co- investigator RfPB fistula study and on HTA Fit4Surgery 2 trial
Lower limb AmputationsProf lan Chetter, HullRCS Surgical Speciality Lead; NIHR Senior Investigator; Chair of research committee of Vascular Society of GB&I
Mr David Bosanquet, NewportConsultant vascular surgeon (2019), previous chair of Vascular and Endovascular Research Network
CardiacProf Gavin Murphy, LeicesterRCS Surgical Speciality Lead; experienced triallist; Director of Leicester Clinical Trials Unit
Mr Ricky Vaja, LondonRCS Associate Surgical Speciality Lead; co-founder Cardiothoracic Interdisciplinary Research Network (CIRN)
NeurosurgeryProf Michael Jenkinson, LiverpoolRCS Surgical Speciality Lead; experienced triallist; CI of ROAM and STOP 'EM trials (NIHR HTA)
Miss Ellie Edlmann, PlymouthRCS Associate Surgical Speciality Lead; NIHR Clinical Lecturer; Ex-chair British Neurosurgical Trainees Research Collaborative

Table 1b. Clinical Pillar teams and members – final version after accelerator award and as submitted in full Platform Trial application.

SpecialtyNameAffiliation/job description
Vascular GroinProf Ian Chetter, HullRCS Surgical Speciality Lead; NIHR Senior Investigator; Chair of research committee of Vascular Society of GB&I
Mr Matthew Popplewell, BirminghamAssistant Professor of Vascular Surgery and Honorary Consultant vascular surgeon (appointed 2023)
BreastProf Stuart McIntosh, Belfast & Prof Shelley Potter, BristolRCS Surgical Speciality Lead; experienced triallist; CI of SMALL trial (NIHR HTA) & RCS Surgical Speciality Lead; NIHR Clinician Scientist, Consultant breast surgeon
Miss Katherine Fairhurst, BristolNIHR Academic Clinical Lecturer in Breast Surgery [ST8 level]
Lower limb AmputationsProf Ian Chetter, HullRCS Surgical Speciality Lead; NIHR Senior Investigator; Chair of research committee of Vascular Society of GB&I
Mr David Bosanquet, NewportConsultant vascular surgeon (appointed 2019), previous chair of Vascular and Endovascular Research Network
CardiacProf Gavin Murphy, LeicesterRCS Surgical Speciality Lead; experienced triallist; Director of Leicester Clinical Trials Unit
Mr Ricky Vaja, London & Mr Luke Rogers, BristolRCS Associate SSL and Cardiac Surgery Specialist Trainees x2 ; co-founders Cardiothoracic Interdisciplinary Research Network (CIRN)
NeurosurgeryProf Michael Jenkinson, LiverpoolRCS Surgical Speciality Lead; experienced triallist; CI of ROAM and STOP ’EM trials (NIHR HTA)
Miss Ellie Edlmann, PlymouthRCS Associate SSL; NIHR Clinical Lecturer; Ex-chair British Neurosurgical Trainees Research Collaborative
ObstetricsProf Katie Morris, BirminghamProfessor of Obstetrics and Maternal Fetal Medicine; Experienced triallist and Director Birmingham Clinical Trials Unit
Ms Victoria Hodgetts-Morton, BirminghamAssociate Professor and Honorary Consultant in Obstetrics (appointed 2023)

The application acceleration award proposal also contained six dedicated pots of money (£6000 per pillar) for the coordination and conduct of meetings between surgeons, trainees, and patient groups during the development of each clinical pillar. This included mini-Delphi exercises, patient focus groups, and room hire/logistical support for clinicians to come together for a parallel session at a national meeting. Each mini-budget was to be utilized at the discretion of the individual pillar team, with support from the core group.

Key issues to be addressed within the application accelerator award

During the early development phase of the proposal and the conversations undertaken in planning for the application acceleration award, a list of key design, delivery, and organizational questions or issues emerged, all of which needed to be unpicked, discussed, and finalised as part of the project. These were categorized into a variety of broad themes, as summarised in Table 2, although it was also anticipated that additional items would emerge as part of the iterative process of the full platform trial design. This list of questions was included (verbatim) within the application for the accelerator award.

Table 2. Categorised list of questions and topics to be addressed within the accelerator award.

NIHR156728
Unanswered questions & middleics to be dealt with during Application Accelerator Grant
MethodologicalDefining the over-arching rules for selecting interventions within each pillar
Developing the targeted effect sizes for each intervention and the potential implications of different targeted effect sizes for different interventions
What are the implications of having the same intervention in different clinical pillars, especially when we patient numbers in some pillars might be more limited?
How to deal with different baseline SSI rates in each pillar; presume we will run on a re Criteria and timing for selecting 'winning' intervention arms (individual or combinations of interventions) to move from one pillar to another?
Criteria for dropping arms and whether it matters if an identical arm still is active in other pillars but not yet reached interim analysis stage (presume continue)
Compare the use of a MAMS 'selection' design to create empty slots when arms are dropped with a standard MAMS in which it is harder to predict overall size and duration?
Role of adaptive randomisation to either drop an apparently failing arm more quickly, or assess/prove success more rapidly in another pillar when it has been robustly successful elsewhere?
LogisticalHow will we run the separate RCTs under on over-arching governance/approvals structure at a central level and a site level?
Protocol – one over-arching generic protocol with specific subprotocols for the separate pillars? Which elements can be genericised and which cannot?
Fine detail of trial management - presume one oversight TMG and individual 'normal'
TMGs for each pillar? What about CTU staff and oversight between pillars? Follow-up structure for primary outcome of SSI-options to add video-assisted remote
follow-up vs telephone vs face-to-face. Lessons from SUNRRISE and other trials
Follow-up timing – presumed 30 days as per CDC definitions, but what about operations where implants are used or should we exclude these?
Health EconomicIs there robust evidence on the cost of an SSI in each of the clinical specialties / wound types? How different are these costs from each other and does it matter? Do we need to capture new health economic data from this trial in each of the pillars?
Do we need to obtain resource use data for each stage of each trial, or can we just collect this in the final confirmatory stage, as per ROSSINI 2?
Do the interventions within a pillar need to cost roughly the same as each other?
Should health economic considerations be built into the decision to drop arms, or do we not care about costs if interventions/combinations are being dropped due to non-effectiveness? What if they do work somewhat, just not as much as other arms?
Should we move away from the binary Yes/No criteria for SSI into the more granular mild/moderate/severe categorisation for SSI. Is this deliverable on this scale?
PPIWhat do patients and the public feel about the acceptability of interventions proposed within each pillar?
How are the overall study design and recruitment/follow-up pathways received by patients?
How should we optimise advertising and dissemination of the trials to maximise patient uptake and retention?
ClinicalWhich limited number of interventions should be picked within each clinical pillar? Including careful consideration of: evidence base, clinician equipoise, availability and cost
How do we maximise clinical buy-in and importance of overall research question?
Can we make recruitment predictions and assess what is deliverable in structured and systematic way?
Achieve consensus on the control arm policies & procedures for each pillar

Timelines for the application accelerator award

The specification of the grant stream involves relatively tight time and cost constraints. The team prespecified the proposed work packages, timelines, and meeting schedules/frequency for the accelerator award, which are shown in the Gantt chart (Figure 3).

de3df01e-1110-43e9-8440-6a6942b16ba7_figure3.gif

Figure 3. Gantt Chart of timelines and activities for the accelerator award.

Financial planning for the full trial

In addition to addressing the key questions and design considerations previously outlined, the team also pledged to maintain careful oversight of the core value proposition inherent within any full platform trial application during the accelerator project. It was important to identify and utilise every opportunity to achieve logistical and financial efficiency. It remained clear that the subsequent proposal for the full platform trial could not be equivalent to requesting funding for six fully costed standalone trials, plus additional platform oversight costs. We aimed to ensure that a cogent, streamlined, and efficient trial proposal would be submitted, making full use of any shared elements.

Results

The application acceleration award

The accelerator project grant was awarded in February 2023, with a start date of 1st March 1, 2023, and a 9-month project duration. The deadline for submission of a call to the full 23/95 Platform studies to efficiently evaluate the clinical effectiveness of multiple interventions in areas of strategic importance was 28th November 28, 2023.

During the course of the project, the team undertook a variety of meetings within both specific subgroups and clinical pillar teams as well as bringing the whole group together for the three major meetings, as follows:

Tuesday 14th March 2023 [online] – Launch meeting

Tuesday 27th June 2023 [face-to-face at MRC CTU, London] – Mid-point meeting

Thursday 9th November 2023 [face-to-face at UoB, Birmingham] – Final meeting

These meetings were also attended by a variety of additional invitees, including representatives from) NIHR Clinical Research Network (CRN), ) NHS trust Research Application Support (RAS) team, ) Research Strategy & Services Division (RSSD), University of Birmingham, and the Head of Research Governance & Integrity, University of Birmingham. All these additional parties provided invaluable further advice and opinions into the design, costing, and proposed conduct of the full platform trial application, and we are grateful for their input.

Outcomes – overarching principles

Firstly, we agreed a set of key design/delivery tenets within the first month of the project:

  • One protocol, with one overarching trial pathway across the different pillars, which are each individually-powered trials.

  • Sites will open for whichever clinical pillars they have availability, interest and equipoise.

  • Benefits come from efficiencies such as costs, approval of one protocol, and a single overarching trial team.

  • Calculating the cost savings of one platform trial versus six separate trials will strengthen the case for the platform.

  • Outcome assessment will use common methods across pillars, despite interventions (and surgical wound site locations on the body) being different.

  • High yield interventions in one pillar can be moved across to other specialties for testing.

Outcomes – pre-specified questions/issues

The meetings were run in an iterative format with loose outline agendas and the list of predefined questions/issues (Table 2) used as a structure to aid discussions, although we were not proscriptive or confined by these categorisations. An example would be the discussion around the selection and prioritisation of interventions for the clinical pillars, as these conversations involved methodological, statistical, logistical, PPI, and health economic considerations. As such, the meetings tended to structure themselves around discussion ‘themes’ – this was therefore the optimal format for presenting the project outcomes and decisions below.

Discussion theme A – Overarching statistical policies

How do we cope with different baseline SSI rates between pillars?

Answer/decision: This does not matter. It is a fact that each clinical area will have a different baseline SSI rate. We will be essentially running six separately powered trials under one overarching governance/structure. Therefore, they will have their own sample size calculations, and from a mathematical point of view, be separate trials.

Are we aiming for same relative risk reduction/effect size in each pillar?

Answer/decision: No, not necessarily. While it might make it easier for the reader or funder to understand, the fact that these are separately powered trials (see above) means that we can cope with different proposed effect sizes or relative risk reductions. This is important as it allows us to mould the trial according to the currently available evidence available for the specific interventions in the specific clinical contexts. It also allows us to potentially aim for a larger effect size (where justifiable) in those specialties with a lower baseline event rate to keep the pillars within what is a practicably deliverable sample size.

Should we aim to use Adaptive randomisation?

Answer/decision: No - it does not really add anything useful from a statistical or time-efficiency point of view in this context.

Timing of interim analyses – do they need to occur at the same time?

Answer/decision: There will be different sample size calculations for each pillar, and they are likely to open at different times and deliver differing recruitment rates. As such there is no way we can time the interim analyses to happen at a similar time between pillars. We are accepting of this. It will, however, make more work for the over-arching Data Monitoring and Ethics Committee (DMEC) and Trial Steering Committee (TSC).

Do we need a unified single endpoint across all pillars?

Answer/decision: We could perhaps use different endpoints within this study, if desired, under the umbrella of the platform design. This could be useful for different endpoint timings, or in those areas where the primary outcome of interest might not be pure SSI, such as in amputation stumps, when other wound issues cause major ramifications to ambulation rates. However, this would make the inter-pillar transfer of interventions much harder. In addition, running the trial under one master protocol in terms of outcome measures and their timing would be much more challenging. As such, we probably have to set some boundaries to maintain some similarly/inter-digitation between pillars. Timing and diagnosis of primary outcome seems like a major one to unify across pillars (also see later discussions on diagnosis of SSI).

Overall statistical design options

Answer/decision: When we wrote the original application acceleration award, we assumed that the platform would utilise a MAMS selection design18. Our discussions led onto consideration of two overall design models:

Option A = MAMS selection design (a.k.a. ‘drop the loser’): This helps pre-define the maximum sample size and reduces the uncertainty about the actual sample size of the study in practice, which is, in turn, easier to cost. This is how we eventually got ROSSINI 2 funded as it decreased the number of unknowns.

Option B = Factorial Basket MAMS19: means that pillars or interventions can start at different times which may have benefits considering the scale of what is proposed. This is essentially multiple MAMS trials. Also easy to cost, and sample sizes will also be smaller than a MAMS selection design. The decision to choose between the two options depends heavily on whether we anticipate significant interaction between interventions or not.

Discussion theme B – Choice of interventions

Do we need core rules for standardised selection across pillars?

Answer/decision: No – we do not need to be too strict. We would rather have interventions that have a credible chance of working in each clinical context, as backed up by the available evidence, but they are not in current standard usage, and/or surgeons would have equipoise to accept randomisation to both use and not use them in their operations. The key tenet to remember is that pillars must not choose internally cancelling interventions (i.e., they must be able to use all during one operation).

Should we have a centrally-created list of ‘approved’ interventions for teams to pick from?

Answer/decision: This was not felt to be necessary and would be very challenging to the police. There are interventions which are context-specific, which would never be applicable in other pillars (e.g., how to deal with the bone flap after craniotomy in neurosurgical practice). The currently available evidence of effectiveness is also very context-specific. We would therefore rather the clinical pillars did their own explorations into the most appropriate interventions for their area. Secondary discussions were had about cost of interventions and whether there needs to be ‘rules’ on this, and/or if the interventions in each pillar need to be roughly equivalent in cost to each other. The agreed answer was that we do not need to do this. Note – this also feeds into early discussions on how to incorporate health economic evidence into the arm-dropping decisions at each interim analysis – see section D.

Can identical interventions appear in multiple pillars?

Answer/decision: Yes, we are happy with this. This is likely to happen, but it might be a good thing as this trial would be the first to show whether the same intervention works better, worse, or the same in different parts of the body.

What level of evidence/clinical effectiveness do we need?

Answer/decision: The best available. We are not prespecifying a certain effect size or level of evidence; it will be up to each pillar to meet and discuss, both within their teams and the wider clinical community to decide and agree.

Must all interventions be currently available in UK hospitals and CE/UKCA-marked – or could we do a CTIMP?

Answer/decision: While it might be easier to not propose to run a CTIMP platform trial, most felt that it would be a shame to up-front rule out interventions not yet fully authorised for market, or which are classified as a drug, and that the extra work in making this a CTIMP trial would not be that significant. The group, therefore, remained open to this, and no limitations were placed on clinical teams in their selection of interventions.

Cost of interventions – is it a consideration?

Answer/decision: No – It is not important at this stage of intervention selection and justification.

When do we explore equipoise of clinicians – and of patients?

Answer/decision: Within the latter phases of the application accelerator award, as soon as candidate interventions have been identified. In the full application we need to propose interventions which are acceptable and deliverable, from the point of view of all parties.

Do we need to standardise control arm policies?

Answer/decision: Yes, to some extent - but within each pillar and not across pillars. This will of course remain a pragmatic trial and as such we will aim to be as non-proscriptive as possible. It is likely that, as per ROSSINI 2, there may be 10–20 other peri-operative interventions or procedural variables which need to be collected on an individual patient-level in order that they can be controlled for in the analyses. Clinical pillars will be responsible for selecting these factors as part of the detailed design phase at stage two. We will need to consider this more when interventions have been determined.

Do we need to consider the timing of intervention delivery within the operative pathway?

Answer/decision: Specifically, picking interventions that sit in different operative phases – for example, pre-, intra-, and post-operation – might be one way to give flexibility and correlation across disciplines. However, it was decided that this would also be too restrictive.

What about sustainability and ‘Green’ aspects?

Answer/decision: It was agreed that this is a hot topic; we need to be mindful of ensuring that interventions are acceptable in this sphere. We may also be able to suggest the potential for offsetting any “carbon cost” against the reduction in hospital time/morbidity/re-admission associated with SSIs. It was also agreed that we should be sure to promote any antibiotic-sparing potential of interventions, where appropriate.

Discussion theme C – Outcome measures

What definition of SSI should used, and is the 30 day timing of assessment fixed (noting the likely used of implants in some specialties)?

Answer/decision: Most clinical pillar teams felt that a 30-day primary endpoint would be ok. In cardiac surgery (due to implant usage in sternal wires and other implants) this timepoint is less typical, however it was agreed that the majority of SSIs still present before 30 days, so this would work probably. Measuring ongoing impact would be important to the cardiac team. There are similar concerns in vascular procedures involving prostheses.

What is the optimal standardised method for follow-up – including phone/video?

Answer/decision: Many hours of discussion, and additional dedicated meetings, were held on this point. All parties agreed that it is critical to get this right if the trial was to be deliverable and impactful. We have learnt a lot during the pandemic about how to do this, and move forwards from the previous ‘gold standard’ of a face to face review of the patient with in-person wound examination.

Wound photos can have issues around consent, centralisation, white balance and adjudication. The quality of photo can have major implication on judging severity of SSI. Photos alongside descriptions from patient can work better. Photos could be used as part of a triage tool, with phoning of the patient where SSI is suspected. It may depend if the patient has someone who can take a photo for them. Some patients may not be comfortable sending photo e.g. breast surgery. Video can be effective by enabling patient to move as required but this depends on capability of the patient (e.g. more elderly patients may struggle?)

The TALON study assessed high-quality remote wound assessments within the CHEETAH trial and compared CDC assessments in person versus remotely via video. All agreed that the ‘Bluebelle’ wound healing questionnaire must be incorporated; it has now been validated across several trials.

There were ongoing questions and discussions about whether more regular post-discharge interactions would be feasible, and what it is that patients want.

See later section on the Centralised Digital Wound Hub (CDWH).

What about the severity of SSI concept, rather than a binary yes/no system?

Answer/decision: The current system, as used by everyone, is based on CDC definitions, and is a binary yes/no. All agreed that a new paradigm of adding in an “if yes, is it mild, moderate, or severe SSI’ could be beneficial. Key determinants of severity may include both the impact on patients and health services. This is exploratory at this stage, as it is not validated; therefore, it cannot be used as a primary outcome. The severity of an SSI will likely also vary between clinical contexts, for example, a severe neurosurgical SSI would be very different from a severe breast SSI.

The breast team already uses a similar scheme in some of their work – one delineator may be: Mild SSI – Outpatient antibiotics; Moderate SSI – IV antibiotics; Severe – Return to theatre or other reinterventions with definite prolonged LOS. It was agreed that a substudy within this large proposed multispecialty trial could define the severity strata/definitions for each context, which would be of great help to future researchers.

Discussion theme D – Arm dropping and inter-pillar transfer

Criteria for dropping arms – and are health economic (HE) considerations included?

Answer/decision: Potentially yes – but this is fairly new territory given that such modelling normally takes time and we will need to be much more agile than traditional trials, if HE data are to be taken into account during the arm-dropping discussions. It was felt that it might be possible to pre-populate context-specific HE models during the early phases of the trial, based on the contemporaneous costs of SSI in each context and the costs of interventions, into which the clinical outcome data are dropped when available. This scheme requires further consideration and careful road testing.

Does it matter if same arm continues in a different pillar?

Answer/decision: No, it does not matter. As mentioned above, it is likely that the clinical pillars will select overlapping interventions. It is possible that these might work better in one clinical area than in others. We need to demonstrate this and undertake an appropriately powered assessment of their clinical effectiveness (or lack thereof).

How and when to transfer arms into other pillars?

Answer/decision: It will be up to both the overarching executive TMG to propose such an adaptation, with significant input and sign-off from the independent oversight committees and the funder.

Discussion theme E – Logistics, delivery and costings

How will the executive oversight group work and what is the level/height of interface with pillar teams?

Answer/decision: The platform will be designed with carefully arranged relationships between devolved/retained responsibilities. There is precedence from other platform trials that the MRC has successfully delivered.

Governance considerations – are multiple Chief Investigators (CIs) allowed?

Answer/decision: The platform will only be able to have one CI – and there is no way round this due to HRA regulations, and sponsorship/governance policies. Each pillar would have a named leader (and a senior oversight person). The protocol can perhaps list a co-CI for each pillar but only one name in the overall official CI role. A lot of the responsibility can be devolved. NB - co-CIs were allowed during the Covid pandemic but seen as an exception due to circumstances. The contract will need to reflect the responsibility of the pillar leads.

Can sites open for one over-arching Platform trial and pick which pillars to engage in (which could change over time)?

Answer/decision: Yes; that’s exactly how we want it to work

Protocol structural considerations – is it a three-level model?

Answer/decision: A Modular protocol is planned. There are examples in existence. We need to be able to add/drop/change arms without having to have the entire master protocol amended each time. Aim to leave quality assurance elements out of protocol – these should be defined in the protocol at a high level, but detail on implementation can be left out, which avoids having to make amendments many times throughout the trial.

How will the central trial management structures be organised and delivered in an efficient manner?

Answer/decision: Significant discussions and then subsequent CTU meetings on this – resulting in the creation and refinement of a whole new proposed structure for the successful delivery of the platform. This structure moves away from the standard single trial vertical infrastructure towards a more flexible horizontal system which will be able to cope with the inevitable vicissitudes of a platform trial in a far more efficient and adaptive manner. Rather than having separate teams for each pillar trial, we propose to take advantage of the single over-arching protocol and run a largely cross-speciality delivery team.

Independent oversight (TSC/DMEC) plans

Answer/decision: We definitely want one overarching TSC / DMEC for the whole platform. Need to avoid separate TSCs & DMECs for each pillar making independent decisions. These committees are likely to need to meet more frequently than usual, and make more decisions than usual – as such we would need to carefully consider who is appointed to these committees and how many core members there should be.

How will we manage the SoECAT (Schedule of Events Cost Attribution Template)?

Answer/decision: All parties agreed that it would be optimal to not have a separate SoECAT form for each pillar, but to have one over-arching one for the whole trial. This would help with the optics and psychology of trial opening and delivery at each site. Further discussions with NIHR CRN and RDS colleagues led to this single SoECAT plan being operationalised for the full application, although it was complicated.

Clinical pillar activities

In the launch meeting, we discussed and agreed the initial activities for each of the clinical teams to undertake over the first half of the accelerator award:

  • Mobilise or create a clinical research network of engaged clinicians +/- trainees

  • Establish the baseline SSI rate in their speciality based on best available evidence

  • Review potential specific interventions and their evidence within that clinical area

  • Review ongoing research activities and parallel or competed trial in each specialty

  • Start to explore clinician equipoise and appetite for both the platform trial and the proposed interventions

  • Establish or engage an existing pillar-specific PPI team

The pillar teams engaged well with these activities and reported progress back at the mid-point face-to-face meeting in London. It was agreed that each pillar would prepare standardised information for inclusion in the full application, according to the following template:

1. PICO:

•   Patient populationincluding key inclusion/exclusion criteria
•   Interventionsthe three interventions chosen and why
•   Comparatorconsiderations on control arm practice that need stating up front
•   Outcome measuresany speciality-specific additional secondary outcomes / timings to consider

2. Baseline SSI event rate - and how robust this estimate is

3. Practicalities/logistics for delivery

  • Size of the proposed population pool (number of operations performed for this type each year in the UK)

  • Evidence of community and surgeon equipoise and engagement

  • Delivery Networks including trainee collaboratives

  • PPI engagement activities and inputs

  • Follow-up issues - any specific factors warranting consideration

Each pillar also spent their allocated £6000 for a variety of engagement activities, both with clinicians and patients. These included hosting meetings of research networks to discuss the proposal with subsequent reviewing and preference-scoring of potential interventions, including undertake live voting or consensus exercises. Some groups held dedicated PPI meetings as stand-alone activities or alongside other meetings – issues such as the patient pathway, overall acceptability of randomisation, follow-up routes and options on potential interventions were discussed. The major lower limb amputations (MLLA) pillar decided to use the majority of their money to undertake a national prospective snapshot audit of MLLA operations to capture real-world complication rates and supplement the published information on the true rates of SSI at present in the UK, captured from sites across their network. This also carried the further benefit of establishing throughout rates for this operation, as well forging early successful collaboration between the active parties in this field.

Central PPI activities

The two core PPI co-applicants on the accelerator project were also very active – both in contributing to the specific pillar PPI activities and meetings, and in establishing the structure and framework for proposed PPI involvement in the full platform trial. This required work to set out the key roles, remits and responsibilities of PPI partners within the proposal, and then establishing the ‘level’ at which these should sit. An organogram for PPI activities was proposed and refined throughout the award, as shown in Figure 4.

de3df01e-1110-43e9-8440-6a6942b16ba7_figure4.gif

Figure 4. Proposed organogram for Patient and Public Involvement in ROSSINI-Platform.

Other changes made during the course of the Accelerator award

Given the iterative nature of the award, other improvements or refinements naturally developed during the preparation of the full stage 1 application for the platform trial, as outlined below.

i] Change to clinical pillars involved in the platform

The six originally proposed clinical pillars had gone through a vetting process based on factors including the magnitude and impact of SSI in that clinical area, ongoing parallel trials, pre-existing national delivery networks, clinician engagement and equipoise. A key activity for the pillar teams was to undertake detailed reviews of currently available evidence on baseline SSI rates, upon which sample size calculations could be accurately based. It became clear that there is very limited evidence on the true SSI rates in the planned thoracic pillar, and much recent activity centres on minimal access thoracoscopic surgery which carries an alleged SSI rate of <1%. A decision was made that high quality, prospective and contemporaneous information on SSI rates was needed before an RCT could be deemed deliverable. Alongside this, we were approached by specialist obstetric clinical researchers who had already conducted much background work on SSI and were designing a major RCT in this area due to be submitted imminently. As such we were delighted to welcome this new team to join the consortium. The final clinical pillar teams are shown in Table 1b.

ii] Centralised outcome assessment hub

A central challenge in undertaking high-quality SSI research is the delivery of accurate, reproducible and blinded wound assessments for the determination of the primary outcome. With >60% of SSIs occurring after discharge, the previous gold standard has been to bring all patients back to hospital for a face-to-face wound assessment and review at around 30 days after surgery. This system is expensive, increasingly challenging in the post-pandemic NHS and would simply not be deliverable in a trial of >26,000 patients. After much deliberation, and inviting additional wound surveillance experts to join the team, we agreed that a radical rethink was needed to take advantage of modern telemedicine technology, pre-existing infrastructure investment and dedicated expertise. The centralised patient follow-up hub model was proposed within the final platform application.

iii] Overall statistical design for the pillar MAMS trials

As mentioned above, we originally assumed that the constituent trials would all utilise a ‘MAMS selection’ design as per ROSSINI 2. The consortium since concluded that a ‘factorial MAMS’ statistical design provided significant advantages in terms of both flexibility and overall sample size required to answer the primary effectiveness research question for the three selected interventions. We have seen limited evidence of interaction so far in the ROSSINI 2 trial and feel it is reasonable to presume that this will continue into the platform. Based on other recent trials in SSI, it seems likely that only a small number of interventions will actually be found effective and that this design may be the most efficient way to identify them2024. Some patients will be naturally allocated to receive combination arms of multiple different interventions (A&B, A&C, B&C, A&B&C), and we will be able to assess these groups for any signal of effect at the interim analysis phase, and then formally undertake a fully powered assessment of any promising combinations in the next trial stage.

iv] Criteria for determining proposed effect sizes across the pillars

A key issue for the accelerator discussions was how to deal with the variable baseline SSI event rates between pillars and potential target effect sizes to achieve the best trade-off between deliverability and clinical impact. As a result, a fixed relative risk reduction model was postulated but we ultimately felt that a cross-pillar absolute risk reduction (ARR) strategy linked to control arm SSI rate was the most appropriate and was proposed in the full platform application.

Full platform trial application and progress thereof

As a result of the Application Acceleration project, a robust application was submitted to the NIHR HTA 23/95 Platform studies to efficiently evaluate the clinical effectiveness of multiple interventions in areas of strategic importance on November 28, 2023.

The application was successfully put through to Stage 2 of the process, and board feedback was received on January 26, 2024. This led to many meetings of the group to discuss the feedback and plan for the full stage 2 application. We were also pleased to be given permission by our NIHR Project Manager to utilise some of the underspend from the Application Accelerator Award to hold a hitherto unplanned fourth face-to-face meeting of the consortium on Wednesday March 13, 2024, to review and finalise our stage 2 application.

Discussion

The application acceleration award was highly impactful and directly enabled the creation and submission of a much-improved funding application to the NIHR HTA platform trial call. The small but efficient grant allowed us to assemble and convene the large and hitherto disparate group of experts to come together regularly and work on the project. The iterative and deliberately problem-unpicking nature of the discussions enabled maximal creative freedom and thereby identification of solutions that would have been outside the likely conclusions of a normal grant application team.

By pre-identifying a list of design or delivery challenges, we were able to start with a loose structure to guide discussions, although these tended to naturally shift themselves into theme-based discussions as the conversations progressed. The face-to-face nature of three out of the four team meetings added to fruitful discussion and engagement. Multiple additional meetings were also required alongside the main large group meetings; these were particularly important for the clinical groups and in terms of the finance and trials unit delivery aspects.

A successful element of the acceleration award was the small allocations of money (£6000) assigned to each pillar, for them to spend on whatever was felt necessary to deliver the aims and actions required. Many groups used this for engagement work with patients and/or clinicians, both to verify acceptability and engagement with the whole premise, then later on to undertake intervention refinement work. One pillar chose to also undertake a network-building prospective audit which served to add evidence about both operation throughput rates and SSI rates. This decentralised small funding pots were appreciated by the clinical pillar teams and we would recommend them to other groups looking to undertake similar project development work.

The benefits of the application acceleration award were demonstrated when our stage 1 application progressed through the funding committee discussions and a full-scale application was subsequently invited. We were able to continue the fruitful collaboration and reassemble the group to respond to board feedback and create a cogent full application. The final outcome is awaited.

We strongly endorse the application acceleration awards scheme and feel that the NIHR should consider offering more of these excellent funding opportunities ahead of major commissioned calls in the future.

Consent statement

Not applicable

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 12 Dec 2024
Comment
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
VIEWS
832
 
downloads
60
Citations
CITE
how to cite this article
The ROSSINI-Platform Investigators Collaborating Group*. ROSSINI-Platform - Reduction of surgical site infection with a platform trial utilising a 'Basket-MAMS' design - Application acceleration award [version 1; peer review: 1 approved with reservations]. NIHR Open Res 2024, 4:76 (https://doi.org/10.3310/nihropenres.13641.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 12 Dec 2024
Views
8
Cite
Reviewer Report 30 Jan 2025
Nida Zahid, The Aga Khan University, Karachi, Pakistan 
Approved with Reservations
VIEWS 8
Overall 
The article effectively describes the process of designing and planning a large-scale, multi-specialty platform trial (ROSSINI-Platform) aimed at reducing surgical site infections (SSIs). The authors provide a comprehensive overview of the challenges and considerations involved in developing such ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Zahid N. Reviewer Report For: ROSSINI-Platform - Reduction of surgical site infection with a platform trial utilising a 'Basket-MAMS' design - Application acceleration award [version 1; peer review: 1 approved with reservations]. NIHR Open Res 2024, 4:76 (https://doi.org/10.3310/nihropenres.14811.r34367)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 12 Dec 2024
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Are you an NIHR-funded researcher?

If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research.

You must provide your first name
You must provide your last name
You must provide a valid email address
You must provide an institution.

Thank you!

We'll keep you updated on any major new updates to NIHR Open Research

Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.