Keywords
Surgical Site Infection, Randomised Controlled Trial, Adaptive designs, MAMS Platform Trial, Acceleration Award, Surgery, Research Methodology
Surgical site infection (SSI) is the most common surgical complication, resulting in significant morbidity, mortality, and major costs to health service providers. Multi-arm, multi-stage (MAMS) trials allow multiple individual interventions to be evaluated simultaneously, rendering them highly efficient at both speed and cost. Our group is running the successful ROSSINI 2 MAMS trial in abdominal SSI reduction, which was funded by HTA and proved that such trials are deliverable and effective in a modern surgical context.
Our proposed platform study will run multiple parallel RCTs in different surgical cohorts, utilising a ‘Basket-MAMS’ design to simultaneously assess multiple interventions and will share learning between cohorts to take successful arms from one area to another for rapid testing. This would provide robust and context-specific evidence for rapidly improving patient care.
We identified a list of key methodological, logistical, health economic, PPI-related, and clinical design challenges. A large study team was assembled, consisting of both core experts and national clinical research leaders, including robust plans for capacity building, Chief Investigator (CI) development, and multi-level PPI engagement. The working groups designed a series of parallel work packages to address the identified challenges, both centrally and in clinical networks, convening regularly to discuss and plan the full application. Iterative modifications and improvements to the overall proposal were allowed and encouraged.
We report the progress, activities undertaken, and decisions made throughout the application acceleration award project. This information may be beneficial to planning or preparing a similar large-scale and multi-specialty platform trial.
The acceleration grant allowed us to design a fundable and deliverable clinical trial for the HTA platform trial call in late 2023. The full trial has the potential to significantly impact the rates of this highly impactful complication, thereby benefitting both patients and health services worldwide.
Up to a third of patients undergoing major surgery develop an infection of the wound (called a surgical site infection (SSI)). SSI is a major problem and costs NHS at least £700 million per year.
There are many ways in which surgeons can try to prevent SSI, but very few have been properly tested to determine how well they work. Our group is currently running a trial called ROSSINI 2 to look at reducing SSI after abdominal (tummy) surgery. ROSSINI 2 is a multi-arm, multi-stage trial. This means that at the beginning of the trial, there was a set number of treatments used during the operation to reduce infection, which were reviewed at set times during the trial. Treatments that have not performed well are dropped, and this continues until the end of the trial, when only the treatment(s) that work best remain.
The proposed ROSSINI-Platform Trial builds and expands upon ROSSINI 2, creating a trial across six surgical specialties or ‘pillars’: breast surgery, vascular groin surgery, cardiac (heart) surgery, neurosurgery (brain), leg amputations, and thoracic (lung) surgery.
By running these six otherwise separate trials together within one ‘platform’ of a single overarching design structure, we can make the management of trials simple, faster and cheaper. We will also be able to have an overview across all pillars and move the treatments proven to work in one part of the body into other groups for rapid testing. This is a very efficient design and will result in improved patient outcomes much quicker than traditional trials. Both patients and the NHS will benefit.
Before starting the full-platform trial, we need to address several design and logistical challenges. This award allowed us to bring our team together and undertake the necessary planning work.
Surgical Site Infection, Randomised Controlled Trial, Adaptive designs, MAMS Platform Trial, Acceleration Award, Surgery, Research Methodology
In November 2022, there were 7.07 million patients waiting for planned hospital treatment within the NHS, the majority of which were elective surgical operations1. The rising backlog and ongoing significant pressures on the service mean that there has never been a more important time for preventable postoperative complications to be avoided.
Surgical Site Infection (SSI) is now the most common healthcare-associated infection, overtaking MRSA, Clostridium difficile, hospital-acquired pneumonia, and catheter-associated infections after significant and sustained investment in these other areas in recent years2.
An unacceptably high proportion of patients undergoing surgery develop an SSI and, as such, are exposed to preventable pain, suffering, delayed further treatment, and even a risk of death. The complication also costs NHS at least £700 million per year3–5.
A key shortcoming of all current national and international guidelines on the reduction of SSI is their highly generic nature: the recommendations made apply to all types of surgery and are not speciality-specific or surgical site/target organ-specific6–8. This is largely a result of the small number of high-quality trials making up the evidence base across surgery, meaning that research findings from one surgical indication are inevitably extrapolated to other settings. This can lead to both overpromotion of interventions that have shown clinical effectiveness within a discrete area of surgery and the discarding of others that might confer benefit when utilised in a different setting.
Recommendations are often made based on the contamination level of the wound. This primary classification system for wounds by contamination (clean, clean-contaminated, contaminated, and dirty) has been in place since 19649. Although the system can be useful to demonstrate the range of operation types and subsequent highly variable risk of SSI from <2% to >40%, it is overly simplistic. Within this classification, an elective wrist operation, open abdominal aortic aneurysm repair, and leg amputation are all pooled together as ‘clean’ operations. Similarly, elective operations for both lung cancer and rectal cancer can be ‘clean-contaminated’. While the etiology of SSI is multifactorial, it is increasingly clear that mechanisms of SSI development, including the source and nature of the pathogenic organisms responsible, are not the same across the body.
We believe that more accurate testing of SSI prevention measures would allow evidence-driven stratification according to the specific procedure a patient is undergoing. This would confer significant benefits for both patients and the health service due to a reduction in SSI across the panoply of surgery.
It is increasingly clear that a single ‘magic bullet’ to prevent SSI does not exist; rather a cumulative package of measures for a particular operation type or surgical setting are likely to be most effective. However, most ‘bundle’ trials of interventions to reduce SSI have failed to produce convincing or reliable results.
Some years ago, our group realized that the multi-arm, multi-stage (MAMS) platform design allows multiple individual interventions to be evaluated simultaneously, utilising factorial design but a nonfactorial analysis in which interactions between interventions can be allowed and formally explored. This ‘unbundling’ design produces evidence-based combination(s) of interventions identified as the optimised preventative strategy for SSI in that setting. The simultaneous nature of multi-arm assessment renders it highly efficient in terms of both speed and cost. The methodology has been previously proven and highly successful in the STAMPEDE suite of trials in prostate cancer10–13 but had not been used in a surgical context until that time. We established a collaboration with the pioneer of MAMS design, Professor Mahesh Parmar at the Medical Research Council Clinical Trials Unit (MRC CTU), and his team to create the ROSSINI 2 trial.
ROSSINI 2 is a major multicentre RCT exploring the effectiveness of three intraoperative interventions for the reduction of SSI in patients undergoing abdominal surgery, utilising a MAMS selection design14,15. Within it, the three interventions are assessed both individually and in combination, resulting in seven treatment arms which are compared against a single (double-recruited) control arm in a 2:1:1:1:1:1:1:1 ratio. The least effective arms are dropped at prespecified interim analysis stages, leaving a final confirmatory phase where the best arms (which may be individual interventions or combinations of interventions) are compared against control. The original ROSSINI 2 trial schema is shown in Figure 1.
ROSSINI 2 was funded by NIHR HTA in 2018, and commenced recruitment in March 2019. By the time of our application for the accelerator award, ROSSINI 2 had recruited >3700 patients from 52 sites across the UK and proved the principle that MAMS RCTs are both deliverable and efficient in a modern surgical context. The first planned interim analysis was performed in early 2022, at which point four treatment arms were dropped owing to a lack of benefit.
ROSSINI 2 was and remains the highest-recruiting surgical trial in the NIHR portfolio. Given the ongoing success of the trial, despite the pandemic, we communicated with HTA regarding the potential cost and practicalities of adding new interventions into the existing trial structure. This was subsequently awarded, and the extension phase of the trial went live in April 2024).
While ROSSINI 2 will deliver robust information on interventions to reduce SSI in abdominal surgery, the findings will not necessarily be translatable to other types of surgery for the reasons outlined earlier.
In mid-2022, we created the outline of a proposal for a new platform trial that will build upon and extend the principles established in ROSSINI 2 across a wide variety of surgical settings. The proposed platform trial would utilize a bespoke ‘Basket-MAMS’ methodology in which multiple full-scale MAMS RCTs will be run in parallel within different surgical cohorts (‘pillars’), organized by wound type/site, under one over-arching protocol and governance structure. These MAMS trials will simultaneously assess multiple carefully-selected specific interventions alone or in combination and will crucially share learning, both clinical and statistical, between pillars to take effective arms (individual interventions or combinations of interventions) from one surgical cohort to the others for rapid re-testing. This iterative approach would be doubly efficient because the MAMS design allows within-trial interim analyses and dropping of ineffective arms, and then the prespecified transference of successful arms into vacated slots in other pillar RCTs within the lifetime of the overall planned protocol. This will provide robust and context-specific evidence in a highly efficient manner, in terms of both time and cost. Overall learning and efficiencies may be further enhanced by utilising the over-arching and standardized protocol, outcome measures, and homogenized datasets to synergize new data in two additional ways: i) considering borrowing information for patients across different pillars who receive the same or similar interventions for combined analysis and ii) undertaking prespecified individual patient data meta-analyses (IPDMA) of outcomes, including those in other (external) trials using the FAME structure16. This has the potential to further shorten the timelines to patient benefit.
The platform element of running these parallel RCTs under one overall organizational structure would yield further benefits from a governance, oversight, delivery, and analysis point of view. We anticipated a ‘master protocol’ which would cover the generic elements of trial procedures and governance, and then individual sub-protocols for the separate cohorts. Hospital sites would open for the overall trial and then select which of the constituent studies to participate in after appropriate capability and capacity assessments and site initiation/opening procedures, which will depend on local facilities, interest, and equipoise.
The reduction of surgical site infection has been a clear area of research need for our patients for many years, as witnessed by the repeated identification of this topic across a range of priority setting partnerships in several different surgical specialities.
Significant PPI work was done during the development of ROSSINI 2, and this was taken into account in the development of the scaled-up platform trial proposal. The project has been further developed in conjunction with an experienced patient representative, who is the patient representative sitting on the independent Trial Steering Committee of the current ROSSINI 2 trial. She has therefore already been involved already in the concept and design of the project and understands the overall premise and the MAMS design. Similarly, her direct personal experiences of undergoing multiple major operations has been invaluable.
We have also invited an experienced PPI manager with many years’ experience of leading PPI activity and working with patients and the public on health-related research to be a key part of the core management team. Both individuals will continue to be involved going into the full project, if funded, and have agreed to sit on the executive trial management group for the project.
The scale of the project has led us to involve two core patient and public representatives in this planning stage, with the expectation that each clinical pillar will subsequently also appoint multiple specific PPI representatives going forwards.
It is clear that significant PPI work will need to be undertaken both during the design phase and then as an active and ongoing key component of all of the parallel trials as well as the overarching management group if the platform trial is to be successful. We will build upon and scale up the positive models created within the ROSSINI 2 trial to deliver this – including the concept that a large MAMS trial is more complex, rapid and iterative compared to a normal RCT and as such a minimum of three PPI co-applicants should sit on each pillar’s trial management group. In addition to bringing a wealth of viewpoints, this system will limit the onus of responsibility on one individual going into the future.
Within this Application Accelerator Award, a key aspect for the individual clinical groups will be ensuring PPI remains central in each of their pillar trial designs and delivery proposals. An early activity (see Gantt chart) will be to identify suitable patient groups and/or individuals who can help in these roles In a similar manner to how we will define the key design elements for selection of interventions and other protocol factors within this initial study design award, we will also create a minimum set of standards for PPI engagement and activities which each speciality pillar will adhere to.
There were multiple factors within the complex fledgling proposal which needed careful and structured planning if a credible and fundable bid for the 2023 HTA Platform Studies call was to be created. These comprised logistical and organizational challenges of a scale not previously seen in surgical research. We were therefore very grateful for the opportunity to apply for the Application Acceleration Award to bring the team together to undertake this necessary prework.
Multiple over-arching elements of overall design, high-level methodology, analysis strategy, oversight, governance, health economics, and core PPI elements all needed unpicking – in addition to the equally important clinically driven aspects, including evidence reviews to select interventions, exploring community and individual surgeon equipoise, delivery network development, patient pathway work, speciality-specific PPI engagement, and input. The management structure for the accelerator project, therefore, intentionally mirrored that of the proposed full platform trial in that the necessary specialist groups were key partners from the outset, working alongside the central team members with regular meetings and interactions. The initial organogram of the platform is shown in Figure 2, and the two main teams are described below.
We assembled a core team with internationally recognized expertise in methodology, statistics, trial delivery, health economics, and PPI. Most of the core team members already had a track record of successful collaboration in the ROSSINI 2 trial. This team will work up the proposal into a fundable HTA application and then go on to form the central trial management team with oversight and responsibility for the core elements of the Basket-MAMS platform if the platform is funded.
The application acceleration award contained costs for a dedicated Project Manager to coordinate all activities and meetings, ensure progression against the proposed timelines, and maintain and develop logs of discussions and outcomes to feed into the full application.
Individual pillar RCTs were proposed to be run by separate but linked teams of specialist surgeons and collaborative surgical research groups within each surgical area, along with a dedicated trial delivery team. A similar model of having separate specialist PIs for constituent trials within a larger trial structure has been successfully used by others, including within STAMPEDE trials.
The UK surgical community came together behind the platform accelerator bid with high-level engagement from experienced research leaders in each of the specialties involved. We worked with the Royal College of Surgeons of England Clinical Research Surgical Speciality Leads (SSLs), who are charged with developing and delivering clinical research networks across the country; the SSLs have often led HTA trials or major research programs previously themselves. Therefore, we created a model where each will mentor a relatively novice or early career researcher to run their individual pillar RCT, thereby increasing research capacity and expertise within the relatively protected environment of the overall platform trial. A similar structure of mentorship and training is currently being successfully used in the MRC CTU ACORD Fellows Academy, which we will emulate17. The clinicians involved in the original application, and their backgrounds, are shown in Table 1 a.
The application acceleration award proposal also contained six dedicated pots of money (£6000 per pillar) for the coordination and conduct of meetings between surgeons, trainees, and patient groups during the development of each clinical pillar. This included mini-Delphi exercises, patient focus groups, and room hire/logistical support for clinicians to come together for a parallel session at a national meeting. Each mini-budget was to be utilized at the discretion of the individual pillar team, with support from the core group.
During the early development phase of the proposal and the conversations undertaken in planning for the application acceleration award, a list of key design, delivery, and organizational questions or issues emerged, all of which needed to be unpicked, discussed, and finalised as part of the project. These were categorized into a variety of broad themes, as summarised in Table 2, although it was also anticipated that additional items would emerge as part of the iterative process of the full platform trial design. This list of questions was included (verbatim) within the application for the accelerator award.
The specification of the grant stream involves relatively tight time and cost constraints. The team prespecified the proposed work packages, timelines, and meeting schedules/frequency for the accelerator award, which are shown in the Gantt chart (Figure 3).
In addition to addressing the key questions and design considerations previously outlined, the team also pledged to maintain careful oversight of the core value proposition inherent within any full platform trial application during the accelerator project. It was important to identify and utilise every opportunity to achieve logistical and financial efficiency. It remained clear that the subsequent proposal for the full platform trial could not be equivalent to requesting funding for six fully costed standalone trials, plus additional platform oversight costs. We aimed to ensure that a cogent, streamlined, and efficient trial proposal would be submitted, making full use of any shared elements.
The accelerator project grant was awarded in February 2023, with a start date of 1st March 1, 2023, and a 9-month project duration. The deadline for submission of a call to the full 23/95 Platform studies to efficiently evaluate the clinical effectiveness of multiple interventions in areas of strategic importance was 28th November 28, 2023.
During the course of the project, the team undertook a variety of meetings within both specific subgroups and clinical pillar teams as well as bringing the whole group together for the three major meetings, as follows:
Tuesday 14th March 2023 [online] – Launch meeting
Tuesday 27th June 2023 [face-to-face at MRC CTU, London] – Mid-point meeting
Thursday 9th November 2023 [face-to-face at UoB, Birmingham] – Final meeting
These meetings were also attended by a variety of additional invitees, including representatives from) NIHR Clinical Research Network (CRN), ) NHS trust Research Application Support (RAS) team, ) Research Strategy & Services Division (RSSD), University of Birmingham, and the Head of Research Governance & Integrity, University of Birmingham. All these additional parties provided invaluable further advice and opinions into the design, costing, and proposed conduct of the full platform trial application, and we are grateful for their input.
Firstly, we agreed a set of key design/delivery tenets within the first month of the project:
One protocol, with one overarching trial pathway across the different pillars, which are each individually-powered trials.
Sites will open for whichever clinical pillars they have availability, interest and equipoise.
Benefits come from efficiencies such as costs, approval of one protocol, and a single overarching trial team.
Calculating the cost savings of one platform trial versus six separate trials will strengthen the case for the platform.
Outcome assessment will use common methods across pillars, despite interventions (and surgical wound site locations on the body) being different.
High yield interventions in one pillar can be moved across to other specialties for testing.
The meetings were run in an iterative format with loose outline agendas and the list of predefined questions/issues (Table 2) used as a structure to aid discussions, although we were not proscriptive or confined by these categorisations. An example would be the discussion around the selection and prioritisation of interventions for the clinical pillars, as these conversations involved methodological, statistical, logistical, PPI, and health economic considerations. As such, the meetings tended to structure themselves around discussion ‘themes’ – this was therefore the optimal format for presenting the project outcomes and decisions below.
Discussion theme A – Overarching statistical policies
How do we cope with different baseline SSI rates between pillars?
Answer/decision: This does not matter. It is a fact that each clinical area will have a different baseline SSI rate. We will be essentially running six separately powered trials under one overarching governance/structure. Therefore, they will have their own sample size calculations, and from a mathematical point of view, be separate trials.
Are we aiming for same relative risk reduction/effect size in each pillar?
Answer/decision: No, not necessarily. While it might make it easier for the reader or funder to understand, the fact that these are separately powered trials (see above) means that we can cope with different proposed effect sizes or relative risk reductions. This is important as it allows us to mould the trial according to the currently available evidence available for the specific interventions in the specific clinical contexts. It also allows us to potentially aim for a larger effect size (where justifiable) in those specialties with a lower baseline event rate to keep the pillars within what is a practicably deliverable sample size.
Should we aim to use Adaptive randomisation?
Answer/decision: No - it does not really add anything useful from a statistical or time-efficiency point of view in this context.
Timing of interim analyses – do they need to occur at the same time?
Answer/decision: There will be different sample size calculations for each pillar, and they are likely to open at different times and deliver differing recruitment rates. As such there is no way we can time the interim analyses to happen at a similar time between pillars. We are accepting of this. It will, however, make more work for the over-arching Data Monitoring and Ethics Committee (DMEC) and Trial Steering Committee (TSC).
Do we need a unified single endpoint across all pillars?
Answer/decision: We could perhaps use different endpoints within this study, if desired, under the umbrella of the platform design. This could be useful for different endpoint timings, or in those areas where the primary outcome of interest might not be pure SSI, such as in amputation stumps, when other wound issues cause major ramifications to ambulation rates. However, this would make the inter-pillar transfer of interventions much harder. In addition, running the trial under one master protocol in terms of outcome measures and their timing would be much more challenging. As such, we probably have to set some boundaries to maintain some similarly/inter-digitation between pillars. Timing and diagnosis of primary outcome seems like a major one to unify across pillars (also see later discussions on diagnosis of SSI).
Overall statistical design options
Answer/decision: When we wrote the original application acceleration award, we assumed that the platform would utilise a MAMS selection design18. Our discussions led onto consideration of two overall design models:
Option A = MAMS selection design (a.k.a. ‘drop the loser’): This helps pre-define the maximum sample size and reduces the uncertainty about the actual sample size of the study in practice, which is, in turn, easier to cost. This is how we eventually got ROSSINI 2 funded as it decreased the number of unknowns.
Option B = Factorial Basket MAMS19: means that pillars or interventions can start at different times which may have benefits considering the scale of what is proposed. This is essentially multiple MAMS trials. Also easy to cost, and sample sizes will also be smaller than a MAMS selection design. The decision to choose between the two options depends heavily on whether we anticipate significant interaction between interventions or not.
Discussion theme B – Choice of interventions
Do we need core rules for standardised selection across pillars?
Answer/decision: No – we do not need to be too strict. We would rather have interventions that have a credible chance of working in each clinical context, as backed up by the available evidence, but they are not in current standard usage, and/or surgeons would have equipoise to accept randomisation to both use and not use them in their operations. The key tenet to remember is that pillars must not choose internally cancelling interventions (i.e., they must be able to use all during one operation).
Should we have a centrally-created list of ‘approved’ interventions for teams to pick from?
Answer/decision: This was not felt to be necessary and would be very challenging to the police. There are interventions which are context-specific, which would never be applicable in other pillars (e.g., how to deal with the bone flap after craniotomy in neurosurgical practice). The currently available evidence of effectiveness is also very context-specific. We would therefore rather the clinical pillars did their own explorations into the most appropriate interventions for their area. Secondary discussions were had about cost of interventions and whether there needs to be ‘rules’ on this, and/or if the interventions in each pillar need to be roughly equivalent in cost to each other. The agreed answer was that we do not need to do this. Note – this also feeds into early discussions on how to incorporate health economic evidence into the arm-dropping decisions at each interim analysis – see section D.
Can identical interventions appear in multiple pillars?
Answer/decision: Yes, we are happy with this. This is likely to happen, but it might be a good thing as this trial would be the first to show whether the same intervention works better, worse, or the same in different parts of the body.
What level of evidence/clinical effectiveness do we need?
Answer/decision: The best available. We are not prespecifying a certain effect size or level of evidence; it will be up to each pillar to meet and discuss, both within their teams and the wider clinical community to decide and agree.
Must all interventions be currently available in UK hospitals and CE/UKCA-marked – or could we do a CTIMP?
Answer/decision: While it might be easier to not propose to run a CTIMP platform trial, most felt that it would be a shame to up-front rule out interventions not yet fully authorised for market, or which are classified as a drug, and that the extra work in making this a CTIMP trial would not be that significant. The group, therefore, remained open to this, and no limitations were placed on clinical teams in their selection of interventions.
Cost of interventions – is it a consideration?
Answer/decision: No – It is not important at this stage of intervention selection and justification.
When do we explore equipoise of clinicians – and of patients?
Answer/decision: Within the latter phases of the application accelerator award, as soon as candidate interventions have been identified. In the full application we need to propose interventions which are acceptable and deliverable, from the point of view of all parties.
Do we need to standardise control arm policies?
Answer/decision: Yes, to some extent - but within each pillar and not across pillars. This will of course remain a pragmatic trial and as such we will aim to be as non-proscriptive as possible. It is likely that, as per ROSSINI 2, there may be 10–20 other peri-operative interventions or procedural variables which need to be collected on an individual patient-level in order that they can be controlled for in the analyses. Clinical pillars will be responsible for selecting these factors as part of the detailed design phase at stage two. We will need to consider this more when interventions have been determined.
Do we need to consider the timing of intervention delivery within the operative pathway?
Answer/decision: Specifically, picking interventions that sit in different operative phases – for example, pre-, intra-, and post-operation – might be one way to give flexibility and correlation across disciplines. However, it was decided that this would also be too restrictive.
What about sustainability and ‘Green’ aspects?
Answer/decision: It was agreed that this is a hot topic; we need to be mindful of ensuring that interventions are acceptable in this sphere. We may also be able to suggest the potential for offsetting any “carbon cost” against the reduction in hospital time/morbidity/re-admission associated with SSIs. It was also agreed that we should be sure to promote any antibiotic-sparing potential of interventions, where appropriate.
Discussion theme C – Outcome measures
What definition of SSI should used, and is the 30 day timing of assessment fixed (noting the likely used of implants in some specialties)?
Answer/decision: Most clinical pillar teams felt that a 30-day primary endpoint would be ok. In cardiac surgery (due to implant usage in sternal wires and other implants) this timepoint is less typical, however it was agreed that the majority of SSIs still present before 30 days, so this would work probably. Measuring ongoing impact would be important to the cardiac team. There are similar concerns in vascular procedures involving prostheses.
What is the optimal standardised method for follow-up – including phone/video?
Answer/decision: Many hours of discussion, and additional dedicated meetings, were held on this point. All parties agreed that it is critical to get this right if the trial was to be deliverable and impactful. We have learnt a lot during the pandemic about how to do this, and move forwards from the previous ‘gold standard’ of a face to face review of the patient with in-person wound examination.
Wound photos can have issues around consent, centralisation, white balance and adjudication. The quality of photo can have major implication on judging severity of SSI. Photos alongside descriptions from patient can work better. Photos could be used as part of a triage tool, with phoning of the patient where SSI is suspected. It may depend if the patient has someone who can take a photo for them. Some patients may not be comfortable sending photo e.g. breast surgery. Video can be effective by enabling patient to move as required but this depends on capability of the patient (e.g. more elderly patients may struggle?)
The TALON study assessed high-quality remote wound assessments within the CHEETAH trial and compared CDC assessments in person versus remotely via video. All agreed that the ‘Bluebelle’ wound healing questionnaire must be incorporated; it has now been validated across several trials.
There were ongoing questions and discussions about whether more regular post-discharge interactions would be feasible, and what it is that patients want.
See later section on the Centralised Digital Wound Hub (CDWH).
What about the severity of SSI concept, rather than a binary yes/no system?
Answer/decision: The current system, as used by everyone, is based on CDC definitions, and is a binary yes/no. All agreed that a new paradigm of adding in an “if yes, is it mild, moderate, or severe SSI’ could be beneficial. Key determinants of severity may include both the impact on patients and health services. This is exploratory at this stage, as it is not validated; therefore, it cannot be used as a primary outcome. The severity of an SSI will likely also vary between clinical contexts, for example, a severe neurosurgical SSI would be very different from a severe breast SSI.
The breast team already uses a similar scheme in some of their work – one delineator may be: Mild SSI – Outpatient antibiotics; Moderate SSI – IV antibiotics; Severe – Return to theatre or other reinterventions with definite prolonged LOS. It was agreed that a substudy within this large proposed multispecialty trial could define the severity strata/definitions for each context, which would be of great help to future researchers.
Discussion theme D – Arm dropping and inter-pillar transfer
Criteria for dropping arms – and are health economic (HE) considerations included?
Answer/decision: Potentially yes – but this is fairly new territory given that such modelling normally takes time and we will need to be much more agile than traditional trials, if HE data are to be taken into account during the arm-dropping discussions. It was felt that it might be possible to pre-populate context-specific HE models during the early phases of the trial, based on the contemporaneous costs of SSI in each context and the costs of interventions, into which the clinical outcome data are dropped when available. This scheme requires further consideration and careful road testing.
Does it matter if same arm continues in a different pillar?
Answer/decision: No, it does not matter. As mentioned above, it is likely that the clinical pillars will select overlapping interventions. It is possible that these might work better in one clinical area than in others. We need to demonstrate this and undertake an appropriately powered assessment of their clinical effectiveness (or lack thereof).
How and when to transfer arms into other pillars?
Answer/decision: It will be up to both the overarching executive TMG to propose such an adaptation, with significant input and sign-off from the independent oversight committees and the funder.
Discussion theme E – Logistics, delivery and costings
How will the executive oversight group work and what is the level/height of interface with pillar teams?
Answer/decision: The platform will be designed with carefully arranged relationships between devolved/retained responsibilities. There is precedence from other platform trials that the MRC has successfully delivered.
Governance considerations – are multiple Chief Investigators (CIs) allowed?
Answer/decision: The platform will only be able to have one CI – and there is no way round this due to HRA regulations, and sponsorship/governance policies. Each pillar would have a named leader (and a senior oversight person). The protocol can perhaps list a co-CI for each pillar but only one name in the overall official CI role. A lot of the responsibility can be devolved. NB - co-CIs were allowed during the Covid pandemic but seen as an exception due to circumstances. The contract will need to reflect the responsibility of the pillar leads.
Can sites open for one over-arching Platform trial and pick which pillars to engage in (which could change over time)?
Answer/decision: Yes; that’s exactly how we want it to work
Protocol structural considerations – is it a three-level model?
Answer/decision: A Modular protocol is planned. There are examples in existence. We need to be able to add/drop/change arms without having to have the entire master protocol amended each time. Aim to leave quality assurance elements out of protocol – these should be defined in the protocol at a high level, but detail on implementation can be left out, which avoids having to make amendments many times throughout the trial.
How will the central trial management structures be organised and delivered in an efficient manner?
Answer/decision: Significant discussions and then subsequent CTU meetings on this – resulting in the creation and refinement of a whole new proposed structure for the successful delivery of the platform. This structure moves away from the standard single trial vertical infrastructure towards a more flexible horizontal system which will be able to cope with the inevitable vicissitudes of a platform trial in a far more efficient and adaptive manner. Rather than having separate teams for each pillar trial, we propose to take advantage of the single over-arching protocol and run a largely cross-speciality delivery team.
Independent oversight (TSC/DMEC) plans
Answer/decision: We definitely want one overarching TSC / DMEC for the whole platform. Need to avoid separate TSCs & DMECs for each pillar making independent decisions. These committees are likely to need to meet more frequently than usual, and make more decisions than usual – as such we would need to carefully consider who is appointed to these committees and how many core members there should be.
How will we manage the SoECAT (Schedule of Events Cost Attribution Template)?
Answer/decision: All parties agreed that it would be optimal to not have a separate SoECAT form for each pillar, but to have one over-arching one for the whole trial. This would help with the optics and psychology of trial opening and delivery at each site. Further discussions with NIHR CRN and RDS colleagues led to this single SoECAT plan being operationalised for the full application, although it was complicated.
In the launch meeting, we discussed and agreed the initial activities for each of the clinical teams to undertake over the first half of the accelerator award:
Mobilise or create a clinical research network of engaged clinicians +/- trainees
Establish the baseline SSI rate in their speciality based on best available evidence
Review potential specific interventions and their evidence within that clinical area
Review ongoing research activities and parallel or competed trial in each specialty
Start to explore clinician equipoise and appetite for both the platform trial and the proposed interventions
Establish or engage an existing pillar-specific PPI team
The pillar teams engaged well with these activities and reported progress back at the mid-point face-to-face meeting in London. It was agreed that each pillar would prepare standardised information for inclusion in the full application, according to the following template:
1. PICO:
2. Baseline SSI event rate - and how robust this estimate is
3. Practicalities/logistics for delivery
Size of the proposed population pool (number of operations performed for this type each year in the UK)
Evidence of community and surgeon equipoise and engagement
Delivery Networks including trainee collaboratives
PPI engagement activities and inputs
Follow-up issues - any specific factors warranting consideration
Each pillar also spent their allocated £6000 for a variety of engagement activities, both with clinicians and patients. These included hosting meetings of research networks to discuss the proposal with subsequent reviewing and preference-scoring of potential interventions, including undertake live voting or consensus exercises. Some groups held dedicated PPI meetings as stand-alone activities or alongside other meetings – issues such as the patient pathway, overall acceptability of randomisation, follow-up routes and options on potential interventions were discussed. The major lower limb amputations (MLLA) pillar decided to use the majority of their money to undertake a national prospective snapshot audit of MLLA operations to capture real-world complication rates and supplement the published information on the true rates of SSI at present in the UK, captured from sites across their network. This also carried the further benefit of establishing throughout rates for this operation, as well forging early successful collaboration between the active parties in this field.
The two core PPI co-applicants on the accelerator project were also very active – both in contributing to the specific pillar PPI activities and meetings, and in establishing the structure and framework for proposed PPI involvement in the full platform trial. This required work to set out the key roles, remits and responsibilities of PPI partners within the proposal, and then establishing the ‘level’ at which these should sit. An organogram for PPI activities was proposed and refined throughout the award, as shown in Figure 4.
Given the iterative nature of the award, other improvements or refinements naturally developed during the preparation of the full stage 1 application for the platform trial, as outlined below.
i] Change to clinical pillars involved in the platform
The six originally proposed clinical pillars had gone through a vetting process based on factors including the magnitude and impact of SSI in that clinical area, ongoing parallel trials, pre-existing national delivery networks, clinician engagement and equipoise. A key activity for the pillar teams was to undertake detailed reviews of currently available evidence on baseline SSI rates, upon which sample size calculations could be accurately based. It became clear that there is very limited evidence on the true SSI rates in the planned thoracic pillar, and much recent activity centres on minimal access thoracoscopic surgery which carries an alleged SSI rate of <1%. A decision was made that high quality, prospective and contemporaneous information on SSI rates was needed before an RCT could be deemed deliverable. Alongside this, we were approached by specialist obstetric clinical researchers who had already conducted much background work on SSI and were designing a major RCT in this area due to be submitted imminently. As such we were delighted to welcome this new team to join the consortium. The final clinical pillar teams are shown in Table 1b.
ii] Centralised outcome assessment hub
A central challenge in undertaking high-quality SSI research is the delivery of accurate, reproducible and blinded wound assessments for the determination of the primary outcome. With >60% of SSIs occurring after discharge, the previous gold standard has been to bring all patients back to hospital for a face-to-face wound assessment and review at around 30 days after surgery. This system is expensive, increasingly challenging in the post-pandemic NHS and would simply not be deliverable in a trial of >26,000 patients. After much deliberation, and inviting additional wound surveillance experts to join the team, we agreed that a radical rethink was needed to take advantage of modern telemedicine technology, pre-existing infrastructure investment and dedicated expertise. The centralised patient follow-up hub model was proposed within the final platform application.
iii] Overall statistical design for the pillar MAMS trials
As mentioned above, we originally assumed that the constituent trials would all utilise a ‘MAMS selection’ design as per ROSSINI 2. The consortium since concluded that a ‘factorial MAMS’ statistical design provided significant advantages in terms of both flexibility and overall sample size required to answer the primary effectiveness research question for the three selected interventions. We have seen limited evidence of interaction so far in the ROSSINI 2 trial and feel it is reasonable to presume that this will continue into the platform. Based on other recent trials in SSI, it seems likely that only a small number of interventions will actually be found effective and that this design may be the most efficient way to identify them20–24. Some patients will be naturally allocated to receive combination arms of multiple different interventions (A&B, A&C, B&C, A&B&C), and we will be able to assess these groups for any signal of effect at the interim analysis phase, and then formally undertake a fully powered assessment of any promising combinations in the next trial stage.
iv] Criteria for determining proposed effect sizes across the pillars
A key issue for the accelerator discussions was how to deal with the variable baseline SSI event rates between pillars and potential target effect sizes to achieve the best trade-off between deliverability and clinical impact. As a result, a fixed relative risk reduction model was postulated but we ultimately felt that a cross-pillar absolute risk reduction (ARR) strategy linked to control arm SSI rate was the most appropriate and was proposed in the full platform application.
As a result of the Application Acceleration project, a robust application was submitted to the NIHR HTA 23/95 Platform studies to efficiently evaluate the clinical effectiveness of multiple interventions in areas of strategic importance on November 28, 2023.
The application was successfully put through to Stage 2 of the process, and board feedback was received on January 26, 2024. This led to many meetings of the group to discuss the feedback and plan for the full stage 2 application. We were also pleased to be given permission by our NIHR Project Manager to utilise some of the underspend from the Application Accelerator Award to hold a hitherto unplanned fourth face-to-face meeting of the consortium on Wednesday March 13, 2024, to review and finalise our stage 2 application.
The application acceleration award was highly impactful and directly enabled the creation and submission of a much-improved funding application to the NIHR HTA platform trial call. The small but efficient grant allowed us to assemble and convene the large and hitherto disparate group of experts to come together regularly and work on the project. The iterative and deliberately problem-unpicking nature of the discussions enabled maximal creative freedom and thereby identification of solutions that would have been outside the likely conclusions of a normal grant application team.
By pre-identifying a list of design or delivery challenges, we were able to start with a loose structure to guide discussions, although these tended to naturally shift themselves into theme-based discussions as the conversations progressed. The face-to-face nature of three out of the four team meetings added to fruitful discussion and engagement. Multiple additional meetings were also required alongside the main large group meetings; these were particularly important for the clinical groups and in terms of the finance and trials unit delivery aspects.
A successful element of the acceleration award was the small allocations of money (£6000) assigned to each pillar, for them to spend on whatever was felt necessary to deliver the aims and actions required. Many groups used this for engagement work with patients and/or clinicians, both to verify acceptability and engagement with the whole premise, then later on to undertake intervention refinement work. One pillar chose to also undertake a network-building prospective audit which served to add evidence about both operation throughput rates and SSI rates. This decentralised small funding pots were appreciated by the clinical pillar teams and we would recommend them to other groups looking to undertake similar project development work.
The benefits of the application acceleration award were demonstrated when our stage 1 application progressed through the funding committee discussions and a full-scale application was subsequently invited. We were able to continue the fruitful collaboration and reassemble the group to respond to board feedback and create a cogent full application. The final outcome is awaited.
We strongly endorse the application acceleration awards scheme and feel that the NIHR should consider offering more of these excellent funding opportunities ahead of major commissioned calls in the future.
Not applicable
All data underlying the results are available as part of the article and no additional source data are required.
The ROSSINI-Platform Investigators Collaborating Group consists of the following: Professor Thomas Pinkney* (Lead Applicant); Dr Babak Choodari-Oskooei*, Miss Elizabeth Li*, Miss Ellie Edlmann*, Dr Kelly Handley*, Dr Laura Magill*, Mr Ricky Vaja*, Miss Kayley King*, Miss Sue Blackwell*, Miss Veena Surendrakumar*, Mr David Bosanquet*, Mr James Glasbey*, Mr Mark Monahan*, Mr Matthew Popplewell*, Mr Michael Bahrami-Hessari*, Mr Michael Wall*, Mrs Manjinder Kaur*, Professor Aneel Bhangu*, Professor Babu Naidu*, Professor Dion Morton*, Professor Gavin Murphy*, Professor Ian Chetter*, Professor Mahesh Parmar*, Professor Michael D Jenkinson*, Professor Peter Hutchinson*, Professor Shelley Potter*, Professor Stuart McIntosh*, Professor Tracy Roberts*, Mr Matthew Lee, Dr Birgit Whitman, Dr Leona Morton, Miss Charlotte Sabine, Mrs Kady Hutton, Mrs Susie Harrison, Mrs Helen Duffy, Mr Luke Rogers, Mr Bryar Kadir, Mr Brenig Gwylim, Ms Katherine Fairhurst, Mrs Judith Long, Dr Raymond Oppong, Ms Melissa Rochon, Professor Judith Tanner.
*denotes applicant/co-applicant on the application acceleration grant award
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
References
1. The ROSSINI-Platform Investigators Collaborating Group*: ROSSINI-Platform - Reduction of surgical site infection with a platform trial utilising a 'Basket-MAMS' design - Application acceleration award. NIHR Open Research. 2024; 4. Publisher Full TextCompeting Interests: No competing interests were disclosed.
Reviewer Expertise: Cancer
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | |
---|---|
1 | |
Version 1 12 Dec 24 |
read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with NIHR Open Research
Already registered? Sign in
If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research.
We'll keep you updated on any major new updates to NIHR Open Research
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)