Acceptability, fidelity and trial experience of four intervention components to support medication adherence in women with breast cancer: A process evaluation protocol for a pilot fractional factorial trial

Background The Refining and Optimising a behavioural intervention to Support Endocrine Therapy Adherence (ROSETA) programme has developed four intervention components aiming to improve medication adherence in women with early-stage breast cancer. These are (a) text messages, (b) information leaflet, (c) Acceptance and Commitment Therapy-based guided self-help (ACT), (d) side-effect management website. Guided by the Multiphase Optimisation Strategy, our pilot trial will use a fractional factorial design to evaluate the feasibility of undertaking a larger optimisation trial. The pilot will include a process evaluation to maximise learning regarding the fidelity and acceptability of the intervention components before proceeding with a larger trial. The trial process evaluation has three aims: to assess the (1) fidelity and (2) acceptability of the intervention components; and (3) to understand participant’s trial experience, and barriers and facilitators to recruitment and retention. Methods The process evaluation will use multiple methods. Fidelity of the intervention components will be assessed using self-reported questionnaire data, trial data on intervention component adherence, and observations of the ACT sessions. Acceptability of the intervention components and trial experience will be explored using an acceptability questionnaire and interviews with patients and trial therapists. Trial experience will be assessed using a questionnaire and interviews with participants, while barriers and facilitators to recruitment and retention will be assessed using a questionnaire completed by research nurses and participant interviews. The pilot trial opened for recruitment on 20th May 2022 and was open at the time of submission. Conclusions This process evaluation will provide information regarding whether the intervention components can be delivered with fidelity within a national healthcare setting and are acceptable to participants. We will also better understand participant experience in a pilot trial with a fractional factorial design, and any barriers and facilitators to recruitment and retention. Registration ISRCTN registry ( ISRCTN10487576, 16/12/2021).

undertaking a larger optimisation trial.The pilot will include a process evaluation to maximise learning regarding the fidelity and acceptability of the intervention components before proceeding with a larger trial.The trial process evaluation has three aims: to assess the (1) fidelity and (2) acceptability of the intervention components; and (3) to understand participant's trial experience, and barriers and facilitators to recruitment and retention.

Methods
The process evaluation will use multiple methods.Fidelity of the intervention components will be assessed using self-reported questionnaire data, trial data on intervention component adherence, and observations of the ACT sessions.Acceptability of the intervention components and trial experience will be explored using an acceptability questionnaire and interviews with patients and trial therapists.Trial experience will be assessed using a questionnaire and interviews with participants, while barriers and facilitators to recruitment and retention will be assessed using a questionnaire completed by research nurses and participant interviews.The pilot trial opened for recruitment on 20th May 2022 and was open at the time of submission.

Conclusions
This process evaluation will provide information regarding whether the intervention components can be delivered with fidelity within a national healthcare setting and are acceptable to participants.We will also better understand participant experience in a pilot trial with a fractional factorial design, and any barriers and facilitators to recruitment and retention.

Plain English summary
Background: The majority of women with early-stage breast cancer are recommended adjuvant endocrine therapy (AET) to reduce the chances of their cancer coming back.Many women given this medication don't take it every day or stop taking it earlier than they should.We have developed four different interventions to help women take AET.These are; text messages reminding women to take AET; an information leaflet explaining how AET works and its benefits and side-effects; a therapy programme to reduce distress, consisting of five support sessions and four module booklets; and a website with strategies to manage AET side-effects.We are now testing whether these interventions can be delivered within the NHS in different combinations, in a small trial.

Amendments from Version 1
We have revised the manuscript in line with the reviewer's comments.The main changes to this new version include: -Additional detail in the introduction about existing interventions to support medication adherence in this population, and the limitations of these.
-Additional detail in the introduction and discussion regarding the Multiphase Optimisation Strategy, where this fits in intervention development and evaluation, and what our future plans for this work are.
-Inclusion of a figure of our conceptual model, detailing hypothesised mechanisms of actions for the intervention components.
-Additional detail about the theoretical rationale for our intervention components, along with clearer signposting to our published intervention development paper.
-Further detail about the predefined progression criteria we will use to decide whether to proceed to a full optimisation trial.
-Description of the theoretical domains framework earlier on in the manuscript.
-Additional paragraph in the data analysis section detailing how we will combine the findings from the process evaluation.
-Clarification of the wording of the study aims and methods in the plain English summary, with clarification as to how these correspond with the aims in the main paper.
-Additional description of Table 2 and Table 3 in the main text.

Plain english summary Background
The majority of women with early-stage breast cancer are recommended adjuvant endocrine therapy (AET) to reduce the chances of their cancer coming back.Many women given this medication don't take it every day or stop taking it earlier than they should.We have developed four different interventions to help women take AET.These are; text messages reminding women to take AET; an information leaflet explaining how AET works and its benefits and side-effects; a therapy programme to reduce distress, consisting of five support sessions and four module booklets; and a website with strategies to manage AET side-effects.We are now testing whether these interventions can be delivered within the NHS in different combinations, in a small trial.

Study methods
We have three aims: 1.To find out if the interventions can be given and are received in the way they were supposed to (fidelity).

Introduction
Breast cancer is the most common cancer in the UK 1 .Adjuvant endocrine therapy (AET) is prescribed to women with oestrogen receptor-positive (ER+) breast cancer for 5-10 years to reduce recurrence and mortality 2,3 .However, up to three quarters of women do not adhere to AET, either taking medication inconsistently or stopping prematurely [4][5][6] .Suboptimal adherence can lead to increased risk of recurrence and mortality, reduced health related quality of life, and reduced quality adjusted life years [7][8][9] .Multiple factors affect non-adherence behaviours, and these are often described as intentional (e.g., not believing the medication is necessary, and side-effects), and unintentional (e.g., forgetting) [10][11][12][13] .
There is a lack of evidence for effective interventions to support medication adherence to AET, with most interventions focusing solely on written educational components, and not targeting the range of barriers to adherence [14][15][16] .Moreover, these interventions are typically evaluated using parallel group randomised controlled trials (RCTs).RCTs are able to evaluate whether an intervention package is more effective than a comparator, but they are unable to estimate the contributions of individual components, or the interactions between components.As such, intervention packages demonstrating a statistically significant effect in an RCT could contain ineffective or redundant components, reducing the efficiency of the overall intervention package 17,18 .

The ROSETA programme
The Refining and Optimising a behavioural intervention to Support Endocrine Therapy Adherence (ROSETA) programme aims to develop and optimise an intervention package to support adherence to AET in women with early-stage breast cancer.The ROSETA programme is guided by the Multiphase Optimisation Strategy (MOST), which is a framework used to optimise multicomponent behavioural interventions 17,19,20 .MOST consists of three stages; (1) preparation, in which intervention components and a conceptual model detailing proposed mechanisms of action are developed, and any pilot testing is carried out, (2) optimisation, in which highly efficient experimental designs, such as factorial designs, are used to estimate the main effects and interaction effects of individual intervention components to build an optimal intervention package, and (3) evaluation, in which the optimised intervention package is compared with a comparator, typically using a parallel groups RCT 17,19,20 .
During the preparation phase of MOST, we combined the intervention mapping framework with MOST to develop four theory-based intervention components targeting distinct, unintentional and intentional barriers to AET adherence: (a) SMS messages to target forgetfulness; (b) information leaflet to target medication beliefs; (c) Acceptance and Commitment Therapy (ACT) guided self-help programme to increase psychological flexibility and reduce psychological distress; (d) side-effect self-management website to target AET side effects 21 (Table 1).As part of this preparatory work, we developed a conceptual model for the intervention, detailing the mechanisms of action (Figure 1).A full description of our approach to intervention development, along with detailed descriptions of the intervention components is available elsewhere 21 .
Experimental versions of the intervention components are currently being assessed for feasibility in an external, multisite exploratory pilot trial, for which a detailed protocol has been published 22 .This pilot trial is using a highly efficient 2 4-1 fractional factorial design.Participants are randomised to one of eight experimental conditions, in which they may receive usual care plus a combination of intervention components.Each intervention component has two levels; on or off.Across five UK sites, the pilot trial will recruit approximately 80 women over 18 with stage I-IIIa breast cancer who have been prescribed AET and have completed curative hospital-based treatment.Follow up will be at 2 and 4 months post randomisation.The primary objectives of the pilot trial are to establish eligibility, recruitment, retention and follow up rates, intervention component adherence and availability and feasibility of outcome and process data (ISRCTN: 10487576).

ROSETA pilot trial process evaluation
Process evaluations of complex interventions can maximise learning from trials, through investigating how an intervention was implemented, clarifying causal mechanisms and exploring contextual factors 23 .As highlighted in the UK Medical Research Council's (MRC) guidance, process evaluations can have differing functions dependent on the trial stage 23 .In the context of a pilot trial, understanding the feasibility, fidelity and acceptability of an intervention is often the focus, falling within the implementation and context themes in MRC guidance 23 .
The fractional factorial design in the ROSETA pilot trial is uncommon in healthcare research 24 .When used with effect coding (-1, +1), factorial designs can efficiently estimate the main effects and interaction effects of multiple intervention components 25 .However, this design requires multiple experimental conditions, thereby reducing the number of participants randomised to each experimental condition (the condition corresponds to a unique combination of the levels of intervention components).While adequate recruitment and retention of participants is challenging in all healthcare trials 26,27 , it is particularly important when using factorial designs, as empty cells may threaten the integrity of the trial 17 , or increase complexity in analysis and interpretation.This trial design brings further uncertainties, such as whether multiple intervention components Informed by evidence of side-effect selfmanagement strategies can be delivered with fidelity, participants are not unduly burdened with multiple intervention components, and whether the trial design is acceptable to, and understood by participants.Understanding participant trial experience and exploring barriers and facilitators to recruiting and retaining participants within a factorial design will provide important insights prior to proceeding with a larger optimisation trial.The Theoretical Domains Framework is a helpful tool to understand barriers and facilitators to recruitment and retention.The TDF synthesises 33 theories of behaviour change into factors that could influence behaviour, for example knowledge or available resources 28 .
Intervention fidelity refers to whether an intervention component was implemented as originally intended 29 .Fidelity assessments commonly focus solely on fidelity of delivery of an intervention, with less regard for the fidelity of training, receipt or engagement 30,31 .The US National Institute of Health Behaviour Change Consortium (NIHBCC) framework suggests fidelity as a multi-faceted construct and recommends five domains of fidelity that should be assessed; design, training, delivery, receipt and enactment 32 .Assessing fidelity at multiple levels can guide targeting of efforts to improve fidelity and hence improve the internal and external validity of a larger trial 31,33,34 .
Assessment of acceptability is also recommended in the pilot phase 23,35 .However, acceptability is frequently assessed as one singular construct, and the definition varies considerably 36 .The Theoretical Framework of Acceptability (TFA) seeks to address this limitation by conceptualising acceptability as a multi-faceted construct consisting of seven components; affective attitude, burden, ethicality, intervention coherence, opportunity costs, perceived effectiveness and self-efficacy 36 .While preliminary work has confirmed the prospective acceptability of the ROSETA intervention components, the pilot trial offers the chance to evaluate acceptability of the intervention components as experienced by intervention recipients, with the potential to improve acceptability prior to a larger optimisation trial.

Aims and objectives
This process evaluation has three aims: to assess (1) the fidelity of the intervention components; (2) the acceptability of the intervention components; (3) participant's experience, and barriers and facilitators to recruitment and retention in a fractional factorial trial.These aims address the implementation and contextual factors themes outlined in the MRC framework 23 .
The specific objectives for each aim are:

Design
We will use quantitative and qualitative methods to address the three main aims, nested within the ROSETA pilot trial.The process evaluation will involve assessments with three participant groups; participants of the ROSETA pilot trial, therapists delivering the ACT intervention component, and research nurses (RNs) involved in recruitment.
Fidelity.Fidelity will be assessed in relation to the intended design and delivery, across five domains using a mix of methods.Table 2 describes the data collection methods for each objective and intervention component, along with the time point at which the data will be collected.Fidelity of design.We will assess whether the intervention components show fidelity to the intended design, in terms of BCTs.BCTs are considered 'active ingredients' of behaviour change interventions (e.g.problem solving, action planning).They can be coded using the behaviour change taxonomy version 1 (BCTTv1), which contains 93 BCTs 37,38 .The research team (SG, SS, LH and CG) have coded which BCTs are present for each intervention component 21 .Two independent coders, external to the research team, will also code the intervention components.Coders will be provided with all intervention component materials and a coding manual and will be asked to identify which BCTs from the taxonomy are present in each intervention component.This process will be conducted once the trial has begun recruitment.The coders will have experience in using BCTs and will have completed BCTTv1 training.They will only code whether a BCT is present or absent, not the frequency of occurrence.Coding from the independent coders will be compared to the original coding 21 , and an agreement coefficient will be calculated 39 .All discrepancies will be discussed between one member of the research team (SG) and the two independent coders, and a final code list will be produced.
Fidelity of training (competency of delivery).The only intervention component to be delivered by healthcare professionals that requires training is the ACT component.Therapists will receive two half days of bespoke training from a clinical psychologist with expertise in ACT (CDG).To establish the adequacy of the training for delivery of the ACT component, a clinical psychologist (CDG) will assess the therapists recording of their first ACT session using the ACT Fidelity Measure (ACT-FM), which assesses therapist fidelity to ACT principles 40 .The number of booster training sessions the therapists require will be monitored.Semi-structured interviews with the therapists will explore barriers to adequate training and scope for any improvements.Fidelity of delivery.SMS receipt data will indicate whether a message has been successfully sent by the online system.The number of participants who opt-out of the SMS messages will be recorded.The number of information leaflets and website login details successfully sent to participants will be recorded by the site.For the ACT component, therapist scores on a procedural fidelity checklist will indicate which parts of the component were delivered (e.g., reflection on the utility, relevance and barriers to home practice tasks and introducing the following module).An expert ACT practitioner independent to the trial team will review 10% of all ACT session recordings using the ACT-FM, which assesses therapist fidelity to ACT principles.Reviewed recordings will include those conducted early and late in the trial and from all therapists involved.Fidelity of receipt.Participants will be asked a single self-reported item to assess which intervention components they received.Self-reported engagement with the intervention component(s) will be collected, along with reasons for nonengagement.For the website, tracking data will be collected, including number of logins, time spent on website, pages visited and videos watched.Each participant will only be asked about the components they were randomised to receive.Semi-structured interviews will investigate fidelity of receipt and elicit any barriers or enablers to the receipt of all intervention components.ACT therapists will report the number of sessions attended, cancelled or missed by each participant, and the participant's engagement with module materials (participant manual, associated audio files and home practice tasks).Participant's engagement with the module materials will also be assessed by an external reviewer assessing 10% of the ACT session recordings.Fidelity of enactment.Semi-structured interviews will be used to assess the extent to which the participant uses the intervention component(s) in their day-to-day life, and any barriers or facilitators to this.

Intervention component acceptability.
The assessment of acceptability of the intervention components will be guided by the TFA 36 (Table 3).As described in Table 3, acceptability will be assessed using an acceptability questionnaire specific to each intervention component at the 4 month follow up 41 , and semi-structured interviews with trial participants and therapists at the end of the trial.

Trial experience, recruitment and retention.
A mix of methods will involve trial participants and RNs (Table 4).Trial experience will be assessed in participants using a questionnaire assessing experience before, during and after the trial, a single item assessing overall trial acceptability, and during semi-structured interview with questions guided by the TFA 36 .The assessment of the barriers and facilitators to recruitment and retention to the factorial trial will be guided by the TDF 28 .Assessment will include one questionnaire for RNs exploring barriers and facilitators to recruitment, and interviews with participants with questions focusing on any barriers to participation and retention in the trial.

Sampling and recruitment
Trial participants.Women over 18 years old taking AET (tamoxifen, raloxifene, anastrozole, letrozole or exemestane) for early stage (I to IIIa) breast cancer, who have completed their last hospital treatment within the previous 12 months, are eligible to be part of the ROSETA pilot trial (Table 5).All trial participants will be asked to complete the quantitative assessments of the process evaluation at baseline, 2 and 4-month follow-up alongside the pilot trial assessments.
Participation in the end of trial interview is optional.When providing initial consent at baseline, participants will be asked if they are willing to be contacted for an interview at the end of the trial.If willing, participants will be contacted with further information approximately three-months post-randomisation.
Consent may be written or via telephone.We aim to interview at least one participant from each experimental condition (corresponding to unique combinations of intervention components) in which they receive at least one component.Maximum variation purposive sampling will aim to achieve a mix of participants above and below 50 years old.Participants will be interviewed until we have gained sufficient information power for each study aim (fidelity, acceptability and trial experience) 42 .Information power will be discussed in regular team meetings to inform further sampling.The specific aims of this study, high specificity of the sample and use of established theoretical frameworks to inform the interview indicates a smaller sample size may be appropriate.The interviewer (SG) has had experience interviewing women with breast cancer taking AET for similar purposes 43 and therefore a good .
quality of dialogue is expected 42 .However, this will require ongoing evaluation as this is dependent on the interviewer and participants.The analytic strategy will explore themes across cases, rather than individual in depth analysis, which is likely to require an increase in sample size.Thus, a small to moderate sample size is expected to be required 42 .

Intervention deliverers (ACT therapists).
Therapists delivering the ACT component will be Health and Care Professions Council (HCPC) or United Kingdom Council for Psychotherapy (UKCP) registered band 7a or above practitioner psychologists, or psychotherapists.Once the site has opened to recruitment, the therapist will be sent an invitation letter, information sheet and consent form.All consenting therapists delivering the ACT intervention component will be asked to complete the quantitative aspect (e.g., procedural fidelity checklists, fidelity assessments), and to participate in an end of trial interview.
One month before the end of the intervention delivery period, one therapist from each site will be randomly selected to be interviewed from those that have consented.Further therapists will be interviewed until the sample holds sufficient information power or there are no new consenting therapists left to interview.Information power will be discussed regularly in team meetings to inform further sampling 42 .Based on the specific study aims, high specificity of the sample, use of theoretical frameworks, expected good quality dialogue and cross-case analysis, a small to moderate sample size is expected 42 .
Research nurses.All RNs (or equivalent) involved in the recruitment of participants to the ROSETA pilot trial will be invited to participate in the process evaluation.Once the site has opened to recruitment, the RN will be sent an invitation letter, information sheet and consent form.If they consent, they will be asked to complete a questionnaire regarding the barriers and facilitators to recruitment at the end of the trial recruitment period.

Assessment measures Participant
Adherence to intervention components.To assess fidelity of receipt of the intervention components, participants will be asked two self-report items for each intervention component they were allocated to receive.The questions ask whether they received each intervention component, and how much of the intervention component they read.In relation to the ACT component, participants will be asked two additional items asking how much of the home practice tasks they completed, and how many of the audio files they listened to.Responses are on a 3-point scale; "none", "at least some" and "all of".
Acceptability Questionnaire (AQ) 41 .The acceptability questionnaire, based on the TFA, will assess the acceptability of the individual intervention components, modified for each component 36,41 .Three items have been removed (ethicality, self-efficacy and opportunity costs) to reduce participant burden for those allocated to multiple intervention components.Five items remain assessing general acceptability, affective attitude, burden, perceived effectiveness and intervention coherence.Participants will answer on a 5-point Likert Scale.
For participants randomised to the ACT component, 15 extra items ask specifically about the acceptability of elements of ACT sessions and format.For participants randomised to the SMS component one additional item asks about the acceptability of the frequency of the SMS messages.One open question asks for any further comments about the acceptability of intervention components.The 'general acceptability' item from the AQ will assess the acceptability of participants' experience in the trial as a whole.Participants will answer on a 5-point Likert Scale.
Study Participant Feedback Questionnaire (SPFQ) 44 .The SPFQ will assess participants' experience throughout the trial.This has been modified for the current trial.At baseline, participants will be asked two items about information received prior to the trial.Participants will answer on a 5-point Likert Scale.At the 2-month follow up, participants will be asked to complete three yes/no items regarding their experience during the trial.At the 4-month follow up participants will complete two items about their overall satisfaction with their experience in the trial, answering on a 5-point Likert Scale.

Therapist
ACT Fidelity Measure (ACT-FM) 40 .ACT-FM is a measure of therapist fidelity to ACT principles when delivering treatment (i.e., competency).The ACT-FM therapist stance subscale will be used to assess fidelity of training and delivery of the ACT component.Only the therapist stance subscale will be used, as this is the subscale most relevant to therapist delivery of the ACT component.Four items assess ACT-consistent behaviours, and three items assess ACT-inconsistent behaviours.Each item is scored on a 4-point scale (0-3).Scores can range from 0-9 on each subscale.A score of >4 on ACT consistent behaviours, and <5 on ACT inconsistent behaviours will be considered competent.
Procedural Fidelity Checklist.A procedural fidelity checklist for each individual session will assess fidelity of delivery of the ACT component.The checklist is designed specifically for the ACT component in ROSETA, for therapists to self-rate whether they undertook core intervention procedures, such as reflection on the skills exercises and home practice tasks.It includes eight items for session 1 and 2, seven items for session 3, six items for session 4, and four items for session 5.A percentage score is calculated for each session by dividing the score achieved by the maximum score achievable in that session and multiplying by 100.Additional items ask therapists to record to what extent a participant has engaged with the module materials (participant manual, home practice tasks and associated audio files).In each checklist, the therapists will also record the number of times the session has been cancelled.

Research Nurse
Barriers and facilitators to recruitment.RNs will complete a questionnaire to report their experiences of recruiting to a factorial trial.Questions are based on the TDF 28 and cover knowledge of the trial design, the influence of resources, social pressures and emotional factors on recruitment, their motivation levels in recruiting to the trial, their decision processes when contacting a patient to schedule a recruitment appointment, and any procedural changes that would facilitate recruitment.

Qualitative interviews
A rapid qualitative approach will be used to ensure findings from the pilot trial process evaluation can inform the future optimisation trial in a timely manner 45,46 .Semi-structured interviews with trial participants and therapists will last up to one hour, will be conducted via telephone or videoconferencing software (e.g., Microsoft Teams), and will be recorded.During the interview, the interviewer will take notes.Immediately following each interview, the interviewer will complete an individual Rapid Research Evaluation and Appraisal Lab Rapid Assessment Procedure (RREAL RAP) sheet for the participant.RREAL RAP sheets are a two-column table summarising the information collected in an interview.Different RREAL RAP sheets will be used for therapists and participants.
Trial participants.Semi-structured interviews will assess fidelity of receipt and enactment, acceptability of the intervention components, and barriers and facilitators to recruitment and retention in the trial.Questions will be based on the NIH fidelity domains, the TFA, and the TDF.Interview guides have been reviewed and amended based on suggestions from our patient and public involvement group (participant interview questions can be found as Extended data 47 ).
ACT therapists.Semi-structured interviews with therapists will assess fidelity of training and delivery and any barriers to these, relationship building with participants in the intervention component, and acceptability of the ACT component (therapist interview questions can be found as Extended data 47 ).We will also explore any suggested improvements to the intervention component and establish whether it may be implementable in routine care in the NHS.

Data analysis plan
Quantitative analysis Intervention fidelity.Descriptive statistics will summarise the quantitative intervention component fidelity assessments (Table 2).For the ACT component, these will also be presented by site.Specific to the fidelity of design assessment, the first-order agreement coefficient statistic (AC1) will estimate the interrater reliability between the independent coders and the BCTs that the components intended to target 21,39 .Strength of fidelity of design will be defined in terms of pre-established AC1 thresholds; <0.2=poor, 0.2 ≤ 0.4=fairly poor, >0.4 ≤0.6=moderate, >0.6 ≤0.8=good, >0.8 and ≤ 1=very good 48 .Discrepancies will be resolved through discussions between SG and the coders.
Intervention component acceptability.Descriptive statistics will be calculated for the individual items on the acceptability questionnaire, and overall, for each intervention component.Additional questions regarding the acceptability of specific aspects of the ACT and SMS components will be summarised.All summaries will be presented by component.Missing data will be summarised.
Barriers and facilitators to recruitment and retention of participants.Descriptive statistics will be presented for the RN barriers to recruitment and retention questionnaire, alongside qualitative data from the questionnaire.Summaries will be presented overall and by site.Missing data will be summarised.
Participant experience of trial participation.Descriptive statistics will be presented (1) for overall trial experience, and split by experience before starting the trial, during the trial and at the end of the trial; (2) for the one item of the AQ aimed at assessing general acceptability of the trial overall, by experimental condition and by site.Missing data will be summarised.
Qualitative analysis.Individual RREAL RAP sheets completed immediately after the interview will be collated into higher level RREAL RAP sheets; one per component for data relating to intervention component acceptability and fidelity, one overall for data relating to participant trial experience, and one overall for therapist interview data 45,46,49 .Regular team meetings will review emerging findings, changes to the interview schedule or RREAL RAP sheets, and data sampling.The interviews will be recorded and selectively transcribed verbatim.Specific quotes will be extracted from the transcripts and added to the RREAL RAP sheets to inform analysis.
For each overall aim (fidelity, intervention component acceptability, and trial experience), the relevant quantitative and qualitative data will be combined narratively to determine whether the intervention components could be delivered with fidelity overall, whether each intervention component was acceptable to women with breast cancer, and whether the experience in the trial was acceptable to women with breast cancer, or if there were barriers to recruitment and retention within the pilot trial.Together these findings will be used to make improvements to the intervention components and trial processes, should these be necessary.
evaluation aims to investigate whether four intervention components can be delivered with fidelity, whether the intervention components are acceptable to participants and trial therapists, and to explore participant's experiences in the trial along with barriers and facilitators to recruitment and retention.Given there is little guidance available for the conduct of process evaluations within factorial trial designs, our protocol may guide others seeking to conduct similar process evaluations.
The results of this process evaluation will be used to adapt and improve the intervention components where necessary, should progression criteria to a larger optimisation trial be met.
Our progression criteria to progress to a full optimisation trial are based on predefined criteria for consent rates, intervention component adherence, and availability of outcome data.These are described in detail elsewhere 22 .The full optimisation trial would aim to identify the most effective combination of intervention components to support medication adherence in women with early stage breast cancer, without exceeding a prespecified cost of £3397 per patient, based on health economic modelling 21 .
This process evaluation has the potential to highlight key issues in the specific intervention components in terms of fidelity and acceptability, and regarding the overall trial design and its implementation within an NHS setting.As factorial designs are relatively uncommon in behavioural trials, we hope that this process evaluation can offer useful insights for others.I still don't feel that the Plain English summary sets up the study as the authors are aiming to achieve more than "find out" stuff, and will be doing more than interviewing and asking.There's a balance between making papers accessible and dumbing down and I don't think the authors are striking it here.This is a matter of style however.

Thomas Mills
PHIRST South Bank, London South Bank University, London, England, UK Technically speaking, this is well thought out and looks to be an excellent process evaluation.One thing I'd like the authors to consider is *why* they are doing this.The initial set of aims needs developing: surely an aim here is to facilitate research-informed improvements to the intervention components ahead of the trial?I also feel there's a gap pertaining to this aim in the protocol whereby the researchers draw on the findings to inform intervention changes, as well as their use of analytical frameworks to guide this: the TDF framework is mentioned on occasions but only in passing.A final section/sub-section after, or at the end of the data analysis section to outline a proposed plan for pulling it all together would add the cherry onto the cake, so to speak.

Study methods
The aims need further development.I suggest having a set of aims and a set of objectives.Also, if the first set of aims is numbered, could the second set be letter-ed, so as to differentiate between them further?○ Aims 2 and 3 are similar -exploring women's acceptability and experiential outcomes.I wonder if there is another aim, which is alluded to elsewhere in the paper, which is to use the process findings to refine intervention components ahead of the full trial.

○
In the "To do this we will" section, 3. implies that intervention components are all readable, when therapy sessions are planned also.All of these points relate to data collection activities: I presume you'll be analysing the data too, to fulfil the aims?Key analytical strategies could be included here.

Roseta process evaluation intro
No mention of analytical strategy/data analysis -e.g., TDF framework

Protocol -Aims and objectives
How do these, new aims relate to the aims stated in the study methods?These different sets of aims need to be linked.

Design
Fidelity -another introduction to the five domains -it feels like you're going backward because the domains have been introduced already: focus this sentence on describing Table 2 in more detail.

Fidelity design
Example BCTs would help the reader -possibility of a logic model to show the causal assumptions? ○

Fidelity of enactment
Ditto -text could focus describing Table 3 in more detail.

Trial experience, retention and recruitment
First mention of TDF framework -this is important because it will help the researchers understand the data and highlight areas of improvement

Qualitative interviews
Why will the interviewer be taking notes during the interview?This risks obstructing the flow of the interview.The interview will be recorded.I suggest the interviewer write up the interview immediately after the interview, using the recording as a guide; then check the summary against the transcript, once the recording is transcribed.
○ Is the rationale for, and objectives of, the study clearly described?Partly

Are sufficient details of the methods provided to allow replication by others? Yes
Are the datasets clearly presented in a useable and accessible format?Yes Thank you for taking the time to review our manuscript and for your constructive comments.
The initial set of aims needs developing: surely an aim here is to facilitate researchinformed improvements to the intervention components ahead of the trial?I also feel there's a gap pertaining to this aim in the protocol whereby the researchers draw on the findings to inform intervention changes, as well as their use of analytical frameworks to guide this: the TDF framework is mentioned on occasions but only in passing.RESPONSE: In line with our decision to undertake an external pilot trial, as opposed to a feasibility trial, using the findings from the process evaluation to refine the intervention components ahead of the optimisation trial was not a main aim of the research.The aims were to determine the acceptability of the intervention components, determine if they could be delivered with fidelity and to assess women's experience within the trial.As such, this process evaluation contributes towards the broader goal of determining if a larger trial to optimise these components is feasible to undertake.We will make some minor amendments to the intervention components if it will improve their acceptability, fidelity or women's trial experience, but this was not the primary purpose of doing a pilot trial with an embedded process evaluation.
In terms of our used of analytical frameworks, we have now added an introduction to the TDF in the introduction.In the introduction we had already introduced the other two theoretical frameworks we are using; the theoretical framework of acceptability and the five domains of fidelity.In the 'design' section in the methods, we then explain which framework we are using for each aim of the process evaluation.The five domains of fidelity is being used to address the fidelity aim (aim 1), the theoretical framework of acceptability is being used to determine acceptability of the intervention components (aim 2), and the TDF is being used to assess participant trial experience and barriers and facilitators to recruitment and retention (aim 3).
A final section/sub-section after, or at the end of the data analysis section to outline a proposed plan for pulling it all together would add the cherry onto the cake, so to speak.RESPONSE: We have added an additional paragraph into the data analysis section as suggested to outline our proposed plan for pulling the quantitative and qualitative data together for each aim.
Methods: "For each overall aim (fidelity, intervention component acceptability, and trial experience), the relevant quantitative and qualitative data will be combined narratively to determine whether the intervention components could be delivered with fidelity overall, whether each intervention component was acceptable to women with breast cancer, and whether the experience in the trial was acceptable to women with breast cancer, or if there were barriers to recruitment and retention within the pilot trial.Together, these findings will be used to make improvements to the intervention components and trial processes, should these be necessary."

Study methods
The aims need further development.I suggest having a set of aims and a set of objectives.Also, if the first set of aims is numbered, could the second set be letter-ed, so as to differentiate between them further?
○ RESPONSE: This comment is in reference to the aims within the plain English summary/ abstract of the work, with a word limit of 300 words.We therefore request that we do not go into detail about the additional objectives within each aim (as we also have not done in the scientific abstract), as there is insufficient space to include these here.In the main body of the paper, we have included aims, and specific objectives for each aim, using numbering and lettering to differentiate between these.Aims 2 and 3 are similar -exploring women's acceptability and experiential outcomes.I wonder if there is another aim, which is alluded to elsewhere in the paper, which is to use the process findings to refine intervention components ahead of the full trial.
○ RESPONSE: Aim 2 refers to the acceptability specifically of each of the four intervention components, while aim 3 refers to women's experience within the trial in relation to the novel trial design we are using, and general trial processes, such as the length of questionnaires.We have made some amendments to the wording of these aims in the plain English summary to differentiate them further.Please see our earlier response with regard to including an additional aim.
"Plain English Summary Aims: 2. To find out if the support received as part of the trial was acceptable to women with breast cancer (acceptability).

To find out what women's experience was of taking part in the trial overall (trial experience)."
In the "To do this we will" section, 3. implies that intervention components are all readable, when therapy sessions are planned also.All of these points relate to data collection activities: I presume you'll be analysing the data too, to fulfil the aims?Key analytical strategies could be included here.
○ RESPONSE: As part of the therapy intervention component, there are module booklets, which are the readable element that would be assessed here; we have clarified this in the background section of the plain English summary where we describe the intervention components, and we have included this information in Table 1, where we provide a full description of the intervention components.
We have not included the analysis strategies here as this is the plain English summary with a small word limit, and we have also not included analytic strategies in the scientific abstract.We are using a range of different analysis approaches for the different aims and quantitative and qualitative data in this process evaluation, and therefore we feel that summarising these analytical methods within this plain English summary would add confusion more than it would provide clarity.We have described our planned analysis in detail in the main paper, under the heading "data analysis plan".

Roseta process evaluation intro
No mention of analytical strategy/data analysis -e.g., TDF framework ○ RESPONSE: We have now added in a description of the TDF into the introduction in the relevant section.The other frameworks we are using (theoretical framework of acceptability, and fidelity domains framework) are already referred to in the introduction.
We have not included specific details of data analysis as we feel this is more appropriate in the methods section.
Introduction: "The Theoretical Domains Framework is a helpful tool to understand barriers and facilitators to recruitment and retention.The TDF synthesises 33 theories of behaviour change into factors that could influence behaviour, for example knowledge or available resources (36)."

Protocol -Aims and objectives
How do these, new aims relate to the aims stated in the study methods?These different sets of aims need to be linked.
○ RESPONSE: These are not new aims; the aims in the 'study methods' the reviewer refers to, are the aims written in a plain English summary.In both sets of aims, (1) refers to investigating intervention fidelity, (2) refers to intervention acceptability, and (3) refers to trial experience, which are the 3 overall aims of this process evaluation.These have been listed in the same order each time to indicate that they are the same aims.In the plain English summary we have now added detail in brackets to make the link clearer to the aims in the main text.

Design
Fidelity -another introduction to the five domains -it feels like you're going backward because the domains have been introduced already: focus this sentence on describing Table 2 in more detail.
○ RESPONSE: We have revised this section you mention, focusing on describing Table 2 rather than referring to the five domains.Methods: "Fidelity will be assessed in relation to the intended design and delivery, across five domains using a mix of methods.Table 2 describes the data collection methods for each objective and intervention component, along with the time point at which the data will be collected."

Fidelity design Example BCTs would help the reader -possibility of a logic model to show the causal assumptions?
○ RESPONSE: We have now added in two example BCTs to the text, and have included our conceptual model as a figure within the paper (Figure 1), which details our proposed mechanisms of action for each intervention component.We have also referenced our intervention development paper in the introduction, which contains the full list of BCTs we have coded for each intervention component.Introduction: "A full description of our approach to intervention development, along with detailed descriptions of the intervention components is available elsewhere (14).As part of this preparatory work, we developed a conceptual model for the intervention, detailing the mechanisms of action (Figure 1)." Methods: "BCTs are considered 'active ingredients' of behaviour change interventions (e.g.problem solving, action planning)."

Fidelity of enactment
Ditto -text could focus describing Table 3 in more detail.
○ RESPONSE: Here, we assume the reviewer is referring to the "Intervention component acceptability" section which relates to Table 3 mentioned, rather than the fidelity of enactment section referred to above.We have kept the sentence referring to the TFA here, as it is the first time we explicitly state that we are using the TFA (we only introduce the framework in the introduction).However, we have added an additional sentence describing Table 3 as suggested by the reviewer.
Methods: "The assessment of acceptability of the intervention components will be guided by the TFA (30)(Table3).As described in Table 3, acceptability will be assessed using an acceptability questionnaire specific to each intervention component at the 4 month follow up (35), and semi-structured interviews with trial participants and therapists at the end of the trial."

Trial experience, retention and recruitment
First mention of TDF framework -this is important because it will help the researchers understand the data and highlight areas of improvement ○ RESPONSE: We have now introduced the TDF earlier in the manuscript, in the introduction, in line with the other theoretical frameworks we are using in this work (theoretical framework of acceptability, and five domains of fidelity).
Introduction: "The Theoretical Domains Framework is a helpful tool to understand barriers and facilitators to recruitment and retention.The TDF synthesises 33 theories of behaviour change into factors that could influence behaviour, for example knowledge or available resources (36)."

Qualitative interviews
Why will the interviewer be taking notes during the interview?This risks obstructing the flow of the interview.The interview will be recorded.I suggest the interviewer write up the interview immediately after the interview, using the recording as a guide; then check the summary against the transcript, once the recording is transcribed.
○ RESPONSE: The interviewer will be taking notes during the interviews as this is part of the recommended process in rapid approaches.We have previously used this method and did not find it disrupted the interviews.The note taking will not be extensive and we will make every effort to ensure it does not disrupt the flow of the interview.Using the recording as a guide to write up the interview would take a substantial period of time to re listen to the interview, and would not fit with the rapid analysis approach that we are taking.
Competing Interests: No competing interests were disclosed.Introduction: The team are employing a novel trial design, using MOST, or a pilot trial with a fractional factorial design.This may be new to many readers.It would be interesting and helpful to readers if the authors could include some comment of where a MOST trial sits in the intervention development and evaluation process.What would the next steps of this study be?An full scale optimisation trial?What would the aims of this trial be?Where is this work going?Some discussion of these important questions would really strengthen this paper and help move along discussions about the place of novel trial designs like these in the development/evaluation/implementation pipeline.

2.
One of the key steps in an optimisation trial is developing a conceptual model of the intervention, detailing the components and proposed mechanisms of action.Did the team develop a conceptual model for this intervention?It would be helpful to readers to see that.

3.
Related to point 3., there is very little in the paper about the intervention components and their behaviour change techniques.Can the authors include any more detail about the intervention components?Is there any theoretical rationale for the intervention?This discussion is very limited in the introduction.

4.
The proposed timeframe for follow up (4 months) seems short.Is there a clinical rationale for selecting this timeframe?Would this be the proposed timeframe in a future optimisation trial?

5.
There is not mention of economic considerations or an economic evaluation of the interventions.Is this being included in the work?The intervention components appear very variable in terms of cost, which is an important consideration in the evaluation of the interventions.Are there plans to consider this aspect in furfure research?6.
Discussion: Can the authors clarify the 'progression criteria', which referred to in the 7.
interventions are typically evaluated using parallel groups randomised controlled trials (RCTs).RCTs are able to evaluate whether an intervention package is more effective than a comparator, but they are unable to estimate the contributions of individual components, or the interactions between components.As such, intervention packages demonstrating a statistically significant effect in an RCT could contain ineffective or redundant components, reducing the efficiency of the overall intervention package (16, 49).

Introduction:
The team are employing a novel trial design, using MOST, or a pilot trial with a fractional factorial design.This may be new to many readers.It would be interesting and helpful to readers if the authors could include some comment of where a MOST trial sits in the intervention development and evaluation process.What would the next steps of this study be?An full scale optimisation trial?What would the aims of this trial be?Where is this work going?Some discussion of these important questions would really strengthen this paper and help move along discussions about the place of novel trial designs like these in the development/evaluation/implementation pipeline.RESPONSE: We have added in additional explanation of the MOST framework, and the three main stages, to demonstrate where optimisation sits within the development and evaluation process.We have also made it clearer that this work falls within the preparation phase of MOST.We have now added additional detail regarding the next stages of this work in the discussion section, as that is where we felt it was most relevant.This includes an explanation of our progression criteria to move to the larger optimisation trial, and what the aim of the optimisation trial would be.
Introduction: "The Refining and Optimising a behavioural intervention to Support Endocrine Therapy Adherence (ROSETA) programme aims to develop and optimise an intervention package to support adherence to AET in women with early-stage breast cancer.The ROSETA programme is guided by the Multiphase Optimisation Strategy (MOST), which is a framework used to optimise multicomponent behavioural interventions (15-17).MOST consists of three stages; (1) preparation, in which intervention components and a conceptual model detailing proposed mechanisms of action are developed, and any pilot testing is carried out, (2) optimisation, in which highly efficient experimental designs, such as factorial designs, are used to estimate the main effects and interaction effects of individual intervention components to build an optimal intervention package, and (3) evaluation, in which the optimised intervention package is compared with a comparator, typically using a parallel groups RCT (15-17).
During the preparation phase of MOST, we combined the intervention mapping framework with MOST to develop four theory-based intervention components targeting distinct, unintentional and intentional barriers to AET adherence: (a) SMS messages to target forgetfulness; (b) information leaflet to target medication beliefs; (c) Acceptance and Commitment Therapy (ACT) guided self-help programme to increase psychological flexibility and reduce psychological distress; (d) side-effect self-management website to target AET side effects (14)(Table1).As part of this preparatory work, we developed a conceptual model for the intervention, detailing the mechanisms of action (Figure 1).A full description of our approach to intervention development, along with detailed descriptions of the intervention components is available elsewhere (14)."Discussion: "The results of this process evaluation will be used to adapt and improve the intervention components where necessary, should progression criteria to a larger optimisation trial be met.Progression criteria to progress to a full optimisation trial are based on predefined criteria for consent rates, intervention component adherence, and availability of outcome data.These are described in detail elsewhere (45){Smith, 2023 #560}.The full optimisation trial would aim to identify the most effective combination of intervention components to support medication adherence in women with early stage breast cancer, without exceeding a pre-specified cost of £3397 per patient, based on health economic modelling (14)." One of the key steps in an optimisation trial is developing a conceptual model of the intervention, detailing the components and proposed mechanisms of action.Did the team develop a conceptual model for this intervention?It would be helpful to readers to see that.RESPONSE: We did develop a conceptual model for the intervention, which we have published elsewhere and have referenced in the current paper (Green et al., 2022, BMC Health Services Research).We have now included our conceptual model as a figure in the present paper (Figure 1), and have referred readers to this in the introduction.
Related to point 3., there is very little in the paper about the intervention components and their behaviour change techniques.Can the authors include any more detail about the intervention components?Is there any theoretical rationale for the intervention?This discussion is very limited in the introduction.RESPONSE: We have now made it clear that the intervention components were developed based on psychological theory in the text, and we have included details of this in Table 1, alongside a description of the intervention components.We undertook an extensive intervention development process, which has been published elsewhere, and we have now made this more explicit to the reader here.Included in this previous publication is a full list of BCTs that we coded in each of our intervention components.
Introduction: "During the preparation phase of MOST, we combined the intervention mapping framework with MOST to develop four theory-based intervention components targeting distinct, unintentional and intentional barriers to AET adherence: (a) SMS messages to target forgetfulness; (b) information leaflet to target medication beliefs; (c) Acceptance and Commitment Therapy (ACT) guided self-help programme to increase psychological flexibility and reduce psychological distress; (d) side-effect self-management website to target AET side effects (14)(Table1).As part of this preparatory work, we developed a conceptual model for the intervention, detailing the mechanisms of action (Figure 1).A full description of our approach to intervention development, along with detailed descriptions of the intervention components is available elsewhere (14)." The proposed timeframe for follow up (4 months) seems short.Is there a clinical rationale for selecting this timeframe?Would this be the proposed timeframe in a future optimisation trial?RESPONSE: A short follow up was chosen in this pilot trial for a number of reasons.Firstly, retention was not included as part of the progression criteria to a full optimisation trial, and therefore we did not need to have a long follow up to assess this.The progression criteria (based on consent rates, intervention component adherence and availability of outcome data) are all based on short term outcomes, so a longer term follow up was not required.Finally, the funding structure for this work means that intervention development, a pilot trial and a full optimisation trial with a 12 month follow up are taking place within 6 years; therefore a shorter follow up period in the pilot trial felt appropriate to allow for this body of work to be completed within the timeframe.
There is not mention of economic considerations or an economic evaluation of the interventions.Is this being included in the work?The intervention components appear very variable in terms of cost, which is an important consideration in the evaluation of the interventions.Are there plans to consider this aspect in furfure research?RESPONSE: We are planning an economic analysis in the larger optimisation trial, for which we will use self-reported data, alongside NHS reference costs and costs from the Personal Social Services Research Unit, who estimate unit costs for healthcare professionals and services in the NHS.The economic analysis will form part of the main trial paper, as it is not considered part of the process evaluation, and therefore we have not included it within this protocol.However, we have now included our optimisation objective for the full optimisation trial in the discussion, which highlights that we are considering cost in the optimisation of the intervention package.
Discussion: "The full optimisation trial would aim to identify the most effective combination of intervention components to support medication adherence in women with early stage breast cancer, without exceeding a pre-specified cost of £3397 per patient, based on health economic modelling (14)."Discussion: Can the authors clarify the 'progression criteria', which referred to in the Discussion section.RESPONSE: We have now added in a description of the progression criteria for this pilot trial, along with a reference to the trial protocol where progression criteria are described in greater detail.Discussion: "The results of this process evaluation will be used to adapt and improve the intervention components where necessary, should progression criteria to a larger optimisation trial be met.Progression criteria to progress to a full optimisation trial are based on predefined criteria for consent rates, intervention component adherence, and availability of outcome data.These are described in detail elsewhere (45){Smith, 2023 #560}." Competing Interests: No competing interests were disclosed.
To establish to what extent the intervention components target the behaviour change techniques they are intended to, and to what extent the components are distinct from one another.All BCTs coded as present in each intervention component, based on the BCTTv1 To evaluate the adequacy of therapist training for the delivery of the intervention component.To assess if each intervention component was delivered as planner.To evaluate if participants receive and understand the intervention components they were allocated to.To understand the degree to which participants implement the intervention components in their daily life.Identify barriers to the fidelity of training, delivery, receipt and enactment of the intervention components, including any influential contextual factors.Key: BCT= Behaviour Change Technique.BCTTv1 = Behaviour Change Technique Taxonomy version 1. ACT = Acceptance and Commitment Therapy.ACT-FM: Acceptance and Commitment Therapy Fidelity Measure.SMS = Short Message Service.CRF = Case Report Form.

Table 1 . Summary of intervention components in the ROSETA pilot trial.
A website containing strategies to self-manage common AET side effects including; arthralgia, fatigue, vulvovaginal symptoms, gastrointestinal symptoms, hot flushes, sleep difficulties.The website uses a rating system to summarise the strength of evidence for each strategy.

Figure 1. Conceptual model for the ROSETA intervention. This
figure has been reproduced with permission from Green et al., (2023) 21 .

Table 5 . Eligibility criteria for participation in the ROSETA pilot trial. Inclusion Criteria Exclusion Criteria
Need for treatment for a severe mental health disorder or crisis, which is likely to interfere with participation (e.g., active psychosis, bipolar disorder, significant issues with addiction or self-harm or expressing active suicidal ideation with active plans and intent*) Note, if concerned about the possible presence of risk of suicidal ideation with active plans and intent, then this can be assessed with the following questions, with patients ineligible if they answer 'yes' to 5c.Items marked with * are collected via self-report.Key:ROSETA= Refining and Optimising a behavioural intervention to Support Endocrine Therapy Adherence.SMS= Short Message Service.ACT= Acceptance and commitment therapy.This table has been reproduced with permission from Smith et al., (2023) 22 1.An informed consent form (signed and dated) 2. Capacity to provide informed consent 3. Women with early stage (1 to 3a) breast cancer according to the Tumour, Node, Metastasis (TNM) / American Joint Committee on Cancer (AJCC) staging system.Note.Women being treated for a second primary breast cancer or a breast cancer local recurrence are eligible for the study, providing the most recent cancer is being treated with adjuvant endocrine therapy, and they meet all eligibility criteria.Women with bilateral breast cancer are permitted, providing at least one breast is affected by hormone receptor-positive disease.4. Aged ≥18 years old at time of screening for ROSETA's pilot study 5. Have sufficient proficiency in English to be able to adhere to all intervention components and data collection required 6. Treated with curative intent 7. Completed their hospital-based treatment (e.g., surgery, radiotherapy and/or chemotherapy) for the current breast cancer within the last 12 months.Note.Women are still eligible for the study if they are being treated with monoclonal antibody-based therapy such as trastuzumab, Kadcyla, pertuzumab, and Phesgo.8. Currently prescribed oral adjuvant Hormone Therapy (tamoxifen, raloxifene, anastrozole, letrozole, exemestane) 9.The participant is willing to complete the study questionnaires* 10.The participant is willing to be audio recorded during the therapy sessions* 11.The participant is willing and able to attend all ACT sessions either via video conference or telephone* 12.The participant is willing and able to complete home practice tasks* 13.Access to a mobile phone to receive SMS messages* 14.Willing to receive frequent SMS messages* 15.Access to a computer or smart device that can access the internet* This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland This is an excellent paper which I enjoyed reading.The team are commended for this ambitious and important programme of research.They are addressing a clinically important question.The paper is well-written and describes a novel trial design which has the potential to address important questions about active ingredients of complex interventions.I have a few suggestions which I believe would improve the proposal:Introduction: There is a lack of discussion and information about what we know already in relation to improving medication adherence.The research is presented in a bit of a vacuum, and would be strengthened by including a little more detail summarising what is already known, on which this research is building.