Keywords
factorial, intervention optimisation, multiphase optimisation strategy, breast cancer, information leaflet
Adherence to adjuvant endocrine therapy (AET) is low in women with breast cancer, which increases the risk of recurrence and mortality. A consistently reported barrier to adherence is low perceived necessity of AET and high concerns. Existing interventions to support medication beliefs have mixed effectiveness and rarely target medication beliefs specifically. We developed an information leaflet with five candidate components aiming to increase necessity beliefs about AET and reduce concerns; (1) diagrams explaining how AET works; (2) icon arrays displaying the benefits of AET; (3) information about the prevalence of side-effects; (4) answers to common concerns and (5) quotes and pictures from breast cancer survivors. Guided by the multiphase optimisation strategy (MOST), we aimed to optimise the content of the information leaflet. We planned for the dataset to be open access to provide an exemplar for other investigators to use.
The content of the leaflet was optimised in a fully powered online 25 factorial experiment. Each candidate component of the leaflet was operationalised as a factor with two levels; on vs off or enhanced vs basic. Healthy women (n=1604) completed the beliefs about medicines questionnaire and were randomised to view one of 32 versions of the information leaflet. The 32 versions comprised unique combinations of the factor levels corresponding to the five candidate intervention components. Time spent on the information leaflet page of the survey was recorded. After viewing the information leaflet, participants completed the beliefs about medicines questionnaire again, a true/false questionnaire assessing their objective knowledge of AET, a subjective rating of their knowledge of AET, and a questionnaire evaluating their satisfaction with the information they received.
The factorial dataset provides the opportunity for other investigators interested in using the MOST framework to learn about complex factorial designs, using a real dataset.
Most women with breast cancer are treated with adjuvant endocrine therapy (AET) to reduce the chance of breast cancer coming back. However, many women do not take the medication as recommended. Women’s beliefs about the medication are a common reason for not taking AET. Some women do not think AET will help them, and some women have lots of concerns about AET. At the moment, we do not know the best way to change women’s beliefs about AET. Therefore, we ran a study to help us understand what combination of information might help change women’s beliefs about AET.
We developed a written information leaflet with five parts; (1) diagrams about how AET works; (2) visual figures of the benefits of AET; (3) information about how likely each side-effect is; (4) answers to common concerns about AET; and (5) pictures and quotes from women who have taken AET. In an online survey, 1,604 healthy women answered questions about their beliefs about the medication. Each woman was shown one version of the information leaflet picked at random. There were 32 possible versions of the information leaflet, which contained unique combinations of the five parts of the leaflet. After women read the leaflet, they were asked to complete the same questionnaire about their beliefs about the medication. They were also asked questions about how satisfied they were with the information they received, true or false questions about AET to assess their knowledge after reading the leaflet, and a rating of how informed they felt about AET. We also recorded how long women spent looking at the leaflet. One of our aims was to make the dataset from this experiment openly available so other scientists could use it to learn how to conduct similar experiments.
factorial, intervention optimisation, multiphase optimisation strategy, breast cancer, information leaflet
Adjuvant endocrine therapy (e.g., tamoxifen, anastrozole, letrozole, exemestane) reduces breast cancer recurrence and mortality in women with early-stage (I-III) breast cancer1,2. However, up to three-quarters of women do not take AET as prescribed3–5. A consistently cited barrier to AET adherence is medication beliefs. Low perceived necessity of AET and high concerns surrounding AET (e.g., concerns of potential side-effects) have been associated with lower adherence6–10.
Educational interventions are commonly used to address medication beliefs11. However, evidence is mixed regarding the effectiveness of such interventions in changing medication beliefs11,12. Moreover, medication beliefs are often targeted within a larger complex intervention aiming to support AET adherence. Therefore, the effectiveness of components specifically targeting medication beliefs is unclear11,12.
To increase necessity beliefs and reduce concerns in women prescribed AET, we developed a theory-informed multicomponent educational information leaflet intervention13. The information leaflet had five candidate components; (1) diagrams explaining how AET works; (2) icon arrays stating the benefits of AET; (3) side-effect prevalence information; (4) answers to common concerns; and (5) quotes and pictures from other breast cancer survivors13.
Typically, an information leaflet intervention may be evaluated using a parallel group randomised controlled trial (RCT), comparing whether the leaflet as a whole is more effective than a suitable comparator, such as usual care. However, in a parallel group RCT, effects of the individual components of the leaflet, and interactions between intervention components cannot be estimated14. It is possible some intervention components could be redundant, or could have a negative effect on beliefs about AET, meaning the effectiveness and efficiency of the leaflet may be compromised14.
The multiphase optimisation strategy (MOST) is an engineering-inspired framework aiming to optimise complex interventions to balance effectiveness with efficiency14. MOST proposes an optimisation phase prior to definitive evaluation of complex interventions. In the optimisation phase, highly efficient and fully powered experimental designs, such as factorial designs, can be used to estimate the individual and combined effects of intervention components14. Empirical data from an optimisation trial can be used to select an optimised intervention15; for example, selecting an intervention comprising only intervention components estimated to contribute to a positive effect on beliefs about AET. The MOST framework therefore offers the potential to balance intervention effectiveness with efficiency.
As the MOST framework is a novel approach to optimising and evaluating complex interventions, there are relatively few open access examples of data of optimisation trials. Open access datasets of factorial experiments would be useful to provide scientists learning about the framework with some real data to practice analysing and interpreting the data. Our aim was to create a dataset of our factorial experiment that is available for other investigators to use. The original aim of the factorial experiment leading to creation of this dataset was to optimise the content of an information leaflet intervention targeting beliefs about AET13.
A panel of four women prescribed AET for breast cancer were involved in this project. The panel were recruited in 2021 via a local charity supporting people affected by cancer. Through regular (approximately quarterly) meetings with the investigator(s), the panel had input into the design of the study (i.e., targeting medication beliefs), the targets of the intervention components and the content and design of the intervention components making up the information leaflet. They additionally provided quotes of their motivations for taking AET and advised on wording of the scenario presented at the beginning of the survey to provide context around being prescribed AET13,16. The panel did not contribute to the conduct or recruitment of the study. After further evaluation of the information leaflet in a larger trial17, the panel will be consulted on methods of dissemination of study results.
Ethical approval was granted from the University of Leeds School of Medicine ethical review board (MREC 21-033, 21/03/2022). Written informed consent was obtained electronically from all participants. All procedures, including the informed consent process, were conducted in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000.
Full methods and a detailed description of the development of the candidate intervention components are described elsewhere13,18.
The information leaflet was made up of five candidate intervention components, each operationalised as factors with two levels: (1) diagrams explaining how AET works (levels: on/off); (2) icon arrays visually displaying the benefits of taking AET in terms of reduced recurrence and mortality (levels: enhanced/basic); (3) information about potential side-effects from AET and their prevalence (levels: enhanced/basic); (4) answers to common concerns women have about taking AET (levels: on/off); (5) quotes and pictures from breast cancer survivors, stating their motivations for taking AET (levels: on/off).
Participants were required to be female, over 18 and able to read English. A market research company recruited participants via sending the survey link to potential respondents in the UK. A total of 1,604 women completed the survey. One participant was excluded due to being under 18.
We used a 25 (2x2x2x2x2) factorial design (Table 1). After completing basic demographic information, participants were shown a scenario asking them to imagine they had been diagnosed with breast cancer and prescribed AET. This scenario is available elsewhere13. Participants could not proceed to the next page of the survey until 30 seconds had passed.
Note. Each component had two levels: on vs off, or enhanced vs basic.
This table was taken directly from Green et al.,13.
Participants completed the beliefs about medicines questionnaire. They were then randomised to one of 32 experimental conditions. Each condition corresponded to a unique version of the information leaflet made up of different combinations of the factor levels corresponding to the five intervention components. All possible combinations of factor levels made up the 32 experimental conditions. Participants could not proceed with the survey until three minutes had passed. After viewing the leaflet, participants were asked the same questions about their beliefs about AET, in addition to true or false questions about their knowledge of AET, their subjective knowledge of AET and their satisfaction with information they had received about AET. The questionnaires are available online (DOI 10.17605/OSF.IO/3CZ9Q)19.
Participant demographics. Age, marital status, education level, ethnicity, menopausal status, previous breast cancer diagnoses and any close relations diagnosed with breast cancer were collected at the beginning of the survey. For women diagnosed with breast cancer, further questions about the stage and whether they had been prescribed AET were asked.
Beliefs about Medication Questionnaire-AET (BMQ-AET). The BMQ-AET is a 10-item scale used to assess specific medication beliefs20. It is made up of two subscales; necessity and concerns, each with five items. Concern scores were subtracted from necessity scores to create a BMQ differential score. A BMQ differential score was calculated for the BMQ completed before and after viewing the information leaflet.
Satisfaction with information about Medicines (SIMS). A modified version of the SIMS was used21. Seven of the original SIMS items were removed due to not being relevant. One item was added asking about the benefits of taking AET. Participants were asked to rate their satisfaction with 11 different aspects of information about AET on a 5-point scale; too much, about right, too little, none received or none needed. Responses of “about right” and “none needed” indicated satisfaction with information and scored 1, while all other responses indicated dissatisfaction and scored 0. A total score was calculated (0–11).
Objective knowledge. Eight true or false items assessed objective knowledge of AET, relating to the mechanisms, benefits and side-effects of AET. Items answered correctly were scored as 1, items answered incorrectly were scored as 0. A total knowledge score was calculated.
Subjective knowledge. One item assessed participants subjective knowledge of AET; “How informed do you feel about hormone therapy for women with breast cancer”. Participants answered on a 0 (not very well informed at all) to 10 (very well informed) subscale.
Engagement. To observe engagement with the information leaflet, the length of time participants spent on the information leaflet page of the survey was recorded.
Missing data. There was no missing data as all survey items were mandatory. Any non-completed responses were not recorded.
Sample size. A sample size of 1,524 was required to detect an effect size of 0.15, with power of 0.9 and alpha set to 0.1. Alpha was set to 0.1 as a decision-priority approach was taken in which Type I and Type II errors are equally detrimental to selecting an optimised information leaflet15. Assuming 5% of participants would be speed responders (not completing the survey correctly), we increased the sample size to 1,604. The ‘MOST’ package in R Studio version 4.2.0 was used to calculate sample size22,23.
All data cleaning was conducted using R Studio, in R Statistical Software version 4.2.023. R packages used to clean the data included ‘tidyverse’ version 2.0.024, ‘dplyr’ version 1.0.1025, ‘tibble’ version 3.2.126 and ‘descr’ version 1.1.827. Data cleaning involved four main steps: (1) renaming variables; (2) scoring the questionnaires; (3) effect coding the factors relating to the five intervention components (i.e., -1, +1); and (4) exclusion of participant that was under 18. The raw dataset and R code used to clean the data are available at https://doi.org/10.5518/146728.
Written informed consent was obtained electronically from all participants. All procedures, including the informed consent process, were conducted in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000.
The dataset is available in a restricted access data repository. The data is restricted access due to data protection concerns; the data is not directly identifiable but could be viewed as indirectly identifiable. Ethical approval was granted to share the dataset open access in the University of Leeds data repository. All participants consented to their data to be stored in this way, and to be available for use in other research.
Repository Name: Restricted Access Data Repository Leeds
Request to access the data set can be made at: https://doi.org/10.5518/146728
The dataset contains the following files:
Open Science Framework: Extended data for ‘Dataset for a randomised factorial experiment to optimise an information leaflet for women with breast cancer’, https://doi.org/10.17605/OSF.IO/3CZ9Q19
This project contains the following files:
Data are available under a Creative Commons Attribution 4.0 International Licence (CC-BY 4.0).
We would like to thank our patient contributors for their input into the development of the information leaflet, and to Health Creatives for designing the leaflet.
Is the rationale for creating the dataset(s) clearly described?
Yes
Are the protocols appropriate and is the work technically sound?
Yes
Are sufficient details of methods and materials provided to allow replication by others?
Yes
Are the datasets clearly presented in a useable and accessible format?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health literacy, cancer screening, intervention design and evaluation. I recently published a paper evaluating the NHS cervical screening leaflet.
Is the rationale for creating the dataset(s) clearly described?
Yes
Are the protocols appropriate and is the work technically sound?
Yes
Are sufficient details of methods and materials provided to allow replication by others?
Yes
Are the datasets clearly presented in a useable and accessible format?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Psycho-oncology, clinical psychology, breast cancer, adjuvant endocrine therapy adherence and medication management
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 1 30 May 24 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with NIHR Open Research
Already registered? Sign in
If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research.
We'll keep you updated on any major new updates to NIHR Open Research
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)