Timing and value of participant vouchers to improve questionnaire return rates in the STADIA (STAndardised Diagnostic Assessment for children and adolescents with emotional difficulties) Trial

SWAT question

Does the timing and value of participant vouchers have an effect on questionnaire return rate in a randomised trial involving children and young people and their parent/carers?

Patient and Public Involvement

Patients and the public were directly involved in the development of the SWAT research question. Consultation with the STADIA Patient and Public Involvement (PPI) Lead (CE) and both the parent/carer panel and the STADIA Youth Lab (CYP panel) indicated that offering voucher payments at both time-points may promote engagement with the trial throughout the follow-up period. In response to this, the SWAT was devised to assess the effectiveness of voucher timing and value on questionnaire return rate.

SWAT protocol

The SWAT is registered on the SWAT repository as number 157 (Sayal et al., 2020).

Host trial

A two-arm, parallel randomised (1:1 allocation ratio) SWAT was embedded in the STADIA Trial (prospectively registered on ISRCTN, reference number: ISRCTN15748675). STADIA assessed the clinical and cost effectiveness of a Standardised Diagnostic Assessment tool called the Development And Well-Being Assessment (DAWBA) as an adjunct to usual clinical care in children and young people presenting with emotional difficulties referred to Child and Adolescent Mental Health Services (CAMHS). The primary outcome was a clinician-made diagnosis of an emotional disorder within 12 months from randomisation which was obtained from clinical records, however, secondary outcomes such as depression and anxiety symptoms in the child/young person and resource use information for the cost effectiveness analyses were obtained from 6- and 12-month participant completed questionnaires. The trial recruited 1,225 children/young people between 27^th August 2019 and 17^th October 2021 from eight NHS Trusts in England.

Participant-reported outcome data were collected from participating parents/carers (P/C) and/or children/young people (depending in child age, see Table 1) at 6- and 12-months post-randomisation through online questionnaires, which participants were alerted to via email and / or text message.

The parent/carer was the primary participant for all children/young people aged <16. Children/young people aged 11–15 were also invited to participate alongside their parent/carer by completing their own questionnaires. Questionnaires for the child/young person aged 11–15 were sent via the parent/carer.

Children/young people aged 16–17 years were the primary participant and were sent follow-up questionnaires directly for completion. For them, parent/carer participation was optional. If participating, they were considered the secondary participant and were sent their questionnaires directly.

Participants

All host trial participants (both CYP and P/C) who were randomised between 07^th May and 17^th October 2021, were included in the SWAT. Follow-up questionnaires were returned between 07^th November 2021 and 17^th January 2023.

Intervention and comparator

Participants were allocated to receive either:

a) one £10 voucher conditional on completion of the 6-month questionnaire plus another £10 voucher conditional on completion of the 12-month questionnaire or

b) one £20 voucher conditional on completion of the 12-month questionnaire.

Participants were informed of the conditional voucher and its value at each follow-up time-point when the questionnaires were sent. Reminders to complete both 6- and 12-month questionnaires also mentioned the voucher and its value for those who were allocated to receive two £10 vouchers, whereas, the 6-month reminder did not reference the 12-month voucher for those who were allocated to receive one £20 voucher at 12-months. e-voucher were used as they were cost effective, as well as easy to resource and distribute to participants via an email address.

Outcomes

We did not specify a primary outcome for the SWAT as we did not anticipate the study would have sufficient power to detect small but important differences. However, we investigated the following outcomes:

Sample size

We estimated that during the lifetime of the SWAT, approximately 500 participants would be recruited to the host trial and therefore into the SWAT. Table 2 shows the smallest detectable risk difference for a binary outcome with 80% power for each group of respondents. This was based on a conservative baseline response rate of 50% and 5% two-sided significance with no adjustment for multiplicity.

Randomisation

Following the host trial randomisation, participants had a secondary randomisation into the STADIA SWAT with a 1:1 allocation ratio using randomly permuted blocks (with block sizes of 2 and 4) stratified by host trial allocation. Child/young person and parent/carer dyads received the same allocation. The Trial Statistician created the allocation lists and then concealed randomisation was carried out automatically using the same online system as for the host trial.

Blinding

All participants were informed in the participant information sheet that if they completed the questionnaires, they would receive an e-voucher (value unspecified). Participants were not aware they were involved in an embedded SWAT. They were informed of the voucher denomination upon receipt of the questionnaire, which was at 6- and 12-months for the £10 group, and 12-months only for the £20 group. The STADIA trial management team were not blinded to the SWAT allocation.

Statistical analysis

For each group of respondents, outcomes were analysed using appropriate mixed-effects regression models dependent on data type. For binary outcomes, mixed-effect logistic regressions were used. For number of questionnaires returned we used original regressions and then for time to event outcomes a cox proportional hazards model was used. The models adjusted for host trial allocation and included a random effect for recruiting site. Comparisons of binary outcomes were presented as both an absolute (risk difference) and relative (risk ratio) effect, along with 95% confidence intervals. Time to event outcomes were presented as hazard ratios and ordinal outcomes were presented as common odds ratios, with 95% confidence intervals presented for both.

Analysis was carried out using Stata v18.0.

Implications for trial practice

Due to the results suggesting that providing £10 vouchers at each time point increases questionnaire return rates, trialists should consider a little and often approach with vouchers to encourage questionnaire return. This is also important for more complete capture of resource use and health economics data, which is crucial for trials evaluating cost effectiveness. However, again due to the small sample sizes and specific study population, further research is required to confirm these findings.

Implications for SWAT research

Previous SWATs attempted to assess the effectiveness of providing financial incentives to improve retention within clinical trials (Cooke et al., 2013; Parker et al., 2020), however they have not assessed whether the timing and value of the incentives have an effect. This SWAT indicates that the timing of providing monetary incentives may play a role in improving return rates of participant questionnaires, but further SWATs in the area will help to enhance the evidence base.