Improving Measurement and Priorities for Assessment in Clinical Trials of childhood-onset Dystonia International Core Outcome Set (Dystonia IMPACT-COS): A protocol

Aim

To synthesise published research outcomes and views from children, families, and professionals to develop consensus on a core outcome set for trials for children with non-degenerative dystonia as a predominant motor feature impacting function, which might be combined with associated HMDs or neurological features.

To meet this aim, DI-COS has three phases ( Figure 1):

Figure 1. Dystonia IMPACT-COS study overview.

Phase 1. Identification of outcomes and Delphi survey development

Objective 1: Identify the interventions and outcomes reported in the published evidence for this population, and map them to the International Classification of Functioning Framework (ICF) (in progress),²⁶

Objective 2: Gather the views of children and their parents/carers on what is important to measure to inform the future development of a core outcome set for interventional trials and clinical care, including those underrepresented in research.

Objective 3: Gather the views of health, social care, and education-based professionals internationally on what is important to measure to inform the future development of a Delphi study to establish a core outcome set for interventional trials and clinical care.

Phase 2. Determining the core outcomes

Objective 4: Conduct a three-round, international modified e-Delphi study via an online survey with children, parents/carers, and professionals to gain consensus on important areas of functioning and core outcome domains.

Objective 5: Hold an expert panel consensus meeting with professionals and lived experience interest-holders to review Delphi findings and recommend the priority areas as outcome domains for the COS to be used in future trials, audit, evaluation, and practice.

Phase 3. Identification of outcome measurement tools

Objective 6: Identify the available outcome measurement tools for the inclusion in the final COS domains established in Phase 2 through an evidence review and consultation with interest-holders.

Objective 7: Systematically evaluate the methodological quality and measurement properties of the identified outcome measurement tools using the COSMIN framework to formulate evidence-based recommendations for their use and identify gaps in the existing validation evidence.

Objective 8: Conduct a key interest-holders and expert group consultation to review the findings of this phase and ratify the recommendations made and establishing gaps in available outcome measurement tools for measuring the core outcomes.

Phase 1a: Scoping review of outcomes measured and reported in interventional studies (Objective 1).

Research Questions

Methods

The six step Arksey and O’Malley scoping review framework will be used.³¹ Searches will be conducted in Medline and Embase via Ovid, and CINHAL Plus via EBSCOhost using terms related to non-degenerative childhood-onset dystonia and associated HMDs, all types of interventions, and all types of outcomes.²⁶ Outcomes reported in the studies will be categorised across the International Classification of Functioning and the COMET outcome taxonomy.

Findings will guide topics for the qualitative interviews and professional survey, and longlist of candidate outcomes for the subsequent e-Delphi in Phase 2.

Phase 1b: Qualitative study with children and parents/carers (Objective 2)

Sampling and recruitment

This work explicitly seeks to include and represent underserved groups in healthcare research³² including: (i) children with rare disorders, intellectual disabilities, and those who communicate non-verbally; (ii) children and families from the UK’s most deprived, multi-ethnic/cultural areas. To ensure maximum variation and representation, we will use purposive sampling based on child’s age, functional level, ethnicity, and Index of Multiple Deprivation (IMD) using a sampling matrix. Given the heterogeneity of this population, the concept of information power was used to support determination of the proposed sample size for exploring children and families’ lived experiences to ensure the diversity and depth needed to achieve the studies aims. Thus, the proposed sample size is 60–80 participants (30–40 children and 30–40 parent/carers).

Inclusion and exclusion criteria are shown in Table 1.

Selection and recruitment routes are shown in Figure 2.

Figure 2. Recruitment process and routes and for Phase 1-qualitative study.

All adult participants will provided informed consent to participate in the study. For young people without capacity, a consultee will be identified to advise on their participation. Children 7 years and older will provide written assent (where physically able to and children under 7 will provide verbal assent only), and parents will provide written informed consent for their child’s participation.

Interview format data collection

Topic guides will be developed with the Dystonia Impact PPI groups, informed by the findings of the scoping review, and centred around the 6 F’s framework.³³ The 6 Fs is a family-centred model emphasising six domains (functioning, family, fitness, fun, friends, and future), to support strengths-based care in childhood disability. Each domain aligns with a component of the ICF.³⁴ PPI will support further refinement of questions on topic guides ensuring they are intelligible, appropriate and sensitive, and support adaptation for children of different ages and cognitive abilities. A creative methods resource pack will also be developed, led by KK (research psychologist) and refined through PPI workshops with children to ensure activities and graphics are fun, engaging, and age and developmentally appropriate.

Resources include Talking Mats®, switches, cameras and photo boards, and prompts for talking or drawing including an ‘all about me cloud’, ‘change tree’, and ‘emoji cards’ to support children to communicate their feelings and thoughts.

Children and parent/carers will be invited to take part in semi-structured interviews lasting approximately 1 hour at a mutually agreed venue, e.g., home, community or online. For younger children, interviews are expected to last 30–60 minutes, including breaks. Interviews will use open-ended questioning based on topic guides. Creative methods (i.e., photography, drawing) will be used particularly with young children to support identification of priority outcomes and administration preferences. Non-verbal communication methods will be explored by the lead researcher (HG) and the research team supporting implementation. When necessary, children who communicate non-verbally will be supported by a clinician known to them to support communication (i.e., Talking Mats®, access to Assistive Technology via eye gaze/switches). Translation for non-English-speaking families will be available. Interviews will be digitally recorded and transcribed verbatim using a transcription service.

Analysis of semi-structured interviews

Interviews will be analysed through content analysis^35,36; with deductive coding³⁷ informed by the ICF and the COMET taxonomy classification. Content analysis data will be summarised into a single data set using Microsoft Excel. A matrix will be created whereby each row will represent an individual participant, and each column a COMET domain. This will inform the development of the subsequent e-Delphi survey for Phase-2.

Phase 1c: International survey of health, social care, and education-based professionals (Objective 3)

Sampling and recruitment

Sampling will combine purposive (interest-holders targeted on the basis of being a medical, nursing, social care, or allied health professional working with this population³⁸) and convenience approaches. Study advertisements will be disseminated through social media, professional associations, and through Dystonia IMPACT expert panel membership networks, directing potential participants to the online Qualtrics™ survey.

Data collection

The survey will be informed by ICF-mapped outcomes from the scoping review, with open questions to identify other important outcomes, designed using the Consensus-Based Checklist for Reporting of Survey Studies (CROSS).³⁹

Participants will be invited to complete the online Qualtrics™ survey in their language of choice, including English, Spanish, Portuguese, Polish, French, Italian, Arabic, and Dutch (translation will be supported by functionality within the Qualtrics™ platform and checked for accuracy by a native speaking individual or with proficient fluency in each language).

The survey will be structured into sections covering participants’ professional background (profession, years of experience, and country of practice), current use and selection of outcome measures in clinical practice, perceived importance of key characteristics of outcome measurement tools, challenges encountered when selecting or implementing outcome measures, and open-ended questions exploring gaps, recommendations, and areas for improvement in the evaluation of children with dystonia.

Survey data analysis

Descriptive statistics will be used to summarise participants’ responses. Categorical variables will be presented as frequencies and percentages. Likert-scale items will be treated as ordinal variables and summarised using medians and interquartile ranges (IQRs). When response distributions are sparse or highly skewed across categories, Likert responses will be collapsed into broader categories for analysis.

Where appropriate, differences between professional groups, countries, and years of experience will be examined using the Chi-square test or Fisher’s exact test for categorical variables, and the Mann–Whitney U or Kruskal–Wallis tests for ordinal variables. All tests are two-sided, and a p-value <0.05 will be considered statistically significant.

Open-ended question responses will be analysed initially using content analysis^35,36; deductively coding³⁷ informed by the COMET taxonomy³⁰ and the ICF³⁴ to inform the subsequent e-Delphi survey (Phase-2). Then we will use thematic analysis,^40,41 using an inductive coding approach,³⁷ with the assistance of NVivo coding software, to develop codes and themes.

Synthesis of phase 1 findings

Findings from the scoping review, qualitative interview study and professional survey will be triangulated to inform the development of the e-Delphi survey to use in Phase 2. Dystonia IMPACT PPI experts and professional group will validate the survey prior to starting Phase 2.

Phase 2a: International online Delphi survey with experts (professionals and people with lived experience) (Objective 4)

The Delphi method has been widely used to gain consensus in COS studies as it facilitates participants completing sequential rounds of surveys, with feedback on the previous round to inform decisions.^42–44 It is recognised as a robust way to gain consensus from different interest-holder groups simultaneously and has been used with most age groups.⁴⁵ The study design follows relevant guidance,^46,47 including the Conducting and REporting DElphi Studies (CREDES).⁴⁸

Sampling and recruitment

Participation in the Delphi study will be open to:

Recruitment will take place across at least five countries using multiple strategies to encourage broad participation. Materials and surveys will be available in multiple languages, and participants will be recruited through patient groups, charities, professional networks, study website, social media, and snowball sampling.

Data collection

Qualtrics software will be used for online data collection. Parents, professionals and young people aged 16+ who can consent for their own participation will be invited to complete the online Delphi. Parents of children aged 6–15 years old who complete the survey will be asked at the end whether their child would also like to participate. Parents can indicate if their child would prefer to complete an online survey or take part in a workshop. If the child prefers the online survey, the parent will receive a child version survey link. If a workshop is preferred, the child will be invited to join a workshop (online/in-person/hybrid). Alternatively, children’s workshops will be advertised via our sites and networks to ensure independent participation.

Consultees (who knows the young person well; usually the parent) will be identified for children 16–18 without capacity. The young person will be invited to attend a workshop to communicate their views, or a survey link will be sent to their parent/carer to support completion.

Materials

To ensure children who communicate non-verbally and/or have complex needs can participate, we will use a simple toolkit developed in earlier workshops with individuals with lived experience. Age-tailored content will support more meaningful participation from children and will also be suitable for older children with intellectual disabilities.

The survey will be co-created with Dystonia Impact PPI members ensuring plain language is used throughout. PPI recommendations already include the use of age-appropriate response formats, including 1–5 facial expression scales, icons, and short survey sections to reduce cognitive load. An ‘I am not sure’ option will be included. Full definitions, measurement examples, free-text comment boxes and read-aloud options will be provided.

PPI workshops with children will be conducted at the end of each round to ensure their perspectives are fully considered.²⁰ The Delphi survey, survey workshops, and PPI workshops will be followed by an interest-holders consensus meeting to finalise core outcome set.^25,26

Round 1: Participants will rate the importance of the identified outcome items using a Likert scoring system (not that important – to - critically important) with an option of ‘unable to score’ and the opportunity to add additional missing outcomes for the second round. For professionals, parent/carers, and young people 16–18 (with capacity), a 7-point scale will be used. For children aged 12–15 years old, a 5-point scale will be used. For children aged 8–11 and children with intellectual disability, a 3-point scale will be used. For new proposed outcomes, at least two research team members (including a young person with lived experience) will manage proposed new outcomes and create new labels and definitions.

Round 2: Group median rating for each outcome and group (children/parent/professionals) will be shown for re-rating of outcomes retained from the previous round. The same process for newly proposed outcomes will follow as in Round 1.

Round 3: A final round of outcomes will be presented for rating.

Data presentations between rounds will include visual presentations with simple bars and median markers but also traffic-light system (i.e., green = consensus, red = consensus out, amber = uncertain). The wording for young people will be developed with PPI advisors; for example, ‘most people thought this was very important’.

Data analysis

Data analysis will present scores for each interest-holder group and the total sample in each round. Scores will be summarised using the median and percentage meeting criteria. The median rating by item and group will be shared in the subsequent rounds to inform decision-making and support consensus development. Missing data will not be imputed; mean scores across and within participant groups will be used to dictate inclusion of outcomes. Only those who have completed round 1 and/or 2 will be invited for round 3.

Inclusion or exclusion consensus will be determined based on pre-specified criteria³⁰ detailed on Table 2.

Phase 2b: Expert panel consensus meeting (Objective 5)

Agree on the final list of outcomes and measures utilising rating and ranking by.

professionals and lived experience interest holders.

Consensus defined as inclusion of outcomes rated as important by a majority (70%), with a small minority (<15%) considering it to have little/no importance.

Expert interest holder consultation groups

The multi-disciplinary panel will consist of paediatric dystonia expert clinicians/clinical researchers (i.e., neurologists, developmental paediatricians, rehabilitation physicians, and therapists) from the UK, the Netherlands, France, Australia, Canada, Spain, and the United States. Experts with lived experience panel will consist of children with dystonia and their families from the UK who are members of the Dystonia IMPACT Children and Family Advisory Groups and international charities.

Methods

This review will be carried out according to the COSMIN guidelines and reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. Searches will be conducted in Medline and Embase via Ovid, and CINHAL Plus via EBSCOhost using terms relating the names of tools identified in the Scoping Review in Phase 1, validity studies, children, and dystonia.

Interest holders and the expert panel will be consulted to identify any additional outcome measurement tools which are not in the literature.

The methodological quality of studies evaluating each OMT will be assessed using the COSMIN Risk of Bias checklist. Measurement properties (including content validity, structural validity, internal consistency, reliability, measurement error, construct validity, and responsiveness, where available) will be evaluated against COSMIN criteria for good measurement properties. Where multiple studies are available for an instrument, findings will be synthesised and the evidence quality graded using a modified GRADE approach as recommended by COSMIN.

Outcome measurement tools will be recommended only if they demonstrate good content validity and internal structure and feasibility. Where no suitable tools are identified this will highlight the need for a domain-specific instrument. Findings will be reviewed with interest-holders and the expert panel to confirm the recommendations and identify gaps.