Early special educational needs provision and its impact on unplanned hospital utilisation and school absences in children with isolated cleft lip and/or palate: a demonstration target trial emulation study protocol using ECHILD

Ethics and dissemination

Research ethics committees have approved the use of the ECHILD database; access to the ECHILD database is approved by the ECHILD team, who are contactable at ich.echild@ucl.ac.uk for proposals for projects using ECHILD.Stake holder groups consisting of focus groups of young people, parents and service providers will help us frame the research question, interpret, and communicate our findings to policy makers, health and education services and families to promote translation of our findings into practice.

Stakeholder involvement

Prior to developing this protocol, two independent meetings were conducted with stakeholders (parents, pupils, teachers) to understand which medical conditions are of interest and which entry timepoints are important for child development. The first meeting was with the Department for Education’s national young SEN advisory group (FLARE) on the 18 of September 2021 and the second with the Young Persons Advisory Group for research at Great Ormond Street Hospital on the 27 of November 2021. This engagement identified that school entry is an important key milestone when SEN provisions are required. Therefore, in the proposed study, we have used school start as our entry point and will generate further target trials based upon further patient engagement.

Study design and setting

The study will be an observational study based on data from the ECHILD dataset previously described in (Mc Grath-Lone et al., 2022) which includes individuals born between 1 September 1995 and 31 August 2020 in England. To reduce confounding-by-indication and other forms of biases when using observational data, we will use a target trial framework to guide eligibility, entry, and an appropriate follow-up period (Hernán et al., 2022). Analyses will be conducted in the Office for National Statistics Secure Research Service using Stata ver. 17 (proprietary, StataCorp) and R ver. 4.0.2 (open source, R Foundation) and the code for the study will be made available via GitHub.

Dataset and linkage

The data source we propose is the ECHILD database, a pseudo-anonymised dataset that links the National Pupil Database (NPD) with Hospital Episode Statistics (HES). In brief, the ECHILD's extract of the NPD contains data from academic terms (Summer, Autumn, and Spring) between 2006 and 2020 and contains information on (but not limited to) school, local authority, year/month of birth, gender, ethnicity, first language, socioeconomic status, free school meal status, absence related data, social care/children in need related data and SEN provision.

The ECHILD’s extract of HES contains details on accident and emergency attendance, admitted patient care, critical care, and outpatient appointments between 1997 until 2021. It contains details on birth admissions, sex recorded by physician, ethnicity, clinical information recorded during hospital admissions, including details of diagnoses, and operations. HES covers 99% of public hospital activity in England (Herbert et al., 2017). HES records since 1998 are also linked to ONS Mortality data covering information on causes and timing of deaths. The linkage coverage periods are described in Mc Grath-Lone et al. (2022). ECHILD has been shown to have a linkage rate between NPD and HES of 95%; the high linkage rates are attributable through a two-stage linkage process (Libuy et al., 2021).

Full methodology of creating the ECHILD dataset is described by Mc Grath-Lone et al., 2022. Enquiries to access to the ECHILD dataset is obtainable by contacting ich.echild@ucl.ac.uk; all researchers accessing the ECHILD dataset will need to be an Office of National Statistics (ONS) accredited researcher. Access to the ECHID dataset will be made through the ONS Secure Research Service, a trusted research environment which requires the researcher’s institution to have Assured Organizational Connectivity, Population

Our population consists of children with isolated CLP recorded in hospital records and followed from Year One of school (the first full year of compulsory education, where pupils are five years old on the first day) between academic years 2008/2009 and 2018/2019 (i.e., born between 1 September 2003 and 31 August 2013). This period was chosen as it contains information of school readiness tests (known as Early Years Foundation Profile - a known good predictor of SEN provision) and 2018/19 was the last entry-into-school academic year prior to the COVID-19 pandemic where access to hospitals and provision of education vastly changed. To identify pupils who started Year One between 2008/2009 and 2018/2019, the earliest recording of “1” from the “NCActualYear” (National Curriculum Actual Year) variable in the NPD dataset will be used; for children whose “NCActualYear” variable is marked as empty or X, we will use “AgePartAtStartOfAcademicYear” equal to 5.

To identify pupils with isolated CLP, International Classification of Diseases version 10 (ICD-10) codes will be applied to primary and secondary HES diagnoses in any hospital admission prior to the start of compulsory education using the following codes: Q35*, Q36* and Q37*. For each pupil, the earliest recorded date in HES will be considered the “diagnosis” date. Pupils whose first recording of CLP in HES is after their first year in school will not be included as SEN allocation needs to proceed diagnosis to avoid reverse causality. Children born with further major congenital anomalies, will be identified using the EUROCAT code list (‘EUROCAT Guide 1.5 Chapter 3.3’, 2023) and excluded using the ICD-10 codes listed in Table 1 to reduce competing needs for SEN provision. The EUROCAT code list was used as it captures major congenital anomalies and not minor anomalies.

We will also restrict our analyses to those born in an NHS funded English hospital due to the importance of birth characteristics such as maternal age, birth weight and gestational age. Additionally, as congenital anomalies are disproportionately recorded in those born in hospital, we have further reasons to restrict our population to those with a birth record in HES to avoid misclassification of congenital anomalies.

Intervention

The intervention will be defined by the categories of recorded SEN provision (including none) in Year one of school (ages five or six). Whilst SEN provision can change throughout a CYP’s educational journey, our implementation of trial emulation focusses on an intention-to-treat analysis (ITT) with the intervention defined at the start of compulsory education. Therefore, we are analysing the initial assignment of treatment and not whether treatment was adhered to or provided. We choose the start of compulsory education as we believe this is a population in need of SEN provision at the start of their educational journey based upon prior evidence of educational (Fitzsimons et al., 2018) and healthcare needs.

To capture differences in type of SEN provision due to severity of CLP, along with other confounders (see covariates section), we aim to classify our exposure variable as “categories of SEN provision” (None, SEN, EHCP) as opposed to a binary outcome (i.e., SEN vs no SEN). To establish SEN status at Year One, we will use the January (Spring) census in Year One of school due to funding being calculated using these censuses. See Table 2 for a list of variables describing SEN in the NPD. See Statistical Analysis section for more detail on analysing comparison groups.

Figure 1 shows our planned Consolidated Standards of Reporting Trials (CONSORT) flow diagram for identifying the population and classifying it according to the exposure variables (categories of recorded SEN provision).

Figure 1. Flowchart of how the proposed population would be derived.

Our population would consist of those who are identified in HES with cleft lip and/or palate before Year 1, who have a birth record in HES and who are linkable to NPD. SEN levels will be based upon SEN recorded by Year 1. No SEN represents either not having recorded SEN provision or receiving SEN after Year 1. Children with cleft lip and/or palate (including bilateral and unilateral): Q35x Q36x Q371, Q373, Q375, Q379, Q370, Q372, Q374, Q378. Exclusion criteria is MCA.

Follow-up

The study population will be followed-up from the initial January census in Year One (to account for time to apply for SEN) until the end of primary school (end of July of Year Six), lost to follow-up, or end of study, whichever occurred first. Children will be considered lost to follow-up if they no longer appear in any NPD school census during primary education; this could be due to a variety of reasons including, transfer to a non-state-funded school, emigration, death, or off-rolling (Jay et al., 2022).

Although ECHILD can be used to follow up children beyond the end of primary school for some academic cohorts, we will limit our follow-up period in this protocol to Year 6 for two reasons: firstly, entering secondary (or middle) education will re-evaluate the need for SEN, and many pupils may no longer be offered SEN provision, and secondly, the time between the assignment of SEN considered here and the outcome may be too long if beyond Year Six, with the outcome affected by many intermediate factors (pupils starting Year One in 2008, would have 13 years of follow-up in HES).

Outcome variables

The outcomes of interest include both unplanned hospital usage and school absences due to the health needs for SEN provision and the intervention being delivered in an educational setting.

Unplanned hospital usage will be measured in days between January of Year One (recorded allocation of SEN provision) and end of follow-up. To identify unplanned hospital usage in HES Admitted Patient Care, we will use the “admission method” variable of the first episode of each admission in HES (admimeth) (Table 3 for the case definition). For hospital utilisation that did not require an overnight admission, we will use the HES Accident and Emergency dataset to account for non-admitted unplanned hospital utilisation (Harron et al., 2018). We aim to combine the “Admitted Patient Care” and “Accident and Emergency” datasets to create a timeline of unplanned hospitalisation between the January census in Year One and the end of Year Six. When an unplanned admission and recording in accident and emergency occurs on the same day, we will only count this as one day, for example when the pupil is initially presented in accident and emergency and is then admitted on the same day.

Absences in the NPD are recorded termly as sessions, corresponding to half-days in school; the total number of potential sessions in a term is also provided. School absences will be measured as sessions between January of Year One and the end of follow up (maximum: end of Year Six). As the population of interest is based upon health needs, our school absences will include those related to medical need. See Table 4 for the case definition of medically related absences during school. In addition to medically-related absences, we will separately evaluate the impact of SEN on unauthorised absences; such evaluations are needed as absenteeism is related to poor academic performance (Allen et al., 2018).

We are not planning to study academic performance by the end of primary education as an additional outcome. This is motivated by the limited number of children in our population who would have recorded Key Stage Two academic outcomes (only those born between 2003–2008). With an estimated prevalence rate of 900 CLP births per year (CRANE, 2021), of whom an estimated 62% are expected to have isolated CLP (Fitzsimons et al., 2023), we would have very small groups of children receiving (some of) the interventions. We will re-evaluate the suitability of evaluating the impact of SEN provision on academic outcomes and include this in our analyses according to the numbers with Key Stage Two data.

Covariates

To account for non-random SEN provision assignment, we will use information on several covariates that are known or suspected to be associated with SEN provision and hospital contacts based upon prior literature. These include CLP specific influences, further clinical/birth, socio-demographic, geographical, and educational influences. See Table 5 for a list of potential confounders from relevant literature. The distribution of these potential confounders by exposure status will be examined (see Table 5 for an outline) and directed acyclic graphs representing the assumed relationships among these variables, SEN exposure, and the outcome of interest will be drawn to identify the variables that will be controlled for to estimate the causal effect of the interventions, using the opensource software, DAGitty ver 3.0.

Specifically, these covariates, measured at birth or before or at the start of Year One, include: cleft severity (based upon prior literature - (Fitzsimons et al., 2018) – see Table 1 for ICD10 codes to differentiate cleft severity), comorbidities (categories based upon prior literature (Hardelid et al., 2014)), gestational age, maternal age, prior hospital contact (unplanned, and outpatient contacts), gender, ethnicity (latest recorded in NPD to reduce missingness), English as a first language, Income Deprivation Affecting Children Index (IDACI) quintile, free school meal eligibility, month of birth, academic cohort (to account for changes in policy over time) and standardised school ready assessments (Early Years Foundation School Profile). Additional school-level variables we aim to include is the proportion of children in the school the child attends in Year One who were recorded as receiving SEN support/EHCP in the previous academic year, and current pupil teacher ratio.

Biases

To reduce confounding and other sources of bias affecting observational data, we will adopt a Target Trial Emulation (TTE) framework (Hernán et al., 2022). TTE enables observational data to be mapped to a hypothetical target experimental trial counterpart by creating the specification of an ideal (pragmatic) trial and using this as a basis to shape the observational study design. TTE consists of one, defining the specifications of a hypothetical target experimental trial of the causal question of interest (including the corresponding effect), two, emulating the specifications of the ideal target trial using observational data and three, estimating the effects of interest using the emulated trial data. The first component of TTE involves defining inclusion/exclusion criteria for entry, a treatment strategy (including time of treatment assignment), follow-up frequency and modality, outcome measures, causal contrasts of interest (estimands) and estimation methods. The second component of TTE involves handling the observational data to emulate the structure of the data that would be gathered in the specified target trial. Finally, the third component of TTE concerns dealing with the inevitable confounding that affects observational data and explicitly outlining the analytical methodology ahead of the data wrangling. In Table 6 we describe the (ideal) target trial one would design to investigate the causal effect of SEN provision on the selected health and educational outcomes in the first year of compulsory education on CLP children and the equivalent emulated trial to be generated from ECHILD.

The estimands we will target are firstly, the average treatment effect (or average causal effect): this is a causal contrast of average potential outcomes for the whole isolated CLP population, secondly, the average treatment effect in the treated: this is a causal contrast restricted to the “treated”, i.e., those that received SEN and finally, the average treatment effect in the not treated; the causal contrast in those who did not receive SEN in Year One (Wang et al., 2017).

Analysis

Explorative analyses. To estimate the representativeness and external validity of the derived cohort, we will compare the following distributions against existing literature, firstly, the rates of CLP children who start Year One between 2008/09 and 2018/19 (CRANE, 2021) and secondly, previously published rates such as school academic attainment (Fitzsimons et al., 2018).

To understand whether pupils who are recorded to have received different categories of SEN provision had the chance to be recorded with another category and therefore for the intervention levels to be comparable using the available data (i.e. to assess whether the positivity assumption could be invoked when performing casual inference (Zhu et al., 2021)), we will examine the distribution of the propensity scores for the recording of SEN support/EHCP across the subgroups of children defined by their observed characteristics. The propensity scores will be predicted using logistic regression, with the covariates mentioned above included as predictors. As there are three categories of SEN (None, SEN Support and EHCP), pairwise propensity score comparisons (Rassen et al., 2013) will be evaluated for common support between: None vs SEN Support, None vs EHCP and SEN Support vs EHCP. The robustness of the selected propensity score model would be assessed by triangulating the predicted scores with those derived using machine learning methods (such as Classification and Regression Trees) (Lee Brian et al., 2010).

Causal inference. Once explanatory analyses have been completed, causal inference will be conducted for pairs of interventions which have common support. To account for the distribution of our outcome, that is number of unplanned hospital contacts and number of absence sessions, we will use Poisson (or negative binomial) regression models. To account for differential follow-up time, we will use the logarithm of one of the following as offsets: days between January of Year One and end of follow-up for hospitalisation usage and total number of full sessions between January of Year One and end of follow-up for school absences. The likely clustering of pupils within local authority will be addressed either by fitting mixed effects models or by using robust inference (or both).

We will triangulate complementary estimation methods to address confounding bias due to non-random assignment of SEN provision. We will compare results obtained assuming no-unmeasured confounding (that is, we have data on all the relevant confounders) and assuming instead that we have an instrumental variable (if there is for example variation in SEN provision by local authorities). Our analyses will involve three general approaches.

The first approach will include “traditional” epidemiological methods such as reporting of crude associations (“naïve estimator”) between intervention and outcome, and of conditional associations obtained by fitting appropriate regression models (Bender, 2009); due to the conditional nature this method, the evaluation of these effects cannot evaluate our causal contracts of interest, such as the average effect on the whole population.

The second approach will involve dealing with measured confounding using outcome-based models (Smith et al., 2022), such as the parametric g-formula, inverse probability weighting of marginal structural models, and inverse probability weighting regression adjustment. Confidence intervals for these models will be estimated using bootstrapping (1000 replicants). These estimation methods target our estimands of interest, including the average treatment effect, average treatment effect in the treated and the average treatment effect in the not treated (see above).

The third set of methods, instrumental variable analysis, will only be possible if suitable instruments for SEN provision are identified (Greenland, 2018), for example if there are policy changes in provision that are implemented at different times across local authorities or changes in school policy for example brought about by governance change (Liu et al., 2020). There is a well-documented change in SEN provision from 2014, which may also allow a difference-in-difference approach to be implemented.

Missing data. Depending upon the proportion of missingness affecting the data and the mechanisms of missingness, we will first use information across data sources to fill the missing information prior to data imputation; for example, using the variable “Sex” held in HES to complement missing values in the NPD variable “Gender”. We will use Imputation using Chained Equations under a missing at random (MAR) assumption to singly imputed missing values, as opposed to multiply imputed, because the imputation will be embedded within the bootstrapping conducted to estimate confidence intervals of point estimates (Schomaker & Heumann, 2018). All relevant variables (including interactions and non-linearities) will be used to predict missing data including the exposure and the outcome (Azur et al., 2011).

Sensitivity analyses. To account for uncertainty in the recording of observational data that may lead to measurement errors, we aim to conduct sensitivity analyses. First, we will conduct a sensitivity analysis to mitigate against a delayed recording in SEN provisions, by expanding the exposure window to the first term in Year Two as part of the January census; this analysis will include information collated during Year Six as part of the adjustment set of baseline covariates. Secondly, to understand the driver of unplanned hospitalisation, we will decompose our outcome of unplanned hospitalisation into three categories firstly, the number of days recorded in Accident and Emergency or in Unplanned Admitted Patient Care, secondly, the number of days recorded in Accident and Emergency, and thirdly, the number of days recorded in Unplanned Admitted Patient Care. Similarly, we will examine absences in subgroups defined by whether they were medically related or unauthorised. Finally, we propose to analyse the differences between using recorded child sex (reported by physician in HES at birth) and gender (submitted by parent/carer during school registration in NPD) and produce point estimates tables of the intervention variable when using either measure.