Impact of special educational needs provision on hospital utilisation, school attainment and absences for children in English primary schools stratified by gestational age at birth: A target trial emulation study protocol

Patient and Public Involvement

Prior to developing this protocol, independent meetings were conducted with stakeholders (parents, pupils, teachers) from existing patient advocacy groups including the Young Person’s Advisory Group (YPAG), Council for Disabled Children’s group (FLARE) and the Great Ormond Street National Children’s Bureau Families Research Advisory Group (FRAG). On 14 November 2020, FLARE were introduced the ECHILD dataset and it’s use of linked administrative data and to the observational study design with warm reception Further meetings were held with FLAREon the 18th of September 2021 and with YPAG for research at Great Ormond Street Hospital on the 27th of November 2021. This engagement identified that school entry is an important key milestone when SEN provisions are required. Therefore, in the proposed study, we have used school start as our entry point and will generate further target trials based upon further stakeholder engagement. The Great Ormond Street Hospital for Children’s NHS Foundation Trust Young People’s Forum voiced that school absences were an important topic for research on 20 March 2021. Therefore, using these interactions, we’ve created our research question, which was presented to the HOPE study steering committee, and includes parents of children with disabilities who will review and advise the on the presentation and dissemination of the study findings. Records and learnings from public engagements can be found here.

Study design

Trial emulation framework applied to observational educational data linked to healthcare data. Analyses will be conducted in the Office for National Statistics Secure Research Service using Stata 17 and R version 4.0.2 (open source, free software). Once written, the code for the study, including algorithms to identify the population, exposure, outcomes, and confounders, will be made publicly available on publication of the full manuscript.

Data source and linkage

We will use the ECHILD database, a pseudo-anonymised dataset that links Hospital Episode Statistics (HES) with the National Pupil Database (NPD). A linkage rate of 95% has been reported between NPD and HES in ECHILD, with high linkage rates attributed to a two-stage linkage process (Libuy et al., 2021).

In brief, the ECHILD's extract of NPD contains pupil-level data from state schools in England for academic terms between 2006 and 2020 (Mc Grath-Lone et al., 2022). This includes school, local authority, age, gender, ethnicity, first language, socioeconomic status, free school meal status, recorded absences, social care/children in need related data and SEN status. In addition to the NPD, school level characteristics such as school type (including special or mainstream), school rating, and governance are available through the Department for Education’s opensource ‘Get Information about Schools’ (GIAS) register, and linkable to ECHILD using the school’s unique reference number (GOV.UK, 2022).

The ECHILD’s extract of HES contains details on admitted patient care, outpatient appointments, accident and emergency utilisation, and critical care between 1997 until 2021. It contains details on admission and discharge dates, patient characteristics (e.g., sex, ethnicity, area of residence) and clinical information recorded during hospital admissions (such as, details of diagnoses and operations). HES covers 99% of public hospital activity in England (Herbert et al., 2017). HES also contains birth records which record characteristics such as gestational age, birthweight, maternal age; missingness in an individual’s birth record can be complemented using the corresponding mother’s delivery record. Furthermore, since 1998, HES records are also linked to ONS Mortality data covering information on mortality causes and timing of deaths.

Further details of the ECHILD dataset are documented by Mc Grath-Lone et al., 2022.

Population and follow-up

Our population is singleton children who were born in NHS-funded hospitals in England between 1 September 2003 and 31 August 2008 and were enrolled in year one of a state-funded primary school in England at age five/six years (see Figure 1). Children will be excluded if they do not have complete information on gestational age. Child will also be excluded from analyses of educational outcomes if they have missing data on the early years foundation stage profile (in reception, age four/five). We will also exclude children with a gestational age of <24 or >44 weeks or those with implausible gestational ages based on birthweight because of a high risk of misclassification. Comparisons of included and excluded children will help to inform whether there are issues of selection bias. This population was chosen as these children can be followed up to the end of primary school (year six, age 10/11) in ECHILD, with the latest academic year of follow up before the COVID-19 pandemic.

Figure 1. Expected age at entry into primary school years one to six, by birth year and follow-up year; Y = year; ^adefined according to the academic calendar (i.e., 2003/04 includes 1 September 2003 to 31 August 2004, inclusive.

The study population will be followed-up from the January census in year one (age five/six) until the first chronological event of: end of primary school (year six, age 11 at exit), lost to follow-up or end of study (30^th July 2019). Children will be considered lost to follow-up if they no longer appear in any NPD school census; this may be due to transfer to a non-government funded school or alternative provision, off-rolling (where pupils are illegally excluded from school) (Jay et al., 2022), emigration or death. We begin follow up in year one rather than reception (the first year of primary school in England) as it is the first full school year when education is compulsory for all children. We use the January census (rather than the October census) to allow for time for pupils to be assigned SEN provision.

Subgroups

We expect the impact of SEN to vary according to the child’s need for SEN, which is correlated with decreasing gestational age at birth (Libuy et al., 2023). We will therefore conduct all analyses separately for five subgroups, defined by completed weeks gestation at birth: extremely preterm (24-<28 weeks); very preterm (28-<32 weeks); moderately preterm (32-<34 weeks); late preterm (34-<37 weeks); full term (37 to <42 weeks) (ONS, 2015).

Intervention variable

Our intervention consists of four categories of recorded SEN provision in the January census of year one of school: none; SEN support (previously known as School Action/School Action Plus) at mainstream school; EHCP (previously known as statement of SEN) at mainstream school; and special school attendance (where the vast majority of children have an EHCP). Whilst SEN provision can change throughout a child’s educational journey, our implementation of trial emulation focusses on an observational-analogue of intention-to-treat analysis (ITT) of SEN at the start of compulsory education. This analyses the assignment of treatment and not whether treatment was adhered to or provided. We choose the start of compulsory education as we believe this is a population in need of SEN provision from the start of their educational journey based upon prior evidence of educational (Libuy et al., 2023) and healthcare needs (Coathup et al., 2020).

Outcome variables

We will evaluate both health and educational outcomes.

For health outcomes, we will evaluate unplanned hospital utilisation, consisting of the number of unplanned admissions to hospital (defined by the admission method in the first episode of care) and contacts with an accident and emergency departments between January of start of year one (age five/six) and at the end of year six (age 11) (Harron et al., 2018).

For educational outcomes, we will evaluate key stage two English and mathematics assessments (taken in Year six, at ages 10/11), including whether assessments are taken (yes or no) and, if taken, attainment in the assessments. To account for time-varying changes in recording of educational outcomes, we will use standardised scores within academic year.

We will also evaluate the number of absences during primary school (January year one to the end of year six) including unauthorised absences and absences related to illness and dental or medical appointments.

Covariates

To account for determinants of SEN provision assignment in children with similar gestational ages, we will use information on covariates known or suspected to influence (or be associated with) SEN provision based upon prior literature (Coathup et al., 2020; Hutchinson, 2021; Libuy et al., 2023). Table 1 shows our preliminary list of sociodemographic, educational and health related covariates which are related to SEN provision and both educational and health related outcomes. We will use DAGitty version 3.0, an open-source piece of software create directed acyclic graphs (DAGs) to guide our selection of variable adjustment set to reduce the risk of unaccounted confounding, overadjustment and potentially mediating away any true effects.

Bias

To reduce confounding and other sources of bias impacting data collected outside of a randomised controlled trial setting, we will adopt the Target Trial Emulation (TTE) framework (Hernán et al., 2022). TTE maps observational data to a hypothetical target experimental trial counterpart by creating the specification of an ideal (pragmatic) trial and using this as a basis to shape the observational study design. TTE consists of firstly, defining the specifications of a hypothetical, ideal experimental trial of the causal question of interest (including the corresponding causal contrast), secondly, emulating the specifications of the ideal target trial using observational data and thirdly, estimating the effects of interest using the emulated trial data. The first component of TTE includes defining an inclusion/exclusion criterion on entry, a treatment strategy (including time of assignment and entry), follow-up frequency and modality, outcome measures, causal contrasts of interest and the analytical estimation methods for an ideal trial. Using the second component of TTE, observational data are wrangled to emulate the distribution of the data if it were to have been gathered prospectively in the ideal trial. Finally, the third component of TTE requires using methods to adjust for known and suspected confounding. In Table 2, we describe the ideal target trial that would be designed to investigate the causal effect of SEN provision (by the upcoming January Census in the first year of compulsory education) on the relevant outcomes and the equivalent emulated trial to be generated from ECHILD.

Statistical analysis

Data wrangling. Based upon the proportion and mechanisms of missingness in the data, we will first use future recordings to complement missing baseline covariates such as gender; secondly, we will complement non-missing data in HES and NPD prior to data imputation; for example, using sex variable from HES to complement missing values in the NPD variable gender (Azur et al., 2011).

Exploratory Analysis. We will first analyse the feasibility counts of the ECHILD data, including gestational age subgroups, the distribution of variables including our exposure (SEN provision) and confounders (Table 1). This will include assessing the feasibility of including children attending alternative provision (including pupil referral units) in our eligibility criteria and follow up; these groups are assumed to have small numbers and hence, their inclusion, may pose violations the positivity assumption.

To understand whether there are violations of the positivity assumption (i.e., whether pupils who are recorded to be requiring different categories of SEN provision are comparable), we will calculate and compare the propensity score distributions for each SEN category within each gestational age group. We will compare the density distribution between each pairwise of groups (Rassen et al., 2013), for example, none versus SEN support in mainstream school, SEN support versus EHCP in mainstream school, none versus EHCP in mainstream and so on. Propensity scores for each SEN provision category will be estimated using logistic regression; to assess their robustness, binary machine learning predictors of each SEN provision category, such as tree-based algorithms, will be used and the resulting propensity scores compared to those obtained when using logistic regression (Lee Brian et al., 2009).

Causal inference. Our causal analyses will be conducted for pairs of interventions where the causal assumptions of non-interference, consistency, positivity, and conditional exchangeability are assumed to hold (Hernán, 2012) (see Table 3). For health outcomes and school absences (which are count data) and educational outcomes (which are continuous variables), we aim to triangulate results from three groups of methods: methods traditionally used in epidemiology, methods that rely on the no-unmeasured confounders assumption and, if possible, methods that exploit instrumental variables or difference in difference methods.

Our first group of methods will implement the naïve and adjusted estimators using general linear models as part of our traditional epidemiological estimates including Poisson based link functions (with the logarithm follow-up time as an offset) for counts of individual health outcomes and absences, and linear link functions for individual educational scores (Arnold et al., 2021). The second group of methods includes outcome-based methods which rely on the no-unmeasured confounding assumption and expand on traditional epidemiological methods by focussing on marginalising results over the population using models such as the parametric g-formula, inverse probability weighting, and inverse probability weighting using regression adjustment (Smith et al., 2022). With these methods inference will be based upon bootstrapping. For both health and educational outcomes, we will calculate and compare the following causal contrasts: observational analogue of the ITT, the overall average treatment effect, the average treatment effect in the treated and the average treatment effect the not treated (see Table 4 for definitions).

The third group of methods includes instrumental variable and difference-in-difference methods and are only suitable if instruments for SEN provision are identified, for example if there are policy changes in provision that are implemented at different times across local authorities (Greenland, 2018). These would lead to estimate (under the assumption of individual homogeneity of effects) the observational analogue of the ITT. Related to these are difference-in-difference based methods that to estimate group differences against predicted trajectories between different groups of recorded SEN provision, leading to estimating the ATT (Richardson et al., 2023). See Table 4 for the research these causal contrasts are addressing.

Missing data. To deal with missing covariate values (there are no missing exposure data by design) we will use Imputation using Chained Equations (ICE) as part of the bootstrap-based estimation of confidence intervals of point estimates, we will use in each replicant as part of bootstrap imputation (Schomaker & Heumann, 2018). All variables will be used to predict missing data including the exposure and the outcome, and any other variables assumed to be informative of the missing values (Azur et al., 2011).

Sensitivity analyses. We aim to conduct a series of sensitivity analyses to estimate the robustness of our results. Firstly, we will adjust our assignment of recorded SEN provision from year one to year two to account for the administrative time it takes for parents/carers to apply for SEN provision. One of our criteria is that pupils must have data on their EYFSP school readiness tests as this is a major confounding variable; this may restrict our population to those able to take the test. Hence, to account for this non-participation, we will use a missingness indicator to capture the information held in missing the test and avoid excluding those without a record (Groenwold et al., 2012). Furthermore, we suspect there maybe missingness in outcome data, particularly for key stage two scores based upon prior knowledge of systematic teacher strikes; in such cases we will use imputation to estimate these key stage two outcomes using year of testing in the imputation model. Finally, we propose analysing the correlation between recorded child sex (reported by physician in HES) and gender (submitted by parent/carer during school registration in NPD). To understand the validity of our models, we will produce a table of how using either variable impacts our point estimates of the intervention variable only.

Ethics

Permissions to use de-identified data and linked from Hospital Episode Statistics and the National Pupil Database were granted by DfE (DR200604.02B) and NHS Digital (DARS-NIC-381972); consent from patients is not required for HES as the data provided by NHS Digital is pseudo-anonymised and reduces identifiability to researchers; further information on opting out of Hospital Episode Statistics for secondary usage can be found here. Ethical approval for the ECHILD project was granted by the National Research Ethics Service (17/LO/1494), NHS Health Research Authority Research Ethics Committee (20/EE/0180) and UCL Great Ormond Street Institute of Child Health’s Joint Research and Development Office (20PE06).