Cost-effectiveness decision modelling in social care: exploring the why, the how, and the what next

Introduction

The most that can be expected from any model is that it can supply a useful approximation to reality: All models are wrong; some models are useful (Box et al., 2005).

Social care, also known as ‘long term care’ outside of the UK, provides social work, personal care, protection and social support services to children and adults who are deemed to be in need or at risk due to illness, disability, frailty, or poverty. As well as supporting service users or clients, it can support informal unpaid carers, such as family members of the service user. In the UK, social care receives funding through the statutory (public) sector, however supply is limited and need for care is routinely means tested, resulting in many users self-funding. The voluntary, community, and social enterprise as well as the private sector are important partners for statutory health and social care agencies in providing social care, meaning that public, private, and third sectors are all part of the delivery of social care.

The objectives of adult social care in England, and responsibilities of local authorities in relation to social care, are set out in the Health and Social Care Act (Glover-Thomas, 2013). Key objectives of social care include supporting individuals to remain independent in the community, supporting their dignity, personal choice, and control over their daily lives, as well as supporting social-care related quality of life and safety. More broadly, social care supports the health care system with its role in prevention and reducing health care service use. Alternative outcome measures relevant to social care, such as the Adult Social Care Outcomes Toolkit (ASCOT), ICECAP-O, and ICECAP-A, provide valuable metrics for capturing the broader impacts of social care interventions beyond traditional health outcomes.

The primary purpose of this paper is to identify further research needs in the field of economic evaluation within social care and to provide comprehensive guidance for researchers and practitioners interested in the methodologies of social care interventions. By addressing these areas, the study aims to bridge existing gaps and enhance the robustness and applicability of decision modelling in social care settings.

Cost-effectiveness analysis, the summation of the costs and effects of alternative courses of action, is considered an informative component in the development of National Institute for Health and Care Excellence’s (NICE) guidance related to best social care provision in England (NICE, 2014). By seeking to estimate the value gained from resources used to implement an intervention or policy, cost-effectiveness analysis aims to determine the best alternative use of limited resources, and the benefits that would have been gained from taking the alternative course of action.

The challenges of conducting robust cost-effectiveness analysis in social care, in contrast to the other areas such as health technology assessment, have previously been defined and discussed (Bauer et al., 2021; El-Banna et al., 2021; Suh & Holmes, 2022; Weatherly et al., 2017). The challenges include: defining the objectives of the intervention and services, and relatedly, the complexity of outcomes of interest, the range of stakeholders in the public, private, and third sectors, complexity of interventions, and availability of research and data. These challenges are especially evident when compared to the evaluation of new pharmaceuticals, for which the methodological approaches to conduct such evaluations were originally developed and for which the consideration of health maximisation subject to costs borne by the public healthcare system are considered sufficient (Drummond et al., 2015). Furthermore, reviews of cost-effectiveness analyses of social care interventions (Bauer et al., 2021; Weatherly et al., 2017) reveal that an increasing number are being undertaken in the social care field but with heterogeneity in methodological approaches used, raising the question of limited standardisation of evaluative methods.

Establishing the cost-effectiveness of an intervention, broadly employs one of two approaches with which to generate findings: empirical data analysis from a single study and decision modelling (Drummond et al., 2015). The use of empirical data relies on evidence generated from sources such as clinical trials and observational and routinely collected data. In contrast, in decision modelling the pathways of care are simulated, and multiple data sources used to impute costs and benefits as they are assumed to occur at different decision-making points or key events (Briggs et al., 2006). This method allows extrapolating costs and effects beyond the time-period of an experimental study.

This paper explores why cost-effectiveness decision modelling has the potential to be informative in a social care setting, how it can and has previously been implemented, and what next steps are needed to ensure consistent, robust, and informative models are produced to inform social care decisions. To do so we first introduce decision modelling as an approach, exploring its distinctive features and aims and its conceptual value to informing decision making in social care. Through a purposeful review of existing published models, we consider how decision modelling has been operationalised in social care to date, and finally seek to define a route forward to increase the usefulness and application of decision modelling in social care.

Why cost-effectiveness decision modelling has a role in social care

To evaluate the effectiveness and cost-effectiveness of a commissioning decision it is necessary to understand what the aims and objectives are of all relevant stakeholders and consider how the cost and outcomes of interest to each are impacted by different available commissioning decisions (Drummond et al., 2015) Given the potential for social care interventions to have multiple decision makers, sectors, objectives, and budgets, this can be challenging. Decision modelling can address many of the conceptual and logistical challenges of conducting cost-effectiveness analyses of complex, real-world interventions. Decision, or ‘economic’ modelling, is a broad term to describe the use of mathematical analysis to simplify the complexity of real-world situations. In health economics decision modelling has been defined as the use of such analyses to estimate a series of possible health outcomes and resource use implications which would result from alternative interventions (Briggs et al., 2006).

In practice, decision models are created by conceptualising and simplifying a complex reality in which interventions exist, focussing on the areas where a demonstrable difference exists between the alternative course of action. This simplification focuses on a limited number of definable states in which the individual can exist. These may be health states, e.g. pre-frail, frail, and dead, or service states, e.g. living at home, admitted to care home, admitted to hospital. Movement between these states, and the impact of each mode of care being evaluated on these transitions is then estimated, with the outcomes of interest and resource use implications of each state reported. The model can then be ‘run’ over the time-period considered relevant, and the sum of the outcomes and resource use implications estimated and compared for the completing modes of care.

The approach is commonly implemented to inform the deliberations of a national commissioner of health services where the quality adjusted life years (QALYs) of the treated individual are the primary outcome of interest (Briggs et al., 2006). While a healthcare QALY based model may not be of primary interest in social care commissioning deliberations, a social care QALY based model might do (Stuttard et al., 2021). Decision modelling as an approach is flexible so long as it is possible to conceptualise and estimate the relevant states individuals may occupy, the costs and outcomes associated with their membership, how they transition between them, and the impact of competing modes of care.

Decision modelling approaches can be argued to have five key benefits in informing deliberations in any setting:

In this section we explore the role of each of these elements related to social care decision making.

How has decision modelling previously been implemented in social care

In England, since 1999, NICE has provided national guidance on the promotion of good health and the prevention and treatment of ill health. Under the Health and Social Care Act 2012 NICE was given additional responsibility to develop guidance and quality standards for social care, making it the first health technology agency internationally to do so. As part of its remit NICE developed a reference case for the economic evaluation of interventions with a social care focus, a generalisable specification of the methods considered appropriate (NICE, 2014). The reference case has been summarised elsewhere, a key finding of which is that compared to HTA guidance, economic evaluation methods guidance for use in social care is much less prescriptive and the range of suggested methods much broader (Bauer et al., 2021; Weatherly et al., 2017). Published systematic reviews have shown that over recent years more economic evaluations of social care interventions have been undertaken, but that the scale is still limited (Tinelli et al., 2020; Weatherly et al., 2017).

The latest general NICE reference case provides some guidance regarding the appropriate approach to conducting decision modelling to inform cost effectiveness analysis (NICE, 2014) and there are existing good practice guides when conducting decision modelling informed cost-effectiveness analysis which have focussed on health technology assessment (Briggs et al., 2006). However, the extent to which these are generalisable to social care of implemented is unclear, therefore with the aim of exploring the scale and scope of decision modelling studies in social care we conducted a purposeful review of the literature. The review, conducted between August and November 2023, focussed on published studies, either in peer reviewed journals or research reports, or are listed in the ESSENCE Toolkit (Tinelli et al., 2020).

The ESSENCE Toolkit is a comprehensive repository of economic evaluations and decision models in social care, which enabled us to identify studies directly aligned with our research objectives. It allowed us to focus on decision modelling methods most relevant to social care, particularly in addressing key methodological trends and gaps. Since our goal is to provide insights specific to decision modelling in social care, rather than conducting a broad review of all available research, a purposeful review was sufficient for capturing the most relevant and impactful studies without the need for a full systematic review.

The literature selected are those that met the following criteria:

We are aware that there is also a literature describing the use of agent-based modelling in social care to, for example, predict social care demand (Gostoli & Silverman, 2019; Gostoli & Silverman, 2022) however these do not extend to cost-effectiveness analyses, nor do they evaluate specific interventions and thus for brevity were excluded from this review. However, they are informative when considering available methodological approaches to decision modelling in this setting. Future research should explore integrating agent-based models to capture the multifaceted nature of social care interventions.

This approach identified 55 studies from an initial search of the titles by criteria 1), with eight being considered relevant to the aims of this paper (Bauer et al., 2010; Bauer et al., 2017; Dixon et al., 2014; Mavranezouli et al., 2014; McDaid et al., 2017; Public Health England, 2018; Tong et al., 2017), one of which contained six independent models (McDaid et al., 2017). The purposeful nature of this review implied pragmatic inclusion and exclusion criteria but broadly the aim was to identify economic evaluations of social care interventions which incorporated some element of a clearly described decision model. We extracted data on key aspects such as the type of decision model used, perspective adopted, time horizon, data sources, measures of effect, uncertainty analysis, headline results, and identified challenges. This systematic extraction process ensured consistency and comprehensiveness in capturing relevant information from each study. The majority of the 55 studies identified as economic evaluations did not incorporate a decision modelling element, consisting of estimates of cost and benefits from primary data sources, for example trials. While not a complete description of the published decision models in social care we consider these eight studies to be an indication of the state of play in the discipline sufficient for the aims of this paper. We conducted a quality assessment of the included studies using the Drummond's criteria to evaluate the robustness of their decision modelling approaches. Despite the purposeful nature of this review, incorporating a quality assessment helps mitigate potential biases and provides a more comprehensive understanding of the existing literature. A summary of the models produced is available in Supplementary Table 1 (Refer extended data)

Supplementary Table 1 demonstrates that there is a wide range of approaches taken to modelling in social care, despite the relatively small number of available studies. Approaches range from relatively simple decision trees exploring outcomes over a short time horizon (Bauer et al., 2010) to individual/client level simulations over the lifetime of the individual (Tong et al., 2017). In general, many of the decision models considered a short time horizon, for example a year, focussing on the period of intervention, with most limiting their horizon to five years or less. This was consistent with the model design and nature of the intervention being evaluated, with the case made that the costs and benefits would only occur over the short term, e.g. support at home services (Dixon et al., 2014) and falls prevention (Public Health England, 2018).

The models included evidence from a wide range of sources, primarily the published literature but also evidence from trials (Cottrell et al., 2018; Mavranezouli et al., 2014), observational data (Dixon et al., 2014), and hypothetical values (Dixon et al., 2014) were used. Most model parameters were drawn from published literature or relevant trial data.

While the more complex models reviewed included extensive uncertainty analysis, including probabilistic and scenario-based sensitivity analyses (Bauer et al., 2017; Cottrell et al., 2018; Tong et al., 2017) the majority of the models focussed on a single base-case analysis. Where it was presented, uncertainty was only explored in terms of impact of the headline results of the analysis with no exploration of advanced modelling approaches such as value of information methodologies or distributional cost-effectiveness analysis, methods which seek to estimate the value of investing in addition research and the differential impact of interventions by socio-economic factors respectively (Weatherly et al., 2017). Additionally, distributional cost-effectiveness analysis, a relatively novel approach in health economics, has begun to gain traction and offers a way to account for equity considerations in social care decision-making. Although not widely adopted yet, its potential to capture the distributional impacts of social care interventions should not be overlooked.

Overall, the published decision models demonstrate the lack of availability of a single appropriate reference case to apply when deciding the optimal decision modelling approach, with a wide range of approaches taken to conduct economic evaluation but importantly no clear logic as to why the different approaches were selected. While reference was made to the complexity of social care decision making in some of the studies (e.g. McDaid et al. (2017) and Public Health England (2018)), few attempts were made to reflect this complexity in the decision model, with the focus most often on costs falling on the public sector, specifically NHS and personal social services, and benefits to the intervention recipients. In some cases, a broader perspective was additionally defined as a secondary analysis (Tong et al., 2017), however, the lack of any widely agreed approach to incorporating the wider implications of social care beyond the public sector cost and health implications makes the extension of any decision model challenging. However, incorporating broader economic evaluation methods such as cost-benefit analysis or cost-consequence analysis could provide a more comprehensive understanding of the multifaceted impacts of social care interventions.

Discussion: what is next for decision modelling in social care?

Economic evaluation is increasingly being applied in social care settings to provide valuable insights to inform the effective and cost-effective use of limited resources. Decision modelling represents a valuable tool to synthesise and analyse data for economic evaluation as it facilitates the incorporation of evidence from multiple sources, the extrapolation of evidence over a longer period than may be directly available, comparisons across all relevant interventions, and the characterisation of uncertainty.

Modelling has been employed in social care programs with examples of highly complex models (Tong et al., 2017) and attempts to incorporate the complex set of stakeholders and relevant outcomes evident with social care delivery in England (McDaid et al., 2017). However, as demonstrated by our review, the overall scale of such models is limited, with only eight separate studies being identified as having implemented decision models relevant to a social care setting. There are many potential reasons for this lack of scale, but key challenges facing decision modelling in this setting can be considered in three elements: contextual, methodological, and evidential (Squires et al., 2016).

In addition to the challenges facing economic evaluation of social care more broadly, and discussed elsewhere (Bauer et al., 2021; Weatherly et al., 2017), the contextualisation of the decision problem and care pathway represents a key challenge for decision modelling in this area. The wide variation in the form of intervention and real-world service provision makes summarising the decision problem in a robust way in the form of a decision model much more challenging in social care, with the risk being the issue is over simplified.

While a key strength of decision modelling techniques is the ability to draw information from a range of sources including trial, observational, and expert elicited data, the complexity of the evidence requirement and relatively limited history of data collection in social care settings, limits the opportunity to conduct meaningful decision modelling in this setting. Efforts are, however, being made to better report and make available routine data in social care (e.g. The Catalogue of Social Care Data (CPEC, 2023)) and to increase the level of funding of research for example by NIHR in the UK (NIHR, 2022).

In recent years there has been significant progress made in methodological developments related to conducting economic evaluation in more complex areas of health and social care provision, for example Skivington et al. (2021) provide a framework for developing and evaluating complex interventions and economic considerations are considered core in determining the comparative resource and outcome consequences of the interventions for those people and organisations affected. They suggest that broad economic approaches to evaluation such as cost-consequence analysis or cost-benefit analysis might be relevant to capture the full range of non-health as well as health costs and benefits across different sectors, and NICE (2013) (2014) also supports use of these economic evaluation approaches of complex intervention. Squires et al. (2016) have produced a similar framework when conducting decision modelling applied to public health. However, there are still numerous areas still requiring methodological development including the modelling of impacts across different sectors and spillover effects (e.g. accounting for the network of support including family and friends), the incorporation of social care specific outcome measures into decision modelling, and the relevant opportunity cost threshold in social care.

While these are all issues that have contributed to the limited application of decision modelling in social care it is evident that it is not the fundamental nature of decision modelling to which they apply, rather to the broader challenges of conducting robust evaluations in a social care setting. As the models identified in the pragmatic review demonstrate, the use of decision modelling in social care is both possible and informative to the setting. Modelling is well used in these studies to highlight the potential costs and benefits of interventions and importantly identify where data is lacking (Dixon et al., 2014).

In terms of ‘what next for decision modelling in social care’, our review has demonstrated that a range of economic evaluation methods and modelling approaches are being used, applied to social care. The range of methods used is consistent with NICE methods guidance in social care. As a tool for undertaking cost-effectiveness analysis, models offer a flexible and somewhat creative approach to undertake rigorous analysis, using available data to best serve the needs of the relevant stakeholders. However, while the current flexibility in approaches has ensured relevance of each decision model to the setting in which it is applied, care is needed that this is not to the detriment of comparison across the findings of each model. Furthermore, understanding how local authorities interpret the results of cost-effectiveness analyses is crucial. Local authorities often focus on cost savings and resource allocation, which may influence the prioritization of interventions. Providing clear guidelines and training on interpreting CEA results can help mitigate potential misinterpretations and ensure informed decision-making.

Another area ripe for future development is the inclusion of equity and distributional considerations within decision models. Current approaches largely focus on average outcomes, but there is increasing recognition that social care interventions often benefit different population subgroups in varying ways. Incorporating distributional cost-effectiveness analysis into future models would allow for more nuanced conclusions about how benefits and costs are distributed across different socio-economic groups, ensuring that decision-makers can account for equity when prioritising resources.

Conclusions

George Box’s eponymous statement that ‘all models are wrong; some are useful’ could be argued to never have been as fitting as in the setting of decision modelling in social care, a setting in which complexity and immaturity of data and research combine to limit the ability to create robust decision models. However, as has been shown through the models that do exist in this setting, there is simultaneously significant potential for decision modelling approaches to provide clarity to evaluations in this setting. As increasing research funding (NIHR, 2022), policy interest, and routine data collection develops, decision modelling will represent an important tool in the conducting of economic evaluation.