Stakeholder perspectives on the benefits and challenges of engagement in public health evidence synthesis: a systematic review

Introduction

Engaging stakeholders in health-related systematic reviews can provide many benefits, including improving reviews’ quality, relevance, usefulness and uptake,^1,2 and supplying important contextual insights. There now exists the ACTIVE framework to guide researchers through the process of stakeholder engagement in systematic reviews.³

Terminology is seldom neutral. Whilst we have retained use of the term ‘stakeholder’ due to its common use and wide scope, we are aware that its use is problematic due to colonial connotations.⁴ Alternative terms, such as ‘actor’, ‘interest-holder’ and ‘knowledge user’ have been proposed,^4,5 although these terms have a narrower focus and do not encompass the full range of functions.

In 2012, a seminal paper within outcomes research proposed a 7P model to characterise different stakeholders.⁶ By privileging alliteration over nuanced categories, the framework presents as user-friendly rather than scientifically rigorous. Nevertheless, it offers a useful backdrop to our own exploration ( Table 1).

Table 1 has been adapted for a UK, rather than North American, focus and, within this table, category 1 has been broadened to include producers of the review, rather than just other interested research teams, the categorisation generally resonates within the context of public health evidence synthesis activities.

Notwithstanding this expansive categorisation, literature on the process of stakeholder engagement and its benefits has largely privileged the views of researchers and their funders (category 1), seen in the enhanced relevance and credibility of the review product. Recent years have witnessed increased interest in the perspectives of patients and the public (category 2), and how their involvement can benefit them individually and the public collectively.

As a funded and bespoke evidence synthesis team for the National Institute of Health and Care Research (NIHR) Public Health Research (PHR) Programme, it was our perception that benefits for stakeholders in categories 3 to 6, for example, for providers/practitioners (Category 4), have received very little consideration around the benefits of engagement in evidence synthesis.³ We identified these categories collectively as ‘stakeholders by role’ to differentiate them from those in category 2, who are 'stakeholders (or experts) by experience'. Potentially, omission of this perspective could result in the unique contribution and value of professional experts being underestimated. Conceivably, it could also lead to challenges in promoting the benefits of involvement in the review process and/or in recruiting necessary representation to synthesis projects.

Due to the benefits to researchers listed above, stakeholder engagement in public health evidence synthesis is becoming increasingly common, however there is little broad understanding of stakeholders as people, with motivations, needs and interests in the way that there is with patients and the public. The use of patient and public involvement and engagement is well-established in health research generally and systematic reviews specifically.^7,8,9 Benefits to patients and members of the public include empowerment and gaining knowledge about the topic and methods, however poor communication from researchers, excessive/additional time demands, and low self-esteem and self-doubt can be challenges to involvement among lay people.⁷

To date, there has been little effort to articulate the benefits and positive experiences of stakeholder engagement for these stakeholders themselves, and, aside from the role of public involvement, the stakeholder perspective on public health evidence synthesis engagement is largely missing. We have therefore undertaken an evidence synthesis to identify, assemble and synthesise data from the research literature on the perspectives of professional stakeholders (‘stakeholders by role’) regarding the challenges and benefits of engagement in public health evidence synthesis.

Methods

Results

Study selection and included studies

1085 records were identified through database searches following removal of duplicates, of which 1027 were excluded at title and abstract screening, leaving 58 to be examined at full text. Fifty-seven studies were excluded at full text screening and one study (reported in one paper) was included (see Figure 1).

Figure 1. PRISMA flow diagram of study selection.

Subsequently, CLUSTER searching¹¹ originating from one key included study,¹⁴ using the full-text search capabilities of Google Scholar, scite and LitSense, identified 10 potentially relevant references; six had already been identified from the database searches but four were additional yields.

Studies excluded at full text screening and reasons for exclusion are available from https://doi.org/10.15131/shef.data.31492525. Most commonly, records were excluded because the involvement described was not related to evidence synthesis. The second most frequent reason centred on study design limitations, including review articles where references were simply checked, documents that did not constitute actual studies, study protocols rather than completed research, and publication errata. Additionally, a notable proportion of records were excluded because they were not related to optimising stakeholder involvement or perspectives, while a smaller number failed to examine stakeholder engagement specifically within reviews.

Less common exclusions included non-health-related content, studies focused on patient and public involvement (stakeholders by experience) rather than stakeholders by role, and materials in non-eligible languages. These exclusions highlight the specialised nature of the review’s focus, suggesting that literature on stakeholder engagement in evidence synthesis represents a relatively narrow subset of the broader stakeholder involvement literature.

Discussion

Studies of stakeholder involvement in evidence synthesis are more plentiful, although still scarce when compared with the perspectives of principal investigators/providers or patients and the public (see Table 4). In qualitative research by the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care (EHC) Program,¹⁵ a wide range of stakeholders reported developing “a sense of investment in the process and findings, as opposed to feeling that the results were simply 'announced' to them at the end”.¹⁶ As with findings from our own review, time was a key consideration for systematic reviewers and stakeholders alike.¹⁶ Additionally, review teams should be mindful of the stress and pressure that stakeholders may feel, including pressure to contribute unique insights or serve as representative voices for large populations.¹⁶ However, it is noteworthy that Cottrell’s study specifically excluded any ‘personal benefits’ for stakeholders as outside the scope of their inquiry.

Perspectives of stakeholders categorised by their professional roles or knowledge contributions remain particularly underexplored. Sharma and Bansal offer insights on this dimension, identifying benefits in three key areas for managers operating as research partners in systematic reviews: changes to self, understanding others, and developing new perspectives.¹⁷ Managers reported gaining knowledge on topics, drawing personal satisfaction, experiencing personal growth, and having their assumptions challenged. They valued seeing the topics and their own work in a different light through engagement with researchers. Furthermore, both researchers and managers in Sharma and Bansal’s study developed deeper insights regarding differences in perspectives; managers appreciated hearing multiple evolving opinions throughout projects and systematic research processes, whereas researchers developed nuanced understandings of managers.¹⁷

Haddaway et al. proposed a framework for stakeholder engagement during systematic reviews in environmental management that offers useful ways to conceptualise stakeholder engagement success.¹⁸ Their taxonomy includes three key elements: stakeholders feeling included; stakeholders feeling their opinions are valued; and stakeholders supporting the review. This framework aligns with Lemke et al.’s (2024) three related categories for procedural justice in stakeholder identification: inclusion (who is given a voice); influence (to what extent participation affects outcomes); and information (availability of materials and transparency throughout the process).¹⁹

Findings from our included study,¹⁴ though privileging the researcher perspective, identified key factors for successful stakeholder involvement that relate to these frameworks. Addressing power dynamics through careful workshop constitution and facilitation, providing financial support, undertaking appropriate training, maintaining flexible engagement methods, and allocating sufficient time all contributed to creating conditions where stakeholders could feel included, valued, and supportive of the review process.

Kuhn et al. contributed a useful process diagram for involvement, highlighting the systematic nature of effective stakeholder engagement in marine ecosystem management that may have broader applications to public health contexts.²⁰

Despite these insights, it remains noteworthy that we did not identify any research specifically designed to explore the benefits and challenges of engagement in public health evidence synthesis from the perspectives of the stakeholders themselves. This represents a significant gap in the literature that future research should address, particularly as stakeholder engagement becomes increasingly recognised as an important component of evidence synthesis methodology.

However, we found ourselves looking sideways at the proximal field of environmental science, which is considerably more advanced in documenting stakeholder perspectives. Environmental science has developed robust frameworks for stakeholder engagement that acknowledge the complex socio-political contexts in which environmental decisions are made. Adherents have pioneered approaches that recognise stakeholders not merely as data sources but as partners with legitimate interests in both process and outcomes.¹⁸ This corresponds with a shift from ‘tokenistic’ to ‘empowering’ participation, where stakeholders’ knowledge contributions are valued on their own terms rather than being subordinated to scientific expertise.²¹

The environmental science literature offers valuable insights for public health evidence synthesis. First, environmental researchers have extensively documented how stakeholder engagement enhances the legitimacy of evidence by incorporating diverse forms of knowledge and expertise.²² This recognition that legitimacy stems not only from methodological rigor but also from inclusive processes could strengthen public health evidence synthesis, particularly when addressing complex social determinants of health. Second, environmental science has developed sophisticated approaches to managing stakeholder conflicts and power differentials,²³ acknowledging that stakeholders bring competing values and interests to the table. These approaches could inform nuanced stakeholder engagement strategies in public health evidence synthesis.

The environmental science literature also highlights the importance of reflexivity in stakeholder processes, arguing that researchers should critically examine how their engagement practices constitute particular types of ‘publics’ and ‘stakeholders’, rather than assuming categories are pre-given.²⁴ This reflexivity is largely absent in public health evidence synthesis, where stakeholders are often engaged according to predetermined professional categories without consideration of how these categorisations might influence the knowledge produced. Adopting this reflexive stance could help public health researchers develop equitable engagement practices that recognise stakeholders as complex actors with multiple identities and commitments.

The contrast between environmental science and public health approaches to stakeholder engagement reflects deeper epistemological differences. Environmental science has been influenced by constructivist and post-normal science perspectives that emphasise the value-laden nature of knowledge production.²⁵ These perspectives have led to greater attention to the benefits stakeholders derive from engagement and to the democratic aspects of knowledge co-production. Public health evidence synthesis, despite its commitment to addressing health inequities, has prioritised methodological considerations over democratic ones. This explains, in part, why the perspectives of stakeholders by role remain under-explored in public health evidence synthesis.

Learning from environmental science approaches would require public health evidence synthesis to embrace a more symmetrical relationship between researchers and stakeholders; one that acknowledges both groups as knowledge producers with legitimate insights and interests. This would necessitate attention to the ‘social life’ of evidence²⁶ – the ways in which evidence is shaped by and shapes social relationships. Such approaches would require public health researchers to move beyond seeing stakeholder engagement primarily as a way to enhance the quality or relevance of evidence, and instead to recognise engagement as an inherently political process with implications for whose knowledge counts and whose interests are served.

Conclusions

In our experience as researchers involved in evidence synthesis, it is relatively unusual to encounter topics with such a stark absence of directly relevant evidence. This was a particularly surprising finding given that it was the specific perspective as evidenced by full-text examination, not the overall topic, that remains relatively underexplored. However, evidence synthesis is equally about seeking potential answers and, by casting our attention to the adjacent field of environmental science, we have unearthed some potentially valuable insights. Collectively, these might be useful in presenting benefits of engagement to stakeholders by role or by function and in suggesting potential routes to overcoming possible obstacles to such involvement.

In the interim we have been able to deliver against our subsidiary objective through complementary work, to be published elsewhere, that has interviewed professional and non-public stakeholders (stakeholders by role) associated with our Public Health Review Team outputs over the course of our NIHR contract 2019–2025.

Ethical considerations

Ethical approval was not obtained as this review did not involve obtaining data directly from human participants. Only data from a source already published has been reported on. The review has been reported transparently and in line with PRISMA guidelines.

Consent to participate

Not applicable.

Consent for publication

Not applicable.