NIHR Open Res NIHR Open Research 2633-4402 F1000 Research Limited London, UK 10.3310/nihropenres.13423.1 Research Article Articles Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs)

[version 1; peer review: 1 approved, 2 approved with reservations]

 Jacob Nina Conceptualization Formal Analysis Methodology Project Administration Writing – Original Draft Preparation Writing – Review & Editing 1 Howick Jeremy Conceptualization Formal Analysis Funding Acquisition Investigation Methodology Project Administration Resources Writing – Original Draft Preparation Writing – Review & Editing a 1 2 Svobodova Martina Conceptualization Investigation Methodology Project Administration Resources Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0001-7949-4039 1 Treweek Shaun Conceptualization Formal Analysis Funding Acquisition Methodology Writing – Review & Editing 3 Gillies Katie Conceptualization Funding Acquisition Methodology Writing – Original Draft Preparation Writing – Review & Editing 3 Edwards Adrian Conceptualization Funding Acquisition Methodology Writing – Original Draft Preparation Writing – Review & Editing 4 Bower Peter Conceptualization Funding Acquisition Methodology Writing – Original Draft Preparation Writing – Review & Editing 5 Bostock Jennifer Conceptualization Methodology Writing – Original Draft Preparation Writing – Review & Editing 6 Hood K Funding Acquisition Methodology Supervision Writing – Original Draft Preparation Writing – Review & Editing 1 Centre for Trials Research, Cardiff University, Cardiff, Wales, UK Stoneygate Centre for Empathic Healthcare,, University of Leicester, Leicester, England, UK Health Services Research Unit, University of Aberdeen, Aberdeen, Scotland, UK Division of Population Medicine, Cardiff University, Cardiff, Wales, UK Division of Population Health, Health Services Research & Primary Care, The University of Manchester, Manchester, England, UK Lay Research Advisor, Centre for Trials Research, Cardiff University, Cardiff, Wales, UK jh815@leicester.ac.uk

No competing interests were disclosed.

 21 8 2023 2023 3 42 17 7 2023 Copyright: © 2023 Jacob N et al. 2023 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Background:

The way information about potential benefits and harms of trial is presented within participant information leaflets (PILs) varies widely and may cause unnecessary ‘nocebo’ effects. The Medical Research Council (MRC) funded a project that developed seven principles to reduce this variation. However, guidance has not been produced to facilitate the implementation of the principles. Stakeholder involvement is recommended to optimise the way these principles are disseminated and explained. To co-produce recommendations for developing: (1) user-friendly guidance for users of the principles; and (2) resources that support the implementation of the principles.

 Methods:

We held a co-production workshop with representation from the following professional groups: the Health Research Authority (HRA), research ethics committee members, and trial managers. Two rounds of discussions focused on generating recommendations for guidance and resources that support the implementation of the seven principles. Extensive low inference style ethnographic notes were taken, and the data were analysed thematically using deductive codes. The data was collected on October 14, 2022.

 Results:

25 participants attended a hybrid workshop. Participants recommended that both researchers designing PILs and research ethics committee members should use the principles, and that that they should be simple, mention both benefits and harms explicitly, include examples of visual representations, and provide the evidence base for the principles.

 Conclusions:

We were able to co-produce recommendations for developing and implementing the seven principles within PILs. These recommendations can now be implemented to reduce unexplained variation in the way potential benefits and harms are shared within PILs.

 Plain language summary

Trial participants need to know about the potential benefits and harms of trial interventions to make an informed decision about whether to take part in a clinical trial. Yet the way they are told about these benefits and harms varies widely. We developed seven principles that can reduced this variability. We held a workshop with members of ethics committees and other stakeholders to develop the best way to implement the principles. 25 participants attended the workshop, and we were able to make useful recommendations that will improve the way trial participants are told about risks and benefits of trial treatments.

 Co-production patient involvement harms nocebo placebo research ethics adverse events participant information leaflet Medical Research Council MR/V020706/1 Health Technology Assessment Programme 15/40/05 This project is funded by the Medical Research Council (MRA reference MR/V020706/1); and the Health Technology Assessment Programme (15/40/05). Cardiff University is the sponsor for this research. Neither the funder nor the sponsor was involved in any other aspect of the project, such as the design of the project’s protocol and analysis plan, the collection, and analyses. The funder had no input on the interpretation or publication of the study results. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Introduction

Trial participants need to know about the potential benefits and harms of trial interventions to make an informed decision about whether to take part in a clinical trial 1, 2 . However, research has shown that the way such information is shared within participant information leaflets (PILs) varies widely and is often unbalanced, with potential treatment benefits frequently not mentioned at all 3 . Additionally, overemphasising harms can induce unnecessary information-induced harm (‘nocebo’ effects) 4 , which in turn may adversely affect trial recruitment 5 .

To help reduce this variation and unnecessary nocebo harms, the Medical Research Council funded a project called ‘Developing and Testing Participant Information Leaflets that Inform and Do Not Cause Harm (PrinciPIL)’ 6, 7 . This project included a Delphi process involving a range of stakeholders (participant representatives, ethics committee members, industry representatives, medico-legal experts, applied researchers, research nurses and trial managers) to identify principles that can be used to guide the way in which information about potential benefits and harms of trial interventions is shared within PILs. The principles are as follows:

All potential harms of the intervention should be listed.

The harms should be separated into serious (life threatening, causing permanent damage) and less serious (like a mild headache that goes away quickly).

The fact that not all potential harms are known needs to be explicit.

All potential benefits of the intervention should be listed.

The potential benefits and harms of a clinical trial need to be compared with what happens if the participant does not take part in the trial.

Suitable visual representations are recommended where appropriate to describe potential intervention benefits and harms, such as pictograms of faces.

Information about potential benefits and harms should be presented in proximity (for example, on the same page).

PILs that are informed by the principles above are called PrinciPILs, and PrinciPILs are designed to have three main benefits. First, they will reduce the variability in the way information about the potential benefits and harms of trial treatments is described within PILs 3 . Secondly—and this is a consequence of the above—the principles have the potential to reduce research waste arising from different trial and research ethics committee (institutional review board) teams, developing their own ‘best’ way to share information about the potential benefits and harms of trial treatments. Third, they may improve recruitment rates and reduce nocebo effect harms. To test this, PrinciPILs are being compared with standard PILs in a series of studies within a trial (SWATs) to evaluate whether they influence recruitment rates and trial-related clinical outcomes (especially participant-reported harms, which may be nocebo effects) 8 .

Given their likely benefits, it is important to implement the principles with user-friendly guidance that researchers who design PILs and ethics committees who evaluate them understand and can use. When developing guidance, stakeholder engagement is considered crucial for ensuring priority issues are considered and identified 9 . The importance of involving practitioners and other stakeholders is also considered key to ensuring guidance is adopted, implemented and maintained in the contexts for which it is intended 10, 11 .

Aims

To co-produce recommendations for developing:

user-friendly guidance for users of the principles; and

resources that support the implementation of the principles.

 Methods Ethical statement

In accordance with Health Research Authority (HRA) guidance, this study did not require ethical approval as this study was conducted with professionals to discuss their professional opinions. Attendees (all professionals) provided their verbal consent to publish the results of the workshop at the outset of the workshop. The sponsor of the study agreed that oral consent sufficed, given that no patients were involved in the workshop, no patients data was included, and because the workshop attendees were all acting in their professional capacities and providing their views related to their professional work.

 Patient and public involvement

A patient and public involvement (PPI) representative (JB) was involved in acquiring the funding for this study, question development, research design, and background research. The same PPI representative is involved in our ongoing active dissemination plan for this study.

 Recruitment of the co-production group

Participants were chosen from the group of stakeholders (patient and public representatives, research ethics committee members, industry representatives, medico-legal experts, psychologists, and trial managers) who had registered their interest in designing, evaluating, or using PILs from a previous study 7 . We then used a purposeful sampling strategy to select participants, and contacted them via email. A purposeful strategy helped ensure that group members were drawn from a range of networks whilst also ensuring representation of stakeholders from key organisations who examine PILs (HRA and research ethics committee members) and those who design them (trial managers and other clinical trial representatives). A choice of attending virtually or in person was offered to encourage maximum participation, although most stakeholders attended in person.

 Co-production workshop

In advance of the face-to-face workshop held on October 14 th, 2022, an email was sent to specify that the scope of the workshop was to discuss the guidance required alongside the principles and suggestions for required implementation resources. It was also specified that a discussion of the principles themselves was beyond the scope of the workshop, as they had already been developed with extensive stakeholder input 12 .

Verbal consent to publish the results was obtained at the outset of the workshop. We followed the methodology used in a related project 13 The research team presented an overview of the evidence underpinning the principles (JH). A brief general discussion of the PrinciPIL findings followed. Next, small groups (n=3) were chosen at random to discuss the two questions listed below.

How can we best present the principles so that they can be easily understood and implemented?

What resources (for example, web-based resources) are required to support implementation of the principles?

Iterative rounds of feedback and discussion were conducted after each question, with opportunities to raise conflicting opinions provided. One researcher (NJ) was tasked to take detailed notes during the day. This included minuting of the reports from the small groups. Key points were reflected and summarised, and areas of widespread agreement and of disagreement were noted.

 Analysis

As part of the co-production workshop, extensive low inference style ethnographic notes were taken and then expanded upon following the meeting. Data were analysed thematically using deductive codes that had been identified prior to the workshop, based on the two main questions asked during the workshop. Additional inductive codes relating more broadly to the seven core principles were added where required 14 . For the purpose of this study, the analysis concentrated on: (i) feedback on the two main questions asked of the group; (ii) the stakeholders’ broad observations on the utility of the seven core principles; and (iii) how the feedback can be incorporated into the main principles.

 Results Demographics of the co-production group

We were able to meet our target number of attendees (n=25). The attendees came from the East Midlands, Southeast, London, Yorkshire and the Humber, and the Southwest. There were 19 female attendees (76%), and five (20%) were from non-White backgrounds.

 Presenting principles

In discussing how best to present the principles so that they can be easily understood and implemented, five main themes arose. These focused on the need for (i) the guidance to be easily understood, (ii) clarification of the proper use of appendices, (iii) examples of appropriate visual representation, (iv) provision of the rationale for describing potential benefits, and (v) the need to highlight the evidence base underpinning the principles.

Guidance should be useable by all researchers designing PILs ethics committee members. Whilst some welcomed a single guidance document understandable for both applicants and reviewers, others questioned whether a one-size-fits-all document would be appropriate. For research teams, this could come in the form of a standard operating procedure (SOP) or similar protocol. For research ethics committees, it was recommended that the PrinciPIL guidance should serve as a template for a conversation with research teams. The usual practice for research ethics committees is to accept the risks and benefits presented to them by study teams. It was suggested that the PrinciPIL guidance could thus serve as a change in practice by encouraging a conversation around what has been included and why. It was also suggested that a different approach might be needed for different trials. It was noted that the guidance had to take into consideration that PILs for different trials and trial populations needed to be different. For example, explaining risks and benefits to children is likely to be very different from explaining risks and benefits to adults.

The proper use of appendices needs to be clarified. Appendices are sometimes used to avoid overly lengthy risks and benefits sections in certain trials. The extent to which these are accessed by participants was queried, although it was agreed that their use may sometimes be required to ensure readability.

Examples of visual representation should be provided. The types of visual representation of risks that would be appropriate were not clear to the attendees. The attendees recommended that examples of visual representation be provided. One participant queried the use of smiley faces and suggested using pictograms that reference the text as an alternative approach.

The rationale for including information about potential benefits should be clear. A great deal of discussion centred on whether the potential benefits of trial treatments should be mentioned at all within PILs. Some of the participants claimed that potential benefits are subjective and not known (whereas many harms were considered to be known). In addition, the group thought it would be useful to think about benefits as direct (from the intervention i.e., therapeutic) and indirect (wider participation in research i.e., non-therapeutic) and to be clear about what the benefits listed are.

The evidence base underpinning the principles must be highlighted. Many workshop attendees had not done the suggested background reading and were unaware of the evidence underpinning the principles. Stakeholders therefore considered it useful to be reminded of the evidence base that underpins the seven principles 3, 4, 12 .

 Specific feedback relating to the core principles

The group was invited to make specific suggestions with regards to the guidance for the principles. Below is a summary of the main points of discussion and will be incorporated into the PrinciPIL guidance (see Table 1).

Proposed actions based on results of co-production workshop.
Category Suggestion How we are taking suggestion into account in the development of dissemination of the principles
Presenting principles Guidance should be usable by all ethics committee members and researchers designing PILs On our website we make it clear that the principles are to be used as a helpful guide and that the way and extent to which they are implemented will vary from trial to trial depending on several factors.
The proper use of appendices needs to be clarified We have taken this into account by making it clear in the PrinciPILs that (a) not all potential harms are known, and (where applicable) that (b) a complete list of known potential harms is contained in an appendix and (c) that additional information can be obtained from trial team.
Examples of visual representation should be provided The way we present potential benefits and potential risks includes visual elements such as contrasting colour. On our website, we have also listed the types of visual representations and provided references.
The rationale for including information about potential benefits should be clear We have written a paper citing regulations from the UK, US, and EU where it is clearly stated that mentioning potential benefits of a trial intervention (where they exist) is required 15– 17 .
The evidence-base underpinning the principles must be highlighted On our website we have included a description of our extensive background studies.
Specific feedback on principles The source of potential risks should be made clear We have included this in our guidance document.
An individualised approach needs to be adopted when separating the harms into serious and less serious On our website we make it clear that the principles are to be used as a helpful guide and that the way and extent to which they are implemented will vary from trial to trial depending on several factors.
The sources of potential benefits need to be made explicit. On our website, we have added an explanation of where information about potential benefits is likely to be found.
The answer to the question of what the risks of not taking part in the study requires elaboration. We recommend that, where applicable, the following standard statement be added to the PrinciPILs: ‘if you do not come into this trial, you will get standard care. To learn more about the risks and benefits of standard care please ask the trial team or your physician.’
Resources for implementing principles Using resources, training, webinar, and a SOP amendment for CTUs We have developed several resources including a website, a video, and recommendations for amending SOPs. For all of these, take measures to discourage slavish adherence to principles that does not account for the need for adaptation to specific trials and trial populations.
Keep it simple We developed infographics for the principles that are the key features of our website.

Abbreviations: SOP=standard operating procedure; CTU=clinical trials unit

All potential harms should be listed: The group agreed with the importance of this principle, especially because ethics committee members are seldom qualified in the disciplines they review. For similar reasons, it was suggested that accompanying guidance should include information on appropriate sources of information for non-drug studies. For example, Cancer Research UK (CRUK) has specific support groups to ensure that a variety of different pathways of information are sourced. It was suggested that an appendix or link should be used for risks over a specified amount.

The harms should be separated into serious (life threatening, causing permanent damage) and less serious (like a mild headache that goes away quickly): Participants agreed with this principle and noted that an individualised approach needs to be taken during additional consent conversations.

The fact that not all potential harms are known needs to be clear: No suggestions were made regarding guidance related to this principle.

All potential benefits of the intervention should be listed: It was suggested that clear guidance was required to define what constitutes a benefit and where the evidence for the benefit came from. In addition, participants suggested that direct potential benefits (that might arise from the intervention) be delineated from indirect benefits (wider participation in research).

Potential benefits and harms of a clinical trial need to be compared with what happens if the participant does not take part in the trial: A suggestion was made to present this information such as ‘Consequences of taking part and the accompanying risks and benefits.’

Suitable visual representations are recommended where appropriate to describe potential intervention benefits and harms, such as pictograms of faces: No additional suggestions were made regarding this principle. It was reiterated that examples would be useful.

Information about potential benefits and harms should not be presented apart by one or more pages: There was unanimous agreement that this was a good idea, and there were no suggestions for guidance related to this principle.

 Resources for implementing the principles

When discussing how to support the implementation of the principles, several suggestions were made, including the use of resources such as webinars and SOPs. It was also recommended that we keep things simple, with one participant stating that a simple list of the seven principles may suffice. We have described these below.

Use of resources/training/webinars/SOPs for clinical trials units. Participants noted that appropriate implementation of resources is relative to the target audience. For example, researchers need more detailed instructions, whereas ethics committees may require exemplars. When developing resources, it would be useful to test these out with different research ethics committees. Several attendees noted that the implementation resources should not encourage slavish and unthinking adherence to principles. It was suggested that this could be achieved with worked examples.

Keep it simple. Participants agreed that a strength of the principles is their brevity. It was even suggested that a single-page explanation of the seven principles could suffice.

 Implementing the suggestions into the PrinciPIL project

We implemented all the suggestions made at the workshop to improve the way the principles are disseminated and explained (see Table 1).

 Wider issues

Whilst the focus of the workshop was guidance and implementation, several wider points were raised during the workshop. We agreed to make a note of these for future research, and they are described below:

Importance of the broader consent procedure: The PrinciPIL approach is focussed upon the written communication around consent practices. It was noted that oral communications surrounding consent are at least as important 18 . Relatedly, the trial participants’ understanding of likelihood and risk was likely to be variable, and an individualised approach needs to be taken during consent conversations.

Are all harms and benefits equal? It was thought that it would be beneficial to think beyond the potential benefits and harms of the intervention itself to more general considerations, including time taken, increased monitoring and jumping queues. The potential to consider benefits to organisations (such as research organisations that generate income from research and pharmacological companies) was also raised, although no consensus was achieved on this point.

The legality of mentioning potential benefits: A vocal minority of participants stated that European guidance claimed to recommend against mentioning potential trial treatment benefits within PILs. This was disputed in the workshop and was subsequently found to be false. The European Union clinical trials regulations state that mentioning benefits is required:

Greater definition about what constitutes a risk:The way risks (of trial treatments) are defined varies widely, and there is no consensus around the best way to define or present them 19 . There was general agreement that natural frequencies should be used (whole numbers rather than percentages), and this needs to be made clear in the guidance.

Rationale for taking part in the trial. Some of the attendees considered it essential for the PIL to answer to the question ‘Why should I take part in this trial and what will happen if I don’t?’ at the beginning of the PIL. As this goes beyond the scope of our current project, we did not incorporate it into our plans and instead made note of it for future research.

 Discussion Summary of results and general interpretation

We were able to co-produce several recommendations for PIL guidance and resources to support the implementation of the seven principles that can harmonise the way in which information about potential intervention benefits and harms is shared within PILs. The suggestions included the following: ensuring that guidance was useable by both research ethics committees and researchers; highlighting the evidence base underpinning the principles; the need for clarity regarding the use of appendices; the need for clarity regarding appropriate visual representation; and the need for greater clarity regarding the discussion of potential benefits. Overall, the group considered the simplicity of the principles to be their strength and, as a result, that minimal additional resources were required to implement them.

 Comparison with related research

The need to reduce variability in the application of research ethics principles has been recognized 20 , and calls have been made to harmonize research ethics guidance 6, 21 . Unfortunately, and with few exceptions 22 , guidance for sharing potential trial intervention benefits and harms is under-researched 7 . By providing suggestions that can reduce the variability in the way in which potential trial intervention benefits and harms can be shared within PILs, our study addresses the identified need to harmonize research ethics guidance.

 Strengths and weaknesses of the study

A limitation of our co-production workshop was that the attendees had not done the required background reading outlining evidence that underpinned the seven principles. This led to tangential discussions, especially about the need to mention potential benefits, which threatened to undermine the groups’ confidence in the value of the principles.

Future research should evaluate the extent to which the guidance and resources we produce are used and able to reduce the variability in the way potential benefits and harms are described within PILs. In addition, our methodology can now be expanded. Whereas our principles were applicable to written materials, we can now develop parallel principles that can inform verbal discussions of the potential benefits and harms of trial treatments. Additionally, there is a need to overcome the mistaken belief that mentioning potential trial/intervention benefits within PILs is unlawful.

 Conclusion

Stakeholders were able to co-produce suggestions for guidance and resources for implementing the seven principles that will harmonise the way information about potential trial intervention benefits and harms are described within PILs. These principles have the potential to help trial teams and research ethics committees consider what risks and benefits should be included, reduce variation in the way these are communicated and reduce research waste arising from both trial teams and research ethics committees having to develop their own optimal way to share this information. The recommendations can now be implemented.

 Data availability Underlying data

The workshop records used in this study are restricted to ensure the anonymity of the participants due to the small sample size. Ethical approval and consent was obtained on the basis that data would remain anonymous. Preserving anonymity requires that additional data not be shared. All unrestricted data is available in the manuscript, any additional data sharing would de-anonymise the sample. To request access to the restricted data, please contact Professor Jeremy Howick ( jh815@leicester.ac.uk).

 Faden RR Beauchamp TL King NMP : A history and theory of informed consent.New York; Oxford: Oxford University Press,1986. Reference Source Howick J : Unethical informed consent caused by overlooking poorly measured nocebo effects. J Med Ethics. 2021;47(9):590–594. 32063581 10.1136/medethics-2019-105903 Kirby N Shepherd V Howick J : Nocebo effects and participant information leaflets: evaluating information provided on adverse effects in UK clinical trials. Trials. 2020;21(1): 658. 32680561 10.1186/s13063-020-04591-w 7368797 Howick J Webster R Kirby N : Rapid overview of systematic reviews of nocebo effects reported by patients taking placebos in clinical trials. Trials. 2018;19(1): 674. 30526685 10.1186/s13063-018-3042-4 6288933 Treweek S Pitkethly M Cook J : Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;2(2): MR000013. 29468635 10.1002/14651858.MR000013.pub6 7078793 Centre for Trials Research: PrinciPILs Wales. Cardiff University, 2021; accessed August 19 2022. Reference Source Svobodova M Jacob N Hood K : Developing principles for sharing information about potential trial intervention benefits and harms with patients: report of a modified Delphi survey. Trials. 2022;23(1): 863. 36209242 10.1186/s13063-022-06780-1 9548137 Howick J Svobodova M Treweek S : Patient reported outcomes and recruitment rates following the introduction of principled patient information leaflets (PrinciPILs): Protocol for a meta-analysis [version 1; peer review: awaiting peer review]. NIHR Open Res. in progress,2023;3:29. 10.3310/nihropenres.13420.1 Petkovic J Riddle A Akl EA : Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation. Syst Rev. 2020;9(1): 21. 32007104 10.1186/s13643-020-1272-5 6995157 Carroll C : Qualitative evidence synthesis to improve implementation of clinical guidelines. BMJ. 2017;356:j80. 28093384 10.1136/bmj.j80 Schunemann F Meerpohl JJ Schwingshackl L : [Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise]. Z Evid Fortbild Qual Gesundhwes. 2021;163:76–84. 34023244 10.1016/j.zefq.2021.01.009 Svobodova M Hale R Hood K : Developing Core Principles for Sharing Information about Potential Intervention Benefits and Harms in Patient Information Leaflets Using a Modified Delphi Survey. Open Science Framework. 2021. 10.31219/osf.io/upnf4 Bruhn H Campbell M Entwistle V : What, how, when and who of trial results summaries for trial participants: stakeholder-informed guidance from the RECAP project. BMJ Open. 2022;12(3): e057019. 35338065 10.1136/bmjopen-2021-057019 8961145 Braun V Clarke V : Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. 10.1191/1478088706qp063oa EUR-Lex: Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC Text with EEA relevance. 2022; accessed 6 January 2023. Reference Source Legislation.gov.uk: The Medicines for Human Use (Clinical Trials) Regulations 2004.London: The National Archives; 2004; accessed 6 January 2023. Reference Source Office for Human Research Protections: 2018 Requirements (2018 Common Rule). Rockville MD: Department of Health and Human Services (HHS),2018; accessed 6 January 2023. Reference Source Koyfman SA Reddy CA Hizlan S : Informed consent conversations and documents: A quantitative comparison. Cancer. 2016;122(3):464–9. 26505269 10.1002/cncr.29759 4724216 Coyle M Gillies K : A systematic review of risk communication in clinical trials: How does it influence decisions to participate and what are the best methods to improve understanding in a trial context? PLoS One. 2020;15(11): e0242239. 33196663 10.1371/journal.pone.0242239 7668608 Goodyear-Smith F Lobb B Davies G : International variation in ethics committee requirements: comparisons across five Westernised nations. BMC Med Ethics. 2002;3: E2. 11964190 10.1186/1472-6939-3-2 111059 Independent Review of Research Bureaucracy Final Report.In: Department for Business EISaURaI, ed.: Gov.uk,2022. Reference Source Schwartz LM Woloshin S Welch HG : Using a drug facts box to communicate drug benefits and harms: two randomized trials. Ann Intern Med. 2009;150(8):516–27. 19221371 10.7326/0003-4819-150-8-200904210-00106 10.3310/nihropenres.14557.r30610 Reviewer response for version 1 Nair Gonasagrie (Lulu) 1 Referee https://orcid.org/0000-0001-9005-2442 Desmond Tutu Health Foundation, Mowbray, South Africa

Competing interests: No competing interests were disclosed.

 16 9 2024 Copyright: © 2024 Nair G( 2024 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

This is a very useful study and will be of relevance to those who develop PILs and for ethics committees who review them. 

The work is clearly presented. However, the literature cited appears to be restricted to the developed world context. More and more frequently clinical trials are now multisite, spanning both developed and developing countries. To make the recommendations generalizable and globally relevant, perhaps views of stakeholders/experts from developing countries could have been included. Cultural beliefs and practices play a major role in adoption of or adherence to interventions under investigation in trials. This is especially. true for prevention studies/trials among healthy volunteers (e.g non adherence to study product among young women in HIV prevention trials in SSA Africa). 

The co-production workshop is an appropriate design to solicit feedback on the development of guidance and resources for implementing the 7 principles. However, more information is required on the characteristics of the workshop participants: Were community members or community advisory board members included?  Noted also that participants were allowed to join online and it would have been feasible to allow participants from developing countries to join, making the feedback more broadly relevant. 

The justification of not making source data available to preserve anonymity is acceptable but the researchers could have considered including comments from participants made during the discussions to support the conclusions. For example, what were thoughts on what constitutes a benefit? Should "jumping the queue" or receiving a service at no cost such as contraceptive access or a screening procedure be considered a benefit. Or should a benefit be restricted to that of using/adhering to the experimental drug. 

Did any of the discussion focus on the access to resources/guidance tools in different settings and how likely busy ethics committee members who are often volunteers would seek out these resources. 

I agree that the strength is the simplicity of the principles and look forward to seeing the accompanying guidelines and resources.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

No source data required

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

HIV and TB treatment and prevention clinical trials. Research Ethics. I have a keen interest in the informed consent process and am currently a member of a research ethics committee.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

 10.3310/nihropenres.14557.r30404 Reviewer response for version 1 Nicholls Stuart 1 Referee https://orcid.org/0000-0003-0485-9069 Ottawa Hospital Research Institute, Ottawa, Canada

Competing interests: Co-author Gillies is a co-applicant with me on a recent submitted grant proposal in another area related to clinical trial design.

 25 10 2023 Copyright: © 2023 Nicholls S 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

Co-production of guidance and resources to implement principled participant information leaflets (PrinciPILs)

In the present manuscript the authors provide an overview of a workshop undertaken to inform the development of guidance documents for the implementation of the PrinciPIL principles. Improving the consent process to ensure potential participants can make informed decisions is a key ethical marker of research and work to improve this process is important. I do, however, have some specific comments for consideration (listed under the section headings) as well as several larger comments for the authors to consider. In terms of the larger comments:

The workshop is focused on developing user friendly guidance to improve the use and evaluation of PILs. The Principles themselves are framed in terms of impact on trials: Yet data is not presented as to whether PILs designed in accordance with the PrinciPILs actually achieve the proposed benefits: does the PrinciPIL designed PIL reduce nocebo effects, for example? Should one not have evidence that adherence to the 7 principles does indeed have benefits? Seeking to improve uptake or adherence, absent evidence of benefits (or lack of harms) seems pre-emptive and it is notable that this was not raised as a theme (even though evidence of benefits to be listed in the PIL was).

A second aspect I found challenging was that the term ‘user-friendly’ is vague. How does one define or evaluate the ‘user-friendliness’ of the guidance? Was this framed in a specific way for participants? It is relevant as it could relate to language, visuals, format, as well as accessibility. Perhaps the notion of user friendly could be expanded upon or greater specificity in the objectives offered.

Finally, can the authors expand more on their definition of co-production? The term is used throughout but the work presented seems more akin to a consultation, unless the individuals engaged are/were also involved in the actions taken (as detailed in the 3rd column of Table 1). If a definition could be provided, along with how the study and future work meets the definition, that would be helpful.

 METHODS

While the paper discusses purposeful sampling, this seems to only reflect on role. Were other  experiential or demographic characteristics used to inform sampling? Was there any representation, for example, from people with design experience or with expertise in knowledge translation/implementation or other skills that could provide input on best approaches to uptake of information?

ANALYSIS

What deductive codes were used? How do the final themes relate to the deductive coding vs indictive coding?

RESULTS

Given the noted fact that many had not done the background reading on the evidence base for the principles; to what extent can these views be considered informed or deliberated vs initial or reactionary?

I am also not completely sure on the goal of the analysis, which again goes back to the relatively broad aim. In some parts the results seem to reflect a desire to share the range of issues participants deemed relevant, yet in other areas it is more focused, and in some cases quasi-quantitative in the way that there is reference to “unanimous agreement” or a single suggestion being made.

Some of the proposals seem somewhat circular to the questions posed. For example, a key question posed was “How can we best present the principles so that they can be easily understood and implemented?” and a theme in response is to make them useable by REC members and researchers. This seems to go without saying doesn’t it? One wouldn’t make them unusable. It seems to me that the more important question is HOW can they be made useable to researchers and how can they be made useable to RECs. Put the other way, it would seem important to identify what might be barriers to their use?

The theme of having a rationale for including benefits is interesting as it seems to in part be more about refining the principle itself (e.g. the specification as direct and indirect benefits) as opposed to a way to present the principle so it can be understood. This is consistent with the serious/less serious specification of harms and how that is incorporated into the principles themselves.

In addition, there seems to be some overlap between the theme relating to the rationale for including benefits, and the specific comments about listing the potential benefits of an intervention (both for example, discuss direct and indirect benefits). Perhaps these sets of results could be merged rather than duplicated.

I am not clear on the concerns about “slavish and unthinking adherence to principles.” Is that not what is exactly desired – that these principles are universally adopted? If not, would that not lead to the variation in content that is lamented in the previous review and the introduction? Indeed, the fact that they are principles and not actions seems to require consideration and thought about how to they are applied in practice. Or was it the case that the comment reflects concern for a focus solely on these principles to the detriment of considering other important information? Perhaps clarifying this or offering perspective would be helpful.

There is also variation in the results reporting. For some comments the results are provided solely as description: e.g. a topic was raised. In other areas of the results there is commentary made on the issue (e.g. the legality of mentioning potential benefits). Some are highlighted but then explained as not relevant, so I wasn’t sure of the rationale for raising them in the manuscript results (as opposed to reporting them as issues raised and then addressing them in the discussion as to whether they are misinformed).

DISCUSSION

The discussion would benefit from greater integration with the literature. While I realise the focus here is on the suggestions around approaches that can be made to improve the uptake of the PrinciPILs principles there has, for example, been lots of work in the area of participant information leaflets and especially in relation to required content (e.g the US regulations stipulate core components: https://www.hhs.gov/ohrp/regulations-and-policy/guidance/faq/informed-consent/index.html ). This is built into regulations, so it would be useful to contrast the regulatory approach to the integration of content, and this principled approach. Similarly, reviewing the literature relating to the topics raised would be helpful – for example, the visual representation of risks.

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Research ethics, patient-oriented research

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

 10.3310/nihropenres.14557.r30351 Reviewer response for version 1 Fleming Melanie 1 Referee Robinson Barbara 1 Co-referee University of Oxford, Oxford, England, UK

Competing interests: No competing interests were disclosed.

 15 9 2023 Copyright: © 2023 Fleming M and Robinson B 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

This qualitative study aimed to generate and collate suggestions for guidelines of use for a set of principles which were created to help standardise the way in which harms and benefits are presented in participant information leaflets (PILs). Stakeholders across different professional research settings were invited to a workshop to discuss recommendations of guidance based on the principles, and these recommendations were thematically presented.

The paper is well written, and does a good job of highlighting the importance and need of having consistent and accurate guidelines that can be used by research and ethics teams to implement the principles laid out, and benefits from considering the viewpoints of a diverse range of stakeholders.

I have several minor comments/suggestions for consideration:

Due to the workshop building on the previous work in which the principles were generated, the narrative within the introduction was a little confusing in terms of what this study was doing, and what had already been done (e.g. when mentioning the Delphi process, I initially thought that was in relation to the current study, rather than the principle generation). The introduction would benefit from some rewording to clarify what was past and what is the current work. 

The introduction also presents the SWATS, although this is useful and relevant to the work, this information would be better placed in the discussion section of the paper as it seems like “next steps” rather than work leading up to this part of the project. 

The methods note that there was a “purposeful sampling strategy” employed. It would be useful to include more detail as to which criteria was used for this sampling. It is currently unclear whether patient representatives were included or not, as the “patient and public involvement” subsection implies that they were not involved in this workshop, but the “recruitment of the co-production group” does mention patient and public representatives as part of the group of stakeholders from which participants were chosen for this study.

The authors note that many of the workshop attendees did not read the background reading, however there was no description of the background reading provided. Please could details of the suggested reading be included in the methods section.

Under the ‘ wider issues’ section of the results, point 2 mentions benefits/harms beyond the intervention itself. However, this point is somewhat covered in the ‘ specific feedback relating to core principles’ section already, and I recommend that this therefore be incorporated into this section (specifically points 2 and 4) rather than being separate.

The conclusion points to the guidance being applicable and useable by both researchers creating PILs and ethics review boards. However, given some of the feedback provided in the results (specifically on pg 4: “ Whilst some welcomed a single guidance document understandable for both applicants and reviewers, others questioned whether a one-size-fits-all document would be appropriate.”), I am not wholly convinced by this conclusion, and further discussion of this point may be more apt within the paper.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

Partly

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Research involving human participants, including clinical trials. Ethical oversight. Experience in patient and public involvement in research.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.