A Protocol for a Rapid Realist Review of Literature Examining Co-Production in Youth Mental Health Services

Abstract

An overview of internationally published literature on what works for co-production in youth mental health services is missing, despite a practice and policy context strongly recommending this approach. The proposed rapid realist review aims to develop a theory about how and why co-production methods in youth mental health services work (or do not work), for whom, in which contexts, and through what mechanisms. Relevant evidence will be synthesised to develop context–mechanism–outcome (CMO) configurations that can inform policy and practice. Stakeholders will be iteratively involved in the development of these theories (CMO configurations) by engaging an expert panel and youth advisory group. The review results will be reported according to the RAMESES guidelines and are intended to be published in an academic journal. Additionally, a plain English summary will be produced with the support of the youth advisory group.

1. Introduction

Co-production in healthcare is the practice of involving service users in decision-making and service design/delivery as equal partners, for example, through advisory groups, peer support worker roles, or in service design (amongst many other methods). Considerations for further defining co-production are discussed below. It is also widely evidenced that most long-term mental ill health has its onset by adolescence [1]. Therefore, young people are a critical cohort for the co-production of mental health services. A clear overview of the evidence base to understand different models of co-production and what works for the global youth mental health context is unavailable, despite it being increasingly written into policy, clinical guidelines, and legislation that young people should be involved in co-producing mental health services [2,3,4,5,6,7,8,9,10]. The review proposed in this protocol aims to fill this gap.

There is broad consensus that human rights abuses have occurred and continue to occur within mental health services, and that where people seek help or are subject to compulsory treatment in these systems, they frequently encounter harm. A recent report identified nine harmful practices that represent human rights violations that are ongoing within mental health systems: seclusion, restraint, compulsory/coercive treatment, police killings and violence, use of comas, neglect, institutionalisation, racism, and locked wards [11]. Though this report was focused on one locality, the United Nations also states that such human rights violations during treatment and poor support for mental health conditions are seen across mental health systems worldwide [12]. These practices result in an array of harms, including (but not limited to) psychological trauma, reduced life expectancy, and premature deaths [11]. The role of lived experience in shaping and delivering future services has been widely promoted as a possible route to system reform and service transformation to help ensure such abuses do not occur and that people who use mental health services feel the services are appropriately configured to their needs. It has been proposed that the principles and practices of co-production in particular have transformative potential in this process [13,14,15].

An initial scope of the literature was completed to find any other literature reviews on this topic. Five were identified that captured elements related to co-production in youth mental health. However, none used a realist methodology, and only Norton’s 2021 systematic review [16] focused specifically on co-production in youth mental health. Moreover, this review included only two papers (38 total participants), highlighting the paucity of literature available. Despite the recency of this review, Norton [16] indicates that the search strategy will likely have excluded some relevant studies and suggests a different review approach should be attempted for this topic. In response to this, the suitability of a rapid realist review approach is presented below in the Section 3.

Of the four remaining literature reviews, none are focused exclusively on co-production in youth mental health services (these look at policymaking, services for all ages, user involvement and co-production in research). Nevertheless, two of these reviews highlight the lack of research in this field. This is a conclusion seen both in a 2022 scoping review by Yamaguchi et al. [17] into youth participation in mental health policymaking and a 2013 scoping review [13] into co-production across adult and youth mental health. In the same year, Viksveen et al.’s larger systematic review on user involvement (including co-production) in youth mental health was published [18]. Though it included more studies (24), with a total of only 587 participants contributing to them, this review does not challenge the claim of research paucity identified by Norton [16], Yamaguchi et al. [17], and Slay and Stephens [13]. Moreover, many of the studies on user involvement in Viksveen et al.’s [18] review are not investigating co-production but other types of user participation or engagement. The specificity of co-production is of interest to the review proposed here. Finally, a 2023 systematic review of youth engagement in mental health research from McCabe [19] provides some insight into effectively co-producing with this cohort, though these recommendations for research may not be transferable to the context of co-producing within mental health services.

Thus, despite these various relevant reviews, the gap remains for a comprehensive overview of evidence specific to investigating how co-production works in youth mental health services. The rapid realist review proposed in this protocol aims to contribute to this gap by scrutinising existing relevant literature and, from this, answering the question:

For whom and in what circumstances does co-production work in youth mental health services?

This review, which adopts a rapid realist review method, is interested in identifying contexts, generative mechanisms, and outcomes for co-production in youth mental health [20]. To focus on these components, additional sub-research questions are:

-

What are the important contextual factors in understanding co-production in youth mental health services?

-

What mechanisms explain the impact of co-production in youth mental health services?

-

What are the outcomes for service users that result from co-production in youth mental health services?

2. What Is Co-Production in Youth Mental Health Services?

The age range for “youth” varies between studies and settings; herein, this will be defined as aged 10–25. This age range has been selected because the World Economic Forum Global Framework for Youth Mental Health found that 10–25 is in line with the World Health Organisation (WHO) and United Nations definitions of a “young person” [21].

Mental health care is the support, treatment, and prevention of mental health conditions and mental distress/trauma. The WHO definition of “mental health services” will be adopted, which includes mental health care practised within three areas: healthcare services (inpatient mental health and primary care/general hospitals), community mental health services, and beyond the health sector (e.g., schools, social services, and third sector organisations) [1,22].

Literature describing co-production in youth mental health services across the world is of interest to this review (though the search is limited to English-language documents). For this reason, these internationally recognised definitions of youth and mental health services have been selected as they encompass mental health provision globally. Extending beyond mental health institutions by selecting this definition of mental health services reflects the WHO recommendations to continue the work of de-institutionalisation by moving mental healthcare out of hospitals and towards community provision [1]. Research on both statutory and third-sector youth mental health services using these definitions of youth and mental health services will be included in this review.

Co-production is more difficult to define. It has been widely discussed that a consensus over the meaning of co-production is lacking, along with much conflation with distinct engagement practices such as stakeholder consultation [3,23,24,25,26,27]. Furthermore, it has been identified that the contested definition makes the comparison between studies investigating co-production challenging [3]. Meaningful study comparison is evidently required for a literature review. Thus, co-production in youth mental health services will be defined in this review as:

Stakeholders (at a minimum, including young people with lived experience of mental illness aged 10–25 and mental health professionals) working together to improve a youth mental health service. The practices must include service users from the outset of the programme and involve attempts at equal power-sharing/democratisation.

Considerations for Defining Co-Production

The following discussion considers the debates around an agreed definition of co-production and highlights how the above definition, which will be used in the proposed review, was reached.

Despite the problem of definitional slippage of the concept of “co-production” and the problem of cobiquity (a rise of different ‘co’ words describing collaborative practices) [23], there is some agreement in the literature on definitions. Heap et al. [28] identify how different theorists repeatedly conceptualised degrees of participation as ladders or continuums with differences in terminology but which all range from “consumerism” to “democratisation.” On these various scales, service users have little power or tokenistic participation at the bottom (consumerism), and at the top they are in control or share control (democratisation). Ladders of participation have a long history, dating back to Arnstein’s 1969 ladder of citizen participation [29]. When co-production features on these various scales, it is at or towards the top.

Though most definitions of co-production describe stakeholders as equal partners, in practice, equity is very difficult to achieve in youth mental health services (for individuals aged 10–25). This is due to the power differentials between service users and professionals [14] and youth and adults [30]. If no, or very little, service user engagement work reaches the standard required to be considered “perfect” co-production, where participants are equal partners, then the concept’s utility can be called into question. Nonetheless, to apply to the real world of healthcare practice, this review needs to include investigations into a range of “imperfect” co-production interventions without the definition becoming so broad as to include everything service users and staff do together.

Combining two approaches resolves the dilemma of what qualifies as imperfect co-production. Firstly, Robert et al.’s [3] position will be adopted, in which co-production has two key components: “technocratic benefits” and “democratic rationales.” A definition of co-production that captures these two components is offered by Slay and Stephens [13] and will be adopted. By this definition, co-production is:

“a relationship where professionals and citizens share power to plan and deliver support together, recognising that both partners have vital contributions to make in order to improve quality of life for people and communities”

Secondly, a continuum of co-production itself has been proposed in a study by the National Institute for Health Services [31]. Co-production on this continuum is classified on five levels: “conventional, provisional, moderate, committed, or radical,” with increased empowerment and emancipation towards the radical end [31]. “Radical” or “emancipatory” co-production is not always achievable (or even always desired) [31]. Accepting such a scale provides space for imperfect co-production to qualify, as it would fall towards the “conventional” end.

Using the two criteria of power-sharing (or democratic rationales) and quality improvement (or technocratic benefits) alongside the idea of a spectrum of co-production (for example, the continuum “conventional” to “radical”) allows inclusion in the review of co-production that is aiming for shared power and service improvement even when this is not perfectly achieved. Where equal power is sought but not achieved, the programme sits further down a continuum of co-production.

Therefore, by accepting that there is a scale of empowerment within the concept of co-production and applying the two requirements of democratic rationales and quality improvement, each co-production initiative described does not have to be practised perfectly to meet the definition. Having used these criteria to establish if the study is investigating genuine co-production and should be included, the concern for the review is then identifying which factors enable co-production that works effectively.

Many have raised concerns about the impact of co-production, arguing that it can privilege some views [32], unfairly transfer responsibility to service users [33], or perpetuate existing power imbalances within the practised systems [34,35,36]. Moreover, many theorists and practitioners have expressed disquiet about tokenism masquerading as co-production, which is often characterised as a harmful type of participation, on a lower rung of participation ladders/continuums [29,37,38]. However, the research question in this review does not assume a priori that co-production is a good thing. By asking for whom and in what circumstances it works, this review will also investigate for whom and in what circumstances it does not.

3. Methods

3.1. Study Design—Rapid Realist Review Methodology

The chosen method of review must fit the research question that it aims to answer [39]. Though systematic reviews are widely used in healthcare research, such an approach is not appropriate for this study because there is a paucity of studies. When there is little evidence to review, systematic reviews in health research have been found to be frequently inconclusive [40] or to conclude vaguely that an intervention works “‘to some extent’ and ‘sometimes’” [41]. A realist review, on the other hand, first described by Pawson et al. [42], aims to provide practical guidance to practitioners and policymakers on how to alter the context or resources to be more likely to trigger mechanisms that produce the hoped-for outcome for complex programmes [43].

Realist evaluation consists of developing and testing programme theories (hypothetical statements) that describe how a programme works to produce outcomes. In this case, this review will investigate the theory that co-production works for some people in some circumstances in youth mental health services. Initial programme theories are then developed from the literature, often in statements structured “if…then” [44]. The process of extracting data from the literature in this review will consist of identifying the contexts, mechanisms, and outcomes for co-production in youth mental health services and proposing explanatory theories from this data to describe how these interact.

Jagosh [44] summarises these three elements as follows: “(c)ontext is the backdrop of programmes, whereas mechanism is how stakeholders respond to resources. Outcome is measurable impact at the behavioural, clinical, or system level.” More developed programme theory then emerges in the form of context–mechanism–outcome configurations (CMOs). Instead of investigating the co-production programmes themselves, a realist synthesis investigates the proposed theories (CMOs), which offer explanations of what works, for whom, and in what circumstances regarding the programme. The objective is to explain how complex programmes work, or do not, in specific contexts.

Realism is the philosophical position that reality is independent of us but can only be understood through human interpretation and perception [45], often described as sitting between positivism and constructivism. Taking a realist approach to evidence synthesis, therefore, means committing to the view that all data sources on a subject are “potentially fallible, limited, and subjective (socially constructed) by their very nature” [40]. Therefore, causal insights can be found in studies using various methods, provided the data available is rich, scientifically rigorous, and relevant to the subject being scrutinised [46]. Using this approach facilitates broadening the search to include grey literature and small qualitative studies, which are more likely to be participatory or co-produced research and might be excluded from a systematic review.

This approach is supported by arguments made by the service user and survivor movements regarding epistemic justice, which state that literature reviews should include a broad range of materials ast lived experience is just as valid a form of knowledge as research [47,48]. The process enables reviewers to find “nuggets” [46] of wisdom that can provide insight about for whom and in what circumstances co-production works (underlying causal mechanisms). Synthesising such nuggets in this review will be valuable to practitioners and policymakers currently using or promoting co-production who wish to improve practice based on evidence.

A rapid realist review is considered “robust but not comprehensive” [20]. An essential characteristic of a rapid realist review is the role of the expert panel in suggesting relevant papers for inclusion; this can be especially helpful for locating grey literature that contains causal insights [20,49]. This is to mitigate the risk of significant sections of the literature being missed [20].

Rapid reviews can be accelerated by using a reverse chronology quota, limiting the number of databases searched, or working with a librarian to focus the search strategy to find the most relevant literature (and limit the total search results) [50]. This review uses the latter of these three approaches. Although using a narrower search can mean that relevant articles are missed, this will be mitigated here by using expert panel suggestions and through forward and backward citation searching.

This is the first proposed rapid realist review to synthesise evidence on co-production in youth mental health services. Ethical approval is not required. This review is designed to be completed within three to four months. This review has been designed in line with the process outlined by Saul et al. [20] for rapid realist reviews and is described below in five stages.

3.2. Review Stages

3.2.1. Stage 1: Research Question Development and Preliminary Programme Theory

In 2020, the Institute of Mental Health’s Youth Advisory Group at the University of Birmingham (hereafter referred to as the “reference group”) co-wrote a book chapter on youth involvement in mental health service design and delivery in conjunction with researchers in the field [30]. The chapter concluded that regarding youth involvement in mental health services, “We don’t know what works, for whom, in what circumstances, and why” [30], highlighting an area requiring further research to answer and inspiring this review. Using this chapter to identify the review topic ensures that the review is focused on a question that stakeholders are most interested in answering, as the chapter authors consisted of young people with lived experience of mental illness, lived experience researchers, and subject experts.

Following this, the lead reviewer met with the reference group in March 2023 and proposed a research question based on their chapter conclusion. The proposed question was reframed in a realist formulation and focused on co-production: “Does co-production work in youth mental health services? For whom does it work and in what circumstances?”

The group considered this question and prioritised suggested areas of future research (both gathered from research literature and brainstormed by the young people) (see Figure A1, Appendix A). From this prioritising exercise, it became clear that who is involved/engaged in co-production (and who is not) was a priority area for investigation. From this feedback, the question was reframed to centre the ‘who’ aspect, as follows: “For whom and in what circumstances does co-production work in youth mental health services?”

This process is in line with Saul et al.’s [20] recommended steps for a rapid realist review: the area of interest and research question should be identified and refined with stakeholders. See Figure 1 below for planned stakeholder involvement at each review stage. The reference group members will be provided honoraria payments for their time contributed to the project (e.g., attendance at focus group discussions).

Preliminary programme theories have been developed to support the search strategy. For realist reviews, preliminary programme theory is a tentative explanation of the topic under examination, which will be tested by the evidence in the literature and then accepted or rejected. Consultation with stakeholders and the literature is used to identify potential contexts, mechanisms, and existing theories to support the search. This was completed through the prioritising exercise and discussion with the youth reference group in March 2023, which identified priority themes (see Appendix A). A similar workshop was carried out in August 2023 with researchers in the field (see Appendix B). The factors from both workshops, as well as the initial scoping of relevant literature and feedback from the review expert panel, have been collated into a table listing possible contexts, generative mechanisms, and outcomes of interest. Preliminary programme theories (hypotheses) in the form of “if…then” statements have been produced from these by the lead reviewer. This is available in the Supplementary Materials.

3.2.2. Stage 2: Search Strategy

The development of search terms and identification of papers to be included will be conducted as follows: Three key concepts have been identified for the literature search: mental health services, youth, and co-production. In a rapid realist review, search terms are recommended to be identified collaboratively [20]. Synonyms will be identified through initial brainstorming and supplemented by incorporating some search terms utilised in two recent, related reviews [16,19]. A full list of the terms used is included as Supplementary Materials. A University of Birmingham librarian will be consulted to assist with refining the search strategy, and members of the PhD supervisory team (with expertise in the field) will provide feedback on the strategy and terms identified. Subject headings relevant to each database will also be used where databases allow, and appropriate terms/phrases identified in this way will be incorporated into the synonyms list for keyword searching on databases without subject heading functionality.

The inclusion and exclusion criteria that will be employed are outlined in

Table 1. Inclusion and exclusion criteria.

Inclusion and Exclusion Criteria

Inclusion Criteria Published in English Empirical studies and grey literature that meet criteria for relevance, richness, and rigour Studies investigating/papers concerned with: ○ Co-production of youth mental health services (service-initiated) ○ Youth mental health service improvement or (re)design in which the study uses a co-produced research method (research-initiated) NB qualifying co-production requires: Evidence of aspiring for power sharing and quality improvement Including young people with lived experience of mental illness aged 10–25 and mental health professionals from the outset Exclusion Criteria Studies investigating/papers concerned with: ○ Other involvement practices (e.g., peer support only, shared decision-making/collaborative care, consultation/involvement/participation/engagement) ○ Co-production of research investigating youth mental health ○ Commentary without a case study

below. These have been selected to identify peer-reviewed research papers as well as project reports and other grey literature that describe co-production that has substantially taken place within youth mental health services. Much of this work is described outside of the research literature, and realist methodology supports the inclusion of such non-research material, which can provide causal insights [44]. Papers that describe or analyse the co-production that has occurred in youth mental health services will qualify for inclusion. This can include both service-initiated and research-initiated co-production initiatives and programmes. Examples of research-initiated co-production within mental health services are likely to include participatory study designs with the research aim of service improvement or re-design. Regarding research-initiated papers, if young people with lived experience are included from the start with efforts towards shared power and the aim is service improvement, then the paper is considered to be describing the co-production of a youth mental health service and will qualify for inclusion. Robert et al. [3] posit that co-design can be considered a specific activity within co-production; this review will include co-design projects only where there is an implementation of what has been designed in a mental health service.

The definition of co-production provided above (in short, aiming for shared power and stakeholders included from the outset) will be used to screen for whether the level of involvement qualifies. Therefore, papers describing peer support, shared decision-making/collaborative care, consultation, involvement, participation, and engagement, all of which fall outside of this definition of co-production, will be excluded as they do not seek equal power sharing [14,51]. Studies that do not refer to co-production in a youth mental health service will be excluded (for example, those investigating co-production of research in youth mental health or commentary on the subject without a case study). This is because this review seeks information on effective uses of co-production in youth mental health services (with the realist framing “what works?”), and therefore, papers based upon real-world co-production initiatives will be best placed to provide insights.

Title, abstract, and keyword searching will be completed for each synonym list, separated by ‘AND’ operators in eight databases: PsycINFO (Ovid), CINAHL Plus (Ebsco), Medline (Ovid), Web of Science (Core Collection), SCOPUS, Social Policy and Practice (Ovid), ProQuest Dissertations and Theses Global, ASSIA (ProQuest). In addition, backwards and forward citation tracking will be utilised on papers identified for inclusion. No date or study method limits will be applied.

As is usual in a rapid realist review, alongside this database searching, the expert panel will be invited to suggest documents for inclusion; this could include empirical studies in journal articles and also grey literature such as government documents, evaluation reports, or public websites, as these can also provide valid information for the programme theory. Documents suggested by the expert panel will also be assessed against the inclusion and exclusion criteria for selection. From the papers identified for inclusion through the database searches and expert panel recommendations, forward and backward citation searching (pearling) will be employed. Covidence will be utilised to remove duplicates and collate bibliographic records.

3.2.3. Stage 3: Document Selection and Appraisal

Titles and abstracts will be screened, and full-text versions will be sought if the source meets the inclusion criteria. A total of 10% of the sources will be screened by a second reviewer. The technique of screening a 10% portion is recommended when resources do not allow for a second reviewer to screen the complete set of abstracts and titles; this helps to ensure that studies are not being missed or the inclusion/exclusion criteria are not being applied incorrectly [52]. Literature in realist reviews is assessed for richness, scientific rigour, and relevance to the research question [20,43,53], and these criteria are used to identify if a study contributes to a theory and/or testing theory and should therefore be included [53]. The lead reviewer will assess the selected papers for quality against these criteria. Dada et al. [53] recommend that reviewers practise transparency and offer processes and considerations for evaluating richness, rigour, and relevance. This process is described diagrammatically in their paper and will be followed for this review [53]. The scientific rigour will be additionally assessed according to the reliability criteria Wong [54] proposed: if the data collection methods are unclear, then additional sources of data are required to support any part of a programme theory, or the programme theory should be excluded.

3.2.4. Stage 4: Data Extraction, Analysis, and Synthesis (with Theory Refinement)

Data describing characteristics of study/project methods, setting, participants, co-production activities, etc., will be recorded in a spreadsheet in MS Excel.

Separately, data will be extracted to build realist programme theories using an extraction template, aiming to find information that helps identify contextual conditions and mechanisms that facilitate effective co-production in youth mental health services (outcome). Examples of what might be extracted will be identified iteratively during the data extraction. Contextual factors might include the type of healthcare setting, any existing participation structures in the organisation, social norms around hierarchy in the setting, or geographical location. Mechanisms might include the co-production principles applied, the resources available, and how these influence the behaviour of participants. Outcomes might include service improvement, new service designs, the psychosocial impact of participation in a co-production initiative, or who is included/excluded in the process. Further examples can be seen in Supplementary Table S2, which contains the CMO brainstorming data. Once these have been extracted, the aim is to identify key recurrent patterns of contexts and outcomes in the data and to explain these through the mechanisms by which they occurred [55]. This is a retroductive approach that is found in realist evaluations, which uses both inductive and deductive logic as well as explanatory accounts, or “hunches” [56].

First, explanatory accounts (causal statements from the literature) will be extracted. These accounts will then be expressed as “if…then” statements. Once this has been completed for all sources, the complete set of explanatory accounts will be collated; where possible, statements will be combined, and the reviewer will look for patterns (called “demi-regularities” [57]) between the explanatory accounts. Demi-regularities will then be grouped by theme, and a CMO configuration will be developed for each theme. All included documents will be reviewed for other data supporting, refuting, or changing these configurations. Finally, these CMO configurations will be used to develop and refine an initial programme theory (also using elements of the preliminary programme theory from the start of the review).

Data synthesis will include a validation of the findings by content experts and a written report of the findings. The initial programme theories that emerge from the data will be reviewed by the expert panel (via email) and reference group (in an online focus group meeting). This is to ensure that the findings from the literature reflect the experiences and learnings of practitioners and lived experience experts, and to identify any gaps. Based on feedback, the theory may be further refined, confirmed, or refuted. This process of stakeholder review and refinement is how an initial programme theory is tested in this type of review [42]. Once theoretical saturation is satisfied and all agree that further examination of the CMO configurations is unlikely to result in changes, this refined programme theory will be a middle-range theory and thus apply across various contexts. Middle-range theory is described by Jagosh as “theory that is not abstract to the point of being disconnected from the on-the-ground workings of programs, yet not so specific to pertain to one program” [40].

3.2.5. Stage 5: Disseminating the Findings

The review results will be reported according to the Realist and Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) quality and publication standards [43]. This rapid realist review has been registered with PROSPERO (ID number 456623) and is intended to be published in an academic journal. To widen the potential audience, findings will also be available as a plain English summary, including recommendations. The reference group and expert panel will be consulted for suggestions of creative methods for disseminating the findings to reach the broadest possible audience and be encouraged to share outputs in their existing networks.

The review findings and recommendations will shape the subsequent data collection stages of a more comprehensive PhD research project (see Appendix C for more details). A benefit of using a rapid realist review method is that the participants of the expert panel will benefit from experiential learning through the process of contributing to the review, which will impact their practice decisions and consequently effect change through this research [20]. Furthermore, the products of this review will be suitable for guiding policy on co-production in the sector and relevant to stakeholders in co-production in youth mental health.

4. Discussion

This rapid realist review will synthesise evidence on the generative mechanisms and context that influence the reasoning and resources of actors to support effective co-production (outcome) in youth mental health services. This will be relevant to young people and professionals using or intending to use co-production within these settings, as well as to policymakers in the sector. Due to its rapid nature, the review will provide practical knowledge in a short period of time, and its relevance will be ensured by involving stakeholders in the review process to help identify the research question, guide the formation of programme theory, and generate feedback on data analysis. The gathered knowledge on what factors support the success or failure of co-production programmes will also be used in the subsequent stages of this study (see Appendix C for an overview).