Towards Youth Mental Health System Reform: An Evaluation of Participatory Systems Modelling in the Australian Capital Territory

Abstract

Australia’s mental health system is failing young people. Calls for accountability, strategic long-term policy planning, and regional leadership have been identified as solutions to guide mental health reform. Developing system dynamics models using a participatory approach (participatory systems modelling (PSM)) is recognized as a useful method that can support decision-making for strategic reform. This paper reports evaluation findings of a youth mental health PSM process conducted in the Australian Capital Territory (ACT). Baseline and follow-up mixed-methods evaluation data were collected in 2022 across diverse stakeholder groups to investigate the feasibility, value, impact, and sustainability of PSM. Although youth mental health system reform was viewed as desirable and a necessity across all stakeholder groups, shared perceptions of disabling powerless was observed regarding their ability to influence current decision-making processes to improve the youth mental health system. This suggests greater accountability is required to support systemic reform in youth mental health. PSM offers promise in improving transparency and accountability of decision-making for youth mental health, as exemplified in the ACT. However, more support and time are required to facilitate transformational change. Future research should investigate empowerment strategies to complement the implementation of findings from dynamic models developed through PSM, as well as the effectiveness of regional youth mental health policy decision-making supported by systems modelling.

1. Introduction

Australia is one of the wealthiest nations, ranking third among the world’s largest economies in 2022 [1], and one of only seven economies recognised as high-income in the Western Pacific region by the World Bank Group for the current 2023 fiscal year [2]. Despite no shortage of wealth by global standards, Australia’s mental health system is failing to meet the needs of those with mental ill-health, resulting in at least 55 high-profile public inquiries between 1991 and 2021 [3]. As there is no obligation that the recommendations delivered by these resource-intensive inquiries are implemented, many remain unfulfilled [3,4]. Australia’s mental health system has consequently been described as a “failure in need of treatment” [5], faring poorly in comparison with other countries in relation to important systemic markers including suicide and hospital readmissions [6,7], leaving many to wonder why the system is so inadequate for such a wealthy nation.

The youth mental health system is particularly in crisis, with concerns increasing due to recent events that continue to undermine the mental health and well-being of young people, such as the impacts of COVID-19, extreme weather events causing disruption to education and employment, and economic instability [8]. A projected 1,093,000 young Australians experienced mental ill-health within a 12-month period in 2020 [8], costing approximately $AUD2 billion in health care expenditure for those under the age of 25 [9]. A substantial amount of mental health care costs for children and adolescents are borne by individuals and their families, with out-of-pocket costs for subsidised services and medicines estimated to be 16% [10]. Out-of-pocket costs are likely to be much higher for non-subsidised services and medicines [11].

Youth mental health system reform is thus critical. Not only are significant investments needed to restore the most productive years of life [12], but performance and accountability mechanisms are required to ensure funding is better prioritised and, importantly, its outcomes assessed [6,13]. Accountability in mental health, however, has been described as “outcome-blind”, and failure to manage youth mental health challenges has profound and lifelong consequences for young people and their families, communities, and the economy [14,15]. Thus, continuing to invest in existing or new programs with minimal understanding of their potential impact within an already fragmented system will not propel necessary changes to deliver substantial reductions in mental ill-health in young people [15], and new solutions are required to achieve genuine youth mental health system reform.

Increased accountability, regionalisation towards strategic long-term policy planning, and strengthened regional leadership have been identified as possible solutions to guide mental health system reform [6,15,16]. Decentralised planning processes can provide regional communities information, incentives, and accountabilities to ensure that local decision makers deliver improved outcomes [6]. Achieving accountability and integrated regional planning was first proposed as a priority in 1992 in the First National Mental Health Plan, and remains the top priority area on the current Fifth National Mental Health Plan [17]. Most recently, the Productivity Commission’s Inquiry report on mental health has noted regional planning to be “patchy”, with a lack of accountability and strategy undermining the impact on mental health outcomes [6,18,19,20]. Youth mental health system reform also requires the consideration of broader social and economic factors, such as housing and employment, which impact youth mental health outcomes [6,21].

Systems modelling via a participatory approach (referred to as participatory systems modelling (PSM)) has been recognised as a useful method to support complex system reform, particularly in the field of environmental sciences [22]. This approach has the potential to increase capacity to support youth mental health reform by offering sophisticated forecasting and decision support tools to prospectively simulate mental health reform options through the consideration of complex and interconnected areas of health, social, and economic systems [21,23]. PSM can also improve accountability through transparent tracking of such mental health reform options [16,24]. Importantly, the PSM process can strengthen a multi-sectoral response to youth mental health reform by inviting diverse sectors to be part of the participatory process. The inclusion of diverse stakeholders can also improve the accountability of regional and national leaders to make more strategic long-term decisions on how to best invest limited funding to improve multi-sectoral system coordination and efficiency. This approach aims to deliver the greatest benefit and better protect the mental health and well-being of young people [23,25].

A team of multidisciplinary researchers at the University of Sydney’s Brain and Mind Centre partnered with the Office for Mental Health and Wellbeing (the Office) in 2022 with the aim to strengthen regional decision-making capacity and accountability for youth mental health in the Australian Capital Territory (ACT). Through a PSM process, stakeholders [26], including community support professionals, front-line health professionals, health administrators, and young people with lived experiences from diverse sectors and organisations, were invited to contribute to the development and customisation of a system dynamics model (the model) to the regional context of the ACT. Further information regarding engagement with the Office is described in Box 1.

Box 1. Summary of engaging in a PSM process with the Office.

The Office was established in 2018 by the ACT government as a response to Australia’s current Fifth National Mental Health Plan, which highlights the need for strengthened regional mental health systems planning [27]. The primary function of the Office is to act as a catalyst for change to improve the mental health and well-being of the whole ACT population [27]. This includes strengthening youth mental health systems, which have been recognised as a priority area since its establishment [28], with ACT having the second highest rate of suicide deaths per 100,000 in Australian children and adolescents aged 5–17 between 2017–2021 [29]. The process of developing the technical model for the ACT, including key data insights, is described elsewhere [23,25]. In summary, the model was developed to provide regional leaders with a sophisticated forecasting and strategic decision-support tool to inform investments on how to achieve the greatest population outcomes for young people in the ACT. This model is delivered via an online platform which can simulate real-time hypothetical “what-if” scenarios of likely health, social, and economic impacts of individual (e.g., first scenario in Figure 1) and/or a combination of programs and services (e.g., second scenario in Figure 1).

Figure 1. The ACT system dynamics model interface aims to allow end-users to ask “what if” questions and determine the most impactful strategic investments.

Figure 1. The ACT system dynamics model interface aims to allow end-users to ask “what if” questions and determine the most impactful strategic investments.

Objective

The participatory development of a system dynamics model for the ACT aims to support more strategic long-term youth mental health policy planning and regional leadership—both of which have been identified as potential solutions to guide mental health system reform [15,16]. However, no literature to date has comprehensively evaluated the adoption of PSM to support youth mental health. Additionally, just as previous mental health inquiries in Australia have sought to learn from the personal stories of individuals who have experienced poor-quality care [3], it is critical that young people with lived experience of mental ill-health challenges, service use, periods of healing, and/or experience supporting someone through such challenges [30] can meaningfully contribute (i.e., non-tokenistically) throughout the entirety of the PSM process. This paper focuses on the PSM process rather than the technical model itself, and reports baseline and first follow-up evaluation results examining the feasibility, value, impact, and sustainability of the PSM process in the ACT. The diverse perspectives of stakeholders, including young people with lived experience of mental ill-health, are explored and reported.

2. Materials and Methods

2.1. Research Context

This evaluation is part of a broader five-year national research program, Right care, first time, where you live. The research program aims to strengthen youth mental health systems by supporting regional leaders to make informed decisions so that young people in their region can access the right level of care, and have it delivered in a timely manner early in the course of illness [23,25,31]. This program works with eight geographically diverse regions across Australia to develop and implement system dynamics models designed to reflect the needs of each region through a PSM process. Located in Australia’s capital city Canberra, the Office in the ACT was the first site to participate in the program. Although the Office was the primary partner of the University of Sydney’s Brain and Mind Centre, this research program focused on a holistic participatory process that involved diverse stakeholders from across the ACT region. The term “site” thus interchangeably refers to both the Office and the broader ACT region, unless specified.

A comprehensive multi-scale evaluation framework underpins this evaluation study. This framework aims to longitudinally assess the feasibility, value, impact (change and action), and sustainability of PSM processes in the context of the broader youth mental health research program [26,31,32]. A flexible and iterative research approach is enabled through participatory action research principles designed to enable each site to determine their own research processes [31]. Participatory action research also facilitates the iterative improvement of the PSM process by acting on key themes identified by site participants throughout the evaluation, allowing them to take greater ownership as co-researchers to meaningfully participate in the decisions aimed at improving research implementation to optimise stakeholder engagement.

2.2. Participants and Sampling

To best capture the local ACT mental health context, diverse stakeholders across the youth mental health sector were invited to attend and contribute to three co-design PSM workshops between March and October 2022. As the Office has a strategic role in mental health services in the ACT, they played an integral role in identifying and inviting key stakeholders to participate in the workshops. Active snowball sampling was also deployed and supported by participatory action research to enable all participants to identify additional stakeholders who should be invited to participate. Over the three workshops, stakeholders worked together to collectively nominate what should be incorporated into the system dynamics model, allowing local contextual priorities to be integrated into the final model. Diverse stakeholders were included, such as representatives from the Education, Housing, and Drug and Alcohol Services sectors. For the sake of the analysis, the perspectives of four broad stakeholder groups were explored: (i) community support professionals (e.g., formal carers’ network), (ii) front-line health professionals (e.g., clinicians), (iii) health administrators (e.g., policy makers), and (iv) young people with a lived experience of mental ill-health (≥14 years). All stakeholders invited to participate in the PSM workshops were invited by the first author (G.Y.L.) to contribute to the PSM evaluation study, which is the focus of this paper.

2.3. Study Design

A mixed-methods approach was adopted to collect evaluation data throughout the PSM process, aligned with the three co-design workshops. Specifically, data were collected through gamified online surveys and semi-structured interviews prior to the first workshop in March 2022 and immediately after the third workshop in October 2022.

The online surveys had four main components: (i) questions to elicit responses regarding priorities for youth mental health system reform in the ACT, (ii) expectations and experiences participating in the PSM process, (iii) social network analysis, and (iv) patient journey mapping (which will be reported elsewhere). To reduce participant burden and increase participation, online surveys were shortened at follow-up by excluding component (ii) as this data were additionally collected through qualitative interviews. To support respondent engagement (particularly among young people), the online surveys incorporated elements of gamification hosted via the Cogniss platform, which is a no-code software tool that enabled the collection of survey data via sophisticated gamified activities [33,34]. This included the incorporation of gamified activities for respondents to complete, as opposed to standard survey templates.

Interviews were conducted online via video or teleconference by G.Y.L. and had three main components: (i) questions to elicit a greater understanding about the ACT context (e.g., youth mental health challenges and how regional decisions are made that impact youth mental health care); (ii) motivations, expectations, and experiences participating in the PSM process; and (iii) social network analysis. A fourth component was included in the follow-up interviews, namely (iv) facilitators, barriers, and experiences using the model.

Additional data were collected via researcher observations and recordings from the three PSM workshops, meetings with local stakeholders outside the PSM workshops, as well as through reflections and field notes written by G.Y.L.

2.4. Data Analysis

Descriptive statistics were utilised to analyse survey data via IBM SPSS Statistics 28, a statistical software package developed for data management and analysis [35]. Survey and interview data were triangulated to develop social network diagrams via Gephi, a free online open-source network analysis and visualisation software package written in Java [36]. Interviews were audio-recorded and transcribed via Otter.ai, a subscription-based speech-to-text transcription application [37], and subsequently checked for quality and de-identified by G.Y.L. and O.I. Transcriptions were then coded and analysed by G.Y.L. via NVivo, a qualitative data analysis computer software package [38]. Qualitative analysis was guided by a latent, inductive codebook approach to thematic analysis, whereby a semi-structured coding process was undertaken, in conjunction with reflective analytical practice [39]. Specifically, G.Y.L. independently analysed each interview and progressively developed a codebook. Throughout this process, G.Y.L. met with O.I. on a biweekly basis to reflectively discuss broader themes, stories, patterns, and meaning across the interviews [39]. The analysis was also progressively reviewed by J.O., I.B.H., V.L., and L.F., where further reflective practice supported continuous refinement of the analysis through open discussions regarding assumptions, interpretations, and coding of the data. This approach was also applied to triangulate data from other sources, such as researcher field notes, to form a richer understanding of broader determinants that may affect participation in and outcomes of the PSM process. The cross-validation of data was further complemented by participatory action research as part of a broader reflective cycle, whereby strong themes and patterns of data were shared with key stakeholders from the site as they emerged in order to identify the best course of actions, prompting continuous improvement throughout the PSM process. The findings from data analysis were reported utilising broad categories (i.e., feasibility, value, impact, and sustainability of the PSM process) extracted from the multi-scale evaluation framework, which underpins the study design.

2.5. Research Ethics

This study was approved by the Sydney Local Health District Human Research Ethics Council (protocol no. X21-0151 & 2021/ETH00553) and by the Aboriginal Health and Medical Research Council of NSW (1875/21). The evaluation tools have also been approved by the Youth Lived Experience Working Group at the University of Sydney’s Brain and Mind Centre.

3. Results

3.1. Demographics

With regards to the surveys, 32 of the 37 respondents who consented to participate at baseline fully completed the survey and were hence included in the analysis. For follow-up surveys, the proportion was 26 of the 31 respondents. A total of 24 and 25 participants consented to contribute to baseline and follow-up interviews, respectively. Eleven participants fully completed both baseline and follow-up surveys, and 14 participants contributed to baseline and follow-up interviews. Further demographic information is presented in

Table 1. Participant demographic information.

		Baseline (March 2022)		Follow-Up (October 2022)
		n	%	n	%
Online Survey	Completion of surveys
	Fully completed	32	86.5%	26	83.9%
	Partially completed	5	13.5%	5	16.1%
	11 respondents fully completed baseline and follow-up surveys
	Age of respondents
	14–17 years old	1	2.7%	1	3.2%
	18–24 years old	3	8.1%	6	19.4%
	25–34 years old	6	16.2%	5	16.1%
	35–44 years old	11	29.7%	7	22.6%
	45–54 years old	12	32.4%	5	16.1%
	55–64 years old	4	10.8%	7	22.6%
	65 years old or older	0	0.0%	0	0.0%
	Profession of respondents 1
	Community support professional	9	28.1%	6	23.1%
	Front-line health professional	11	34.4%	8	30.8%
	Health administrator	4	12.5%	6	23.1%
	Young people with lived experience	3	9.4%	5	19.2%
	Did not disclose	5	15.6%	1	3.8%
	Other information of respondents
	Culturally and linguistically diverse	5	15.6%	3	11.5%
	LGBTQIA+	4	12.5%	7	26.9%
	Religious and/or spiritual	2	6.3%	0	0.0%
	None of the above	22	68.8%	17	65.4%
Interviews	Completion of interviews	24		25
	14 respondents completed baseline and follow-up interviews
	Profession of respondents
	Community support professional	12	50.0%	9	36.0%
	Front-line health professional	1	4.2%	6	24.0%
	Health administrator	9	37.5%	6	24.0%
	Young people with lived experience	2	8.3%	4	16.0%

¹ Only fully completed responses included.

.

3.2. Feasibility of PSM to Support Youth Mental Health Systems in the ACT

A draft version of the system dynamics model for youth mental health was delivered to the ACT at the third co-design workshop in October 2022. While 87.5% of the survey participants (n = 28) had no previous experience utilising such a model to support decision-making, diverse stakeholders contributed to the participatory workshop process, in which they interacted with the model, discussed the implications of the findings for policy and planning, and provided feedback to the research team to further refine the model.

The opportunity to engage and collaborate with others was identified as a key motivator for baseline survey respondents to participate in the PSM process, with 75.0% (n = 24) selecting “engage with people from other health services and organisations”, and 62.5% (n = 20) selecting “opportunities for further collaborations with other health services and organisations”. This finding was further reinforced in the baseline and follow-up interviews across all stakeholder groups, where respondents not only noted opportunities for collaboration with others as a key motivator to participate, but as a necessity. Importantly, interview responses suggested that the PSM (co-design) process enabled the participants time needed to develop more trust that a robust model would reflect the local ACT context. Figure 2 illustrates how participants emphasised the importance of trust-building processes in model building. These processes included community consultation and co-design, as well as the hands-on participation of young people with lived experience of mental ill-health.

Although opportunities for collaboration were regarded highly by participants, they also noted challenges working with diverse stakeholders. Specifically, participants suggested further training and standardisation among workshop facilitators were required to better manage power relationships between diverse stakeholder groups and ensure that the PSM process was more inclusive, accessible, and transparent for all involved. For example, one front-line health professional noted, “it was sometimes hard to get any other ideas across [as] there were one or two people dominating with their ideas” and suggested in future that the “facilitator could do more containing and asking quieter voices if they wanted to add anything”.

Although workshop facilitation to better manage power relationships was identified as an area for improvement, the way facilitators respected all participants was noted as a key motivator to continue contributing throughout the PSM process. A young person with a lived experience of mental ill-health shared how their contributions to the PSM process differed from similar co-design workshops they attended:

“… [the project] went past what I was expecting because I was anticipating reluctance like, ‘It’s the system, and we can’t change it.’ But being able to hear people who are actually running the system saying that they want to make it better made me feel like there could be positive change”.

Although most participants conveyed that they felt respected during the process, another area for improvement was how information was presented to stakeholders. Figure 3 demonstrates how modelling language and technical aspects could be alienating for participants, many of whom lacked prior knowledge and experience of modelling. It was, therefore, “critical” that time was taken to enable participants to understand the value of the tool. In addition to providing less technical explanations, community support professionals and health administrators suggested that demonstrating what the model could tangibly look like at each of the three workshops would improve engagement throughout the PSM process, as stakeholder engagement varied and new registrants attended each workshop.

3.3. Value of PSM for Youth Mental Health System Strengthening in the ACT

Survey participants identified “address the current gaps of youth mental health care” (90.6%, n = 29), “improve the health and well-being of young people” (87.5%, n = 28), and “assist to make better decisions to improve youth mental health treatments or programs” (84.4%, n = 27) as the top benefits that they hoped the system dynamics model would provide for the ACT. These responses were further reinforced during the interviews, with a health administrator sharing in their baseline interview their hopes for the system dynamics model tool:

“…to make people think about what the options are. Meaning, funds can be directed in the most optimal way rather than wasting money on things that are not available or unlikely to make a difference [in youth mental health]”.

Another health administrator noted in their follow-up interview that the model has the potential to “help write better business cases as sometimes we are left with such a small amount of time to ask for funding that may last years”. The value of technical analyses offered by the model that may not be readily available to stakeholders was identified, with one health administrator sharing:

“We’re asked to provide a cost benefit analysis [for our business cases] that we’re not really equipped to do … Having this [model] that’s quite accessible and can give us that level of detail is great”.

There was consensus in the baseline and follow-up interviews on the importance of improving the health and well-being of young people in the ACT through better policy decisions. As interviews allowed for further probing of the responses, systemic challenges impacting youth mental health in the ACT were identified. The challenges identified were similar across the participant stakeholder groups, with a common theme being that transparency was lacking around system-level decision-making regarding youth mental health in the ACT. In Figure 4, quotes from participants occupying different roles within the system convey a lack of transparency regarding how decisions are made. This suggests stakeholders sensing a “barrier” in understanding who has the power to make decisions to strengthen the youth mental health system in the ACT, which was further emphasized in the follow-up interviews, as discussed below. Further and notably, there were different views about where responsibility for decision-making was placed across the stakeholders. For example, young people with a lived experience of mental ill-health placed responsibility on front-line health professionals or community support professionals, professionals placed responsibility on health administrators, and administrators placed responsibility of ultimate decision-making power at a level above their role (such as the Minister for Mental Health or the federal government).

Despite this confusion, participants across all stakeholder groups shared similar views regarding the value of the model to support improved policy decision making (Figure 5). The model itself was viewed by participants as a tangible outcome to look forward to, incentivising them to be involved in the PSM process to gain access to the model. Additionally, the model was viewed as a credible and easy-to-use tool, with sentiments shared by a health administrator that they “could put my faith into it [the model]” through the evidence-based approach utilised by the research team of including empirical information into the model. Participants also saw value in the participatory action research approach, which supported “reflexive practice in action”, where areas for improvement and solutions could be identified and actioned by the site, providing stakeholders more autonomy in the PSM process.

Limitations of the model were noted by participants, including the lack of data infrastructure, not only in the ACT, but nationally. For instance, as population level data in Australia are currently disaggregated by males and females, some participants expressed concern that the model did not completely capture the whole ACT youth population (e.g., transgender, non-binary, etc.). Additionally, as system dynamics modelling provides insights at a whole population level, concerns that the model “doesn’t reflect the marginalised communities and only helps those organisations that are mainstream” were expressed. One community support professional noted that “the lack of data is phenomenal”. Despite these limitations, another community support professional expressed that the PSM process of working with diverse stakeholders from various disciplines and seniority levels supported an environment where participants were “…able to actually highlight in front of some key government people [which] was very beneficial because it does recognise the fact that there are still challenges around these things [data]”. These responses not only reinforced systemic issues such as the need to strengthen data infrastructure, but also further highlighted the need for clearer explanations and communication regarding the model to enable participants to better understand both the parameters and limitations of the model.

3.4. Impact (Change and Action) of PSM to Support Youth Mental Health Systems in the ACT

A health administrator noted tension “between different sectors” during the first workshop, but by the time of the final workshop, noted that “sectors were working better together”. This interview quote aligned with observations made by the first author, where it was written in G.Y.L.’s field note entry for workshop one, “I overhead a workshop participant saying to someone, ‘It’s hard to not feel attacked’”. Comparatively, GYL’s field note entry for the final workshop included:

“…when the model insights were presented, you could visibly see how everyone in the room was very engaged. This was a stark difference from the first workshop where I noticed quite a few people on their phones or answering emails during larger group sessions like these”.

This suggests that the PSM process not only led to a tangible tool—which was recognised as a facilitator for engagement—but it also supports a broader participant engagement and learning process. This was also noted by a health administrator in their follow-up interview, “… the modelling can definitely shape our thinking and is fantastic for looking at various scenarios”.

These qualitative findings were also somewhat reflected in the quantitative survey data. The PSM workshops provided the opportunity for site stakeholders to prioritise broad strategic intervention categories for inclusion in the final system dynamics model (Figure 1), including, but not limited to, programs and services that target prevention, early intervention, and community-based youth mental health care. As presented in

Table 2. Baseline and follow-up survey results to understand if participant perceptions shifted throughout the PSM process to align with the insights of the final system dynamics model.

Broad Strategic Intervention Category Prioritised by Site Stakeholders	Baseline Mean 1 (SD), N = 32	Follow-up Mean (SD), n = 25 2	Linear Mixed-Effects Model p-Value	Estimates of Fixed Effects for p-Values <0.05 (SE)
Prevention programs focusing on health and well-being (e.g., education programs)	4.59 (0.71)	4.36 (0.70)	p-value for time = 0.13	N/A
			p-value for profession = 0.03	Community professional = 0.10 (0.33)
				Front-line professional = −0.49 (0.32)
				Health administrator = 0.28 (0.35)
				Young people = −0.16 (0.37)
Early mental health intervention programs (e.g., school counselling services)	4.75 (0.67)	4.80 (0.41)	p-value for time = 0.90	N/A
			p-value for profession = 0.05	Community professional = 0.11 (0.27)
				Front-line professional = −0.45 (0.25)
				Health administrator = 0.24 (0.24)
				Young people = −0.03 (0.30)
Community-based mental health intervention programs (e.g., alcohol and other drugs support programs)	4.59 (0.56)	4.76 (0.44)	p-value for time = 0.16	N/A
			p-value for profession = 0.17	N/A
Hospital-based mental health intervention programs (e.g., hospital-to-home services)	4.34 (0.70)	4.20 (0.58)	p-value for time = 0.31	N/A
			p-value for profession = 0.76	N/A
Cultural programs (e.g., Aboriginal Community Controlled Health Organisations)	4.56 (0.62)	4.52 (0.59)	p-value for time = 0.27	N/A
			p-value for profession = 0.64	N/A
Access programs (e.g., creating opportunities to receive better access to care such as after-hours care or shortened waiting times)	4.81 (0.59)	4.64 (0.67)	p-value for time = 0.16	N/A
			p-value for profession = 0.59	N/A
Practical employment or educational skills programs (e.g., financial counselling)	4.50 (0.67)	4.36 (0.49)	p-value for time = 0.57	N/A
			p-value for profession = 0.15	N/A

¹ Mean = 1 (not important at all); mean = 5 (very important). ² N = 26 for completed follow-up surveys; however, one respondent was excluded from analysis as they skipped this question.

, the means were calculated utilising survey data on how stakeholders rated the importance of broad strategic categories for inclusion in the model. The means shifted toward the direction of the final model insights—for instance, in favour of programs and services that target early mental health intervention and community-based mental intervention programs, in line with the simulations of the model presented to the site at the third (and final) PSM workshop. However, analysis using linear mixed-effects models detected no statistically-significant change in stakeholder ratings for all broad strategic categories between baseline and follow-up. These results may be inconclusive due to the insufficient sample size. Statistical significance was observed for respondents’ profession in relation to the prevention and early mental health intervention categories. Community support professionals and health administrators were likely to provide a higher importance score in their Likert-scale responses for both strategic categories (i.e., early mental health and community-based intervention programs), compared with front-line health professionals and young people with a lived experience of mental ill-health.

Another area where shifts were observed was how young people with a lived experience of mental ill-health were included in the PSM process. Although participants across all stakeholder groups agreed at baseline that “young people [should be] at the forefront” throughout the PSM process, a lack of lived experience representation was identified after the first workshop. Participatory action research principles underpinning the study design supported not only reflective practice to jointly identify this area for improvement by the research team and the site, but also led to collaborative actions. As explained by a young person with a lived experience during their follow-up interview, “At the first workshop, I was the only young person there”. However, this same participant noted positive changes made by the research team (S.H. and J.O.) in collaboration with the site (E.M., N.J., S.L., J.B., and E.B.) to engage more broadly with young people with a lived experience in the ACT, ensuring their voices were not only included, but genuinely integrated into the PSM process (Box 2).

Box 2. Young person with lived experience of mental ill-health sharing changes made by the research team and the site to engage and include youth lived experience voices into the PSM process.

“… most of the young people had dropped out because of COVID and things like that [at workshop one] … So before workshop two, there was a young people only workshop, which basically replicated the events of workshop one. … Since it was just me and other young people there [at the youth workshop] it was really great getting to bounce off everyone and have that environment where it’s a safe space without stakeholders or other government people. … A lot of the young people who were at the youth workshop obviously came to workshop two. … And then after workshop two, for those who couldn’t come to workshop two or those who felt like they didn’t have a chance to properly say stuff, we had a debrief meeting. And we all had another proper opportunity to discuss our thoughts which was really, really good”.

Young person with lived or living experience of mental ill-health

The shift to include more young people with a lived experience in the PSM process was viewed as “invaluable” by participants across all stakeholder groups, with suggestions to engage with young people with a lived experience as early in the PSM process as possible in the future. Young people also expressed not only “feeling heard [which] makes a big difference”, but having confidence that the research team was “actually going to do something with my story”.

A health administrator shared how they had noticed the change in youth lived experience participation throughout the PSM process, attributing the involvement of young people to creating the right balance of representation of diverse stakeholder groups at the workshops. This health administrator shared, during their follow-up interview, the methods adopted throughout PSM that supported improved engagement with young people (Box 3). Improved youth engagement led to young people no longer being motivated by financial incentive alone, but valuing other outcomes such as feeling heard and respected without “feeling like the need to receive financial reimbursement for my time—that was just a bonus”.

Box 3. Reflection on strategies to improve youth engagement.

“…I think once we got that balance between the sector and the services with the young people in the room, I think it tipped it back. And I think because we did that specific youth [only] workshop where we really made sure that the young people were set up to succeed when they attended workshop two, they knew what they were walking into. So we had youth workers there [at the PSM workshops] so that if they [young people] felt overwhelmed, they had someone to talk to. We made it a really safe space, which I think allowed them [the young people] to feel brave enough to kind of challenge the sector. And they did particularly [challenge the sector] in workshop two, which I was really proud of. … We ended up with quite a beautiful process between young people having very much a loud role in that [PSM process]. … if I was going into this process again, I would absolutely say that there needs to be a large emphasis on connecting with lived experience as early on as possible”.

Health administrator

In addition to youth engagement, participants shared during follow-up interviews that the PSM process created opportunities for further collaboration between workshop attendees. For instance, a front-line health professional shared their experience of the “gems that have come out of that [PSM] process” regarding developing new collaborations:

“…in terms of connection and just people saying, ‘Can you come talk to my team??’ … The Office also started a [reference] group bringing services together around the issue of youth mental health. … I don’t know whether that’s a direct outcome of this [PSM process], but that’s how I perceived it which I thought was fantastic”.

While the reference group arose from a separate initiative, qualitative responses highlighted that the PSM process helped to strengthen stakeholder relationships as well as the commitment for diverse sectors to work together. These findings align with survey responses, with 53.8% (n = 14) of respondents reporting that they “see more opportunities to work with other organisations and/or people after my involvement in the co-design research program”. Subsequently, 38.5% (n = 10) of respondents reported that their “current relationships with organisations and/or people have improved”. Social network analysis, utilising the responses of those who completed both baseline and follow-up surveys, triangulated with qualitative interview data, showed more linkages (or collaborations) were formed between stakeholders from the baseline (Figure 6a) and follow-up evaluation data collection points (Figure 6b), as depicted through circle size and line thickness changes, demonstrating the strength of collaboration between organisations.

Changes were also made to the online surveys during the PSM process, which incorporated elements of gamification to support respondent engagement, particularly with young people with a lived experience of mental ill-health. Participatory action research principles supported a process of iteratively enhancing the follow-up surveys based on feedback received at baseline. Overall, 80.8% (n = 21) responded that the gamified activities made the survey more enjoyable at follow-up compared with 56.3% at baseline. Interestingly, 100.0% (n = 3 at baseline; n = 5 at follow-up) of the respondents who identified as young people with a lived experience of mental ill-health found gamified activities made the surveys more enjoyable, whereas front-line professionals and community professionals were more likely to find the surveys confusing or too long (

Table 3. Perceptions of the gamified online survey.

Baseline (N = 32) Community support professional, n = 9 Front-line health professional, n = 11 Health administrator, n = 4 Young people, n = 3 Did not disclose profession, n = 5	Gamified activities made the survey enjoyable	Overall, n = 18 (56.3%)      Community support professional, n = 5 (27.8%)      Front-line health professional, n = 4 (22.2%)      Health administrator, n = 3 (16.7%)      Young people with lived experience, n = 3 (16.7%)      Did not disclose, n = 3 (16.7%)
	Gamified activities made the survey confusing	Overall, n = 10 (31.3%)      Community support professional, n = 1 (10.0%)      Front-line health professional, n = 7 (70.0%)      Health administrator, n = 1 (10.0%)      Young people with lived experience, n = 0 (0.0%)      Did not disclose, n = 1 (10.0%)
	The survey was too long	Overall, n = 7 (21.9%)      Community support professional, n = 2 (28.6%)      Front-line health professional, n = 4 (57.1%)      Health administrator, n = 0 (0.0%)      Young people with lived experience, n = 0 (0.0%)      Did not disclose, n = 1 (14.3%)
	The survey was no different to any other survey	Overall, n = 3 (9.4%)      Community support professional, n = 2 (66.7%)      Front-line health professional, n = 0 (0.0%)      Health administrator, n = 1 (33.3%)      Young people with lived experience, n = 0 (0.0%)      Did not disclose, n = 0 (0.0%)
Follow-up (N = 26) Community support professional, n = 6 Front-line health professional, n = 8 Health administrator, n = 6 Young people, n = 5 Did not disclose profession, n = 1	Gamified activities made the survey enjoyable	Overall, n = 21 (80.8%)      Community support professional, n = 4 (19.0%)      Front-line health professional, n = 6 (28.6%)      Health administrator, n = 6 (28.6%)      Young people with lived experience, n = 5 (23.8%)      Did not disclose, n = 0 (0.0%)
	Gamified activities made the survey confusing	Overall, n = 2 (7.7%)      Community support professional, n = 1 (50.0%)      Front-line health professional, n = 0 (0.0%)      Health administrator, n = 0 (0.0%)      Young people with lived experience, n = 0 (0.0%)      Did not disclose, n = 1 (50.0%)
	The survey was too long	Overall, n = 3 (11.5%)      Community support professional, n = 1 (33.3%)      Front-line health professional, n = 2 (66.7%)      Health administrator, n = 0 (0.0%)      Young people with lived experience, n = 0 (0.0%)      Did not disclose, n = 0 (0.0%)
	The survey was no different to any other survey	Overall, n = 1 (3.8%)      Community support professional, n = 0 (0.0%)      Front-line health professional, n = 1 (100.0%)      Health administrator, n = 0 (0.0%)      Young people with lived experience, n = 0 (0.0%)      Did not disclose, n = 0 (0.0%)

).

3.5. Sustainability of PSM for Youth Mental Health Systems Strengthening in the ACT

Overall, participants expressed during the baseline and follow-up interviews that the PSM process was “worthwhile to commit to” as the model had the “potential to assist with advocacy, planning, and policy making”. The uniqueness of the ACT compared with other states in Australia was highlighted during the follow-up interviews, particularly around how “…the ACT is small enough that this [implementing the model] could work”.

One health administrator admitted that there “…is a little bit of doubt, with any model [about how the model will be used]”. However, that same participant also shared that “it’s not to say I don’t have confidence in the model”. Another health administrator expressed that the PSM process supported a “narrative that we must fail forward”, suggesting that despite any scepticism towards the model, that the alternative (i.e., current decision-making practices) is not any better “and we have to do something different [to support youth mental health in the ACT]”.

Most participants expressed hope that the model would be sustainably used to support decision-making for youth mental health systems policy and planning decisions. Although participants saw value in the PSM process in that “the co-design makes me more willing to use the model because I understand it more”, participants also expressed that more awareness of system dynamics modelling as a method is required in the youth mental health sector, to increase recognition of the model as a legitimate tool to support decision-making in youth mental health policy and planning in the ACT (Figure 7).

In addition to building more awareness and legitimacy of the model as a youth mental health policy and planning tool, participants expressed that “…if it [the model] got into the hands of the right people, it could be really helpful”. However, as discussed earlier, uncertainty where actual responsibility for decision-making lay, who was going to be accountable for making such decisions considering potential limitations of the model, and how to best incorporate the model in decision-making processes was uncovered during follow-up interviews (Figure 8). A health administrator shared their concerns of “defaulting to old and less reliable methods” if those who are perceived to be in power do not trust or understand the model. A front-line health professional also shared their concerns and questioned how stakeholders in the ACT could be “confident that they [the decision makers] will actually implement any of the recommendations [from the model]”, comparing their observation of recommended evidence-based changes repeatedly not being implemented in the youth mental health system to “…our big inquiries that we have by parliament [in Australia]. … they never implement the findings, so what’s the point?”.

Strategies identified by stakeholders to facilitate sustainable implementation of the model in the ACT included: (i) ongoing adaptations to the model to reflect the dynamic needs of youth mental health in the ACT, so that the model will not “…just die on the vine like most things do at the end of projects”; (ii) widespread training on how to use and interpret the model to prevent “incorrect usage of the model” or the possibility that the model “…might be used in a negative way”; and (iii) implementation of the model within diverse local sectors or organisations in the ACT to mitigate potential challenges, including staff turnover, to “champion” the model and support strategic decision-making for youth mental health planning and policy.

4. Discussion

This paper reports on the feasibility, value, impact, and sustainability of PSM in the ACT, exploring diverse perspectives of stakeholders, including young people with a lived experience of mental ill-health, community support professionals, front-line health professionals, and health administrators from diverse sectors and organisations. In summary, a system dynamics model was delivered in the ACT via a PSM process. While most stakeholders had no previous experience in PSM nor utilising such a model to support decision-making, diverse stakeholders contributed to the development of and discussions on model insights for youth mental health policy and planning.

A lack of transparency in current decision-making processes regarding youth mental health was uncovered in the ACT. The participatory (co-design) process of developing the model with diverse stakeholders—particularly with young people with a lived experience of mental ill-health—was viewed as a facilitator to develop trust that the model could support transparency and improved policy decision-making to best allocate limited funding and achieve optimal mental health outcomes for young people in the ACT. There was, however, confusion among stakeholders at baseline regarding where responsibility resided for youth mental health system policy decision-making in the ACT. This confusion was also observed in follow-up interview responses, where there was uncertainty among stakeholders regarding where the model would be best implemented to inform decision-making. This suggests that stakeholders felt a lack of agency or powerlessness in their ability to influence current decision-making processes to improve the youth mental health system in the ACT. This also suggests that greater accountability is required to support systemic reform in youth mental health.

This sense of powerlessness expressed is not unique to the ACT and has been identified as an obstacle that continues to undermine mental health system reform in Australia. Described as the “yes minister” culture, those who are perceived as being in power can also feel disempowered by the bureaucracies that operate within Australia’s mental health system, influencing policy success and failure [40]. This may create the passive acceptance that there are no impactful solutions to achieve youth mental health system reform, which may consequently justify inaction. In PSM research, it has been suggested complex systems, such as youth mental health, fail to undergo reform because the solutions being sought are only those that are politically popular at a national level, as opposed to considering solutions that can first enact change on a regional level, before going on to effect changes at national and global system scales [22].

While our results found no evidence that the PSM process had (as yet) led to shifts in how stakeholders would prioritise interventions or strategies to support youth mental health care, there was evidence of improved relationships among stakeholders and understanding of what the barriers and opportunities are to regional youth mental health reform. This may, therefore, indicate that the ACT may be in the early stages of “transformative change” [41]. Transformative change requires diverse stakeholders to first develop new kinds of capacity and relationships, which then fosters new ways of practice [42]. However, many modelling exercises “fail to reach the goal of empowering stakeholders to take ownership of the decision-making process required”, which can hinder transformative change [42].

Thus, the focus in the ACT should now be on maintaining that trust and empowering all regional stakeholders—led by the site—to realise their potential on how they could both contribute to and hold regional leaders accountable for regional youth mental health change in order to support the sustainable implementation of the model. National leaders should also be held accountable to support regional models of governance and decision-making, through improvements to systemic challenges that can limit the system dynamics model, such as current data infrastructure and information collection systems [21]. Strategies on how to support sustainability were identified by stakeholders during follow-up interviews. It is conceivable that over time, with repeated engagement with the model, youth mental health system strengthening priorities would further align with the modelling insights. Therefore, although youth mental health reform is a difficult endeavour, we argue that the PSM process offers advantages over the current decision-making processes.

Further research should investigate empowerment strategies that can complement the implementation of such models developed through a PSM process, holding regional and national leaders more accountable to make more transparent and informed decision-making practice. Future research should also further explore the effectiveness of engaging with the use dynamic systems models and its impacts on regional youth mental health policy decision-making, in the hopes that these regional impacts can catalyse both positive and lasting changes to national and global youth mental health system scales.

Limitations

Due to the nature of participatory action research—where a process of reflective practice prompted continuous improvement throughout the PSM process—changes to the evaluation study protocol were made when participant fatigue was noted. Specifically, as participants engaged in three full-day workshops across a six-month period, participating in additional interviews and surveys was noted as an additional burden. To minimise participant burden, follow-up surveys were shortened to exclude Likert-scale response questions that could be asked during the interview, particularly around participant expectations and experiences participating in the PSM process. The implication of this change was that comparative statistical tests were unable to be performed. Additionally, although 100.0% of youth lived experience respondents reported that gamified activities made the surveys more engaging at baseline and follow-up, future studies should consider a more appropriate balance of questions to include within surveys, as gamified activities may lengthen the time it takes to complete the surveys (a barrier for time constrained stakeholders). The authors also acknowledge that the overall sample size was small, with n = 32 and N = 24 fully completing the baseline survey and interview, respectively, and n = 26 and N = 25 fully completing the follow-up survey and interview, respectively. Only 11 participants fully completed both the baseline and follow-up survey, which had an impact on the final social network analysis. With that being said, a response rate of 75.5% and 56.4% was achieved for baseline and follow-up surveys, respectively. Furthermore, due to the triangulation of data, the social network analysis, despite its limitations, was still able to uncover significant findings regarding the impact of the PSM process on stakeholder collaborations. Baseline and follow-up interview results also demonstrated responses that reached saturation across diverse perspectives of stakeholder groups who contributed to the PSM process. Finally, while the perspectives of young people with a lived experience of mental ill-health are recognised as a crucial voice in the PSM process, the authors acknowledge that three and two young people contributing to baseline surveys and interviews respectively, and five and four young people contributing to follow-up surveys and interviews, respectively, is relatively low. However, as the study aimed to explore the perspectives of diverse stakeholders, rather than emphasising the number of young participants involved in our study, what holds greater significance is the genuine and equitable opportunities provided to young people to contribute to the PSM process. The process of increasing youth lived experience engagement was co-led and implemented by a young person, which ensured that engagement was of a high quality (i.e., youth-friendly and non-tokenistic), and allowed young people to have a genuine seat at the decision-making table throughout the PSM process.

5. Conclusions

Across 30 years of Australian mental health plans and policies, one of the most enduring challenges for mental health reform has not been the articulation of desirable goals or change, but its implementation [16]. While this paper focused on describing a case study in the development of PSM, it highlights, more generally, the complexity and resources required to build new ways of thinking to support local mental health reform. Systems modelling via a participatory approach is not only feasible, but can also add value to a regional youth mental health system, as exemplified by our work in the ACT. The inclusion of diverse stakeholders, including young people with a lived experience of mental ill-health, in the PSM process facilitated trust towards the final system dynamics model built for the ACT, and enabled all stakeholders to be actively engaged as co-researchers to cyclically identify areas for improvement and suitable actions to support the participatory process itself. Although youth mental health system reform was viewed as both desirable and a necessity across stakeholder groups, a lack of agency or powerlessness was observed by stakeholders on how they could themselves contribute to improving the youth mental health system in the ACT. Similar observations have been reported in other research in Australia and identified as a barrier to mental health system reform. Ongoing collaboration between researchers and regional stakeholders to facilitate the implementation of the insights derived from the model and, perhaps more importantly, to empower stakeholders to realise their potential on how they could contribute to change, will be critical for the successful and sustainable implementation of the learnings of the PSM process to support regional youth mental health system reform in the ACT.