Institutionalizing Reflexivity for Sustainability: Two Cases in Health Care

Abstract

Unsustainability in health care comprises diminishing returns and misalignment between the health care regime and the needs of the population. To deal with complex sustainability problems, niche solutions can be collaboratively designed and implemented through reflexive methods. For second-order sustainability, however, the institutionalization of the reflexive element itself is also needed. This paper aims to provide insight into the possibilities of embedding reflexivity into institutions to support second-order sustainability by reporting on two consecutive participatory research programs that sought to address unsustainability in terms of misalignment and diminishing returns. The first case study reflexively monitored the system’s innovation toward an integrated perinatal care system. Reflection within the project and implementation was supported successfully, but for stronger embedding and institutionalization, greater alignment of the reflexive practices with regime standards was needed. Building on these lessons, the second case study, which was part of the IMI-PARADIGM consortium, collaboratively built a structured tool to monitor and evaluate “the return on engagement” in medicine development. To institutionalize reflexivity, the creation of “reflexive standards” together with regime actors appears to be most promising. Broader and deeper institutionalization of reflexive standards can be attained by building enforcement structures for reflexive standards in the collaborative process as part of the reflexive methodologies for addressing complex sustainability problems.

1. Introduction

The sustainability crisis in most Western healthcare systems comprises ever-rising costs (not just in absolute terms, but as a percentage of the gross domestic product) caused by changing demographics and new and often expensive care products in combination with critiques on the quality of care from practitioners and care consumers. In general, healthcare faces a problem of diminishing returns, where the cost increase does not necessarily equate to growth in care quality [1,2]. “Sustainability”, a fluid and broad term, then refers to development that “meets the needs of the present generation without compromising the ability of future generations to meet their own needs” [3] (p.43). The question is whether current healthcare systems will still be able to provide accessible high-quality services for future generations. Sustainability in that sense is a balancing act of different needs and perspectives. Essink [4] (p.197) frames this as a “dynamic equilibrium in which the system easily rests on its constituent values (e.g., affordability, accessibility, acceptability and quality)”. By following the transition management and system innovation literature [2,5], unsustainability can be conceived of as friction between “landscape” developments [6] and the “regime” [7]. The dominant institutions and structures have become unequipped to face current societal problems. Diminishing returns and negative side effects are corollaries of system success [8]. The well-known Multi-Level Perspective (MLP) [6,9] can be used to describe how the initial success of a social system drives changes on the landscape level, leading to a misalignment between the successful regime and the needs of the population. It has provided some guidance for prescriptive modeling of sociotechnical transitions [5]. Although it has been criticized, for instance, for lacking a distinction between “radical” change and regular systemic renewal [10] and an overemphasis on structures [11], the model is helpful for eliciting processes of mis- and realignment between the landscape and regime. The model dissects (sociotechnical) systems in three layers: the landscape (societal structures that emerged over multiple generations, like globalization), the regime (structural layer that constitutes the context of common practice with its dominant institutions, physical infrastructure, and culture), and the niche level (protective spaces in which organizers of “initiatives” that deviate from the regime focus on solving problems they recognized on the regime level). Following the MLP, a transition is a regime change that can occur due to sustainability problems related to developments at the landscape level the regime cannot handle, instigated by deviating practices in niches [5].

Addressing misalignment, and thus systemic unsustainability, through niche innovations is not straightforward. The problems underlying systemic unsustainability are seen as wicked [12], unstructured [13], or persistent [8], meaning the regime actively works against sustainability solutions. To address these problems, support niches, and overcome systemic barriers, the system innovation literature argues for applying reflexivity or, more precisely, a set of methods to enhance reflexivity in (forms of) experimental collaborative practices [14]. Promoting reflexivity within such initiatives is argued to embed solutions to sustainability problems in existing systemic contexts [15].

Across health care contexts, the need for reflexivity is increasingly recognized. It has been shown, for instance, that reflexivity at an individual level is an essential part of diagnostic knowing in general practice [16]. This has led to an increased emphasis on including reflexivity in health profession education (e.g., [17,18]). Moreover, the role of team reflexivity in achieving improvements to care provision (e.g., [19,20,21]) and health care innovation (e.g., [22,23]) is being explored in an increasing number of settings. Studies that explore the role of reflexivity within transformation of the health care system (i.e., beyond the individual and team level) are, however, sparse.

Furthermore, as this paper will argue, addressing sustainability problems through embedding solutions with the help of reflexive practices cannot equate to “true” sustainability. Even if misalignment is addressed, the implemented, embedded “solutions” will ultimately effectuate new side effects and misalignment. We therefore make a distinction between first-order sustainability, a system in which current sustainability problems are addressed, and second-order sustainability, a system in which the structures are in place to monitor and adapt to misalignment continuously. For such a system, reflexivity itself, as a practice of internal (organizational) accountability [24], needs to become embedded as a form of structured flexibility. This implies that reflexivity itself needs to become embedded through habitualized ways of thinking and acting [25] supported by institutional structures [8].

The aim of this paper is to provide insight into the possibilities of embedding reflexivity into institutions to support second-order sustainability by reporting on two consecutive participatory research programs (6 years of research in total) in different health care contexts where both sought to address unsustainability in terms of misalignment and diminishing returns.

In Section 2, we explore sustainability issues in maternal care and medicine development in terms of diminishing returns and misalignment, introducing the rationale for interventions based on reflexive methods. Building on that, we argue that reflexivity itself needs to be institutionalized for sustainable systems because of the complexity of sustaining success in addressing sustainability issues. To define our analytical lens, we introduce three main phases of institutionalization: habitualization, objectification, and sedimentation.

In the Results section, we analyze the challenges and opportunities faced in building the “set of methods to enhance reflexive thinking” among the participants in response to sustainability threats. Both programs aimed to facilitate (1) processes of reflection among the project participants on an individual level and (2) the creation of structures to embed these processes sustainably in organizational networks in order to deal with problems of diminishing returns and misalignment. Through this work, we sought to gain insight into the challenges and opportunities of institutionalizing reflexivity.

2. Complexity of Systemic Sustainability Problems: The Need for Reflexivity

2.1. Dimishing Returns and Misalignment in Perinatal Care

The perinatal care system in the Netherlands is in transition, following debates on why perinatal mortality rates in the Netherlands became relatively high in comparison with other European countries. Historically, the Netherlands has fostered a system in which community-based midwifery plays a dominant role in maternity care. While many countries increasingly have medicalized childbirth and organized perinatal care as secondary care, the Netherlands is known for a model in which risk selection is key. Women considered “low risk” receive primary care (i.e., midwife-led care) and may choose to give birth in an outpatient clinic or at home. Secondary and tertiary care is only provided in (academic) hospitals in cases of medium or high medical risk [26]. Internationally, this model is and has been considered exemplary for limiting (unnecessary) medicalization of maternal and perinatal care. In a 2016 review on 15 trials involving 17,674 women, Sandall et al. [27] suggested that midwife-led continuity models contribute to (1) higher maternal satisfaction, (2) a cost-saving effect compared with other care models, and (3) women who received midwife-led continuity models of care being less likely to be subjected to medical interventions, with no increase in adverse outcomes [27]. The Netherlands has one of the lowest caesarean section rates of any European nation [28].

However, in 2004, a European comparative study caused quite a stir by suggesting that perinatal mortality rates in the Netherlands were relatively high and ameliorating at a slower rate, downgrading the Netherlands to number 23 of the 26 ranked European countries [29]. In response, the “Steering Committee Pregnancy and Childbirth” was formed, and their report, “A Good Start”, focused on the autonomy of both professionals and organizations as a key barrier for collaboration and coordination, resulting in fragmentation and discontinuity of care [26]. The (institutional) structure and organization of the care system was considered to be in need of radical change.

Important to note, however, is that the Netherlands did not perform worse in absolute numbers; other countries were merely performing better faster [29]. In other words, where the Netherlands used to be leading in quality of care and low perinatal and maternal death, current investments yield comparatively less. The Dutch maternal and perinatal care system suffers from diminishing returns.

Furthermore, a misalignment in terms of changes in what the population expects from perinatal care occurred. The output indicators, following broader societal trends, have changed to include “client-centered outputs”, like respect or autonomy [30]. The system thus functioned suboptimally and needed to integrate risk selection and responsive care.

To enhance the quality (addressing both diminishing returns and misalignment), obstetric partnerships (in Dutch; Verloskundige SamenwerkingsVerbanden, normally abbreviated as VSVs) were formed in which birth care professionals from maternity, primary, secondary, and tertiary care were supposed to design integrated care pathways for patient-centered care [31]. Obstetric partnerships were considered as platforms on which professionals would make formal agreements on protocols, procedures, and care pathways, share knowledge and expertise, smooth out day-to-day collaboration, and as such be instrumental for improving quality of care in terms of responsiveness [30], where the broader (not only medical) well-being of pregnant women can be improved by the interactions they have with the birth care system. Interprofessional collaboration [32] in these partnerships sought to overcome the boundaries between care professionals in their different organizations with their different educational backgrounds, different (epistemic) cultures, financing structures, (professional) protocols, and political lobbies.

The sustainability issues, however, had some wicked [12] characteristics involving conflicting values and perspectives on the role and place of risk selection and organization of “continuity” of care, substantially hampering the formation and effectiveness of these new partnerships [26]. In the results, we (1) describe how we, by reflexively monitoring the system innovation toward an integrated perinatal care system, aimed to support interprofessional collaboration and patient centered care in order to (2) analyze our effectiveness in embedding reflexivity for second-order sustainability.

2.2. Dimishing Returns and Misalignment in Medicine Development

The model of pharmaceutical innovation, which proved extremely profitable for the majority of the twentieth century, is now considered as inherently unsustainable [33]. As each new treatment raises the current standard of care, this raises the costs needed to achieve any incremental improvement, making it more difficult to improve further, which also results in diminishing benefits and added value for medicine users alongside extremely high costs for (health system) payers. For instance, in June 2021, the US FDA approved a new Alzheimer’s medicine for the first time since 2003 [34], despite reservations over the drug’s effectiveness. The reasons proposed for this diminishing trend include rising costs of R&D processes such as clinical trials, more stringent regulatory environments, and pressures from increasing generic competition [33].

In addition to market forces, changes in actor configurations are important for understanding an evolving sociotechnical landscape. Specifically, definitions of patient communities or “medicine users” are becoming more pluralized. One example is the substantial rise of the “rare disease” category in the twentieth century and subsequent increases in patient advocacy in the rare disease domain [35]. Advocacy efforts have focused on providing a voice to these diverse patient populations in medicine R&D, particularly in clinical trials. Sustained calls from social science and health system advocacy have been raised for medicine innovation and governance practices to take better account of the “real-world” impacts of drugs on patients and caregivers and move away from the “narrowly” defined notions of clinical (cost-)effectiveness.

These examples reflect a growing misalignment between an evolving landscape and the current (institutional) structures of medicine’s innovation regime. These changes—and their resultant diminishing returns—stipulate stringent attention to broaden the scope of “return on investment” in medicine development to include new perspectives.

In line with a growing international discourse of public and patient involvement in health systems [36], “patient engagement” has been envisaged to serve as one means of approaching the chronic problems of medicine innovation. In the USA for instance, the Patient-Centered Outcomes Research Institute (PCORI) was established in 2010 and aims to improve the quality and relevance of evidence produced and used within health systems, with patient engagement forming a key program of the PCORI’s work. While practices vary across diverse contexts, the PCORI defines patient engagement as the “meaningful involvement of patients and other stakeholders throughout the planning, conduct, and dissemination” of healthcare research which, as Sigal et al. [37] (p.8) stated, “is becoming institutionalized and incorporated into several funding schemes”. Patient engagement is increasingly financed and organized by established actors, including the biopharmaceutical industry, patient organizations and networks, and regulatory (EMA) and health technology assessment (HTA) bodies.

Embedding meaningful patient engagement, however, still runs into systemic barriers, like that the inclusion of other (patient, “n = 1”) perspectives in medical research clashes with the dominant research methodology [38]. In the results, we describe how we, by means of reflexive methods, aimed to build systemic structures that presuppose and facilitate reflexive thinking by collaboratively designing a structured tool to monitor and evaluate “the return on engagement” in medicine development.

2.3. Institutionalizing Reflexivity for Second-Order Sustainability

In this section, we elaborate on the link between complex, systemic sustainability problems and reflexive approaches to (1) devise sustainability solutions and (2) work on embedding such niche innovations through institutionalizations. Based on that, we make a distinction between first-order sustainability, in which the focus is on implementing sustainability solutions, and second-order sustainability, in which reflexivity itself is institutionalized for a sustainable system.

A common denominator of complex or wicked problems is that they are inherently social and dynamic, implying a unique clash of ever-shifting values and perspectives [12], as is the case in both cases described above. Early conceptualizations have defined persistent problems [39] as complex (multiple causes and consequences exist), uncertain, difficult to manage, and difficult to grasp (unclear structure and boundaries), with no agreement on values and where every solution will produce negative side effects. More recently, Schuitmaker [8] defined persistent problems as systemically reproduced side effects of success factors, emphasizing regime actors, who act through institutionally and culturally paved pathways, and the regime rules and resources on which they base their actions. Addressing misalignment thus runs into the same complex pattern of actors and problem definitions the misalignment is in itself the product of.

Reflexivity, or more precisely the set of methods to enhance reflexive thinking by actors in collaborative settings, is a common approach to support system innovation (practices) in relation to complex (sustainability) problems. Voß et al. [40] used the term first-order reflexivity in reference to Beck et al. [41], who described the process in which modern society impacts itself negatively through modernization processes. Here, the outcomes of institutional and social structures have become disconnected from the original intentions underpinning their development. Second-order reflexivity then refers to awareness of this reflexivity of society, which is an essential prerequisite to tackle negative impacts of progress.

In the system innovation literature, research focuses on promoting second-order reflexivity within novel or niche initiatives in order to facilitate their embedding in existing systemic contexts [15]. The aim is to challenge institutionalized but undesirable practices, critically scrutinizing “taken for granted” notions by stimulating inquiry, dialogue, interactive learning, and learning by doing [14,42]. However, challenging “taken for granted” notions is not sufficient for the embedding of initiatives. As negative side effects of progress are themselves embedded in institutional and social structures [8], habitual ways of thinking and acting or routine discourse are themselves barriers for system transformation [25].

In the context of reflexive governance, Genus and Stirling [43] made the case that reflexivity needs to be framed as more than a quality of individual social actors in private processes of self-reflection and instead as a collective practice and capacity of governance networks, which can be enhanced through the application of shared codes of conduct and standards [40]. “Institutionalizing” can be seen as aligning an initiative to standards that govern the practice. Tolbert and Zucker [44], building on the work of Giddens [45], contended that institutionalization is a process of increasing structuration. They defined the process of institutionalization with three main phases. The first is habitualization, wherein an innovation is created by a small number of actors in response to a recurring problem and as such achieves some sort of habitualized form. As mostly uncoordinated activities, novel practices tend to lack stability and permanence, often disappearing with the actors who initially established them. Reflexive practices run into underling systemic barriers, and stakeholders’ reflexivity needs to be encouraged structurally to overcome them [38]. The second is objectification, in which some degree of social consensus emerges among organizational decision makers. This relies on broader work by initiating actors, such as problem and solution framing, persuasion, theorizing, making alliances, and mobilizing resources. A “collective rationality” about the innovation has to be formed and taken up, and variance of the innovation decreases. The last step is sedimentation, which relies on historical continuity and is characterized by the perpetuation of structures over a lengthy period of time. The structure has become normative, with changes in design rare and failures rather low [44]. At this stage, alignment between the sustainability solution and the institutional context through adaptation of the dominant (organizational) regime has occurred. A system innovation initiative is “successful only if the institutional setting of the initiative changes alongside the initiative itself” [46] (p.418).

Institutionalization of sustainability solutions by means of reflexive practices, however, does not equal sustainable implementation. First of all, renewed alignment of the regime to societal needs cannot be seen as “truly” sustainable. If a society aims to “meet the needs of the present generation without compromising the ability of future generations to meet their own needs” [3], the interventions themselves will have (unexpected) side effects and thus generate new misalignments between the regime and the needs of the population. Embedding of successful sustainability interventions then also needs an embedding of the reflexive element of the intervention itself to allow for continued adjustment. We frame institutionalized reflexivity, therefore, as second-order sustainability.

Two important properties are drawn from such a line of conceptualization. First, regime dynamics and forces alter interventions [47]. As institutionalized does not mean rigid, these forces lead to a disconnect between the intentions and effects. In order to stay true to the intentions, the underlying values need to be monitored. This is also known as internal accountability: whether an organization stays true to its intentions [24]. The reflexive question for actors needs to be “Are we still doing it right?” An organization needs to be resilient in the dynamic context in order to work according to the earlier-implemented, reflexivity-based quality standards [46]. Second, institutionalizing is a balance of power. If “sustainability is the dynamic equilibrium in which the system easily rests on its constituent values” [4], then the values are time- and place-dependent. The goals and direction of the reflexive sustainability intervention are then a mixture of these problems and what systemic elements actors reproduce [8]. For the continued meaningful inclusion of underrepresented voices, a shift in power balance is essential, which can be structurally facilitated by reflexivity, but only when institutionalized. The ingrained structures or routines must be continually reflexively questioned in order for the system to be sustained.

The question then becomes this: if “institutionalizing” can be seen as aligning to standards that govern the practice, how then can reflexivity be aligned with the practice of implementation? For second-order sustainability, embedding of reflexivity might be a structured approach, but institutional structures need to be in place to allow for actors’ reflexivity.

In this paper, we analyze two action research programs in different healthcare contexts (6 years in total), in which we facilitate the creation of institutional structures that support reflexivity. Both the perinatal care and medicine development contexts are characterized by a high level of standardization in organization and knowledge. In both cases, the effort became to standardize reflexivity, or to habitualize [44] reflexive problem solving by a core group in order to support the core group to reach objectification, with the ultimate aim of reaching sedimentation (although we acknowledged this was outside the scope of the projects, as such transformations take a generation [5]). Nevertheless, the aim was to create supporting structures for the envisaged system transformation by supporting care and research professionals to be reflexive by developing tools that align with their respective regime practices and values while inviting the actors to structurally reflect in order to enhance their practice, slowly transforming practices while being embedded in the regime. The strategy to co-create new structures together with regime actors resembles what Grin [15] described as second-generation initiatives, or experiments no longer exclusively undertaken by “alternative” networks, but with a leading role for established regime players. In such initiatives, the proximity of the regime may function as leverage to scale up niche experiments, supporting the embedding of standardized reflexivity through institutionalization. In these analyses, we focus on the challenges and opportunities faced in building the “set of methods to enhance reflexive thinking” among participants in response to sustainability threats. Both programs draw on reflexive methodologies for supporting system innovations to facilitate (1) processes of reflection among project participants and (2) the creation of structures to embed these processes in organizational networks.

3. Materials and Methods

We report on the challenges and opportunities faced when we attempted to institutionalize reflexivity in two consecutive participatory research programs, in which the aim was to address unsustainability in terms of misalignment and diminishing returns. In both programs, we collaboratively designed practices with regime actors through reflexive methodologies in order to enhance embedding of both the niche innovation and the reflexive element. The first is the (policy-driven) transformation of perinatal care into a more integrated model with high continuity of care. The second is the European IMI-PARADIGM project aimed at supporting the implementation of meaningful patient engagement in medicine development.

In the two cases, the participatory research approach of Reflexive Monitoring in Action (RMA), as developed by Van Mierlo et al. [14], was applied both as a research methodology and as a template for institutional structures that support reflexivity. We used this interventionist approach to reflexively monitor the formation of the obstetric partnerships, co-develop a monitoring and evaluation tool within PARADIGM, and develop RMA-based tools to support professionals to work reflexively after the program ends. RMA emphasizes the importance of integrating reflection in the process and promotes reflexive governance, as it encourages actors to scrutinize and reconsider their underlying assumptions, institutional arrangements, and practices in order to steer toward sustainability [48]. Next to that, RMA aims to stimulate system learning, in which actors learn to (1) recognize the “wickedness” of recurrent problems, (2) acknowledge the systemic barriers but redefine them into opportunities, and (3) design activities that can contribute to systemic change [49].

Reflexive monitoring follows the circle of observation, analysis, reflection, and adjustment of system innovation initiatives. Research activities undertaken by the monitor include interviews, organizing meetings, and analyzing notes from the meetings and transcripts from the interviews in order to guide the reflection sessions in which action plans are adjusted [14,50,51]. For this paper, we analyzed the transcripts, observation notes, recordings, meeting reports, action agendas, and other materials we gathered in our role as the monitor in order to elicit occurrences of reflexive thinking (or explicit rejection) and the level of habitualization and objectification of this way of working, as described per case below. This analysis thus focused on both the occurrences of reflexive learning on an individual level as well as on the contextual (cultural and institutional) reasons and factors that explain (the lack of) reflexive learning by participants in both projects. We have published elsewhere the results of the programs in terms of integrated care and impact of patient engagement [26,31,52,53], while in this paper, we focus on the overarching goal of institutionalizing reflexivity.

We used theories on institutionalization and reflexivity to recognize instances of reflexive learning in relation to institutional structures. Themes were drawn around enabling and hampering factors underlying embedding of reflexivity during the two processes in which we employed the principles of RMA, including the relationship of facilitated project activities to wider institutional practices (impact or transformative aspect) and the relations between reflexive practices and institutional resources (i.e., standards).

In the Results section, we present the case analyses separately in order to maintain (1) the logic of how (stages of) case one informed the subsequent stages and case two and (2) to improve the internal validity of the analysis.

3.1. Case 1: Supporting Obstetric Partnerships to Reflect on Perinatal and Client-Centered Care

From 2014 to 2016, we conducted the action research study “North West Netherlands Aligned” (In Dutch: Noordwest Nederland op één lijn), which supported system innovation toward an integrated perinatal and maternal care system by enhancing the collaboration and organizational integration of birth care professionals within obstetric partnerships, with a special focus on client-centered care. The research team, which included two of the authors (T.J. Schuitmaker-Warnaar and J.E.W. Broerse), supported obstetric partnerships for 1 year to build a shared vision on optimal perinatal care and concrete strategies and plans to reach these objectives through attaining short-term goals. The study was embedded within the Maternity Care Network Northwest Netherlands (MCNNN), a regional organization of obstetric partnerships.

The MCNNN is the largest consortium in the Netherlands with around 20% of the national total of births, and at the time, it consisted of 18 active obstetric partnerships, which were all included in this study. Maternity care assistants, primary care and clinical midwives, and obstetricians were always present in meetings and reflection sessions. Some partnerships invited other professionals, like nurses, pediatricians, residents, general practitioners, or youth healthcare professionals. Interviews were held with partnership members, who were selected in consultation with the coordinator of the MCNNN in order to assure that all partnerships and professions would be represented. We held 73 semi-structured interviews and 7 questionnaires among professionals (2 in the whole region in 2015 and 2016 and 5 in specific partnerships as part of the RMA), investigating the desired form of (and barriers and facilitators to) integrated care. Based on the analyses, we organized 18 reflection sessions, with 5 partnerships selected based on geographical spread, implementing the Dynamic Learning Agenda (DLA) [50,51,54]. As for operationalization of the RMA approach [50], the DLA was used to manage the situated dynamics between different modes of coordination, as it reframes contextual barriers to transformation (toward the intended direction) into learning goals and solutions to overcome them. A DLA explicitly requires reflection from participants as it asks for (1) reframing a problem into a vision on integrated care, (2) systemic barriers for reaching that vision, (3) which other stakeholders can be included, and (4) in what way to work on a solution and the concrete actions to take, culminating in participants drawing up “learning questions”.

Building on the lessons of the program, the DLA approach was developed into a reflexivity supporting toolbox, and we organized four training sessions with four other partnerships who joined voluntarily to explicitly instruct and guide these partnerships to implement the DLA in their own practice, including the reflexive element.

3.2. Case 2: Building a Tool for Structured Reflexivity in Medicine Development

Within the IMI-PARADIGM project, which took place from March 2018 to November 2020, one of the work packages (WP3) worked on developing a tool to monitor and evaluate “the return on engagement” in medicine development. The overall goal of patient engagement throughout the research and development of medicines (PARADIGM, https://imi-paradigm.eu/ accessed on 18 October 2021) is to develop safer, more effective treatments for patients closely related to their needs and to deliver them faster and more efficiently. In WP3, the Patient Engagement Monitoring and Evaluation Framework (with metrics) was created in order to support organizations in medicine development in their evaluations of the outcomes and the impact of patient engagement in three key phases of medicine development: research priority setting, clinical trial design, and early dialogues with regulatory authorities such as the European Medicines Agency and HTA bodies. The team responsible for the execution of WP3 included three of the authors of this article (T.J. Schuitmaker-Warnaar, C.J. Gunn, and J.E.W Broerse).

The aforementioned RMA approach [14] was applied to develop and refine an M&E framework. The researchers’ role was to partner with companies and organizations to facilitate discussions and support early attempts to monitor and evaluate their patient engagement initiatives by identifying and selecting relevant metrics for measuring PE impact. Twenty-four case studies of patient engagement initiatives were followed, which elicited in total 47 interviews and 23 reflection sessions with representatives of the pharmaceutical industry, patient organizations, and health governance organizations. The outcomes of the case studies fed into the development of an overall M&E framework among a working group consisting of partners from these stakeholder groups. As well as monthly 1-hour teleconferences (TCs) between the working group across a 2-year period, this process included two 1-day multi-stakeholder workshops (March 2019 and April 2020) which developed consensus on the final overall framework tool. Additionally, two questionnaires were distributed to the working group and their patient engagement networks, with one focusing on their initiative and its (desired) impact and the other about the suitability of the overall framework tool for measuring PE. The framework was presented at three “patient engagement open forum” events, which also provided input from all stakeholder groups.

4. Results

4.1. Institutionalizing Reflexivity in Perinatal Care on the Level of Practitioners

Through the 18 reflection sessions with the participating obstetric partnerships structured by the Dynamic Learning Agenda (DLA), the professionals habitualized reflexivity with the help of the researchers in their role as a reflexive monitor, guiding the DLA process. The partnerships gained insight into the success factors and barriers for integrated care with “client-centricity” and used these insights to challenge institutionalized but undesirable practices and scrutinize “taken for granted” notions by following the four steps of the DLA approach. These reflexive insights were used to create and implement a structured plan to strengthen the care in their region, improve collaboration, and give substance to the client-centered care.

The researchers guided the process of drafting and acting on a DLA by focusing on the contents and quality of care first—something the professionals largely agreed on—laying the groundwork for implementation of the following reflexive question for actors (concerning internal accountability): “Are we (still) doing it right?” In the sessions, professionals collaboratively placed visions on quality at the center of mind maps, operationalized their perspectives on high-quality care in sub-elements and formulated barriers and possibilities for these elements in practice. In other words, they were asked to specify what would the collaboration look like and what would be the possibilities to organize this, given the fact that the mentioned barriers exist. Professionals then formulated reflexive learning questions like "How can good cooperation in the partnership be achieved through shared responsibilities when there are no clear cooperation agreements and protocols on this contribute to the “protocols mountain”?” (VSV 3, Reflection Session 1) and "How can we as healthcare providers take into account the diversity in backgrounds and personal preferences of the client population, given that we want to (and must) deliver a protocoled care process?" (VSV 4, Reflection Session 1). Based on these and other guiding questions, they formulated action plans such as joint intakes of midwives and gynecologists, accompanying each other in the workplace, joint training, drawing up a birth plan with each client (and acting accordingly as care providers), training for the partnerships to generate more from client conversations and structurally embed input from clients in formal meetings (through interviews, surveys, and a client council). In adjusting the action plans based on the results of these actions, the professionals already reflected on their own practice and standardized, embedded, and sedimented modes of working.

Over the course of the project, obstetric partnerships professionalized in terms of the trust between (levels of) professionals, integration of work routines, and overall organization (two questionnaires (one in 2015 and a follow-up in 2016) evaluated the progress in collaboration). They transformed from one large partnership into a smaller board with a mandate and thematic working groups to continuously evaluate care paths and practices (analysis of evaluative interviews in 2015–2016). The reflection sessions showed that the DLA helped to make clear agreements regarding role division, responsibilities, and overall decision making. These agreements and the layered structure of the obstetric partnerships ensured more involvement of members and increased efficiency and effectiveness.

However, the further institutionalization of reflexivity ran into existing structures. In the interviews and reflection sessions, birth care professionals mentioned feeling (unspoken) tension, hierarchy, and mistrust between members, partly arising from the different views between and within different professional groups. Physiological versus pathological views of pregnancy and birth remained a barrier to interprofessional collaboration. Core groups like boards or working group members of obstetric partnerships often managed to overcome this barrier by applying reflexive methods, but in daily practice—regularly and unexpectedly—it became relevant again among the wider group of professionals in the partnerships. In general, the biggest challenge became national developments and news items on those, agitating professionals withing the partnerships and re-introducing distrust. The reflexive process with a focus on quality of care, which all participants could agree on, became disrupted by a policy focus on financial structures, causing care providers to feel that they were forced to think in terms of their own interests instead of the common interest for the pregnant woman.

Furthermore, the process of institutionalizing the reflexive practice of integrating the client perspective did not reach the level of habitualization. Client centeredness was discussed in all reflection sessions and interviews, and its importance was recognized. Partnerships have set up structures for client consultation and participation, mostly through working groups where clients are invited to provide feedback on care paths or protocols. However, structurally shaping client participation remained difficult, in particular due to the lack of knowledge on and experience in client involvement and the fact that caregivers have to do this on top of their already burdensome workloads, as mentioned in sessions and interviews.

Based on these observations, the researchers created a toolbox called “Gezonde geboortezorg met de dynamische leeragenda” (Healthy birth care with the DLA) [51] that supports professionals working autonomously on applying the DLA within their own obstetric partnership, including the reflexive element. This could potentially increase both the scope and perceived ownership of the reflexive process to explicitly make the improvement of maternity care a project of the care providers themselves. Following the difficulties described above, we assumed "cold" implementation (simply making available and distributing the toolbox) would not yet offer sufficient tools to allow partnerships to work with the toolbox independently. We thus set up an educational program of 1 year together with ZonMW (as part of funding for implementation of successful programs) and the MCNNN, designed to help obstetric partnerships in the Netherlands to work with the toolbox. In this implementation project, we trained representatives to take up the reflexive monitor function in their own partnership to be able to enhance the continuity and quality of perinatal and maternal care with the support of the toolbox and a website (www.gezondegeboortezorg.nl accessed on 18 October 2021) with a forum for questions and discussion. The four sessions helped to further develop the toolbox based on the experiences from the field in such a way that the representatives of the obstetric partnerships learned to work with the DLA without external support and integrate this with their regular routines, thus habitualizing the reflexive element of the tool.

The toolbox is still (anno 2021) regularly downloaded from the website and other platforms, although the extent to which the reflexive elements are still being applied is unknown. After the different research phases (the reflexive monitoring of the partnerships, the creation of the toolbox, and the training in the use of the toolbox), the following was concluded to be essential for habitualization:

• Organization of several consecutive meetings accelerates the process, because results are periodically evaluated and further developed;
• External and impartial guidance remains useful;
• Participation in physical meetings is stressful but concurrently motivating because of the interaction and tailor-made solutions;
• The use of many interactive, playful working methods increases the yield;
• Interaction between members of the same partnership greatly helped continuation after the program;
• Accreditation (or another tangible incentive from existing institutions) of meetings is an important extra motivation to participate.

In particular, this last lesson implies that broader institutional support remains useful for objectification.

4.2. Institutionalizing Reflexivity in Perinatal Care on the Regional Level

To support institutionalization of the progress of the obstetric partnerships, the steering committee of the regional Maternity Care Network Northwest Netherlands (MCNNN) was also reflexively monitored through recurrent reflection sessions, in which the research team presented the results of the interviews, questionnaires, and progress of the obstetric partnerships. In line with the idea of “institutionalizing reflexivity”, the steering committee was taken as a research object with the aim to create a reflexive knowledge infrastructure capable of providing support to continuous learning within the obstetric partnerships. The steering committee was evaluated by means of (1) observations of meetings, (2) interviews with members, (3) questionnaires, and (4) action-oriented reflection and brainstorming sessions. Visions on perinatal care, tasks, activities, and points for improvement to enhance reflexivity were discussed. The researchers explicitly invited the members to discuss contents of care and not see the committee as a group of representatives from different echelons. The aim was to create a “knowledge network” centered around learning and further development.

The evaluation concluded that the steering committee, as an overarching multidisciplinary body, made an important contribution within the MCNNN. Region-wide research was facilitated, for which individual partnerships did not have the time and financial scopes. Barriers to good care for the entire region were discussed jointly, and collaboration between birth care providers was emphasized and shaped. Various activities and products supported this, including the regional perinatal consultations, the regional protocols, the app, the website, the newsletters, and the studies.

Even though the birth care providers from different echelons within the steering group worked together constructively, friction still arose following national developments, as happened within the obstetric partnerships. The collaboration was further complicated by the rapid changes of the steering committee members and the national pressure on integrated maternity care with associated full personal professional agendas, which precluded the time and energy spent on the steering committee. Important points for attention in the collaboration were increasing the effectiveness of decisions, limiting the individual burden, and offering the space and opportunities for steering committee members to find each other and contribute ideas so that the members remain motivated.

4.3. Difficulties in Reflecting with Professionals

As described, the program itself was rather successful in stimulating reflection on current practices and re-designing new perinatal care practices based on integrated care. However, it remained difficult to institutionalize reflexivity. Even though core groups like the boards of obstetric partnerships adopted the approach, we acknowledge that (1) the process required significant effort to encourage participating professionals to take an actual reflexive stance; (2) after the researchers left, the DLA was usually not maintained; (3) changes in board members further eroded adoption, as they were not trained by the researchers; and (4) boards have tried to organize reflexive sessions with their wider partnerships but were largely unable to convince their colleagues to continue using the reflexive elements of the DLA, reverting to more practical agendas.

Underlying these impediments, we see several systemic features being reproduced, underlining the wickedness of not only first- but also second-order sustainability. First of all, a lot of professionals do not want to “reflect” on their work. When sessions were organized, several participants actively resisted participating in the exercises. A gynecologist, voicing his aversion against “vague and non-scientific” methods, left a reflection session after stating “I’m not going to write on post-its!” (VSV 1, Reflection Session 1). This particular gynecologist repeatedly asked the researchers what the actual “goal” was, a question posed often by others. The act of reflecting was experienced as “vague” as it (by design) expanded the scope of problems before specifying solutions through the creation of action plans. Interestingly, the researchers noticed that the more educated the professionals were, the more they resisted “vague” work that did not seem to have a clear endpoint. On the other hand, some professionals with professional education classified formulating learning questions as part of the DLA as “too academic”.

In the end, however, the organized sessions did add to a more reflexive stance and were perceived as useful by the participants. One participant remarked in a training session on the DLA that “We have been discussing this issue for over 15 years and this is the first time I know what to do next!” (VSV 5, Reflection Session 3). Devising learning questions is a hurdle that takes effort, which needs a well-trained facilitator. After this first step, actions and barriers are discussed, and “reflecting” becomes tangible, making it more interesting for professionals to participate. This, however, is not synonymous with institutionalized reflexivity.

Institutions forced themselves into niche practices regularly. As described above, the continuous focus on financial integration put pressure on the professionals. Mandatory membership of obstetric partnerships was introduced simultaneously with the message that integrated care and bundled payment were the final goal, diverting attempts to co-design new collaborations to complicated discussions on integration. Basic principles for good interdisciplinary teamwork, like leadership, management, vision, and mutual trust, still required development within the partnerships (e.g., [26]).

Overall, after the various stages aimed at embedding reflexivity, we concluded that better alignment with extant institutional structures might be beneficial for achieving this. More structures need to be in place to allow for different forms of reflexivity. This idea was taken up and incorporated into the proposal for the PARADIGM research project, in which the aim was to draw on the feature of an institutionalized preoccupation with “return on investments” to build structures that presupposed reflexive thinking.

4.4. Institutionalizing Reflexivity through the PARADIGM Framework Exercise

To institutionalize reflexivity, we sought to align with existing structures. The purpose of the PARADIGM project, as introduced in Section 2.2, was to develop a range of agreed-upon resources and tools for strengthening patient engagement practice, one of which was a framework of evaluation metrics. Through interviews and monthly TCs, the noticeably “louder” industry actors expressed interest in developing a standardized set of metrics that would conclusively “prove” the value of engagement (phrased as the “return on engagement” in popular industry discourse). In the emerging context of patient engagement, where its “embedding” in drug development is of key interest to initiatives and projects like PARADIGM, interests in showcasing and selecting “best practices” undergirded these interests in developing such “rigorous” assessments of the value of patient engagement. Particularly from an industry perspective, impact metrics for patient engagement were already relatively well conceived at the beginning of the project. During the first “Patient Engagement Open Forum” in 2018, participants articulated the importance of various impact indicators related to clinical trial design, including “lower recruitment time” (of participants to a clinical trial), “more diversity (of trial participants) in recruitment”, “retention (of trial participants) rate”, and “fewer trial protocol amendments”.

While these kinds of perspectives on relevant criteria remained relatively unchanged throughout, a “reflexive approach” would need to pay attention to different interests and values in defining impacts for patient engagement, resulting from different actor commitments to medicine development that could not be simply “aligned” in a universal or objective framework of impact metrics. With this in mind, we attempted to facilitate a process that would highlight the relevance of acknowledging and appreciating different stakeholder values and interests in PE in order to generate a final framework that would both align with the voiced interest in standardized metrics and reflect these multiplicities. We attempted to inscribe reflexivity in a seemingly standardized tool.

The approach taken was to construct a framework through a process of negotiating between different perspectives that are invariably present in PE. This process was organized by developing a “general” framework structure that could be adapted to different individual PE initiatives through a tailoring process, following the structure and methodology of RMA.

4.4.1. Designing the Framework Tool

To align the standards with reflexive thinking, we used the tailoring process to explore the multiplicity of perspectives in naming and framing ‘impacts’ of patient engagement. In other words, the selection of meaningful and feasible metrics for a patient engagement initiative invited participants to reflect on the underlying factors or conditions underlying PE activities and their conceived value. These types of reflection were facilitated by the research team during the testing phase in the construction of each case initiative’s “tailored” framework. In developing these tailored frameworks, the organizers of the initiatives were invited to explain under what grounds certain metrics were “relevant” and “feasible” to their practices based on their overall goals or objectives.

Some cases showed the benefits of the tailoring process in its ability to stimulate reflection on how “impact” may be influenced by engagement contexts, including the different stakeholder interests at play. Developing tailored frameworks under the guidance of the reflexive monitors enabled different levels of reflection in different engagement contexts. Many case studies noted the value of the framework approach in relation to the way it stimulated reflection, such as how the exercise was “Thought provoking with good questions for reflection” (Case 21) and “Useful as a prompt for reflection” (Case 5). Especially in settings where multiple stakeholders could be brought together, the processes of reflection on the different frames that influenced the criteria for PE impact were appreciated by the participants (Case 4). Other partners found additional value in the exercise, as it helped them gain “clarity” over the complex processes of patient engagement in their organizations, enabling them to learn about why patient engagement initiatives are being conducted in the first place.

Some partners found it burdensome to work through the long list of reflective questions in developing a tailored evaluation framework. In several cases, the exercise fell flat when facilitators asked how “contextual factors” might influence an engagement initiative. Here especially, more guidance was sought by partners in understanding how these factors played a role in measuring patient engagement.

Furthermore, while the partners recognized the value of defining metrics using such a “co-creative” approach with different stakeholders, many felt restrained by organizational structures (e.g., a lack of time to dedicate to organize complex evaluation and complicated multi-stakeholder exercises being less pressing than more immediate priorities of business). During the testing phase, feedback from one organization was that “the group discussion version would be preferable but impossible to envision due to time pressures" (Case 1). These issues are linked closely to perceptions of the “complexity” of evaluations, to which the concerns of the partners were often related. In the feedback from the testing phase, it was noted that “most common issues were that [the framework] needed simplifying to suit the time pressures of regular organizational work” (Case 14). The interests of several partners in making evaluations “simpler”, aligned with the concerns of “survey fatigue”, related to the burdens of being requested to complete too many evaluations in professional practice (monthly TC number 12). Furthermore, at an open forum (2019) workshop on PE metrics, one participant suggested the use of the net promotor score (NPS) in order to make “straightforward comparisons” in the evaluation of different PE initiatives which, while yielding some insight into how happy participants were with an engagement initiative, would elicit nothing about the reasons why, which is essential information for reflexive processes.

4.4.2. Implementing the Final Framework Tool

A central issue in habitualizing contextual reflection was thus the tangibility of contexts and the practicality of critically reflecting in routine professional settings. We saw that developing tailored frameworks under the guidance of the reflexive monitor enabled different levels of reflection in different engagement contexts. However, the final tool is intended for use by regime actors without facilitation.

The final, “full” metrics framework described a large list of evaluation metrics (n = 87) that asked users to select which metrics were most meaningful to them based on their own engagement contexts, with the predefined “sets of metrics” based on the objectives of engagement, providing a “rough guide” for selecting metrics that align with particular aims or intentions of conducting patient engagement. Like the case studies, the final tool asked users to select metrics relevant to their own practices from the full list, as opposed to defining a conclusive or universal metrics set. The guidance developed therefore encouraged flexibility in the sense that it required some reflection on organizational contexts in order to produce a meaningful set of metrics for those who are using the framework. Crucially, without this type of reflection, the standalone indicators would certainly feel less “rigorous” than the systematic assessments that many in the project showed interest in developing. While some initial users were enthusiastic about the tool, the complex process of adapting the framework to suit local needs was still experienced as difficult to achieve coherently in practice.

The final tool attempted to inscribe different stakeholder perspectives within the predefined sets of metrics. All sets contained different metrics defined as beneficial for all stakeholders. The final tool also encouraged the inclusion of all relevant perspectives during the process of metrics selection. However, several partners in the working group were still interested in developing a more “global” set of “must-have” metrics (workshop observations), which would reduce the need for contextualized selection by users. When reflecting on the framework’s “implementation”, the participants added that the users needed to be more comprehensively guided on how to measure the impact of PE by specifying the most relevant methods for measuring different metrics. Others suggested that the metrics we had developed should form a “soft guide” for organizations to adopt and adapt where necessary, rather than being a strict standard for the field of PE.

4.5. Conceptions of the Role and the Nature of Patient Engagement Evaluation

The development of the framework tool, which encompassed the (facilitation of) various instances of reflection and learning, showed a slow adaptation of the notion of meaningful engagement, meaning that for engagement to yield value, the inclusion of and reflection on multiple perspectives is essential. By the end of the project, this particular stipulation was widely acknowledged.

Furthermore, shifts were seen in the way that the evaluations of patient engagement could be organized to enhance meaningful or valuable engagement practices. In contrast to the “conclusive” impact evaluations envisaged by some, the project leader toward the end of the project began to advocate the value of organizing evaluation as an “internal monitoring” exercise in order for organizations to track their own progress (in achieving valuable or impactful patient engagement) over time (monthly TC number 15).

The research team advocated throughout that local tailoring processes were “better” when performed collectively, including the values of different stakeholders, as this enabled a reflective process more readily. In congruence with previous findings, reflection was found to be more accessible with the support of deliberate (trained) facilitation, such as during the case studies of the testing phase in this project, where researchers could probe the grounds under which the evaluation metrics would be relevant or feasible to different patient engagement practices. At this stage, it remains to be seen how the final framework will be used (i.e., how the envisaged “reflexive approach” to developing metrics for patient engagement will be adopted, adapted, avoided, or resisted by different medicine development actors). Ongoing insights into these dynamics will tell us more about whether and how the strategy of organizing reflexivity proved productive to patient engagement and medicine development.

4.6. Understanding and Supporting Sustained Forms of Reflexivity

Ultimately, developing PARADIGM’s framework for the monitoring and evaluation of patient engagement was an exercise in compromise, with enough coherence for a multi-stakeholder community in medicine development to have their divergent interests in patient engagement (partly) integrated. This required drawing upon features of the system (in this case, a rhetoric of “return on engagement” and a vernacular of impact metrics) to support the institutionalization of more reflexive approaches. The fact that several of the “final” metrics were already well-defined at the start of the project shows that the task was more about striking a balance between different stakeholder interests and values in developing some forms of shared meaning in patient engagement through a process of objectification. This included the development of a collectively defined framework for evaluation between different actor perspectives in patient engagement.

We sought to include meaningful and sustained processes of reflection (the reflexive element) as part of the institutionalization of patient engagement, which came about through the process of tailoring a general (collectively defined) framework into a specific local PE context. We included actors from all levels of the regime, including regulatory authorities such as the European Medicines Agency and HTA bodies, and aligned with their need to stimulate pharmaceutical companies to include more patient-centered outcome measures to be sent for evaluation next to the regular clinical ones. The developed reflexive standards were thus designed to be taken up by—and thereby enforced through—these regulatory agencies. Although we successfully built a tool for standardized reflexivity, it remains to be seen whether these standards will be taken up by the regime.

5. Discussion

The aim of this paper was to provide insight into the possibilities of embedding reflexivity into institutions to support second-order sustainability. We reported on how we applied reflexive methods in two consecutive participatory research programs, in which the aim was to address unsustainability in healthcare in terms of misalignment and diminishing returns. In these analyses, we focused on the challenges and opportunities faced in building the “set of methods to enhance reflexive thinking” among the participants. Both programs aimed to facilitate (1) processes of reflection among the project participants on an individual level and (2) the creation of structures to embed these processes in organizational networks.

In both cases, we were mostly successful in organizing reflexivity within the practices and embedding standardized reflexivity through institutionalization by aligning with regime elements, which was an exercise in compromise, reaching habitualization and some extent of objectification. In perinatal care we, for instance, drew on tangible incentives from existing institutions, and in PARADIGM, we created a framework with standardized metrics to align with industry needs. We supported professionals to enhance internal accountability [24] and to monitor underlying values. On an individual level, standardized tools with clear steps proved supportive in inviting participants to reflect on their “regular way of doing” because these drew on the systemic element of project-based and solution-oriented professionals in perinatal care by “standardizing” a reflexive process through the Dynamic Learning Agenda and with the M&E framework by enabling interactive learning and priming stakeholders with a broader problem definition. We were able to create “reflexive standards” [55], implementing the idea that reflexivity can be part of standardization by explicating where reflexivity is situated and who gets to do the reflecting, as well as what is being reflected upon.

In general, the participants in both projects valued the exercises when performed. In perinatal care, for instance, a participant knew what to do next after 15 years of deliberation, and in PARADIGM, the participants valued the “clarity” the framework brought for complex processes of patient engagement. However, the participants also expressed hesitation in engaging with reflexive questions. This aligns with both the increasingly acknowledged value of reflexivity [16,19] as well as with the need for more training in different forms of problem solving, like in the case of shared decision making and supporting doctors to consider financial arguments in care decisions [56,57], where reflection on the medical practice is a prerequisite. Resistance to reflecting is recognized in the literature, which emphasizes the complexity of the reflexivity concept and the practical complications to the development of reflexive skills [58].

Reaching habitualization and some extent of objectification, however, did not lead to sedimentation of the new reflexive structures, even though we actively worked on the necessary organizational networks by having relevant regime actors participate in the projects. The willingness of actors to reflect on their daily activities is essential, but the space to maneuver is largely determined by existing institutions. An important systemic feature of the health research system—the results-oriented medical culture—impedes the willingness and ability of participants’ reflexivity, and alignment with contextual factors to enhance the success of system-innovative initiatives is not enough for institutionalization [38]. In the perinatal care case, the political pressure made professionals less inclined to reflect teamwise. One core assumption of the RMA approach is that focusing on contents and goals helps to overcome strategic conflicts [14], but this has not been thoroughly examined empirically, and our findings suggest more research is needed on strategies to deal with political conflicts disrupting the reflexive process.

In the PARADIGM case, the participants mentioned, for instance, a lack of time to dedicate to organizing complex evaluation as an example of limited space to maneuver, or that complicated multi-stakeholder exercises are less pressing than the more immediate priorities of businesses. Boström et al. [59] underscored that reflexive learning and continuous reform is challenging because institutional structures tend to reproduce themselves, unless they are self-confronted by the side effects of their own operations. In PARADIGM, we not only focused on professionals as change agents but explicitly aligned with (the needs of) regime actors, including regulatory authorities, following the lessons from case one and organized a “second-generation initiative”, where the closeness of the regime can function as leverage to scale up niche experiments [15]. The developed reflexive standards are designed to be taken up by, and thereby enforced through, these regulatory agencies.

6. Conclusions

In the perinatal care and PARADIGM cases, we objectified reflexivity, supporting the transformation to a more sustainable perinatal care system and medicine development process. The creation of reflexive standards [55] seems promising for the embedding of reflexive practices, but opportunities for broader institutionalization depend on the implementation and enforcement of these standards by regime actors.

In future research, it may prove useful to consider how the (flexible) monitoring of accountabilities can be supported through different forms of intervention. Next to enforcement by regulators, reflexive standards might be useful for public benchmarking of companies in terms of meaningful patient engagement [59], forcing companies to review their own assumptions and commitments and expose them to critical and public contestation. In line with the idea of regime stress opening up space for niches to scale up [5], this increases opportunities for practices that apply the developed reflexive standards. It appears that for second-order sustainability (i.e. sedimentation of reflexive practices) pressure remains important.