Next Article in Journal
Remote and Technology-Based Dialogic Development during the COVID-19 Pandemic: Positive and Negative Experiences, Challenges, and Learnings
Previous Article in Journal
Educating Clinicians to Improve Telemedicine Access for Patients with Limited English Proficiency
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Challenges
Volume 13
Issue 1
10.3390/challe13010001
challenges-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Scoping Review Examining Governance, Co-Creation, and Social and Ecological Justice in Living Labs Literature

by
Lindsay P. Galway
1,*,
Charles Z. Levkoe
1,
Rachel L. W. Portinga
1 and
Kathryn Milun
2
1
Department of Health Sciences, Lakehead University, Thunder Bay, ON P7B 2X5, Canada
2
College of Arts, Humanities, and Social Sciences, University of Minnesota—Duluth, Duluth, MN 55812, USA
*
Author to whom correspondence should be addressed.
Challenges 2022, 13(1), 1; https://doi.org/10.3390/challe13010001
Submission received: 27 October 2021 / Revised: 27 December 2021 / Accepted: 29 December 2021 / Published: 31 December 2021
Browse Figures
Review Reports Versions Notes

Abstract

:
Living Labs (LLs) are increasingly being used as an approach to address complex sustainability-related challenges. Inspired by existing knowledge and practice gaps, calls for further examination of governance and co-creation in relation to LLs work, and our experiences in the Lake Superior Living Labs Network, we conducted a scoping review of the recent (2015–2019) LLs literature. This review focused on peer-reviewed LLs literature aimed at addressing sustainability-related challenges and involving universities as key collaborators specifically. This scoping review addressed the research questions: how are LLs conceptualized, described, and applied? how are LLs governed? How is co-creation supported in LLs work? and, are social and/or environmental justice considered in LLs work? From the 729 citations gathered in the electronic database searches, 48 papers were identified as relevant through the screening and eligibility assessment. We found that this literature is growing rapidly, highly interdisciplinary, and predominantly taking place within European urban centres. We summarize the findings in relation to our research questions and outline implications for interrogating governance, unpacking co-creation, and working towards social and ecological justice in LLs research and practice. We conclude by outlining four key research directions to advance LLs work, including, (1) expanding research across a greater diversity of settings; (2) examining and analyzing governance and power dynamics; (3) exploring how learning evolves via co-creation; and (4) examining how universities are impeding and/or supporting advances in relation to governance, co-creation, and justice in LLs work.

1. Introduction

Living Labs (LLs) are increasingly being used as an approach to address complex sustainability-related challenges. Although there exists substantial diversity in terms of how LLs are defined, understood, and practiced, they are generally spaces and places for ‘innovative’ and collaborative research and action focused on co-creating solutions in the context of real-world environments [1]. LLs have been applied across a range of sectors and for myriad purposes [2]. Given the growing interest in LLs, and the need to synthesize emerging knowledge and evidence, others have conducted reviews of the LLs literature over the past decade [3,4,5,6,7,8]. While these existing knowledge syntheses have advanced our collective understanding and generated important findings to advance and inform LLs work, key knowledge and practice gaps exist and calls remain for additional reviews of the LL literature. Greve et al. [5] for example, argue that “...the landscape of LL research remains largely unclear and underexplored”. Houssain et al. [6] question the extent to which LLs have received adequate scholarly attention and examination. Inspiring this study specifically, von Wirth [9] and Schuurman [10] argue that the underlying processes, methods, and tools that support LL governance and co-creation remain poorly understood. Moreover, despite the growing recognition that sustainability and justice are deeply intertwined and the subsequent imperative to center social and ecological justice in sustainability work [11,12], the ways in which LLs research and practice engage with and advance justice has not been examined to date. Addressing these gaps is essential if LLs research and practice are to make meaningful contributions towards the grand challenge of our time: co-creating healthy, sustainable, and equitable transitions.
Building on our collective interests in experimenting with a LL approach and learning-by-doing, the authors of this paper are part of the core team that established the Lake Superior Living Labs Network (LSLLN) in 2018 [13]. All co-authors are currently members of the LSLLN steering committee. The LSLLN is a platform to connect academics, students, and community groups to develop and expand relationships and LL initiatives across the Lake Superior watershed. The overarching goal of the LSLLN is to collectively increase the impact of sustainability-related teaching, research, and action with a focus on issues and opportunities at the nexus of water, land and food, climate and energy, and well-being (See Figure 1). We also aim to have a stronger collective influence on issues of social and ecological justice and sustainability regionally, nationally, and internationally while sharing our collective learnings about using a LL approach across nested scales (i.e., campus-community-watershed). The LSLLN functions at a watershed scale and is rooted in four local hubs spanning the North, East, South, and West directions of the Lake Superior watershed: Thunder Bay (Ontario, Canada) in the North, Sault Ste. Marie (Ontario, Canada) in the East, Houghton (Michigan, US) in the South, and Duluth (Minnesota, US) in the West. Our team considers the network itself as a LL and the four local hubs are designing, implementing, and learning from a range of LL projects.
Inspired by key knowledge and practice gaps identified in recent reviews of the LLs literature as well as our ongoing work in the context of the LSLLN, we conducted a scoping review of the recent (2015–2019) LLs literature. Our review focused on the recent LL literature aimed at addressing sustainability-related challenges and involving universities as key collaborators specifically, to deepen our understanding of governance, co-creation, and social and ecological justice within the emerging LLs scholarship. In this paper, we describe the scoping review process used and summarize key findings in relation to the following research questions: (1) how are LLs conceptualized, described, and used? (2) how are LLs governed? (3) how is co-creation supported in LLs work? and (4) are social and/or environmental justice considered in LLs work? We discuss the relevance of our findings in relation to existing scholarship and our own LSLLN work. We conclude by identifying what we see as four key directions for future LL research.

2. Methods

Scoping reviews are a form of knowledge synthesis used to map existing literature on a specific topic and are particularly useful when a topic is complex and heterogeneous [14,15]. This is certainly the case with LLs literature. “Scoping reviews are gaining in popularity across many fields as authors look to replace the traditional narrative literature review with more rigorous, transparent, and replicable methods for synthesizing research” [16]. Compared to systematic literature reviews, the scoping review process allows for a broader search strategy while still following a clearly outlined, transparent, and reproducible approach to enhance rigor. Scoping reviews are most appropriate for emerging and interdisciplinary bodies of literature [17]. In this study, we applied the 5 stages of the scoping review process as outlined by Arksey and O’Malley [17]: (1) identifying the research question(s); (2) identifying relevant studies; (3) selecting studies for inclusion in the review; (4) data extraction; and (5) collating, synthesizing, summarizing, and reporting results. In practice, these phases unfold iteratively rather than linearly. Also noteworthy, following the recommendation by Levac et al. [18], the members of our core scoping review team met throughout the process to review progress, address challenges, and discuss discrepancies.

2.1. Identifying the Research Questions

A draft set of guiding research questions were developed by the co-authors drawing on existing LLs literature and our experiences designing and implementing the LSLLN. Then, during a hybrid (i.e., virtual and in-person) interactive workshop with LSLLN members from the different network hubs, the purpose and process of conducting scoping reviews was presented along with the draft set of research questions, followed by group discussions about which research questions should guide the scoping review process. This workshop led to the development of the final set of research questions guiding the scoping review process.

2.2. Identifying Relevant Studies

A librarian from Lakehead University assisted with the selection of electronic databases, search strategy, and identification of relevant studies. Three electronic databases were used in our search: (1) ScienceDirect; (2) Web of Science; and (3) PubMed. These databases were recommended by the librarian to enable breadth and comprehensiveness in terms of journals and disciplinary perspective. Additionally, preliminary searches in other potential databases indicated a very high-level of duplication across identified articles. To cast a wide net and to identify original peer-reviewed LL literature with a focus on addressing sustainability-related challenges in our initial database search, “living lab” (living lab*) and “sustainability” (sustainab*) were used as keyword search terms. A five-year time period, between 2015 and 2019, was selected as appropriate to answer our specific research questions and because we were interested in synthesizing recent knowledge and emerging trends to add to existing LLs literature reviews [3,4,5,6,7,8]. Database searches were concluded at the end of November 2019 such that any publications in December 2019 were not included. The database searches identified a total of 729 papers. All articles identified from the searches were transferred to a reference manager software (Zotero). Once all duplicates were removed (n = 26), 703 papers remained for relevance screening.

2.3. Selecting Studies for Inclusion in the Review

This step consisted of an initial screening of titles and abstracts followed by a full-text review. The title and abstract of each citation were assessed for relevance based on the inclusion and exclusion criteria presented in Table 1. These criteria were developed, tested, and refined by two independent reviewers (one author and one research assistant) to ensure a set of criteria was developed that was sufficiently robust to capture relevant studies while also excluding non-eligible and irrelevant studies. Articles meeting any exclusion criteria were excluded. Title and abstract screening was conducted by the same two independent reviewers. A final set of citations (n = 187) was included for full-text review. At this step, if there was any uncertainty about relevance, citations remained for consideration in the full-text review. Subsequently, the full text of all 187 potentially relevant articles were assessed using the a priori developed inclusion and exclusion criteria. After applying the inclusion and exclusion criteria to all full-text articles, a total of 139 articles were excluded (See PRISMA diagram, Figure 2, for overview and reasons for exclusion). A total of 48 papers were selected as eligible and included in the data charting and synthesis process.

2.4. Data Extraction

Once the articles were selected, data were extracted and recorded in a data extraction spreadsheet. Data extraction was guided by the set of research questions in addition to general descriptive information about the articles. The extraction spreadsheet included a total of 37 data items that were coded as either binary (yes/no), categorical, textual description, or verbatim text (See Supplementary Materials File S1 for the full data extraction spreadsheet). The data extraction form was pre-tested on four articles by all authors for clarity and consistency and adjusted as needed. A set of data items were coded at the paper level and another set were coded using information about specific LL projects that were described, presented, and/or analyzed within the papers in our review. The final dataset was reviewed and cleaned after data extraction.

2.5. Collating, Synthesizing, Summarizing, and Reporting Results

The reviewers used extracted data to tabulate, describe, and synthesize the content using frequencies, percentages, tables, and graphs. Where appropriate, textual analyses using a constant-comparative strategy were conducted.

3. Results

From the 729 citations gathered in the electronic database searches, 48 papers were identified as relevant through the screening and eligibility assessment process. In the sub-sections below, findings from our analysis of these 48 papers are presented.

3.1. Characteristics of Recent LLs Publications

Overall, we found that the published LLs literature involving universities as key collaborators and focused on sustainability is growing, highly interdisciplinary, and predominantly taking place within Europe. We saw a consistent increase in the number of publications between 2015 and 2019 illustrating a growing interest in LLs research. The LLs literature crosses disciplinary boundaries and articles are published across a wide range of journals with no clear disciplinary home. There is a very strong geographic concentration of LLs research in Europe, with very little research emerging from Asia, the Americas, Africa, or Oceania. Additionally, the large majority (77%) of lead authors work in academic institutions located in European countries—Sweden and the Netherlands in particular. This strong European focus is likely explained, in part, by funding programs such as the Joint Programming Initiative (JPI) Urban Europe, an intergovernmental research and innovation programme aimed at addressing the challenges of sustainable urban development, and the existence of The European Network of Living Labs (since 2006) which have supported LL research and work across Europe. In 2015, Schuurman et al. [10] found that LLs research was almost exclusively taking place in Europe. Our findings support this finding and demonstrate that there has been limited progress in expanding LLs research beyond Europe since 2015.
The majority (66%) of publications were empirical papers, 19% were review papers, and 15% were papers with a theoretical, conceptual, and/or practical focus. Among those publications coded as empirical, qualitative data collection and analysis were most commonly used (30%) followed by mixed-methods (21%).

3.2. Conceptualization, Description, and Application of LLs

Through the scoping review process, we aimed to interrogate and advance our collective understanding of how LLs are being conceptualized and described in the recent scholarly literature while also examining how sustainability-focused LLs are being used, particularly when universities are key collaborators. We coded for eight specific categories in our data extraction process to examine how LLs are conceptualized, described, and used.
We examined how authors are conceptualizing LLs by extracting the verbatim definitions presented in papers and analyzing these definitions with textual analysis. As other scholars have noted [6,19], our findings illustrate that there is no common or consistent definition of LLs, rather there is a wide variety of definitions presented and used in the literature. The following definitions are illustrative of this diversity:
“[LLs are] a form of experimental governance, whereby urban collaborators develop and test new technologies, products, services and ways of living to produce innovative solutions to the challenges of climate change, resilience, and urban sustainability.”
[20]
“...spaces to facilitate experimentation about sustainability solutions… [urban LLs] represent sites that allow different urban actors to design, test and learn from socio-technical innovations in real time.”
[9]
“Urban Living Labs are “transition arenas” which provide “protected spaces” for experimentation and learning in the context of governing urban sustainability transitions.”
[21]
“A ‘living lab’ is defined as a situation or circumstance where real-world sustainability challenges are formally addressed in collaborator partnerships…”
[22]
“…Living Lab is a tool or a framework for creating services, public goods, market products or ideas.”
[23]
Despite the diversity of definitions presented and used in the literature, our textual analysis identified the following four common elements of LL definitions: (1) a general descriptor term, (2) participants, (3) key processes used, and (4) intended outcome(s). The most used descriptor terms were ‘approach’ and ‘methodology’. Other examples of general descriptor terms included: ‘spaces’, ‘transition arenas’, ‘innovation arenas’, and ‘networks’. The second common definitional element identified was a description of who participates; in other words, the actors or collaborators that participate in LL activities. Most definitions refer to ‘stakeholders’ broadly and emphasized that multiple stakeholders participate in LLs, such as ‘a broad range of stakeholders’ or ‘stakeholders from public-private-people partnerships’ Third, a description of key process elements are commonly outlined within LL definitions. References to ‘co-creation’, ‘experimentation’, ‘open innovation’ and ‘learning’ were most used within definitions presented; often several of these three process elements were included together in LL definitions. Although less common, some definitions also speak to the intended outcome(s) of LLs. Across the range of definitions analyzed here, commonly referenced outcomes included ‘solutions’, ‘products’ and ‘services’. Frequently, outcomes were captured by the overarching idea of ‘innovative solutions’. These common intended outcomes emphasize the technological and technocratic focus that characterize the LLs literature.
We also used a categorical data item to examine the extent to which key LLs characteristics were explicitly referred to when authors were describing, defining, and presenting the LL concept (drawing on the work of [19]). As summarized in Table 2, a large majority of papers in our review explicitly described LLs as involving “multi-stakeholders collaboration in LL activities” (89%), “aimed at innovation” (85%), and “co-creation as a central process” (79%) as key characteristics of LLs. Most papers (70%) also clearly articulate that “living lab activities take place in real-life context” emphasizing that LLs are largely place-based, place-responsive, and generally aiming to identify solutions to sustainability challenges in particular places. Notably, ‘shared decision-making power among stakeholders’, was rarely (21%) described as a characteristic of LLs. Some papers identified ‘other’ key characteristics of LLs beyond the set that we coded for, examples include experimentation, self-organization, and open innovation.
Given the explicit focus on place and real-life settings within LL research and practice, we also coded for an urban versus rural focus with the papers in our review. We found a strong bias towards urban settings (and large cities in particular) in the recent LL literature; 70% of the papers focused on LLs in urban settings compared to rural or peri-urban settings. This highlights a blind-spot in the current LLs literature while also representing a continued marginalization of rural and peri-urban settings that are commonly overlooked in sustainability research and action broadly.
In terms of advancing our collective understanding of how LLs are being used for sustainability-related research involving universities as a key collaborator, we gathered textual data about the stated justification for LLs described in the papers. In terms of the stated justification for LLs, the need to test and experiment with place-relevant social and technological solutions/transitions was common. Many authors also articulated the rationale for LLs as creating learning environments/spaces and opportunities for collaboration among diverse participants. The following set of quotes provides examples of the ways in which collaboration was described with respect to justification:
“The living laboratory provides a focal point around which stakeholders can work together, generating communities of interest that may well outlive and transcend the living laboratory itself”
[24]
“[LLs] are intended to promote the collaboration between a variety of actors and, with their experimental sites, are also considered to have value for long-term sustainability transitions”
[1]
“...to bring stakeholders together in a process of co-learning”
[25]
Some authors also justified the need and value of LL work in relation to governance. Lovell et al. [26] argue that LLs provide a much-needed space for “governance by experiment”, Purcell et al. [22] write that LLs can “function as a convening framework supporting formal intra- and inter-organizational governance”, and von Wirth et al. [9] argue that LLs are particularly useful as “new ways of governing transformative change”. In the next subsection, we consider governance dimensions of LLs work more fully.

3.3. Governance of LLs

By governance, we refer to the processes and practices whereby collaborative groups make decisions, build relationships and structures, and establish accountability [27]. To enhance our collective understanding of LL governance, a set of data items were collected about decision-making, power, and structure.
A majority (72%) of papers explicitly spoke to and/or described governance aspects of the LLs, highlighting a general recognition of the importance of governance in LLs research and practice. Given the strong focus on collaboration, this recognition is imperative. We also coded whether or not issues related to power relations and imbalances among LL participants are explicitly recognized and/or discussed in the 48 papers in our review. Somewhat concerning, issues related to power were only recognized/discussed in half of the papers we reviewed. When this was discussed in the papers, we collected details about how power imbalances were addressed (or recommendations for addressing power imbalances). Key recommendations included the following: (i) explicitly recognizing/naming the existence of power imbalances early on, (ii) establishing trusting relationships among all collaborators, (iii) ensuring that all collaborators are involved in developing the guiding vision(s) of LL projects, (iv) drafting contracts to inform expectations about relationships and practices among collaborators, and (v) ensuring there are ample opportunities for dialogue among collaborators. Together, these findings emphasize a need for greater attention to processes of decision-making, efforts to ensure all actors have a voice and role in decision making and efforts aimed at addressing power imbalances present in collaborative work.
In terms of governance-related data for specific LL projects described within the papers, we coded for LL structure, how decisions were made, and characteristics of the project lead. Among those LL projects where decision-making was described, decisions were made primarily by the lead organization (40%) while joint decision making or decision-making through committees were used less commonly (25% and 20%, respectively). Therefore, although it is increasingly argued that participants should have decision-making power within LLs work, this does not appear to be practiced in a significant proportion of LL projects described in our review papers. Academic institutions and city/municipal authority were the most common LL project/initiative lead. The most common governance structure was centralized (i.e., decision making power and control is concentrated in a central, authoritative individual or institution and all participants must act according to their determined role in the hierarchy) while decentralized (i.e., decision making power and control is distributed among all participants in an inclusive and/or democratic way) or polycentric (i.e., decision making power and control includes multiple actors and groups that coordinate according to an agreed-upon set of rules/processes) were less common. Importantly, we found that details relating to decision-making, structure, power and accountability are often omitted or described minimally when LL projects are described in the academic literature.

3.4. Co-Creation within LLs

Co-creation is a central defining characteristic of LLs work [10,19]; however, co-creation was not specifically defined in 27 (56%) of the papers in our review. Among those papers that did explicitly define co-creation, key commonalities across definitions include that co-creation is understood as a process (that supports the achievement of an intended aim/outcome), the process is often specifically connected to learning, problem-solving, and/or experimentation, and that co-creation is derived from inclusive collaboration/relations. Definitions of co-creation, extracted verbatim from our review papers are provided to illustrate these common themes:
“Co-creation as the intended learning process in living labs, ideally, encompasses joint problem-definition and problem-solving using improvisation and experimentation—this in designing, implementation and testing of solutions in an iterative way, while interacting and co-constructing personalized experience”
[28]
“...co-creation brings together different policy and practice stakeholders with the aim to develop an outcome that will be valued by all the parties involved in the process”
[29]
“A widely accepted generic and literal definition of co-creation is ‘making something together...co-creation is not only a methodology to achieve (product, service, or process) innovation, but a way to create values that are shared between participants.”
[1]
We also found that co-creation was sometimes used interchangeably with ideas of co-design and/or co-production as general concepts conveying interactions among stakeholders from various sectors. Notably, some authors made the links between co-creation and decision-making. Specifically, that effective co-creation processes must recognize power dynamics and decision-making processes. For example, as Steen [19] argues, “to qualify as co-creation, the targeted users need to be involved in the various development phases of the LL process… they should have decision-making power throughout the phases.” Menny et al. [30] write that “[c]o-creation refers to the ability of citizens to actively engage in decision-making processes with at least equal power to influence them compared to other decision-making bodies.” Additionally, noteworthy, several authors clearly describe the kinds of stakeholders/participants that should be involved in LL co-creation processes emphasizing the need for participants across diverse sectors.
To better understand how co-creation is supported in LLs work, we also collated information about the specific kinds of methods, tools, and processes that are being used and/or are recommended in the recent LLs research. Somewhat surprisingly, very little detail is provided in most academic papers about how to support co-creation in practice. Some specific examples of methods, tools, and processes to support co-creation included workshops, visioning sessions, reflective practice, mapping (mind mapping and asset mapping), online dialogue sessions, design thinking seminars, creating group guidelines (i.e., terms of reference), and field trips.

3.5. Social and Environmental Justice in LLs Research and Practice

Social and environmental justice are increasingly recognized as essential in our collective efforts towards achieving sustainability [11,12]. We coded whether or not social and/or environmental justice was explicitly considered and/or discussed in the 48 paper and if so, we extracted key findings/implications/recommendations concerning justice. Overall, 36 papers (76%) did not explicitly consider or discuss social or environmental justice. Among those 12 papers that did, social justice was considered more commonly than environmental justice (or both social and environmental justice). Finally, we also coded two binary (yes/no) data items for LL projects described within the papers we reviewed. Specifically, whether LLs explicitly aim to address issues related to equity and justice and whether or not the LLs use planning and decision-making processes based on equity and/or representation. We found that approximately 5% of LL projects were aimed at addressing issues related to equity and/or justice while approximately 7% used planning and decision-making processes that were based on equity and/or representation. Therefore, we found that social and environmental justice were largely not considered in LLs work.

3.6. Limitations

There are limitations that should be recognized when interpreting the findings and considering implications of the findings presented herein. First, our review only searched for English language papers given that our research team does not have skills in any other languages. Although the inclusion of only English language articles may impact findings, implications would be minimal. Second, our search focused on published peer-reviewed research; the gray literature was not examined. The grey literature would likely contain relevant information about our research questions, in particular details about co-creation practices, but we aimed to examine and synthesize findings and trends from recent scholarly literature specifically. Third, our process did not assess the quality of articles included in our review. Although this is common and consistent within the scoping review methodology, it is worth noting. Finally, our review focused on literature published between 2015 and 2019 and therefore does not represent the longer history of LLs literature. Relatedly, our database searches were conducted at the end of November 2019 and therefore we have not captured the most recent literature (i.e., 2020 and 2021). We acknowledge that the exclusion of the 2 most recent years of literature from our study is a particularly noteworthy limitation. Given the collective reckoning around the need for transitions to support social and ecological justice prompted, in large part, by the COVID-19 pandemic, it is possible that most recent LLs literature engages with justice dimensions more meaningful and explicitly than the pre-pandemic research that we have used in our study.

4. Discussion

As noted previously, this scoping review study aimed to characterize and synthesize the scholarly literature on LLs with a focus on the recent literature oriented towards sustainability challenges and involving universities as key collaborators. The following subsections discuss our findings in relation to existing scholarship and our experiences engaging in LLs work within the LSLLN. We discuss opportunities for interrogating governance, unpacking co-creation, and working towards social and ecological justice in LLs research and practice.

4.1. Interrogating Governance

LLs are increasingly recognized as unique and timely opportunities for novel governance practices [31,32]. However, governance has also been identified as a core challenge and key knowledge gap in LLs research and practice [6]. Our findings have emphasized that governance challenges and opportunities are rarely a core focus of LLs research or projects. Moreover, scholarly publications generally include little detail and information about governance generally and power relations specifically.
Based on our review, we echo Van Geenhuizen [28] who argues that additional efforts interrogating governance is imperative to “find ways in coping with (considerable) gaps in interests and power between certain stakeholders”. LL decision-making processes, who makes decisions, and the contexts surrounding a LL and its participants inevitably enable and/or constrain social dynamics and influence the success of a particular project. However, the lack of critical reflection on these issues in the recent scholarly literature is noticeable, as outlined in our findings, and problematic in our view. Failing to explicitly acknowledge, interrogate, and address power dynamics surrounding the governance of LLs is not only a missed learning opportunity but could unintentionally limit diverse engagement, innovation, and new ideas—a goal of LLs. Among the papers we reviewed that did explicitly recognize and address power relations and imbalances, key recommendations focused on the need to identify and discuss concerns as well as establishing structures and processes that enable better collaboration as a way to build trust, establish meaningful relationships and encourage new and different ideas. We endorse these suggestions but also argue that they need to go further in the realm of decision-making power. In our scoping review, shared decision-making power among stakeholders was rarely (~20%) described as a key characteristic of LLs. Moreover, we found that decision-making was centered with the host organization or institution for most of the LL projects described in the reviewed papers despite arguments in the literature that decision-making power should be shared among stakeholders. For LLs to adequately address the needs of a particular group or community, in a particular place, those most impacted should have a more central role in the development and decision-making processes of the LL.
It is essential to recognize that all governance processes, whether formal decision-making structures or informal collaborations are manifestations of power relations. While some structures and forms of power may be obvious, others may be less visible, especially to those that benefit most from existing structures and resource flows. Moreover, while a multistakeholder approach to governance may work for some LLs, polycentric forms of governance may be more desirable in others. For example, in exit interviews of a LL project in Germany focused on a restoration planning process, Zingraff-Hamed et al. [23] demonstrate that polycentricity facilitated trust, learning, and the co-design of a resilient waterscape. The authors argued that a polycentric governance approach can be particularly useful when autonomous and multi-scale decision-makers are collaboratively involved in the design of policy solutions. Additional research interrogating LL governance structures specifically, along with a more in-depth engagement with governance theory [33], is an opportunity to further advance LLs work.
LL researchers and practitioners should more explicitly identify and describe governance structures as a fundamental element of LLs. This might include more attention to considering both the internal (e.g., decision-making processes, relationships among participants, funding, and budget-making) and external (e.g., social and environmental enablers and constraints) dynamics and more attention to accountability. Relevant to the climate crisis, LLs attentive to and engaging with governance and power dynamics overlaps with recent scholarly attention to climate stewardship [34]. When research and practice focus more directly on governance processes, LLs can become transformative experimental and learning spaces, not only to accomplish desired impacts but also to develop new and more effective ways to collaborate and co-create. Lovell et al. [26] argue that LLs provide a much-needed space for “governance by experiment” and Purcell et al. [22] wrote that LLs can “function as a convening framework supporting formal intra- and inter-organizational governance”. Relatedly, Von Wirth [9] argued that LLs are particularly useful as “new ways of governing transformative change”. In these ways, the process of governing LLs, the relationships among participants and enabling and constraining power dynamics should be understood as an essential element of the research and practice of all LLs.
In our own LL work in the context of the LSLLN, we have engaged in efforts aimed at reciprocal and respectful relationship-building to foster trust, equity, and motivation as a foundation for co-creation. However, we now also recognize that, moving forward, we need to engage more fully in critical reflection and intentional processes to identify and address power imbalances and ensure accountability in our own LLs work. Moreover, the importance of documenting and sharing details about how we go about this (e.g., the tools and process that support this work) is an important lesson that we take from this scoping review.

4.2. Unpacking Co-Creation

Our review has underscored that co-creation is central to LLs as understood and used in recent scholarly literature. Most papers in our review defined and described co-creation as a central characteristic of LLs. As Steen [19] write, co-creation is the “essence” of a LL. However, we also found that clear definitions of, and in-depth detail about, co-creation processes were commonly lacking from the papers we reviewed. Given the centrality of co-creation in terms of how LLs are conceptualized, described, and applied, the lack of clarity around what exactly co-creation means and how it is supported and achieved is surprising and problematic. Challenges with the co-creation processes can limit and stagnate development in LL work [35].
Puererri et al. [1] write, “making something together” is a rather literal starting place for understanding and thinking about co-creation in LL research and action. However, those of us doing research about and applying a LL approach should unpack co-creation more fully and explicitly describe the tools and processes used to support co-creation in practice. We recognize that these details may be outlined more fully in other spaces and outputs outside of peer-reviewed journals (e.g., reports, websites, other grey literature) but argue that these details should not be excluded from scholarly outputs to advance research and practice. In particular, we argue that explicit answers to the following questions should be clearly articulated: what does co-creation mean, and which scholarly lineage(s) inform this understanding? What is being co-created (i.e., knowledge, services, technology, solutions, etc.) and why? What are the tools and processes used to foster collaboration and support co-creation? Who is (and is not) participating in the co-creation process? Van Geenhuizen [28] has done this well, clearly describing and defining co-creation, linking to related scholarly works, and detailing those involved as well as the tools and processes used to support the co-creation process such as interviews, designing future scenarios, focus groups, and developing common aims, language and trust through relationship-building. The question of who is and who is not participating in co-creation processes is particularly important; deeply and purposefully considering this question aligns with several of our thoughts about governance and justice as they relate to LLs. Greater attention to, and transparency about, the people participating in and excluded from co-creation processes, including lived experiences, positionality and power, and values, is an opportunity to support more meaningful and impactful co-creative processes.
Like Verbeek [36], we see transparency about, and greater attention to, the specific tools and processes used to foster respectful and reciprocal relationships at the outset of LLs research and work as an essential foundation for co-creative work. People and relationships are, after all, “the driving force behind any and all collaborative initiatives” [35]. Moreover, systems thinking and complexity theory outline that initial conditions, including the nature, structure, and function of relationships among LL participants, play a huge role in shaping emergence [37]. Relatedly, a deeper and more nuanced exploration of iteration in relation to co-creative processes and LL research and practice generally is lacking in the recent literature; we see this as a missed opportunity for unpacking and advancing LL research and work. Although Steen et al. [19] outline an iterative approach as a defining characteristic of LL work, few of the papers in our scoping review spoke to iteration as central to LLs and none considered the interplay between iteration and co-creation explicitly or how learning, power dynamics, and the outcomes of LL research and action. This interplay between iteration, learning, and power in relation to co-creative processes is a specific realm that we hope to unpack more fully within the LSLLN.
Finally, our LSLLN team has wide-ranging experience and expertise in community-based and participatory action research. Moving forward in our own LL research and practice, we aim to foster cross-fertilization across the recent LL research and the community-based [38,39,40] and participatory action research scholarship [41,42]. We suspect that this limited cross-fertilization to date is explained, in part, by the continued technocratic focus within the LL literature and the relatively limited involvement of the social sciences broadly speaking. Workshops, colloquia, and other events to cultivate shared learning among LLs researchers and practitioners and those with expertise in community-based and participatory action research would support the cross-fertilization of ideas, inspiration, and practices.

4.3. Working towards Social and Environmental Justice

In our scoping review, we identified that the large majority of articles about LLs did not explicitly consider nor discuss social or environmental justice in meaningful ways. This illustrates a ‘justice gap’ in the recent LL research and practice. Dawson et al. [43] outline that justice gaps in scholarship and practice exist due to interacting normative (e.g., values and knowledge systems) and implementation (e.g., governance) challenges.
This absence of social and environmental justice within the recent sustainability-related LL scholarship is particularly concerning when participants aim to engage in co-creative processes, yet do not address the inevitable power relations that are present within and external to collaboration. As discussed above, in relation to LL’s governance structures, when people with multiple interests, ways of knowing, and perspectives come together, it is necessary to explicitly address these dynamics and structures proactively and responsively. A key concern here is that, despite a desire to address sustainability challenges and make positive change for a particular community or group, by failing to recognize and address power differences, LL’s risk reproducing existing and/or creating inequities. Scholarly literature and practitioners across diverse sectors have long argued that environmental degradation impacts vulnerabilized groups (i.e., people that have been racialized, gendered and face a range of economic and social oppressions) much more severely [44]. Thus, while many LLs may set out with “good intentions” aiming to address complex sustainability challenges, they generally ignore and/or fail to address structures of class oppression, racism, settler colonialism, and patriarchy that are at the core of sustainability challenges. Moreover, without a focus on (in)justice and (in)equity, efforts aimed at addressing sustainability are too commonly conflated and co-opted. For example, greenwashing is described as a way that companies or institutions mislead the public about the environmental performance and benefits of a product or service [45]. Thus, we argue that social and environmental justice (e.g., the equitable distribution of benefits and burdens and the participation of all people in decision-making) must take a more central role in LL work.
An example of this is expressed through the integration of issues relating to social justice with the discourse and practice of sustainability. Agyeman [46] writes that “it has become increasingly apparent that the issue of environmental quality is inextricably linked to that of human equality. Wherever in the world environmental despoliation and degradation is happening, it is almost always linked to questions of social justice, equity, rights and people’s quality of life in its widest sense” [47]. Engaging with and working towards social and environmental justice demands critically reflecting on who is involved in the LL research and/or project (e.g., is it largely white, educated, and middle-class people?) and what kinds of issues are being addressed by its projects (e.g., who do they serve and how do they address justice and equity?). For sustainability to become a transformative force, as opposed to simply making small changes, the concept of ‘just sustainabilities’ is proposed as “the need to ensure a better quality of life for all, now and into the future, in a just and equitable manner, whilst living within the limits of supporting ecosystems” [47]. Adopting a ‘just sustainabilities’ framework, as we have done in the LSLLN, may not only increase the functioning and influence of LL research and practice, but could also greatly enhance co-creation and innovation ultimately enhancing the relevance and impact of a LL overall. Adopting a ’just sustainabilities’ [46] framework is, of course, only a starting point; governance structures and co-creative processes must also be oriented towards addressing injustices.
Efforts aimed at working towards justice in LL research and practice may also benefit from the three-dimensional conceptualization of justice as theorized by Fraser [48,49] and Schlosberg [44]. From this perspective, research and practitioners must engage with, and create the conditions for distributional justice (fair and equitable distribution of harms, benefits, and resources); recognitional justice (i.e., recognition of and respect for marginalized groups, perspectives, and ways of knowing), and representational justice (i.e., procedures to ensure representation of diverse perspectives in decision-making) [50]. We see this as another promising pathway for working towards social and ecological justice in LLs work [44,51,52].
Finally, our own experiences with LLs and the LSLLN specifically have underscored that universities themselves, commonly lead organizations in LL initiatives as shown in our data, should also be prioritizing social and ecological justice. Universities engaging in LLs work may benefit from adopting a ’just sustainabilities’ framework [46] and enacting this work through the three-dimensional conceptualization of justice described above [48,49]. If universities are engaging in LLs research and practice without prioritizing social and ecological justice, they risk reproducing injustices and the systems and structure that create and amplify them. Moreover, our experiences have also highlighted the need for spaces that foster honest, transparent, and generative sharing about LLs work in relation to social and ecological justice.

5. Conclusions and Future Research Directions

Drawing on the scoping review findings and our experiences, we recommend four priority research directions. First, we see the need for research to expand across a greater diversity of settings. Our data have illustrated a blind spot in the existing LLs literature in terms of research in, or focused specifically on, rural, remote, and peri-urban settings alongside a dearth of published research outside of Europe. This gap in research in rural, remote, and peri-urban settings, which tend to be marginalized in sustainability work, is itself an injustice that demands action. LLs research in more diverse contexts is needed to address the existing urban and European bias to garner new insights for advancing governance, co-creation, and justice in LLs work. Second, future research should more explicitly identify and analyze governance structures with particular attention to power dynamics in relation to LLs research and practice. Hopefully, if future research focuses more explicitly and directly on power imbalances, LLs can become valuable experimental and learning spaces, not only to accomplish desired impacts but also to work towards addressing social and ecological injustices. Third, expanding on calls from others [1,9], we propose that future LLs studies should pay greater attention to how learning evolves via co-creation and if/how co-creative tools and processes create shared values and visions among those participating. Finally, given the central role of academic institutions in sustainability-related LLs work, we call for research examining how academic institutions are impeding and/or supporting advances in relation to governance, co-creation, and justice in LLs work. This is particularly important given that academic institutions tend to uphold power imbalances, create barriers for meaningful co-creative processes, and replicate systems and structures that contribute towards social and ecological injustices. This study, and our own experiences engaging in LLs work, have underscored that LLs can make contributions towards co-creating healthy, sustainable, and equitable transitions. We see these four research directions as imperative for LLs to meaningfully support these transitions.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/challe13010001/s1, File S1: Final Data Extraction Form.

Author Contributions

Conceptualization, L.P.G.; Methodology, L.P.G.; Data Collection, Analysis and Interpretation, L.P.G., C.Z.L., R.L.W.P. and K.M.; Writing—Original Draft Preparation, L.P.G., C.Z.L.; Writing—Review and Editing, L.P.G., C.Z.L., R.L.W.P. and K.M. Funding Acquisition, C.Z.L., L.P.G. and K.M. Authors are listed in order of contribution. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Social Sciences and Humanities Council of Canada (Award number: 890-2018-0015).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The co-authors acknowledge and express gratitude to research assistants (Shadiya Aidid and Kelsey Jones-Casey) and all members of the Lake Superior Living Labs Network.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Puerari, E.; de Koning, J.; von Wirth, T.; Karré, P.; Mulder, I.; Loorbach, D. Co-Creation Dynamics in Urban Living Labs. Sustainability 2018, 10, 1893. [Google Scholar] [CrossRef] [Green Version]
  2. Bronson, K.; Devkota, R.; Nguyen, V. Moving toward Generalizability? A Scoping Review on Measuring the Impact of Living Labs. Sustainability 2021, 13, 502. [Google Scholar] [CrossRef]
  3. Dekker, R.; Franco Contreras, J.; Meijer, A. The Living Lab as a Methodology for Public Administration Research: A Systematic Literature Review of Its Applications in the Social Sciences. Int. J. Public Adm. 2020, 43, 1207–1217. [Google Scholar] [CrossRef]
  4. Følstad, A. Living Labs for Innovation and Development of Information and Communication Technology: A Literature Review. Electron. J. Organ. Virtualness 2008, 10, 99–131. [Google Scholar]
  5. Greve, K.; Leminen, S.; De Vita, R.; Westerlund, M. Unveiling the Diversity of Scholarly Debate on Living Labs: A Bibliometric Approach. Int. J. Innov. Manag. 2020, 24, 2040003. [Google Scholar] [CrossRef]
  6. Hossain, M.; Leminen, S.; Westerlund, M. A Systematic Review of Living Lab Literature. J. Clean. Prod. 2019, 213, 976–988. [Google Scholar] [CrossRef]
  7. Schuurman, D.; Baccarne, B.; Coorevits, L.; Georges, A.; Logghe, S. Knowledge Exchange for Innovation Development in Open Innovation Systems. Tijdschr. Voor Commun. 2014, 42, 378. [Google Scholar]
  8. Westerlund, M.; Leminen, S.; Rajahonka, M. A Topic Modelling Analysis of Living Labs Research. Technol. Innov. Manag. Rev. 2018, 8, 40–51. [Google Scholar] [CrossRef] [Green Version]
  9. von Wirth, T.; Fuenfschilling, L.; Frantzeskaki, N.; Coenen, L. Impacts of Urban Living Labs on Sustainability Transitions: Mechanisms and Strategies for Systemic Change through Experimentation. Eur. Plan. Stud. 2019, 27, 229–257. [Google Scholar] [CrossRef]
  10. Schuurman, D. Bridging the Gap between Open and User Innovation? Exploring the Value of Living Labs as a Means to Structure User Contribution and Manage Distributed Innovation. Ph.D. Thesis, Ghent University, Ghent, Belgium, 2015. [Google Scholar]
  11. Agyeman, J. Introducing Just Sustainabilities: Policy, Planning, and Practice; Zed Books Ltd.: London, UK, 2013; ISBN 978-178-032-410-4. [Google Scholar]
  12. Sze, J. Sustainability: Approaches to Environmental Justice and Social Power; NYU Press: New York, NY, USA, 2018; ISBN 978-147-989-456-7. [Google Scholar]
  13. Lake Superior Living Labs Network Lake Superior Living Labs Network. Available online: https://livinglabs.lakeheadu.ca/ (accessed on 20 October 2021).
  14. Peters, M.D.J.; Godfrey, C.M.; Khalil, H.; McInerney, P.; Parker, D.; Soares, C.B. Guidance for Conducting Systematic Scoping Reviews. Int. J. Evid.-Based Healthc. 2015, 13, 141–146. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Pham, M.T.; Rajić, A.; Greig, J.D.; Sargeant, J.M.; Papadopoulos, A.; McEwen, S.A. A Scoping Review of Scoping Reviews: Advancing the Approach and Enhancing the Consistency. Res. Synth. Methods 2014, 5, 371–385. [Google Scholar] [CrossRef]
  16. Marshall, R.E.; Levison, J.K.; McBean, E.A.; Brown, E.; Harper, S.L. Source Water Protection Programs and Indigenous Communities in Canada and the United States: A Scoping Review. J. Hydrol. 2018, 562, 358–370. [Google Scholar] [CrossRef]
  17. Arksey, H.; O’Malley, L. Scoping Studies: Towards a Methodological Framework. Int. J. Soc. Res. Methodol. 2005, 8, 19–32. [Google Scholar] [CrossRef] [Green Version]
  18. Levac, D.; Colquhoun, H.; O’Brien, K.K. Scoping Studies: Advancing the Methodology. Implement. Sci. 2010, 5, 69. [Google Scholar] [CrossRef] [Green Version]
  19. Steen, K.; Van Bueren, E. The Defining Characteristics of Urban Living Labs. Technol. Innov. Manag. Rev. 2017, 7, 21–33. [Google Scholar] [CrossRef]
  20. Voytenko, Y.; McCormick, K.; Evans, J.; Schliwa, G. Urban Living Labs for Sustainability and Low Carbon Cities in Europe: Towards a Research Agenda. J. Clean. Prod. 2016, 123, 45–54. [Google Scholar] [CrossRef] [Green Version]
  21. Sharp, D.; Salter, R. Direct Impacts of an Urban Living Lab from the Participants’ Perspective: Livewell Yarra. Sustainability 2017, 9, 1699. [Google Scholar] [CrossRef] [Green Version]
  22. Purcell, W.M.; Henriksen, H.; Spengler, J.D. Universities as the Engine of Transformational Sustainability toward Delivering the Sustainable Development Goals: “Living Labs” for Sustainability. Int. J. Sustain. High. Educ. 2019, 20, 1343–1357. [Google Scholar] [CrossRef] [Green Version]
  23. Zingraff-Hamed, A.; Martin, J.; Lupp, G.; Linnerooth-Bayer, J.; Pauleit, S. Designing a Resilient Waterscape Using a Living Lab and Catalyzing Polycentric Governance. Landsc. Archit. Front. 2019, 7, 12–31. [Google Scholar]
  24. Omrcen, E.; Lundgren, U.; Dalbro, M. Universities as Role Models for Sustainability: A Case Study on Implementation of University of Gothenburg Climate Strategy, Results and Experiences from 2011 to 2015. Int. J. Innov. Sustain. Dev. 2018, 12, 156. [Google Scholar] [CrossRef]
  25. van der Jagt, A.P.N.; Smith, M.; Ambrose-Oji, B.; Konijnendijk, C.C.; Giannico, V.; Haase, D.; Lafortezza, R.; Nastran, M.; Pintar, M.; Železnikar, Š.; et al. Co-Creating Urban Green Infrastructure Connecting People and Nature: A Guiding Framework and Approach. J. Environ. Manag. 2019, 233, 757–767. [Google Scholar] [CrossRef]
  26. Lovell, H.; Hann, V.; Watson, P. Rural Laboratories and Experiment at the Fringes: A Case Study of a Smart Grid on Bruny Island, Australia. Energy Res. Soc. Sci. 2018, 36, 146–155. [Google Scholar] [CrossRef]
  27. The Institute on Governance. Available online: https://iog.ca/research/ (accessed on 20 October 2021).
  28. Van Geenhuizen, M. Applying an RRI Filter in Key Learning on Urban Living Labs’ Performance. Sustainability 2019, 11, 3833. [Google Scholar] [CrossRef] [Green Version]
  29. Zavratnik, V.; Superina, A.; Stojmenova Duh, E. Living Labs for Rural Areas: Contextualization of Living Lab Frameworks, Concepts and Practices. Sustainability 2019, 11, 3797. [Google Scholar] [CrossRef] [Green Version]
  30. Menny, M.; Palgan, Y.V.; McCormick, K. Urban Living Labs and the Role of Users in Co-Creation. GAIA-Ecol. Perspect. Sci. Soc. 2018, 27, 68–77. [Google Scholar] [CrossRef]
  31. Cellina, F.; Castri, R.; Simão, J.V.; Granato, P. Co-Creating App-Based Policy Measures for Mobility Behavior Change: A Trigger for Novel Governance Practices at the Urban Level. Sustain. Cities Soc. 2020, 53, 101911. [Google Scholar] [CrossRef]
  32. Bulkeley, H.; Coenen, L.; Frantzeskaki, N.; Hartmann, C.; Kronsell, A.; Mai, L.; Marvin, S.; McCormick, K.; van Steenbergen, F.; Voytenko Palgan, Y. Urban Living Labs: Governing Urban Sustainability Transitions. Curr. Opin. Environ. Sustain. 2016, 22, 13–17. [Google Scholar] [CrossRef] [Green Version]
  33. Kooiman, J. Governing as Governance; Sage: London, UK, 2003. [Google Scholar]
  34. Merenlender, A.; Buhler, B. Climate Stewardship: Taking Collective Action to Protect California; University of California Press: Oakland, CA, USA, 2021; ISBN 978-052-037-894-0. [Google Scholar]
  35. Kalinauskaite, I.; Brankaert, R.; Lu, Y.; Bekker, T.; Brombacher, A.; Vos, S. Facing Societal Challenges in Living Labs: Towards a Conceptual Framework to Facilitate Transdisciplinary Collaborations. Sustainability 2021, 13, 614. [Google Scholar] [CrossRef]
  36. Verbeek, H.; Zwakhalen, S.M.G.; Schols, J.M.G.A.; Kempen, G.I.J.M.; Hamers, J.P.H. The Living Lab in Ageing and Long-Term Care: A Sustainable Model for Translational Research Improving Quality of Life, Quality of Care and Quality of Work. J. Nutr. Health Aging 2020, 24, 43–47. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Meadows, D.H. Thinking in Systems: A Primer; Chelsea Green Publishing: Chelsea, VT, USA, 2008. [Google Scholar]
  38. Halseth, G.; Markey, S.P.; Manson, D.; Ryser, L. Doing Community-Based Research: Perspectives from the Field; McGill-Queen’s University Press: Montreal, QC, Canada; Kingston, ON, Canada; London, UK; Chicago, IL, USA, 2016; ISBN 978-077-354-727-8. [Google Scholar]
  39. Israel, B.A.; Schulz, A.J.; Parker, E.A.; Becker, A.B. Review of Community-Based Research: Assessing Partnership Approaches to Improve Public Health. Annu. Rev. Public Health 1998, 19, 173–202. [Google Scholar] [CrossRef] [Green Version]
  40. Israel, B.A.; Eng, E.; Schulz, A.J.; Parker, E.A. Methods for Community-Based Participatory Research for Health; John Wiley & Sons: Hoboken, NJ, USA, 2012; ISBN 978-111-802-186-6. [Google Scholar]
  41. Chevalier, J.M.; Buckles, D.J. Participatory Action Research: Theory and Methods for Engaged Inquiry, 1st ed.; Routledge: Abingdon, UK; New York, NY, USA, 2013; ISBN 978-041-554-032-2. [Google Scholar]
  42. McIntyre, A. Participatory Action Research; SAGE Publications, Inc.: Thousand Oaks, CA, USA, 2008; ISBN 978-141-295-366-5. [Google Scholar]
  43. Dawson, N.; Coolsaet, B.; Martin, A. Justice and Equity: Emerging Research and Policy Approaches to Address Ecosystem Service Trade-Offs. In Ecosystem Services and Poverty Alleviation; Routledge: New York, NY, USA, 2018; pp. 22–38. [Google Scholar]
  44. Schlosberg, D.; Collins, L.B. From Environmental to Climate Justice: Climate Change and the Discourse of Environmental Justice. Wiley Interdiscip. Rev. Clim. Change 2014, 5, 359–374. [Google Scholar] [CrossRef]
  45. Delmas, M.A.; Burbano, V.C. The Drivers of Greenwashing. Calif. Manag. Rev. 2011, 54, 64–87. [Google Scholar] [CrossRef] [Green Version]
  46. Agyeman, J. Toward a ‘Just’ Sustainability? Continuum 2008, 22, 751–756. [Google Scholar] [CrossRef]
  47. Agyeman, J.; Bullard, R.D.; Evans, B. Introduction: Joined-up Thinking: Bringing Together Sustainability, Environmental Justice and Equity. In Just Sustainabilities; Routledge: New York, NY, USA, 2003; ISBN 978-184-977-177-1. [Google Scholar]
  48. Fraser, N. Scales of Justice: Reimagining Political Space in a Globalizing World; Columbia University Press: New York, NY, USA, 2009; ISBN 978-023-114-680-7. [Google Scholar]
  49. Fraser, N. Abnormal Justice. Google Scholar. Available online: https://scholar.google.com/scholar_lookup?title=Abnormal%20justice&publication_year=2008&author=Nancy%20Fraser (accessed on 21 October 2021).
  50. Blue, G.; Bronson, K.; Lajoie-O’Malley, A. Beyond Participation and Distribution: A Scoping Review to Advance a Comprehensive Justice Framework for Impact Assessment; University of Calgary: Calgary, AB, Canada, 2020. [Google Scholar]
  51. Schlosberg, D. Defining Environmental Justice: Theories, Movements, and Nature; Oxford University Press: Oxford, UK, 2009; ISBN 978-019-956-248-0. [Google Scholar]
  52. Schlosberg, D. Political Challenges of the Climate-Changed Society. PS Political Sci. Politics 2013, 46, 13–17. [Google Scholar] [CrossRef]
Challenges 13 00001 g001 550
Figure 1. The Lake Superior Living Labs Network Orienting Diagram.
Figure 1. The Lake Superior Living Labs Network Orienting Diagram.
Challenges 13 00001 g001
Challenges 13 00001 g002 550
Figure 2. PRISMA Diagram. (* used in search for variations of word endings).
Figure 2. PRISMA Diagram. (* used in search for variations of word endings).
Challenges 13 00001 g002
Table
Table 1. Inclusion and Exclusion Criteria.
Table 1. Inclusion and Exclusion Criteria.
Inclusion CriteriaExclusion Criteria
Published between 1 January 2015 and serach datePublished prior to 1 January 2015
Article is published in an academic journal Conference abstracts or proceedings, books, book chapters, theses
Article is published in EnglishLanguage other than English
Living lab(s) are an explicit and central focus of the paperLiving lab(s) are NOT an explicit and central focus of the paper
Paper explicitly addresses sustainability challenge(s) Paper does NOT explicitly address sustainability challenge(s)
Paper is not focused solely on creating a technological innovation as an end-point/objectivePaper is focused solely on creating a technological innovation as an end-point/objective
University(ies) a key collaborator University(ies) NOT a key collaborator
Table
Table 2. Summary of explicit reference to key LL characteristics.
Table 2. Summary of explicit reference to key LL characteristics.
Characteristic of Living Labs 1Number of Articles
Multi-stakeholder collaboration in LL activities 42(89%)
Aimed at innovation40(85%)
Co-creation as a central process37(79%)
LL activities take place in real-life contexts and communities33(70%)
Aimed at formal learning27(57%)
Iterative approach 21(45%)
Shared decision-making power among all actors10(21%)
Other5(11%)
1 Based on [19].
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Galway, L.P.; Levkoe, C.Z.; Portinga, R.L.W.; Milun, K. A Scoping Review Examining Governance, Co-Creation, and Social and Ecological Justice in Living Labs Literature. Challenges 2022, 13, 1. https://doi.org/10.3390/challe13010001

AMA Style

Galway LP, Levkoe CZ, Portinga RLW, Milun K. A Scoping Review Examining Governance, Co-Creation, and Social and Ecological Justice in Living Labs Literature. Challenges. 2022; 13(1):1. https://doi.org/10.3390/challe13010001

Chicago/Turabian Style

Galway, Lindsay P., Charles Z. Levkoe, Rachel L. W. Portinga, and Kathryn Milun. 2022. "A Scoping Review Examining Governance, Co-Creation, and Social and Ecological Justice in Living Labs Literature" Challenges 13, no. 1: 1. https://doi.org/10.3390/challe13010001

APA Style

Galway, L. P., Levkoe, C. Z., Portinga, R. L. W., & Milun, K. (2022). A Scoping Review Examining Governance, Co-Creation, and Social and Ecological Justice in Living Labs Literature. Challenges, 13(1), 1. https://doi.org/10.3390/challe13010001

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop