A Systematic Review on the Application of the Living Lab Concept and Role of Stakeholders in the Energy Sector

Abstract

The living lab concept is identified as having the potential to provide a platform to test technologies and support energy transition. However, the application of the concept to the energy sector is limited, though emerging. This study undertook a systematic literature review to understand the extent of the application of the living lab concept, with the particular aim of informing the processes to establish such a platform in urban Africa. Using a sample of 35 papers, only 17 papers were related to energy-living labs, while 18 papers were outside the energy field. The scale and contexts of the application of living labs were diverse. However, not all initiatives that defined themselves as living labs were characterised by elements typical of the concept of a living lab. Further, how the stakeholders were identified, and the stakeholder recruitment process in energy living labs was unclear in the sampled studies. A recommendation is to improve transparency in the stakeholder identification, engagement, and recruitment process in energy living labs and to incorporate gendered issues into the setup and management of urban energy living labs.

1. Introduction

The living lab concept in energy transition research is still relatively new, but it has piqued the interest of policymakers, researchers, and practitioners [1]. While Følstad [2] claims that the living lab concept originated in the 2000′s with private firms conducting real-life testing and experimentation for information and communication technologies, Leminen et al. [3] contend that the concept first appeared in 1749.

The living lab concept has been applied and studied in a wide array of disciplines over the last 20 years, including human–computer interaction [4] and public health sciences [5]. The concept has had an impact on rural, peri-urban, and urban settings. By linking stakeholders from different spheres of the community, living labs within the territorial fabric increase the potential for technological innovations and the promotion of a digital culture [6]. In addition to technological outcomes, the living lab has been successful in fostering inclusive local development among all stakeholders through collective learning, innovation co-creation, and knowledge exchange [7].

It is, however, unclear how living labs are operationalised and how their outcomes are measured, and there is no published evidence on their design, implementation, and reporting performance [1]. Habibipour [8] and Paskaleva and Cooper [1] point out some of the existing gaps in our understanding of the living lab, including: (i) a lack of universally agreed-upon conceptualisation of a living lab; (ii) the benefits claimed for using a living lab are not based on the assessed outcomes or impacts; (iii) limited evidence exists to show that living labs deliver the benefits claimed; (iv) the quality of the evidence concerning living labs is unclear; (v) few studies provide the location of evidence to support claims about the performance of living labs; and (vi) the source of funding with major stakeholders having divergent interests affects the objective assessment of the benefits claimed. For instance, Ondiek and Moturi [9] illustrate how living labs in Nairobi, Kenya, have failed because they are mostly donor-funded projects and are not designed to ensure their sustainability beyond the funding duration.

In addition to the latter, most living lab applications are found in Europe, and few living labs in Sub-Saharan Africa have been documented, despite the establishment of the African Network of Living Labs (ANoLL) in 2010. A few documented studies on Africa’s living labs fall outside the energy sector. Living labs primarily emerged in Africa as a tool to address socioeconomic development, sustainable livelihoods, and social ills, such as substance abuse and gangsterism [9,10]. This study therefore aims to understand the extent of the conceptualisation of the living lab and its application in the energy context. The study is part of a larger project that aims to design and implement an energy living lab in urban informal settlements with one of the case studies in South Africa. Acknowledging the existing gaps in the living labs literature, but not presently addressing all, the focus of this study is on exploring the existing literature to inform the process of setup and management of energy living labs in urban informal settlements.

The study utilised a systematic literature review to explore the following seven questions: (i) How is the living lab concept defined in the literature? (ii) How has the concept of a living lab evolved? (iii) What are the emerging categories of living lab for sustainability? (iv) What are the scales and field of application of the living lab concept? (v) Who are the stakeholders involved in a living lab outside and within the energy sector? (vi) How are the stakeholders identified to participate in living labs? and (vii) What are the stakeholder recruitment and engagement processes in living labs?

The rest of the paper is organised as follows: Section 2 describes the process of the systematic review of the literature utilised in the study. Section 3 discusses the results from the systematic literature review to address the first four questions; Section 4 discusses the last three questions; Section 5 provides a discussion of the study results; and Section 6 presents the conclusions.

2. Methods

This study utilised a systematic literature review, which is a method to synthesise research findings in a systematic, transparent, and reproducible manner based on particular criteria [11,12]. The study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) structure suggested by Moher et al. [12] to ensure rigour and transparency. The PRISMA structure consists of four steps: identification, screening, eligibility, and inclusion, as shown in Figure 1.

2.1. Identification Phase: Information Sources and Search Strategy

The selection of the primary information sources was aimed at searching grey literature, search engines, online databases, and electronic journals. Therefore, the literature was identified from the Scopus database [13], Google Scholar [14], ENERGIA [15], and later the Technology Innovation Management Review journal [16].

Scopus searches were limited to titles, abstracts, and keywords to eliminate irrelevant papers since there is an increasing amount of literature on living labs. All topics were selected to ensure that every subject area, document type, and year of publication were covered.

Google Scholar was chosen to identify any non-peer-reviewed documents relevant to the study and supported quick searches through snowballing, particularly for highly systematic reviews of living labs. Publications from ENERGIA, an international organisation researching gender and energy [17], were relevant to the study. Google Scholar and ENERGIA database searches were limited to articles published between 2007 and 2020. The search dates were informed by scholarly articles that had increased after the launch of the European Network on Living Labs (ENoLL), an international association that took in many living labs in Europe and the rest of the world after 2006 [18,19].

The searches from these information sources were last revised on 22 April 2021. However, there were no outcomes from the manual search in the ENERGIA publications. As a substitution, a second search was later done on 25 June 2021 in the Technology Innovation Management Review journal. The second search was done because this journal was identified as the top journal in the living labs field [8], and the purpose was to check for any articles that might be missing from Scopus and Google Scholar.

The study utilised three criteria to determine the scope of the search: (i) only documents written in the English language were selected from all databases; (ii) the date restrictions for the Technology Innovation Management Review journal and Google Scholar publications were 2007–2020; and (iii) the Scopus database search covered all dates. The search strategy and keywords used in each database are described in

Table 1. Information sources and search strategies.

Information Source	Search Strategy
Scopus database (all dates)	The query search strategy was TITLE-ABS-KEY (stakeholder* OR “role player” OR actor* OR “end-user” OR user* OR partner*) AND TITLE-ABS-KEY (“living* lab*” OR “living* laboratory*” OR “living* labs*” OR “living* laboratories*” OR “living* labbing*”) AND TITLE-ABS-KEY (“gendered* energy*” OR “energy* gender*” OR energy* OR gender* OR gendered*).
Google Scholar (2007–2020)	The adapted Scopus search strategy used for advanced search in Google Scholar was with at least one of the words “stakeholder” OR “role player” OR “actor” OR “end-user” OR “partner” AND with all of the words “gendered energy” AND with the exact phrase “living lab”.
Technology Innovation Management Review (2007–2020)	The adapted Scopus search strategy used for the advanced search in Technology Innovation Management Review contained any of the words “gendered energy” AND “stakeholder” AND containing the phrase “living lab”.

* Search documents with specific word.

.

2.2. Screening Phase

A total of 229 records from Scopus, Google Scholar, and the Technology Innovation Management Review dataset were identified following the search strategies shown in Table 1 after duplicates had been removed. The PRISMA methodology requires that these items undergo the process of screening [12], which means screening the titles and abstracts of the identified literature items to determine which records are irrelevant to the study and, therefore, should be excluded from the review process.

The screening process also comprised the exclusion of literature items in the following instances: (i) the record was either a poster or the full-text document was unavailable, meaning that the document only had an executive summary and a content page available, or one had to pay a certain amount to gain access to the full-text document. (ii) The record was a systematic literature review. A total of 177 literature items were excluded through the screening process, and 52 were put through the eligibility assessment described in Section 2.3.

2.3. Eligibility Phase

In the eligibility phase, the full text of the document was read to determine whether it met or did not meet the eligibility criteria. Given the number of descriptive and case study-related papers on living labs identified, the following eligibility criteria were developed:

(i)

If the full-text document only mentioned ‘living lab’ as a phrase in passing and no description of the concept or approach was provided, the document was excluded.

(ii)

If the full-text document did not describe at least one role of a stakeholder or mentioned an activity in which one or more stakeholders or users were involved, the document was excluded.

(iii)

When living labs were discussed outside the energy context, the article was included.

(iv)

When the article was about living labs but did not mention any gender aspect related to the stakeholders, the research field, or technology innovations, it was included.

The third and fourth criteria provided an overall context or coverage of the subject of living labs and stakeholders, and insights from other fields in which living labs had been implemented and informed significant parts of this study.

2.4. Inclusion Phase

A total of 17 articles were excluded from the eligibility phase, and only 35 articles were included and used for the qualitative review and synthesis in Section 3 and Section 4. The 35 sample papers included are organised by year of publication, with the majority of articles published in 2017 and 2018 (Figure 2).

3. Results

Table 2. List of sampled papers by region, sector, and year.

Region/Country/City	Sector	Year Established	Reference
Norway, Sweden, Denmark, Lithuania and Iceland	Energy	2016 (completed)	Krogstie et al. [20]
Germany	Energy	2010	Jakobi and Schwartz [21]
Italy and Germany	Energy	-	Jahn et al. [22]
Harlow (England)	Education	2010	Chin and Callaghan [23]
Finland	ICT	2009	Suopajärvi et al. [24]
Sweden	Energy	2008–2010	Ståhlbröst [25]
Lulea (Sweden), Amsterdam (The Netherlands), Helsinki (Finland) and Lisbon (Portugal)	Energy	-	Nina et al. [26]
Hungary and Styria (Austria)	Energy	2012 (evaluated)	Kovács [27]
Europe	Information technology (IT)	2012–2014	Ståhlbröst et al. [28]
Austria, Denmark, Italy, Germany, United Kingdom, Hungary and Poland	Energy and policy	2012	Dvarioniene et al. [29]
Finland and Sweden	Social sciences	2014	Buhr et al. [30]
Europe	Innovation management and design	-	Bergvall-Kåreborn et al. [31]
Sweden	Energy/sustainability	2016	Andersson and Rahe [32]
Berlin (Germany)	Energy/ transportation/ICT	2013–2015	Canzler et al. [33]
Sant cugat del Valles (Barcelona)	Education	2011	Masseck [34]
Fremantle (Australia)	Energy	2015	Eon et al. [35]
Cahors (France)	Energy	2016	Claude et al. [36]
London (England)	Internet of things (IoT) and sensing systems	2017	Jackson et al. [37]
Germany	IT and human computer interaction (HCI)	2015	Jakobi et al. [38]
Northwest Europe	Business	-	Burbridge et al. [39]
Finland	Innovation management	-	Leminen et al. [3]
Edinburgh (Scotland)	Energy	-	Morgan et al. [40]
Australia	Health	-	Pedell et al. [41]
Savona (Italy)	Sustainability/smart city	2017	Bracco et al. [42]
Germany	IT and feminist studies	2017	Ahmadi et al. [43]
Roeselare (Belgium)	Urban farming	2018	Pertry et al. [44]
European Union	Research and innovation	-	Provenzano et al. [45]
Fremantle (Australia)	Energy	2015	Eon et al. [46]
Karditsa (Greece)	Energy	2016	Giannouli et al. [47]
Belgium	Innovation management	-	Schuurman and Protic [48]
Evanstad (Norway)	Energy	2018	Woods and Berker [49]
Germany	IT and feminist studies	2017	Ahmadi et al. [50]
Birmingham, Newcastle, Manchester (England) and Bridgend (Wales)	Energy	2017	Sovacool et al. [51]
Cahors (France) and Vitoria-Gasteiz (Spain)	Energy	2016	Egusquiza et al. [52]
Vancouver (Canada)	Education	2014	Save et al. [53]

provides a list of the 35 papers included in the study according to the region where the living lab is framed, the sector under which it falls, and the year of establishment. Of the 35 sampled papers, only 17 papers were related to energy living labs, while 18 papers were outside the energy field. The results were discussed, addressing each of the seven questions. In this section, the paper discusses the first four questions: (i) How has the living lab concept been defined in the literature? (ii) How has the concept of a living lab evolved? (iii) What are the emerging categories of living lab for sustainability? (iv) What are the scales and fields of application of the living lab concept?

3.1. How Has the Concept of a Living Lab Been Defined?

There were 12 definitions of the term ‘living lab’ identified in the 35 analysed papers, summarised in

Table 3. Definitions of the term living lab.

Living Lab Definitions		Source
1	“A user centric research methodology for sensing, prototyping, validating, and refining complex solutions in multiple and evolving real life contexts”.	Giannouli et al. [47] p. 15
2	“A space often geographically bounded, for example, a city, neighbourhood, park, etc.) where research and innovation happen in a public-private-people partnership”.	Jackson et al. [37] p. 1
3	“A real-life place for user (those living in the lab and stakeholders, i.e., business, society, academic) co-creation of innovations in knowledge, products, services and infrastructures”.	Burbridge et al. [39] p. 393
4	“A real-life place that supports the co-creation and testing of innovation whilst also focusing on user awareness and providing insights into user behaviour and daily practices”.	Eon et al. [35] p. 274
5	“A living environment which houses both people and technology, in a semi experimental setting that promotes symbiotic innovation, development and research”.	Chin and Callaghan [23] p. 93
6	“An open innovation, user participative approach which takes place in an ecosystem of some kind”.	Ahmadi et al. [50] p. 2
7	“A user-centred sort of social experiment with the objective of testing a novel technology or intervention in a real-world and real-time environment”.	Sovacool et al. [51] p. 3
8	“An open innovation environment in a real-life setting in which user-driven innovation is the co-creation process for new services, products and societal infrastructures”.	Provenzano et al. [46] p. 574; Kovács [27] p. 52
9	“An innovation organisation in which the whole value chain is involved in the development of innovative services in co-creation with users in a real-world context”.	Ståhlbröst [25] p. 2
10	“Physical regions or virtual realities, or interaction spaces, in which stakeholders form public–private–people partnerships (4Ps) of companies, public agencies, universities, users, and other stakeholders, all collaborating for creation, prototyping, validating, and testing of new technologies, services, products, and systems in real-life contexts”.	Leminen et al. [3] p. 21; Save et al. [53] p. 2
11	“A user-centric innovation milieu built on every-day practice and research, with an approach that facilitates user influence in open and distributed innovation processes engaging all relevant partners in real-life contexts, aiming to create sustainable values”.	Bergvall-Kåreborn et al. [31] p. 38
12	“A user-centred, open innovation ecosystems based on systematic user co-creation approach, integrating research and innovation processes in real life communities and settings”.	Pertry et al. [44] p. 157

Source: Compiled by authors from various sources.

. While there is no consensus on the definition of the term, a consistent train of understanding exists when defining the living lab, which consists of three components. The first component is experimentation, which is built on everyday practices in a real-time environment [46,51] or a geographically bound space [37]. The second component focuses on innovation processes and the development of new products, services, societal infrastructures [27,45,47], knowledge, and research [23,24,39]. The third component is on the importance of collaboration–the involvement of both users, in other words those living in the lab, and multiple stakeholders from different sectors in cocreation [35,37]. Steen and Van Bueren [54] identified similar components when operationalising living labs and suggested context, aims, participants, and activities as four primary characteristics of living labs.

The authors argue that not all initiatives recognised as living labs in the sampled papers are characterised by the elements mentioned above, indicating a mismatch of theoretical understanding of the living lab concept and its practical implementation. Given the scope of the paper, the discussion is based on a recent living lab definition suggested by Mukama et al. [55] p. 4, who defined the term as “a research and innovation concept for experimental and experiential learning in real-life environment, involving users and multiple private and public stakeholders, aimed at tackling the problem of energy insecurity in urban poor environments”.

3.2. How Has the Concept of a Living Lab Evolved?

The application of the living lab concept appears to be constantly evolving. According to Schuurman and Protic [48], the evolution of the concept has occurred within three time periods: (i) the 1980s and 1990s, when the living lab was first initiated for long-term field experiments; (ii) the 1990s and 2000s, when laboratory infrastructures were designed to test innovations in environments that reproduced real-life conditions; and (iii) the 2000s and 2010s, when real-life experimentation coupled with cocreation became a crucial part of innovation methods.

The emergence of the living lab term is widely accredited by William J. Mitchell, a professor at the Massachusetts Institute of Technology [23,29,49]. Based on a strong awareness of users, Mitchell used the concept as a tool to observe people’s interactions with and reactions to new IT innovations over extended periods in a ‘living’ or quasi-naturalistic space, such as residential buildings or a city [29,43]. However, some authors still trace back the roots of the living lab to Knight and his work in 1749. He first described the “elements and conditions of a body and an environment of an experiment” as a living laboratory [3]. It was only after the Finnish Presidency officially adopted the concept through the launch of the ENoLL as a multinational organisation in 2006 that the practice of living labs gained momentum internationally and became valued in research [23,39].

Andersson and Rahe [32] suggest the existence of four generations of living lab networks, illustrated in

Table 4. Generations of living labs.

Generation zero	Assesses the performance of a single technology	Tests and standards are normalised
First generation	Assesses a system’s performance	No demonstrators occupy the living lab
Second generation	Assesses the performance of a user and a single technology	Demonstrators occupy the living lab
Third generation	Optimises the interface between systems and human behaviour	Facilitates a cocreation environment	Occurs within flexible and modular spaces

Source: Andersson and Rahe [32] p. 237.

.

Generation zero assesses the performance of a single technology. The first generation assesses a system’s performance, with no demonstrators occupying the living lab. The second generation assesses the performance of a user and a single technology, and demonstrators occupy the living lab. The third generation optimises the interface between systems and human behaviour, facilitates a cocreation environment, and occurs within flexible and modular spaces.

Viewing a living lab in urban contexts, Leminen et al. [3] p. 22,describe the third generation as a platform that depends on a participatory, representative, and open system of governance that shares its resources amongst all stakeholder networks and engages in “diverse activities and methods to gather, create, communicate, and deliver new knowledge, validated solutions, professional development, and social impact in real-life contexts”. Looking at a home environment, Andersson and Rahe [32] point out that a third-generation living lab is a research platform that intends to improve the interface between human behaviour and innovation while facilitating a cocreation environment in adaptable spaces.

The generations of living labs presented in Leminen et al. [3] and Andersson and Rahe [32] highlight two possible types of living labs that Woods and Berker [45] brought forward. The first one is technology and innovation driven and places emphasis on product development for advancing technologies in settings that resemble a ‘real-life’ environment but with limited social contact. The second type of living lab is centred around people and user groups and acknowledges the social, physical, and political context of the living lab environment. This type of citizen-centred living lab could potentially foster transitions towards sustainability in complex environments such as urban areas. One could also argue that although living labs seem to be ‘evolving’ with time, there are still living labs from the last 10 years that resemble the second generation and are designed as a technology test bed for new applications or infrastructure, such as the living lab of Oulu, described in Suopajärvi et al. [24].

3.3. What Are the Emerging Categories of Living Labs for Sustainability?

The concept of sustainability is of growing interest worldwide, and as a result, living labs have become an important part of a sustainable transition. Five studies have mentioned different categories of living labs that stem from the following: the need to advance sustainable development (with the realisation of all three pillars—economy, society, and environment) [30], a response to the ratification of the Paris Agreement in 2015 and reducing CO₂ emissions [32], changing national energy policies and systems [47], and improving energy consumption of the built environment [36,53]. These are therefore labelled either as a ‘regional living lab’, a ‘sustainability living lab’, or an ‘urban living lab’.

A sustainability living lab is focused on enhancing environmental sustainability, being economically viable, and improving people’s lives through sustainable living [32]. According to Claude et al. [36], an urban living lab is an experimental form of urban governance that creates solutions to confront urbanisation challenges and address climate change and resilience issues. Similarly, when describing a regional living lab, Giannouli et al. [47] acknowledges the governance aspect that a living lab enables in integrated sustainable energy planning. For instance, a regional living lab as a strategy and environment provided a balance between bottom-up and top-down approaches, while bringing together multilevel structures and stakeholders in the governance of transitioning energy systems across the Karditsa region in Greece [47].

With an understanding that sustainable development is an evident and pressing issue everywhere, Buhr et al. [30] show the application of a living lab approach in marginalised urban areas in need of social uplift in Finland and Sweden. The authors express the importance of having the common goals of society, which are conveyed through users and municipalities, at the heart of the living lab for sustainable development in these areas. For example, the living lab project in Alby, Sweden, provided energy-efficient light-emitting diode (LED) lights along a walkway to increase the residents’ sense of security in public spaces around the community. The paper also highlights how a living lab strategy in marginalised communities helps to consider the residents’ ideas in development processes, gives them a sense that they are being heard, and increases both the likelihood of obtaining widespread support for large changes, as well as giving the people a chance to take ownership of the changes happening around their community [30].

Both urban and sustainability living labs set up favourable conditions for linking sustainable innovations with society and the market [53] and enabling sustainable transitions from an individual to a governance perspective. However, the transition offered by a living lab method for sustainable development does not come without challenges. Several barriers noted relate to (i) the poor visibility of living labs, which may impede the ability to disseminate their outcomes to a wider audience and be competitive in the development and research field, and (ii) the limited focus on and interest in sustainable technologies or products, which may potentially hinder the long-term deployment of sustainable systemic improvements [32].

3.4. What Are the Scales and Fields of Application of the Concept of a Living Lab?

The sampled studies show that the living lab concept has been applied in various fields and at different scales, as shown in

Table 5. Scales/context and fields of application.

Scale/Context	Supporting Reference	Field of Application	Supporting Reference
University campus	Save et al. [54]; Bracco et al. [42]; Woods and Berker [49].	ICT, HCI and IoT	Ståhlbröst et al. [28]; Suopajärvi et al. [24]; Jakobi and Schwartz [21]; Jakobi et al. [38].
Cross-border living lab networks	Nina et al. [26]; Bergvall-Kåreborn et al. [31]; Jahn et al. [22]; Sovacool et al. [51]; Krogstie et al. [20]; Egusquiza et al. [52]; Dvarioniene et al. [29]; Kovács [27]; Buhr et al. [30].	Small and medium-sized enterprises (SMEs) innovation process and business models	Ståhlbröst [25]; Burbridge et al. [39].
Private homes and public/semi-public buildings	Claude et al. [36]; Jakobi et al. [21]; Sovacool et al. [51]; Krogstie et al. [20]; Jakobi and Schwartz [21]; Eon et al. [46]; Eon et al. [35]; Morgan et al. [40].	Education and health	Masseck [34]; Chin and Callaghan [23]; Pedell et al. [41].
Regional level	Giannouli et al. [51]; Dvarioniene et al. [29]; Provenzano et al. [45].	IT and feminist studies	Ahmadi et al. [50]; Ahmadi et al. [23].
City-wide/urban areas	Claude et al. [36]; Leminen et al. [3]; Buhr et al. [30]; Egusquiza et al. [52]; Jackson et al. [37]; Suopajärvi et al. [24].	Sustainable urban farming	Pertry et al. [44].

. Moreover, research studies on the subject were identified as case studies, reviews of living lab projects, and papers that provided project descriptions where the living lab approach/method enabled a real-life environment for project design and implementation.

Most of the articles included in the study were written from a European perspective, from countries such as Hungary [27,29], Belgium [48], Norway [49] and Sweden [32], while only three were identified from Australia [35,41,46]. This observation indicates that the living labs in the sampled studies are mainly perceived through a European lens and are barely understood outside this context.

While living labs have been set up across multiple scales or contexts, Table 5 shows that eight papers pertained to private homes and public/semi-public buildings. The latter also accounts for rooms in commercial buildings and living lab environments set up within different organisations in these buildings. Another dominant environmental context is a living lab as a city-wide/urban area initiative, as mentioned in six papers [3,24,30,36,37,52]. These environments include schools, neighbourhoods, buildings in city centres, municipalities, and Olympic parks or stadiums. These living labs served as a platform for people to engage in development and planning initiatives and did not necessarily function as business-driven technological research environments.

Similarly, the literature on living labs realised at regional levels was found not to focus on specific projects but rather to aim at promoting sustainable regional planning [47], as well as policymaking in innovation and research to improve people’s quality of life and reduce territorial and economic disparities [29,45]. Where living labs are implemented as networks across different cities or countries, these projects are noted to have the same aims, operations, dwelling types, and socio-economic characteristics, such as high unemployment and low education levels [26,30,51]. Three studies [42,49,53] allude to the context of the university, mainly using the entire campus or a building of a particular department at the university as a living lab experiment.

At its inception, the application of a living lab was concentrated in the corporate world as a novel way to include customers and have them commit to advancing technology [52]. Currently, living labs are being used practically in diverse fields, such as urban farming on warehouse rooftops in cities [44], the health sector [41], the education sector [34], IT, and feminist studies for addressing gender-blind innovations [43], enhancement of business models in the corporate field [39], and more widely in the IoT [21], ICT [24], and computer science fields [38].

An emerging research field in living labs is ‘smart cities’. According to Caragliu et al. [56], a city is smart when a high quality of life, coupled with long-term economic growth and sustainable resource management, is fuelled by its investments in modern ICT infrastructure and human/social capital in a participatory governance system. Smart cities are a multidisciplinary topic of interest within numerous development sectors, such as mobility, participatory governance, intelligent buildings, environment, and energy [42].

Several studies relating to living labs in the energy sector have been documented within a smart city context [33,49] and more specifically as ‘smart homes’ [20,26,51] or ‘smart buildings’ [22]. This field of research often integrates new energy-saving or energy-efficient technologies, such as smart metres measuring indoor climate, heat, and electricity consumption, microgrids, and renewable energy technologies with ICT and digital services. These papers are mostly identified within the framework of urban sustainability. What is also apparent in the literature on living labs for energy improvements is the emphasis on using smart energy systems embedded in homes and buildings as a method to affect, stimulate, and motivate environmental awareness and behavioural change of people in these spaces.

4. What Are the Roles of Stakeholders in a Living Lab?

This section addresses the last three questions on stakeholders: (v) Who are the stakeholders involved in a living lab outside and within the energy sector? (vi) How are the stakeholders identified to participate in living labs? and (vii) What are the stakeholder recruitment and engagement processes in living labs?

4.1. Stakeholders and Their Roles in a Living Lab Outside the Energy Context

The literature has significantly acknowledged the importance of involving end users and multiple stakeholders in a living lab. End users in living lab studies are perceived as co-designers who actively participate in developing products and services [23,24]. A stakeholder is described as an individual or group that is affected or potentially affected by a particular decision or development [28]. With this perspective, Dvarioniene et al. [29] p. 514, define stakeholder participation as “a process of involving everybody who takes an interest (or “stake”) in a project to foster its acceptance, get contribution and support as well as manage possible conflicts and oppositions”.

The stakeholders involved in living labs outside the energy sector were mentioned in 11 of the sampled studies (see

Table 6. Stakeholders and their roles in a living lab outside the energy sector.

Stakeholder	Role	Reference
Industry representatives (technology providers, designers, manufacturers and entrepreneurs)	- Offer technical support - Involved in designing, developing, and implementing technology	Buhr et al. [30]; Pertry et al. [44]; Jakobi et al. [3]; Jackson et al. [37]; Suopajärvi et al. [24]; Ahmadi et al. [50]; Ahmadi et al. [43].
Researchers, trainees and students (universities/companies)	- Provide methodological support for the living lab processes - Involved in prestudies, need finding, developing concepts, and testing innovation before implementation	Buhr et al. [30]; Pertry et al. [44]; Ahmadi et al. [50]; Jackson et al. [37]; Suopajärvi et al. [24]; Pedell et al. [41]; Chin and Callaghan [23]; Ståhlbröst et al. [28].
Users (residents, students, households, staff members, the elderly, and citizens)	- Contribute to contextual understanding of living lab by expressing their values, goals and needs regarding a particular situation - Participate in the design of technology - Test technology in real-life scenarios and provide feedback for evaluation	Buhr et al. [30]; Suopajärvi et al. [24]; Pedell et al. [41]; Chin and Callaghan [23]; Ståhlbröst et al. [28]; Jakobi et al. [38]; Masseck [34].
Financiers	- Fund the development and research of the living lab - Evaluate the progress of the project	Suopajärvi et al. [24]; Ståhlbröst et al. [28].
Public sector authorities (city councils and building managers)	- Provide real-world context by contributing their knowledge and experiences of a problem in a particular area - Also considered as problem owners	Buhr et al. [30].
Private sector companies and third sector organisations	- Provide a communication platform between users and project initiators - Involved in project planning, design, and implementation	Ahmadi et al. [50]; Buhr et al. [30].
Pilot manager and panel manager	- Interact with users and the wider community involved in a living lab - Provide coordination of real-world experiments and stakeholder engagements - Recruit users and disseminate information about the living lab	Ståhlbröst et al. [28].
Project manager	- Identifies and decides which actors can take part in the living lab project - Oversees research and technological innovations - Disseminates research results	Ståhlbröst et al. [28].
Business manager	- Develops business models and is responsible for the commercialisation of products or services	Ståhlbröst et al. [28].

). These studies fall within the sectors of health, education, urban farming, computer and social sciences, and ICT. The stakeholders identified include: (i) private companies and third sector organisations; (ii) researchers; (iii) financiers; (iv) project managers; (v) pilot and panel managers; (vi) business managers; (vii) users; (viii) industry representatives; and (ix) public sector authorities.

The most common stakeholders in the sampled studies outside the energy sector are the users, industry representatives and researchers. Furthermore, research institutes [23,24,34] at times partner with non-profit organisations [43,50]. Seldomly, municipal authorities [30] are the predominant initiators of living lab projects in these papers.

Apart from providing financial resources for technological innovations and research in living labs, the role of a financier is also to consistently evaluate the progress of the living lab [24,28]. Like the project manager, who has the role of overseer, living lab financiers are present from the initiation through the implementation and evaluation phases. However, they are not actively engaged in living lab processes on the ground. Evidence highlighting industry representatives comprises designers, manufacturers, technology providers [37], and entrepreneurs [30]. These actors are involved in designing, developing, and implementing technical solutions and providing general technical support or management in a living lab.

In an urban living lab initiated by the municipality, the private sector and third sector organisations adopted the role of being an intermediary that provides a communication platform between municipal authorities and civil society to express their goals and needs as well as being a coordinator that gathers all human and financial resources required to achieve the objectives of a living lab initiative [30]. Reviewing a ‘smart city living lab process’, Ståhlbröst et al. [28] further highlight the involvement of a pilot manager who is involved in setting up pilot projects within the living lab, a panel manager who primarily recruits and interacts with users, and a business manager who is responsible for the commercialisation of technology.

Regarding researchers as stakeholders, these actors are shown to be either from private research institutes or public universities and are sometimes identified as participatory design or action researchers, students, and trainees [28,30,50]. Researchers are involved in prestudies to discover the needs and stories of potential users, collect data during the testing phases, develop concepts, and support other methodological processes in the living lab [24,41].

Public sector authorities were identified as city councils and building managers. They contribute knowledge that provides the context of a problem in a particular area, similar to what users do during the need-finding phase. Users (mostly referred to as ‘end users’ or ‘participants’) differ according to the context of the living lab. For instance, in private home settings, they are noted as residents [30] or households [38], while in a medical context, they are described as ‘elderly people’ [41]. Ståhlbröst et al. [28] add that end users, municipal authorities, and building managers are stakeholders who have a moral claim on the living lab and are the owners of problems embedded in that place.

In theory, users are depicted as actual cocreators; however, this has been proven to be limited in practice. Leminen et al. [3] state that within a living lab, a user may adopt the role of being an informant, a contributor, a tester and/or a cocreator. A cocreator is described as an inexperienced technological designer who, without being made aware of the possibilities, may not accurately articulate what they want during the design processes. Hence, they work with other (‘expert’) stakeholders to design, develop, and produce tangible solutions in a mutual learning environment [41].

In practice, the role of the user was therefore primarily described as an informant during the conceptual phase, an experimenter or tester of existing technology during the implementation phase, and a contributor/collaborator influencing designs in the evaluation of existing technology or systems [24,28,38,41]. In a city-wide living lab environment, poor user involvement is attributed to the inability to define who the user is and the lack of financial resources [24].

4.2. Stakeholders and Their Roles in a Living Lab within the Energy Sector

The stakeholders involved in living labs in the energy sector are mentioned in 14 of the sampled papers and include the following: (i) financiers; (ii) public entities; (iii) researchers; (iv) SMEs; (v) large private businesses; (vi) trainees, teachers, and students; (vii) administrative personnel; (viii) users; and (ix) industry experts (

Table 7. Stakeholders and their roles in a living lab within the energy sector.

Stakeholder	Role	Reference
Users (building occupants, building managers, homeowners, residents, and staff members)	- Take part in need-finding surveys and other consumer research studies - Participate in brainstorming sessions to generate ideas for energy solutions - Test energy technologies in homes and provide feedback to increase usability	Jahn et al. [22]; Jakobi and Schwartz [21]; Nina et al. [26]; Sovacool et al. [51]; Ståhlbröst [25]; Morgan et al. [40]; Andersson and Rahe [32]; Eon et al. [46]; Eon et al. [34].
Public entities (municipal authorities and energy companies owned by local authorities)	- Provide a legal framework and public support for the living lab - Improve competence in qualification requirements for grants - Take part in brainstorming sessions for generating energy solutions - Support pilot projects initiated by local SMEs in the living lab - Municipal energy departments are a bridge between local authorities, SMEs, and universities, providing a platform that incentivises living lab methods	Claude et al. [36]; Krogstie et al. [20]; Nina et al. [26]; Giannouli et al. [47]; Egusquiza et al. [52].
Industry experts (architects, craftspeople, hardware developers, and engineering consultancies)	- Provide innovative solutions, skills and technical support for the living lab - Take part in brainstorming sessions to generate ideas for energy solutions - Involved in testing, implementation, evaluation, and management of systems and products - Support research projects in the living lab	Andersson and Rahe [32]; Ståhlbröst [25]; Claude et al. [36]; Krogstie et al. [20]; Egusquiza et al. [52]; Woods and Berker [49].
Researchers (universities and research bodies)	- Contribute scientific knowledge to support living lab processes - Provide evidence-based considerations in the decision-making process and adjust the customisation of testing strategies according to regulatory requirements, objectives, and resources of the living lab - Involved in data collection processes - Facilitate energy-efficient systemic innovations	Nina et al. [26]; Sovacool et al. [51]; Ståhlbröst [25]; Egusquiza et al. [52]; Woods and Berker [49].
Trainees, teachers and students	- Involved in data collection processes and administering of questionnaires - Involved in vocational training during living lab activities	Claude et al. [36].
Administrative personnel	- Deliver general communication about the living lab and specific stakeholder engagements	Woods and Berker [49].
Private businesses and SMEs	- Own energy technologies that are tested - May request funding from the project to develop technology as solutions to be tested in the living lab	Krogstie et al. [20]; Nina et al. [26]; Ståhlbröst [25].
Financiers	- Fund the living lab project to enable operations	Andersson and Rahe [32]; Ståhlbröst [25].

).

The prevalent stakeholders are the users, industry experts, researchers and public authorities. Moreover, partnerships between municipalities, universities (or research institutions), and private companies [26,32,36,40] are the main initiators of living lab projects relating to energy in the sampled studies.

The role of public entities has been stated in five studies [20,26,36,47,52]. These entities include actors such as state-owned energy companies and municipal authorities that provide a regulatory and legal framework for the living lab and incentivise SMEs to implement pilot projects. The financier plays a role in enabling operations within the living lab by contributing financial resources [25,32]. In Ståhlbröst [25], the SME actor is represented as the owner of energy technology that is tested in the living lab within private homes and can thus be seen as playing the role of a living lab developer or initiator. In contrast, private businesses, also referred to as energy market actors in Krogstie et al. [20], are seen as technology designers who are also at liberty to request funding from project initiators to develop energy products.

Another stakeholder group identified in energy living labs in six papers is industry experts, which include architects, craftspeople [52], hardware developers [25], and suppliers of technology [20]. This group provides industry input about techniques and relevant skills, tests products and systems in the living lab, assists in research analysis, and participates in brainstorming sessions with users and other stakeholders to come up with solutions to energy problems.

Researchers from research institutions contribute scientific and evidence-based knowledge to support decision-making and help realign testing strategies according to the living lab’s regulatory requirements, objectives, and resources [52]. Furthermore, in writing about the first energy living lab in the United Kingdom, Sovacool et al. [51] illustrate how researchers are the actors who primarily interact with end users in the living lab, from interviews before the installation of energy systems to surveys relating to users’ satisfaction with the system after installation and use. Actors such as trainees, teachers and students assist in data collection processes and partake in vocational training [36]. The administrator is responsible for sending out emails concerning living lab activities and engagements to other stakeholders [49].

The end users in energy living labs, commonly referred to as ‘participants’, are actors such as homeowners, residents, building occupants, and managers [22,51]. Their roles are elaborated in 9 of the 14 energy living lab papers, which allude to the end users’ involvement in various research studies and surveys as consumers of technology as well as their participation in the implementation and testing of installed software and energy devices such as smart metres to assess their effectiveness [21,26]. Similar to user roles observed in living labs outside the energy context described in Section 4.1, end users in operational energy living labs can be seen to serve as informants, contributors/collaborators and testers for evaluating existing energy innovations to increase usability rather than as actual cocreators of tangible solutions.

4.3. How Are Stakeholders Identified to Participate in a Living Lab?

There is limited evidence highlighting how stakeholders are identified in a living lab project. Only 2 out of 35 sampled studies [29,47] provided information on how stakeholders were identified. Stakeholders can be identified through the mind-mapping method or reverse mind mapping, done individually or during a group brainstorming session. In the two studies by Dvarioniene et al. [29] and Giannouli et al. [47], the mapping of stakeholders was used to generate a list of actors involved in a specific sector (i.e., energy), persons who are considerably affected by a decision or project, or diverse interest groups based on their skills, economic, and political interests or knowledge [29,47]. A visual matrix that clusters stakeholders according to their importance, influence, and power during planning or later implementation may also be drawn from the stakeholder list. In an energy lab context for enabling energy-conscious communities, Dvarioniene et al. [29] further allude to an in-depth stakeholder analysis that claims to be the best method for obtaining a well-represented final list of stakeholders. The authors assert that this simplifies how and when the stakeholders can be involved and helps determine their position or role in the project. The main issues to be considered in the deeper analysis of stakeholders are summarised in

Table 8. Summary of stakeholder analysis.

Stakeholder Group	What Are the Advantages That the Stakeholders May Have when They Contribute to or Are Involved in the Project?	What Are the Disadvantages That the Stakeholders May Have when They Contribute to or Are Involved in the Project?	Evaluations of the Stakeholders’ Contribution or Position
……………	……………..	………………	……………..

Source: Adapted from Dvarioniene et al. [24] p. 515.

.

4.4. What Are the Stakeholder Recruitment and Engagement Processes?

Literature explaining the recruitment process of stakeholders and end users involved in a living lab is limited. Nonetheless, cold-call techniques that involve sending out email newsletters that call for participation, individually contacting potential stakeholders via telephone, and attending subject-related events to engage with relevant persons are highlighted in one study [43].

End-user recruitment in research to enhance the design interface of smart home technology began by first publicising the living lab study via local radio stations and newspapers [38]. Participants were thus selected when they fulfilled the criteria set by the researchers, which were done through an online portal and later through telephone interviews. Households had to fall within a particular postcode, had to have a stable internet connection, had to provide adequate reasons for wishing to participate, and had to express their project expectations and their knowledge of products and hard-/software prototypes [38].

Stakeholder recruitment processes in most papers concerning living labs in the energy sector are vague. The papers state that end users were either recruited voluntarily [25] or invited to test and evaluate technology [26], subject to their interest in energy solutions [20,36]. In Jahn et al. [22], end users as building occupants seemed not formally recruited but rather participated in the living lab study because operational changes were occurring within their living space. Only Sovacool et al. [51] elaborate on the process of recruitment of end users, which was through the use of telephone interviews. Further, the user recruitment in Sovacool et al. [51] energy living lab was based on a screening process that included households with specific technological infrastructure, permanent residents, and people possessing knowledge about smart energy technologies.

Evidence indicating the operations of engagement amongst various stakeholders and end users involved in a living lab is elaborated in 12 of the 35 studies. Researchers, technology developers, and end users mostly engage directly with each other through a series of face-to-face interviews [46], telephone interviews [25,51], user-pool brainstorming workshops [20], kick-off events [38], questionnaires [49], roundtable discussions [30], and technical workshops [21]. The latter engagements are utilised as instruments for data collection for researchers and developers to enhance their contextual understanding of users and their needs during the early phases of the living lab, inform users about the research programme, and train users on how to install smart plugs.

The literature shows that after the setup and installation phase and during the final technology or system development phase, end users can exchange their immediate experiences with the user community, convey problems, and receive solutions from researchers via online discussion forums, instant messenger groups, and regular phone calls [25,38]. This encourages ongoing dialogue and cultivates close relationships between users and researchers. Overall, engagements between all participating stakeholders in a living lab ensue through regular meetings [50], occasional technical and codesign workshops, conferences [29,40], and focus groups [30,32]. This is where certain stakeholders present the plans and progress of the living lab, and the codesign of technological and scientific solutions amongst expert stakeholders and municipal officials occurs [49,52].

Keeping stakeholders and end users engaged throughout the living lab process is one of the obstacles to collaboration. Some partners either become passive in their participation due to undefined roles and expectations, or the level of commitment decreases, resulting in their absence from certain activities [32,50]. In the case of end users, valuable quantitative data were lost due to household renovations, specific inhabitants travelling or selling their homes while the study was in progress, and users not providing feedback [24,46].

In the literature, there is a gap regarding gendered considerations in living labs in the energy context. The power relations of gender in innovation and design decision-making processes in the IT and ICT industries were, however, highlighted by three studies [24,41,50].

5. Discussion

Upon reflecting on insights, existing gaps, and the incongruity between the living lab concept and practical implementation, as revealed in Section 3 and Section 4, the authors therefore propose a generic process that can be utilised to support the setup and management of an energy living lab in urban informal settlements in South Africa, which consists of six phases, each specifying a purpose and the relevant action (Figure 3). We posit that to conceptualise and support measuring the outputs, outcomes, and impacts of the energy living lab, the framework needs to capture the vision.

The first phase specifies the societal problem or challenge to be addressed. It involves asking probing questions to capture the issue addressed, identifying the initiator of the issue, understanding why it is crucial, and specifying the project’s vision. The second phase requires describing the application context. For example, it can be an urban, regional, or rural case study. Questions that can be explored capture the application and identified geographic territory.

The third phase discusses the type of living lab, namely a sustainability living lab, an urban living lab, an urban transition living lab, or any other type. The fourth phase specifies the field of application of living lab, for example, the energy sector, health, information technology, and so on.

The fifth phase maps the stakeholders involved. A major gap concerning living lab stakeholders is the vagueness of the process of stakeholder identification, recruitment, and engagement in the sampled studies on energy living labs. A critical aspect to therefore consider is identifying the key stakeholders and describing their roles, how they are recruited, the selection process, and their contribution to the cocreation of solutions. Various stakeholders were identified in energy living lab studies, which include (i) financiers; (ii) public entities; (iii) researchers; (iv) SMEs; (v) large private businesses; (vi) trainees, teachers, and students; (vii) administrative personnel; (viii) users; and (ix) industry experts. Mapping the stakeholders as proposed in the framework can contribute to the energy living lab setup and management knowledge base.

The sixth phase focuses on monitoring and evaluation, which compares anticipated and expected benefits to actual benefits, outputs, outcomes, and impacts. It entails the living lab going through an iterative follow-up process to keep track of its activities and milestones. The authors recommend documenting these processes in order to inform the development of energy living labs and to enable monitoring and evaluation of the outputs, outcomes, and impact of the living labs.

Another aspect to consider in the conceptualisation of energy living labs is the integration of gender issues to account for the different roles and responsibilities of men and women in the energy transition. Initiators may need to identify which aspects of the energy living lab are associated with gender issues. In the context of urban households, living lab initiators may need to collect gender-disaggregated data on household heads and their potential influence on household energy transitions. The energy living lab at the regional level may entail asking questions about gender-conscious and gender-blind policies that persist and their implications for social planning for energy transitions.

6. Conclusions

This paper explored the concept of a living lab to understand the scope of application within the energy sector and the roles of the stakeholders involved. Using a systematic literature review, we sampled 35 studies that were relevant for analysis. Of the 35 studies, 17 were on energy-related living labs, and 18 were on living labs in other fields. The authors do, however, acknowledge that, given that the main focus of the study was on the energy sector, the list of papers describing the living lab concept is not exhaustive. As such, an area of future study may be to systematically identify projects involving energy living labs, what the stakeholders and end-users do, and later assess the related scientific publications within those projects. To conclude, the study findings show that although the concept of a living lab has been applied in other fields, its application within the urban energy sector is limited but emerging. Furthermore, evidence describing how key stakeholders are recruited and their specific contributions to co-creation is vague. As our results have illustrated, there is a gap in the literature regarding the application of the living lab concept in the energy sector, particularly in urban Africa. The majority of the sampled studies included living lab cases in Europe. This study therefore contributes to the growing body of knowledge of energy living labs, with a focus on urban Africa. It provides insight into the conceptualisation of an energy-living lab in urban Africa, informed by consideration of context-specific aspects. The proposed generic process can help living lab initiators understand how to set up and manage the energy living lab, as well as iteratively monitor outcomes and impacts to ensure that the living lab is sustainable. The process can serve as a foundation for scoping a specific energy problem based on the context, as well as strengthening the coordination of major and vulnerable groups who contribute to the living lab’s knowledge base. This can improve the well-being of individuals and foster access to innovative, safe, and reliable energy services, all of which contribute to achieving sustainable energy for all.