**Preface to "Climate Change and Environmental Sustainability-Volume 2"**

The Earth's climate is changing; the global average temperature is estimated to already be about 1.1 °C above pre-industrial levels. Indeed, we are now living in conditions of a climate emergency. Climate change leads to many adverse events, such as extreme heat, flooding, bushfire, drought, and many other associated economic and social consequences. Further warming is projected to occur in the coming decades, and climate-induced impacts may exceed the capacity of society to cope and adaptive in a 1.5 °C or 2 °C world. Therefore, urgent actions should be taken to address climate change and avoid irreversible environmental damages.

Climate change is interrelated with many other challenges such as urbanisation, population increase and economic growth. For instance, cities are now the main settlements of human being and are major sources of greenhouse gas emissions that are key contributors to climate change. Moreover, rapid and unregulated urbanisation in some contexts further causes urban problems such as environmental pollution, traffic congestion, urban flooding and heat island intensification. In the absence of well-designed measures, increasing urbanisation trends in the next two–three decades are likely to further aggravate such problems. Overall, climate change and many other challenges have deteriorated the sustainable development of the world.

The United Nations proposed the Sustainable Development Goals in 2015. Goal 13, Climate Action, emphasises the need for urgent action to combat climate change and its impacts in order to enhance sustainability. To achieve this, there is a need to develop a holistic framework that considers mitigation—the decarbonisation of society—to address the challenge of climate change from the root, and adaptation—an immediate action—to increase the resilience of and protect society from climate-induced hazards. The framework prioritises the transformation of the traditional methods of environmental modifications in various fields, including transportation, industry, building, energy generation, agriculture, land use and forestry, towards sustainable ones to limit greenhouse gas emissions. The framework also highlights the significance of sustainable environmental planning and design for adaptation in order to reduce climate-induced threats and risks. Moreover, it encourages the involvement and participation of all stakeholders to accelerate climate change mitigation and adaptation progress by developing sound climate-related governance systems.

The framework also calls for the support and engagement of all societal stakeholders. To support the achievement and implementation of the framework, this book focuses on climate change and environmental sustainability by covering four key aspects, including climate change mitigation and adaptation, sustainable urban–rural planning and design, decarbonisation of the built environment in addition to climate-related governance and challenges. Climate change mitigation and adaptation covers topics of greenhouse gas emissions and measurement, climate-related disasters and reduction, risk and vulnerability assessment and visualisation, impacts of climate change on health and well-being, ecosystem services and carbon sequestration, sustainable transport and climate change mitigation and adaptation, sustainable building and construction, industry decarbonisation and economic growth, renewable and clean energy potential and implementation in addition to environmental, economic and social benefits of climate change mitigation.

Sustainable urban–rural planning and design deals with questions of climate change and regional economic development, territorial spatial planning and carbon neutrality, urban overheating mitigation and adaptation, water-sensitive urban design, smart development for urban habitats, sustainable land use and planning, low-carbon cities and communities, wind-sensitive urban planning and design, nature-based solutions, urban morphology and environmental performance in addition to innovative technologies, models, methods and tools for spatial planning. Decarbonisation of the built environment addresses issues of climate-related impacts on the built environment, the health and well-being of occupants, demands on energy, materials and water, assessment methods, systems and tools, sustainable energy, materials and water systems, energy-efficient design technologies and appliances, smart technology and sustainable operation, the uptake and integration of clean energy, innovative materials for carbon reduction and environmental regulation, building demolition and material recycling and reusing in addition to sustainable building retrofitting and assessment. Climate-related governance and challenges concerns problems of targets, pathways and roadmaps towards carbon neutrality, pathways for climate resilience and future sustainability, challenges, opportunities and solutions for climate resilience, the development and challenges climate change governance coalitions (networks), co-benefits and synergies between adaptation and mitigation measures, conflicts and trade-offs between adaptation and mitigation measures, mapping, accounting and trading carbon emissions, governance models, policies, regulations and programs, financing urban climate change mitigation, education, policy and advocacy of climate change mitigation and adaptation in addition to the impacts and lessons of COVID-19 and similar crises.

Overall, this book aims to introduce innovative systems, ideas, pathways, solutions, strategies, technologies, pilot cases and exemplars that are relevant to measuring and assessing the impact of climate change, mitigation and adaptation strategies and techniques in addition to public participation and governance. The outcomes of this book are expected to support decision makers and stakeholders to address climate change and promote environmental sustainability. Lastly, this book aims to provide support for the implementation of the United Nations Sustainable Development Goals and carbon neutrality in efforts aimed at achieving a more resilient, liveable and sustainable future.

Our world is facing many challenges, such as poverty, hunger, resource shortage, environmental degradation, climate change, and increased inequalities and conflicts. To address such challenges, the United Nations proposed the Sustainable Development Goals (SDG), consisting of 17 interlinked global goals, as the strategic blueprint of world sustainable development. Nevertheless, the implementation of the SDG framework has been very challenging and the COVID-19 pandemic has further impeded the SDG implementation progress. Accelerated efforts are needed to enable all stakeholders, ranging from national and local governments, civil society, private sector, academia and youth, to contribute to addressing this dilemma. This volume of the *Climate Change and Environmental Sustainability* book series aims to offer inspiration and creativity on approaches to sustainable development. Among other things, it covers topics of COVID-19 and sustainability, environmental pollution, food production, clean energy, low-carbon transport promotion, and strategic governance for sustainable initiatives. This book can reveal facts about the challenges we are facing on the one hand and provide a better understanding of drivers, barriers, and motivations to achieve a better and more sustainable future for all on the other. Research presented in this volume can provide different stakeholders, including planners and policy makers, with better solutions for the implementation of SDGs.

Prof. Bao-Jie He acknowledges the Project NO. 2021CDJQY-004 supported by the Fundamental Research Funds for the Central Universities. We appreciate the assistance from Mr. Lifeng Xiong, Mr. Wei Wang, Ms. Xueke Chen and Ms. Anxian Chen at School of Architecture and Urban Planning, Chongqing University, China.

> **Bao-Jie He, Ayyoob Sharifi , Chi Feng, Jun Yang** *Editors*

### *Article* **A Framework to Evaluate the SDG Contribution of Fluvial Nature-Based Solutions**

**Tatiana Andrikopoulou 1,\*, Ralph M. J. Schielen 1,2, Chris J. Spray 3, Cor A. Schipper <sup>2</sup> and Astrid Blom <sup>1</sup>**


**Abstract:** Nature-based solutions (NBSs) are measures reflecting the 'cooperation with nature' approach: mitigating fluvial flood risk while being cost-effective, resource-efficient, and providing numerous environmental, social, and economic benefits. Since 2015, the United Nations (UN) 2030 Agenda has provided UN member states with goals, targets, and indicators to facilitate an integrated approach focusing on economic, environmental, and social improvements simultaneously. The aim of this study is to evaluate the contribution of fluvial NBSs to the UN 2030 Agenda, using all its components: Sustainable Development Goals (SDGs), targets, and indicators. We propose a four-step framework with inputs from the UN 2030 Agenda, scientific literature, and case studies. The framework provides a set of fluvial flooding indicators that are linked to SDG indicators of the UN 2030 Agenda. Finally, the fluvial flooding indicators are tested by applying them to a case study, the Eddleston Water Project, aiming to examine its contribution to the UN 2030 Agenda. This reveals that the Eddleston Water Project contributes to 9 SDGs and 33 SDG targets from environmental, economic, societal, policy, and technical perspectives. Our framework aims to enhance the systematic considerations of the SDG indicators, adjust their notion to the system of interest, and thereby enhance the link between the sustainability performance of NBSs and the UN 2030 Agenda.

**Keywords:** nature-based solutions; river; flooding; sustainability; sustainable development goals; indicators

### **1. Introduction**

According to the United Nations (UN) 2015 report, 'The Human Cost of Weather-Related Disasters' [1], flooding has negatively affected 2.3 billion people over the last 20 years. This accounts for 56% of all those negatively affected by weather-related disasters such as droughts, storms, landslides, and extreme temperatures (64.4 million/year). Especially for fluvial floods, the number of affected people under the most extreme river flooding scenario and without further adaptation may rise from 39 million people per year to 134 million people per year by 2050. Approximately two-thirds of this increase can be attributed to increases in the severity and frequency of flooding due to climate change and the remainder due to population growth in flooding-prone areas [2].

Rapid development combined with the expansion of infrastructure, agricultural intensification, transport, and other linked socioeconomic systems has increased society's vulnerability to environmental disasters, especially in floodplain areas [3]. At the same time, climate change is an important driver for implementing sustainable practices in protecting and managing river ecosystems. In this context, the UN 2030 Agenda [4] has provided international and national governments with goals, targets, and indicators to facilitate an integrated approach focusing on economic, environmental, and social improvements

**Citation:** Andrikopoulou, T.; Schielen, R.M.J.; Spray, C.J.; Schipper, C.A.; Blom, A. A Framework to Evaluate the SDG Contribution of Fluvial Nature-Based Solutions. *Sustainability* **2021**, *13*, 11320. https://doi.org/10.3390/ su132011320

Academic Editor: Antonio Miguel Martínez-Graña

Received: 1 August 2021 Accepted: 5 October 2021 Published: 13 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

simultaneously. Since 2015, all UN member states are expected to pursue these Sustainable Development Goals (SDGs), tailoring a path towards a peaceful and prosperous planet.

Nature-based solutions (NBSs) can help in addressing many of the SDGs as established in the UN 2030 Agenda. The inclusion of natural elements could create manifold benefits for all the three pillars – 'People', 'Planet', and 'Prosperity' which reflect the three sustainability principles (society, environment, economy) and are adopted by the UN 2030 Agenda. From a societal perspective, they could provide access to nature and recreation while adding cultural and heritage value to the landscape. From ecological and environmental perspectives, they could enhance biodiversity and contribute to water and air purification. From an economic viewpoint, they could promote sustainable and responsible resource management, resulting in cost-effective practices. In Europe, nature-based protection measures (green-blue-hybrid) have already gained increasing prominence in application [5–8]. Green/Blue infrastructure indicates a strategically planned network of natural and semi-natural areas with other environmental features designed and managed to deliver a wide range of ecosystem services. It incorporates green spaces (or blue if aquatic ecosystems are concerned) and other physical features in ter-restrial (including coastal) and ma-rine areas. Hybrid solutions mix hard infrastructure with ecosystem-based infrastructure (https://portals.iucn.org/library/sites/library/files/documents/2016-036.pdf, accessed on 10 October 2021). In global relevance, the effort to learn, implement and promote NBSs is worldwide and supported by many programs and data pools [9].

To date, several NBSs frameworks have been developed to comprehensively describe, analyze, and assess the planning, implementation, and operationalization of NBSs projects. Typically, they include indicators for benchmarking, assessing, or measuring the performance or (co-)benefits of NBSs under several hydro-meteorological hazards (HMHs). Kumar et al. [10], Ruangpan et al. [11], Shah et al. [12], and Albert et al. [13], for instance, developed a single NBS framework covering at least four HMHs. The type of environment under consideration typically differs; Kumar et al. [10], Ruangpan et al. [11], Calliari et al. [14], and Nesshöver et al. [15] do not focus on a single environment, in contrast to many other studies wherein a specific type of environment is the focus. The environments most studied are urban, large rivers (250–300 km), and coasts [11,13,16–21].

Sustainability is addressed in recent NBSs-related frameworks either with the inclusion of the three pillars in the assessment of the NBSs' performance or their (co-)benefits or by measuring the sustainability performance according to the components of the UN 2030 Agenda. Initially, studies such as those by ones of Artmann et al. [22], Pakzad et al. [23], Raymond et al. [20] showed that NBSs interact across and within society, economy, and environment. Building on that, subsequent studies (e.g., [18,21,24]) examined the potential contributions of NBSs to the UN 2030 Agenda by examining the SDGs and/or their targets. Schipper et al. [21] developed the Sustainability Impact Score (SIS) Assessment Framework, which uses a selection of SDGs and SDG targets to score the sustainability performance of coastal management projects. Whilst it is apparent that some of the recent frameworks address the SDGs and/or SDG targets from the UN 2030 Agenda, they omit consideration of the SDG indicators.

However, in scrutinizing the UN 2030 Agenda, it is noticeable that (i) often, SDG targets refer to multiple elements which are broken down into SDG indicators, and (ii) 12 SDG indicators are repeated (some with slight amendments) under different SDG targets. The SDG indicators (rather than the SDG targets) seem to have the right abstraction level to serve as a connection between the NBSs and the UN 2030 Agenda. Although reaching the SDGs in itself is a promising achievement in preserving our planet, as stated by the United Nations, using the SDG indicators could bring a new perspective to the effort of linking the NBSs to SDGs and hence to assessing the contribution of NBSs to the achievement of the SDGs.

To bridge the gap identified in the research to date, the aim of this study is to evaluate the sustainability performance of NBSs projects with respect to the UN 2030 Agenda, involving all its three components: SDGs, SDG targets, and SDG indicators. In other

words, we seek the SDGs, SDG targets, and SDG indicators to which NBSs projects could contribute. We focus on NBSs projects for fluvial flood risk mitigation (FFRM) implemented in riverine ecosystems up to 100 km2, which is a smaller scale than that examined in riverine environments to date. Specifically, we aim to evaluate the sustainability performance of NBSs projects for FFRM by:


We consider case studies from countries with high-income economies only, as NBSs projects in countries with upper-middle, lower-middle, and low-income economies typically aim to cover more fundamental needs, such as water quality and scarcity, and flood mitigation is seldom the main driver for NBSs implementation. Interaction of the river ecosystem with the coastal environment is out of the scope of this research.

### **2. Methodology**

The methodology of this study uses the SIS Assessment Framework in a way that looks to build on the systematic methodology introduced by Schipper et al. [21], but with focus on river ecosystems, recognizing and introducing new elements reflecting the scope of the research. With the UN 2030 Agenda as a starting point, four steps are considered that eventually lead to the formation of the framework. Subsequently, the framework is presented along with the four steps through which our aim is accomplished and, ultimately, a case study to test its applicability.

### *2.1. The Sustainability Performance Evaluation Framework*

The Sustainability Performance Evaluation Framework presented here is derived from the SIS Assessment Framework with the necessary alterations. It encompasses a systematic methodology for creating a set of fluvial flooding indicators, linking them to the SDG indicators, and evaluating the sustainability performance of an FFRM NBSs project through four steps. Starting with the components of the UN 2030 Agenda as input (Figure 1), Step I defines the fluvial flooding indicators which relate to NBSs for FFRM. Subsequently (Step II), the SDGs relevant to NBSs for FFRM, along with the respective SDG targets and SDG indicators, are selected from the UN 2030 Agenda. In Step III, Step I and Step II are brought together, creating a set of NBSs fluvial flooding indicators that demonstrate the FFRM NBSs' contribution to the UN 2030 Agenda. The definition of fluvial flooding indicators and their subsequent connection to the SDG indicators (instead of directly using the SDG indicators) makes it possible to overcome an apparent lack of conceptual clarity inherent in some of the SDG indicators due to their universal nature. Finally, Step IV consists of the assessment of the fluvial flooding indicators based on their application to a case study with available project metadata, with the sustainability performance of the project as the main output.

### *2.2. Step I—Definition of Dimensions and Fluvial Flooding Indicators*

The scope of the sustainability evaluation is defined through the identification of the NBSs dimensions. The word 'dimension' is chosen above terminology such as 'property' or 'aspect' to emphasize the broadness of the NBSs (co-)benefits. The NBSs dimensions express sectors that are affected by FFRM NBSs projects in river ecosystems. For instance, floodplain ponds will, in addition to temporarily storing water during floods, provide habitats for wildlife and support biodiversity. Therefore, wildlife and biodiversity are two sectors that are affected by floodplain ponds and are expressed by the Environmental dimension in our study.

**Figure 1.** The Sustainability Performance Evaluation Framework, using a selection of SDGs, SDG targets, and SDG indicators published in the 2030 Agenda for Sustainable Development. The Sustainability Performance Evaluation Framework output shows the contribution of an NBSs project for FFRM (and the reason for it) to the UN 2030 Agenda (Figure adapted from [21]).

The identification of the dimensions is based upon an analysis of 7 existing NBSs frameworks or reviews ([14,15,20,22,25–27]), complemented by the examination of three case studies (Table 1). Case studies are used in order to validate that the framework/review findings are realistic and to add any relevant findings that might have been neglected in the literature. The selection of the case studies was based on the following five independent criteria. The criteria are not prioritized; the order is indicative.



**Table 1.** The selected projects examined as case studies. They contribute to the definition of the dimensions and their fluvial flooding indicators (Step I).

Each dimension consists of fluvial flooding indicators, as shown in Figure 2, that act as a metric that condenses complexity and provides relevant information [48]. The aim of the fluvial flooding indicators is to list specific effects that might occur in a dimension when implementing an NBSs project for FFRM. For instance, biodiversity abundance is a fluvial flooding indicator that can be found under the Environmental dimension (Figure 2). A preliminary list of fluvial flooding indicators was created by:

(i) Collecting existing indicators from literature. The collection of indicators comes from the 7 frameworks reviewed for the dimensions. However, starting with the already identified frameworks and using snowballing techniques, three additional frameworks were identified that also revealed additional indicators [23,49,50].

(ii) Using the indicators derived from (i) in analyzing the three case studies (Table 1), chosen to reflect a reasonable geographical coverage, spread in surface and in geomorphological aspects and with enough information at hand to quantify the indicators. In this process, the case studies gave rise to several new indicators that were not included in Step (i).

**Figure 2.** Within each of the five dimensions, the fluvial flooding indicators represent potential effects of NBSs projects for FFRM in the respective dimension. This figure aims to show the structure of the dimensions and their fluvial flooding indicators. The exact dimensions and their fluvial flooding indicators are fully explained in the Results section (Steps I and III).

### *2.3. Step II—Selection of Relevant SDG Targets and SDG Indicators*

The Sustainable Development Goals, targets, and indicators constitute very broad but versatile milestones that users may need to adapt depending on the context and their precise area of interest. As the goals themselves are very broad, the starting point for examination in this study is the 169 targets, followed by the 247 indicators. In reviewing these, the aim is to establish what they address and then select those SDG targets and SDG indicators that are relevant to NBSs for FFRM. For this purpose, a screening process has been developed (Figure 3) to help select relevant SDG targets and SDG indicators according to (i) the boundary conditions (high-income economies, river ecosystem) and (ii) potential NBSs' contribution to them for FFRM. The latter, in particular, has been developed from insights gained from the literature review and case studies. The selection process is presented in Figure 3:

**Figure 3.** The screening process that selects SDG targets and SDG indicators relevant to NBSs for FFRM. Eventually, the relevant SDG indicators can be linked to the fluvial flooding indicators, as explained in Step III (hexagonal purple box).

### *2.4. Step III—Connection of the SDG Indicators with the Fluvial Flooding Indicators*

Having selected the relevant SDG targets and SDG indicators from the UN 2030 Agenda (Step II), the SDG indicators were connected with the preliminary list of fluvial flooding indicators, as formed in Step I. The connection was made at a conceptual level: matching the description of the fluvial flooding indicator with the SDG indicators. For instance, the biodiversity abundance fluvial flooding indicator was connected to an SDG indicator that addresses the presence and diversity of species.

### *2.5. Step IV—Assessment of the Fluvial Flooding Indicators Based on Project Metadata*

Finally, the connected fluvial flooding indicators (Step III) are applied to a selected case study using project metadata as input to them. Through examination of the fluvial flooding indicators, the sustainability performance of a selected FFRM NBSs project with respect to the UN 2030 Agenda is evaluated.

### **3. Results**

### *3.1. Step I—Definition of Dimensions and Fluvial Flooding Indicators*

Five dimensions are defined: Environment, Economy, Society, Policy—Procedural, and Technical. The Environment, Economy, and Society dimensions represent the three pillars of sustainability. These three pillars, along with the Policy—Procedural dimension, can all be found as broad divisions within the UN 2030 Agenda [51]. The Environment, Economy, and Society dimensions are also used by other frameworks addressing either NBSs sustainability or the additional benefits that NBSs bring [17,19–21,52]. The Technical dimension is a new addition that is considered highly relevant because it refers to the fulfillment of the objective of the intervention (flood protection) and to characteristics that the intervention should comply with, including structural integrity, reliability, ease of implementation, adaptability, and resilience. The Technical dimension has recently been introduced in the literature. The study of Pugliese et al. [19] uses the framework introduced by PHUSICOS [53], where the Technical dimension is used as an ambit to examine the NBSs' technical and economic feasibility aspects. The Policy—Procedural dimension is usually found in the Society dimension. This is the case both in the EC Handbook for Practitioners [52], which places the 'Participatory Planning and Governance' under the People pillar of Sustainable Development, and in the PHUSICOS framework [53] that includes the 'Community Involvement and Governance', in the Society ambit. In the Sustainability Performance Evaluation Framework, we distinguished the Policy— Procedural dimension from the Society one, aligning with the UN 2030 Agenda [4], which devotes entire goals to partnerships (Goal 17) and inclusive collaboration (Goal 16).

A preliminary list of 32 fluvial flooding indicators was identified spread across the five dimensions (Environment 8, Society 5, Economy 5, Technical 6, Policy—Procedural 8) to describe the effects of NBSs for FFRM. Overall, 24 of the fluvial flooding indicators were collected from the NBSs frameworks and reviews examined; three emerged from the case studies, and five technical fluvial flooding indicators were introduced by the authors adjusted after Slinger, J.H. [54]. A detailed table with all the dimensions, their fluvial flooding indicators, and their use is presented in the description of Step III to avoid repetition.

### *3.2. Step II—Selection of Relevant SDG Targets and SDG Indicators*

Table 2 shows the selection of 10 SDGs, 42 SDG targets, and 51 SDG indicators as relevant to NBSs for FFRM, as derived from the screening process. The selection starts by examining, one by one, all the SDG targets following the screening process (Figure 3). For each SDG target that was considered relevant to the FFRM NBSs, at least one of its SDG indicators also had to be FFRM-NBSs-relevant. The explanation as to which SDG indicator is considered relevant is shown in the fourth column of Table 2 and is derived from our examination of the literature and case studies.

### *3.3. Step III—Connection of the SDG Indicators with the Fluvial Flooding Indicators*

The preliminary list of fluvial flooding indicators was coupled with the relevant SDG indicators, resulting in 21 out of 32 fluvial flooding indicators being linked to various of the 51 SDG indicators. It was expected that not all the fluvial flooding indicators would be linked to the relevant SDG indicators since the intention was to match a targeted—to fluvial flooding—list specifically derived from the authors' examination of literature and practice to a universal solid agenda. However, since the UN 2030 Agenda is a universally recognized policy framework and the aim of this study is to examine the FFRM NBSs's

contribution to it, the preliminary list was extended to cover all the relevant SDG indicators. A total of 12 fluvial flooding indicators were added in the Policy—Procedural dimension, which can be seen in Table 3, rows #29–34 and #39–44. Therefore, the final list comprises 33 fluvial flooding indicators coupled with all the 51 relevant SDG indicators.

**Table 2.** In total, 10 SDGs, 42 SDG targets, and 51 SDG indicators are considered relevant to NBSs projects for fluvial flood risk mitigation. These 51 relevant SDG indicators will be connected with the fluvial flooding indicators in Step III. For a full description of the SDG targets and SDG indicators, reference should be made to the UN 2030 Agenda.



**Table 2.** *Cont*.


**Table 2.** *Cont*.

**Table 3.** All the 44 fluvial flooding indicators. The first 32 fluvial flooding indicators that belong to the preliminary list come from (i) literature review [14,15,20,22,25–27,54] and (ii) case study examination [28–47]. From these, 21 link to SDG indicators whilst the 11 remaining do not (-). In rows #29–34 and #39–44, there are 12 new fluvial flooding indicators. On this base and with project metadata, the final list of 33 fluvial flooding indicators will show the contribution of an NBSs project for FFRM to the UN 2030 Agenda (Step IV).



**Table 3.** *Cont*.


**Table 3.** *Cont*.


**Table 3.** *Cont*.

Table 3 presents the five dimensions with their respective fluvial flooding indicators (first four columns) and their connection with the SDG indicators (last two columns).

### *3.4. Case Study: The Eddleston Water Project*

The Eddleston Water Project was selected as a representative case study to test the use of the fluvial flooding indicators. Importantly for our assessment, the Eddleston Water Project adopts a multi-benefit approach to the use of NBSs aiming at (i) exploring whether flood risk can be reduced by means of NBSs, (ii) the use of NBSs for improving the ecological condition of the river, and (iii) working with landowners and communities to maintain and enhance sustainable land management practices and farm businesses. Furthermore, since the measures were implemented in 2013, preliminary outcomes from the monitoring campaigns are already available. Finally, it is also part of the EU North

Sea Region (NSR) Interreg Building with Nature (BwN) program (https://northsearegion. eu/building-with-nature/), providing good links with experts if further consultation was needed (see also Appendix A).

The details of the Eddleston Water Project are summarized in the first column of Figure 4. A full description is available on the project website (https://tweedforum.org/ our-work/projects/the-eddleston-water-project/). The Eddleston Water Project started as a learning-by-doing project, which is successfully evolving and revealing valuable insights as to how a catchment approach reduces flood risk, involving both structural measures and natural flood management (NFM), and may help improve resilience to climate change. A key element throughout the project has been close stakeholder consultation because uptake of NBSs measures is voluntary, and all the locations for NBSs measures within the project catchment are privately owned. Local land managers and the wider community had been engaged from the very beginning of the project, and these and other stakeholders are still actively involved through regular meetings and surveys, ensuring productive continuation and uptake of the project.



**Figure 4.** Eddleston Water Project key information. Photos of the Lake Wood site, Eddleston Water Project: the previously straightened reach (top photo), the site immediately after the completion of the re-meandering works (middle photo), and the site one year after the completion of the re-meandering works with small consecutive floodplain ponds and the new re-meandered reach (bottom photo). All were retrieved from the Tweed Forum website.

The present year (2021) was the end of the 5 years of matched funding for the Eddleston Water Project from the EU NSR Interreg BwN program. The Interreg program focused on assessing the costs and benefits of implementing NFM through improved monitoring and modeling. With ongoing support for the current phase (2021–2024) from the Scottish Government, and the participation of local farmers and landowners, the study continues with the implementation of different types of NFM measures across the catchment, alongside detailed hydrological and ecological monitoring. Some of the headline outcomes from the Eddleston Water Project so far are summarized in the second column of Figure 4. For a more extended description of the project, reference is made to the Eddleston Water Project Report [55]; to the paper on flood risk reduction [56]; and to the Tweed Forum website (see above) where all the reports, including those from the Building with Nature program, are made publicly available.

### *3.5. Step IV—Assessment of the Fluvial Flooding Indicators Based on Project Metadata*

To assess the utility and effectiveness of the fluvial flooding indicators, they were applied to the Eddleston Water Project to examine its sustainability performance in terms of:


The application was performed with input metadata from the Eddleston Water Project to the fluvial flooding indicators. The metadata for the Eddleston Water Project in Table 4 were collected from the Project Reports and [56–59]. The output of the evaluation is presented in Table 4, with three columns and the following format: each fluvial flooding indicator (Column I) contributes to none/one/or more SDGs and SDG targets (Columns II and III) as justified by the Eddleston Water Project metadata (Column IV). The contribution of the Eddleston Water Project to the UN 2030 Agenda is presented in Table 4 in terms of SDGs because this is more practical and easier to remember as a take-home message. However, by referring back to Table 3, it is possible to see the derivation and connection between the relevant SDG targets and respective SDG indicators. For instance, in Table 4, it can be seen that the Eddleston Water Project contributes to SDG 15 and SDG target 15.1, as assessed by the biodiversity abundance (fluvial flooding) indicator according to available project metadata. By referring back to Table 3 and the biodiversity abundance indicator (#1), it can be seen that the Eddleston Water Project contributes to SDG indicator 15.1.2. Therefore, the Eddleston Water Project contributes to the SDG indicator 15.1.2, SDG target 15.1, and SDG 15 of the UN 2030 Agenda in terms of biodiversity abundance.

**Table 4.** Contribution of the Eddleston Water Project to the relevant SDGs and SDG targets (second, third column), examined per fluvial flooding indicator (first column) according to Eddleston Water Project's metadata (fourth column). Divisions are according to the dimensions, as in Table 3.



**Table 4.** *Cont*.


### **Table 4.** *Cont*.


**Table 4.** *Cont*.

Overall, the Eddleston Water Project contributes to 9 SDGs: 1, 6, 8, 9, 11, 12, 13, 15, and 17, and to 33 SDG targets, as can be seen graphically in Figure 5. Figure 5 complements Table 4 since it shows all the relevant SDGs and SDG targets (as established from Step II). The SDG targets in bold black color are the ones that the Eddleston Water Project contributes to, while in red, the ones that it does not. We showed that the Sustainability Performance Evaluation Framework follows a systematic methodology that allows to identify the interactions of the Eddleston Water Project within the five dimensions and define fluvial flooding indicators, which showed the Eddleston Water Project's contribution to the UN 2030 Agenda. Table 5 presents the Eddleston Water Project's SDGs under their respective dimensions. As expected, most of the Eddleston Water Project's SDGs contribute

to multiple dimensions at the same time due to the repetition of some of the SDG indicators under several SDG targets.

**Figure 5.** NBSs for FFRM could address 10 SDGs (legend) and 42 SDG targets (circle), as set by the UN 2030 Agenda. The framework application to the Eddleston Water Project revealed that the Eddleston contributes to 9 out of these 10 SDGs (not to the third) and to 33 SDG targets (not to the ones in bold red color in the outer circle).


**Table 5.** The Eddleston Water Project's SDGs under their respective dimension.

### **4. Discussion**

### *4.1. Regarding the Framework Itself: From Structure to Output*

The Sustainability Performance Evaluation Framework was developed from literature review, insights from three case studies, and then applied to one independent case study. On the one hand, this enabled us to assess the performance of the fluvial flooding indicators in depth, as we tried to find and access metadata for each fluvial flooding indicator. On the other hand, application to more case studies would have provided more insights that could extend or alter the NBSs–SDG coupling. For instance, the research showed that SDG seven, which assesses energy resources, could potentially be linked to the fluvial flooding indicators. More specifically, the Noordwaard polder project provided some clues about energy production from biomass, but the other case studies did not (https: //www.ecoshape.org/en/cases/wave-attenuating-willow-forest-noordwaard-nl/), and thus SDG seven was not included in the link between NBSs and SDGs. A broader case study examination will provide further insights regarding the potential of energy production in NBSs for FFRM interventions.

For data-dependant frameworks, such as the Sustainability Performance Evaluation Framework, data accessibility and/or method availability are key factors. Literature indicators such as 'population viability', although meaningful in the context of NBSs, are difficult to measure in practice. Similarly, within the Tier Classification for the Global SDG Indicators [60], Tier three includes the need for new or re-examination of existing measuring methods. Shah et al. [12] recognize the need for making the level of information even more local and specific alongside primary data collection for local NBSs or their indicators. From a more general point of view, Kumar et al. [10] state that the challenge of inadequate or insufficient data hinders the acceptance, assessment, and potentially successful operationalization of NBSs. Recent studies have addressed this challenge; Schröter et al. [9] provide extensive lists of online data pools on NBSs, and the EC's Handbook for Practitioners [52] devotes a chapter to types of data, data sources, and data generation techniques for NBSs monitoring and impact assessment. Therefore, current sources seem to allow for the effective use of such data-based frameworks while acknowledging the need for new or enhanced measuring methodologies.

The output of the Sustainability Performance Evaluation Framework is qualitative. We acknowledge, however, the potential to extend it to a quantitative one. Quantitative outputs, such as scored evaluation against pre-defined targets, enhance the evidence base of NBSs effectiveness [13]. To date, several studies provide quantitative results regarding effectiveness, co-benefits, and NBSs' sustainability contribution. Schipper et al. [21] provide a methodology for scoring the sustainability performance of coastal management projects using numeric data. Pugliese et al. [19] apply a multi-criteria tool to assess the effectiveness of NBSs for a specific case study compared to a grey alternative. Martín et al. [18] use qualitative analysis of Fuzzy Cognitive Maps (FCM) alongside semi-quantitative analysis of the co-benefits to examine the effectiveness of different NBSs and their co-benefits. Liquete et al. [17] perform an ex-post assessment of the environmental, social, and economic benefits of multi-purpose NBSs for water pollution control based on Multi-Criteria Analysis (MCA). The proposed framework is used to couple to the SDGs but can (with minor changes) also be used as an independent framework for the ex-post evaluation of individual NBSs projects, as a tool to compare grey–green alternatives for an NBSs project, or even as a tool to compare different NBSs projects. Andrikopoulou [61] describes the development of such a framework. We speculate that application of such a framework to a 'Room for the River' project, where pre-defined targets for the Rhine's conveyance capacity were set (https: //www.tandfonline.com/doi/abs/10.1080/02508060508691839), would have resulted in a better understanding regarding the potentials of the Sustainability Performance Evaluation Framework to derive a scored evaluation. A Multi-Criteria Analysis would both bring the framework output closer to reality and better assist decision makers in prioritizing river management options. In this research, flood safety was the primary river function under consideration, while ecosystem development, water quality, and recreation issues were also

examined. However, rivers typically have a larger number of functions (e.g., water supply, navigation, water quality, nature development), the importance of which may have greater or lesser weighting at any one time and location, and to different stakeholders, depending on the focus of the project. Therefore, prioritization of the fluvial indicators based on the primary river functions of a specific case would greatly benefit the framework output.

### *4.2. How Do the Outcomes of Our Framework Relate to Other Relevant Studies?*

We have set up a framework to create a set of fluvial flooding indicators to evaluate the ex-post contribution of the Eddleston Water Project to the UN 2030 Agenda. Ligtvoet [2] and Ge et al. [62] have also explored the relationship between rivers and the SDGs. Ge et al. [62] have defined SDGs for river basin scale in terms of water, ecosystems, and socioeconomic capabilities. Ligtvoet [2] has identified those SDGs related to people and the economy that are negatively affected by river flooding.

Framework application to the Eddleston Water Project shows that it contributes to six out of the nine SDGs mentioned in Ligtvoet [2]. In the study of Ligtvoet [2], Ward and Winsemius have identified the negative effects of fluvial flood risk on SDGs 1, 2, 3, 6, 8, 9, 10, 13, and 16 for people and the economy (specifically on agriculture). Although Ward and Winsemius established a broader link (river flood risk and SDGs) than ours (NBSs for FFRM and SDGs), the outcomes of our framework application show that the Eddleston Water Project can ameliorate most of the negatively affected SDGs by river flooding. Exceptions constitute SDGs 2, 10, and 16. SDG two addresses food security issues and agricultural practices, which were considered out of scope for the functions of fluvial flood risk mitigation NBSs considered in this research. SDG 10 refers to the reduction of inequalities between countries in terms of providing the same means of coping with flood risk between high-income, upper-middle, lower-middle and low-income economies. NBSs could address such an aspect; however, it needs examination in a broader context combined with geopolitical considerations. Similarly, SDG 16 talks about justice and inclusivity in societies, which can be promoted as part of the general NBSs conceptual framing but are out of scope for this study. Leaving these aside then, it is apparent that the Eddleston Project positively affects the following 6 SDGs: 1, 3, 6, 8, 9, 13.

The SDGs to which the Eddleston Water Project contributes align with seven out of eight SDGs proposed by Ge et al. [62]. Ge et al. [62] link river basins (e.g., Amazon, Nile, and Heihe river basins) to SDGs in terms of water, ecosystem, socioeconomic, and ability-related issues, and they find all SDGs relevant apart from SDG seven. The water-related SDGs (6, 11, 12, 14) and the ecosystem-related SDGs (14, 15) coincide with the SDGs derived from the present Eddleston Water study, apart from SDG 14 (which was out of scope for the current research context because it focuses on the coastal environment). The socioeconomic-related SDGs were omitted because they focus mostly on food security, justice, and inequalities which, whilst they might be added in other situations, were not within the scope of the impacts of the fluvial NBSs examined for the present research. However, although they were not specifically considered, it could be argued that by including and elaborating on the third aim in the Eddleston Water Project's objectives (working with landowners and communities to maintain and enhance sustainable land management practices and farm businesses), the project contributes to an element of this SDG. The ability-related SDGs (9, 11, 13, 17) coincide with the ones derived from our research because they refer to structural actions and strategies for conserving and protecting the rivers. Hence, SDGs 6, 9, 11, 12, 13, 15, 17 are shared between Ge et al. [62] and the Eddleston Water Project with respect to water, ecosystem, and ability-related issues.

### *4.3. Standardization and Scale of Sustainability Assessments*

As mentioned by Pohle et al. [63], standards, in general, have a twofold role: they can serve as a source of information and enabler for the development and transfer of technology. The EU Research and Innovation program Horizon 2020 has recognized standardization as a measure that underpins innovation since it bridges the gap between research and

the market but also facilitates the propagation of research outcomes to the European and international markets [64]. That said, standardization of sustainability assessments could provide harmonization in indicators, reliability and transparency in calculation methods, and comparability of results [65]. To date, standardization for sustainability assessments has already been discussed in the literature [66–68]. Although most of the studies recognize the aforementioned benefits of standardization, they also recognize the risk of compromising agendas, contexts, and needs when treating larger scales. Similarly, in the present study, although the aim was to derive indicators as widely applicable as possible, given the scope of the study, we acknowledge that most of the indicators would need further consideration when used in projects of different scales and contexts. Some of the indicators, e.g., the environmental and technical indicators, could relatively easily be standardized – some with slight amendments—for larger scales. For others, such as the Social, Policy—Procedural, and Economic indicators, this is more difficult because they are geopolitically dependent. For instance, in a transboundary water body apart from the international legislation, the in-between the country-member treaties and arrangements should also be considered. Such a view was out of scope in our research and thus requires further research.

Sustainability assessments should be able to be carried out at any scale. However, the necessary data are not always available or accessible at any scale. For instance, it is likely that the Sustainability Performance Evaluation Framework cannot be applied globally to any NBSs project due to the lack of data. Most of the data collected for such projects either suit national aims, which do not always align with the global SDG indicators, or come from private sources. Therefore, it seems that currently, the main challenge lies in finding adequate, available, and accessible data to upscale the sustainability assessments, and although there is still a lot to accomplish in this direction, the EC's Handbook for Practitioners [52] and Schröter et al. [9] have made promising steps (as discussed in paragraph 4.1).

### **5. Conclusions**

The aim of this study is to propose a ready-to-use methodology to evaluate the sustainability performance of Nature-Based Solutions (NBSs) with respect to the United Nations (UN) 2030 Agenda, involving all its components (i.e., Sustainable Development Goals (SDGs), targets and indicators). This was achieved by building on the Schipper et al. [21] systematic framework and adjusting it to fit our needs. The focus is on NBSs for Fluvial Flood Risk Mitigation (FFRM) in river basins of sizes up to 100 km2. The derived framework is called the Sustainability Performance Evaluation Framework. It encompasses four steps through which the end-user creates a set of fluvial flooding indicators that can then be linked to the SDG indicators, and by applying the fluvial flooding indicators to a specific FFRM NBSs project, it is possible to ascertain the project's contribution to the UN 2030 Agenda. The Eddleston Water Project was used as a case study to test the effectiveness of the fluvial flooding indicators. Application to the Eddleston Water Project has shown that it contributes to 9 SDGs and 33 SDG targets. In developing the Sustainability Performance Evaluation Framework and testing its fluvial flooding indicators, the findings are:


To further develop the Sustainability Performance Evaluation Framework, a key recommendation is its trial application in other areas and by different end-users. This should focus on three aspects:


We recognize that the more the framework is reviewed and applied, the more insights will be gained with respect to its biases, limitations, and gaps, including opportunities that could extend or alter the NBSs–SDG coupling.

**Author Contributions:** Conceptualization, T.A. and R.M.J.S.; methodology, T.A., R.M.J.S., C.A.S., A.B. and C.J.S.; formal analysis T.A. and R.M.J.S.; investigation T.A., R.M.J.S. and C.J.S.; data curation T.A., R.M.J.S. and C.J.S.; writing—original draft preparation, T.A.; writing—review and editing T.A., R.M.J.S., C.A.S., A.B. and C.J.S.; supervision R.M.J.S., C.A.S., A.B. and C.J.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** For the framework application, data were mostly collected from publicly available sources, with a few of them from expert consultation. Details regarding data availability and accessibility for the Eddleston Water Project are available in Appendix A.

**Acknowledgments:** The research behind this paper was a collaboration of TU Delft and Rijkswaterstaat in The Netherlands. It was funded by Rijkswaterstaat and greatly benefited from discussions with TU Delft and RWS colleagues, examining nature-based projects and their potentials with respect to the UN 2030 Agenda. The EU Interreg Building with Nature program's projects were an inspiration and key evidence-contributor to the paper. We would also like to thank the Tweed Forum that provided all the information needed for the Eddleston Water Project, as well as the external reviewers for their time devoted to this manuscript since we feel their comments helped to improve the quality of the manuscript.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **Appendix A. Data Availability and Accessibility for the Eddleston Water Project**

One of the main reasons for choosing the Eddleston Water Project was the data availability and accessibility. Indeed, most of the fluvial flooding indicators (85%) were filled in with data publicly available online, while only 15% of the indicators needed a project specialist —either to verify data found online or to provide additional information. Expert consultation was needed for the following five fluvial flooding indicators: CO2 emissions, recreation/leisure value, enhance attractiveness, employment, and adaptability. Both qualitative and quantitative data were gathered since not all indicators required a numerical value – for example, those covering the Policy—Procedural dimension. On the contrary, fluvial flooding indicators such as 'well-being' or 'extent of water-related ecosystems' could be filled in with quantitative data.

During the framework application, a few overlaps of the project metadata per fluvial flooding indicator were observed. For example, in looking at the 'recreation/leisure value' and 'enhance attractiveness', similar data were used for both fluvial flooding indicators. Although the attractiveness of the area has been enhanced and it is being used by the public, new plans for a cycleway will further increase its recreational value. To this end, there is a limited extent of data for these two indicators, leading to their current overlap. However, for a case study where all the interventions had been finalized, these two indicators might provide different information.

### **References**


### *Article* **Tracking Topics and Frames Regarding Sustainability Transformations during the Onset of the COVID-19 Crisis**

**Mariana Madruga de Brito 1,\*, Danny Otto <sup>1</sup> and Christian Kuhlicke 1,2**


**Abstract:** Many researchers and politicians believe that the COVID-19 crisis may have opened a "window of opportunity" to spur sustainability transformations. Still, evidence for such a dynamic is currently lacking. Here, we propose the linkage of "big data" and "thick data" methods for monitoring debates on transformation processes by following the COVID-19 discourse on ecological sustainability in Germany. We analysed variations in the topics discussed by applying text mining techniques to a corpus with 84,500 newspaper articles published during the first COVID-19 wave. This allowed us to attain a unique and previously inaccessible "bird's eye view" of how these topics evolved. To deepen our understanding of prominent frames, a qualitative content analysis was undertaken. Furthermore, we investigated public awareness by analysing online search behaviour. The findings show an underrepresentation of sustainability topics in the German news during the early stages of the crisis. Similarly, public awareness regarding climate change was found to be reduced. Nevertheless, by examining the newspaper data in detail, we found that the pandemic is often seen as a chance for sustainability transformations—but not without a set of challenges. Our mixed-methods approach enabled us to bridge knowledge gaps between qualitative and quantitative research by "thickening" and providing context to data-driven analyses. By monitoring whether or not the current crisis is seen as a chance for sustainability transformations, we provide insights for environmental policy in times of crisis.

**Keywords:** frames; SDG; green deal; content analysis; natural language processing; NLP

### **1. Introduction**

Historically, crises have motivated fundamental social, ecological, economic, and political transformations [1]. The evolving COVID-19 pandemic is no exception. In a matter of weeks, it cast a stark light on how changes can take place overnight. This includes, among others, short-term negative impacts such as an increased waste generation due to the lockdowns [2,3] but also positive environmental changes such as an improved air quality [4,5]. What remains to be understood is if fast-moving crises such as the COVID-19 pandemic can usher in lasting and far-reaching transformations [6].

Particularly at the beginning of the pandemic, an increasingly vocal debate on potential long-term changes with a rather optimistic view was initiated, with proponents arguing that the COVID-19 crisis may have opened a window of opportunity to spur on systemic ecological sustainability transformations [7]. This narrative was highlighted, for instance, by António Guterres: "We must turn the recovery from the pandemic into a real opportunity to build a better future" [8]. Similarly, multiple scholars have stated that they see an opportunity for accelerated sustainability transitions in the post-crisis period—mainly in the form of "restarting" society or by "rebuilding it better" [6,9,10]. The adherence to this narrative is echoed by both scientists and politicians. Yet, empirical evidence for such a dynamic currently lacking.

**Citation:** de Brito, M.M.; Otto, D.; Kuhlicke, C. Tracking Topics and Frames Regarding Sustainability Transformations during the Onset of the COVID-19 Crisis. *Sustainability* **2021**, *13*, 11095. https://doi.org/ 10.3390/su131911095

Academic Editors: Baojie He, Ayyoob Sharifi, Chi Feng and Jun Yang

Received: 18 September 2021 Accepted: 5 October 2021 Published: 7 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Previous experience suggests that whether or not crises result in an opportunity for initiating or accelerating sustainability transformations depends on factors such as the crisis severity, the public and media framing [11], socio-economic capacities, and political interests [7,12,13]. Hence, to understand whether or not the COVID-19 crisis can prompt sustainability transformations, it is necessary to monitor political, public, and media frames and their translation into policies. In our view, this calls for a systematic monitoring of discursive material and an in-depth study of how the relationship between the pandemic and ecological sustainability is established. This is crucial because the way environmental concerns are framed sets the stage for how they are addressed [14].

Since the beginning of the pandemic, a plethora of discursive material on COVID-19 has been produced. To varying degrees, this material has influenced the perceptions of and the reactions to the crisis. Indeed, it is widely acknowledged that traditional news media play an important role in shaping public awareness [15–17]. While social media and other forms of media have gained relevance, the analysis of newspapers is still a valuable tool for monitoring public knowledge [18]. Media coverage has been shown to influence public concern regarding climate change [19]; it can impact policy processes [20] and people's behavioural intentions [21,22]. The influence of the media in shaping opinion is even more evident in times of crisis [23]. This has been studied in relation to the Ebola outbreak in 2014 [24], the Brexit referendum [25], and the European drought of 2018 [26,27]. In this context, media news can serve as a proxy for understanding how public discourses on ongoing transformations evolve. Similarly, online search behaviour can provide insights into public awareness levels [28].

Given the multitude of discourse material available, new and variably applicable analytical approaches are needed to analyse these data, aiming to produce timely results that allow researchers to make cross-context generalisations. To this end, "big data" tools such as natural language processing (NLP) techniques [29,30] can be used to offer a "bird's eye view" of how discourses change. At the same time, context-specific and interpretative qualitative methods (i.e., "thick data" tools) are needed to elucidate the underlying patterns observed in data-driven analyses, as they allow for an in-depth analysis of prominent frames [31] (for a definition of discourse and frames see [32]).

Here, we propose the integration of NLP and qualitative content analysis on newspaper data to understand how sustainability aspects are discussed within news related to COVID-19. More specifically, we analyse how the reporting about topics underpinning sustainability has evolved in the media debate about COVID-19. The proposed approach is demonstrated by following the COVID-19 discourse on sustainability during the onset of the crisis in Germany. In a first step, we used topic modelling and quantitative content analysis to investigate the extent of the discussion on sustainability topics from March until June 2020. We also analysed how public awareness developed compared to the previous year by considering online search behaviour. In a second step, we conducted a qualitative content analysis on a smaller sample of articles, focusing on news published during the first COVID-19 wave. This allowed us to delve deeper into the discursive material and track specific frames. Emphasis was given to sustainability topics in line with the European Green Deal framework and the UN Sustainable Development Goals (SDGs) related to ecological sustainability.

### **2. Big Data and Thick Data: Methodological Considerations**

Big data has been discussed as a central issue for empirical research. Some see it as a challenge for empirical social sciences [33,34]. Others weigh up the related threats and opportunities [35–38] or question whether the use of big data is truly innovative, arguing that traditional types of big data—such as administrative records and newspaper articles have been studied for centuries [39]. Additionally, the definition of what constitutes big data has been contested, and it remains a "rather loose ontological framing" [40].

Overall, characteristics commonly associated with big data include large volumes of data that: are accessible for computational analyses, are rapidly created and shifting, have a complex and untidy structure, and are exhaustively captured instead of sampled [35,39–41]. Due to challenges arising from pure data-driven quantitative analyses, integrating qualitative components into the research design is often advised [38,39,41,42]. In this context, the goal of mixed-methods designs can be to: (a) provide additional coverage by adding more perspectives and data on a particular issue, (b) verify results by "convergent findings" of quantitative and qualitative methods or (c) achieve "sequential contributions", meaning that what is learned in one method serves as input for the other [43]. For big data, all three aspects are applicable.

Following this rationale, we argue that the monitoring of sustainability transformations needs both big and thick data. Thick data is understood here as qualitative and detailed data which support the understanding of the context in which a pattern occurs [44]. While big data tools can help to track and describe changes in discourses over time, it is difficult to understand these transformations and the reasons behind them based solely on data-driven approaches [31]. In this regard, thick data methods can be used to add a deep-diving and context-sensitive dimension to the data-driven outcomes, enabling more complex interpretations.

Here, we propose to "thicken" the data by reducing the number of data points while enhancing the thickness of their descriptions. Hence, instead of solely providing a trend analysis of the topics mentioned in the media, we analyse how they are framed. We use the metaphor of "thick data" in reference to Geertz's "thick description" [45], which describes the need for ethnographic descriptions to go beyond recording what people are doing, as this only provides a superficial account of actual situations. Taking golfing as an example, a "thin" description portrays a person "repeatedly hitting a little round white object with a club-like device" [46]. A "thick" description sensitively interprets the context of the behaviour. It adds the frames of a golf course, gaming rules, and equipment handling in an attempt to grasp the situation fully. Thick descriptions not only enable an understanding of cultural contexts beyond "thin" statements of observable behaviour, but they also allow a further understanding of patterns emerging in big data. Conversely, big data can support sampling strategies or highlight hotspots for thick data tools. Big data can also take on the role of thick data if, for instance, qualitative text analyses are complemented by the study of large databases.

### **3. Material and Methods**

### *3.1. Newspapers Sample Selection*

Newspaper data were collected from a news aggregator database (wiso-net.de). The articles in our sample were published between 1 March to 30 June 2020. By using the search string "Corona\*" or "Covid" or "SARS-CoV-2", 459,129 articles were retrieved. Given that the aforementioned database does not allow web scraping, the articles had to be downloaded manually. Hence, to reduce the sample size to a manageable number while at the same time ensuring topic coverage and geographical equity [26], only the newspapers with the highest circulation rates in each German federal state were considered. Hence, 2 nationwide and 19 regional newspapers were used (Table S1). This reduced the sample to 84,587 articles. The Jaccard similarity coefficient [47] was used to identify duplicate articles. A threshold of 0.7 was applied, where 0 indicates no similarity and 1 indicates full similarity. The final data set consisted of 61,514 unique records (see Figure S1 for sampling flowchart).

### *3.2. Automated Text Analysis*

Before the analysis was conducted, common NLP methods were applied to clean the data. First, we removed numbers and punctuation and converted the characters to lowercase. Then, the articles' sentences were tokenised into individual words. Stop words such as articles, pronouns and prepositions were removed from the corpus (see Table S2 for additional stop words). Finally, metadata relating to the date of publication and newspaper were extracted using regular expressions. All coding was carried out in R.

### 3.2.1. Topic Modelling

In the first step, topic modelling was conducted to provide an overview of the content of the articles and investigate if sustainability issues emerged as a relevant topic. Topic modelling is an unsupervised machine-learning algorithm, where patterns of word cooccurrences are used to identify topics that describe the corpus [48]. Each topic contains a cluster of words that frequently occur together and refer to similar subjects [49].

The Latent Dirichlet allocation (LDA) was chosen to identify predominant topics in our dataset. The LDA model assumes that articles can have multiple topics. For instance, an article might be 40% about topic 1 and 60% about topic 2. The number of topics in our LDA model was defined as 70 after computing the coherence when varying the number of topics from 10 to 100. Additional stop words used are shown in Table S2. As an outcome of the LDA, a list of the most common words and the topic probabilities for each article were obtained. The topic titles were defined inductively based on the 20 most unique words from each topic (i.e., words that tend to occur mainly in a specific topic).

### 3.2.2. Quantitative Content Analysis

Pattern matching was used to classify the articles into different sustainability topics. The articles were classified according to eight environmental topics drawn from the European Green Deal framework and the UN SDGs.

The keywords (Table S3) used to classify the articles within these topics were defined based on two sustainability glossaries [50,51]. Additional terms were identified by analysing unigram frequencies and word co-occurrences (i.e., words that occur in the same sentence). Furthermore, ten experts in sustainability transformations were consulted to select unambiguous terms. Based on that, we tagged sentences where a keyword or combination of keywords occurred as related to a given topic.

Of the 61,514 unique articles, only 2343 mentioned at least one of the sustainability keywords. A random sample of 40% of the articles was read to validate the automatic classification and identify false positives and false negatives. Missclassifications found were manually corrected. In a final step, we plotted the results obtained for each of the eight topics against the German national SDG performance [52].

### 3.2.3. Bibliometric Analysis

To add a temporal perspective to the quantitative content analysis, we conducted a bibliometric analysis [53] on the whole wiso-net database, only considering German newspapers (>39 million articles, 240 newspapers). The same sustainability keywords (Table S3) were used, but with a different time frame: from August 2017 to July 2020. For these searches, the keywords related to COVID-19 were omitted. This allowed us to determine the frequencies of articles referring to sustainability over time and see if their number increased or declined during the first wave of the pandemic. The outputs were normalised according to the total number of articles per month. This reduced the bias due to the variation in the monthly number of news.

### 3.2.4. Co-Occurrence of Sustainability Topics

To analyse the topics that were reported simultaneously by the same article, a cooccurrence analysis was conducted in the sustainability dataset (n = 2343 articles). Spearman correlation coefficients were computed to assess whether they co-occur by chance or follow a pattern. Circos plots [54] were used to visualise the interdependencies between the sustainability topics.

### 3.2.5. Word Frequencies

NLP procedures, such as term-document matrices, word frequency analyses and bigram analyses, were employed. Based on that, a network analysis graph was developed to visualise how the corpus' words are associated. In addition, word clouds were created to demonstrate how the terms' usage has changed over time and according to different sustainability topics. For the compilation of the comparison word clouds, all mentions of a word in the corpus were counted. Then, the proportional use of the word was calculated based on the total words used in all articles across the specified topic or month.

### 3.2.6. Public Awareness on Sustainability

In a final step, Google Trends (GT) data (trends.google.com) were used to measure public awareness on environmental topics. GT measures the search popularity in relative terms according to a random sample of all search terms used in queries within the investigated period. A detailed description of the GT measure can be found in Rousseau and Deschacht (2020). The GT database can be searched according to user-defined "search terms" and "topics". The user-defined keywords used here are shown in Table S3. Furthermore, the pre-defined "Environment" topic was considered.

To identify the effect of the COVID-19 crisis on the population's awareness regarding sustainability topics, we compared the GT search popularity indicator from March to June 2020 with data from the same period in 2019. The nonparametric Wilcoxon test was used to identify significant differences.

### *3.3. Qualitative Content Analysis*

A qualitative content analysis was conducted to provide context and an in-depth analysis of the frames which connect COVID-19 and sustainability. Based on the previous steps, 461 articles were selected as a thick data sample. These articles discussed two or more of the sustainability topics considered, and hence, were more likely to add to a "window of opportunity" narrative for sustainability transformations.

Following Schreier [55] and Mayring [56], we applied a multi-step approach using a combination of inductive and deductive reasoning. First, we developed three broad categories: (a) COVID-19 impacts with implications for the environmental SDGs and Green Deal goals; (b) COVID-19 recovery measures and their relation to the environmental SDGs and Green Deal goals; and (c) COVID-19 as a window of opportunity for sustainability transformations. Two coders worked through all the articles to identify segments related to the above-mentioned categories. Disagreements were discussed in this process and code descriptions were refined.

After the first coding round and based on the discussion between the coders, subcategories were added in vivo to create a final version of the coding scheme. In the end, eight prominent frames were identified. Once all the articles were analysed, we calculated the number of articles in each category. We also compared the frames with the automated text analysis outputs by developing a heatmap and computed the co-occurrence of frames and topics using the same method described in Section 3.2.4.

### **4. Results**

*4.1. Quantitative Analysis: Media Debate on Topics Regarding Sustainability during the Early Stages of the COVID-19 Crisis*

4.1.1. Media Attention and Public Awareness on Sustainability

Overall, there was a sharp decline in the number of articles that mention sustainabilityrelated keywords after the start of the COVID-19 pandemic in Germany (Figure 1). This could indicate that ecological concerns were brushed aside during the initial shock of COVID-19. Before the onset of the pandemic, clear peaks occurred during protests such as the Global Climate Strikes (GCS) (Figure 1).

With regard to the public awareness on sustainability, an adverse effect of the COVID-19 crisis on the online search behaviour for the topics climate change and environment was found (Figure 2). Conversely, there was an increase in searches regarding air quality, with more searches in 2020 compared to the same period in 2019.

**Figure 1.** Bibliometric analysis showing the frequency of articles that mention sustainability-related keywords (Table S3) in the German media according to the wiso-net database. The data were normalised against the total N of published articles per month.

**Figure 2.** Search behaviour in 2019 and 2020 according to Google Trends data. Higher values indicate a higher amount of searches. For both years, data from March to June were considered. Wider sections represent a higher density of people searching for a term; the thinner sections represent a lower density of searches. *p*-values were obtained by using the nonparametric Wilcoxon test.

4.1.2. Media Statements about Sustainability (MSS) during the Onset of the COVID-Crisis

To analyse the relationship between sustainability and the COVID-19 crisis, only articles that mention the pandemic were considered (for sampling procedure, see Section 3.1). Topic modelling outcomes suggest that environmental problems are underrepresented in contrast to other topics (Table S4). The topics that dominate the corpus were related

mainly to family and everyday life issues (3.5%), financial insecurity (3.4%), and quarantine restrictions (3.1%). Of the 70 topics, only one is directly related to environmental issues (1.5% of the corpus, n = 932). It includes news about outdoor recreation, gardening, and smart mobility (see topic 62 in Table S4). Hence, based solely on this unsupervised machine-learning algorithm, it was not possible to detect sustainability as a core issue in the COVID-19 discourse.

To further investigate the COVID-19 discourse on sustainability, all COVID-19-related articles were classified into eight sustainability topics following the scheme presented in Table S3. Based on that, 2935 media statements about sustainability (MSS) were identified. Most of the MSS refer to climate change aspects (39%, n = 1142), followed by food supply (22%, n = 637) (Figure 3). These numbers are low compared to the number of articles included in the analysis (n = 61,514). Indeed, for each week and topic, an average of only 0.6% of the COVID-19 articles reported about sustainability issues. This suggests that the early COVID-19 discourse in Germany was not centred on sustainability issues, which was expected, because issues that directly impact people tend to become a priority in an immediate crisis.

**Figure 3.** Temporal distribution of the MSS topics. The keywords used for the searches are provided in Table S3.

When considering the MSS frequency in relation to the total number of news articles, we see an increase in the proportion of climate change MSS (Figure 4). In March 2020, about 0.9% of all articles (n = 61,514) reported about climate change issues. This number increased to 2.7% in April 2020. This is also reflected in the usage of terms, which shifted drastically over time (Figure S2a). Indeed, the word stem "*Klima*\*" (climate in english) appeared at a frequency of 0.0038 (words appeared 815 times across 210,311 words for this period) in April, whereas in March, the frequency was only 0.0016 (n = 411 out of 248,046 words). With regard to food supply MSS, there was higher interest in this topic in March 2020, with a rapid decline in the following months. In May, the discussion about smart mobility intensified, as reflected by the usage of the words "car", "Tesla", "railway", and "bicycles". Finally, in June, concerns regarding the "future" and the economy (e.g., "euro", "economic stimulus package") became more evident. In summary, there was a shift in the discourse, from a primary focus on food supply in March, to increasing references to a "chance" for climate change protection in April, followed by an intensified discussion on mobility in May and a focus on the restoring the economy in June.

**Figure 4.** Temporal distribution of the sustainability topics with a 95% confidence interval. For each topic, the frequency of articles that contain the keywords in Table S3 in the text is shown for each week.

### 4.1.3. Co-Occurring Sustainability Topics and Keywords

To better understand the patterns behind the articles that mention multiple sustainability topics, a co-occurrence analysis was conducted. An average clustering coefficient of 0.91 was obtained, which indicates that most of the topics are connected at least in some of the articles (Figure 5). Still, the majority of the MSS were reported alone (64.1%, n = 1880). Climate change MSS were frequent drivers of network connections (present in 71.8% of all co-occurrences, n = 481). Indeed, energy, biodiversity, and air quality MSS were mentioned mainly together with climate change MSS. This is reflected in the strong correlations found between climate change and other topics (Table S5). Conversely, food supply, which was the second-most dominant MSS in our database, was primarily reported in isolation (n = 594), with few co-occurrences with climate change. This could indicate that these MSS are not directly related to sustainability but refer to individual food supply needs. This was verified when validating our classification system, where it became clear that the discourse on food supply is centred on panic buying and not on long-term food security.

To identify weakly integrated themes, a biography network was elaborated (Figure S3). It illustrates how the words in the corpus are associated. Terms related to the COVID-19 impacts cluster together (e.g., "consequences", "people", "million"). On the other hand, sustainability-related terms are side-lined (e.g., "climate change", "environment"). Several word pairs are disconnected (e.g., "Green Deal", "Fridays for Future", "renewable energy"). This indicates that there is little crossover between sustainability and the COVID-19 consequences as portrayed by the German media. It also shows that sustainability is not the central theme depicted in the corpus, even in the sample of articles that discuss at least two sustainability topics.

### 4.1.4. Results Validation

We evaluated the accuracy of the automatic classification system by conducting close reading and annotating 40% of the articles, as mentioned in Section 3.2.2. The automatic classification of the MSS was accurate in 96.1% of the cases, with a standard deviation of 2.2% (Table S6). The misclassifications correspond mostly to articles that mention sustainability-related keywords but outside of the COVID-19 context. Overall, MSS topics such as circular economy and food supply presented the highest levels of accuracy (98.9%

and 98.7%, respectively). Conversely, MSS about climate change were overestimated (accuracy of 92.3%).

**Figure 5.** Co-occurring MSS topics. The ribbons connecting two MSS indicate copresence links, and their size is proportional to the number of links. Ribbons without connections indicate MSS that were reported alone. For clarity purposes, only co-occurrences >5 are shown. For an interactive visualisation see Supplementary Material S1.

As an additional validation measure, we built a comparison word cloud, which highlights the most unique terms in each topic (Figure S2b). The results indicate that the MSS are consistent with the sustainability topics they were assigned to. For instance, the word "water" was mentioned 168 times in the "water quality" MSS articles and only 13 times in the "biodiversity" MSS articles.

### *4.2. Qualitative Analysis: Frames Connecting COVID-19 and Sustainability*

The big data bird's eye view (Figure 3) shows a decrease in coverage on sustainability since the beginning of the pandemic and only reveals a few articles that link COVID-19 and sustainability. In an in-depth interpretative content analysis, we differentiated these linkages further by analysing three bundles of frames, as shown in Table 1.

Of the 461 articles included in the qualitative analysis, 79.6% (n = 367) had substantive information related to the frames mentioned in Table 1. The following items discuss each of these frames and the main actors supporting them. Notably, these frames are interlinked and overlap, both in terms of occurrence over time (Figure S4) and content (Figure 6). Indeed, they do not occur as separate lines of discourse but instead supplement or counteract one another, suggesting that there is no dominant frame.

4.2.1. The COVID-19 Crisis Led to Observable Short-Term Positive Environmental Impacts (PEIs)

About a quarter of the articles (23.4%, n = 108) highlight that the pandemic has given nature time to recover. These were connected to early scientific reports on the environmental effects of the first lockdowns. It is stated that there was an improvement in air and water quality, as well as biodiversity, due to reduced economic activities and limited human impacts on the environment. The clean canals of Venice, less nitrogen dioxide emissions in China and India, and increased sightings of wildlife are frequent points of reference for this frame. However, this frame rarely ends with this simple description. Most of the articles reference researchers and emphasise that these effects are temporary and are not enough to impact overall climate developments [7]. They might be enough to enable Germany to accomplish its 2020 greenhouse gas emission reduction goals, as some commentators note, but they do not present a solution for the climate crisis. This line of argument is accompanied by warnings of potential rebound effects. Parallels are drawn between post-pandemic scenarios and the fast return to pre-crisis emission levels that accompanied the economic recovery after the Global Financial Crisis (2008/09). This frame is closely linked to air and water quality MSS (Figure 7).


**Table 1.** Overview of sustainability frames and percent of articles (n = 367) that mention them.

**Figure 6.** Relationships between the frames. For the acronyms, see Table 1. The ribbons connecting two frames indicate copresence of links and their size is proportional to the number of links. Ribbons without connections indicate frames that were reported alone. For an interactive visualisation, see Supplementary Material S2.

**Figure 7.** Relationships between the MSS and the frames identified in the thick data analysis. The numbers represent the percentage of articles in the subsample (n = 367) that are tagged according to an MSS and present a specific frame. For instance, 64.6% of the air quality articles in the subsample were also classified as having the positive environmental impacts (PEIs) frame.

4.2.2. The COVID-19 Crisis Spurred Behavioural Changes That Have an Impact on Sustainable Development (BC)

A total of 112 articles (24.3%) emphasised that the COVID-19 crisis has led to multiple behavioural changes that affect the environment either positively or negatively. These include, for instance, increased use of bicycles, mobility reduction, and a decline in oil demand. The pandemic has also affected consumption patterns (e.g., panic buying) and waste generation, as indicated by the high linkage between the BC frame and circular economy and food supply MSS (Figure 7). Similar to the positive environmental impacts (PEIs) frame, it is stated that these changes may be reversed after the crisis is over. Still, it is argued that the digitalisation of work will remain a feature of the post-COVID world, thus reducing mobility needs and overall fossil fuel consumption. Nevertheless, countervailing negative behavioural changes are also expected, such as a preference for using private cars and the avoidance of public transportation. Furthermore, experts highlight that recreational air travel is not likely to decrease after the pandemic.

4.2.3. The COVID-19 Recovery Programme and Measures Address or Should Address Environmental Sustainability Goals (MAS)

This is the most common frame in our sample (27.3%, n = 126). It is reasoned that the recovery plan should boost the economy while at the same time focusing on longterm strategies for advancing the climate agenda. Experts and politicians argue that the UN SDGs may act as a normative guide for selecting the targets for future investments. Overall, the ecological modernisation of transport and the car industry is a central point of this frame, as evidenced by the linkage between this frame and MSS on smart mobility (Figure 7). Of the measures discussed, the most prominent (and controversial) is the environmental bonus programme for the purchase or lease of electric or plug-in hybrid vehicles. We also found broader arguments that called for recovery programmes to focus on sustainability or include "climate-friendly" technologies and energy. From a long-term perspective, it is argued that the breakdowns of normality caused by the pandemic should be used to assess the possibilities for a more sustainable recovery (i.e., innovative, digital,

low CO2 emissions). It is noteworthy that the water quality MSS are not linked to any of the measures frames (MAS, MNDE, and MFEF) (Figure 6). This could indicate that water issues may be overlooked in the news media reporting.

4.2.4. The COVID-19 Recovery Measures Are Not Doing Enough to Achieve Sustainability Goals (MNDE)

This frame (7.4%, n = 34) criticises the recovery measures for not including regulations or practices that would accelerate sustainability efforts or even for hampering those efforts. Politicians and environmental NGOs question the use of environmental bonus programmes for transforming the mobility sector. Furthermore, they are critical of government rescue programmes for airline companies. They argue that investments in environmental protection projects are too low to achieve the climate goals of the Paris Agreement. This frame is usually mentioned together with the MAS frame (COVID-19 recovery programme and measures address or should address environmental sustainability goals) by pointing out existing challenges (Figure 6).

4.2.5. The COVID-19 Recovery Measures Focus on the Economy First—Rollback for Sustainability (MFEF)

The MFEF frame emphasises a quick economic recovery that should not be hampered by ecological concerns. This frame is consistent with policy discourses that propose halting or postponing environmental standards and goals [57,58]. Only a few newspapers (4.6%, n = 21) report on this position. Nevertheless, this number should be interpreted cautiously, as only articles that reported on at least two sustainability topics were included in the qualitative content analysis (Figure S1). This frame is supported mainly by politicians from federal states with extensive vehicle production facilities or industry sector representatives. It focuses on the situation of large industries (especially the car industry) during the pandemic and suggests rollbacks of various climate regulations (such as CO2 taxes or emission levels). The articles also mention the pressure to loosen up regulations regarding the circular economy and to postpone the adoption of environmental measures for the energy and car industries (Figure 7).

4.2.6. The COVID-19 Crisis Is an Opportunity for Change towards More Sustainable Development (OCS)

These articles (25.2%, n = 116) highlight that the government should consistently pursue socio-ecological transformations to deal with the aftermath of the pandemic. A central challenge mentioned is the transformation of transport, energy, and agriculture in an economically sensible, socially acceptable, and sustainable way. At the heart of the debate is the Green New Deal. Both politicians and experts emphasise that the COVID-19 crisis and the climate crisis must be tackled together. Similarly, economic institutes argue that the ongoing crisis is a good time to make longer-term investments in climate protection. Another point often mentioned is that the COVID-19 crisis has shown how society can be mobilised and how quickly changes can be made. Indeed, structures, institutions, and behaviours that were considered unchangeable were put to the test. Hence, the COVID-19 crisis is seen as an opportunity to break up ossified structures to trigger a wave of green recovery in Germany. Articles in this class also point out that the COVID-19 crisis is an opportunity for the renewable energy industry because it is accelerating the coal phase-out and, as some put it, "ending the oil age".

### 4.2.7. The COVID-19 Crisis Poses Challenges for Sustainability Transformations (CST)

Various factors that might challenge sustainability transformations are connected in this frame (21.9%, n = 101). From an economic perspective, the financial feasibility of climate objectives is questioned due to the costs of recovery programmes and the strain that COVID-19 regulations are putting on industry and the service sector. Articles that are sceptical about the successful implementation of climate protection measures or regulations in light of the pandemic tend to reference industry actors and (conservative) politicians in particular. In addition to these economic challenges, the articles also describe a shift of attention away from climate issues. As society is focused on the immediate COVID-19 response, there are limited capacities to address the climate crisis with its uncertain timeframes. Furthermore, this loss of attention is linked to secondary effects on climate activism and (international) climate diplomacy. Focal points of this frame are the postponement of major climate conferences and changes in how climate activists stage protests (e.g., virtual formats for Fridays for Future). Both issues are seen as challenging for the sustainability efforts, as the lack of demonstrations reduces the attention given to environmental protection and the postponement of significant international meetings limits climate policy. Similar delays and shifts are reported for environmental research, technology development, and legislation processes. Interestingly, all MSS were well-represented in this frame, indicating that the challenges encompass a broad range of sustainability aspects (Figure 7).

### 4.2.8. The COVID-19 Crisis as an Analogy for the Climate Crisis (ACC)

Articles mentioning this frame (6.3%, n = 29) indicate that the COVID-19 outbreak may provide an illustrative analogy for sustainability challenges. This frame exemplifies how the COVID-19 and climate change crises require an analogous response even though they occur on different temporal scales. The analogy for the climate crisis frame is mainly supported by scientists who argue that similar to the COVID-19 pandemic, innovative and far-reaching interventions are needed to address the climate crisis [59]. These may include, for instance, the creation of new institutions and radical changes in behavioural patterns and policy governance. Along with this frame, it is stated that to curb the spread of coronavirus, ingrained societal behaviours were put to the test, including citizens´ freedom of movement. Likewise, deeply entrenched behaviours related to resource consumption and travel need to be challenged to tackle climate change. Energy and circular economy were the MSS most mentioned within this frame (Figure 7). Furthermore, even though these classes were not prominent in the MSS database (8% and 6% of the MSS, respectively), together with climate change MSS, they were the ones that were represented more strongly in the investigated frames.

### **5. Discussion**

Monitoring how the media portrays different topics and frames on sustainability transformations is challenging due to the complexity of the social systems and processes involved [60] and the large amounts of discursive material available. Against this background, we proposed a mixed-methods approach [61] for following the COVID-19 discourse on sustainability in Germany. In this section, we discuss the empirical and methodological contributions of the article and the limitations of the proposed approach.

### *5.1. Empirical Findings: Evidence for a "Window of Opportunity" Narrative*

Our empirical results show that the frequency of articles that mention sustainability topics in the early stages of the pandemic is limited (Figure 1). Indeed, an average of 0.6% of the articles in the corpus discussed sustainability topics each week (Figure 4). Topic modelling results indicate that (Table S4) only 1.5% of the articles were connected to environmental issues. Furthermore, google searches on sustainability on topics such as climate change, and environment were significantly lower between March and June 2020 compared to the same period in 2019 (Figure 2). This is not surprising, as during the first months of the crisis, media attention focused on the pandemic's social, economic, and epidemiological aspects rather than the climate crisis.

More broadly, these results support empirical studies that show that environmental sustainability is being overshadowed by COVID-19 [62]. Findings by [63] show that environmental issues have become less of a priority for international organisations since the COVID-19 crisis. Recent assessments indicate that the pandemic will likely undermine progress towards 12 of the 17 SDG goals [64,65]. Indeed, it is estimated that two-thirds of the 169 SDG targets are under threat due to the pandemic [66].

To discuss how our results relate to the SDGs, we linked our findings to Germany's national SDG performance [52] (Figure 8). We found that two SDGs in which Germany is not performing well were widely reported by the German news media: the SDG 13 (climate action) and the SDG 2 (food security and sustainable agriculture). A possible interpretation is that the COVID-19 crisis put the existing fragilities into the spotlight. Conversely, it could be argued that SDG 13 is intrinsically linked to all other SDGs, and the high number of SDG 2 articles is linked to panic buying during lockdowns and not necessarily connected to long-term food security. The analysis also showed that SDG 12 (sustainable consumption and production) is underrepresented in the COVID-19 media discourse, even though it is another area in which Germany is not performing well [52]. Still, the evidence indicates that COVID-19 has aggravated consumerism and the generation of waste [2,3]. This was identified in the challenge for sustainability frame (CST), where challenges for the circular economy were frequently pointed out.

**Figure 8.** Matrix comparing Germany's national SDG performance [52] and the number of MSS. The matrix shows four patterns: (**a**) high number of MSS and low performance; (**b**) high number of MSS and high performance; (**c**) low number of MSS and low performance; and (**d**) low number of MSS and high performance.

By looking closely at the qualitative data, we found that frames supporting a more sustainable green COVID-19 recovery programme (MAS, n = 126) are more frequent than those promoting economic recovery without advocating for climate action (MFEF, n = 21). Nevertheless, this cannot be translated into unequivocal support of a "window of opportunity" frame. It became apparent in our qualitative content analysis that, even in articles that see opportunities for accelerated sustainability transformations, such acceleration does not come on its own. Both the environmental impacts (PEIs) and challenge for sustainability transformation (CST) frames point out that the pandemic could also lead to an increased environmental burden.

In sum, there certainly are thoughts about rebuilding better and initialising sustainability transformations with COVID-19 recovery measures. There is, however, no dominant frame that sees the pandemic as a chance for sustainability. In fact, the frames are closely interwoven (Figure 6). News media give contradictory representations of the relationship between COVID-19 sustainability that range from stories on environmental recreation during lockdowns to accounts that prioritise an economic recovery and see sustainability regulations as an obstacle. Depending on the actors involved, the pandemic is seen as a

chance to rebuild more sustainably, or it is perceived as an economic threat that side-lines sustainability because more urgent issues are at hand. Therefore, it is necessary to continue monitoring the discourses as the pandemic unfolds since policy priorities and opinions are expected to change based on the severity of further waves of infection.

### *5.2. Methodological Considerations: Thickening Big Data*

By linking big data and thick data tools, we were able to deal with large volumes of text data; a volume that surpasses researchers' ability to conduct a close reading [67], yet the analysis remains true to the principles of traditional qualitative approaches. The use of NLP tools allowed us to scrutinise huge collections of documents across 21 news outlets and identify thematic clusters as a means of following discursive changes. Therefore, our results can be more effectively generalised. To delve deeper into the data, qualitative content analysis was employed to track specific frames.

The use of this approach allowed us to bridge knowledge gaps between qualitative and quantitative research by providing context to the "big data" through rigorous empirical qualitative research. This offers both a broad overview on relevant topics and a close analysis of particular frames. As such, the approach can yield new insights not easily achievable through traditional text mining or qualitative social science methods alone. On the one hand, NLP techniques can provide an overview of the topics that are being discussed in large-scale textual corpora through time. On the other hand, qualitative tools can yield a background to these findings by adding context. Therefore, here, we do not consider data-driven quantitative tools as an "end in itself", nor do we see traditional qualitative tools as a mere "supplement". Instead, we promote the use of both approaches alongside each other.

### *5.3. Limitations and Further Research*

Despite its advancements, the approach is not without limitations. First, we relied on a stratified sample of articles as the articles need to be downloaded manually, given that the database used does not allow for web scraping (see Section 3.1). Hence, less popular newspapers were ignored. Similarly, the thick data analysis focused on articles that mentioned at least two sustainability topics (Table S3, Figure S1, n = 461). As a result, they may be skewed towards the "window of opportunity" frame.

A further drawback is that we only considered the first wave of the COVID-19 pandemic, which limits the extrapolation of the findings. With this in mind, we recommend that future studies consider how the frames on sustainability have changed during different COVID-19 waves. This will allow one to assess whether or not the pandemic has had a discernible lasting impact on the media frames used to discuss sustainability.

Future work should more deeply examine how the media debate around sustainability within the COVID-19 crisis is shaped by both the actors involved and the power relationships of the organisations that promote these discourses. Additionally, the results of the qualitative analysis can also be retrofitted to the big data analysis, to capture the extent to which the identified frames and actors were mentioned in the broad sample.

### **6. Conclusions**

In this study, we examined the COVID-19-related discourse regarding sustainability transformations during the early stages of the crisis in Germany. The results show that sustainability topics were not prominent in contrast to other topics. When we looked deeper into the data, we found that there is no dominant "window of opportunity" frame that sees the pandemic as an unequivocal chance for sustainability. Some frames support a green COVID-19 recovery programme, and they appear more often than articles that support economic recovery without advocating for climate action. Still, they only represent a small fraction of the published articles. Furthermore, even in articles that perceive an opportunity for accelerated sustainability transformations, such acceleration is not taken

for granted. In fact, the presence of a window of opportunity does not necessarily imply that this opportunity will actually be taken [7].

Pragmatically, these findings are significant because reduced coverage about sustainability could dampen political will to act on the climate crisis. Indeed, as stated by Weber and Stern [17], the news media can influence people's thoughts and actions regardless of whether they are accurate or not. Even if the articles do not influence the public's opinions, the attention they bring to the topic can still be relevant as they can influence future news coverage. Thus, monitoring the immediate and long-term impacts of different COVID-19 waves on the sustainability discourse and their translation into policies remains a task for future investigation.

The mixed-methods approach described here can assist with this task by equipping scientists with a reliable toolkit for investigating discourses on rapidly evolving transformation processes. Researchers can use it to explore ongoing media discourses on sustainability transformations and detect trends in large-scale textual corpora.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/ 10.3390/su131911095/s1, Table S1. Selected newspapers, number of articles per month, duplicates and articles included in the analysis, Table S2. Additional stop words for the word cloud, network and topic modelling, Table S3. Sustainability transformation topics, their related SDGs and Green Deal goals, and keywords used for coding the newspaper articles and for the google trends search, Table S4. Topics obtained by applying the LDA model to the 61,514 COVID-19-related articles. Each topic consists of a series of keywords, a topic name (assigned by the authors based on manual inspection of keywords and articles), the topic coherence and prevalence. Following LDA, one article can be assigned to more than one topic, Table S5. Correlation matrix showing Pearson correlations between the MSS, where 0 indicates a low correlation and 1 a strong correlation, Table S6. Automatic classification accuracy (%) for different MSS topics. Of the 2343 articles, 40% were randomly selected and read to determine the accuracy of the classification system, Figure S1. Sampling flowchart, Figure S2. Comparison word cloud of the changing discourse in the news media in Germany. The size of the word is proportional to its frequency. It compares the predominance of specific terms for each (a) month and (b) sustainability topic. For instance, the word "Klimawandel" (climate change) was used 158 times in April 2020 (frequency = 0.0007) and 91 times in May 2020 (frequency = 0.0004). Because this word was used more frequently in April, it only appears in the April section of the comparison cloud, Figure S3. Network analysis graph with word relationships in the MSS corpus. It portrays words that are strongly vs. weakly integrated. Each node represents a word, which must have at least 30 occurrences in the entire corpus to be present in the graph. The size and colour of the node represents the word's frequency. The darkness of the connecting lines depicts the strength of the word pair's relationship, Figure S4. Distribution of articles (n = 367) according to different frames over time. The size of the bubbles varies according to the number of articles per week.

**Author Contributions:** M.M.d.B.: Conceptualisation, methodology, software, validation, formal analysis, visualisation, data curation, writing—original draft, writing—review and editing. D.O.: Conceptualisation, methodology, validation, formal analysis, writing—original draft, writing—review and editing. C.K.: Conceptualisation, supervision, writing—review and editing. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Data Availability Statement:** Data used for the analyses are available upon request.

**Acknowledgments:** Christian Kahmann and Tim Golla are thanked for their valuable research assistance.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


## *Article* **Investigating the Drivers of Supply Chain Resilience in the Wake of the COVID-19 Pandemic: Empirical Evidence from an Emerging Economy**

**Mohammad Ali Yamin**

Department of Human Resources Management, Collage of Business, University of Jeddah, Jeddah 23454, Saudi Arabia; mayameen@uj.edu.sa

**Abstract:** The COVID-19 pandemic has disrupted supply chain operations globally. Nevertheless, resilient firms have the capacity to combat an unprecedented situation with the right strategic approach. The current research has developed an integrated research model that combines factors such as supply chain intelligence, supply chain communication, leadership commitment, risk management orientation, supply chain capability and network complexity to investigate supply chain resilience. The research model of this study was empirically tested with 309 responses collected from supply chain managers. Results revealed that supply chain resilience is measured with supply chain intelligence, supply chain communication, leadership commitment, risk management orientation, supply chain capability and network complexity and demonstrated a substantial variance *R*<sup>2</sup> of 0.548% towards supply chain resilience. Practically, this study suggests that supply chain managers should focus on factors such as big data analytics, risk management orientation,1 supply chain communication and leadership commitment to enhance supply chain resilience and sustainable supply chain performance.

**Keywords:** supply chain resilience; supply chain intelligence; communication; leadership commitment; big data analytics; sustainable supply chain performance

### **1. Introduction**

In this dynamic environment, the major challenge for supply chain practitioners is to deal with supply chain upheavals, disruption, and unforeseen events [1]. If a firm faces upheavals in supply chain operations and continue to perform, that situation is characterized by resilience [2]. Supply chain resilience is defined as operational capacity of a firm to return to its initial state after being disrupted and be stronger than before in a supply chain process [3]. The importance of supply chain resilience is highlighted in earlier studies [1,3–5]. According to Brandon-Jones et al. [1] firms are facing more disruption due to natural and manmade events and therefore resilience phenomenon should be investigated to understand how resilience help firms to recover quickly after disruption. In current situation wherein COVID-19 pandemic has left devastated impact global economy and badly affected supply chain operations and therefore organizations are now seeking resilient kind of strategies to confront unforeseen events [6,7].

Concerning supply chain disruption, the literature indicates that the pandemic has destabilized supply chain operations and negatively impacted customer needs requirements and satisfaction [5,8,9]. Therefore, the current study fills a research gap and develops an integrated supply chain resilience model with a combination of factors such as supply chain intelligence, supply chain communication, leadership commitment, risk management orientation, supply chain capability, network complexity and big data analytics to investigate supply chain resilience during the COVID-19 pandemic and sustainability supply chain performance in a post-pandemic context. To enhance supply chain resilience, the researcher has paid attention to supply chain intelligence and communication strate-

**Citation:** Yamin, M.A. Investigating the Drivers of Supply Chain Resilience in the Wake of the COVID-19 Pandemic: Empirical Evidence from an Emerging Economy. *Sustainability* **2021**, *13*, 11939. https://doi.org/10.3390/ su132111939

Academic Editors: Marc A. Rosen, Baojie He, Ayyoob Sharifi, Chi Feng and Jun Yang

Received: 4 September 2021 Accepted: 21 October 2021 Published: 28 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

gies. Supply chain intelligence is a process of integrating knowledge that is derived from suppliers, customers and competitors and using that knowledge to manage supply chain operations [10,11]. Therefore, communication is the extent to which a firm facilitates supply chain partners through communication, messages, and communication networks [12]. The impact of leadership commitment is found to be positive in measuring supply chain resilience. For instance, authors such as Speier, Whipple, Closs, and Voss [13] postulated that leadership commitment has been used as a foundation in designing and implementing supply chain strategies. Similarly, prior research conducted by Wieland and Marcus Wallenburg [14] confirmed that risk orientation reduces the failure chances in the supply chain process. Furthermore, supply chain capability and network complexity have shown a positive influence on measuring supply chain resilience [15,16].

The research model as shown in Figure 1 is extended with the moderating role of big data analytics. Big data analytics is identified as a combination of technologies, processes and techniques that enable organizations to collect, organize, visualize and analyze data and bring swiftness into supply chain operations [17]. The moderating effect of big data analytics is examined between supply chain resilience and sustainable supply chain performance. To the best of the authors knowledge, this study is the first that integrates factors such as leadership, technology and network factors altogether to investigate supply chain resilience and sustainable supply chain performance. This study is significant as it investigates the role of supply chain resilience and sustainable supply chain performance during the COVID-19 pandemic. In addition to that, the results of this research disclose several useful findings for the manufacturing industry to understand how to bring resilience in supply chain operations whenever they confront unforeseen events. The remaining part of the research is included in the literature review, methodology, data analysis, discussion and the conclusion of this study.

**Figure 1.** Research framework.

### **2. Literature Review**

### *2.1. Supply Chain Intelligence and Communication*

The concept of supply chain intelligence is extracted from knowledge-based resources and is explained as the extent to which a firm integrates knowledge that is derived from suppliers, customers and competitors and uses this to manage supply chain operations [11]. Supply chain intelligence is a process of knowledge integration among supply chain partners [11,18]. The use of a supply chain intelligence strategy gives a holistic view of the supply chain process and reduces disruption in supply chain operations [18–20]. Earlier studies have established that intelligence characteristics enable organizations to forecast accurately, reduce risk and make firms more resilient in response to supply chain uncertainty [18,19,21]. Therefore, communication is the extent to which a firm facilitates supply chain partners through communication, messages and communication networks [12] The literature has shown that intelligence and communication bring supply chain integration, responsiveness, information exchange and leverage to a higher supply chain performance [12,21,22]. A recent study conducted by Asamoah et al. [22] has confirmed that both communication and supply chain intelligence positively impact supply chain resilience. Therefore, the following hypotheses are proposed:

**Hypothesis 1 (H1).** *Supply chain intelligence significantly impacts SC resilience*.

**Hypothesis 2 (H2).** *Supply chain communication significantly impacts SC resilience*.

### *2.2. The Role of Leadership Commitment and Risk Management Orientation*

The importance of leadership commitment is vital in planning, designing and implementing a supply chain strategy [15]. As suggested by Speier et al. [13] leadership commitment is considered as a foundation in the supply chain process. The literature has confirmed that a leader's commitment motivates employees, brings pro-activeness and ensures that resources are being used adequately [13,23–25]. It is established that leadership commitment assists supply chain managers in implementing decisions transparently to avoid supply chain disruption, which, in turn, enhances resilience in operations [26,27]. Concerning risk orientation, earlier studies have confirmed that risk orientation reduces the failure chances in a supply chain process [14]. Organizations can handle uncertainty in a supply chain process through risk orientation, which ultimately boosts supply chain resilience [28–30]. Therefore, the following hypotheses are proposed:

**Hypothesis 3 (H3).** *Leadership commitment significantly impacts SC resilience*.

**Hypothesis 4 (H4).** *Risk management orientation significantly impacts SC resilience.*

### *2.3. Supply Chain Capability and Network Complexity*

Supply chain capability denotes to the ability of a firm to identify, manage, and synchronize information that facilitates a firms internal and external supply chain operations [16]. The literature provides abundant evidence that the supply chain capability improves a firm's operational and financial performance by reducing cost and enhancing resilience in supply chain operations [2,16,31]. Therefore, in the current research, we centered our attention toward the supply chain capability and resilience. According to Nishat Faisal, Banwet, and Shankar [32] strong coordination is required among supply chain partners to manage, forecast and replenish inventory. Authors such as Bhamra et al. [2] asserted that supply chain capability plays a vital role in the flow of goods, reduces lead time, brings transparency in supply chain operations, speeds up the payment cycle, reduces inventory and avoids over production. The network complexity is identified as the degree of connectivity between the network length and the number of nodes [15]. The complexity of the network is increased with an increase in the number of nodes and network length [4,13]. Thus, the researcher assumed that supply chain resilience may enhance with a decrease

in the numbers of nodes and the network length. Therefore, the following hypotheses are proposed:

**Hypothesis 5 (H5).** *Supply chain capability significantly impacts supply chain resilience.*

**Hypothesis 6 (H6).** *Decrease in network complexity significantly impacts supply chain resilience.*

### *2.4. Big Data Analytics*

Big data analytics is an emerging concept and is being acknowledged as a process that enables organizations to collect, process, store and analyze data further to obtain useful insights [17]. In a supply chain setting, big data analytics is conceptualized as a combination of technologies, processes and techniques that enable organizations to collect, organize, visualize and analyze data and bring swiftness to their supply chain operations [17]. The extant literature has confirmed the significant influence of big data in reducing SC disruption while increasing SC resilience [33–38]. According to Janssen et al. [34] data analytics is significantly related to the supply chain innovation process. Another study conducted by Dubey et al. [17] has confirmed that big data analytics positively impacts the predication of supply chain resilience. Following the above arguments and supported by [17]. the current research tests the moderating effect of BDA on the relationship of supply chain resilience and sustainable supply chain performance. Thus, the following hypotheses are proposed:

**Hypothesis 7 (H7).** *Resilience significantly impacts sustainable SC performance.*

**Hypothesis 8 (H8).** *Big data analytics has a moderating impact on SC resilience and sustainable SC performance.*

### **3. Research Methods**

### *3.1. Designing Questionnaire and Instrument Development*

In line with research objectives, the research framework of this study is developed under a quantitative and cross-sectional research approach. The researcher has followed a positivist paradigm to design the research in line with [39]. In addition to this, data are collected through research questionnaires. The research questionnaire in this study comprises construct items and demographic characteristics of the respondents. Construct items were developed by reviewing the literature and previously established scales. Scale items for network complexity were adopted from [15]. Risk management orientation scale is adopted from Wieland and Marcus Wallenburg [14] and then slightly adapted into the current research context. Supply chain resilience scale items were adopted from a previously developed scale by Brandon-Jones et al. [1] then slightly adapted. Scale items for the construct big data analytics were adopted from [17]. Supply chain communication scale items were adopted from [21]. Similarly, supply chain intelligence items were adopted from [22]. Therefore, supply chain capability scale was adapted from [16,22]. Scale items for leadership commitment were adapted from [7]. Construct items for sustainable supply chain performance were adopted from [40,41]. The scale items are measured using a 7-point Likert scale, where 1 denotes strongly disagree and 7 denotes strongly agree [42].

### *3.2. Sampling and Data Collection*

This study uses an empirically tested research framework to examine supply chain resilience and sustainable supply chain performance. Therefore, manufacturing firms are selected as the unit of analysis in this research. Earlier studies have emphasized that manufacturing companies provide in depth understanding about supply chain operations and organizational performance [2,16,31]. The population of this research comprises supply chain managers working in manufacturing companies in Saudi Arabia. As suggested by M. Yamin [43] and consistent with prior power analysis, a sample of 300 respondents was

selected for data analysis. Research data of this study were collected through an online research survey. Data were collected during the month of May 2021. The research survey was designed and conducted during COVID-19 pandemic; consequently, the researcher used online tools such as social media platforms, Google forms and email for data collection. Concerning respondent selection, the researcher selected respondents through a convenience sampling approach. For data collection, 750 questionnaires were forwarded to supply chain managers with a request to fill out an online research questionnaire according to their knowledge about supply chain resilience and sustainable supply chain performance. In response to the online research survey, 317 questionnaires were retrieved from respondents. Nonetheless, 8 questionnaires were discarded due to inadequate and inappropriate filling. Thus, a total of 309 responses were used for inferential analysis. Descriptive analyses were conducted with SPSS software. The results of the descriptive analysis revealed that the data set comprised 198 males and 111 females among 309 respondents. Similarly, the respondents' ages were considered. The results indicate that the majority of the respondents 143 were found to be between 21 and 30 years. Next to this, 35 respondents were found to be between 41 and 50 years old. Furthermore, 131 respondents were aged between 31 and 40 years. Finally, data were estimated with a structural equation modeling (SEM) approach.

### **4. Data Analysis**

### *4.1. Common Method Bias*

The common method variance bias (CMV) needed to be examined before the inferential analysis to ensure that the research data had no bias [44]. Since the data were collected from a single source, a CMV issue may occur in this research [45,46]. Referring to the CMV issue, the literature has suggested two well-known methods, namely, procedural and statistical remedies [45,47–49]. Following the procedural method, the questionnaire items were jumbled up, consistent with [48]. Therefore, CMV is statistically confirmed with Harman's single factor solution. Harman's single factor solution suggests that the variance explained by the first factor must not be higher than 50% [48]. The results, as depicted in Table 1, indicate that the variance explained by the first factor is less than the threshold value (50%), hence confirming that CMV is not a potential issue in this study.

**Table 1.** Harman's single factor analysis.


### *4.2. Structural Equation Modeling*

The theoretical framework of this study was tested with a structural equation modeling (SEM) approach [42]. The researcher opted for a two-stage approach for the structural equation modeling computation, which included the assessment of a measurement model and a structural model [43,50]. The reliability and validity of the constructs was tested with a measurement model [43]. Therefore, the hypothesized relationship was confirmed with a structural model [51]. The data were evaluated with Smart PLS software using a partial least square (PLS) approach [52].

### 4.2.1. Assessing Measurement Model

The measurement model was examined to establish the construct's validity and reliability. The convergent validity of the constructs was achieved with average variance extracted (AVE) following the criterion that that AVE values must be higher than 0.50. Nevertheless, the construct's reliability was achieved following the values of (α) and composite reliability in line with Mohammad Ali Yousef Yamin and Sweiss [53] who

suggested that the CR and Cronbach's alpha values should be higher than 0.70, reflecting adequate construct reliability and validity. The findings, as depicted in Table 2, revealed the adequate reliability, validity and convergent validity of the constructs.


**Table 2.** Measurement model.

The measurement model had established the construct's convergent validity and reliability. Therefore, Fornell and Larcker analysis was incorporated for the assessment of the construct's discriminant validity [24,54]. The Fornell and Larcker analysis suggests that the square root values of the average variance extracted must be higher than the other constructs correlations [24]. The findings indicate that the square root of AVE is higher when compared with other constructs correlations, thus establishing the discriminant validity of the measure. The findings of the Fornell and Larcker analysis are tabulated in Table 3.


**Table 3.** Discriminant validity.

Although the Fornell and Larcker analysis was used extensively, it has some deficiencies in computation [54–56]. The cross-loading method was used in this study as an alternative method and consistent with earlier studies by [54]. The cross-loading method suggests that the indicator loading of the construct must be higher than corresponding constructs loading [57]. The results of the cross-loading method revealed that construct loadings are satisfactory when compared with the corresponding constructs loading and, therefore, establish the discriminant validity of the constructs. The results of the cross loading are exhibited in Table 4.

**Table 4.** Cross loadings.


Prior studies have proposed Heterotrait-monotrait (HTMT) analysis for the testing of discriminant validity. HTMT analysis was introduced by Gold, Malhotra, and Segars [55] and suggests that the HTMT ratios must be lower than 0.85 or 0.90 to indicate the adequate discriminant validity of the constructs [55,56]. The results of the HTMT ratio analysis show that the HTMT values are less than 0.90 and hence, confirm the adequate discriminant validity of the variables. The values of HTMT are presented in Table 5.


**Table 5.** Heterotrait-Monotrait analysis.

4.2.2. Assessing Structural Model

The structural model tests the hypothesized relationship between variables. In order to mitigate the normality issue, data were bootstrapped with dummy data of 4000, as suggested by [58]. The results of the structural model are given in Table 6 comprising path value, standard error, t-statistics and the significance of the hypotheses.



The results of the structural model indicate that although the exogenous variables have a significant impact on supply chain resilience, the relationship between network capacity and supply chain resilience is insignificant. The following results indicate that supply chain intelligence significantly impacts supply chain resilience and support H1: *β* = 0.113 path; significance, *p* < 0.001 and t-statistics, 2.634. Supply chain communication has a positive impact on supply chain resilience and confirms H2: *β* = 0.183 path; significance, *p* < 0.001 and t-statistics, 2.859. Similarly, leadership commitment has revealed a positive impact on measuring supply chain resilience and statistically confirms H3: *β* = 0.210 path; significance, *p* < 0.001 and t-statistics, 2.635. The statistics show that both risk management orientation and supply chain capability have a positive impact on supply chain resilience and are supported by *β* = 0.326 path; significance, *p* < 0.001 and t-statistics, 4.012; *β* = 0.116 path, significance, *p* < 0.05 and t-statistics, 1.933, hence establishing H4 and H5. Contrary to the researcher, the relationship between the expected network complexity and supply chain resilience was found to be insignificant (*β* = 0.028 path; significance, *p* > 0.05 and t-statistics, 0.679) and, therefore, H6 is rejected. Finally, the supply chain resilience has shown a positive impact toward sustainable supply chain performance and confirms H7: *β* = 0.667 path; significance, *p* < 0.001 and t-statistics, 2.678, the results of hypotheses including path coefficient and significant level are shown in Appendix A.

### 4.2.3. Assessing Effect Size, Predictive Power and Coefficient of Determination

The effect size of the variables is examined through *f* <sup>2</sup> analysis [50,51]. According to Samar Rahi et al. [58] the coefficient of determination, *R*2, reveals the collective impact of all the exogenous variables towards the criterion variable. Nevertheless, variable size as a single factor should be assessed with effect size analysis. The results demonstrate that in measuring the supply chain sustainable performance, the effect size of supply chain resilience is substantial and, therefore, a potential construct for managerial implication.

Concerning the coefficient of determination, the results of the structural model revealed that exogenous variables have a substantial variance *R*<sup>2</sup> of 0.548% on supply chain resilience. Similarly, the findings indicate a substantial variance *R*<sup>2</sup> of 0.550% in measuring sustainable supply chain performance, which was predicted by supply chain resilience and big data analytics. Aside from the substantial coefficient of determination for supply chain resilience and sustainable supply chain performance, the predictive power *Q*<sup>2</sup> of the framework was computed. The result of the blindfolding method has established that the research model has substantial power to predict supply chain resilience and sustainable supply chain performance. The results of the effect size, predictive power and coefficient of determination are shown in Table 7.



### 4.2.4. Importance and Performance Analysis

The current research model has integrated numerous factors to determine sustainable supply chain performance. Therefore, the importance and performance of the variable are tested with importance performance matrix analysis (IPMA) consistent with earlier studies by [42,59]. Before applying IPMA analysis, it was important to choose an outcome variable. In this study, the researcher selected supply chain sustainable performance as the outcome variable. The findings of the importance performance analysis indicate that supply chain resilience has the highest importance/total effect in measuring sustainable supply chain performance. Therefore, the importance of big data analytics, risk management orientation, supply chain communication and leadership commitment show an intermediate level of importance. The results of the IPMA analysis are exhibited in Table 8 with the importance and performance indexes.

The importance of the constructs is observed through an IPMA map. The IPMA map exhibits that supply chain capability, intelligence and network complexity have less importance in measuring sustainable supply chain performance. Therefore, the importance of big data analytics, risk management orientation, supply chain communication, leadership commitment and supply chain resilience is considerable. Thus, managers and policy makers should focus on factors such as the importance of big data analytics, risk management orientation, supply chain communication, leadership commitment and supply chain resilience to enhance the sustainable supply chain performance in their organization. The IPMA map exhibited in Figure 2 clearly shows the performance of the constructs on the *Y*-axis and importance on the *X*-axis.


**Table 8.** Results of IPMA.

**Figure 2.** IPMA analysis map.

### *4.3. Moderating Effect of Big Data Analytics*

The impact of technology is substantial in achieving the strategic goals of a firm [60]. In line with the above argument, this study has outlined big data analytics in a research model as moderating the variables between supply chain resilience and sustainable supply chain performance. The moderating effect of big data analytics was tested through a product indicator approach consistent with earlier studies by [53]. For statistical computation, the orthogonalization method was selected. The findings indicate that BD analytics moderates the relationship outlined between supply chain resilience and sustainable supply chain performance and statistically confirms H8 (*β* = 0.138; significant at *p* < 0.001; t-statistics, 4.094). Figure 3 exhibits the findings of the moderating effect.

The moderating effect was further analyzed with simple slope analysis to test the strength of the moderating effect. According to S. Rahi [61] simple slope analysis reveals the trend of the moderating effect i.e., whether it moderates strongly or weakly. Nevertheless, the result of the simple slope analysis shows an upward BDA trend of +1SD on the green line compared to the negative BDA at −SD on the red line, which means that simple slope analysis has a declining trend. Nonetheless, the blue line indicates a neutral impact between the highest and lowest moderating effect. Hence, the result of the moderating effect establishes that the higher use of big data analytics in a supply chain operation will increase the supply chain resilience and sustainable supply chain performance. The simple slope analysis graph is displayed in Figure 4.

**Figure 3.** Moderating analysis.

**Figure 4.** Output of simple slope analysis.

### **5. Discussion**

The results of this research unfold interesting facts for both academic researchers and practitioners. The current research has synthesized the literature into two main streams. At the first stage, supply chain resilience is determined by supply chain intelligence, supply chain communication, leadership commitment, risk management orientation, supply chain capability, network complexity and the substantial variance *R*<sup>2</sup> of 54.8%. Therefore, the second stream of the literature focused on sustainable supply chain performance with supply chain resilience and big data analytics and revealed a considerable variance *R*<sup>2</sup> of 55.0% in sustainable supply chain performance. These findings established the theoretical validity of the research model in determining supply chain resilience and sustainability supply chain performance. Similarly, the literature has confirmed the moderating role of big data analytics between the relationship of supply chain resilience and sustainable supply chain performance. The moderating effect of big data analytics indicates that the higher use of BD analytics in supply chain operations will raise supply chain resilience and sustainable supply chain performance.

Concerning the hypothesized relationships, the findings of the structural model indicate that supply chain intelligence positively influences supply chain resilience, which is consistent with earlier studies by [11,18]. Similarly, supply chain communication has shown a positive impact on supply chain resilience, which is in line with [12,22,38]. Leadership commitment had revealed a positive impact on measuring supply chain resilience, which supports prior studies by [13,15]. Pointing to risk management orientation and supply chain capability, both factors have shown a positive impact on supply chain resilience, which was supported by earlier studies by [28–30]. Nevertheless, network complexity has shown an insignificant influence supply chain resilience that was beyond our expectations. Therefore, supply chain resilience has shown a positive impact toward sustainable supply chain performance, which is consistent with an earlier study by [17]. Following IPMA and effect size analysis, this research suggests that during the COVID-19 pandemic, supply chain resilience and sustainable supply chain process could be enhanced by focusing on factors such as big data analytics, risk management orientation, supply chain communication and leadership commitment.

### *Research Contribution to Theory and Practice*

The findings of this research have contributed to theory and practice. For instance, the current research shows that factors such as supply chain intelligence, communication, leadership commitment, risk orientation, supply chain capability and network complexity positively relate to supply chain resilience. These findings clearly indicate that policy makers and supply chain managers should focus on the outlined factors to enhance supply chain resilience in supply chain operations. Another aspect of this research is to shed light on sustainable supply chain performance. The results of this study reveal that a sustainable supply chain is determined by supply chain resilience and big data analytics. Therefore, supply chain managers should improve supply chain resilience in order to improve sustainable supply chain performance. Aside from industry utilization, this research contributes to the academic literature by developing an integrated supply chain model that comprises market-oriented factors to determine supply chain resilience and sustainable supply chain performance. In addition to that, the variance explained by exogenous variables was substantial for supply chain resilience and sustainable supply chain performance, which, in turn, confirms the validity of the research model. Furthermore, this study contributes to information system literature by testing the moderating effect of big data analytics between supply chain resilience and sustainable supply chain performance. The findings establish that a higher use of big data analytics in supply chain operations will increase supply chain resilience and sustainable supply chain performance. Therefore, managers and policy makers should introduce big data analytics tools in supply chain operations to boost supply chain resilience and sustainable supply chain performance.

### **6. Conclusions**

The aim of this study was to investigate factors that influence supply chain resilience and sustainable supply chain performance during the COVID-19 pandemic. The current study developed an integrated research model that combined factors such as supply chain intelligence, supply chain communication, leadership commitment, risk management orientation, supply chain capability and network complexity to investigate supply chain resilience. On the other side, the research model was extended with the moderating effect of big data analytics between the relationship of supply chain resilience and sustainable supply chain performance. The research model of this study was tested using structural equation modeling. The results of the structural model computation revealed that supply chain intelligence, supply chain communication, leadership commitment, risk management orientation, supply chain capability and network complexity have shown a substantial variance *R*<sup>2</sup> of 0.548% toward supply chain resilience. Therefore, supply chain resilience and big data analytics have an explanation to the variance *R*<sup>2</sup> of 0.550% in measuring sustainable supply chain performance. In addition to that, the validity of the research model was tested with predictive analysis *Q*<sup>2</sup> using a blind folding procedure. The results of the predictive analysis *Q*<sup>2</sup> revealed that the research model has large predictive power to predict supply chain resilience and sustainable supply chain performance 37.4% and 40.0%, respectively. This research contributes to the theory by developing an integrated research model toward supply chain resilience and sustainable supply chain performance. Therefore, for practical implications, this study suggests that supply chain managers should focus on factors such as big data analytics, risk management orientation, supply chain communication and leadership commitment to enhance supply chain resilience and sustainable supply chain performance. In addition to that, the moderating effect of big data analytics is confirmed between the relationship of supply chain resilience and sustainable supply chain performance. These findings established that the use of big data analytics in a supply chain operation will increase supply chain resilience and sustainable supply chain performance. Thus, supply chain managers and policy makers should incorporate big data analytics in supply chain operations to enhance supply chain resilience and sustainable supply chain performance.

### *Research Limitations and Future Direction*

This research has some limitations that reveal a potential area of future research. First, this study was developed as an integrated supply chain research model that combined factors such as supply chain intelligence, supply chain communication, leadership commitment, risk management orientation, supply chain capability, network complexity and big data analytics to investigate supply chain resilience and sustainable supply chain performance phenomenon. Nevertheless, there are some other factors that could impact supply chain resilience such as supplier relationship, uncertainty and inter departmental coordination. Thus, extending the current research model with some additional factors could reveal interesting findings. Another limitation of this research is related to research design. The research design was based on a cross-sectional design and, therefore, respondents were restrained to participate at once in the research survey. It is expected that results may differ in a longitudinal research design. Therefore, the supply chain resilience phenomenon should be investigated with a longitudinal research design. The data were collected through a convenience sampling approach, which is a non-probability sampling approach. It is suggested that future researchers collect data through a probability sampling approach to mitigate any kind of sampling risk. The research model was designed from a developing countries perspective and tested the resilience behavior of a Saudi manufacturer. However, a future researcher may test the current research model in the context of other developing countries to enhance the generalizability of the research model.

**Funding:** This work is funded by the University of Jeddah, Jeddah, Saudi Arabia, under grant No. (UJ-21-DR-46). The authors, therefore, acknowledge with thanks the University of Jeddah technical and financial support.

**Institutional Review Board Statement:** Not Applicable.

**Informed Consent Statement:** Not Applicable.

**Data Availability Statement:** Delivered on demand.

**Acknowledgments:** Author would like to thank anonymous reviewers for the critical and constructive suggestions which in turn give an opportunity to rethink and reconstruct manuscript. In addition to that, author is grateful to the editor-in-chief and assistant editor for their humbleness during the revision process.

**Conflicts of Interest:** The author declares no conflict of interest. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

**Appendix A. Path Coefficient and Significance Level**

### **References**


## *Article* **Evolution of the Global Scientific Research on the Environmental Impact of Food Production from 1970 to 2020**

**Alessio Cimini**

Department for Innovation in the Biological, Agrofood and Forestry Systems, University of Tuscia, Via S.C. de Lellis, 01100 Viterbo, Italy; a.cimini@unitus.it

**Abstract:** Food production and consumption account for a significant share of the impact of various pressing and important environmental concerns such as climate change, eutrophication, and loss of biodiversity. In this work, a bibliometric analysis of the last 50 years of research papers, written in English and indexed on Scopus database, was carried out to highlight the evolution of the global scientific research in the environmental assessment of food production (EAFP). The research papers in EAFP started to significantly increase from 2005, being most frequently published by the *Journal of Cleaner Production* and *International Journal of Life Cycle Assessment*. The United States of America was the first publishing country, followed by China, the United Kingdom, and Italy. Wheat, rice, fish, maize, and milk were the food items mainly studied, with different importance depending on the authors' publishing country. *Life Cycle Analysis, Carbon Footprint*, and *Water Footprint* were the first three standard methods used to assess *climate change, energy consumption,* and *environmental impact*. The *Wageningen University, Chinese Academy of Sciences* and *Research Centre*, and *China Agricultural University* were the main publishing research centers. All the papers published worldwide received 18.1 citations per paper, the UK and Chinese papers being those mostly and minimally cited, respectively. Over the last five years, this research field largely aimed to managing the agricultural practices, mitigating global warming and water use, assuring food security and sustainable food consumption, while minimizing food waste formation. Such an objective evaluation of this research topic might help guide researchers on where to address their future research work.

**Keywords:** environmental analysis; food production; bibliometric study; citation number; authors; affiliations; Scopus

### **1. Introduction**

Earth's environment is affected by human activities. Since the late 20th century, numerous studies and published papers have highlighted how pollution, burning fossil fuels, deforestation and land exploitation may directly or indirectly cause damage to the environment [1,2]. Today, the increasing human overpopulation and its resulting adverse impacts affect the Earth's environment through ocean acidification, global warming, biodiversity loss, soil erosion, air pollution and undrinkable water. Every choice made can exert multifaceted environmental effects, either negative or positive, on four subsystems [3] like renewable and non-renewable materials; water consumed; land used for agricultural, forest, and grazing areas; raw material extraction or private housing [4]; and emissions such as greenhouse gases (GHG) and several other pollutants, namely SOX, NOX, and O3.

Food production accounts for a significant share of the total impact of several important environmental categories, such as climate change, eutrophication and loss of biodiversity. Between 22% and 37% of global anthropogenic emissions may be attributed to the overall food system [5]. Agricultural production involves the manufacture of fertilizers, pesticides, equipment and energy, as well as land-use change, and is responsible for the great majority (72–82%) of the above GHG emissions. The post-production (2.4 Tg

**Citation:** Cimini, A. Evolution of the Global Scientific Research on the Environmental Impact of Food Production from 1970 to 2020. *Sustainability* **2021**, *13*, 11633. https://doi.org/10.3390/su13 2111633

Academic Editors: Baojie He, Ayyoob Sharifi, Chi Feng, Jun Yang and Marc A. Rosen

Received: 30 August 2021 Accepted: 10 October 2021 Published: 21 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

CO2e) and post-sale (1.0 Tg CO2e) steps produce results of only a minor magnitude, while emissions embedded in food wasted at the consumption phase (~1.6 Tg CO2e) are no way insignificant. About 63% of food-related GHG emissions derived from the production and consumption of animal-based products, except fish and fisheries, 8.5 ± 2.4 Tg CO2e in 2010 [5].

Food production systems are often complex and involve biological systems, which are difficult to control and measure. Some examples are the evaluation of *land occupation* generally resulting in different crop outputs; *soil quality*, this depending on the water content, presence of organic matter, nutrients, heavy metals, living organisms and soil texture and structure [6]; *carbon storage in soils* and *standing biomass*; *yield variability*, between years owing to weather conditions and other factors; and *consumer behavior* towards food consumption, portion sizes, packaging, as well as wastage,. Given the many factors involved, it is quite a difficult task to assess the environmental impact of food products and production systems.

Life Cycle Assessment (LCA) dates to the 1960s, when the energy analyses of industrial systems started to be carried out to account for the oil crises of the early 1970s. The rise of environmental awareness in the late 1980s had the effect of increasing the attention paid to LCA as a potentially valuable environmental management tool. The International Standards Organization (ISO) released four ISO LCA standards (ISO 14040 to 14043) from 1997 to 2000. Following this, the LCA became a decision-support tool for several food companies. The Coca Cola Co. had started to account for the environmental impact of its packaging since 1969. Then, other companies (e.g., Tetrapak, Nestlé, Unilever, Arla Foods) and many beer companies engaged in assessing and improving their environmental sustainability [7,8].

Over the last decade there has been significant advances in terms of the environmental analysis of food products (EAFP). A bibliometric analysis of the research studies performed so far might be used as a basis for the comprehensive understanding of current research on the EAFP and, thus, highlight some potential future research directions.

Several bibliometric studies have been recently published to assess the scientific productivity related to climate change [9–11], food security [12], as well as food security in the context of climate change [13] or food waste [14]. Other scientometric reviews analyzed the effect of climate change on carbon sink [15], water quality [16], and human health [13]. However, no bibliometric study has so far attempted to analyze the scientific literature regarding the environmental assessment of food production and consumption.

Different multidisciplinary citation indices are available online, namely ISI Web of Science, Scopus, Google Scholar, these allowing the extraction of bibliometric indicators, which can be classified as *quantity, quality*, and *structural indicators* [17]. *Primary bibliometric indicators* are elementary measures. They consist of the simple count of publications (*quantity indicators*) and/or citations produced/received (*performance indicator*) by a single author, research group, or journal in a certain time interval and represent the starting point of many bibliometric analyses, even if such measures are usually not very suitable for representing the complexity in the various contexts of application. With regards to the citation count, there is a general agreement about its capacity to represent the impact of an article in the scientific community, but the citation behavior might be influenced by its publication date and language, the author's reputation, membership of an important scientific Institution, as well as the journal characteristics, such as the degree of internationalization, accessibility, and so on. At the same time, the citation data extracted can be used to highlight any connectivity between scientific fields, research groups, as well as authors.

The combinations of the primary indicators with other data (i.e., time interval, average number, etc.) allow the definition of a set of *secondary bibliometric indicators*, such as the *Impact* Factor (IF), and *H-Index* [17].

The construction of the *landscape of science* [18] is now considered an established and prolific field of research and application of bibliometrics. Thanks to natural language processing techniques and a linguistic filter employed by the elaboration software, terms occurring in titles and abstracts of a set of identical elements extracted from publications presents in a database of selected literature are represented as circles in a two-dimensional map [19]. The representation of a research field as a *term map*, or *co-word map*, allows strongly related terms to be located close to each other, while the greater their distance the weaker their relationship will be.

The main aim of this work was to trace global research trends and scientific evolution in the EAFP research from 1970 to 2020, by resorting to a bibliometric, textual, and map analysis to provide a basis for the comprehensive understanding of current research and to highlight the countries, institutions, authors, and journals more productive and influent in this research field.

### **2. Materials and Methods**

### *2.1. Bibliometric Analysis of Scientific Literature*

Elsevier Scopus database (that is, one of the largest abstract and citation database of peer-reviewed literature) was consulted in March 2021 to retrieve bibliographic records related to the environmental impact and sustainability of food production from peerreviewed articles that were published from 1970 to 2020 (included), this time period having been also subdivided into two time intervals (e.g., 1970–2015, and 2016–2020). This choice implied the exclusion of even important non-peer reviewed articles, some proceedings, communications, and patents, but they were out of the scope of this work. The outcome obtained from a database research analysis can be heavily conditioned by the query string. Indeed, the latter represents the most important tool to extract reliable results for a bibliometric analysis. The words present in the query string were selected as the most important related terms in the field of the environmental assessment of food production. To overcome the numerous limitations related to this kind of exercise, the database queries included two different categories of selected terms related to the topic, as detailed in the electronic supplement (Table S1). Namely, category 1 (C1) was composed of two thematic groups. The first one (A) was related to the *Standard Methods* usable for measuring the environmental impact of food production and consumption, and included 17 words; while the second one (B) included nine generic terms often used in research papers. Category 2 (C2) was composed of words identifying different *food products*. Owing to the numerous foods available, the FAO database (https://www.fao.org/faostat/en/#data, accessed on 18 October 2021) was consulted to identify the foods most relevant worldwide. To this end, all the words contained in the domain *Production* were extracted and included in a few categories, such as *Crop production* (Table S2), and *Crop processed, Live Animals, Livestock Primary,* and *Livestock Processed* (Table S3), by eliminating redundancies and similarity. Moreover, Table S4 shows both several *Generic* terms and other ones characterizing main foods and beverages as derived from the so-called FoodEx2 catalog, that is the *standardized* system recognized by the European Food Safety Agency (EFSA) (https://www.efsa.europa.eu/it/data/data-standardisation, accessed on 18 October 2021) for classifying and describing *foods* and *beverages*. Thus, a total number of 275 words were listed in Tables S2–S4.

The database queries consisted of a string obtained from all the possible binary combinations of all the terms included in the two categories C1 and C2, thus the 24 words in C1 were combined with each one of the 275 words in C2 using the 'AND' Scopus operator. In this exercise, only documents containing simultaneously the two terms in the *Title* were extracted. For an accurate description of the Scopus logical operators refer to the Scopus Search Guide (https://dev.elsevier.com/tips/ScopusSearchTips.htm, accessed on 18 October 2021). Obviously, it is almost impossible to produce a *perfect query string*, and background noise will be always produced. To improve the performance of this study, the Scopus search was performed by choosing the only *Title* (T) as research field (excluding Abstract and keywords) on the assumption that this was the most accurate strategy in relation to terms and research selection efficiency [20]. Bibliometric analysis is usually applied at three levels, the so-called macro level referring to national systems,

the meso level to institutions, including individual Universities, and finally the micro one to research groups or individual researchers [21].

The resulting database including the title, citation count, year, author, affiliation, country, and abstract was saved in CSV format and elaborated in the following different ways:


The search was restricted to publications written in English because it was almost impossible to translate all the scientific terms and keywords in English for the elaboration and analysis. Thus, many authors that wrote papers in their native language were excluded. Even if the database obtained did not include all the papers published in the EAFP field, the data collected allowed a general picture of the world scientific research in EAFP topic, and were not intended to draw up a ranking among countries, affiliations, or authors. In general, it was possible to assume a 10–15% underestimation of the data retrieved, as approximately evaluated by counting manually the number of articles published by several authors and automatically those extracted from the database using the search keys. Figure 1 shows a schematic diagram to highlight the bibliometric procedure used in this work.

**Figure 1.** A schematic diagram to highlight the bibliometric procedure used in this work.

### *2.2. Time Horizon*

Since the Scopus search had been conducted in March 2021, publications relative to the year 2020 were included in the analysis. Thus, the publications relative to the year 2021 were underestimated, their overall number being not yet completely indexed by the Scopus database. For the same reason, the citation count regarding the papers published in the year 2021 was excluded.

The selected bibliography was used to extract information regarding the number of publications, countries, main affiliations, authors, citations, and journals. The resulting information was also segmented for the five countries with greater publishing rate in this research topic. The Scimago database (https://www.scimagojr.com/, accessed on 18 October 2021) was used to extract journal info.

### **3. Results and Discussion**

### *3.1. Bibliometric Analysis*

3.1.1. Publication Trends from 1970 to 2020

An overall number of 4186 scientific research works resulted from the research performed here, their annual distribution being shown in Figure 2.

Such a trend was in line with that resulting from a research article that analyzed food security in the context of climate change from 1980 to 2019 [13] and retrieved as many as 5960 documents. The first paper appeared in the years 1968–1970 and until 1990 the overall number of papers published remained almost insignificant; then, two increasing trends were clearly identified. The first one covered the latest decade of the 20th century. Such a period represented a turning point of awareness of the scientific community and public opinion. First, the Antarctic ozone hole was discovered in the 1985. Within two years the United States and more than 100 other countries pledged to phase out the use of ozone-depleting compounds. In 1988, the Intergovernmental Panel on Climate Change (IPCC) was established by the World Meteorological Organization (WMO) and the United Nations Environment Program (UNEP) in order to provide the world with a clear and scientifically based view about the current state of knowledge on climate change and its

potential environmental and socio-economic impacts. According to [24], the first Life Cycle Assessment (LCA) study on food products was performed at the beginning of the 1990s and in 1996 was hold the first *LCA food* conference dealing with the environmental impacts analysis of the agri-food sector [25]. A second trend extending from 2000 to 2020 exhibited a definitively steeper growth rate of about 800 papers per year consequently not only to the recent attention to the environmental consequences of food production and consumption, but also to the increasing number of affiliations/authors publishing in this research field.

**Figure 2.** Annual distribution of the overall number of scientific research papers retrieved from the Scopus database from 1970 to 2020.

Table 1 lists the first 20 countries most publishing in this topic, as well as the total number of papers published together with their relative and cumulative percentages. Even if any document might be attributed to multiple countries, the total number of papers retrieved was higher than 4186 (Table 2)

The first 10 nations covered about 53% of the total documents produced, seven of these being European countries. The USA, China, and Italy were the three most productive countries with 548, 465, and 464 documents, respectively, even if the Climate Change Performance Index (CCPI) ranked their efforts to combat climate change at the 61st, 33rd, and 27th position, respectively [26]. In Europe, Italy (464), the UK (328), and Spain (283) were the first three publishing nations with their CCPI at the 27th, 5th, and 4ht position, respectively [26]. Surprisingly, they produced more scientific publications than The Netherlands (170), Denmark (141), and Sweden (161), but, in all, it is likely this achievement is linked to the overall number of inhabitants. Even in this case, in 2021 the CCPI of such countries inversely ranked at the 29th, 6th, and 4th position, respectively [27]. In 2019, the world's largest CO2 emitters were in descending order China (30.4% of total CO2 emissions), the United States (13.43%), India (6.83%), the EU27 + UK (8.69%), Russia (4.71%), and Japan (3.03%), these together accounting for 67% of total global fossil CO2 emitted [27]. As shown in Table 1, China, the USA, and EU28 carried out an intense research activity in this sector (as related to food), showing a certain attention to the problem, differently from Japan and Russia, that ranked at the 17th place and by far lower than the 20th place, respectively. Over the last five years, China, Italy, and USA were the main publishing countries. From 2016 to 2020 the number of publications was higher than that produced

from 1970 to 2015 for most of the countries listed in Table 1. A different behavior was noted for Denmark, Sweden, the Netherlands, Australia, and Canada, probably because their research interest switched to other topics.

**Table 1.** Overall number of publications by country, as well as their relative and cumulative frequencies, as indexed by the Scopus database in different time periods (i.e., 2020–1970; 2020–2016; 2015–1970).


**Table 2.** Top 5 nations ranked by the number of documents published over the 1970–2020 period together with the overall number of citations and average citations per paper index (CPPI).


From the bibliometric analysis carried out here, it was possible to extract an overview of the number of citations per each article. This assessment was just restricted to the year 2020, because many of the articles published in the 2021 have not been cited yet. Table 2 show the first five countries ranked by the number of documents published during the time periods accounted for, together with total citation count and citation per paper index (CPPI). All the scientific papers retrieved from Scopus database received an average number of 20.3 citations per paper. Interestingly, Table 2 displays that CPPI ranged from 16.6 to 29.6 for Chinese and UK papers, respectively. For some countries (i.e., the UK, USA, Spain, and Italy) CPPI was greater than the world average value, whereas for the second publishing country (i.e., China), such an index was smaller than 20.3. Thus, the CPPI might be regarded as an index measuring the scientific relevance of a paper that should no way be confused with its scientific quality.

### 3.1.2. Affiliations

The affiliation search returned a list of the institutions involved in the environmental analysis of food production. Table 3 shows the number of papers published by these affiliations from 1970 to 2020.

The *Wageningen University* & *Research* (NL) resulted to be the first affiliation in terms of number of publications, followed by the *Chinese Academy of Sciences*, which includes other affiliations separately named in Scopus, such as the *Institute of Geographical Sciences* and *Natural Resources Research Chinese Academy of Sciences*, *University of Chinese*

*Academy of Sciences*. Moreover, other Chinese Institutions ranked at the 3rd, 12th, and 16th places, whereas the *Aarhus Universitet* (DK) together with other five European affiliations is in the Top 10 world ones. The first 20 affiliations altogether represented about 20% of the world publications over the years 1970–2020. Thus, there is a widely distributed interest on this scientific research field. Generally, in some countries, and particularly in the USA, the papers retrieved were produced by numerous institutions. Inversely, in other countries, like Sweden, Denmark, and Iran, such papers were performed by specialized research centers.

### 3.1.3. Authors

A search dedicated to the papers' authors returned the list of the most publishing authors, shown in Table 4, together with the overall number of papers released by such authors in their own country over the time intervals examined. Prof. Adisa Azapagic at the University of Manchester (UK) resulted to be the most publishing author with 31 papers, followed by Prof. María Teresa Moreira (27) at the Universidad de Santiago de Compostela (Spain), and Ian Vázquez-Rowe (25) at the Pontificia Universidad Catolica del Peru (Peru), and Feijoo Gumersindo (24) at the Universidad de Santiago de Compostela (Spain). It can be also noted that one author from Iran and another one from Thailand are included in the first Top 10, indicating a high specialization activity in both countries. The research string used did not allow to number all the papers published by the authors listed in Table 4, being the underestimation of the order of 10–15%, as reported above.

### 3.1.4. Journals

A journal-based search returned the list of the peer-reviewed journals publishing more frequently papers related to the topic of concern. Table 5 shows the number of publications hosted by such journals, their ranking in quartiles (Q), Scimago Scientific Journal Ranking (SJR), and H-index, as extracted from https://www.scimagojr.com/, accessed on 18 October 2021. In particular, the quartile ranking classifies scientific journals on the basis of their impact factor or impact index.




*Sustainability* **2021**, *13*, 11633


**Table 5.** Top 20 world journals ranked by the number of EAFP-related publications for the periods examined (1970–2020).

The journals mainly specialized in the EAFP topic were those appearing in the first positions, namely in descending order the *Journal of Cleaner Production* with 499 papers published in the 1970–2020 interval, *International Journal of Life Cycle Assessment* with 179 papers, and *Science of the Total Environment* with 116 papers. Other journals, not exclusively publishing papers dealing with food production, firstly dealt with agricultural and environmental science aspects, and, then, with their climate ones. The great majority of these journals were first-quartile journals with SJR citation index in the top 25% of journals for at least one of its classified subdisciplines.

According to Table 6 that ranks all the above 20 journals with respect to the average citation per paper index (CPPI), the *International Journal of Life Cycle Assessment* yielded the greatest CPPI (35.8), followed by the *Journal of Cleaner Production* (32.4), *Science of the Total Environment* (23.5), and *Sustainability Switzerland* (7.8), the latter being the only open-access journal.

### 3.1.5. Founding Sponsors

Table 7 reports the data extracted from the Scopus database when accounting for the founding sponsors. Over the 1970–2020 period, the *European Commission* was the first sponsor in terms of publications produced, followed by the *National Natural Science Foundation of China*, and *Ministry of Science and Technology of the People's Republic of China*. Such a wide presence of Chinese papers again shows a certain sensitivity of China towards the environmental sustainability of food production, probably because this nation is currently the first CO2 emitter in the world [25].

**Table 6.** Top 5 journals ranked by the overall number of papers published from 1970–2020 together with the overall number of citations, total citations per paper index (CPPI), and citation per paper index (10%-CPPI) and percentage of citations (10%-CG) generated by the first 10% of published articles in order of citation received.




### *3.2. Textual Analysis*

3.2.1. The Most Cited Food-Related Terms

Table 8 shows the mostly used terms (occurrence) and their relative frequency (%) in the database constructed using the Title of the papers produced worldwide or in the five mostly publishing countries (Table 1). The term *milk* was that most cited worldwide, the same conclusion can be drawn for other terms, such as *rice, maize, fish,* and *wheat.* Such terms were those most used also for the five most publishing countries, but in different order. In China, *rice* resulted to be the first term used, owing of course to the high amount of rice produced and consumed (http://www.fao.org/faostat/en/#data/QC/visualize, accessed on 18 October 2021). Italy and Spain differentiate from the other countries for their characteristic terms were related to their diet and/or production. Both countries had *wine* as the third and fourth term most frequently studied, respectively, followed by *wheat*, and *milk*. The term *olive oil* was at 6th place, while the term *pasta* was present in Italian publications only. It is worth noting that worldwide the term *palm oil* resulted to be more studied than the terms *cattle*, *coffee*, or *cheese*. The term *beer* was only present in Chinese (just less cited than *tea*) and Italian papers at the 13th and 14h place, respectively.

**Table 8.** Ranking of the occurrence of the most food-related terms in the Abstract/Title of the papers indexed in the Scopus database, as referred to the world and fivemost publishing countries.


### 3.2.2. The Most Cited Environmental Impact Categories and Standard Methods

Table 9 shows the environmental impact categories and standard methods most frequently used and their relative frequency, as searched in the Title or Abstract of published papers. *Life Cycle Assessment* (LCA), *Carbon Footprint* (CF), and *Water Footprint* (WF) were the first three standard methods mostly used. However, the LCA procedure does not a priori give any clue about the impact categories effectively considered. In any case, the most used methods relied on just a single environmental issue (i.e., CF including the Publicly Available Specification 2050, WF, *Ecological Footprint*, and *Cumulative Energy Demand*), followed by the standard methods that analyze the environmental impact with mid- (i.e., *Environmental Product Declaration*, *TRACI*) or end- (i.e., *IMPACT 2002+*) point impact categories, and more recently by the *Product Environmental Footprint* (PEF) method, where as many as 16 mid-point categories are normalized with respect to the impacts caused by one person living in the world during one year and weighed excluding or including three toxicity related impact categories (i.e., human toxicity cancer, human toxicity non-cancer and freshwater eco-toxicity) to yield a single weighted score. Further details were summarized by Moresi et al. 2021, [28].

**Table 9.** Occurrence of the environmental impact assessment methods in the abstract of the papers published in the 1970–2020 period.


AWCC—Australian Wine Carbon Calculator; CED—Cumulative Energy Demand; EPD—Environmental Product. Declaration; PEF— Product Environmental Footprint.

### *3.3. Map Analysis*

Figure 3 shows the term map referred to the 1970–2020 period. The 299 terms displayed on the map were grouped in six clusters, which appeared well separated and were identified by different colors.

Among the 116 terms marked in red, the three main terms were *development, consumer, challenge* and *food production*, these cluster terms were clearly linked to issues related to environmental policies, decision-making about the mitigation measures to be undertaken to address and resolve the environmental impacts of food production, and communication of environmental and sustainability aspects to different stakeholders. The term *diet* was closely related to another cluster (marked in yellow), and in particular to *meat.* Such a cluster represents an important sector of study regarding the *livestock* impact on food production, *milk* production and *land use*.

**Figure 3.** The term map referred to the 1970–2020 period.

The green-marked cluster included 55 items, regarding the impact categories mostly cited, the terms *global warming*, *eutrophication,* and *acidification* being the prevailing ones. This cluster is near the blue-marked one, where the biggest spot was represented by the terms related to *fossil fuels*, *oil* and *combustion* probably for transportation and *heat* generation. Clearly, the *fossil fuel* problems were emphasized by other terms, such as *biofuel, biodiesel, ethanol,* and *sugar*, these underlining the use of an edible raw material as energy source.

The viola-marked cluster was strictly related to *agriculture* and raw material production (in particular *wheat* and *rice*). *Nitrogen fertilizer*, *methane*, *CO2* resulted to be linked to the *soil* and *global warming potential* spots. One of the biggest spots included the term *water* with connection to all the other clusters and the smallest cluster regarding *waste*, *food waste*, *composting*, and *anaerobic digestion.*

About 70% of the global water consumed by humans is used in agriculture [29], primarily for irrigation. Thus, in food products coming from irrigated land, irrigation will dominate the water use.

Figures 4 and 5 elucidate the main links (i.e., co-occurrence between terms), as referred to the 1970–2016 and 2017–2020 periods. One of the main topics of the first period concerned several terms, namely *sustainability*, *milk*, *fertilizer*, *global warming*, *eutrophication*, and *land use*. On the contrary, in the following four years the main research topics diverted to *water, diet, consumer* and *sustainable food consumption, food waste,* and *waste disposal*, this including *incineration* and *recycling*, such as *composting* and *anaerobic digestion.*

**Figure 4.** The term map referred to the 1970–2015 period.

**Figure 5.** The term map referred to the 2016–2020 period.

### *3.4. Firstly Indexed and Mostly Cited Papers*

Table S6 in the electronic supplement shows the first 10 published papers concerning the EAFP, these having been indexed by the Scopus database since 1968, as well as those mostly cited ones over the latest 50 years.

The term *environmental impact* firstly appeared in 1974 and was related to the disposal of sulfur removal sludges generated by the slaked lime-wet scrubber process, and use of antibacterial drugs in food animals. A paper published in 2006 explored the consumer attitude towards a more sustainable food consumption and resulted to be the most cited paper in this research sector, gathering more than one thousand citations.

### **4. Discussion of Results**

The present bibliometric review highlighted the presence of some novel aspects regarding the actual EAFP sector. The first and most evident conclusion of this study pointed out that the production of the scientific community in terms of number of publications is still relatively small despite the social and scientific relevance of the environmental impact of food production and consumption in terms especially of food security, air and soil quality, diet and health. In all probability, the limited number of publications retrieved depended on the evolution of this sector that has just a recent origin. On the other hand, the data analysis at the macro level highlighted that two of the most GHG emitting countries, namely China and the USA (http://www.globalcarbonatlas.org/en/CO2-emissions, accessed on 18 October 2021) were the first ones in terms of the number of publications on this topic. Moreover, such nations displayed a long and consistent record of investment in research and development (R&D) (https://www.rdworldonline.com/2021-global-rdfunding-forecast-released/), accessed on 18 October 2021). Nevertheless, if their prolific scientific production in terms of total world scientific publications was related to their population and governmental investment in R&D (see Table S7 in the Supplementary Material), the number of scientific papers per each billion USD invested o per million citizens was just of the order of unity, while it roughly increased by a factor of 10 in the case of the most publishing European countries, such as Italy, the UK, and Spain.

Not by chance, the scientific research in the EAFP field resulted to be mainly carried out in Europe, since 10 out of the first 20 nations and 12 out of the first 20 Institutions/Research Centers, as well as the most publishing authors, were European.

The publications resulted to be concentrated in few journals (e.g., the *Journal of Cleaner Production*, *International Journal of Life Cycle Assessment*, and *Science of the Total Environment*), even if the EAFP topic has interested other magazines not always directly related to this research area as measured by a higher number of citations per paper with respect to the world average one.

The textual analysis showed that a great number of studies just focused on a single environmental issue, mainly the climate impact measured via the carbon footprint or water footprint methodology. This choice was probably encouraged by a wide-ranging public debate on this issue, a relatively important share of the overall GHG emissions by the food sector, and a good availability of data, even if such impact categories are regarded to be insufficient to describe the full range of environmental impacts deriving from the food systems. Less attention was up to now given to other standard methods, such as EPD® and PEF, likely for their relatively recent introduction. Since the characterization of the whole environmental profile of a single food or drink product is costly [30,31], and the climate change impact category is quite more reliable than all the other ones used in the aforementioned standard methods [32], the assessment of the product carbon footprint might be regarded as not only a cheaper tool to identify the major hotspots of the food supply chain, but also a proper method to start improving the sustainability of the 99% of the food and beverage small- and medium-sized enterprises [33]. Moreover, since the collection of primary data involves time, cost, and resource efforts, especially for SMEs, the use of secondary data might be more than sufficient for a preliminary identification of the major hotspots of a process or product, as shown in the production of bread [34] and lager beer [35].

The bibliometric map also showed a scientific interest towards other environmental impacts not exclusively related to fossil energy use or climate impacts, such as those linked to the agricultural production affecting the biodiversity, soil quality, water release of nutrients (mainly nitrogen and phosphorus), and pesticides. For instance, the GHG emissions associated to such a phase are in descending order due to: (1) crop residue decomposition;

(2) inorganic fertilizers applied; (3) manufacture, storage, and transportation of inorganic N and P fertilizers, and pesticides to the farm gate; (4) main farming operations, such as tillage, seeding, pesticide spraying, and crop harvesting; (5) soil carbon gains or losses from various cropping systems; and (6) emissions of N2O from fallow areas destined to the growth of the next year crop [36]. Several crop rotation systems, lower N fertilizer rates, and reduced tillage seemed to be effective to mitigate the carbon footprint of several crops, such as durum wheat [37–39]. Even if organic farming is regarded as low-carbon agriculture [40], its lower productivity with respect to conventional one requires more cultivated land, this greatly enhancing the damage to the ecosystem quality, as observed in the case of organic durum wheat [41].

Figure 3 shows an important correlation between nutrition (diet) and meat and food security. Since the world population is expected to grow from about 7 billion to 9.6 billion people in 2050, as well as the global meat and milk consumption, especially in China and India, the promotion of healthy diets can reduce the environmental footprint of food consumption [42,43], as in the case of the Mediterranean diet, which is not only protective against lifestyle diseases, such as cardio-vascular disease, obesity, type 2 diabetes mellitus and certain cancers, but also responsible for a more favorable impact on the environment [44].

As shown in Figure 3, the utmost studied foods coincided with the most impacting ones, such as meat and milk. In this group also rice and wheat were included, even if they are staple food with a relatively low environmental impact but a high worldwide production and consumption.

Over the latest five years (Figure 5), other terms, such as *water*, *energy demand* and *food waste*, underlines the contribution of the consumer and post-consumer steps of the life cycle of different foods and beverages. As concerning the consumer phase, any mitigation action of its environmental impact would ask for the diffusion of more appropriate cooking systems. In the case of dry pasta, its cooking energy consumption might be significantly reduced by using quite smaller water-to-pasta ratios than the conventional one of 10 L per kg of dry pasta [45–47] or adopting novel home eco-sustainable pasta cookers [48,49]. On the contrary, in the case of coffee the use of ground and roasted coffee instead of coffee pods or capsules would drastically cut the GHGs emitted to produce their packaging materials and dispose of post-consumer packaging wastes [50].

Finally, an increasing number of studies focused not only on the analysis of food waste to measure their environmental impacts and suggested some mitigation options, but also on *water* and *land use*, the latter being especially related to *meat* and *milk* production. Food is lost and wasted along the whole supply chain from farms to processing, retailing and consumption at home and restaurants. Food waste not only involves the loss of valuable and often scarce resources, such as water, soil, and energy, but also contributes to climate change. In the European Union, about 88 million metric tons of food are wasted every year, equivalent to 173 kg per person, 53% and 19% of which being wasted by households and processing, respectively [51]. Per capita food waste generation by households was found to be practically independent of the country income, thus any action on food waste would be equally relevant in high, upper-middle and lower-middle income countries [52]. Food waste should be handled to avoid a negative impact on the environment or human health. According to the waste hierarchy set out at Article 4 of the revised Waste Framework [53], food waste formation should be limited as much as possible using for example less material in design and manufacture. Once formed, its entire apparatus or replacement parts should be refurnished to be re-used, recycled, submitted to other recovery options, and finally disposed of via landfilling or incineration with no energy recovery, as the least preferred option.

From such a map analysis, the food and beverage industry is expected in the near future to bear an ever-increasing responsibility towards consumer and environment and to invest more to prevent the development of more serious and costly adverse effects on food security.

### **5. Conclusions**

The bibliometric data analysis, articulated through various indicators and combined to literature mapping clustering tools, allowed a graphical and numerical assessment of the research panorama on the environmental assessment of food production, as well as the geographic origin of this kind of research and growth or erosion of the scientific impact of specific countries. It represented an objective evaluation of this particular research topic, researchers or research Institutions, and could help researchers to address their research work and select the appropriate journals for their papers. The data highlighted the presence of few specialized journals with relevant citations per paper and H-index, as well as many other multidisciplinary journals with H-Indexes like those of the specialized ones. Over the latest five years this research field was mainly aimed at managing the agricultural practices, mitigating global warming and water use, assuring food security and sustainable food consumption, while minimizing food waste formation.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10 .3390/su132111633/s1, Table S1: List of terms used for the query string: Category 1—Selected Terms of two thematic groups, A—Standard methods and B—Generic Terms, Table S2: List of terms used for the query string: Category 2—Food products, Crop production Terms, Table S3: List of terms used for the query string: Category 2—Food products, category Crop processed, Live Animals, Livestock Primary, Livestock Processed, Table S4: List of terms used for the query string: Category 2—Food products, category Generic, Food and beverage, Table S5: List of terms used for textual analysis of abstract, Table S6: Scientific production in EAFP for the top 5 nations ranked by the number of documents published related to their population and governmental investment in R&D.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data available on request due to restrictions e.g., privacy or ethical.

**Conflicts of Interest:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

### **References**


## *Article* **A Quantitative Modeling and Prediction Method for Sustained Rainfall-PM2.5 Removal Modes on a Micro-Temporal Scale**

**Tingchen Wu 1,2,3,4, Xiao Xie 5,\*, Bing Xue <sup>5</sup> and Tao Liu 1,2,3,4**


**Abstract:** PM2.5 is unanimously considered to be an important indicator of air quality. Sustained rainfall is a kind of typical but complex rainfall process in southern China with an uncertain duration and intervals. During sustained rainfall, the variation of PM2.5 concentrations in hour-level time series is diverse and complex. However, existing analytical methods mainly examine overall removals at the annual/monthly time scale, missing a quantitative analysis mode that applies micro-scale time data to describe the removal phenomenon. In order to further achieve air quality prediction and prevention in the short term, it is necessary to analyze its micro-temporal removal effect for atmospheric environment quality forecasting. This paper proposed a quantitative modeling and prediction method for sustained rainfall-PM2.5 removal modes on a micro-temporal scale. Firstly, a set of quantitative modes for sustained rainfall-PM2.5 removal mode in a micro-temporal scale were constructed. Then, a mode-constrained prediction of the sustained rainfall-PM2.5 removal effect using the factorization machines (FM) was proposed to predict the future sustained rainfall removal effect. Moreover, the historical observation data of Nanjing city at an hourly scale from 2016 to January 2020 were used for mode modeling. Meanwhile, the whole 2020 year observation data were used for the sustained rainfall-PM2.5 removal phenomenon prediction. The experiment shows the reasonableness and effectiveness of the proposed method.

**Keywords:** sustained rainfall; PM2.5 removal mode; micro-temporal scale; quantitative modeling; mode prediction

### **1. Introduction**

Air pollution is currently a major environmental challenge for both developed and developing countries worldwide, with increasing industrialization, growing urbanization and energy consumption posing a serious threat to public health [1,2]. According to a report from the World Health Organization, PM2.5 is unanimously considered to be an important indicator of air quality [3,4]. The rainfall processes are typically regarded as strong drivers to remove PM2.5 so as to improve the air quality.

However, in the existing available research on an observation data analysis, there are obvious inconsistency between rainfall and the PM2.5 removal effect, especially for sustained rainfall, a kind of typical but complex rainfall process in southern China with an uncertain duration and intervals. For example, Kao et al. [5] found that in a rainforest environment, summers with high precipitation are negatively correlated with PM2.5 levels; Preethi et al. [6] proposed that the effect of simulated Indian monsoon rainfall removal

**Citation:** Wu, T.; Xie, X.; Xue, B.; Liu, T. A Quantitative Modeling and Prediction Method for Sustained Rainfall-PM2.5 Removal Modes on a Micro-Temporal Scale. *Sustainability* **2021**, *13*, 11022. https://doi.org/ 10.3390/su131911022

Academic Editor: Constantinos Cartalis

Received: 27 August 2021 Accepted: 1 October 2021 Published: 5 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

depends to a large extent on climatic wind speeds; Neal et al. [7] pointed out that there has been no systematic improvement in air quality in mid Wales for 17 years in the face of increased rainfall. In addition to the complexity of the sustained rainfall itself, the variations in regional environmental conditions are possibly another important reason. Therefore, studying the relationship between sustained rainfall and the PM2.5 removal effect becomes a multi-factor related scientific issue.

When expressing the effect of rainfall on the removal of PM2.5 concentrations in the air, the observed time series is usually the most intuitive expression [8,9]. The observed value will be considered as a sign of the presence of the rainfall process [10]. However, sustained rainfall, which is characterized by two or more sustained rainfall events within a given period of time, is intermittent, slow to change and uncertain in its length of formation, and its complexity needs to be fully considered [11,12]. During sustained rainfall, the variation of PM2.5 concentrations in hour-level time series is diverse and complex. However, existing analytical methods mainly examine overall removals at the annual/monthly time scale and mainly use the correlation analysis (CA) between rainfall amount and PM2.5 concentration based on macroscopic monitoring data, which refers to the rainfall time series and PM2.5 time series sampled on an annual or monthly temporal scale [13,14]. For example, using the data of a macro-temporal scale, Shaibu et al. [15] confirmed that monthly PM2.5 concentrations in the Niger Delta region of Nigeria show a significant positive correlation with monthly rainfall. Similarly, through a study of annual datasets from five air quality monitoring stations in Bahrain from 2006 to 2012, Jassim et al. [16] indicated little correlation between rainfall and PM2.5, leading to a year-on-year increase in PM2.5 concentrations. They are all missing a quantitative analysis mode that applies micro-scale time data to describe the removal phenomenon. In order to further achieve air quality prediction and prevention in the short term, it is necessary to analyze its micro-temporal removal effect for atmospheric environment quality forecasting.

In addition, due to the sustained rainfall and the atmospheric pollution particulate matter itself being complex, the large-scale temporal analysis lacks guidance for specific sustained rainfall-PM2.5 removal processes [17]. On this basis, a part of the study proposes to extract the historical single rainfall process using hourly observation data and adopt a predetermined calculation model to quantify the removal effect of the rainfall process: Chhavi et al. [18] analyzed the wet removal effect by calculating the PM2.5 concentration difference before and after rainfall, including the positive and negative removal; Kapwata et al. [19], based on the intensity of rainfall, delineate the rainfall classes, counting the percentage of positive removal to summarize the removal effect according to the influence factors, such as rainfall duration and rainfall volume, which have certain guiding significance. The above methods do not take into account the changes in the effects produced by complex sustained rainfall processes at different stages, including effects such as hygroscopic growth and the secondary transformation of gaseous pollutants which cause PM2.5 concentrations to rise or rebound [20–22]; at the same time, they lack universal law exploration, having difficulties in serving the scientific prediction and early warning of air quality systems and active prevention [23,24].

In this paper, we propose a quantitative modeling and prediction method for sustained rainfall-PM2.5 removal modes on a micro-temporal scale. The detailed contributions are as follows:


of sustained rainfall at the micro-scale than the traditional micro-scale data analysis methods. During sustained rainfall, the variation of PM2.5 concentrations in an hourly time series is diverse and complex. The analysis of hourly scales reveals new characteristic modes that are different from the traditional large scale. These "declining, rebounding, or rising" modes not only allow the analysis of historical data from different regions, but also allow the prediction of PM2.5 removal at hourly intervals using future hourly rainfall, which can help the relevant systems and departments to make timely decisions on air pollution control.

This paper is organized as follows. The study area and data on analytical framework are viewed in Section 2. Section 3 introduces the quantitative definition of the sustained rainfall-PM2.5 removal mode on a micro-temporal scale, then presents the rainfall-PM2.5 removal phenomenon predicting algorithm based on the quantitative model and Section 4 discusses the experimental results. Finally, the discussions are presented in Section 5.

### **2. Study Area and Data**

### *2.1. Study Area*

The experimental area of this paper is located in Nanjing, which is in the middle of the lower reaches of the Yangtze River and has a subtropical monsoon climate with cold and dry winters, high temperatures and rainy summers and a relatively large rainfall variability, making it one of the regions in China with more droughts and floods [25,26]. Some studies have shown that (i) due to the influence of strong convective weather and global warming, there is a significant increase in the amount and duration of rainfall in Nanjing, which has a different degree of the purifying effect on air pollutants [27]; (ii) at the same time, according to the results of a pollutant source analysis in Nanjing, PM2.5 water-soluble ion concentrations have diurnal and seasonal differences, making the average mass concentration of water-soluble ions higher during the day than at night, and the flushing effect of spring and summer rainfall is generally lower than that of autumn and winter [28,29]; (iii) according to the statistics, the annual average humidity in Nanjing is nearly 80%, and some rainfall in this environment may lead to hygroscopic growth of particulate matter, resulting in higher pollutant concentrations [30].

### *2.2. Dataset*

The experimental data in this paper were obtained from hourly PM2.5 concentrations and meteorological observations released by the China General Environmental Monitoring Station and the National Meteorological Information Centre for the period from January 2016 to December 2020. Based on the geographical location of the monitoring stations in Nanjing and the completeness of the data, the PM2.5 data observed at the CCM (Caochangmen) site (32.0572 ◦N, 118.7486 ◦E) and the rainfall data observed at the PK (Pukou) site (32.177 ◦N, 118.706 ◦E) were used for the experiment.

Figure 1 demonstrates the phenomenon of PM2.5 concentration removal by rainfall observed on a macro-temporal scale (yearly and monthly) to a micro-temporal scale (hourly): from the annual scale, the average annual PM2.5 concentration in Nanjing decreases year by year from 2016 to 2020, while the pollutant concentration is relatively lower in years with a higher average annual rainfall; from the monthly scale, the monthly rainfall shows an overall stable trend of first increasing and then decreasing, and the average monthly PM2.5 concentration in the period of concentrated rainfall are significantly lower than those during periods of low rainfall, indicating that rainfall has a significant effect on the removal of air pollutants; on an hourly scale, the hour-by-hour PM2.5 concentration changes during a single rainfall process, thus rainfall has an obvious effect on pollutant removal while there are situations that cause concentrations to increase, and it is not possible to obtain the mode of the effect of regional rainfall on PM2.5 from a particular rainfall process. In summary, the mechanism of PM2.5 removal by sustained rainfall in Nanjing is complex, and it is difficult to satisfy the study of the mechanism of PM2.5 removal by rainfall with the

macro time statistics and the time series of a single rainfall process, so the micro time-series effect model proposed in this paper is applied for a deeper analysis.

**Figure 1.** Macro/micro-temporal scale of sustained rainfall-PM2.5 observation data series and removal phenomenon in Nanjing from 2016 to 2020: (**a**) Yearly temporal-level; (**b**) Monthly temporal-level; (**c**,**d**) Hourly temporal-level.

### **3. Methods**

*3.1. A Quantitative Modeling for Sustained Rainfall-PM2.5 Removal Mode in Micro-Temporal Scale*

### 3.1.1. Overview

In order to accurately describe the study object and the study boundary from the observation dataset, this section first defines a microscopic time-series fragment model of the sustained rainfall process, on the basis of which the time-series process of the removal effect is structured, further modeling the concomitant factors affecting the role of rainfall in PM2.5 removal, and finally establishes an evaluation mode to quantitatively describe the removal effect.

### 3.1.2. Micro-Temporal Modeling of Sustained Rainfall Process

The model needs to accurately identify a complete sustained rainfall process in micro-scale observations due to the large variation in time duration and the existence of uncertain intervals. The sustained rainfall process is characterized by intermittent multi-fragmentation on hourly observation data series.

Time-series segments (TS) are mathematical frameworks for describing and modeling event sequences in the time domain, and are commonly used to construct interpretable time-series analysis models, such as trend prediction and anomaly detection [31,32]. In this paper, the sustained rainfall time-series segment (SRS) is defined at the micro-temporal scale rainfall: {*Rt* : *t* = *hour*}. SRS visualizes the variation of rainfall in the time dimension of a sustained rainfall process, as shown in Figure 2, and is defined as follows:

$$\mathcal{SRS} = \left\{ \left[ \mathcal{R}\_{t'}, \dots, \mathcal{R}\_{t''} \right] \middle| t', \Delta t, t'' \right\} \tag{1}$$

**Figure 2.** Schematic diagram of a micro-temporal model of the sustained rainfall process.

The quantitative measurement steps are:

**Step 1:** Using the hourly rainfall values as a benchmark, the moment of the first occurrence of 0.1 mm and above rainfall is taken as the starting point *t* of the time-series fragment. **Step 2:** Since rainfall and its resulting effects will remain in space for a certain period of time, a threshold value Δ*t* is set to indicate the intermittent duration of the sustained rainfall process, i.e., the sequence before and after when rainfall is zero does not exceed Δ*t* is regarded as the same time-series fragment.

**Step 3:** Satisfy the above conditions and the last occurrence of rainfall greater than zero is the end point *t* , and obtain a complete *SRS*.

The micro-temporal model of the sustained rainfall process provides a quantitative basis for determining the baseline interval for the clearance effect analysis.

### 3.1.3. Sustained Rainfall Removal Concomitant Factor Modeling

Sustained rainfall processes are often concomitant by changes in the accompanying meteorological factors such as temperature, humidity, wind speed and direction, which will affect the removal effect to varying degrees [33]. Therefore, in this paper, we consider the effect and intensity of the existing influence factors [34–37], and balance the availability of their own observational data and the predictability of future trends to establish a set of accompanying influence factors *F*, including the (i) direct factor (*FD*) to describe rainfall characteristics, and the (ii) indirect factor (*FI*) to describe environmental characteristics. The factors and impact effects are shown in Table 1.

On this basis, the removal effects time-series segment (RES) is defined in conjunction with the PM2.5 time series. The RES refers to the process and subsequent effects of rainfall on PM2.5 at the scale of the mechanism of removal, while avoiding the chance of PM2.5 concentration values before rainfall anomalies by expanding the series values for *t* and *t* forward and backward for *n* hours, respectively (Figure 3):


**Table 1.** Table of qualitative descriptions of concomitant factor.

**Figure 3.** Schematic diagram of a micro-temporal model of the removal effects process and concomitant factor.

The concomitant factor model {*F*, *RES*} for sustained rainfall removal identifies the information dimensions and effect intervals for the quantitative modeling.

3.1.4. Quantitative Evaluation Modeling of Removal Effects

The effect evaluation refers to the evaluation of the change in the effect produced by a geographical process considering the influence of relevant factors, and usually uses quantitative indicators to measure and analyze its dynamic change characteristics in spatial and temporal modes [38,39]. The quantitative evaluation model of removal effects is proposed based on the above-mentioned models (*SRS*, *RES*):

$$\mathcal{M} = \left\{ M\_{i \in \{\text{T,P,D,U,R,L}\}} \Big| \langle RP, RF, RS \rangle\_{\prime} \langle TSR, TSE \rangle \right\} \tag{2}$$

(1) Quantitative Measurement of Removal Effect Indicators

Based on the model (*SRS*, *RES*), the following measurable evaluation indicators (*RP*, *RF*, *RS*) that quantitatively characterize the temporal variability in the removal processes are defined below:


(2) Modeling of Removal Mode based on Effect Indicators

The (*SRS*, *RES*) and (*RP*, *RF*, *RR*) were used to modeling the removal process of sustained rainfall and the indicators were combined with zero boundaries, corresponding to the rising and falling changes in PM2.5 concentrations in Figure 4, and combined with the ensemble theory to classify the effect patterns in Table 2.

**Figure 4.** Schematic diagram of the sustained rainfall-PM2.5 removal modes. (**a**) Totally Removal Mode; (**b**) Partly Removal Mode; (**c**) Delayed Removal Mode; (**d**) Totally Ascent Mode; (**e**) Rebounding Ascend Mode; (**f**) Lasting Ascent Mode.


The removal mode, classified according to the above rules, covers all of the combinations of indicators and corresponds to realistic removal phenomena, providing a category a priori information for predicting future sustained rainfall.

### *3.2. The Mode Predicting of Sustained Rainfall-PM2.5 Removal Effect Using the Quantitative Model*

3.2.1. Overview

In this section, based on the quantitative model of sustained rainfall-PM2.5 removal mode, the future rainfall is further fitted using a multiple regression to fit the removal modes, the core steps are as follows (Figure 5): Step 1, using the time-series window according to the constitute time-series sample of removal modes with high-dimensional characteristics; Step 2, extracting the principal components and component features of the pattern classification features through the principal component analysis method for the concomitant factor intensity; Step 3, establishing factorization classification regression machines for removal modes according to the factor intensity, which can predict the removal phenomenon based on the future meteorological and current air quality information.

**Figure 5.** Flowchart for the prediction of sustained rainfall-PM2.5 removal effect using the quantitative model.

3.2.2. Sustained Rainfall Time-Series Sample Construction Using Sliding Time-Series Window

Firstly, according to the definitions of the *SRS* and *RES* proposed in Section 3.1, a small-scale sliding window is applied to process the hourly rainfall time series *Rt*, hourly PM2.5 time series *Ct* and hourly meteorological observations, setting up to capture all of the sustained rainfall processes in the historical data [40].

Secondly, the statistics of the corresponding direct and indirect factors are calculated within the extracted time-series range (*SRS*, *RES*) according to the concomitant factor model *F* proposed in this paper, as Table 3.


**Table 3.** Table of quantitative calculation of concomitant factor.

The quantitative evaluation indicators (*RP*, *RF*, *RR*) of the extracted sustained rainfall processes were calculated based on the PM2.5 concentration series *Ct* according to the evaluation model of removal effects described in this paper.

$$\begin{cases} \begin{array}{c} RP = \frac{(\mathbb{C}\_1 - \mathbb{C}\_{\text{min}})}{\mathbb{C}\_1} \\ RR = \frac{(\mathbb{C}\_2 - \mathbb{C}\_{\text{min}})}{\mathbb{C}\_2} \\ RF = \frac{(\mathbb{C}\_1 - \mathbb{C}\_2)}{\mathbb{C}\_1} \end{array} \tag{11}$$

$$\mathbf{C}\_2 = \frac{\sum\_{t=t'}^{t''+n} \mathbf{C}\_t}{n} \tag{12}$$

$$\mathbb{C}\_{\min} = \min(\mathbb{C}\_{t'}, \dots, \mathbb{C}\_{t''}) \tag{13}$$

where *C*<sup>1</sup> is the initial PM2.5 concentration during the sustained rainfall process, *C*<sup>2</sup> is the PM2.5 concentration at the end of the sustained rainfall process and *Cmin* is the minimum value for the whole sustained rainfall process.

Finally, sustained rainfall time-series samples <sup>→</sup> *V* were constructed in the form of multi-dimensional feature vectors, as follows:

$$\overrightarrow{\mathbf{V}} = \left( \mathbf{P}\_{\prime} \mathbf{D}\_{\prime} \mathbf{T}\_{\prime} \mathbf{\overline{H}}\_{\prime} \mathbf{W}\_{\prime} \mathbf{C}\_{1\prime} \mathbf{S}\_{\prime} \mathbf{K}\_{\prime} \left[ \mathbf{R} \mathbf{F}\_{\prime} \mathbf{R} \mathbf{S}\_{\prime} \mathbf{R} \mathbf{P}\_{\prime} \right] \right) \tag{14}$$

It provides a computable basis for the accurate extraction of analytical reference intervals from hourly observations.

### 3.2.3. Removal Mode-Constrained Component Analysis of Concomitant Factor

According to the removal mode division rules in Table 2, the <sup>→</sup> *V* were classified to obtain six types of labeled samples corresponding to *Mi*, as shown in the following equation. Due to the different degrees of influence of the removal factors on effect indicators in different effect modes, such as the change of effective removal rate magnitude with increasing rainfall duration in complete removal mode, a multi-factor analysis is needed to explore its change pattern.

$$
\overrightarrow{\mathbf{V}} = \begin{pmatrix} P\_\prime D\_\prime \overbrace{T\_\prime H\_\prime} \,\mathcal{W}\_\prime \,\mathcal{S}\_1 \,\mathcal{S}\_\prime \,\mathcal{K} \, \vert \,\mathcal{M}\_i \end{pmatrix} \tag{15}
$$

The principal component analysis is a statistical method that reduces the information of multi-dimensional variables to a few characteristic components by a linear transformation and reflects as much information of the original variables as possible [41]. In this section, the main components {*F*1, *F*2,..., *Fn*} of the sample features under each mode will be extracted separately, and the weights of each effect factor will be calculated by the

eigenvalues λ of each principal component and the variance contribution ratio *E*(*λ*), and the relative strength *S* of the clearance factors under different modes will be evaluated quantitatively and used to screen the effect factors involved in the construction of the mode classifier.

$$S\_{F\_i} = \frac{A\_{F\_i}}{\sqrt{\lambda\_{F\_i}}} \times \frac{E\left(\lambda\_{F\_i}\right)}{\sum\_{k}^{n} E\left(\lambda\_{F\_k}\right)}\tag{16}$$

*AFi* in the above equation is the loading of the initial factor i on the principal component.

The removal mode-constrained component analysis of concomitant factor enables the extraction of effective and stable effect factors that map mode characteristics, reducing the redundancy and sparsity of the factor information and improving the accuracy of the clearance effect prediction.

### 3.2.4. Removal Mode Prediction Based on Factorization Machines

Factorization machines (FM) are a classification method with a good learning ability for sparse data, solving the sparse information generated by the one-hot coding of category features, while taking into account the two-two correlation features between the features to meet the correlation and between the environmental factors in the rainfall process [42].

We use the factors filtered by the factor strengths calculated in Section 3.2.3, normalized by features (scaling all factors to between −1 and 1) with label coding to form the feature vector - *Xi*, which is used to build the factor decomposer *y*ˆ of the time-series effects model.

$$\begin{cases} \begin{aligned} \hat{y} &= \nu\_0 + \sum\_{i=1}^n w\_i + \overrightarrow{X}\_i + \sum\_{i=1}^{n-1} \sum\_{j=i+1}^n \upsilon\_{i\prime} \overrightarrow{\upsilon\_j X\_i} \hat{X}\_j \\\ w\_{i,j} &= \upsilon\_i \upsilon\_j \end{aligned} \end{cases} \tag{17}$$

Assuming that the sustained rainfall time-series sample has *n* features, *vi* and *vj* in the above equation are the implicit vectors of the feature matrix decomposition, and *wi*,*<sup>j</sup>* is the interrelationship between the two features *i*, *j*.

Further construction of a classifier oriented to the removal mode: the loss function loss Equation (18) is designed using the logistic regression theory, the *y*ˆ is mapped into different classes by the step function sigmoid and the logistic loss is used as the criterion for optimization.

$$\begin{cases} \quad \text{loss}(\mathcal{Y}, \mathcal{Y}) = \sum\_{i=1}^{m} -\ln \sigma(\mathcal{Y}^{(i)}, \mathcal{Y}^{(i)})\\ \qquad \sigma(\mathbf{x}) = \frac{1}{1 - \epsilon^{-\mathbf{x}}} \end{cases} \tag{18}$$

We can use the factor classifier constructed from the mode information to achieve a prediction of the removal effect that will result from a sustained rainfall process based on the rainfall forecast information.

### **4. Experimental and Analysis Section**

This paper uses the historical observation data of Nanjing from January 2016 to January 2020 as *Datatrain* for the model analysis and uses the data from January 2020 to December of the same year *Datatest* to verify the feasibility and effectiveness of the method of this paper.

### *4.1. Construct the Sustained Rainfall Time-Series Sample*

Firstly, all of the datasets were processed in a uniform manner.

A sliding window was designed according to Section 3.2.2 to extract the sustained rainfall process, with all of the rainfall durations greater than 1 h, where the general residence time of particulate matter in the air and a window threshold of Δ*t* was set to 3 h, rainfall with an interruption of no more than 3 h was considered as a complete sustained process (starting moment *t* and ending moment *t* ), with a total of 427 sustained rainfalls collected.

While calculating the removal in each concomitant factors for each process, including the rainfall total, the rainfall duration and initial PM2.5 concentration are shown according to Table 3, with seasonal and diurnal values used for labeling; further combining Equation (3) to calculate the removal effect indicators, and using these to classify the above rainfall samples into six modes according to the removal mode classification rules in Table 2, the result is shown in Table 4.


**Table 4.** A sample table of the sustained rainfall time series in Nanjing.

The findings show 85 times of MT, 191 times of MP, MD 80 times, MA 85 times, MR 12 times and ML 14 times. By plotting the numerical distribution of concomitant factors for different modes (Figure 6), the removal mechanism of sustained rainfall in Nanjing was explored.

**Figure 6.** Distribution of the numerical distribution of concomitant factors for different removal modes. (**a**) Rainfall Total *P* and removal modes; (**b**) Rainfall Duration *D* and removal modes; (**c**) Wind Power *W* and removal modes; (**d**) Temperature *T* and removal modes; (**e**) Humidity *H* and removal modes; (**f**) Initial PM2.5 *C*<sup>1</sup> and removal modes; (**g**) Seasonal factor *S* and removal modes; (**h**) Day and night factor *K* and removal modes.

### *4.2. Principal Component Analysis of the Removal Mode*

Due to the large differences in the values of the effectors, the rainfall samples were first normalized by the eigenvector <sup>→</sup> *V*, and the high-dimensional feature samples of each type of mode were further subjected to the principal component analysis. According to the criteria for selecting the principal components, in this case, when the eigenvalues of the components were all greater than 1 and the cumulative contribution rate was 85%, there were the four types of extracted components. The results are shown in Table 5.



Based on the principal component eigenvalues and variance contribution rates, the factor strengths of different modes were calculated Equation (16) and the strengths of the effectors were ranked, and the total rainfall *P*, rainfall duration *D*, initial PM2.5 concentration *C*<sup>1</sup> and wind speed scale *W* were selected to participate in the construction of the mode classifier (Table 6).



### *4.3. Predict the Removal Mode and Phenomenon*

We combine the above removal concomitant factor strengths with the factorization machines in Section 3.2.4 to build a classifier for predicting the removal mode and phenomenon: the total rainfall *P*, rainfall duration *D*, initial PM2.5 concentration *C*<sup>1</sup> and wind speed scale *<sup>W</sup>* in *Datatrain* form the feature vector - *X* and the classifier *y*ˆ in Equation (17), and the loss function in Equation (18) is combined to train the classifier according to the stochastic gradient descent (SGD) method. Figure 7 shows the effect of the removal modes classification, where *C*1,, *W* and *P*, *D* are the factors that form the modal length of the vector, which is used to present the modes distribution based on the effector features; the classifier designed in this paper can classify the sustained precipitation process into the six types described according to the effectors.

To validate the removal effect classifier, all of the sustained rainfall extracted from January 2020 to December of the same year in Nanjing were classified using the samples in *Datatest*, and the accuracy of the classification results was evaluated by plotting the receiver operating characteristic (ROC) curve and the curve area, area under the curve (AUC), to evaluate the accuracy of the classification results (Figure 8), where the true positive rate (*TPR*) and false positive rate (*FPR*) of the tested samples were calculated as follows:

$$\begin{cases} \begin{array}{c} TPR = \frac{TP}{TP + FN} \\ FPR = \frac{TP + TN}{FP + TN} \end{array} \end{cases} \tag{19}$$

**Figure 7.** FM classifier result graph for removal modes, (**a**) Scatter plot of modes classification results; (**b**) Interpolated rendering of modes classification results.

**Figure 8.** The ROC and AUC of removal modes classifier, (**a**) ROC of Totally Removal Mode; (**b**) ROC of Removal Mode; (**c**) ROC of Delayed Removal Mode; (**d**) ROC of Totally Ascend Mode; (**e**) ROC of Rebounding Ascend Mode; (**f**) ROC of Lasting Ascend Mode.

### **5. Discussion**

In this paper, we mainly analyze the effect of sustained rainfall on PM2.5 removal from the perspective of a microscopic temporal scale with historical observation data at the hourly scale. By combining previous research results, we propose a model analysis framework to quantitatively describe the removal effect from a more refined perspective for combining the mechanism of sustained rainfall effect on PM2.5. The primary conclusions are summarized as follows.

In this paper, we use hourly scale observations for proposing models to quantitatively express sustained rainfall processes with intermittent duration and relative complexity. It is able to provide data boundaries for studying the role of rainfall removal. Moreover, we consider a large number of environmental influences and construct a concomitant factor model *F*, which can improve the accuracy and information dimension of the analysis. Based on the above considerations, we conclusively propose the removal modes for a quantitative description of the removal phenomenon.


The method in this paper is able to classify the proposed model by historical observation data. The results show that of the sustained rainfall processes occurring in Nanjing from 2016 to 2020, only 85 were able to provide complete removal of PM2.5, 63.4% of the precipitation processes resulted in PM2.5 rebound and up to 177 sustained rainfall processes ultimately led to elevated PM2.5 concentrations.

Based on the above quantitative modeling framework, we construct a classifier in combination with the model identification method, considering it for the accurate forecasting of future air quality in short periods. The accuracy evaluation results of the model show that the ROC of our constructed classifier performs well, and the AUC refers to more than 0.85, showing the reasonableness and effectiveness of the method in this paper.

Due to the limited data acquisition, more years of hourly and environmental data for PM2.5 are lacking in this paper. Therefore, it lacks the samples of the lasting ascent mode and the delayed removal mode. Future studies are expected to obtain more hourly temporal observations for the purpose of removal modes construction and acquisition, ultimately to improve the accuracy of the prediction classifiers.

### **6. Conclusions**

Rainfall is an effective way to remove major air pollutants such as PM2.5. However, most studies on the relationship between rainfall data and PM2.5 concentrations have only focused on the changes in the air quality under the influence of long-time span rainfall, ignoring the effects of single rainfall processes that lead to increases or rebound changes in the PM2.5 concentrations, in addition to the effect of wet deposition.

Therefore, based on the definition and generalization of the sustained rainfall process and its effects, this paper uses a time-series statistical method to extract and calculate the single sustained rainfall process and its removal effect factors based on microscopic timescale observation data; in this process, the effect evaluation index is specifically proposed to quantitatively describe the degree of the removal effect, so as to establish a time-series effect model of PM2.5 concentration removal by rainfall. The potential, deep-seated effect of rainfall processes on PM2.5 concentrations is explored. The model is further combined with pattern recognition theory to design an effect pattern classifier for the sample characteristics of the rainfall process, and finally realize the micro-temporal prediction of air quality after a single rainfall. Using the hourly observation data of Nanjing from 2016 to 2020, a total of 427 sustained rainfall processes were collected using this micro-temporal time-series effects model, and the rate of process, rate of rebound and rate of final were calculated and classified into six types of modes: 85 totally removal mode, 191 partly removal mode, 14 delayed removal mode, 85 rebounding sscend mode and 12 lasting ascent mode. The classifier was constructed based on the factors, indicators and model categories, and the ROC evaluation index showed that the classifier has good performance and is capable of quantitatively predicting future PM2.5 concentration decreases, increases and rebound effects using easily accessible rainfall and PM2.5 concentration forecast information, with a view to providing decision-making information for future regional ambient air quality forecasting and refined control. For the acquisition of hour-by-hour PM2.5 concentration forecasts, further investigation of the finer variation characteristics within rainfall periods is required on the basis of this study.

**Author Contributions:** Conceptualization, X.X.; methodology, X.X. and T.W.; software, T.W.; validation, T.W. and X.X.; formal analysis, X.X. and T.W.; investigation, T.W.; resources, X.X.; data curation, B.X.; writing—original draft preparation, T.W. and X.X.; writing—review and editing, X.X. and T.W.; visualization, T.W.; supervision, X.X. and T.L.; project administration, B.X.; funding acquisition, B.X. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by National Natural Science Foundation of China (41971166) and LZJTU EP (201806).

**Institutional Review Board Statement:** Ethical review and approval were waived because necessary permissions were obtained from the local governments to which the schools are affiliated for this study. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

**Informed Consent Statement:** Studies not involving human (Not applicable).

**Data Availability Statement:** Data available on request due to ethical restrictions. The data presented in this study are available on request from the corresponding author. The data are not publicly available due to not having participants consent to share their anonymized data with a third party.

**Acknowledgments:** We express our sincere appreciation to Qing Zhu from the Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University and the anonymous reviewer for their constructive comments.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

### **References**


### *Article* **PM2.5 Concentration Prediction Based on Spatiotemporal Feature Selection Using XGBoost-MSCNN-GA-LSTM**

**Hongbin Dai , Guangqiu Huang \*, Huibin Zeng and Fan Yang**

School of Management, Xi'an University of Architecture and Technology, Xi'an 710055, China; daihongbin@xauat.edu.cn (H.D.); zenghuibin@xauat.edu.cn (H.Z.); yangfan@xauat.edu.cn (F.Y.) **\*** Correspondence: gqhuang@xauat.edu.cn; Tel.: +86-152-7710-7077

**Abstract:** With the rapid development of China's industrialization, air pollution is becoming more and more serious. Predicting air quality is essential for identifying further preventive measures to avoid negative impacts. The existing prediction of atmospheric pollutant concentration ignores the problem of feature redundancy and spatio-temporal characteristics; the accuracy of the model is not high, the mobility of it is not strong. Therefore, firstly, extreme gradient lifting (XGBoost) is applied to extract features from PM2.5, then one-dimensional multi-scale convolution kernel (MSCNN) is used to extract local temporal and spatial feature relations from air quality data, and linear splicing and fusion is carried out to obtain the spatio-temporal feature relationship of multifeatures. Finally, XGBoost and MSCNN combine the advantages of LSTM in dealing with time series. Genetic algorithm (GA) is applied to optimize the parameter set of long-term and short-term memory network (LSTM) network. The spatio-temporal relationship of multi-features is input into LSTM network, and then the long-term feature dependence of multi-feature selection is output to predict PM2.5 concentration. A XGBoost-MSCGL of PM2.5 concentration prediction model based on spatio-temporal feature selection is established. The data set comes from the hourly concentration data of six kinds of atmospheric pollutants and meteorological data in Fen-Wei Plain in 2020. To verify the effectiveness of the model, the XGBoost-MSCGL model is compared with the benchmark models such as multilayer perceptron (MLP), CNN, LSTM, XGBoost, CNN-LSTM with before and after using XGBoost feature selection. According to the forecast results of 12 cities, compared with the single model, the root mean square error (RMSE) decreased by about 39.07%, the average MAE decreased by about 42.18%, the average MAE decreased by about 49.33%, but R2 increased by 23.7%. Compared with the model after feature selection, the root mean square error (RMSE) decreased by an average of about 15%. On average, the MAPE decreased by 16%, the MAE decreased by 21%, and R2 increased by 2.6%. The experimental results show that the XGBoost-MSCGL prediction model offer a more comprehensive understanding, runs deeper levels, guarantees a higher prediction accuracy, and ensures a better generalization ability in the prediction of PM2.5 concentration.

**Keywords:** XGBoost; MSCNN; genetic algorithm; LSTM; feature selection; spatiotemporal feature extraction

### **1. Introduction**

With the increasing of environmental pollution, the weather issue of haze is spreading in China's major cities. PM2.5 has become a major problem of air pollution. Recent studies have shown that PM2.5 leads to the occurrence of respiratory diseases, immune diseases, cardiovascular and cerebrovascular diseases and tumors [1,2]. Accurate prediction and early warnings of the concentration of PM2.5 are of great significance. Many scholars have begun to integrate multiple data features, but too many data and factor features will affect the prediction effect, and redundant features will affect the performance of model prediction. Therefore, many scholars have begun to use feature selection to make predictions. For example: In power system, cooperative search algorithm is used to select

H.; Yang, F. PM2.5 Concentration Prediction Based on Spatiotemporal Feature Selection Using XGBoost-MSCNN-GA-LSTM. *Sustainability* **2021**, *13*, 12071. https://doi.org/10.3390/su132112071

**Citation:** Dai, H.; Huang, G.; Zeng,

Academic Editors: Jun Yang, Ayyoob Sharifi, Baojie He and Chi Feng

Received: 18 September 2021 Accepted: 28 October 2021 Published: 1 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

power load features [3], and minimum redundancy and maximum correlation are used to obtain the best feature set of power load [4]; In wind energy, the multi-agent feature selection method is used to establish the wind speed prediction model [5]; In the stock market, using random forest combined with depth generation model is used to predict the daily stock trend [6]; In tourism, random forest is used for feature selection to predict the number of visitors [7]; In agriculture, model feature (MF) and principal component analysis (PCA) is combined with regression algorithm to predict the water content of rice canopy [8]; In the economy, the genetic algorithm-based feature selection (GAFS) method combined with random forest is used to estimate the per capita medical expenses [9]; In the aspect of transportation, XGBoost (extreme gradient enhancement) screening feature combined with long-term and short-term memory network is used to predict airport passenger flow [10].

Aiming at air quality prediction, the main models used in the existing research include linear regression model [11], grey model [12], geographical weighted regression model [13], mixed effect model [14] and generalized weighted mixed model [15]. In essence, these statistical models are still linear, although the complex relationship between PM2.5 and other factors is simplified in the model, the prediction result of PM2.5 concentration still remain uncertain. With the development of computer technology, machine learning (including deep learning) methods are increasingly used in PM2.5 concentration estimation due to their strong nonlinear modeling ability, such as support vector regression (SVR) [16], k-nearest neighbor (KNN) [17], random forest (RF) [18], multilayer neural network (MLP) [19], artificial neural network (ANN) [20], long-term memory network (LSTM) [21], convolution neural network (CNN) [22], and chemical transport model (CTM) [23]. These models all show better performance than traditional statistical models in predicting PM2.5 concentration, and have stronger nonlinear expression capabilities.

In order to better predict air quality, many scholars have also begun to apply feature selection to air pollutants. Jin et al. [24] proposed a hybrid deep learning prediction that decomposes PM2.5 data through empirical mode decomposition (EMD) and Convolutional Neural Network (CNN) so that an air pollution prediction model can be established. Masmoudi et al. [25] combined the multi-objective regression method with random forest to perform feature selection and predict the concentration of multiple air pollutants. Mehdi et al. [26] studied the impact of feature importance on PM2.5 prediction in Teheran urban area, and used random forest, XGBoost and deep learning technology, of which XGBoost was used to obtain the best model. Zhang et al. [27] used XGBoost model to screen out the most critical characteristics and predict the PM2.5 pollutant concentration in Beijing in the next 24 h. Ma et al. [28] used XGBoost and grid importance to predict PM2.5 in the Northwestern United States. Zhai et al. [29] used XGBoost for feature screening and predicted the daily average concentration of PM2.5 in Beijing area of GA-MLP. Gui et al. [30] used XGBoost model to build a virtual ground-based PM2.5 observation network at 1180 meteorological stations in China, as a result, he found that XGBoost model has strong robustness and accuracy for PM2.5 prediction.

At present, some researchers use deep learning method to estimate the spatial and temporal distribution of PM2.5 concentration. Although the traditional prediction model adds multivariate features, it ignores the impact of redundant features on the prediction results, resulting in the impact of features with little correlation and importance on the prediction results. The scale of relevant models is still relatively small, and it still relies on artificial feature selection to a large extent, and does not make full use of deep learning method to give full play to the advantages of deep learning through deeper and wider network structure. In the related research on the prediction of PM2.5 using feature selection, the prediction is mainly based on a single model, not the perspective of spatio-temporal features, and the importance of feature selection is too emphasized in the related research on the application of feature selection for prediction. The problem of insufficient precision still remains unsolved. The single or combined PM2.5 concentration prediction model does not show strong robustness, and the degree of model optimization is not high. Existing

researches are limited to cities in specific regions, ignoring the predictive performance of the model itself, resulting in poor applicability and migration of the model used.

The main contributions of this paper are as follows:


### **2. Study Area and Data**

### *2.1. Study Area*

Fenwei plain is the general name of Fenhe plain, Weihe plain and its surrounding terraces in the Yellow River Basin. It ranges from the north, Yangqu County in Shanxi Province to the south, Qinlin Moutains in Shaanxi Province, and to the west, Baoji City in Shaanxi Province. It is distributed in Northeast southwest direction, about 760 km long and 40–100 km wide. It has a population of 55,5445., including Xi'an, Baoji, Xianyang, Weinan and Tongchuan in Shaanxi Province, Taiyuan, Jinzhong, Lvliang, Linfen and Yuncheng in Shanxi Province, and Luoyang and Sanmenxia in Henan Province. Since 2019, Fenwei plain is still the area with the highest PM2.5 concentration in China. The average PM2.5 concentration in autumn and winter is about twice as much as other seasons, and the days of heavy pollution account for more than 95% of the whole year [31]. In 2020, the average concentration of PM2.5 in Fenwei plain was 70 μg/m3, and serious pollution occurred in 152 days. Table 1 shows the factors of air pollutants [32].

**Table 1.** Air Pollutant Factors of PM2.5 Concentration Prediction Model.


### *2.2. Study Data*

### 2.2.1. Air Quality Data

Since December 2013, the China Environmental Protection Agency (EPA) has published open air quality observation data from China's ground monitoring stations. The study data in this article comes from the atmospheric pollutants of 12 cities in Xi'an, Baoji, Xianyang, Weinan, Tongchuan, Taiyuan, Jinzhong, Luliang, Linfen, Yuncheng, Luoyang, and Sanmenxia from 1 January 2020 to 31 December 2020 (PM2.5, PM10, NO2, SO2, O3, CO) hourly concentration data set, Table 1 is the atmospheric pollutant factors of PM2.5 concentration prediction model. There are 2,838,240 pieces of air quality data and meteorological data in 12 cities.

### 2.2.2. Meteorological Data

The meteorological data of this paper come from the Chinese weather website platform. As shown in Table 2, through data preprocessing, 21 types of meteorological factors are selected in this paper, and they are average surface temperature, maximum surface temperature, minimum surface temperature, daily average wind speed, daily maximum wind speed, daily maximum wind direction, maximum wind speed, maximum wind direction, daily precipitation of maximum wind speed, 20–8 h (mm) precipitation, 8–20 h (mm) precipitation, 20–20 h (mm) precipitation, average temperature, maximum temperature, minimum temperature, daily average pressure, daily maximum pressure, daily minimum pressure, sunshine hours, daily average relative humidity, daily minimum relative humidity, and season.


**Table 2.** Meteorological Factors of PM2.5 Concentration Prediction Model.

### *2.3. Data Processing*

### 2.3.1. Division of Data Set

The data set needs to be divided before it can be input to the model for training. Otherwise, the prediction model will have no additional data for effect evaluation, and the training results may be overfitted due to training on all data. In the experiment, each data set is divided into training set and test set, after that, the training set is divided into training set and verification set. The data ratio of training set, test set, and verification set is 6:2:2. The training set mainly learns the sample data set and establishes a classifier by matching some parameters. A classification method is established, which is mainly used to train the model. The verification set is used to determine the network structure or the parameters controlling the complexity of the model, and select the number of hidden units in the neural network. The test set is used to test the performance of the finally selected optimal model. It mainly tests the resolution of the trained model (recognition rate, etc.).

### 2.3.2. Raw Data Processing

### Identification and Processing of Abnormal Data

Abnormal data may be caused by errors in the process of collecting and recording data. Abnormal data will affect the prediction accuracy of the model, so it is necessary to identify and process the abnormal data. Outlier detection is used to find outliers. Here, quartile analysis is used to identify outliers. First, the first quartile and the third quartile of variables are solved. If there is a value less than the first quartile or greater than the third quartile, the value is determined as an outlier. The horizontal processing method is used to correct the abnormal data.

The calculation formula of horizontal treatment method is shown in Equations (1) and (2) If,

$$\begin{cases} \left| y\_i - y\_{i-1} \right| < \varepsilon\_d \\ \left| y\_i - y\_{i+1} \right| > \varepsilon\_d \end{cases} \tag{1}$$

Then,

$$y\_t = \frac{y\_{t+1} + y\_{t-1}}{2} \tag{2}$$

Among them, *yi* represents the concentration of air pollutants in a certain day or hour, *yi*−<sup>1</sup> represents the concentration of air pollutants in the previous day or hour, and *yi*+<sup>1</sup> represents the concentration of air pollutants in the next day or hour, *ε<sup>a</sup>* represents the threshold.

### Data Normalization

Due to the different meanings and dimensions of physical quantities such as air pressure and evaporation, the input to the prediction model will have an impact on results., so it is necessary to normalize such data. The input of normalized data into the prediction model can effectively reduce the training time of the model, accelerate the convergence speed of the model, and further improve the prediction accuracy of the model. The normalized calculation formula of the data is shown in Equation (3). This method realizes the equal scaling of the original data [33]:

$$\mathbf{x}\_{norm} = \frac{\mathbf{x} - \mathbf{x}\_{\min}}{\mathbf{x}\_{\max} - \mathbf{x}\_{\min}} \tag{3}$$

Among them, *xnorm* is the normalized value, *x* is the original data, *x*min is the minimum value in the original data, *x*max is the maximum value in the original data, and the size of the normalized data is constrained between 0 to 1 interval.

### **3. Method**

### *3.1. XGBoost*

XGBoost is an extreme gradient boosting decision tree, which belongs to a machine learning algorithm. The algorithm introduces regular items during the generation period and prunes at the same time, making the algorithm more efficient and more accurate. [34].

XGBoost (eXtreme Gradient Boosting) can be expressed in a form of addition, as shown in Equation (4):

$$\mathcal{G}\_i = \sum\_{k=1}^{K} f\_k(\mathbf{x}\_i)\_\prime f\_k \in F \tag{4}$$

Among them, *y*ˆ*<sup>i</sup>* represents the predicted value of the model; *K* represents the number of decision trees, *fk* represents the *k* sub-models, *xi* represents the *i*-th input sample; *F* represents the set of all decision trees. The objective function of XGBoost consists of two parts: a loss function and a regular term, as shown in Equations (5) and (6):

$$L(\boldsymbol{\varrho})^t = \sum\_{i=1}^n l(\boldsymbol{y}\_{i\prime}\boldsymbol{\mathfrak{H}}\_i^{(t-1)} + f\_t(\boldsymbol{x}\_i)) + \Omega(f\_k) \tag{5}$$

$$
\Omega(f\_k) = \gamma T + \frac{1}{2}\lambda \|\omega\|^2 \tag{6}
$$

Among them, *L*(*ϕ*) *<sup>t</sup>* represents the objective function of the *t*th iteration, *y*ˆ (*t*−1) *<sup>i</sup>* represents the predicted value of the (*t* − 1) iteration; Ω(*fk*) represents the regular term of the model of the *t*th iteration, which plays a role in reducing overfitting; *γ* and λ represent the regular term Coefficient to prevent the decision tree from being too complicated; *T* represents the number of leaf nodes of the model.

Using Taylor's formula to expand the objective function shown in Equation (7), we can get:

$$\begin{split} L(\boldsymbol{\Phi}) & \stackrel{\scriptstyle \mathcal{D}}{=} \sum\_{i=1}^{n} \left[ \mathcal{g}\_{i} f\_{t}(\mathbf{x}\_{i}) + \frac{1}{2} h\_{i} f\_{t}^{2}(\mathbf{x}\_{i}) \right] + \gamma T + \frac{1}{2} \lambda \sum\_{j=1}^{T} \omega\_{j}^{2} \\ & \stackrel{\scriptstyle \mathcal{D}}{=} \sum\_{j=1}^{T} \left[ (\sum\_{i \in I\_{j}} \mathcal{g}\_{i}) \omega\_{j} + \frac{1}{2} (\sum\_{i \in I\_{j}} h\_{i} + \lambda) \omega\_{j}^{2} \right] + \gamma T \end{split} \tag{7}$$

Among them, *gi* represents the first derivative of sample *xi*; *hi* represents the second derivative of sample *xi*; *ω<sup>j</sup>* represents the output value of the *j*-th leaf node, and *Ij* represents the sample subset of the value of the *j*-th leaf node.

It can be seen from Equation (7) that the objective function is a convex function. Taking the derivative of *ω<sup>j</sup>* and making the derivative function equal to zero, the objective function can reach the minimum value of *ωj*, as shown in Equation (8):

$$
\omega\_j^\* = -\frac{\sum\_{i \in I\_j} \mathcal{S}\_i}{\sum\_{i \in I\_j} h\_i + \lambda} \tag{8}
$$

Equation (9) can be used to evaluate the quality of a tree model. The smaller the value, the better the tree model. It can be easily concluded that we can obtain the scoring formula for the tree to split the node:

$$L(\boldsymbol{\phi})\_{\text{min}} = -\frac{1}{2} \sum\_{j=1}^{T} \frac{\left(\sum\_{i \in I\_j} g\_i\right)^2}{\sum\_{i \in I\_j} h\_i + \lambda} + \gamma T \tag{9}$$

Equation (10) is used to calculate the split node of the tree model.

$$Gain = -\frac{1}{2} \frac{\left(\sum\_{i \in I\_j} \mathfrak{g}\_i\right)^2}{\sum\_{i \in I\_j} h\_i + \lambda} + \frac{\left(\sum\_{i \in I\_R} \mathfrak{g}\_i\right)^2}{\sum\_{i \in I\_R} h\_i + \lambda} + \frac{\left(\sum\_{i \in I} \mathfrak{g}\_i\right)^2}{\sum\_{i \in I} h\_i + \lambda} \big] - \gamma \tag{10}$$

### *3.2. One-Dimensional Multi-Scale Convolution Kernel (MSCNN)*

Convolutional neural network has been successfully applied to image recognition direction, which verifies that the network has a strong extraction of feature map. Based on the analysis of the data set, it is found that the characteristics of the data are multi features, shown in the form of numerical value, rather than in the form of feature map. Therefore, this study preprocesses the data, combines the characteristics of the data into a feature map, and inputs it to the convolution neural network to complete the extraction of the spatial and temporal characteristics of the air pollutant concentration data and meteorological factors [35]. The spatiotemporal feature extraction of single factor PM2.5 is shown in Figure 1. Among them, the feature map is traversed from left to right on the data feature axis through a one-dimensional multi-scale convolution kernel to complete the convolution operation, the number of steps is 1, and the feature vectors output by different convolution kernels are spliced and fused to obtain a single factor. The spatial characteristics of the relationship. On the time axis, as the convolution kernel traverses from top to bottom to complete the convolution operation, the number of steps is 1, and the local trend of the single factor changing over time can be obtained. Finally, the spliced and fused feature vectors are merged in the data feature direction, and the spatio-temporal features of multi-site PM2.5 are output.

**Figure 1.** One-dimensional convolution feature extraction process diagram.

The following is the formula derivation of MSCNN's convolution operation on the special whole. The feature map contains N sample data and M air pollutant factors. Then the feature map formula of single factor *i* is as shown in Equations (11) and (12):

$$X\_i = \begin{bmatrix} x\_i^1, x\_i^2, x\_i^3, \dots, x\_i^N \end{bmatrix}^T \tag{11}$$

$$\mathbf{x}\_{i}^{t:t+T-1} = \begin{bmatrix} \mathbf{x}\_{i}^{t}, \mathbf{x}\_{i}^{t+1}, \mathbf{x}\_{i}^{t+2}, \dots, \mathbf{x}\_{i}^{t+T-1} \end{bmatrix}^{T} \tag{12}$$

In the formula, *X<sup>t</sup> <sup>i</sup>* = [*x<sup>t</sup> i* , *xt*+<sup>1</sup> *<sup>i</sup>* , *<sup>x</sup>t*+<sup>2</sup> *<sup>i</sup>* , ... , *<sup>x</sup>t*+*T*−<sup>1</sup> *<sup>i</sup>* ] ∈ *R* represents the vector of the single factor *i* at time *t*, *Xt*:*t*+*T*−<sup>1</sup> *<sup>i</sup>* represents the *T* group vector of *Xi* in the [*t*,*t* + *T* − 1] time zone, and *T* represents the matrix transpose.

The convolution operation multiplies the weight matrix *Wj* by *Xt*:*t*+*T*−<sup>1</sup> *<sup>i</sup>* .


When the first convolution kernel traverses the entire feature map on the time axis, and the number of steps is 1, the feature vector *a j <sup>i</sup>* is obtained, and its size is *N* − *T* + 1, and the eigenvectors obtained by multiple convolution kernels *Z* merge [*N* − *T* + 1] × *Z* size *Ai* in the data feature direction, and *Ai* represents the single-factor spatiotemporal feature matrix, as shown in Equations (13) and (14).

$$a\_i^j = \begin{bmatrix} a\_{t+T-1,j'}^j a\_{t+T,j'}^j a\_{t+T+1,j'}^j \dots \dots a\_N^j \end{bmatrix} \tag{13}$$

$$A\_i = \begin{bmatrix} a\_{n'}^1, a\_{n'}^2, a\_{n'}^3, \dots, a\_n^Z \end{bmatrix} \tag{14}$$

So far, the single-factor spatiotemporal feature extraction has been completed, but the data set also contains other features, such as NO2, SO2, CO, etc. A total of M factors, so we can extract the M factors through the same operation as above, and then they can be extracted. Single-feature spatio-temporal feature matrix, and then linearly splicing and fusion them to form a multi-factor fusion spatio-temporal feature matrix *A*, as shown in Equation (15):

$$A = [A\_1, A\_2, A\_3, \dots, A\_M] \tag{15}$$

Based on MSCNN convolution neural network, the space-time characteristics of air quality data are extracted. This method makes a simple transformation of the twodimensional feature map to form a side-by-side one-dimensional feature map, which makes the network training show better generalization ability. Meanwhile, the convolution neural network automatic feature extraction method replaces the traditional artificial feature selection method, which makes the feature extraction more comprehensive and deeper.

### *3.3. Genetic Algorithm*

The genetic algorithm is a method to perform crossover and mutation operations on feasible solutions in the population, so the objective function of the genetic algorithm does not require derivable or continuous conditions. The genetic algorithm applies a probabilistic optimization method to automatically obtain and guide the optimized search space, and adaptively adjust the search direction. The genetic algorithm is simple, universal, and suitable for parallel processing. The specific steps of the algorithm are shown in Figure 2.

**Figure 2.** GA algorithm flow.

The GA process can be divided into six stages: initialization, fitness calculation, checking termination conditions, crossover, selection, and mutation. In the initialization phase, a chromosome is selected arbitrarily in the search space, and then the fitness of the chromosome is determined according to the preset fitness function. For optimization algorithms such as GA, the fitness function is a key factor that affects the performance of the model. Chromosomes are randomly selected based on the fitness of the fitness function. Dominant chromosomes have a higher chance of being inherited to the next generation. The selected dominant chromosomes can produce offspring through the exchange of similar segments and changes in gene combinations.

### *3.4. LSTM*

Long Short-Term Memory (LSTM) is an improvement of Recurrent Neural Network (RNN) [36]. RNN has a higher probability of gradient disappearance and gradient explosion during training, and there is a long-term dependence problem. LSTM can effectively solve this problem. LSTM introduces a gate mechanism, which makes LSTM have a longerterm memory than RNN and can be more effective in learning. In LSTM, each neuron is equivalent to a memory cell (cell, *ct*). LSTM controls the state of the memory cell through a "gate" mechanism, thereby increasing or deleting the information in it. The structure of LSTM is shown in Figure 3.

**Figure 3.** LSTM Unit Structure.

In the LSTM cell structure, the Input Gate (*it*) is used to determine what information is added to the cell, and the Forget Gate (*ft*) is used to determine what information is deleted from the cell. The Output Gate (*ot*) is used to determine what information is output from the cell. The complete training process of LSTM is that at each time t, the three gates receive the input vector *xt* at time *t* and the hidden state *ht*−<sup>1</sup> of the LSTM at time *t* − 1 and the information of the memory unit *ct* and then perform the received information Logical operation, the logical activation function σ decides whether to activate *it*, and then synthesize the processing result of the input gate and the processing result of the forgetting gate to generate a new memory unit *ct*, and finally obtain the final output result *ht* through the nonlinear operation of the output gate. The calculation formula for each process as shown in Equations (16)–(20).

Input Gate calculation formula:

$$\dot{\mathbf{u}}\_t = \sigma(\mathbf{W}\_{\text{xi}}^T \mathbf{x}\_t + \mathbf{W}\_{\text{hi}}^T \mathbf{h}\_{t-1} + \mathbf{b}\_i) \tag{16}$$

Forget Gate calculation formula:

$$f\_t = \sigma(\mathcal{W}\_{xf}^T \mathbf{x}\_t + \mathcal{W}\_{hf}^T \mathbf{h}\_{t-1} + \mathbf{b}\_f) \tag{17}$$

output gate calculation formula:

$$\rho\_t = \sigma(\mathcal{W}\_{\text{xo}}^T \mathbf{x}\_t + \mathcal{W}\_{\text{ho}}^T \mathbf{h}\_{t-1} + \mathbf{b}\_o) \tag{18}$$

Memory unit calculation formula, the internal hidden state:

$$\mathbf{c}\_{t} = f\_{t} \times \mathbf{c}\_{t-1} + i\_{t} \times \tanh(\mathbf{W}\_{\mathbf{x}c}^{T}\mathbf{x}\_{t} + \mathbf{W}\_{hc}^{T}h\_{t-1} + b\_{c}) \tag{19}$$

Hidden state calculation formula:

$$h\_t = o\_t \tanh(c\_t) \tag{20}$$

Among them, σ represents generally a nonlinear activation function, such as a sigmoid or tanh function. *Wxi*, *Wx f* , *Wxo*, *Wxc* represents the weight matrices of nodes connected to the input vector *Wt* for each layer, *Whi*, *Wh f* , *Who*, *Whc* represents the weight matrices connected to the previous short-term state *ht*−<sup>1</sup> for each layer, *bi*, *bf* , *bo*, *bc* represents the offset terms of each layer node. In short, the input gate in LSTM can identify important inputs, and the forget gate can reasonably retain important information and extract it when needed. Therefore, this feature of LSTM can effectively identify long-term patterns such as time series, making training convergence faster.

### *3.5. XGBoost-MSCGL Model*

Figure 4 shows the XGBoost-MSCGL process. First, the atmospheric pollutant data and meteorological data are normalized and processed with missing values. Secondly, Pearson analyzes the correlation of the original data and uses XGBoost to select the importance of features. Furthermore, input the data after feature selection into MSCNN, and use the MSCNN algorithm to extract the temporal and spatial features of the data. At the same time, GA is used to optimize the parameters of the LSTM, the best fitness output of the chromosome is used as the global optimal parameter combination of the LSTM network, and then the data extracted from the spatiotemporal features are input into the optimized LSTM for prediction. In order to better verify the effect of the model, finally combined models such as XGBoost-MLP, XGBoost-LSTM, XGBoost-CNN are used for comparison, and then RMSE, MAE, MAPE and other indicators are used for evaluation.

**Figure 4.** XGBoost-MSCGL Model Process.

### *3.6. Evaluation Index*

In order to measure the accuracy of the prediction model, this paper uses Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) as evaluation indicators. The formulas are shown in Equations (21)–(23).

$$\text{RMSE} = \sqrt{\frac{1}{N} \sum\_{i=1}^{N} \left( X\_i - \overline{X} \right)^2} \tag{21}$$

$$MAE = \frac{1}{N} \sum\_{i=1}^{N} |X\_i - \overline{X}|^2 \tag{22}$$

$$MAPE = \frac{100}{N} \sum\_{i=1}^{N} \left| \frac{X\_i - \overline{X}\_i}{X\_i} \right| \tag{23}$$

$$R\_2 = 1 - \frac{\sum\_{i} \left(\mathcal{Y}\_i - y\_i\right)^2}{\sum\_{i} \left(\overline{y} - y\_i\right)^2} \tag{24}$$

Where *y*ˆ represents the predicted value, *yi* is the true value, and *N* is the number of test samples. The ranges of RMSE, MAE, and MAPE are all [0, +∞). Generally, the larger the value of RMSE and MAE, the greater the error and the lower the prediction accuracy of the model. MAPE is the most intuitive prediction accuracy criterion. When MAPE tends to 0%, it means the model is perfect, when MAPE tends to 100%, it means that the model is inferior. Generally, it can be considered that the prediction accuracy is higher when the MAPE is less than 10% [37]. R<sup>2</sup> measures the applicability of the model to sample values and can test the prediction ability of the model. The closer to 1, the higher the fitness of the model, and the closer to 0, the lower the fitness of the model.

### **4. Results**

### *4.1. Analysis of Factor Characteristics*

In order to better analyze the characteristics of the model input factors, the Pearson correlation method is used for analysis. As shown in Figure 5, the factors for the correlation coefficient of PM2.5 in Yuncheng are PM10 (0.9) and CO (0.8), which are highly positively correlated. Further, SO2 (0.5), average humidity (0.5), and seasons (0.5) are moderately positively correlated, and the average surface temperature (−0.5), the highest surface temperature (−0.5), the duration of sunshine (−0.5), the average temperature (−0.5), the lowest temperature (−0.5), and the highest temperature (−0.5) have a moderately negative correlation. The factors of the correlation coefficient of Xianyang PM2.5 are that CO (0.9) is highly positively correlated. Further, PM10 (0.6), the lowest humidity (0.5), season (0.6), etc. are moderately correlated, the average surface temperature (−0.5), the surface, the lowest temperature (−0.5), the highest surface temperature (−0.5), the average temperature (−0.5), the lowest temperature (−0.5), and the highest temperature (−0.5) have a moderately negative correlation. The correlation coefficients of PM2.5 in Xi'an are PM10 (0.8), CO (0.9) and SO2 (0.7), which are highly positively correlated, and season (0.5) is moderately correlated. The average surface temperature (−0.6), minimum surface temperature (−0.6), extremely high wind speed (−0.5), average temperature (−0.6), minimum temperature (−0.5) and maximum temperature (−0.6) are moderately negatively correlated. The correlation coefficient of PM2.5 in Weinan is that PM10 (0.8) and CO (0.8) are highly positively correlated. average humidity (0.5), minimum humidity (0.5) and season (0.5) are moderately correlated, sunshine duration (−0.6) is highly negatively correlated, and maximum surface temperature (−0.5), maximum wind speed (−0.5), average temperature (−0.5) and maximum temperature (−0.5) are moderately negatively correlated. The factors of the correlation coefficient of Taiyuan PM2.5 are PM10 (0.9) and CO (0.9), which are highly positively correlated. NO2 (0.5), SO2 (0.6), average humidity (0.6), minimum humidity (0.6), season (0.5) It is

moderately correlated. The highest surface temperature (−0.5), highest wind speed (−0.5), extremely high wind speed (−0.5), and sunshine duration (−0.5) are moderately negatively correlated The correlation coefficient of PM2.5 in Tongchuan had a high positive correlation with CO (0.9), moderate correlation with PM10 (0.6), average humidity (0.5), minimum humidity (0.5) and season (0.5), and moderate negative correlation with maximum surface temperature (−0.5), maximum wind speed (−0.5), extremely high wind speed (−0.5) and sunshine duration (−0.5). The factors of the correlation coefficient of PM2.5 in Sanmenxia are PM10 (0.7) and CO (0.8), which are highly positively correlated, the average humidity (0.5), the lowest humidity (0.5), and the season (0.5) are moderately positively correlated, and the average surface temperature (−0.5), the highest surface temperature (−0.5), extremely high wind speed (−0.5), average temperature (−0.5), and highest temperature (−0.5) have a moderately negative correlation. The factors of the correlation coefficient of Lvliang PM2.5 are PM10 (0.6), CO (0.7), average humidity (0.6), minimum humidity (0.6), and season (0.6), which are highly positively correlated. The average surface temperature (−0.5) and the surface average temperature (−0.5), extremely high wind speed (−0.5), average temperature (−0.5), maximum temperature (−0.5), sunshine duration (−0.5) are moderately negatively correlated, average humidity (0.5), minimum humidity (0.5), season (0.5), etc. have a moderate correlation. Luoyang PM2.5 correlation coefficient factors are PM10 (0.8) and CO (0.9) are highly positively correlated, the average surface temperature (−0.5), the highest surface temperature (−0.5), the average temperature (−0.5), the highest temperature (−0.5), sunshine duration (−0.5) are moderately negatively correlated, and average humidity (0.5), minimum humidity (0.5), season (0.5), etc. are moderately correlated. Linfen PM2.5 correlation coefficient factors PM10 (0.9) and CO (0.9) are extremely highly positively correlated. Further, SO2 (0.7), average humidity (0.6), minimum humidity (0.7), season (0.6) are highly positively correlated. The average surface temperature (−0.6), the highest surface temperature (−0.7), the average temperature (−0.6), the lowest temperature (−0.5), the highest temperature (−0.6), and the duration of sunshine (−0.5) have a moderately negative correlation. The correlation coefficients of PM2.5, PM10 (0.9) and CO (0.9), S02 (0.7), average humidity (0.6) and minimum humidity (0.7) in Jinzhong were highly positively correlated. The average surface temperature (−0.5), maximum surface temperature (−0.6), average temperature (−0.5), minimum temperature (−0.4), maximum temperature (−0.5), and sunshine duration (−0.5) were moderately negatively correlated. The correlation coefficient of PM2.5 was PM10 (0.7) and CO (0.8), and the average humidity (0.5), minimum humidity (0.5) and season (0.5) were moderately correlated. The average temperature (−0.5), the maximum temperature (−0.5), the average temperature (−0.5), the minimum temperature (- 0.4), and the maximum temperature (−0.5) were moderately negatively correlated.

Meteorological elements affect air quality by affecting the accumulation, diffusion, and elimination of pollutants. In the studies of PM2.5 and PM10 concentration, they are found closely related to meteorological elements (such as temperature, precipitation, wind speed, etc.). According to existing studies, relative humidity has an important a key factor to fine particle concentration [38]. At higher relative humidity, pollutants are attached to the surface of water vapor easier. Water solution is a good place for chemical reaction [39]. Wind direction and speed affect the dispersion of particulate matter in the air [40]. Chen et al. made predictions on PM2.5 concentration in Zhejiang Province, finding that meteorological factors such as air temperature, air pressure, evaporation, humidity are remarkably correlated with PM2.5 concentration [41]. Zhang Zhifei et al. found that O3 h mass concentration has positive correlation with air temperature, solar radiation, visibility and wind speed, whereas NO2 concentration is positively correlated with relative humidity and atmospheric pressure [42]. Precipitation [43], season [44], precipitation [45], sunshine duration [46], and other factors have remarkable impacts on the concentrations of air pollutants. Different city characteristics will also have different impacts on PM2.5, the correlation coefficient of PM10 and CO in Jinzhong and Linfen is 0.9. Further, the two numbers in Lv Liang are 0.6 and 0.7. The correlation coefficients of 12 cities show that temperature, surface temperature, atmospheric pressure, air humidity, and sunshine duration all affect PM2.5. Further analysis is needed in selecting appropriate features for the model.


**Figure 5.** Pearson Analysis of Atmospheric Pollutants and Meteorological Factors in 12 Cities of Fenwei Plain.

### *4.2. Feature Selection*

Through Pearson analysis, it is found that addition to the traditional six atmospheric pollutants, meteorological factors are also main factors to PM2.5 concentration, such as surface temperature, temperature, sunshine duration, humidity, and so on. Consider unrelated and redundant factors, which may obscure the role of important factors and require the mining and refinement of raw data.

### 4.2.1. Feature Importance Sorting Principle

The traditional GBDT algorithm uses first derivative, while XGBoost expands the error function with second-order Taylor, using both first-order and second-order derivatives. XGBoost uses a second-order Taylor expansion of the error function, and XGBoost uses column sampling of features to select the proportion of features used in training and to prevent over-fitting effectively. The parallel approximate histogram algorithm for XGBoost's feature split gain calculation can make full use of multicore CPUs for parallel computation. Traditional feature selection models iterate continuously during operation, and new trees will be generated after each iteration. When dealing with complex datasets, they may

iterate over hundreds of thousands of times, so they are not efficient. To overcome this disadvantage, the XGBoost algorithm uses a regression tree to build models. This system is based on the Boosting algorithm, which has made great breakthroughs in prediction accuracy and training speed. In fact, XGBoost calculates which feature to select as the split point based on the gain of the structure fraction. The importance of a feature is the sum of times it occurs in all trees. The more an attribute is used to build a decision tree in a model, the more important it is. Using gradient enhancement makes it relatively easy to retrieve the importance for each attribute after building an enhanced tree. Generally, importance represents a score, indicating the usefulness or value of a feature in the process of building an enhanced tree in a model. The more attributes used for key decisions in a decision tree, the higher its relative importance is. Generally speaking, importance provides a score indicating how useful or valuable each attribute is in building an enhanced decision tree in a model. The more times attributes are used to make key decisions using a decision tree the higher the relative importance is. This importance is explicitly calculated for each attribute in the dataset so attributes can be ranked and compared with each other. The importance of a single decision tree is calculated by increasing the number of performance indicators per attribute split point, weighted by the number of observations the node is responsible for.

### 4.2.2. Experimental Process and Analysis of Feature Selection

We conduct a feature filtration on some parts of training set, and divided data sets into training sets and validation sets. First, we make XGBoost model which contains that contains all the feature training sets, use the five-fold cross validation to find the optimal parameters, and sort the features based on Fscore. Then we filter the sorted feature sets, evaluate whether a feature can be preserved under Fscore value, and delete the feature set which is scored lowest one by one. The AUC value of the validation set under the new feature subset is used to determine whether the predicted results of the remaining features are better or not. Both the number of features and the model improvement effect should be taken into consideration when selecting features. As some features have limited improvement effect on models, this experiment should use features that have greater impact on prediction of PM2.5 concentration. The threshold h is set (the exact value of H is set according to the experimental results) to select the features. If the AUC value of the validation set increases more than *h*, the recently deleted features are saved. If the AUC value increases less than h or decreases, the deleted features are still removed. The algorithm can filter out the features that have a greater impact on the target variable and reduce the redundancy between the features.

As shown in the Figure 6, features are filtered by XGBoost. we use the "*importance\_type = gain*" method to calculate the importance of features. We use five-fold cross validation meothod and grid search to find the optimal parameters of XGBoost model. The parameters of XGBoost algorithm are according to the weight of features. The importance of a feature can be used as a model explanatory value. This method represents the average gain from the presence of a feature as a split point in all trees.

In all trees, the number of times a feature is used to split nodes is *Weight*, and the total gain that a feature brings each time it splits a node is *Total\_gain*. F Score formula is shown in Equation (25):

$$\text{FScore} = \text{Total\\_gain} / \text{weight} \tag{25}$$

Average gain is calculated as Equation (26):

$$\text{AverageGain} = \frac{Total\\_gain}{Fscore} \tag{26}$$

XGBoost calculates which feature to select as the split point based on the increment of the structure fraction. The importance of a feature is the sum of the number of times it occurs in all trees. The more an attribute is used to construct a decision tree in a model, the more important it is. Using XGBoost to rank the feature importance, as shown in

Figure 6, the top 10 cities are the 12 cities with different feature importance of PM2.5. We input the filtered features into MSCNN-GA-LSTM.

**Figure 6.** XGBoost model flow.

The importance of features is sorted by XGBoost, and the threshold *h* is set to 0.002. As shown in Figure 7 the *y*-axis represents each city, and the *x*-axis coordinates represent each feature. The numbers in the box represent the value of features importance in different cities. The color depth of the box represents the size of Fscore. The darker the color, the more important the feature. The lighter the color, the less important the feature. The top 10 feature importance of 12 cities are listed in the chart. Consistent with the previous Pearson correlation analysis, we found that the air pollutant characteristics with strong correlation, such as PM10 and CO, ranked as first and second in 12 cities in the feature importance ranking, while the factors with strong negative correlation, such as maximum temperature and average wind speed, are also of high importance. The feature importance of PM2.5 varied in different cities. We input the filtered features into MSCNN-GA-LSTM.

**Figure 7.** Atmospheric Pollutant Factors and Meteorological Factors in 12 cities of Fenwei Plain.

### *4.3. GA Optimize LSTM Optimal Parameters*

In the prediction model, genetic algorithm is introduced to globally optimize the initial parameters of the LSTM network. Using traditional experience to set parameter value will make algorithm convergence easily fall into local optimum in the late period of algorithm iteration. To overcome this problem, we dynamically set the initialization parameters of the genetic algorithm. Further, we use a larger probability of perturbation, and avoid local optimum as the number of times of iteration gradually increases. Our repeated experiments, and the final optimization parameters are listed in the Table 3—a good convergence effect has been achieved.


**Table 3.** GA Optimized LSTM Optimal Parameters.

### *4.4. Forecast Results*

4.4.1. Model Comparison before and after Feature Selection

At the beginning of this section, we evaluate the performance of different models by using the predictions from the test set. Figures 8 and 9 show the simulated prediction results of PM2.5 in 12 cities using nine models. First, PM2.5 test set data are input into four single trained models for calculation, and the PM2.5 h predictions are compared with the measured results. The predicted PM2.5 h concentration is close to the measured value when the measured value of PM2.5 h concentration increases rapidly, the predicted values deviate from the measured values significantly. This may be due to the redundancy of features and the influence of space-time characteristics. It is difficult to accurately predict if the model is not trained to filter feature values. The MLP model is similar to the LSTM model in that the predicted values deviate greatly from the measured values when the measured values increase or decrease sharply. The main reason why XGBoost model is not efficient is that it cannot achieve accurate prediction over time series data. When the measured values are small, the predicted values of PM2.5 concentration are consistent with the measured values, and when the measured values are large, the predicted values are larger than the measured values. Comparing the predicted values of PM2.5 concentration of four single models in 12 cities, the LSTM model has the best predicted results.

PM2.5 test set data are input into five trained combination models to calculate. The predicted PM2.5 h concentration values of 12 cities are compared with the measured values which are shown in Figures 8 and 9. In the figure, the predicted values of the XGBoost-MSCGL model PM2.5 are consistent with the measured values, even when some individual PM2.5 h concentration values increase or decrease sharply, the predicted values are close to the measured values. XGBoost-LSTM prediction is similar to XGBoost-MSCGL model in that when the measured value increases or decreases sharply, the predicted value has a smaller deviation from the measured value, but the predicted result is slightly worse than that of XGBoost-MSCGL model. When the measured value of XGBoost-MLP model is higher or lower, the predicted value has a larger deviation from the measured value and the predicted value is smaller than that of the measured value. CNN-LSTM model performs better when the measured value increases or decreases sharply. However, compared with the other eight models, its prediction effect is the worst. For PM2.5 average concentration prediction, the predicted value is larger than the measured value. Comparing XGBoost-MLP, XGBoost-LSTM, XGBoost-CNN, XGBoost-MSCGL with CNN, LSTM, MLP, and CNN-LSTM, we found that the predicted value of the model after feature selection is closer to the measured value than that before feature selection, with a greater increase in accuracy, and a marked decrease in derivation value. Comparing the predicted values of PM2.5 h concentration of the nine models with their corresponding measured values, the XGBoost-MSCGL model had the best prediction effect.

**Figure 8.** Predicted and Measured PM2.5 h Concentration Values of Nine Models in Six Cities: Baoji, Jinzhong, Linfen, Luoyang, Lvliang, and Sanmenxia.

**Figure 9.** Predicted and Measured PM2.5 h Concentration Values of Nine Models in Six Cities: Tongchuan, Taiyuan, Weinan, Xi'an, Xianyang, and Yuncheng.

### 4.4.2. Model Accuracy Evaluation

The accuracy of the four models was evaluated by RMSE, MAPE, MAE, and R2. The smaller the RMSE, MAPE, and MAE, the higher the accuracy of the model, and the larger the R2, the higher the accuracy of the model. In order to better evaluate the error, prediction effect, and prediction accuracy of the nine models, we selected four evaluation indexes to evaluate the performance results of each model in each city, as shown in Table 4.


**Table 4.** Accuracy comparison of nine models in 12 cities.


**Table 4.** *Cont*.

Among the nine models which predicted PM2.5 h concentration value, XGBoost-MSCGL had the best prediction effect. The average MAE (8.26), RMSE (5.6), MAPE (9.9), R2 (0.95) in 12 models were the highest, while the XGBoost model had the worst predictive effect in nine models, with the average MAE (21.67), RMSE (15.25), MAPE (31.94%), R2

(0.69) in 12 cities. R<sup>2</sup> was the smallest of the nine models. The correlation coefficient R2 of the four single models was 79.07%, which may be related to the unstable time series of PM2.5 concentration and no screening of features during the model building process, resulting in no further improvement of model accuracy. From the prediction effect after feature selection, the overall prediction effect of the combination of feature selection based on XGboost with a single model has been remarkably improved. XGBoost-CNN and XGBoost in 12 cities prediction compared with CNN, LSTM, MLP, XGBoost-MSCGL, the values of -LSTM, XGBoost-MSCGL, and CNN-LSTMRMSE decreased by 13.25%, 28.63%, 20.16%, 21.64%, the values of MAPE decreased by 14.86%, 29.96%, 27.31%, 26.25%, respectively, and the values of MAE decreased by 17.02%, 24.90%, 32.26%, 33.68%. R<sup>2</sup> Values increased by 11.98%, 16.62%, 12.70%, and 6.80%. The results show that feature selection based on XGBoost can effectively improve the accuracy of prediction model and reduce the error. For PM2.5 concentration prediction, feature selection can effectively improve the accuracy and reduce the overestimation or underestimation caused by redundant features.

Among the four combination models which predicted PM2.5 h concentration value after feature selection, XGBoost-MSCGL has the best prediction effect. Compared with XGBoost-CNN model and XGBoost-LSTM model, XGBoost-MLP model has slightly higher prediction accuracy with correlation coefficient R2(0.83). The prediction results of XGBoost-CNN model are the worst among the four models, MAE (7.98), RMSE (11.07), MAPE (13.96), R2(0.9). XGBoost-MSCGL is better than XGBoost-MLP, XGBoost-LSTM, and XGBoost-CNN in predicting performance, with RMSE, MAE, and MAPE decreasing 11.11%, 15.97%, 15.36%, and R2 increasing 3%, respectively, in 12 cities. Overall, XGBoost-MSCGL is better than XGBoost-MLP, XGBoost-LSTM, and XGBoost-CNN in predicting performance. As for cities, XGBoost-MSCGL performed best in Xi'an with MAE (3.94), MAPE (5.59), R2 (0.98). The worst in Xianyang was RMSE (12.65), MAE (8.25), and Lv Liang's R2 (85.78).

By analyzing the predicted data of 12 cities in the Fenwei Plain, it is noted that different prediction models have different performances in reducing errors and improving consistency of changes in different cities. The prediction error may be related to the different city characteristics that we choose, and to a different dispersion of air pollutant concentration values in each season. Using four deep learning combination models for training and validating the prediction accuracy, the results show that XGBoost-MSCGL has the highest prediction accuracy for most city training sets, and its prediction performance is better than other models. Through the three indicators of RMSE, MAE, and MAPE, we can see that XGBoost-MSCGL has better prediction performance than XGBoost-MLP, XGBoost-LSTM, XGBoost-CNN. In 12 cities, RMSE, MAE, and MAPE decreased by 11.11%, 15.97%, and 15.36%, respectively. However, XGBoost-LSTM in Xianyang, XGBoost-MLP in Weinan, and XGBoost-CNN in Jin are slightly higher than XGBoost-MSCGL in MAPE. XGBoost-MSCGL in Xi'an, Taiyuan, Sanmenxia, and other cities declined significantly. Overall, the error value of XGBoost-MSCGL in the four combined prediction models is small, the performance is outstanding, and the prediction effect is better.

Through the analysis of the prediction data of the 12 cities in the Fenwei plain, we noticed that the performances of different prediction models were different in reducing errors and improving the consistency of changes of changes in different cities. The prediction errors might have something to do with the different types of city characteristics and the degree of dispersion of the concentration of air pollution in different seasons. We used four models of deep learning to train and test. The results show that XGBoost-MSCGL has the highest prediction accuracy for most of the city's test sets, and it is better than other models in terms of prediction performance.

### **5. Discussion**

In this study, the PM2.5 feature selection based on XGBoost, combined with MSCNN to extract temporal and spatial features, and GA optimized LSTM, were used to establish the XGBoost-MSCGL air pollutant concentration prediction model. Compared with other machine learning, feature selection combined with feature extraction and combined with deep learning is an effective method for processing big data (especially spatio-temporal feature data). Combining spatio-temporal feature and models can improve the performance of spatio-temporal data prediction to a certain extent. In different cities, the importance of PM 2.5 influencing factors are different. It is necessary to select PM2.5 influencing factors in different cities and propose redundant features and delete redundant features in order to avoid influencing the accuracy of the prediction model. The prediction method proposed in this paper is feasible for the PM2.5 h concentration data prediction in multiple cities, and the method can be used in multiple regions and predictions on different atmospheric pollutant concentration. In terms of input variables, regular monitoring data from the National Environmental Monitoring Station, and China Meteorological Administration are used. In terms of modeling methods, machine learning and deep learning algorithms are combined. On the premise of eliminating redundant features, space and time features are considered, and a genetic algorithm is used to optimize the parameters of the LSTM network, enabling it to capture optimal parameters better. With stronger capturing ability, the long-term dependence relationship hidden by air quality data is more accurate, and the prediction accuracy is further improved.

The shortcoming of this research is that in different cities, the performances of XGBoost-MSCGL model may be different due to driving factors, spatio-temporal characteristics, model types, model structure, and model development methods. We find that in cities such as Xi'an, the model performs well. However, in some cities, their performances cannot achieve the same accuracy and prediction effect. The dispersion of PM 2.5 concentration data in different cities and other city air pollutants may also affect the prediction performance of the model. So, it was necessary to further analyze the reason for the difference. At the same time, the data volume, the dispersion between air pollutant concentration values and space features might also affect the performance of model prediction. So, it was necessary to further analyze the reason for the difference. In this study, the range and interval prediction of air pollutants concentrations are not taken into consideration. In following researches, it needs to be discussed in detail. Only in this way could the relevant government and enterprises better monitor and manage the release of air pollution.

### **6. Conclusions**

In this study, based on the hourly concentration data and meteorological data of six air pollutants in 12 cities in the Fenwei Plain in 2020, a PM 2.5 concentration prediction model, based on XGBoost-MSCGL, was established, and the performance of the model was compared with XGBoost-MLP, XGBoost-LSTM, and XGBoost-CNN. The main research results are as follows: In the PM2.5 concentration prediction, the XGBoost-MSCGL model performs better in 12 cities in the Fenwei Plain, with smaller error values and better prediction results. As for feature selection, compared with the prediction of all influencing factors, the prediction effect of the former is significantly improved for the factors of feature selection. From the perspective of spatio-temporal characteristics, the hourly concentration prediction performance of the 12 cities considering spatio-temporal characteristics is better than the prediction model that does not consider spatio-temporal characteristics. From the perspective of the optimized model, the accuracy of the optimized model is significantly improved compared to the unoptimized model. In general, based on feature selection, screening the influencing factors of PM2.5 according to their importance helps to reduce the feature redundancy of the data set. In terms of overall performance, the prediction performance of the XGBoost-MSCGL model is generally better than that of the XGBoost-MLP, XGBoost-LSTM, and XGBoost-CNN models. Compared with other prediction methods, the PM2.5 concentration prediction, based on the XGBoost-MSCGL model, has better performance in accurately predicting the actual data in different cities. Compared with other models, it has a higher accuracy improvement and achieves better prediction especially when the data are at extremely high and low points in the sharp fluctuations. The migration of the model is verified by the prediction results of 12 cities

in Fenwei plain. The concentration change direction and volatility of PM2.5 need to be further considered in future research.

**Author Contributions:** The article was written through the contributions of all authors. H.D.: conceptualization, methodology, modelling, analysis, writing—original draft preparation. G.H.: conceptualization, modelling, writing—reviewing and editing, revision. H.Z.: modelling, analysis, writing—original draft preparation, writing—reviewing and editing. F.Y.: analysis writing reviewing and editing, revision. All authors have read and agreed to the published version of the manuscript.

**Funding:** This paper is funded by the National Natural Science Foundation of China (71874134). Key Project of Basic Natural Science Research Plan of Shaanxi Province (2019JZ-30).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Both air pollutant data and meteorological data come from public data provided by the Ministry of Ecology and Environment of the People's Republic of China (http://www.mee.gov.cn/, accessed on 18 September 2021) and China Meteorological Administration (http://www.cma.gov.cn/, accessed on 18 September 2021).

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


### *Article* **Competitiveness of Industrial Companies Forming the Value Chain of Wind Energy Components: The Case of Lithuania**

**Akvile Cibinskiene 1,\*, Daiva Dumciuviene 1, Viktorija Bobinaite <sup>2</sup> and Egidijus Dragašius <sup>3</sup>**


**Abstract:** Sustainable energy development has attracted attention worldwide, partly because of the value chain of the wind energy industry that focuses on the overall value creation and innovation. In order to achieve not only ambitious goals in the fight against climate change, but also to create significant economic benefits for European Union citizens, it is necessary to ensure the production of renewable energy components in Europe itself and in Lithuania at the same time. This paper aims to evaluate the competitiveness of Lithuanian companies that manufacture wind energy components. The research was conducted applying methods such as a survey of manufacturers of wind energy components, expert assessment and descriptive analysis. The results of the competitiveness assessment revealed that the existing conditions and trends are favourable for the development of their performance and strengthening of their competitiveness. The government solutions to promote industry could facilitate the performance of companies operating in the value chain of wind energy components and encourage new companies to join it. This would encourage the Lithuanian industry to expand its participation in the value chain of the European Union's renewable energy industry, create more jobs, and increase the added value.

**Keywords:** wind power industry; manufacturers of wind energy components; value chain of wind energy components; competitiveness

### **1. Introduction**

Sustainable energy development has attracted attention around the world, partly because of the value chain of the wind energy industry that focuses on overall value creation and innovation. The global wind energy industry is going through major changes. The development of renewable and sustainable energy technologies makes the wind energy industry more intense, efficient, greener and more complex in terms of value creation. It is therefore intended to minimize value-added operations and maximize value creation as much as possible. The value chain model of the wind energy industry allows for a comprehensive analysis of the factors forming the value chain of wind energy components (VCWEC), to determine the overall value creation process.

In order to achieve not only ambitious goals in the fight against climate change, but also to create significant economic benefits for European Union (EU) citizens, it is necessary to ensure the production of renewable energy components in Europe itself and in Lithuania at the same time. Lithuania proposes to include wind production and energy storage technologies in the EU's strategic value chain. The EU needs a stable and strong industrial sector and research and development (R&D) efforts to become a leader in the developing renewable energy technologies.

**Citation:** Cibinskiene, A.; Dumciuviene, D.; Bobinaite, V.; Dragašius, E. Competitiveness of Industrial Companies Forming the Value Chain of Wind Energy Components: The Case of Lithuania. *Sustainability* **2021**, *13*, 9255. https:// doi.org/10.3390/su13169255

Academic Editors: Baojie He, Ayyoob Sharifi, Chi Feng and Jun Yang

Received: 21 July 2021 Accepted: 13 August 2021 Published: 18 August 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

It is strategically important to invest rapidly and actively in the recovery and growth of the Lithuanian economy after the COVID-19 crisis. Firstly, it is vital to exploit the situation and ensure a sustainable, innovative and high-value-added economy. It is also vital to attract investment, reduce dependence on the Asian market and shorten and diversify the value chains. Finally, it is important to increase their resilience, create new jobs and ensure the extraction of green energy and the supply of this energy to other sectors.

Wind energy is one of the types of "green" energy which can address the aforementioned issues. In detail, due to the development of offshore and onshore wind energy, people are supplied with jobs, which accounted for 300,000 jobs in EU in 2019, and the economy is supplied with growth, which was generated from value-added contributions of EUR 37.2 billion and exports in goods and services of EUR 8 billion [1]. Furthermore, wind energy made relevant contributions to the local economy through taxes, which were of EUR 1.3 billion (not linked to corporate profit) and were designated to local governments and communities [1], and income for farmers in a form of lease payments to landlords of the area [2]. Indeed, in the presence of rising carbon and fuel prices, energy demand and more [3], the low operating cost of wind energy [2] kept wholesale electricity prices low and decreasing [3,4]. Being a local industry, wind energy reduced the country's dependence on polluting fossil fuels [5], helped stabilize the cost of electricity and reduced vulnerability to price spikes and supply disruptions [2]. Beyond the economic benefits, wind energy is relevant for environmental purposes as it reduces greenhouse gas (GHG) emissions [6], including lifecycle GHG emissions [7]. The latest research demonstrates that the known environmental, economic and other benefits of wind energy might even be larger in monetary terms if novel solutions are adapted, including wind farm layout optimization [5,8].

Changes, trends, forecasts, and investments of strategic importance to the country require detailed analysis and substantiation. In order for Lithuania to successfully solve the tasks mentioned above and join the VCWEC manufacturing, research of Lithuania's opportunities and perspectives in this field is essential and relevant, not only in the country's context, but also in the context of the EU.

The World Economic Forum (WEF) defines competitiveness as the ability of a country or firm to create more wealth than its competitors under global market equilibrium [9]. The Organization for Economic Co-operation and Development (OECD) argues that competitiveness under favourable market conditions shows the extent to which a firm can produce goods and services for an internationally competitive market while synchronizing real domestic income and growth in living standards [10]. The International Institute for Management Development (IMD) defines competitiveness as an effective mean of achieving growth in living standards and social well-being [11].

Although the concept of competitiveness has been used and applied in research for a relatively long time, there are many studies of the competitiveness of various types of companies, industries and countries. In addition, the regular calculation and publication of national competitiveness indices are promoted (Global Competitiveness Index (WEF), World Competitiveness Rating (IMD), the European Regional Competitiveness Index (European Commission)). In general, competitiveness shows the ability of a company to make business decisions using its available resources, which allows it to occupy a higher position than its competitors do. Therefore, the competitiveness assessment can provide valuable insights in evaluating the development opportunities, perspectives and resilience of the companies forming the VCWEC.

This paper aims to evaluate the competitiveness of Lithuanian companies that manufacture wind energy components (WEC).

The results of this research can provide insights for the Lithuanian government to take actions that will encourage the Lithuanian industry to participate more widely in the value chain of the EU's renewable energy industry, including wind energy, create more jobs and increase added value.

### **2. Theoretical Background of Competitiveness Assessment**

The origins of competitiveness are linked to the ability of individual countries to accumulate more wealth. This concept was later developed and adapted to various levels, not just countries. It was taken into account that the country's assets depend on the industries and individual companies operating within it. In this way, the factors that may affect competitiveness are starting to be examined, and models for competitiveness assessment are created.

### *2.1. The Concept of Competitiveness and the Levels of Its Assessment*

The representatives of the mercantilist worldview may be considered the pioneers of the country's competitiveness and the analysis of its factors. In the 14th century, they held the position that a country that is able to accumulate more wealth by promoting exports and restricting imports is in a better position. A. Smith's [12] concept of absolute advantage and D. Ricardo's [13] theory of comparative advantage, which states that a country has a comparative advantage in the production of a certain product when the opportunity cost of this product is lower than in other countries, can also be considered as the basics of a competitiveness assessment. The representatives of classical and neoclassical economic theory paid great importance to trade, which ensures the country's competitiveness. According to them, competitiveness is gained by being able to profit from international trade by exporting expensive products or services produced cheaply and using inexpensive raw materials. Competitiveness was later developed within the framework of Shumpeter's theory of entrepreneurship and innovation, the Porter Diamond model and even later in Krugman's [14] theory of new economic geography criticizing competitiveness [15].

The term "competition" is derived from the Latin word *concurrentia*, which means collision and embarrassment [16]. The concept of "competitiveness" is treated slightly differently by many authors. Porter [17] can be considered as the originator of the modern concept of competitiveness, and he argued that competitive advantage could be understood as "a country's ability to create an environment that enables companies to develop and innovate faster than foreign competitors". According to Fang et al. [18], the definition of competitiveness varies depending on the context, scale and purpose of its application.

A review of the concepts of competitiveness presented by various authors shows that this category can be examined in very different ways. Summarizing the research of many authors, Travkina and Tvaronaviˇciene [ ˙ 19] state that a country's competitiveness can be analysed by levels (micro, meso, macro), areas (economics, politics, society and technology) and time perspectives (medium or long term). Only the same levels of competitiveness can be compared. However, we can observe that the competitiveness of the lower level (e.g., companies) forms the competitiveness of the higher level (industry), and this also affects the overall competitiveness of the country [20]. This dependence of the country's economic competitiveness on the competitiveness of its constituent industries, which in turn depend on the competitiveness of enterprise that is determined by the competitiveness of employees, was also revealed by Reiljan, Hinrik and Ivanov [21].

### *2.2. The Assessment of Competitiveness*

Many different models can be found to assess competitiveness. Balkyte and Tvaronav- ˙ iˇciene [ ˙ 22] state that many economists develop competitiveness models to adapt them to different factors that affect competitiveness. The broader the analysis of competitive advantages, the more complex the model is, leading to differences in the views of researchers, even when assessing at the same sectors or industries.

One of the most commonly used models for competitiveness assessment is the Porter Diamond Model [17]. It covers four main groups of factors that affect national competitiveness: factor conditions, demand conditions, related and supporting industries and firm strategy, structure and rivalry.

**Factor conditions.** This group of factors includes the factors of production needed to compete in a given industry. These factors include human resources (quantity of labour, skills, costs, etc.), physical resources (natural resources or raw materials used in production, their quantity, quality, price, availability; they may also include the country's natural or climatic conditions), knowledge resources (scientific, technical knowledge resources), capital (amount of capital, cost) and infrastructure (types of infrastructure, quality and cost of use).

**Demand conditions.** These are the conditions that shape the demand for manufactured products or services in domestic and foreign markets. In the absence of demand, it is hardly possible to achieve competitiveness.

**Related and supportive industries.** Nowadays, this is often referred to as a cluster of a particular industry or activity. These are industries or companies that are geographically close to each other, interact, collaborate or complement each other with their specific activities. These may include local suppliers or scientific and other organizations.

**Firm strategy, structure and rivalry.** It is the internal organization, management and strategy of a company that can affect its competitiveness in one way or another.

In addition to these four main groups of factors, Porter [17] adds government and a chance that can affect all four main groups of factors. Direct intervention, significant technological discoveries, jumps in factor prices, significant changes in financial markets, jumps in exchange rates, fluctuations in global and regional demand, political decisions of foreign governments, wars and pandemics are considered as a chance. Porter treats the government not as a factor of the Diamond model but rather as an influencer of its structure and efficiency.

Although the Porter Diamond model has been supplemented and refined in various ways, the main groups of factors identified have become the basis for the various models used in the research. It has become popular for competitiveness assessment because it covers the key elements that shape competitiveness, and the factors that affect each key element can be chosen flexibly depending on the object of the research.

### **3. Porter Diamond Competitiveness Assessment Model for Assessing the Competitiveness of Enterprises Forming the VCWEC**

Zhao et al. [23] used the Diamond model to assess the performance and competitiveness of the Chinese wind energy industry, and Liu et al. [24] applied it to a review of the value chain of the Chinese wind energy industry. Irfan et al. [25], with the help of this model, examined the main factors affecting the Indian wind energy industry. Fang et al. [18] applied this model to assess the competitiveness of national renewable energy in the G20. They took only the four main factor groups of the Porter Diamond model for the assessment of national renewable energy competitiveness but combined it with the indicator system. Zhao et al. [23] applied the improved dynamic Diamond model, which includes government as the fifth determinant that directly affects all main groups of factors except for factor conditions. However, one of the factors under the group of factor conditions discussed in that study includes programs and projects that might be initiated by the government. Therefore, the determinant government should affect the factor conditions as well. Technology in that model is incorporated as an intermediate variable, affecting only demand conditions and firm strategy, structure and rivalry. Under the technology Zhao et al. [23] discuss technical R&D; R&D capacity building and standards and norms building. These elements may have an impact not only on demand conditions and firm strategy, structure and rivalry but also on factor conditions and related and support industries. The chance in this model has influence only on factor conditions and firm strategy, structure and rivalry but in fact may have influence on the other main Diamond model factor groups. Irfan et al. [25] applied the Diamond model with the core four groups of factors with government and chance both affecting all main elements. Technology in this case was not included as an element of the model but showed up in the analysis under the chance and firm strategy, structure and rivalry. Although Zhang et al. [26] discuss Porter's Diamond model for the international competitiveness of China's wind turbine manufacturing industry, they did not apply it for evaluation.

Summarizing the performed research of wind energy competitiveness, the adapted Diamond model to assess the competitiveness of Lithuanian companies forming the VCWEC was applied (Figure 1).

**Figure 1.** Diamond model of the VCWEC (adapted according to Liu et al. [24]).

In assessing the competitiveness of the wind energy industry according to the Diamond model, foreign authors have assigned to Porter's selected factor conditions the appropriately selected factors relevant to each activity. The authors studying the competitiveness of the wind energy industry, renewable energy and wind energy industry value chain have assigned the following factors to the respective groups of factors in the Porter's Diamond model:


As can be seen, some of the above-mentioned factors of the assessment of the competitiveness of Lithuanian VCWEC companies cannot be applied since they were applied at another level of research and become irrelevant in this case. Examples of these factors include local renewable energy companies as a factor in the firm strategy, structure and rivalry of companies. In the case of this study, the element must include the companies that form the VCWEC.

Fang et al. [18] singled out the substitution costs under the demand conditions when examining the competitiveness of renewable energy, which was considered as a possible change in energy prices due to the changed energy structure (an increasing share of energy from renewable sources may lead to an increase in the price to the final consumer, leading to fewer consumers choosing to buy energy from renewable sources). This study chooses the domestic and foreign market demand for "green" products as factors with similar meaning. Fang et al. [18] also included in the demand conditions a factor of political incentives that may encourage consumers to abandon fossil fuel energy and choose energy from renewable energy sources, which may affect demand. The Diamond model used in this study distinguishes a separate group of "government" factors that affect all the major groups of the other factors. Therefore, the factors related to government decisions were assigned to the group of "government" factors in this study. The wind energy rejection factor singled out by Liu et al. [24] is associated with possible restrictions on the supply of wind energy due to the limited capacity of China's transmission networks. The off-grid energy factor examined by Zhao et al. [23] is also specifically relevant to the Chinese market, as it relates to wind energy that is generated in rural areas and consumed locally, which cannot be supplied to the broader market as it would be economically inefficient to build power transmission lines to these areas. Due to the specifics of these factors, which are relevant to the Chinese market and not relevant in Lithuania, they were not included in this study. The manufacturers of WEC are classified as a group of related and supporting industries by Fang et al. [18], Irfan et al. [25] and Zhao [23], as they examined the competitiveness of wind (renewable) energy. In the case of this study, the factor is attributed to the firm strategy, structure, and rivalry because the manufacturers of WEC are competitors to the firms in question. Typically, the output of manufacturers of WEC is very specific and only in rare cases can one manufacturer of WEC be a supplier or customer to the other component manufacturers. For this reason, they are not included in the group of related and supporting industries in this study. Irfan et al. [25] assigned a specialization factor to firm strategy, structure, and rivalry, emphasizing on firm specialization to achieve competitive advantage. In this study, Lithuanian manufacturers of WEC usually produce a wider range of products, only one part of which is WECs; therefore, the inclusion of this factor in the study becomes meaningless. The opportunities and challenges of a chance for foreign authors are related to the conditions of demand and factors, as well as government and technology elements, and therefore, these factors are not applied in this study. To describe a chance in this study, price spike, shortage of raw materials, supply disruptions and other factors are used.

Irfan et al. [25] attributed the technological innovation factor to the element of firm strategy, structure and rivalry, but in the case of this study, it is more appropriate to attribute it to the element of technology. The factors "energy storage" and "construction of ultra-high voltage electricity transmission networks" that were attributed to the group of technological factors by Liu et al. [24] are not relevant for manufacturers of WEC in this study, therefore, they were further omitted.

Summarizing the factors applied in the research of foreign authors, as well as the factors of the business environment, statistics and analysis of the current situation, the following factors in assessing the competitiveness of Lithuanian VCWEC were singled out.

### **Factor conditions:**


### **Demand conditions:**


### **Related and supporting industries:**


### **Firm strategy, structure and rivalry:**


### **Government:**


### **Technologies:**


**Chance:** price spike, shortage of raw materials, supply disruptions.

### **4. Methods of Assessing the Competitiveness of Enterprises Forming a VCWEC**

Various quantitative and qualitative methods are used to assess competitiveness according to the Porter's Diamond model. The applied method is determined by the selected competitiveness assessment factors and data describing them. It is difficult to limit oneself to quantitative methods, as not all of the selected factors can be quantified or there is a lack of quantitative data. Some of identified competitiveness factors for this study describe the business environment, another part is the analysis of statistical data and the third part is the answers to purposefully formulated questions for companies in the questionnaire. Irfan et al. [25], Liu et al. [24] and Zhao et al. [23] performed a competitive assessment

using the method of descriptive analysis. Zhang [26] and Fang et al. [18] used quantitative methods to assess competitiveness. However, in their case, the number of competitiveness factors was lower, and they performed an international competitiveness assessment comparing the competitiveness of renewable energy producers and international wind power producers in different countries, respectively.

In order to cover as many factors of competitiveness of the companies forming the VCWEC as possible, a descriptive logical analysis, a questionnaire survey and an expert assessment were used. The results of the survey of manufacturers of WEC allow for the simultaneous assessment of the business environment and the factors shaping the competitiveness. Based on these, it is possible to draw conclusions about the influence of individual groups of competitiveness factors on the competitiveness of the companies in question. The expert assessment was applied to assess the business environment and the significance of various individual groups of competitiveness factors. In our case, the experts selected were the representatives of energy and industry associations.

The general aim of *the survey of manufacturers of WEC* was to identify and describe the peculiarities of these companies' performance in terms of the group of products they produce and could produce in future, the share of WEC in the total production value, the historical development of sales volumes of WEC, the export markets and the shares of exports, the customers they deal with, the trade intermediaries they have, the competitors in domestic and foreign markets, the purchases of raw materials, the post-sale services they provide, the participation in R&D, the factors of business environment they face, the issues they meet and the role of government improving the conditions for performance. The survey was constructed based on the questionnaire, which consisted of 24 questions covering the activities mentioned above. This paper focuses mainly on responses of the manufacturers of WEC to questions about the factors of the business environment. The identified factors were grouped considering the PEST analysis method which allowed for an analysis of the Political, Economic, Sociodemographic and Technological determinants of the business environment. The manufacturers of WEC were asked to assess whether the factors provide opportunities or threats to their participation in the VCWEC and, later on, to assess the strength of this impact on a scale from 1 of 5, where:


In addition, if manufacturers of WEC supposed that the factor of business environment is irrelevant to their participation in the VCWEC, they could scale it as of impact 0, which means it is insignificant factor.

From the practical perspective, the scaled strength of the impact on opportunities should be understood as the opportunity to increase production value, earn net profit, grow, increase exports, enter new markets or achieve other benefits, which are adjusted to the core aim of that company. In contrast, the scaled strength of the impact on threats means that the factor of the business environment contradicts the main aim of that company's performance and results. A rating of 0 means that the factor of business environment is insignificant as it does not impact the performance and results of manufacturer. No numerical values were attributed to measures of opportunities and threats; therefore, manufacturers were left with personalized understanding. Such a decision was made by the Authors of this paper with the purpose to normalize the responses.

*The expert assessment* aims to assess how strongly the factors of the political, economic, sociodemographic and technological environment impact the participation of companies in VCWEC and evaluate the importance of the factors determining the competitiveness of these companies. The expert assessment questionnaire consisted of seven questions. Of these, four questions were designed to assess the factors of the business environment according to the opportunities and threats (on a scale from 1 to 5, see above), one question

on the assessment of the importance of groups of competitiveness factors (on a scale from 1 of 5) and two open-ended questions. One open-ended question was asked to find out the reasons that prevent companies from getting involved in the VCWEC, as this may not have been reflected in the assessment of business environment. The second open question sought to find even more companies involved in the manufacture of WEC that could be interviewed.

The assessment of the competitiveness of the companies forming the Lithuanian VCWEC was carried out in three stages, as shown in Figure 2.

**Figure 2.** The assessment of the competitiveness of the companies forming the Lithuanian VCWEC.

Factors that describe both the business environment and the results of the analysis of statistics and the current situation were selected to assess the competitiveness of the companies forming the VCWEC. Thus, in the first stage of the research, a survey of companies, experts' assessment, analysis of statistics and the current situation were carried out. In the second stage of the study an analysis of the data collected during the first stage is performed, i.e., assessing competitiveness factors and determining the importance of groups of competitiveness factors. By combining the evaluations carried out in the second stage, their results are summarized in the third stage.

During the research, 25 manufacturers of WEC were identified and contacted electronically by sending the questionnaires or the references to questionnaires during 1 March 2021 and 5 July 2021. In total, seven manufacturers of WEC sent their responses. The responses about business environment factors were summarized considering the weighted average method, which was applied for assessments of factors providing both opportunities and threats. Four experts participated in the survey. Since there have not been many manufacturers forming Lithuanian VCWEC, many industrial associations could not be experts. An expert assessment was organized electronically between 28 May 2021 and 14 June 2021 by sending the questionnaires or the references to experts.

The Kendall concordance coefficient was used to assess the level of coincide (similarity) of experts' opinions [27], as follows:

$$\mathcal{W} = \frac{12S}{m^2(n^3 - n)}\tag{1}$$

where *m*—number of experts, *n*—number of assessed (rated) factors and *S*—sum of squares of deviations from the mean:

$$S = \sum\_{i=1}^{n} \left( R\_i - \overline{R} \right)^2 \tag{2}$$

$$R\_i = \sum\_{j=1}^{m} r\_{ij} \tag{3}$$

where *R* − *Ri*—the average and *rij*—assessment of *i*-th factor of *j*-th expert.

The Kendall concordance coefficient can take values from 0 to 1. The closer the value of the coefficient to 1 is, the more similar the opinions of the experts are.

### **5. Results of Research on Competitiveness of Manufacturers Forming VCWEC**

*5.1. Assessment of Competitiveness Factors of VCWEC*

The calculated overall Kendall concordance coefficient of all experts' responses indicates a mean concordance of opinions, as it is 0.556. The greatest agreement is reached on the impact of the factors of the sociodemographic and economic environments on the participation of Lithuanian manufacturers in the VCWEC. The greatest difference of opinion, on the other hand, is in the assessment of the importance of the factors determining the competitiveness of manufacturers participating in the VCWEC and the impact of the factors of technological environment (Table 1).

**Table 1.** Values of the Kendall concordance coefficient by groups of factors: case of research of Lithuanian VCWEC.


According to experts, within the group of factors of the political environment, the EU and Lithuanian emission reduction targets, regional development policies, aims and supporting policies provide large opportunities to manufacturers of WEC. The first factor creates assumptions to increase demand for WEC, but the second factor creates assumptions to develop WEC business in regions having wind energy potential and relevant infrastructure, such as seaports. The medium threats come from the requirements for the management and utilization of waste generated in the production process. In the group of factors of the economic environment, increasing foreign and domestic market demand for "green" products, active EU and national support for business, as well as access to variety of business financing sources bring the WEC business medium to large opportunities; while increasing prices of goods and services, growth of production cost, as well changes of exchange rate are recognized as unfavourable conditions. The threats they cause are assessed as medium to very high. The experts singled out the shortage of requested qualification staff and emigration as a source of threat within the group of factors of the sociodemographic environment but agreed that immigration, education and society's improving attitude towards "green" products provide opportunities. The status of production facilities, implementation of new technologies and innovation, development of general infrastructure and expenditure on R&D are assessed as providing opportunities to manufacturers of WEC under the technological environment.

Based on the factors singled out in the literature review, eight factors of the economic environment, one factor of technological and sociodemographic environment, two factors describing the external environment and one factor to the question about raw materials used in the production of WEC were prescribed to a group of factor conditions. The impact of environmental factors was estimated as an average of manufacturers of WEC and expert assessment. The descriptive analysis to assess other factors was used (Table 2).


**Table 2.** Assessment of competitiveness factors: a group of factor conditions.

With reference to the information in Table 2, in a group of factor conditions, the lack of requested qualification staff is the most serious source of threat. Respondents noted that the business faces both: a shortage of skilled engineers and low-skilled staff. Other threats come from increasing cost of key production factors, including skilled labour, capital borrowing and energy. Most manufacturers of WEC procure raw materials on EU markets, but a third of them import raw materials from outside the EU and settle for raw materials using exchange rates other than EUR. Therefore, changes in exchange rates bring uncertainty. Respondents assessed the threat coming from the changes in exchange rate as low. Historically, the inflation rate was moderate in Lithuania (up to 10% a year), and during some years, deflation was fixed (during 2014–2015, inflation was 0.1–0.3% a year). Respondents assessed that threat (for example, reducing purchasing power of customers or orders of WEC) coming from increasing prices is low. The current status of production facilities assures the execution of orders. Manufacturers of WEC invest in update and development of production facilities. Since 2016, manufacturers of WEC have invested EUR 24.4 million a year when production volume has been EUR 250–300 million a year. Lithuania implements its Programme for Investment Promotion and Industry Development [30] by using EU Structural Funds and other resources which focus on

linking industry and services to networks and industrial cooperation, increasing production of advanced technologies and using raw materials and energy efficiently. Respondents assessed that changes in investment environment are positive and promising. Moreover, within the Programme, the EU and national support measures are provided to address the aforementioned issues. Respondents assessed this as a medium to high opportunity. The market provides various financing sources to business. However, respondents assessed that the opportunities from this factor are medium, as the access conditions are strict. The share of wind energy in electricity production from renewable sources has been increasing in recent years in the country. The potential of wind energy resources and changes in the structure allow for the assumption that there are favourable conditions for the development of wind energy in Lithuania. The assessment of a group of condition factors and their tendencies show that these conditions are moderately favourable for manufacturers of WEC.

Two factors of the economic environment and two factors focusing on customers of WEC and the dynamics of sales of those components over the last five years included in the manufacturers of WEC survey are assigned to a group of demand conditions. The assessment of three factors described by statistical analysis is additionally added to a group of demand conditions (Table 3).


**Table 3.** Assessment of competitiveness factors: a group of demand conditions.

The changing attitudes of consumers searching for sustainable, environment-friendly solutions increase the demand for "green" products. As it is seen from Table 3, due to expressed demand for "green" products, opportunities arise for the manufacturers of

WEC. Subject to the fact that the Lithuanian market is small for conducting business, opportunities provided by foreign markets demand for "green" products were assessed higher than those provided by the domestic market. Demand structure is not homogeneous as it is formed by the manufacturers of wind power plants and their components, the companies constructing wind power plants and the companies providing maintenance services to wind power plants. The demand structure allows for the assumption that the WEC for wind power plants installed in Lithuania are applied in different stages of the production and maintenance of wind power plants and may have the potential to increase by expanding the construction of wind power plants. The dynamics of demand over the last 5 years show that WECs sales have changed slightly, as 40% of the respondents said that sales were declining, and 30% said that they were increasing or did not change. The installed wind energy capacity has been growing in Lithuania since 2008 but has remained constant since 2016 (0.5 GW). The installed wind power capacity in the EU market is constantly growing, only decreasing by 5.8% in 2020 due to the COVID-19 pandemic. The International Renewable Energy Agency (IRENA) predicts that wind energy will meet 35% of total energy demand by 2050 in order to meet the climate change targets of the Paris Agreement. In this way, the installed wind power capacity must triple by 2030 and increase nine times by 2050 compared to 2018. Asia (mainly China) will dominate in installed onshore wind power capacities (more than 50% in 2050), while in Europe, 10% will be installed. Accordingly, with 60% of installed capacity, Asia will be the leader in offshore wind power capacities, too; in Europe 22% of offshore wind power capacities will be installed [33]. In this way, the demand conditions to produce WEC show positive trends and can be welcomed.

Four factors are assigned to a group of factors of related and supporting industries. They are included in the survey to manufacturing companies and refer to the questions about the customers of WEC, the suppliers of raw materials and the sales intermediaries (Table 4).


**Table 4.** Assessment of factors of a group of related and supporting industries.

Assessing the factors of a group of related and supporting industries, it is found that the manufacturers of wind power plants and their components, the companies constructing wind power plants, and the companies providing wind power plant maintenance services are the key related and supporting industries. Grid constructors were not identified as related and supporting industries for Lithuanian manufacturers of WEC. The suppliers of raw materials are related and supporting industries uphold the competitiveness of manufacturers of WEC. It was found that the majority of manufacturers of WEC have 3–10 raw material suppliers; one-third of those companies demand raw materials from more than 10 suppliers, but up to 10% of manufacturers of WEC are dependent on a single supplier. The sales intermediaries are less important in this group of factors, as a significant proportion (55%) of manufacturers of WEC answered that they do not have sales intermediaries. According to the results of the survey of manufacturers of WEC, it is seen that they usually communicate with their customers without intermediaries, have regular suppliers of raw materials; therefore, there is some synergy between related and supporting industries.

Three factors (Table 5) were prescribed to a group of firm strategy, structure and rivalry. Seeking to analyse them, six questions were formed in the questionnaire for the survey of manufacturers of WEC. The content of those questions referred to the groups of WEC that the manufacturers produce, the shares of WEC in the production structures, the shares of WEC exports, the export markets and the domestic and foreign competitors.


**Table 5.** Assessment of firm strategy, structures and rivalry.

According to the factor of differentiation strategy, it was found that the manufacturers of WEC produce various components, but usually they are not the only products they manufacture. Several companies specialize in WEC production; therefore, WEC make up to 95–100% of the production structure. Exports of WEC are quite widely developed among the surveyed manufacturers. They mainly export to EU countries, but 38% of the surveyed manufacturers of WEC export their products to other countries of the world, too. Lithuanian companies are too small to attract the largest wind power manufacturers. Competition in the Lithuanian market is not high, and some manufacturers of WEC have no competitors at all (40%). Competition in the foreign markets is high, but due to the specifics of the products, it is not very intense. In any case, international competition is not beneficial for Lithuanian producers due to the low costs of producers from other countries. Most of the surveyed companies indicated that they are subdivisions of global enterprise groups. Therefore, they face the following difficulties:


As opportunities to address issues related to firm strategy, structure and rivalry factors, companies see opportunities to look for partners who are already working with power plant manufacturers and aspire to be second or third in the supply chain.

Six factors of political environment and one factor of the economic environment were prescribed to a group of governmental factors. They were assessed as an average of results from the survey of manufacturers of WEC and expert assessment (Table 6).



The group of governmental factors is dominated by the factors of political environment. One factor of the economic environment was included. According to the Respondents, governmental decisions provide companies with more opportunities than threats. The most immense opportunities can be provided by the EU and Lithuanian emissions' reduction targets, followed by targets and support measures for increasing the production and consumption of energy from renewable sources in Lithuania, as well as regional development policy, its objectives and support measures as the factors create demand for WEC, facilitate business doing in places favourable to development of wind energy or to relevant infrastructure (for example, seaports).

Companies, naming opportunities for the development of their activities, indicate that the governmental decisions could stimulate it:


Three factors of technological environment were prescribed to a group of technological factors (Table 7).


**Table 7.** Assessment of technological factors.

As it is seen from Table 7, the introduction of the latest technologies, dynamics of technological changes and innovations and R&D expenditures may provide medium to high opportunities to manufacturers of WEC.

The impact of a chance on the competitiveness of the manufacturers of WEC is assessed only based on the results of an experts' survey. Its importance is scored as 3.25 on a 5-point rating scale.

### *5.2. Assessment of the Importance of Groups of Competitiveness Factors and the Competitiveness of WEC Companies*

Assessment of the competitiveness factors of manufacturers of WEC was performed in three stages. The first stage examined the factors of business environment (analysis of responses of surveyed manufacturers of WEC and experts). In the second stage, the factors of business environment were summarized and supplemented by the factors of statistical and current situation analysis. In the third stage, the assessment results of the importance of the groups of competitiveness factors were summarized according to the results of the surveyed experts (Table 8).

**Table 8.** Assessment of the importance of the factors determining the competitiveness of manufacturers of WEC.


Assessing the importance of the factors determining the competitiveness of manufacturers of WEC, experts unanimously agree that the factor conditions are the most important. However, the other factors mentioned were also considered to be quite important.

The firm strategy, structure and rivalry and the government were assessed as very important by experts too (the average score is 4.0). The analysis of individual factors of these groups revealed that Lithuanian manufacturers of WEC are characterized by their activity differentiation, the development of their exports to the EU and other countries and that they do not face the intense competition.

The importance of related and supporting industries for the competitiveness of manufacturers of WEC was assessed by experts at the average score of 3.5. The analysis of individual factors showed that companies engaged in these activities sell their products without the intermediaries, have regular suppliers of raw materials and a list of different customers, including manufacturers of wind turbines and their components, companies constructing and installing wind power plants and companies providing maintenance services to wind power plants.

The demand conditions and the impact of the chance were assessed as the least important in comparison to other discussed factors. Their average score is 3.3. Nevertheless, the analysis of demand factors has revealed that they are favourable and positively affect the competitiveness of WEC companies.

The methodology developed and applied for the research of the competitiveness of the industrial companies forming the VCWEC in Lithuanian is universal and may be applied to other research in other countries'. The manufacturer surveys and expert assessments may reveal completely different results in larger countries with more developed industries. It would be useful to carry out such a study in the context of EU countries. In this case, it might be challenging to assess government factors as they can vary significantly from one EU country to another. This may be a further area of research.

Competitiveness assessment studies often face difficulties in conducting quantitative assessments, as competitiveness models cover a wide range of factors, but most of them cannot be quantified, so qualitative assessment methods have to be applied.

A short (half a year) project implementation deadline and the quarantine made data collection more complex. With more time for companies to be interviewed and opportunities to meet, the study results might reveal slightly different results.

### **6. Conclusions**

The research was conducted applying methods such as a survey of manufacturers of WEC, expert assessment and descriptive analysis

Summarizing the factors assigned to the groups of factors assessing the competitiveness of Lithuanian industrial companies forming VCWEC assessed according to the results of companies' survey and experts' assessment, it is seen that the greatest opportunities are provided by demand conditions (3.6), followed by factor conditions (3.4), government (3.3) and technology (3.3). The threats are posed by factor conditions (3.7) and government (2.8). The analysis disclosed that the existing conditions are sufficiently favourable for the competitiveness of Lithuanian industrial companies forming the VCWEC.

According to experts, financial reasons and insufficient state legal regulation are the main obstacles that make it difficult for companies to join and/or operate in the VCWEC in Lithuania.

The interviewed Lithuanian industrial companies forming the VCWEC identified the following difficulties for their performance:


The survey of manufacturers revealed that most of them are entities of global enterprise groups. For this reason, they face the following difficulties:


The interviewed manufacturers see the following opportunities for improvement or development of their performance and the measures that would help to implement them:


The research results reveal that government solutions to promote industry could facilitate the activities of Lithuanian industrial companies forming VCWEC and encourage new companies to join it. This could encourage the Lithuanian industry to participate in the value chain of the EU's renewable energy industry, create more jobs and increase the value-added created.

**Author Contributions:** Conceptualization: D.D. and A.C.; methodology: D.D., A.C. and V.B.; software: A.C.; validation: A.C. and V.B.; investigation: D.D., A.C. and V.B.; supervision: E.D.; writing

and review: D.D., A.C. and V.B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research "Feasibility study of Lithuanian industry participation in the value chain of wind energy components. (VEGRAND)" is funded by the Research Council of Lithuania under Grant No. S-REP-21-5.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

### **References**


### *Article* **Bus Load Forecasting Method of Power System Based on VMD and Bi-LSTM**

**Jiajie Tang 1, Jie Zhao 1,\*, Hongliang Zou 2, Gaoyuan Ma 1, Jun Wu 1, Xu Jiang <sup>2</sup> and Huaixun Zhang <sup>1</sup>**


**Abstract:** The effective prediction of bus load can provide an important basis for power system dispatching and planning and energy consumption to promote environmental sustainable development. A bus load forecasting method based on variational modal decomposition (VMD) and bidirectional long short-term memory (Bi-LSTM) network was proposed in this article. Firstly, the bus load series was decomposed into a group of relatively stable subsequence components by VMD to reduce the interaction between different trend information. Then, a time series prediction model based on Bi-LSTM was constructed for each sub sequence, and Bayesian theory was used to optimize the sub sequence-related hyperparameters and judge whether the sequence uses Bi-LSTM to improve the prediction accuracy of a single model. Finally, the bus load prediction value was obtained by superimposing the prediction results of each subsequence. The example results show that compared with the traditional prediction algorithm, the proposed method can better track the change trend of bus load, and has higher prediction accuracy and stability.

**Keywords:** variational mode decomposition (VMD); Bayesian optimization; bidirectional long short-term memory (Bi-LSTM); power system bus load forecasting

### **1. Introduction**

With the development of energy environment, the proportion of power consumption on the user side in energy consumption is gradually increasing. The large-scale introduction of distributed generation and the diversification of user behavior characteristics pose new challenges to power dispatching planning. Because the power on the user side is mostly collected from the bus side, accurate prediction of bus load is of great reference significance to power planning. The bus load can be predicted by exploring the internal connection and development law between the load influencing factors and the bus load. Accurate load forecasting can be applied to many fields such as power system planning, market transactions, dispatching, etc., and serves as an important basis for the work of related departments. Electric energy consumption has become one of the most important modes of energy consumption in the world. The accurate prediction of electric energy consumption can also predict the future energy development trend, provide a basis for the development of renewable energy, and help the sustainable development of humans and the environment.

In recent years, the research on the theory of bus load forecasting has become increasingly mature, and its forecasting methods can be divided into three categories: statistical forecasting methods, intelligent forecasting methods, and combined forecasting methods. The statistical forecasting algorithm analyzes the time series based on the implicit time dependence and recursive relationship between the bus loads at different times, and then obtains the short-term bus load forecast values, including time series models, gray models,

**Citation:** Tang, J.; Zhao, J.; Zou, H.; Ma, G.; Wu, J.; Jiang, X.; Zhang, H. Bus Load Forecasting Method of Power System Based on VMD and Bi-LSTM. *Sustainability* **2021**, *13*, 10526. https://doi.org/ 10.3390/su131910526

Academic Editors: Nicu Bizon and Baojie He

Received: 2 August 2021 Accepted: 14 September 2021 Published: 23 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

etc. However, the time series model has a suitable prediction effect when the relationship between load and time is linear or exponential, but the relationship between bus load and time is irregular and does not have obvious linear or exponential relationship; thus, the application scenarios have certain limitations. The intelligent prediction method realizes prediction by extracting the features of the data. Typical representatives are Deep Neural Network (DNN) [1], Support Vector Machine (SVM) [2], long and short-term memory (LSTM) Network [3–5], DeepAR [6], N-BEATS [7], Transformer [8], etc. These methods have the ability to model the nonlinear load process, can better adapt to nonlinear spikes and more accurately model the data characteristics of the load, and have better forecasting accuracy. They have become the main research direction of short-term bus load forecasting in recent years.

With the increasing application of data preprocessing theories such as wavelet transform, empirical mode decomposition (EMD) [9,10], ensemble empirical mode decomposition (EEMD) [11], and variational mode decomposition (VMD) [12,13], in view of the nonlinear and non-stationary characteristics of the load sequence, the data preprocessing method is used to decompose the original sequence, and each sub-sequence is predicted separately, and the prediction result is obtained by superimposing and reconstructing. With the progress of data preprocessing methods, bus load forecasting has more processing methods, and the combined forecasting method has been developed. Combination forecasting methods can be divided into two categories. One is to weight and synthesize the forecast results of different forecasting methods to obtain the combined forecast. The forecast results are easily affected by weight distribution. The other is based on the nonlinearity of the bus load sequence non-stationary characteristics, using signal processing methods such as wavelet transform, EMD, and VMD to decompose the original sequence, separately model each sub-sequence component, and reconstruct its prediction results through superposition to obtain the combined prediction results that meet the accuracy requirements. Reference [14] uses EMD to decompose the bus load for multi-step prediction, which has achieved ideal prediction results, but EMD is prone to mode aliasing and cannot choose the number of decomposed components. VMD can find the optimal solution of the natural modal function model through repeated iterations within a limited number of times, which can effectively avoid modal aliasing and improve robustness. Reference [15] proposed four hybrid models based on four decomposition methods, EMD, VMD, wavelet packet transform, and intrinsic time-scale decomposition to forecast agricultural commodity futures prices. The results showed that VMD contributed the most in improving the forecasting ability.

In intelligent prediction methods, the forecasting model based on LSTM in the field of deep learning has great prediction performance. Reference [16] improved the multi-level gated LSTM prediction model to effectively improve the accuracy of bus load prediction. However, the hyperparameters of LSTM are artificially set before the machine learning model starts the learning process, rather than parameters such as weights and biases obtained by training. The choice of hyperparameters plays a vital role in the improvement of model performance. Therefore, it is necessary to carry out parameter adjustment work according to different application scenarios. Hyperparameter optimization methods mainly include grid search method, random search method, Bayesian optimization algorithm, and so on. Among them, the grid search method is an exhaustive search method that traverses the hyperparameter space. It has the disadvantages of being time-consuming and low efficiency when searching in the high-dimensional space. The random search algorithm avoids the above to a certain extent through sparse and simple sampling. However, it still has the disadvantage of not being able to use prior knowledge to select the next set of hyperparameters. The basic idea of Bayesian optimization algorithm is to use prior knowledge to approximate the posterior distribution of the unknown objective function and then adjust the hyperparameters, which significantly improve the efficiency and accuracy of search in high-dimensional space. The data preprocessing method is selected for noise reduction preprocessing before bus load curve prediction in order to obtain more stable

prediction results. In reference [17], the bidirectional long-term and short-term memory (Bi-LSTM) network is used to predict the bus load sequence, which effectively improves the learning ability of the network to historical data. However, with the combination prediction model dividing the complete sequence into multiple subsequences, not all sequences are suitable for bidirectional training. Therefore, it is necessary to use the optimization method to select the Bi-LSTM.

Based on this, a bus load forecasting method based on VMD and Bayesian Optimization Bi-LSTM is proposed in this paper. Firstly, VMD is used to stabilize the original bus load series, which is decomposed into a group of subsequence components with different frequencies. Then, the LSTM neural network prediction model of each subsequence component is constructed, the network related super parameters are optimized by Bayesian theory, and whether Bi-LSTM is used is judged to improve the prediction accuracy of a single model. Finally, the prediction results of each subsequence are superimposed to obtain the predicted value of bus load. In the second part of this paper, we expound the theoretical part of the proposed method and compare and verify the method through an example in the third part. The example results show that compared with the traditional algorithm, the prediction model constructed in this paper has a significant improvement in the accuracy of single-step prediction and multi-step prediction, and can better track the change trend of bus load.

The rest of the paper is organized as follows. Section 2 introduces the theories of methods that are used in the bus load forecasting method of power system, which includes VMD, LSTM, BiLSTM, and Bayesian optimization method, and establishes the VMD-BiLSTM combined prediction model. There is a case study in Section 3 to test the model and compare the model proposed in this paper with SVM, LSTM, Bayesian-LSTM, Bayesian-BiLSTM, and VMD-BiLSTM methods to prove its effectiveness. The conclusion and future studies are presented in Section 4.

### **2. Theoretical Framework**

Here, we mainly explain the theoretical framework of the bus load forecasting in this article.

### *2.1. Variational Mode Decomposition*

### 2.1.1. Construction of Variational Mode Decomposition Function

Variational modal decomposition (VMD) is an adaptive signal processing method proposed by Dragomiretskiy, which can be effectively applied to the smoothing processing of nonlinear and non-stationary time series [13]. It iteratively searches for the optimal solution of the variational mode, continuously updates each mode function and center frequency, and obtains a number of Intrinsic Mode Functions (IMF) with a certain bandwidth.

In the process of VMD, each natural mode is a finite bandwidth with a center frequency, so the variational problem can be defined as seeking *k* natural mode functions *uk*(*t*) and making the bandwidth of each mode is the smallest, and the sum of each mode is equal to the input signal *f*. The specific construction steps are as follows:

(1) Through the Hilbert transform, the analytical signal of the modal function *uk*(*t*) is obtained:

$$[\delta(t) + \frac{j}{\pi t}] \* u\_k(t) \tag{1}$$

Among them, *δ*(*t*) is the Dirichlet function, ∗ is the convolution symbol.

(2) Perform frequency mixing on the analytical signal to transform the frequency spectrum of each mode to the fundamental frequency band:

$$
\left[ \left( \delta(t) + \frac{\dot{j}}{\pi t} \right) \* \mu\_k(t) \right] e^{-j\omega\_k t} \tag{2}
$$

(3) The constraints of the optimized variational model are:

$$\begin{cases} \min \left\{ \sum\_{k=1}^{K} \left\| \partial\_{t} \left[ \left( \delta(t) + \frac{j}{\pi t} \right) \* u\_{k}(t) \right] e^{-j\omega\_{k}t} \right\|\_{2}^{2} \\ \qquad\qquad\text{s.t.} \sum\_{k=1}^{K} u\_{k}(t) = X(t) \end{cases} \tag{3}$$

In the formula, *K* is the number of IMFs, {*uk*}={*u*1, *u*2,..., *uk*} is IMFs, and {*ωk*}={*ω1*, *ω2*, ..., *ωk*} is the center frequency of *uk*.

### 2.1.2. Solution of Variational Mode Decomposition Function

(1) Using the quadratic penalty factor *α* and the Lagrangian multiplication operator *λ* (*t*), the constrained problem is turned into a non-constrained problem.

Extended Lagrangian function expression:

$$\begin{aligned} L(\{\boldsymbol{\mu}\_{k}\}, \{\boldsymbol{\omega}\_{k}\}, \boldsymbol{\lambda}) &= \sum\_{k=1}^{K} \left\| \partial\_{t} \left[ \left( \boldsymbol{\delta}(t) + \frac{\boldsymbol{j}}{\pi t} \right) \* \boldsymbol{\mu}\_{k}(t) \right] e^{-j\boldsymbol{\omega}\_{k}t} \right\|\_{2}^{2} \\ &+ \left\| X(t) - \sum\_{k=1}^{K} \boldsymbol{\mu}\_{k}(t) \right\|\_{2}^{2} + \left< \lambda(t), X(t) - \sum\_{k=1}^{K} \boldsymbol{\mu}\_{k}(t) \right> \end{aligned} \tag{4}$$

(2) Initialize *u*ˆ<sup>1</sup> *<sup>k</sup>*, *<sup>ω</sup>k*, *<sup>λ</sup>*<sup>ˆ</sup> 1:

Iteratively update *<sup>u</sup>*ˆ*k*, *<sup>ω</sup>k*, *<sup>λ</sup>*<sup>ˆ</sup> *<sup>n</sup>* under the condition of *<sup>ω</sup>* <sup>≥</sup> 0:

$$u\_{k}^{n+1}(\omega) = \frac{\hat{f}(\omega) - \sum\_{i \neq k} \theta\_i^{n+1}(\omega) + \frac{\hat{\lambda}^n(\omega)}{2}}{1 + 2a\left(\omega - \omega\_k^n\right)^2} \tag{5}$$

$$u\_{k}^{n+1}(\omega) = \frac{\int\_{0}^{\infty} \omega \left| \boldsymbol{\mathring{u}}\_{k}^{n+1}(\omega) \right|^{2} d\omega}{\int\_{0}^{\infty} \left| \boldsymbol{\mathring{u}}\_{k}^{n+1}(\omega) \right|^{2} d\omega} \tag{6}$$

$$
\hat{\lambda}^{n+1}(\omega) = \hat{\lambda}^n(\omega) + \left(\hat{f}(\omega) - \sum\_{k} \hat{\mu}\_k^{n+1}(\omega)\right) \tag{7}
$$

Until ∑*<sup>k</sup> u*ˆ *n*+1 *<sup>k</sup>* <sup>−</sup> *<sup>u</sup>*ˆ*<sup>n</sup> k* 2 2 / *u*ˆ*<sup>n</sup> k* 2 <sup>2</sup> < *ε*.

By using the VMD, different scales or trend components can be decomposed from the bus load sequence step by step to form a series of sub-sequence components with different time scales. The sub-sequences have stronger stationarity than the original series, and regularity help to improve forecast accuracy.

### *2.2. Long Short-Term Memory Neural Network*

### 2.2.1. LSTM Operation Rules

The long and short-term memory (LSTM) network is a special modified version of the cyclic neural network. While retaining the cyclic feedback mechanism, the topology of the LSTM network controls the accumulation speed of information by introducing a gating unit, selectively adding new information, and selectively forgetting. The previously accumulated information solves the long-term dependence problem in sequence modeling.

Compared with ordinary RNN, LSTM neural network is also composed of an input layer, output layer, and hidden layer, but its hidden layer is replaced by ordinary neurons with memory modules containing gating mechanism. Its internal structure is shown in the Figure 1.

**Figure 1.** Internal structure of LSTM.

The memory unit *ct* is the core component of the LSTM memory module. It is realized by controlling the forget gate, input gate, and output gate. It contains the long-term memory information of the sequence. The hidden layer state *ht* contains the short-term memory information of the sequence, which is updated faster than memory unit update speed [16].

Suppose a total of *k* time steps of the input vector sequence are divided into *x*1, *x*2, ..., *xk* according to the input time sequence, and the *t*-th time step is taken for analysis.

The LSTM operation rules are as follows:

(1) Update the output of the forget gate, select the historical information that the memory unit needs to keep, and control the degree of influence of *ct*−<sup>1</sup> on *ct*:

$$f\_t = \sigma(\mathbb{W}\_f \mathbf{x}\_t + \mathbb{U}\_f \mathbf{h}\_{t-1} + \mathbf{b}\_f) \tag{8}$$

(2) Update the two parts of the output of the input gate, select the current input information that the memory unit needs to retain, and control the degree of influence of *xt* on *ct*:

$$f\_t = \sigma(\mathcal{W}\_f \mathbf{x}\_t + \mathcal{U}\_f \mathbf{h}\_{t-1} + \mathbf{b}\_f) \tag{9}$$

$$\widetilde{x}\_t = \tanh(\mathcal{W}\_c \mathbf{x}\_t + \mathcal{U}\_c \mathbf{h}\_{t-1} + \mathbf{b}\_c) \tag{10}$$

(3) Update the cell status according to the input gate and forget gate:

$$\mathfrak{c}\_{t} = f\_{t} \circ \mathfrak{c}\_{t-1} + \mathfrak{i}\_{t} \circ \mathfrak{c}\_{t} \mathfrak{t} \tag{11}$$

(4) Update the output gate output, select the output information that the memory unit needs to retain, and control the degree of influence of *ct* on *ht*:

$$\rho\_t = \sigma \left( \mathbb{W}\_0 \mathbb{x}\_t + \mathbb{U}\_0 \mathbb{h}\_{t-1} + \mathbb{h}\_0 \right) \tag{12}$$

$$h\_t = o\_t \circ \tanh(c\_t) \tag{13}$$

(5) Update the forecast output at the current moment:

$$
\mathfrak{H}\_{\mathfrak{l}} = \sigma(Vh\_{\mathfrak{l}} + \mathfrak{c})\tag{14}
$$

Among them, *ft*, *it*, and *ot* represent the calculation results of the forget gate, input gate, and output gate at time *t*; *Wf*, *Wi*, and *Wo* represent the weight matrix of the forget gate, input gate, and output gate, respectively; *bf*, *bi*, and *bo* represent the weight matrix of the forget gate, input gate, and output gate, respectively. The bias term of the forget gate, input gate, and output gate; *o* is the dot product symbol of the matrix element; *σ*(*x*) is the Sigmoid activation function of each gate:

$$\sigma(\mathbf{x}) = \frac{1}{1 + \varepsilon^{-1}} \tag{15}$$

Its value range is (0, 1). Through the conversion of Sigmoid function, the input can be converted into probability values, so it is widely used as the activation function of artificial neural network transmission.

When *xt* is input to the network, it will be processed by a tanh neural layer and three gates at the same time as the hidden layer vector *ht*−<sup>1</sup> of the previous time step. The tanh function is a hyperbolic tangent activation function, and its expression is:

$$\tanh(x) = \frac{\sinh(x)}{\cosh(x)} = \frac{e^{2x} - 1}{e^{2x} + 1} \tag{16}$$

Its value range is (−1, 1), the output is centered at the origin, and the convergence speed is faster than that of Sigmoid. It is usually used as the activation function of the output gate.

The tanh layer will create a new candidate state vector *ct* . The forgetting gate *ft* determines what information to discard and retain from the cell state *ct*−<sup>1</sup> of the previous time step. The input gate it determines how to update the candidate state vector. After the cell state is updated, the output gate *ot* decides how to filter the new state vector *ct* into output information *ht*.

### 2.2.2. Training Process of LSTM

The training algorithm of LSTM neural network mainly includes two categories: back propagation algorithm over time and real-time cyclic learning algorithm. The concept of backpropagation algorithm is clear and it has advantages in computational efficiency. This paper selects this method to train LSTM neural network.

The back propagation method expands the LSTM into a deep feedforward neural network in time sequence, and then further calculates the relevant parameter gradients according to the error back propagation algorithm of the feedforward network, and trains the model [18]. The LSTM network sequence expansion diagram is shown in Figure 2.

**Figure 2.** Sequence structure diagram of LSTM.

The specific training steps are as follows in Figure 3.

**Figure 3.** Training steps of LSTM.

(1) Forward calculation of the output value of the LSTM memory module.

The *ct* and *ht* of the current time step are calculated by the gating mechanism of LSTM and are retained for the calculation of the next time step. When the calculation of the last step is completed, the hidden layer vector *hk* will be used as the output and the prediction corresponding to this set of sequences' value (tag value) to compare.

(2) Reverse calculation of the error item value of each memory module, including two reverse propagation directions in chronological order and at network level.

Calculate the loss function value according to the comparison result of step 1, and select the square sum error function as the loss function of LSTM; the expression is:

$$L = \frac{1}{2} \sum\_{i \in outputs} (y\_i - y\_i)^2 \tag{17}$$

(3) According to the corresponding error term, calculate the gradient of each weight, and the gradient descent method iterates the parameters including *W*, *U*, *V*, *ct*, and *ht*:

$$
\hat{\theta}\_{\dot{\jmath}} = \theta\_{\dot{\jmath}} - \alpha \frac{\partial}{\partial \theta\_{\dot{\jmath}}} L(\theta\_{\dot{\jmath}}) \tag{18}
$$

Through the gating mechanism and perfect parameter update rules, LSTM realizes the selection and screening of the input information flow, and improves the processing ability of the recurrent neural network for long sequences.

The back-propagation algorithm can compare the predicted value with the real value by using the error function after the forward calculation, and then optimize the network parameters. Therefore, the back-propagation algorithm can back calculate and optimize many parameters of LSTM. The cyclic learning algorithm can deal with some sequence problems, but it has serious long-term dependence problems after multi-stage propagation—the gradient tends to disappear or explode, and it is difficult to continue to optimize within the number of iterations in many cases. However, the introduction of gating mechanism in LSTM solves the gradient disappearance problem and performs well in sequence processing.

### 2.2.3. Bidirectional LSTM

The bidirectional long and short-term memory (Bi-LSTM) network is derived from the bidirectional cyclic neural network [19]. Its main feature is to increase the learning function of the neural network for future information, thereby overcoming the defect that the unidirectional LSTM network can only process historical information. The Bi-LSTM mainly splits the ordinary LSTM into two directions, but the two LSTMs are connected to the same output layer. Such a structure can provide complete upper and lower sequence information for the input sequence of the output layer. Using a Bi-LSTM network to model bus load forecasting is to input historical data into a forward LSTM network and a reverse LSTM network at the same time, so as to capture a complete time series global information. The Bi-LSTM neural network structure diagram is as follows in Figure 4.

**Figure 4.** Bi-LSTM Neural Network Structure Diagram.

Because the bus load has certain regularity and planning, the joint grasp of historical and future information can better learn the characteristics and laws of the load curve. The update formula of the back-to-forward cycle neural network layer is:

$$\mathbf{h}\_{1,t} = f\left(\mathbb{W}\_{h\_1,t}\mathbf{x}\_t + \mathbb{W}\_{h\_1}h\_{1,t-1} + b\_{h\_1}\right) \tag{19}$$

The update formula of the looping neural network layer from front to back is:

$$h\_{2,t} = f\left(\mathbb{W}\_{h\_2,t}\mathbf{x}\_t + \mathbb{W}\_{h\_2}h\_{2,t+1} + b\_{h\_2}\right) \tag{20}$$

The two layers of recurrent neural networks are superimposed and input to the hidden layer:

$$\lg y\_1 = \lg \left( l L\_{h\_1} h\_{1,t} + l L\_{h\_2} h\_{2,t} + b\_{\mathcal{Y}} \right) \tag{21}$$

Among them, *h*1, *t*, and *h*2, *t* are the hidden units of the front pass layer and the back pass layer at time *t*, respectively; *yt* is the model output at time *t*; *f*(\*), *g*(\*) are optional activation functions; *Wh*1,*t*, *Wh*2, *t*, *Wh*1, *Wh*2, *Uh*1, and *Uh*<sup>2</sup> are the weight matrices of the corresponding objects; and *bh*1, *bh*2, and *by* are the bias terms of the corresponding objects.

### *2.3. Bayesian Optimization of LSTM*

2.3.1. Bayesian Optimization Theory

Gaussian Regression Process

In the feasible region, uniformly select points that obey the multi-dimensional normal distribution as candidate solutions to establish a Gaussian regression model [20]:

$$
\begin{bmatrix} y\_1 \\ \vdots \\ y\_n \end{bmatrix} \sim N \left( 0, \begin{bmatrix} k(\mathbf{x}\_1, \mathbf{x}\_1) & \cdots & k(\mathbf{x}\_1, \mathbf{x}\_n) \\ \vdots & & \vdots \\ k(\mathbf{x}\_{\mathcal{U}}, \mathbf{x}\_1) & \cdots & k(\mathbf{x}\_{\mathcal{U}}, \mathbf{x}\_{\mathcal{U}}) \end{bmatrix} \right) \tag{22}
$$

Among them, *K* is the covariance matrix, *x* is the input value, and *y* is the response output value.

Through the training set, the updated value *y*\* is obtained according to the posterior formula:

$$P\left(y\_\*\left|y \sim N\left(K\_\*K^{-1}y, K\_{\*\*} - K^{-1}K\_\*^T\right)\right)\right) \tag{23}$$

Among them, *K*<sup>∗</sup> is the covariance of the training set, and *K*∗∗ is the covariance of the newly added sample.

Update the Gaussian regression model according to the updated value:

$$
\begin{bmatrix} y \\ y\_\* \end{bmatrix} \sim N\left( 0, \begin{bmatrix} K & K\_\*^T \\ K\_\* & K\_{\*\*} \end{bmatrix} \right) \tag{24}
$$

The Gaussian regression model considers the relationship between *yN* and *yN*+1 and establishes input and output functions to provide a search basis for parameter optimization.

### Acquisition Function Process

On the basis of the Gaussian regression model, it is necessary to use the collection function to solve the optimal solution of the function. This paper selects the expectation improvement method to use the mathematical expectation to solve the optimal solution.

$$\mathbf{x}\_{n+1} = \mathbf{a}\mathbf{g}\mathbf{m}\mathbf{a}\mathbf{x}E I\_{\mathbf{n}}(\mathbf{x})\tag{25}$$

$$EI\_n(\mathbf{x}) = E\_n \left[ \left[ f(\mathbf{x}) - f\_n^\* \right]^+ \right] \tag{26}$$

$$= (\mu - f\_n^\*) \left( 1 - \phi \left( \frac{f\_n^\* - \mu}{\sigma} \right) \right) + \sigma \phi \left( \frac{f\_n^\* - \mu}{\sigma} \right) \tag{27}$$

Among them, *φ*(*x*) is the probability density of the normal distribution, *ϕ*(*x*) is the standard normal distribution about *x*, *μ* is the mean value of the input value *x*, and *σ* is the variance of the input value *x*.

The expected value is used for parameter optimization, and the next optimal value is effectively sought within a certain range, so as to find the optimal parameter within the number of iterations [20].

### 2.3.2. Hyperparameter Optimization of LSTM

The hyperparameters of the LSTM neural network used for bus load prediction can be divided into two categories: structural hyperparameters and training hyperparameters. The structural hyperparameters mainly include the number of hidden neurons in the network, etc [21]. The number of hidden layer neurons determines the expressive ability of the network, but also determines whether the network is over-fitting and the network is time-consuming. Reasonable hidden layer neurons help improve the performance of the network's predictive ability. The use of the bidirectional long-term memory network improves the network's ability to learn historical data, but it also leads to slow network prediction and different effects. Choosing different curves to predict whether to use a Bi-LSTM can save prediction time while ensuring prediction accuracy [22].

The training hyperparameters of the LSTM neural network mainly include learning rate, L2 regularization parameters, etc. A suitable initial learning rate helps to significantly improve the iterative convergence speed and prediction accuracy of deep learning models. L2 regularization parameters are additional items of the network loss function and can prevent the over-fitting problem to a certain extent.

In summary, this paper will use Bayesian optimization algorithm to optimize and debug the number of hidden layer neurons of LSTM neural network, whether to use Bi-LSTM, initial learning rate, and L2 regularization parameters.

### *2.4. VMD-Bi-LSTM Combined Prediction Model*

The power system bus load itself has fluctuating characteristics and is affected by distributed power dispatch and user-side behavior characteristics. Its curve has a certain degree of non-linear and non-stationary characteristics. The use of conventional learning forecasting methods to improve the forecasting accuracy is relatively limited. Considering the outstanding advantages of variational modal decomposition technology in sequence smoothing processing and the excellent performance of LSTM networks in time series data modeling, this paper proposes a VMD-Bi-LSTM combined forecasting model. The specific modeling process is as follows, as shown in the Figure 5.

(1) In view of the non-stationary characteristics of the bus load sequence, the VMD method is used to decompose, and each IMF component and residual component are obtained;

(2) Normalize each sub-sequence component separately, and divide the training sample and the test sample according to the same ratio;

(3) Construct an LSTM neural network prediction model for each sub-sequence component, and use Bayesian optimization algorithm to optimize the hyperparameters of a single model to obtain the most suitable hyperparameter combination for decomposing the sequence and determine whether to use Bi-LSTM in sub-sequences;

(4) Train the prediction model after hyperparameter optimization, use the trained prediction model to perform multi-step extension prediction, and superimpose the reconstruction to obtain the multi-step prediction value of bus load;

(5) Compared with actual data, the multi-step prediction performance of the prediction model is evaluated by calculating error indicators.

### **3. Case Study**

Here, we analyze the example of the method proposed in this paper. The example takes the bus load data of Canberra, Australia, from 16 January to 22 January 2016 as the data set, including 270 time steps (30 min as a time step). The first 244 time steps of the data are used as the training sequence and the last 36 time steps are used as the verification sequence.

### *3.1. Temporal Data Decomposition*

The original bus load sequence is decomposed byVMD, and six groups of IMF components and one group of residual components are separated step by step. The decomposition results are shown in Figure 6.

**Figure 6.** (**a**) Original sequence and IMFs; (**b**) IMFs and residual.

Decomposing the bus load sequence through variational modal decomposition shows that the natural modal function consists of multiple sub-sequences with small amplitude and stable frequency. Although the frequency of the remainder is relatively unstable, the amplitude is small, which affects the overall bus load. The forecast trend change of the load has less impact.

### *3.2. Hyperparameter Optinization of VMD-LSTM*

On the basis of smoothing the original sequence, it is necessary to construct the LSTM network prediction model of the sub-sequence components, and to optimize the related structure hyperparameters and training hyperparameters. The hyperparameter results obtained by using Bayesian optimization algorithm in this paper are shown in the Table 1.


**Table 1.** Hyperparameters of each subsequence.

### *3.3. Model Evaluation Index*

In order to visually analyze the predicted error value, the curve correlation coefficient is introduced to evaluate the shape difference between the predicted curve and the real curve, and the root mean square error (RMSE) is introduced to visually analyze the deviation between the observed value and the true value. The accuracy is analyzed, and the standard error is selected as the criterion for predicting the dispersion level.

In order to evaluate the forecasting effect of the proposed bus load forecasting method, the RMSE, NRMSE, standard error, and correlation coefficient are selected as the overall forecasting results evaluation index of the short-term bus load forecasting method.

$$
\sigma\_{RMSE} = \sqrt{\frac{1}{n} \sum\_{i=1}^{n} (y\_i - \hat{y}\_i)^2} \tag{28}
$$

$$x\_{NRMSE} = \sqrt{\frac{RMSE}{mean\left(\sum\_{i=1}^{n} y\_i\right)}}\tag{29}$$

$$e\_{STD} = \sqrt{\frac{\sum\_{i=1}^{n} \left(y\_i - \mathfrak{g}\_i\right)^2}{n}}\tag{30}$$

$$\rho(A,B) = \frac{1}{N-1} \sum\_{i=1}^{N} \left( \frac{\overline{A\_i - \mu\_A}}{\sigma\_A} \right) \left( \frac{B\_i - \mu\_B}{\sigma\_B} \right) \tag{31}$$

where *σ<sup>A</sup>* and *σ<sup>B</sup>* are the standard deviation of dataset *A* and dataset *B*, respectively, and the standard deviation formula is as follows:

$$
\sigma\_x = \sqrt{\frac{\sum\_{i=1}^n (\mathbf{x}\_i - \overline{\mathbf{x}})^2}{n-1}} \tag{32}
$$

where *yi* is the true value of bus load and *y*ˆ*<sup>i</sup>* is the predicted value of bus load. The correlation coefficient *r* is used to characterize the accuracy of the prediction curve and the calibration curve. The closer the correlation coefficient is to 1, the higher the prediction accuracy.

### *3.4. Model Evaluation Index*

After optimizing the Bayesian parameters of each sub-sequence of the bus load, the long and short-term memory network is trained, and the next time step is predicted in Figure 7, and the single-step prediction results of each sub-sequence component are integrated into the historical monitoring data. The new input sequence of the single-step forecasting model can realize the multi-step rolling forecast of each component, which can realize the rolling forecast load value for a period of time in the future. The rolling prediction method is used to predict multiple time steps in the future, and the prediction results are shown in Figure 8.

**Figure 7.** Subsequence training prediction results.

**Figure 8.** Subsequence multi-step prediction results.

The multi-step prediction results of bus load can be obtained by superimposing the predicted values of each subsequence, and the error analysis of each subsequence is carried out by using the prediction evaluation index. The error results are shown in Table 2. From the training prediction results and multi-step prediction results, it can be seen that the *eRMSE* of IMF6, which accounts for a large proportion of the original sequence of bus load, is 1.3635 and *eSTD* is 0.8233 in the next 12 time steps, with small prediction error. With the increase in prediction time steps, the prediction *eRMSE* in the next 36 time steps will increase to 20.2677, *eSTD* and *eNRMSE* will also increase accordingly, but the *r* will increase from 0.9521 to 0.992. The shape of the prediction curve will gradually stabilize and be closer to the shape of the original curve. With the increase in time steps, the prediction accuracy will decline, but the prediction curve will gradually remain stable; the shape gradually approaches the original curve, which has suitable prediction stability.


**Table 2.** Performance comparison of multi-step prediction.

The prediction *eNRMSE* of other subsequences is lower than 16, the *r* is higher than 0.98, the prediction error is small, and the prediction shape remains suitable. The prediction results of a single stable natural mode function subsequence meet expectations.

The decomposition remainder of bus load has many burrs and unstable frequency, so the prediction correlation coefficient is low. However, due to its small amplitude, its prediction error is also small, and its impact on the overall prediction result of bus load is relatively small.

The multi-step prediction results of each subsequence and remainder are superimposed to obtain the multi-step prediction curve of bus load, as shown in Figure 9. The prediction curve fits with the real curve and has accurate prediction results. In order to verify the prediction performance of the proposed method, different models in various cases are selected for comparative analysis. The SVM that uses radial basis function as kernel function for prediction is compared with LSTM to verify the advantages of LSTM in time series prediction. The VMD-Bayesian-BiLSTM model is compared with EEMD-Bayesian-BiLSTM model and EMD-Bayesian-BiLSTM to verify the advantages of VMD in short-term combination forecasting. The VMD-LSTM combined prediction model and LSTM model are selected for prediction to verify the prediction accuracy and stability of the combined prediction model. The VMD-Bayesian-LSTM model is compared with VMD-LSTM model, and the Bayesian-BiLSTM model is compared with the Bayesian-LSTM model and LSTM to verify the improvement effect of Bayesian optimization theory on LSTM prediction accuracy and the necessity to consider the applicability of BiLSTM in sequence prediction. The comparison results are shown in Table 3 and Figure 10.

**Figure 9.** Prediction results of VMD-BiLSTM.



The comparison of different models shows that the prediction accuracy of each time step of LSTM is greatly improved compared with SVM. After decomposing the bus load sequence, the *eRMSE*, *r* and other indices of VMD-LSTM are greatly improved on the basis of the LSTM model, which greatly improves the prediction accuracy and stability. The comparison of VMD-Bayesian-BiLSTM and EEMD-Bayesian-BiLSTM and the comparison of VMD-Bayesian-BiLSTM and EMD-Bayesian-BiLSTM shows that the prediction error of the prediction model considering VMD in multiple time steps is lower than that of EMD and EEMD. With the proposal of Bayesian optimization theory, the *eRMSE* of 36 time steps of combined prediction is reduced from 23.9219 to 14.9219, the *eNRMSE* is reduced from 0.0088 to 0.0055, the *eSTD* is reduced from 20.9806 to 14.8022, and the results of LSTM are also greatly improved after Bayesian optimization. The results of the Bayesian-BiLSTM model are improved compared with those without considering Bi-LSTM. The comparison of various data shown in Figure 10 verifies that the VMD-LSTM combined prediction

model based on Bayesian optimization proposed in this paper has more accurate prediction results and more stable multi-step prediction results.

**Figure 10.** (**a**) Comparison of *eRMSE*; (**b**) comparison of *r*; (**c**) comparison of *eNRMSE*; (**d**) comparison of *eSTD*.

In order to verify the applicability of the model proposed in this paper, this paper randomly selects two groups of 270 time step data sets in the bus load data set in Canberra, Australia, in 2016, which are divided into data set 2 and data set 3 for verification. In order to eliminate the regional influence, three groups of 270 time step data sets are randomly selected from the 2014 Beijing bus load data set, namely data set 4, data set 5, and data set 6, and their errors are tested in Table 4.

**Table 4.** Test of different data sets.



**Table 4.** *Cont*.

It can be seen from Table 4 that through the test of multiple groups of randomly selected data in the same time step, the error value is always controlled in a low range, and the average error of six groups of data has suitable prediction results. This shows that the proposed method has appropriate prediction stability and adaptability.

### **4. Conclusions and Future Studies**

Aiming at the current research hotspot in the field of deep learning, this paper studies the bus load forecasting, establishes the bus load forecasting method based on VMD-BiLSTM, and draws the following conclusions:

(1) The VMD method is used to deal with the non-stationary characteristics of bus load series and reduce the interaction between different time scale information, which is conducive to further mining the characteristics of original series and improving the prediction performance of the model;

(2) The cyclic network structure and gating mechanism of LSTM neural network are used to capture the temporal correlation of each sub sequence component, so as to track the change trend of bus load more effectively. Compared with other models, the VMD-LSTM combined prediction model has a significant improvement in multi-step prediction accuracy;

(3) Bayesian optimization algorithm is used to optimize the super parameter combination of LSTM neural network to overcome the adverse effect of empirical selection on the improvement of model prediction performance;

(4) Bayesian optimization considers the applicability of the sequence to the bidirectional neural network. While using the Bi-LSTM network to enhance the training ability, considering the applicability of the sequence, the prediction performance of the network has been optimized and improved.

This method has certain feasibility in the field of bus load forecasting, can be applied in the direction of energy consumption forecasting and power production planning, and is conducive to the planning and development of clean energy in the future and the sustainable development of energy.

The model proposed in this paper has achieved suitable results in short-term prediction, but it is uncertain whether it can ensure the prediction accuracy and stability in long-term prediction when the data support is sufficient, and whether it is adaptable in other areas. Due to time constraints, this paper does not compare it with ETS, ARIMA, and Thet to reflect the optimization performance. In the future, this proposed model can be implemented in different areas to validate its effectiveness and compared to alternative approaches and stronger baselines. Moreover, by taking a new variety of data input for sustainability studies, the model can be implemented for carbon emission forecasting. The author will study and optimize the excellent prediction methods in the field of machine learning in order to establish an accurate prediction model that can be applied in the field of sustainability.

**Author Contributions:** Conceptualization, methodology, software, writing—original draft preparation, validation, formal analysis, investigation, J.T.; resources, H.Z. (Hongliang Zou), X.J., H.Z. (Huaixun Zhang); data curation, J.Z., G.M.; writing—review and editing, J.Z., J.W. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by State Grid Zhejiang Electric Power Co., Ltd. Science and technology project: Research on bus load forecasting method in spot market (5211TZ1900S4).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data sharing not applicable.

**Acknowledgments:** This work was supported by State Grid Zhejiang Electric Power Co., Ltd. Science and technology project: Research on bus load forecasting method in spot market (5211TZ1900S4). Thanks for the information and data provided by power system company.

**Conflicts of Interest:** The authors declare that there is no conflict of interests regarding the publication of this paper.

### **Nomenclature**


### **References**


### *Article* **The Good, the Bad and the Future: A SWOT Analysis of the Ecosystem Approach to Governance in the Baltic Sea Region**

**Savitri Jetoo \* and Varvara Lahtinen**

Faculty of Social Sciences, Business and Economics, Åbo Akademi University, Tuomiokirkontori 3, 20500 Turku, Finland; varvara.lahtinen@abo.fi

**\*** Correspondence: savitri.jetoo@abo.fi

**Abstract:** The ecosystem approach has been used extensively as a guiding principle in water policies of the Baltic Sea Region since the 1970s. In addition to its operationalization as a management framework in this region, it also has expansive theoretical underpinnings. However, despite extensive literature on this approach, there has not yet been any systematic assessment of the internal and external factors that influence its implementation. This kind of assessment could form the basis for improved thinking around the concept and better implementation actions. As such, this article presents a Strengths, Weaknesses, Opportunities and Threats (SWOT) analysis of the ecosystem approach in the Baltic Sea Region by using content analysis on Baltic Sea documents. This study found that key strengths of the principle are its interdisciplinary focus and its acceptance as a framework for conservation, whilst resource intensiveness and its operational complexity are key weaknesses. The SWOT analysis revealed that a key opportunity in the external environment is the ease of alignment with other policies whilst the key external threat is the difficulty integrating disciplines. This study showed that with a streamlined allocation of resources, more stakeholder engagement through capacity building and political leadership, the ecosystem approach could facilitate interdisciplinary knowledge pooling to achieve a good ecological status of the Baltic Sea.

**Keywords:** ecosystem approach; Baltic Sea; SWOT analysis; implementation; water policy; water governance; barriers

### **1. Introduction**

Humans constantly increase the stressors against the global marine ecosystems that, together with climate change, lead to severe loss of biodiversity, habitats and resources. It has already become apparent from the middle of the 20th century that traditional management practices could not address the ever-increasing loss of biodiversity [1–3]. The Ecosystem Approach (EA) is a concept that arose by the end of the 1980s, initially endorsed for the management of terrestrial ecosystems [1,2] to counteract the increasing loss of biodiversity. It emphasized the integration of information into management practices, the involvement of stakeholders in the decision-making process and the consideration of environmental relationships in different spatial, organizational and biological scales [2].

The term Ecosystem Management, conceptually similar to the ecosystem approach, was used to signify an ecosystem approach to resource management at the end of the 1980s [2]. Another similar concept, ecosystem-based management, emerged in the 1950s for managing terrestrial ecosystems [3–5]. The usage of many different terms (e.g., ecosystem approach, ecosystem-based management, ecosystem management, integrated management, etc.) in reference to similar concepts created confusion among scientists and practitioners [6,7]. Many researchers use ecosystem (-based) management and ecosystem approach interchangeably [5,6,8]. It is argued that there is a difference between the ecosystem approach and ecosystem management with the first being a vision and simultaneously a working plan for its realization, while the second is judged in human terms and may be altered [9].

**Citation:** Jetoo, S.; Lahtinen, V. The Good, the Bad and the Future: A SWOT Analysis of the Ecosystem Approach to Governance in the Baltic Sea Region. *Sustainability* **2021**, *13*, 10539. https://doi.org/10.3390/ su131910539

Academic Editors: Baojie He, Ayyoob Sharifi, Chi Feng and Jun Yang

Received: 22 August 2021 Accepted: 18 September 2021 Published: 23 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

The internationalization of the ecosystem approach occurred through the Convention of Biological Diversity (CBD), which adopted it as the framework for the implementation of its objectives [10]. According to the COP V/6 of the CBD, the ecosystem approach is "a strategy for the integrated management of land, water and living resources and promotes conservation and sustainable use in an equitable way" [11]. Furthermore, the CBD provided a list of 12 interlinked principles (Malawi principles) that characterize the ecosystem approach, and five points were proposed for guiding the operationalization of EA [11].

Regional Sea Conventions such as the Commission of the Convention on the Protection of the Marine Environment of the North-East Atlantic (OSPAR) and the Helsinki Commission (HELCOM) adapted the definition to the needs of their regional marine ecosystems. They jointly adopted "a vision of an ecosystem approach to managing human activities impacting on the marine environment (an "ecosystem approach") in their maritime areas" [6,12]. They defined the ecosystem approach as "the comprehensive integrated management of human activities based on the best available scientific knowledge about the ecosystem and its dynamics, in order to identify and take action on influences which are critical to the health of marine ecosystems, thereby achieving sustainable use of ecosystem goods and services and maintenance of ecosystem integrity" [12]. In 2007, HELCOM adopted the ecosystem approach in its Baltic Sea Action Plan (BSAP) [13] as it was defined in the first joint ministerial meeting of the Helsinki and OSPAR Commissions in 2003, and it calls for its implementation in the management of the Baltic Sea Region.

EA was also adopted by European Union (EU) policy documents. The Water Framework Directive (WFD) of 2000 did not explicitly mention the ecosystem approach but calls for the administrative reorganization of the water institutions to the appropriate scales according to the ecological borders [14]. The Marine Strategy Framework Directive (MSFD), adopted in 2008, explicitly refers to the application of the ecosystem-based approach to the management of human activities in the marine strategies of EU members for attaining good environmental status in the European marine regions and sub-regions [15]. However, the MSFD does not provide a definition for the ecosystem-based approach. The same term is also to be found in the EU Maritime Spatial Planning Directive (MSPD) of 2014, where the ecosystem-based approach will "contribute in the sustainable development and growth of the maritime and coastal economies and the sustainable use of marine and coastal resources" [16]. Furthermore, it identifies the maritime spatial planning as a tool for the application of the ecosystem-based approach in attaining good environmental status [16].

The ecosystem approach has been studied extensively globally but also within the Baltic Sea Region, which is this research's focus. One line of research focuses on the ecosystem approach to fisheries (EAF) or ecosystem-based fisheries management (EBFM) [17–21] that is not a fully cross-sectoral ecosystem approach but instead applies ecosystem-based policies within an individual sector [5]. Another line of research discusses appropriate ecosystem indicators to be developed for the implementation of ecosystem-based management [22–25]. A large part of the literature discusses frameworks as well as guidelines and the sequence of steps for the implementation of the ecosystem approach/management [5,26–28].

Against this backdrop of extensive literature discussing different aspects of EA, there has not yet been any systematic assessment of the internal and external factors that influence EA implementation in the Baltic Sea Region. This article intends to fill this gap by presenting a Strengths, Weaknesses, Opportunities and Threats (SWOT) analysis of the ecosystem approach in the Baltic Sea Region. The analytical framework itself guides the research questions of this study: what are the internal strengths and weakness of the ecosystem approach and what are the external opportunities and threats for the ecosystem approach in the Baltic Sea Region?

The purpose of this article is to provide a clear view of the key features that can promote or hinder the implementation of EA in the Baltic Sea Region and the achievement of good environmental status but also to inform the ongoing policy processes such as the updating of HELCOM's Baltic Sea Action Plan.

### **2. Materials and Methods**

This is a qualitative research process as illustrated in Figure 1. The SWOT and content analysis and further details of the research method are elaborated in the following sections.

**Figure 1.** The research process.

### *2.1. The SWOT Analysis*

The term SWOT derives from business literature and refers to Strengths, Weaknesses, Opportunities and Threats (SWOT) analysis. It is used as an assessment tool for strategic planning to position organizations to achieve their stated goals. The SWOT analysis is an effective tool for taking reconnaissance of the organization's resources and capabilities, its deficiencies, market opportunities and threats to its future [29]. The SWOT analysis views the performance of a given organization with respect to its goals and varies with the internal characteristics of that organization and the broader external context in which that organization must operate [30]. Strengths are internal characteristics that position the organization favorably towards the achievement of its goals, whereas weaknesses are internal factors that can detract from goal realization. Conversely, opportunities and threats are factors in the external environment that can facilitate or thwart the achievement of goals, respectively. These factors can include political, economic, social, technological, legal and environmental factors.

The SWOT analysis is a useful framework for analysis of the use of the ecosystem approach in the Baltic Sea Region as its value stems from the way the features of the concept interact in the external and internal environments to achieve its success. Its usefulness also arises from its ability to be used in the identification of barriers to success and the subsequent implementation of long-term strategies to achieve its objectives [31]. When applied to the ecosystem approach, the SWOT analysis can be applied to the features of the concept of the ecosystem approach that allow for the realization of its goals [32], to:


In this context, strengths can be seen as the features of the ecosystem approach that allow for the achievement of the above stated goals, whilst weaknesses are characteristics of the approach that compromise the realization of these goals. As such, strengths and weaknesses are internal attributes of the ecosystem approach itself whilst opportunities and threats are external features that retard the realization of the above goals. In this case, external features are examined with respect to the Baltic Sea environment and the achievement of the good ecological status of the Baltic Sea. More specifically, the political, economic, sociological, technological, legal and environmental (later the PESTLE framework) factors within the implementation framework of the ecosystem approach in the context of the Baltic Sea Action Plan are considered. This is illustrated in Figure 2, with research questions included.

**Figure 2.** Research questions showing the internal and external SWOT environments.

### *2.2. The Content Analysis*

The qualitative content analysis interprets the text's data content subjectively by using the systematic classification method of coding in the identification of patterns or themes [33]. It has an interpretive character that enables discussion on the points and themes that are essential to the study's research goal [34]. As a qualitative method, it has many advantages such as flexibility and applicability in a plethora of text types, being rather "safe" [35] and having "less rules to follow" by not being strictly linked to any specific science [36]. The disadvantages, on the other hand, are the inability to study other types of communication (e.g., body language, etc.) but the recorded communication-text, the potential loss of useful data due to restricted access or altogether missed data, its complexity and the fact that it is labor intensive and time consuming [35].

Qualitative content analysis was deemed the most appropriate method to be used in this research for a number of reasons. Firstly, due to the data used being existing documents and communications; secondly, due to their relatively limited numbers that renders their study attainable through qualitative content analysis and thirdly, due to qualitative content analysis being the most appropriate research method in answering the posed research questions.

The documents studied in this research were mainly the minutes from the meetings of the Group for the Implementation of the Ecosystem Approach (GEAR) [37]. GEAR is one of HELCOM's working groups, established in 2012 for coordinating the actions in implementing both the Baltic Sea Action Plan (BSAP) and the Marine Strategy Framework Directive (for the Contracting Parties also EU Member States). It also has oversight of pertinent global processes with the overarching goal being the implementation of EA and the achievement of good environmental status in the marine waters of the Baltic Sea [38]. By August 2021, the GEAR Group had 24 meetings. All 24 GEAR minutes and accompanying documents were studied but four of these minutes-documents were chosen specifically for coding analysis. These were the minutes of the GEAR 1 meeting (2012) [39], GEAR 13 (2016) [37], GEAR 18 (2018) [37] and GEAR 24 (2021) [37]. The GEAR 1 [39] and 24 [37] minutes were chosen to observe the changes the Group has undergone within the 9 years of operating. The GEAR 13 of 2016 [37] was chosen as the year when the EU Member States (also Contracting Parties) had to establish and implement Programmes of Measures (PoMs) in order to achieve Good Environmental Status (GES), set in Article 13 of MSFD (MSFD 2008); it was therefore interesting to observe how this mandate was met through the GEAR meeting. Finally, the GEAR 18 of 2018 [37] was chosen as the year of the beginning of the second cycle of MSFD implementation and therefore interesting to note how this process was reflected in the GEAR meeting.

The coding process began after having collected the documents. It is acknowledged that a successful qualitative analysis is the outcome of a well-thought out and well-executed coding process [33]. Therefore, the focus was kept on the research questions in order to avoid too broad or too narrow coding that would not answer them. The coding process did not aim at generating any theory, and therefore a deductive use of existing theoretical framework took place. The theoretical/conceptual framework utilized in this study was the 12 Malawi Principles of EA of the CBD [32]. After studying and discussing the Malawi Principles, some of the principles were somewhat overlapping and therefore condensed into themes that were relevant to the primary content of the principles and relevant to the Baltic Sea Region. The themes that emerged are as follows, with principle number in brackets: stakeholder participation (1), governance scale (2,7,8), institutional arrangements (3,6,9,10), economic considerations (4,5) and science-policy interface (11,12).

For the coding process, an excel spreadsheet was utilized for each GEAR minutes separately, where paragraphs from the minutes were inserted with their assigned theme as a code. Additionally, a column "remarks" was inserted where each minutes' paragraph was classified as internal strength or weakness, or as external threat or opportunity along with the justification for the classification. Many topics discussed in the minutes touched upon more than one theme and therefore assigned more than one code accordingly.

### **3. Results**

This section presents the results of the SWOT analysis, focusing on each part of the SWOT analysis in turn.

### *3.1. Strengths*

One of the key strengths of the ecosystem approach that was evident in all the themes and came across in the document analysis is the integrated approach in practice. This was evident in the meeting minutes, for example, in the Gear 24 min, paragraphs 5.51–5.56, which noted the "plans for assessments for marine litter, integrated eutrophication assessment tool, underwater noise, integrated hazardous substances assessment tool" [37]. These paragraphs describe the latest developments on the topic of the assessments for marine litter, underwater noise, eutrophication, etc. HELCOM has developed the necessary tools for assessing all these stressors, and their efficient utilization can lead to accurate assessments that can facilitate the implementation of EA in management. It exemplifies an internal strength of the approach, which causes seemingly disparate areas to connect with each other in pursuit of one common overarching goal of good environmental status.

Another strength of the approach is its simplicity in communicating the necessity of humans to protect their environment, in this case, the good ecological status of the Baltic Sea. One theme throughout the GEAR meeting portal is that there is consensus that the ecosystem approach to management is the accepted guiding framework for the achievement of good ecological status of the Baltic Sea. The meeting minutes [37] demonstrate

that there is agreement that everyone understands the guiding principles of the ecosystem approach and that humans are an integral part of ecosystems.

Stakeholder participation is one of the key features of EA and one of its internal strengths. The practice of stakeholder participation can be traced during different processes described in the GEAR minutes. The adequacy of participation or the breadth of stakeholders of course can be argued but nonetheless it is present in different occasions. For example, in GEAR 13 min, the paragraphs from 4.14 to 4.17 discuss the national consultation process regarding the first results of HOLAS II. The comments that would occur from this consultation will feed the HOLAS II updating. Additionally, it was discussed the "option to have a regional consultation on the HOLAS II report ... oriented in particular towards an international audience in the Baltic Sea Region ... " [37]. A few more examples of stakeholder participation can be found in the minutes of GEAR 24, in paragraph 5.35, "a project meeting ... to enable interested experts and CPs to contribute more concretely to the development of driver indicators ... " and in paragraph 6.2, "outcome of the HELCOM Stakeholder Conference in 2021 ... " [37]. Finally, already from its first meeting, GEAR discussed "how in principle the projects (steered by GEAR) should be carried out ... good involvement of national experts in the projects and project preparation ... better results than purely scientific . . . of mainly external scientific institutions" [39].

Another internal strength of EA is the constant generation and updating of scientific data and indicators for better monitoring and assessing of different components of marine ecosystem. For example, the GEAR 24 min in paragraphs 5.17 to 5.23 discuss the establishment of a list of commercial species for the HELCOM area and (5.21) the "possible need to include species such as perch ... commercially important in some areas of the Baltic but for which data is not reported under the CFP." [37]. Furthermore, in 5.22, it was "agreed that all commercially exploited species ... should be included on the list ... only those species for which landings data is available can be included in the assessments . . . collect initial information on the availability of landings data for regionally important commercial species for which data is not reported under the CFP." [37]. Though no scientific breakthrough is communicated through this example, the constant effort to improve the existing indicators for better monitoring is however evident.

One strength closely related to the constant generation of data and stakeholder participation is adaptation; an inherent feature of EA. Change is inevitable and is it recognized by EA, which encourages for the right adjustments and adaptations to it. This is exemplified in the minutes of GEAR 24, paragraph 6.6: the "current set up of the HELCOM Roadmap on EA is no longer optimal and that as part of updating the Roadmap in 2022 ... an approach more along the line of a meta-analysis would be more appropriate ... " [37]. The Group recognized the need for improvement and adjustments to fit the input from feedback loops for facilitating further EA implementation.

### *3.2. Weaknesses*

The scientific and interdisciplinary basis of the ecosystem approach requires comprehensive monitoring data which may be difficult to implement due to resources limitations. This incomplete scientific information was a theme in the meeting minutes of the GEAR group. For example, in the first meeting minutes [39], Section 4.4 states that "new approaches for eutrophication targets ... the report should be considered as input from science to the HELCOM work ... acknowledging the views of Germany and Poland that it has its deficiencies in terms of data use ... HELCOM work continues to be based on science and its technical work should follow the best available science" [39]. This statement illustrates that there is incomplete data which hampers the achievement of eutrophication targets, as now HELCOM is forced to use the incomplete information, or best available science. Another inherent weakness is the inconsistency in the application of EA, due to the diverse interpretation of what it means and how it lends itself to the local context. For example, the Gear 1 meeting minutes demonstrate in Sections 4.2 and 4.3: "Poland ... requested information on sampling stations ... Poland to observe that only data from

the 1980s and 1990s had been used ... most useful for the project to have data also from the end of the 1990s and 2000s ... proposals by the project for further sub-regionalized targets" [39]. These comments were made by delegates considering the final report of the HELCOM Approaches and methods for eutrophication target setting in the Baltic Sea Region (TARGREV) project [39]. Since Poland discharges one of the largest nutrient loads in the Baltic Sea Region, meeting participants were pointing out for the need for a more comprehensive data set to be able to set sub-regional targets.

An additional weakness of EA is the vastness of the marine ecosystem itself as well as a number of factors that cannot be monitored easily or issues of national jurisdictions or natural phenomena and climate change that cannot be fully predicted or controlled. This aspect is reflected in the minutes of GEAR 13 in paragraph 3.13 where "took note of the information on the national plans to use exceptions under MSFD Article 14 ... " [37] and it continues in presenting the exceptions used by the different Member States-Contracting Parties (CPs). Article 14 of MSFD [15] in essence gives the opportunity to the Member States (MS) to determine areas within their national marine waters that cannot meet the environmental targets or the GES within the time schedule due to a number of listed reasons. Those reasons include natural causes, unforeseeable circumstances that prevent MS to fulfill their obligation, natural conditions that hinder the prompt improvement of marine waters' status, etc.

### *3.3. Opportunities*

One of the big opportunities of EA was recognized in the very first GEAR meeting minutes, which is the potential for alignment with existing policies and strategies and finding collaborative synergies with other countries and agencies. For example, the GEAR 1 meeting, Section 2.3 recognizes "The importance of starting early on with the regional coordination ... also cooperate with other organizations such as OSPAR and Black Sea Commission" [39]. This was further reiterated in Section 2.4, which went on to say that members "supported the alignment of the delivery of the relevant HELCOM products with timetables under other international frameworks" [39]. This theme continued forward in the GEAR 24 meeting minutes, where Section 6.7 looks into "use GEAR as a platform to share national experiences on implementing ecosystem based management (EBM), in the long term develop joint guidance on EBM, consider how to strengthen interactions with other sectors" [37].

These synergies can have further benefits apart from sharing knowledge, scientific data and expertise. These benefits can be the saving of resources, both in terms of funding and working force, avoiding duplication of work and unnecessary workload. This is clearly stated in GEAR 18 min, paragraph 3.7: "global fora are good channels to present ... high quality and best practices developed in the Baltic Sea Region ... the well-structured assessments ... from EU and RSC (Regional Sea Convention) work should be exported ... to avoid duplication of work . . . " [37].

Another opportunity that surfaced in the GEAR 24 meeting minutes is the possibility to combine the potential for the alignment of policies with funding support. For example, Section 5.64 notes that "Sweden is looking into the possibility of providing additional resources to support the work under HOLAS III" [37] and 5.69 "Finland and Sweden investigate potential funding opportunities to support closer participation by Russia" [37]. The EU also provides funding support in the operationalization of projects that promote the implementation of MSFD and EA such as the BALSAM project, which provided recommendations for marine monitoring in the Baltic Sea. Such reference is found in GEAR 6 paragraph 3.26: "EC that is providing the main source of funding for BALSAM, that project goals and deliverables could in discussion with the EC be adjusted in order to make best use of resources" [40]. More supportive actions, even if their nature is not specified, we find in GEAR 13, paragraph 4.1, where Germany will continue supporting "activities that were not possible to finalize within the timeframe of the project (EUTRO-OPER) e.g., development of GES boundaries for new pre-core indicators . . . " [37].

An additional opportunity in the implementation of EA in the Baltic Sea Region is performing both national and regional assessments for better management and implementation of EA. Such a statement is found in GEAR 24, paragraph 3.7: "regional and national assessment results ... valuable information even if these differ, with regional assessment ... a transboundary overview ... national level assessments enables results to be used for direct management needs which often take place at a local scale" [37].

Finally, the active participation of observers in GEAR meetings but also in actions taking place within HELCOM can be considered as an external opportunity in EA implementation in the Baltic Sea Region. Such an example is to be found in GEAR 18, paragraph 4.19: "the need to include those observers which have been taking part in the HOLAS II work" [37].

### *3.4. Threats*

The top threats to the implementation of EA in the Baltic Sea Region relate to the themes already discussed. For example, inadequate funding is a possible threat to the third HELCOM Holistic assessments of the Baltic Sea environment (HOLAS III). This was captured in the GEAR 24 meeting minutes, Section 5.63: "resources allocated for HOLAS III ... further consideration on resources needs of the planned work ... very important that there are dedicated resources for drafting the HOLAS III texts in a coherent and readable manner" [37]. Planning for the funding is very important and it must be secured before the planned work began. Amidst the lack of resources, wasting them is an additional threat for EA implementation. Such a fear was discussed in GEAR 24 paragraph, 3.24: "careful consideration ... the added value of the resources required for the pre-filling ... to what degree countries will utilize this information" where a careful planning is required to avoid using resources to unutilized information" [37].

Another threat that was clearly articulated in the meeting minutes of GEAR 24 is the challenge of integrating the different disciplines. For example, Appendix 3 of the minutes speak of a future action by 2023 to "integrate economic and social analyses in HELCOM work ... for assessment of the linkages between the marine environment and human wellbeing, including carrying out regionally coordinated economic and social analysis of the marine environment" [37]. This difficulty of coordinating and integrating different approaches was tabled at the minutes of the GEAR meeting nine years earlier in 2012, when the GEAR 1st meeting minutes Section 3.15 stated that "the different approaches, lack of sufficient coordination and harmonization ... will be no possibility to coordinate and harmonize the targets between the neighboring countries' sea areas during the first implementation round of the MSFD" [39]. This quotation also points to the threat of inadequate political leadership to ensure collaborative governance for the meeting of the goals of the ecosystem approach.

An additional threat is the differences in national agendas, which can have various origins (differences in political will, in prioritizations, in resources, in capabilities, in data etc.). These differences can be reflected in GEAR, a group where mutual agreement is required for decisions to be made. Such an example can be found in the minutes of GEAR 13 in paragraphs 3.3 to 3.8 where study reservations on the "Consolidated version of the Joint documentation of regional coordination of programmes of measures in the Baltic Sea area" [37] by Denmark and Germany are discussed. In paragraph 3.6 is stated "the strict approach by Germany ... expected a number of its comments and proposed amendments ... to be agreed by the other CPs as a prerequisite to Germany clarifying its study reservation ... harsh action ... effectively blocks constructive work of the group ... against the good practice of HELCOM work" [37]. These kind of incidents may not be frequent but they certainly can have a disruptive effect on the implementation of EA.

The lack of consistency in the delegates of the CPs' presence in the GEAR meeting can pose an external threat for EA in the Baltic Sea Region. This issue was raised in GEAR 14 min, paragraph 2.3: "only four Contracting Parties were represented ... stressed that a better representation of Contracting Parties is needed Group to steer the implementation of the HELCOM BSAP and to facilitate the regional coordination for the implementation of the EA and the MSFD" [37].

Finally, partly ineffective tools can be considered as an external threat. Such an example can be found in GEAR 24, paragraph 4.9: "the information on the updated HELCOM Explorer" and 4.11 "could be refined to be more easily accessible for the general public ... proposals for simplifying the language" [37]. HELCOM Explorer is a very important tool because it gives the general public the opportunity to be informed about the different joint and national actions in the Region for the implementation of EA and to be part of this process, which mainly takes place among experts. Therefore, it should cater to the needs of the public and the language simplification is a step towards that direction.

### **4. Discussion**

### *4.1. Results of the SWOT Analsyis and Strategies Going Forward*

The results of the SWOT analysis as conducted for the Baltic Sea Region are summarized in Table 1. The research questions that this study posed were: what are the internal strengths and weaknesses of ecosystem approach to use sustainability and people as the heart of achieving good environmental status, and what are the external opportunities and threats to promote or retard the implementation of EA in achieving these goals in the Baltic Sea Region? The Baltic Sea Action Plan (BSAP) is HELCOM's key tool for achieving good ecological status of the Baltic Sea by 2021, while EU MSFD established a framework for achieving or maintaining GES by 2020, both having EA as their guiding principle. However, none of these realized their goal because GES has not yet been achieved in the Baltic Sea Region. Nonetheless, progress towards EA implementation has been made under these frameworks in the Region, a fact that should be not overlooked and understated and that was evident in the minutes of the GEAR meetings examined in this study. However, as Table 1 shows, there are key threats such as the difficulty integrating disciplines, inadequate funding, lack of consistent stakeholder participation and lack of political will that potentially undermine the strengths of the approach.


**Table 1.** Summary of SWOT results.

This study has found confirmation of the usefulness of the ecosystem approach as a framework towards the achievement of the good ecological status of the Baltic Sea. It has extended previous studies on the ecosystem approach as it has applied the SWOT framework and is using it in the discussion of strategies for taking the approach forward into the current update of the Baltic Sea Action plan for more effective implementation. The strategies that will be discussed below flow out of a systematic examination of the threats and weaknesses and the utilization of the strengths and opportunities to mitigate those impediments.

### *4.2. The Future—Strategies Going Forward*

These results are now used to devise strategies for moving forward with EA in the Baltic Sea Region. The SWOT analysis is not only a simple framework used to highlight strengths, weaknesses, opportunities and threats, but its additional value can be found in combining these elements in the development of strategies toward goal achievement. There are many ways to link the strategies from a SWOT analysis, based on the internal and external factors. According to the literature [41,42], four strategies can be developed, as follows: SO (strengths-opportunities), ST (strengths-threats), WO (weakness-opportunities) and WT (weakness-threats). This can be combined in a SWOT matrix, as follows [43]:


### 4.2.1. SO Strategies

SO strategies are those combinations that allow internal strengths of the ecosystem approach to utilize external opportunities in the Baltic Sea environment. This combination of internal strengths of the ecosystem approach with the external opportunities in the Baltic Sea Region and other areas of the world could help to mitigate threats in the environment. Blending the strengths of the ecosystem approach, being an inherently interdisciplinary one with the opportunity of it being a widely held and accepted framework for Baltic Sea protection, could help to mitigate the threat of the difficulty of integrating different disciplines. A strategy recommendation would be to hold facilitated events such as workshops or conferences that bring together different disciplines to discuss the ecosystem approach in the Baltic Sea Region. Such workshops and conferences would also help to raise awareness and create a common understanding of the approach.

Stakeholder participation is an inherent strength of EA that helps to determine the objectives of ecosystem-based management. The Baltic Sea Region presents the great opportunity of broad participation of stakeholders in the process of developing EA. These stakeholders can be national experts participating in GEAR, experts working on EA actions in national contexts or observers from different organizations such as foundations, NGOs, etc. An example of strategy recommendation would be that by highlighting the significance of stakeholder participation in the core of EA, participating experts in the Baltic Sea Region can promote action that would further cultivate and elevate the participatory culture, addressing especially the countries that have less experience of it in their political systems. Furthermore, observers could also promote their experience in participating in Baltic Sea Region actions and the positive impact and effect such participation can have in protecting the environment and the achievement of GES. They could use the websites of their organization as platforms to promote such experiences to gain visibility and raise awareness. This combination of strengths and opportunities can also be used to combat the threat of the loss of political will, as showcasing the framework in public events would help to garner political support.

One of the EA cornerstones is the production of data and scientific knowledge, which is an important strength of the principle. The production of scientific data in the Baltic Sea Region is voluminous as it is a high priority in the region. More tools are developed, and more data are collected. One strategy action could be for HELCOM to export the expertise of creating tools and frameworks that have been effectively utilized in the Baltic Sea Region not only to other regions in Europe but also at an international level. The poor ecological status of the Baltic Sea due to its pollution, adding to its natural vulnerability, has led to a high scientific expertise in the Region that is constantly looking for more effective tools and framework. Such expertise in the Baltic Sea Region should become visible, recognized and exported globally to harness financial resources.

The opportunity of technological advances in the region along with the inherent strength of the scientific focus of the approach could help to mitigate the threat of lack of institutional capacity in some Baltic Sea countries. Environmental forerunners such as Finland, Sweden and Denmark could help in training staff of the environmentally weaker countries such as the Baltic States through mobility staff exchanges.

### 4.2.2. ST Strategies

ST strategies use internal assets to help lessen the impact of external threats in the Baltic Sea and wider world. The integrated, interdisciplinary focus of the ecosystem approach could be combined with the threat of integrating disciplines, leading to a strategy of increasing collaborations among disciplines. It is noted that HELCOM has a natural sciences focus, so collaborations should be extended with the humanities and the social sciences to gather local knowledge to combat stressors such as climate change.

The internal strength of stakeholder participation could be coupled with the threat of the voluntary participation of national delegates in HELCOM processes. A strategy recommendation may be to detect the reasons behind the occasional lack of participation and trying to address those reasons if possible. For example, if delegates cannot be present to a certain venue due to lack of funds or excessive workload, then actions could be taken to find some funds or hold the meetings in a virtual manner, in a hybrid mode, in order for more delegates and observers to participate in the meetings.

Another possible combination may be the internal strength of data collection and knowledge production in conjunction with the threat of developed tools not completely fulfilling their purpose. The constant development of methods and tools for collecting information about the marine ecosystems in the Baltic Sea and their monitoring has been a very important component of EA implementation in the Region. A strategic recommendation would be a more careful design of tools and possibly their pilot use to determine their performance in their designated task before their release to broad use. The constant scientific research with a broadening interdisciplinary focus will lead to gains in knowledge on EA implementation. Therefore, it can inform the process of developing tools suited to purpose, in order to avoid wasting resources by creating impractical tools.

### 4.2.3. WO Strategies

WO strategies are those that use external opportunities in the Baltic Sea environment and wider world to mitigate the internal flaws of the ecosystem approach. One opportunity that recurs in HELCOM's meeting minutes is the alignment of the ecosystem approach with other frameworks such as the sustainable development goals (SDGs). This can be harnessed in overcoming the weakness of the approach in being complex and difficult to apply in practice. A strategy to do this will see experts of EA helping managers and non-specialists to understand the approach and to apply the tools of the approach. They can be part of project teams that help in the implementation of the approach on the ground in the Baltic Sea Region. This could result in greater understanding, uptake and local ownership of the ecosystem approach.

One important external opportunity in the Baltic Sea Region is conducting assessments at both a national and regional level for fulfilling the requirements of the MSFD, among other things, since eight out of the nine countries-CPs of HELCOM are also member states of the EU. This opportunity can be combined with the inherent weakness of EA, which is the incomplete scientific data for EA implementation. A strategic recommendation could call for intensifying the combination of the data collected at a national and regional level in the Region in order to fill in the gaps of scientific knowledge. Furthermore, national assessments can provide data for smaller geographical areas and be more specific. This information should be utilized in a manner that would enhance knowledge of more components of the marine ecosystems of the Baltic Sea and it would promote further the implementation of EA.

An often-reoccurring opportunity in the GEAR meetings is the synergies in processes such as reporting and monitoring that can lead to saving funds as well as less workload. It can be combined with the internal weakness of EA of being resource-intensive and requires many comprehensive data. There is already a broad network of collaboration between HELCOM and other organizations in the Baltic Sea Region and at an international level. A strategic action could be to enhance the existing collaboration and to assign tasks in a coordinated manner so as to share different monitoring and reporting processes and costs. The collaborations aimed at producing data in the Baltic Sea Region could be strengthened in order to produce solid scientific data to export for international assessments in ameliorating the weakness of increased resources and data requirements for EA implementation.

### 4.2.4. WT Strategies

WT is the double whammy strategy, as it consists of strategies aimed at lessening internal weaknesses and external threats at the same time. Lessening the negative effects of threats and weaknesses simultaneously can be confounding. The weakness of the approach being resource-intensive combined with the threat of inadequate funding could be addressed by utilizing funding pools available from the EU. As such, it would be pertinent to identify the funding gaps and their causes and to combine this with matching these gaps to the pools available.

The important internal weakness of EA, the incomplete scientific data information, and the external threat of inadequate resources, e.g., personnel, could be addressed together. A strategic recommendation would be to actively pursue more collaboration agreements with scientific institutes in the Region; for example, with universities that could assist in producing knowledge and at the same time would address the issue of lack of scientific personnel. Universities, having their own pools of funding, would not pose any additional financial burden to the process of EA implementation in the Region and they have the expertise and the experience to provide valuable assistance in accumulating additional scientific data for EA.

Another weakness, that of EA being complex and difficult to apply, may be approached together with the threat of different national agendas. It is very hard for one to address the threat of different national agendas adequately or to change the fact that EA is difficult to apply since at its very heart lies the marine ecosystem that is complex and unpredictable. However, one strategic action could be to increase the visibility of EA at the national level and especially to the broad public. Through events or workshops addressing different parts of the populations and different local groups, EA can be promoted and become less of a complex principle by actively educating the public, emphasizing their participation and support. The more visibility and recognition gains among the broad public, the more chances are for the official national attitudes to change towards it and be more actively involved in EA implementation if they have not been already.

### *4.3. Malawi Principles in the Baltic Sea Region*

This section now considers how the implementation of the ecosystem approach in the Baltic Sea Region corresponds to the vision espoused in the Malawi principles (Table 2). This is based on the results from the SWOT analysis. Table 2 clearly illustrates that there are different degrees of implementation of the ecosystem approach in the Baltic Sea Region. There is management autonomy to choose actions and planning time frames, with management recognizing changes in line with the Malawi principles. However, there is still the sectoral approach, with silos operating, signaling the need for better collaboration

and policy coherence at the local (e.g., municipalities), national (e.g., ministries), regional (EU level) and international levels. There is also not enough recognition and use of local knowledge. The local level is still not adequately represented in the decision-making process. One of the key areas for improvement is highlighted in principle 12, the need for better stakeholder engagement. Communication and capacity building of all stakeholders can lead to better implementation through an increased understanding of ecosystem interactions and hence the adoption of policies by stakeholders, leading to better science policy processes.

**Table 2.** Malawi principles in the Baltic Sea Region (Green-achieved; Red-not achieved; Grey-somewhat achieved).


### **5. Conclusions**

There is much literature on the ecosystem approach, but the novelty of this work is that it systematically studies the ecosystem approach using the strengths, weaknesses, opportunities and threats (SWOT) framework. This method not only highlights key aspects of the ecosystem approach but enables the development of strategies for further optimization of the implementation of the approach in the Baltic Sea Region and beyond. This research presents the findings from the Baltic Sea Region, one of the largest water catchment regions in the world and a part of the European Union. It has shown that the ecosystem approach to management is a means by which to capture the intricacies of the relationship between humans and the Sea (the environment). Findings reveal that one of the advantages of the ecosystem approach that Baltic Sea experts cherish is its simplicity in communicating the importance of Baltic Sea protection. However, the caveat is that the language of the ecosystem approach can be complex and unclear, hence it can serve as the link between research and practical implementation. This research has recommended that the implementation of the ecosystem approach is most effective when using existing tools and incorporating into prevailing policies. On a wider scale, the ecosystem approach is not only useful in management decision making but also as a means of conveying the complexities of the relationship between humans and the environment. Resource allocation backed by strong political leadership can strengthen interdisciplinary research so that the ecosystem approach can be better integrated with sustainability policies and practices.

This study has utilized the SWOT framework to highlight key aspects of the ecosystem approach that can be further developed for better implementation in the Baltic Sea Region and beyond. This research can be seen as the platform from which other dialogue and further research emerges as outlined below:


**Author Contributions:** This article was conceptualized by S.J. and V.L. and outline drafted by S.J. and V.L. Literature review, methods and results and discussion written by both S.J. and V.L. Coding and data analysis done by both V.L. and S.J. Both authors have read and agreed to the published version of the manuscript.

**Funding:** This research received academy of Finland SeaHer funding, grant number 315715. The authors also wish to acknowledge support from the 'Sea' profiling area of Åbo Akademi University.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** All data is stored centrally at Åbo Akademi University.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


### *Review* **A Review of Key Sustainability Issues in Malaysian Palm Oil Industry**

**Lakshmy Naidu and Ravichandran Moorthy \***

Research Centre for History, Politics and International Affairs, Faculty of Social Sciences and Humanities, Universiti Kebangsaan Malaysia, Bangi 43600, Selangor, Malaysia; p103087@siswa.ukm.edu.my **\*** Correspondence: drravi@ukm.edu.my

**Abstract:** The palm oil industry has contributed enormously to the economic growth of developing countries in the tropics, including Malaysia. Despite the industry being a development tool for emerging economies, the oil palm crop is inundated with allegations of its unsustainable plantation practices and viewed as environmentally detrimental and socially adverse. These negative perceptions are amplified through anti-palm oil campaigns and protectionist trade regulations in developed countries, particularly in the European Union (EU). This situation, if further exacerbated, could potentially affect the export of palm oil and the industry as a whole. As such, this article provides a critical review of the key sustainability issues faced by the Malaysian palm oil industry as the second biggest exporter of palm oil to the global market. The various insights and the interpretations of sustainability are contested according to the contexts and the interests of the countries involved. Hence, palm oil is constantly exposed to bias masked by non-tariff barriers from consumer countries to protect their domestically produced vegetable oils. This could constrain the commodity competitiveness in the international market. As issues on palm oil sustainability continue to evolve, policymakers at key stakeholder agencies need to devise strategies to manage global disruption in the palm oil trade.

**Keywords:** palm oil; oil palm; sustainability; trade; European Union; stakeholder agencies

### **1. Introduction**

The rapid growth in the world population and the increase in demand for renewable energy, particularly towards mitigating greenhouse gas (GHG) emissions, have resulted in the rise in demand for oilseeds. This situation has contributed towards the expansion of oilseed cultivation, especially oil palm, in the tropical producing countries. As one of the major exporters of palm oil, Malaysia's oil palm plantation area reached 5.87 million hectares, which supplied 34.3% of the total palm oil trade and constituted 18.3% (17.37 million tonnes) of the global oils and fats industry last year [1]. With the anticipated increase in the global population to nine billion by 2050, the demand for oils and fats is expected to reach 35 million tonnes annually [2]. In addition to food, new uses of palm oil, such as for industrial applications and biodiesel, have further strengthened the demand to cultivate more oil palm.

However, as the total land area available is fixed it means that more crops have to be produced within the same area. In order to fully optimize land usage for oil palm crops, palm oil research has been focused on sustainable plantation and development. Despite various efforts made in research advancements and sustainable practices, oil palm cultivation has been exposed to increasing controversy and scrutiny over the years. The oilseed crop has been associated with environmental degradation, such as peatland conversion, tropical rainforest deforestation, loss of biodiversity, reduced carbon sink source, flood regulator, and the carbon footprint [3–7]. The industry has also been plagued with social issues, such as the violation of human rights, the rise in crime, the influx of foreign labor, conflicts over land rights, labor management, and the displacement of local

**Citation:** Naidu, L.; Moorthy, R. A Review of Key Sustainability Issues in Malaysian Palm Oil Industry. *Sustainability* **2021**, *13*, 10839. https://doi.org/10.3390/ su131910839

Academic Editors: Baojie He, Ayyoob Sharifi, Chi Feng and Jun Yang

Received: 27 August 2021 Accepted: 27 September 2021 Published: 29 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

and indigenous communities [8–10]. Moreover, concerns over the safety and the quality of palm oil contaminants, such as 3-monochloropropane-1,2-diol (3-MCPD) and glycidyl esters (GE) in the food sector have continued to be present over the past decade [11,12].

These negative perceptions have created concerns about the sustainability of the commodity, especially in the developed economies such as the European market. The European Union (EU) established certified and sustainable palm oil standards as part of its importation requirements, which started in 2015, for a completely sustainable palm oil supply chain in Europe by 2020 [13]. Furthermore, the European Parliament agreed that the main criteria for biodiesel are set for no deforestation and voted to ban biodiesels from palm oil beginning in 2021 and to completely phase them out by 2030 in adherence to the EU Renewable Energy Directive (RED) II [14,15]. In addition to the restriction of palm oil-based biodiesel, the European Commission (EC) implemented the maximum limits on 3-monochloropropane-1,2-diol esters (3-MCPDE) and GE in edible oils and fats in January 2021 and March 2018, respectively [16]. Apart from these interventions, non-governmental organizations (NGOs) and consumers advocate boycotting palm oil through palm oil-free campaigns, while private companies promote the use of certified and sustainably produced palm oil through labels on their products [17,18]. The imposed restrictions and the increase in public awareness could suppress the utilization of palm oil in the transportation and food sectors in Europe. It may jeopardize the market share of Malaysian palm oil in the European oils and fats sector if palm oil is continuously viewed as unsustainably produced.

Therefore, there is growing pressure on exporting countries such as Malaysia to meet the sustainable production and development standards in the palm oil supply chain in order to trade in the European market. The formation of the Roundtable of Sustainable Palm Oil (RSPO) in 2004 as a reliable global standard for certified and sustainable palm oil (CSPO) enabled the palm oil industry players to demonstrate commitment towards sustainable development [19]. In 2015, the Malaysian Sustainable Palm Oil (MSPO) was developed as a national standard to monitor the development of the sustainable palm oil sector, and compliance to the standard was made compulsory by 31 December 2019 [20]. These certification standards indicate the importance of responsible practices from the environmental, social, and safety perspectives. The principles and criteria in the standards require involvement from multi-stakeholder agencies to discuss sustainability issues from a policy formulation perspective in order to stay competitive internationally.

Hence, the purpose of this article is to provide a critical review of the numerous academic articles that have been written about sustainability issues in the palm oil industry. By elucidating the discourse on palm oil concerning key sustainability issues, this paper aims to bridge the gap between the issues of the misconceptions, the allegations, and the realities of palm oil. The critical review also intends to contribute intelligibility and coherently to the existing body of knowledge in the palm oil sector. This is important as policymakers could use the information at governmental agencies to develop effective sustainable palm oil strategies, and academicians and researchers could use it for further studies in the field.

The article is structured based on environmental, social, economic, and health sustainability themes. The sub-section on environmental sustainability examines the environmental challenges to Malaysia's palm oil industry and the actions taken by the Malaysian authorities to counter them. The social sustainability sub-section highlights how the palm oil industry has contributed to the social wellbeing of the local people and how the negative campaigning by international agencies is harming the industry and the people. In the economic sustainability sub-section, the article shows the palm oil industry's contribution to Malaysia's economy and the actions taken to reduce the carbon footprint and meet the high standards of the global market. In the health sustainability sub-section, issues such as the health problems resulting from the haze, forest fires, and slash-and-burn farming techniques are addressed. It also discusses issues related to nutritional value and the association with diseases. In the Section 4, the authors discuss the international dynamics that hamper the palm oil industry and the issues surrounding Sustainable Development

Goals (SDGs). The Section 5 summarizes the findings and provides recommendations to policymakers.

### **2. Materials and Methods**

This review article investigates the sustainability issues of the palm oil industry in Malaysia. By using a macro-approach to examine the sustainability of the commodity, the article discusses the development of the palm oil industry at the international and national levels. The article consists of critically reviewed sources ranging from academic books, scholarly journals, proceeding papers, and reports from the leading researchers and academicians in oil palm studies. The data reviewed in this article was obtained from the international and national agencies involved in palm oil sustainability in order to present the most current developments. Among the data sources are the Amsterdam Declaration from the EU, the Report on the Status of Production Expansion from the European Commission, and background information from the RSPO, as well as oil palm statistics and development from the Malaysian Palm Oil Board (MPOB) and the Malaysian Palm Oil Council (MPOC). Collectively, the literature reviewed provides the insight to explore the complexity inherent in the palm oil sector with the aim of achieving sustainable development.

As a measure to discuss the latest progress in the industry, this article examines the literature published in the past two decades, between the years 2000 and 2021. The searching process was conducted in the Scopus database using five main keywords: palm oil, sustainability, trade, European Union, and stakeholder agencies. From the literature review, four main themes in palm oil sustainability are identified for review. The themes are based on the environmental, social, economic, and health perspectives. With the exclusion of health, the other themes are in conformance with the three pillars of sustainable development by the United Nations' SDGs.

Throughout the article, priority has been given to the findings and discussions from scholarly literature to ensure the reliability and validity of this review. However, news articles were reviewed to obtain information on the latest governmental policies and actions that have yet to be academically investigated. Careful consideration was taken to ensure the accuracy of these news articles by counterchecking with other related media releases.

### **3. Results**

As a highly contested oilseed crop, palm oil dominates the discussion on sustainability compared to other vegetables in the oils and fats sector. The exchange of information on this commodity has been prevalent among palm oil-producing countries, palm oilconsuming countries, NGOs, traders, manufacturers, growers, processing and milling facilities, associations, consumers, and policymakers in the past decades. The findings of these discourses can potentially affect the supply chain and the trade in palm oil and its products as they may influence the policymaking processes made by stakeholder agencies. The complexity of palm oil poses a challenge to policymakers to increase the yield produced in each hectare of agricultural land to optimize its socio-economic benefits while reducing the detrimental impacts on the environment. Hence, policy formulation that underpins sustainable development of the palm oil industry needs to balance between environmental, social, and economic stances [21].

The following four sub-sections highlight the issues of palm oil sustainability based on the environmental, social, economic, and health perspectives.

### *3.1. Environmental Sustainability*

The categorization of palm oil-based biodiesel as a renewable energy source has strengthened palm oil exports to Europe. Since 2000, the EU countries have increased their total use of palm oil by 63% and by the year 2017, 87% of palm oil imports were used to produce biodiesel [20,22]. As the third-biggest importer of palm oil after India and China, Malaysian palm oil exports to the EU increased by 9.5% from 1.91 million tonnes in 2018 to 2.09 million tonnes in 2019 [23,24]. The imported palm oil is used as a substitute for

soybean oil as the main feedstock for biodiesel production [17]. Although the EU uses the term biofuel to describe the blending of renewable resources with fossil fuel, palm oil is exported as a feedstock to produce biodiesel due to its compatibility with diesel.

Amid countries expanding their oil palm plantations to meet growing market demands, the EU implemented RED-2009/28/EC, which mandates a binding target of a 10% share of biofuel as a transport fuel by 2020 [25]. The inclusion of biofuel in the regional bloc energy mix aims to reduce GHG emissions, improve energy security and protect domestic farmers' revenue. As stated in Article 17, biofuel products must be environmentally friendly and sustainable to decrease GHG emissions by at least 35% compared to fossil fuels [26]. Nevertheless, the biofuel policy as stipulated in RED has generated criticism from member states since its inception [27].

As a measure to increase the sustainability criteria to take into account member-state grievances, the resolution to eliminate palm oil in biofuel starting in the year 2021 was made a part of the EU RED II renewed policy framework. Palm oil has been classified as a high Indirect Land Use Change (ILUC) risk biofuel, whereby an increase in the demand for palm oil will lead to the indirect conversion of forest into land for biodiesel feedstock to meet the raw materials required for the European market and hence will be phased out by 2030 [17]. Moreover, the risk in ILUC needs to be assuaged as the GHG savings are set at 65% and would impact the GHG balance of biofuels, if unaddressed [28]. Classifying palm oil for biofuel as unsustainable will not be factored into the EU renewable energy target as it is not considered green fuel. Therefore, it is evident that palm oil is specifically targeted due to its high ILUC risk.

The regulation serves the dual purpose of protecting the industry players of domestically produced vegetable oil by using regulatory barriers towards palm oil while adhering to the euro-centric notion of sustainable development. The plan " ... to take measures to phase out the use of vegetable oil that drive deforestation, including palm oil, as a component of biofuel, preferably by 2020", and the plan that " ... recommends finding and promoting more sustainable alternatives for biofuel uses, such as European oil produced from domestically cultivated rape and sunflower seeds" reveal that palm oil is specifically mentioned in the resolution [29] (p. 9). The unwarranted attention has translated into discriminatory action that could impede the export of palm oil into the EU market and has tainted the reputation and acceptance of palm oil and its products internationally. Moreover, there is contention regarding ILUC as defining the criteria for land conversion as it reflects the European perspective rather than an internationally accepted view on environmental issues [30]. The debate on sustainable palm oil is also perceived as an enforcement of Western environmental values on producing countries that are dominated by Asian culture [31].

Recognizing that palm oil is a highly productive oilseed crop, replacing palm oil with other vegetable oils would create a need for more land for cultivation. Hence, it serves as an indicator that the potential replacement by other oils would not be suitable. Even as studies indicate that beef and soy imported into the EU created more deforestation as compared to that of the imported palm oil, the regulation emphasized palm oil only and referred to it as a forest-risk commodity [29,32]. The requirement for establishing a uniform and compulsory certification standard for imported palm oil, detailed product labels, such as palm oil-free and certified palm oil, and campaigns to enhance customer awareness and acceptance have distinguished palm oil from other vegetable oils. These actions can be deemed an unwarranted reinforcement of the misconception that palm oil is harmful to the environment. Notwithstanding the resolution of a shared global responsibility and the need for a global solution on palm oil issues, there exists reasonable doubt as to whether palm oil-producing countries have been engaged in the resolution itself [29].

However, studies on the impact of oil palm plantations on global natural resources indicate environmental consequences in the producing countries. These impacts have caused severe and widespread pressure on the environment as more landscapes are cleared to meet the growing global demand for affordable and versatile vegetable oil [3]. Among the adverse impacts on the ecological systems are deforestation and degradation of tropical forests, which cause destruction of biodiversity, soil erosion, water pollution, and deterioration of the water and air quality [33–35]. Moreover, using slash-and-burn techniques for forest and land clearing leads to forest fires and releases carbon dioxide into the atmosphere, which results in global warming and induces climate change [36,37]. These detrimental impacts on natural resources and ecosystems have positioned the palm oil industry badly, especially among EU member states, as palm oil products are alleged as being imported deforestation [38].

Nonetheless, the Malaysian oil palm plantations can be categorized as systematically organized as they have succeeded from the organized rubber plantations, which span more than a century and are mostly owned by the large plantation companies. Hence, several standard practices adopted to reduce environmental degradation were taken throughout the years. Some of the good agricultural practices are the usage of organic fertilizers to recycle nutrients through the placement of cut fronds, empty fruit bunches and chipped old palm trunks; the planting of cover crops; the construction of terraces and silt pits; the correct placement of cut fronds; the mulching with empty fruit bunches; the practice of precision agriculture; and the utilization of Integrated Pest Management (IPM) to reduce the use of pesticides by using livestock to control weeds and using predators to control pests [1,23,39]. Moreover, the compulsory zero-burning practice for all oil palm replanting returns organic material into the soil to increase fertility as well as minimizing GHG emissions and accidental forest fires [40]. Mature oil palm trees also serve as a green canopy and a carbon sink that captures carbon from the atmosphere [41,42].

In addition to extracting crude palm oil, the mill produces large quantities of organic waste in the form of Palm Oil Mill Effluent (POME), which is used as a potential source of renewable energy [43]. Methane captured from POME is used as biogas in mills to produce electricity and thus reduces the usage of fossil fuels and GHG emissions by 30% [44]. The biogas trapping in palm oil mills can reduce climate change impact, maximize available resources, and fulfil sustainability requirements while enhancing the image of palm oil as an environmentally friendly product. Moreover, POME can be used as an organic fertilizer as it restores soil fertility and provides nutrients to oil palm trees while reducing chemical fertilizer usage and cost. Although POME is a highly polluting waste and may have profound environmental implications if untreated, it has been converted to an economically useful and environmentally friendly by-product. By achieving zero-waste standards, the oil palm industry rebuts negative allegations of environmental impacts through cleaner production and greater sustainability.

Contrary to EU perceptions, the Malaysian palm oil industry is highly governed and regulated. There are currently more than 15 laws and regulations that the industry must conform with, such as the Land Conservation Act 1960; the Environmental Quality Act 1974; the Environmental Quality (Clean Air Regulations) 1978; the Pesticides Act 1974 (Pesticides Registration Rules); the Occupational Safety and Health Act (1977); the Protection of Wildlife Act 1972; and the National Park Act 1984 [45]. The industry also has to comply with the Hazard & Critical Control Points (HACCP) and the Environmental Impact Assessment (EIA) requirements in addition to the MPOB's Codes of Practices (CoP) certification scheme as a guide to retain food safety, quality, and sustainability throughout the palm oil supply chain [46].

Apart from domestic regulations, Malaysia is also a signatory to the Convention on Biological Diversity (CBD), the Charter of the Indigenous Tribal Peoples of the Tropical Forests, and the Cartagena Protocol on Biosafety. In 2006 the Malaysian Palm Oil Council (MPOC) launched the Malaysian Palm Oil Wildlife Conservation Fund (MPOWCF) to financially fund wildlife conservation activities and studies by academicians, government officials, and NGOs on the overall impact of the palm oil industry on wildlife, biodiversity and the environment [47]. The commitments towards national and international laws demonstrate Malaysia's responsibility and transparency in ensuring conservation of the environment and protecting wildlife impacted in the palm oil sector.

### *3.2. Social Sustainability*

As the engine of growth for an emerging nation, the palm oil industry has contributed directly towards poverty eradication through employment opportunities. Land development schemes such as FELDA provided land to 112,635 families for oil palm plantations to create livelihoods and generate income [48]. The organized plantations spurred economic development and promoted social advancements, especially in the early years after independence. New job openings were created in the agricultural sector, which comprises the planting and harvesting of oil palm trees. These new downstream sectors include processing facilities and palm oil-related industries, research and innovation for high-quality planting materials, technology advancement, and new end-products. Further to employment opportunities at oil palm plantations, the palm oil industry has enabled job openings and positions in other spin-off auxiliary services, such as transportation, trading/brokering houses, bulking installations, financial services, and other related supporting industries. As the country ventures into the development of by-products in the palm oil sector, the utilization of biomass through new technology is expected to create an additional 66,000 new jobs and had generated MYR 30 billion by 2020 as stipulated in the National Biomass Strategy 2020 [49].

In spite of the benefits in alleviating the socio-economic status of the society, the industry has been inundated with social issues related to oil palm plantations. Social issues, such as the violation of human rights, the rise in crime, the influx of foreign labor, conflicts over land rights, labor management, and the displacement of local and indigenous communities are some of the pertinent concerns in the industry [8–10]. The conflicts over land rights and forest resources negatively impact the surrounding communities, plantation companies, and governments while working conditions unconducive to safety in the oil palm plantations further expose workers to health and safety risks and ergonomic injuries [50,51]. As development in the oil palm sector continues to progress, studies have found that the diminishing human interaction with nature could lead to a sense of disaffection towards the environment [52].

The recent ban on palm oil import by multinational consumer goods manufacturers in the United States (US) due to allegations of forced labor has also garnered the attention of the industry. Companies such as Nestle, General Mills and Unilever's restriction on the purchase of palm oil was made to exclude the commodity from their global supply chain. Major oil palm growers, such as FGV Holding and Sime Darby Plantation, were accused of exploitative practices of workers, with the latter company being recognized as the world's largest producer of CSPO by RSPO [53]. The US Customs and Border Protection claimed to possess evidence of sexual and physical violence, retaining salary, and restrictions on movement among workers in the plantation. However, there are uncertainties as to whether these contentions were systematically and academically researched to ensure the validity and reliability of the findings.

Moreover, palm oil mills are regulated by the authorities through laws that protect workers' welfare, such as the Occupational Safety and Health Act (OSHA) 1994; the Employment Act 1990; the Passport Act 1966; the Factories and Machinery Act (FMA) 1989; and the Children and Young Persons (Employment) Act 1966 [46]. The implementation of these acts through regulations and policies demonstrates social sustainability in the areas of human rights and labor rights as well as corporate governance. Hence, there exist reasonable doubts that the allegations hurled against the industry are substantiated.

### *3.3. Economic Sustainability*

Many countries, especially those rich in natural resources, are dependent on the agricultural sector as an important mainstay of the economy. Agrarian sectors have an important role in food security and employment in many nations and have historical significance. Economic development studies frequently argue that agricultural productivity is usually followed by rapid industrialization and thus has become a source of long-term sustainable economic growth [54–56]. However, other scholars contend that the natural

resource-rich countries grow at a slower rate than the resource-poor countries, and the countries tend to exhibit an inferior institutional quality. This phenomenon is known as the 'resource curse, the staple trap, or the paradox of plenty' [57]. The revenues generated by the commodity sector may have a positive impact on manufacturing. However, this dependence on natural resources may translate into low economic diversification and greater deindustrialization. Nonetheless, studies suggest that while agricultural exports contribute to the shrinkage of the manufacturing sector, the agricultural sector with strong links with the domestic economy may increase the prospects of industrialization [57,58]. This is true with the case of Malaysia; the palm oil industry has spurred downstream manufacturing activities.

Strong economic growth and trade liberalization in the international economy over the past few decades have led to an increase in the export of palm oil and its related products. The total export of oil palm and palm oil products in 2020 was recorded at 26.59 million tonnes with a revenue of MYR 73.25 billion compared to MYR 67.55 billion in 2019 [1]. Table 1 provides data on the export volume and value of oil palm and its products for 2019 and 2020. Based on the table, this review supports the idea that the income generated from the agricultural commodity exports relates to the production sector output [57]. As an integral agricultural sector, the industry mitigates the food security risks in many countries [54,55]. Moreover, economic modernization in this sector induced technological improvement that reduced dependence on the unskilled workforce [58].


**Table 1.** Export of oil palm and palm oil products for 2019 and 2020.

Source: MPOB (2021) [1].

The export competitiveness of palm oil is attributed to several factors. Oil palm is a highly productive oil crop as it produces up a yield up to ten times higher than other oilseed crops, such as soybean, sunflower and rapeseed, and has a productive life span of over 20 years [23,59]. As such, oil palm plantations record the highest land productivity. With the growth in population and the surge in renewable energy, palm oil can address those demands by increasing the yield from the existing plantation areas. Oil palm growers can maximize the Oil Extraction Rate (OER) through breeding and cloning highly efficient plants. In addition, the innovation and development of scientific research in planting materials can increase palm oil productivity while ensuring sustainable practices.

By virtue of its high productivity, the cost competitiveness of palm oil makes it the most affordable vegetable oil in price-sensitive markets in emerging countries, such as India and China. As a cost-effective solution, palm oil is a suitable replacement for animal fats. The versatility of the crop is evident as it has been used as an ingredient in the food and non-food industries due to its strong stability and ease of conversion [20,60,61]. In addition, the biomass from the palm mesocarp fiber, palm kernel shell, and empty fruit bunches is a source of a renewable energy which is used to generate heat in plantation mills as a cost-saving measure, while palm oil-based biodiesel has been identified as an alternative fuel due to its price competitiveness [62–64]. The utilization of palm oil in various sectors is attributed to technological innovations, quality enhancements, and scientific advancement.

In addition to reducing the carbon footprint, POME can potentially securitize energy supply while generating additional revenue from the sale of surplus energy. Using POME as an alternative energy source not only saves on the costs of mill operation and waste treatment but mill operators can earn up to MYR 3.8 million per year through the generation of electricity [65]. The ability to generate income from POME has changed the perception of the by-product from waste to be treated to a resource that could generate earnings. Unused electricity can be connected to the national grid under the Feed in Tariff (FIT) scheme introduced in the National Renewable Energy Act 2011. This supports the Fifth Fuel Policy target of achieving 5% of national grid-connected electricity generated from renewable sources [66].

The stringent observations of quality and regulatory control and the capacity of domestic processing facilities are instrumental for the industry to meet the high standards of the global market. These factors have enabled Malaysia to consolidate its position as the second leading producer of palm oil in the international trade of oils and fats. The development brought about by palm oil has led the commodity crop to be referred to as the development engine [30] and the driver of development [67].

However, the rise of palm oil from the 1960s onwards has created disruptions in the global market dominated by other vegetable oils, notably the soybean oil and the rapeseed oil industries. Hence, countries, especially member states in the EU, sought to protect their domestic oil industries by introducing tariff barriers and disguising non-tariff barriers, such as safety, societal and environmental concerns. Furthermore, negative information was disseminated through the anti-palm oil campaigns as part of the politics to defend locally produced vegetable oil in order to win back its market share. The backlash against palm oil is attributed to the competitiveness of palm oil in comparison with soybean oil and rapeseed oil as feedstock for biodiesel production. These barriers are discriminatory as they inhibit producing countries from exporting palm oil on the global market. The restriction could limit the use of palm oil-based biodiesel in importing countries and hence contradicts free trade practice principles regulated by the World Trade Organization (WTO).

Therefore, this review posits that the protectionism policy has evolved from traditional trade barriers such as import tariffs, quota restrictions, and import bans to contemporary issues such as health, workers' protection, and the environment. These current concerns are multifaceted and are interlinked with international trade. Moreover, preventing the conversion of forest land to cultivate commodity crops would hamper developing countries such as Malaysia from using palm oil as a development tool. Denying the growth of developing economies by enforcing natural-resource conservation might be seen as prejudice because established nations in the West have abused their environment for economic advancement for millennia.

### *3.4. Health Sustainability*

Constant clearing of land for oil palm plantation and expansion using the slash-andburn technique has led to accidental fires. The fires resulted in the recurring haze that has increased the occurrences of ill health among the people in Southeast Asia [68]. In spite of national laws and the plantation companies' practice of restricting the use of fire to clear forests, transboundary haze continues to be an annual air pollution problem that affects people's livelihood and health. Moreover, oil palm trees which are planted on peatlands are highly susceptible to the fires that lead to the transboundary haze. The transboundary haze issue is intricate, and politically motivated interventions are ineffective as the system is safeguarded by patron-client relationships [69].

Another health-related issue concerning palm oil relates to its perceived nutritional value and association with diseases. One prominent incident regarding the health standard of palm oil occurred in the 1980s in the US [70]. National campaigns were undertaken to convince the public that tropical oils such as palm oil and coconut oil in food products contributed to coronary heart disease due to their higher saturated fat. Interest groups successfully compelled food manufacturers to replace tropical oils with hydrogenated soybean oil. The substitution was successful initially as "almost all of the nearly 2 billion pounds of tropical oils removed from annual use in the US food supply in the late 1980s was replaced, pound for pound, by hydrogenated oil ... " [71] (p. 236). However, it backfired as the partially hydrogenated oil in food products produces trans fatty acid, which is harmful to cardiovascular health as it increases total cholesterol. This has led to a surge in palm oil exports to the US as the solid-fat component makes it an ideal substitute to eliminate trans-fatty acid in hydrogenated vegetable oils.

In spite of the assertion that the consumption of a high amount of saturated fats from palm oil is linked with cardiovascular diseases, studies revealed that there is no concrete evidence associating palm oil intake with the risk of heart disease or the negative impact on children's health [72,73]. Furthermore, the link between the dietary intake of palm oil and obesity, in comparison with other vegetable oils, is not supported by adequate evidence [74–76]. There is also no scientific proof to validate the association between palm oil intake and the incidence of cancer [77]. In fact, the nutraceuticals and phytonutrients from palm oil reduce cholesterol and diabetic levels in adults [78,79]. Research on the effects of palm oil tocotrienol as an antioxidant in the prevention and treatment of bone-related illness was conducted [80]. In addition, the anti-inflammatory effect of oil palm phenolics may prevent the formation of neurodegenerative diseases such as Alzheimer's and Parkinson's [81,82]. The findings by these scholars have strengthened the neutral and positive effects of palm oil intake in the context of obesity and non-communicable diseases in humans.

Thus, this review postulates that the nutritional content of palm oil has contributed significantly to improving the health of the global population. Existing literature and studies conducted on the health aspects of palm oil indicate that there are potential benefits that require further scientific investigation. As palm oil is essential in developing good human health, this review suggests the need for more research to gather concrete evidence to support the industry and invalidate adverse claims regarding palm oil safety and quality. The most pertinent question is the relationship between the type and quantity of oils and fats with health, while taking into consideration other nutrients in the dietary intake as well as lifestyle habits.

### **4. Discussion**

Based on the deliberation of sustainability issues in this paper, this review contends that there is a shift in the context of international attacks on the palm oil industry. The alleged health hazards of palm oil consumption, as claimed in the 1980s, have evolved to that of the abuse of worker human rights and the detrimental impact on the environment in current times. The reposition of bombardment towards palm oil indicates the tendency of importers and consumers in developed economies to continuously develop negative perceptions towards commodities that are in direct competition with their domestically produced vegetable oils. Despite the nature of oil palm as a perennial crop with a competitive edge, the continuous and damaging accusations on the oilseed crop can be surmised as unsubstantiated and politically motivated. Hence, this review premises that palm oil is constantly exposed to bias masked by non-tariff barriers on its cultivation and industrial application in the varying contexts of usage and consumer countries. Therefore, further research is required to mitigate the impact of the evolution these discriminatory measures. The findings of the research would be beneficial in protecting the interests of palm oil-producing countries such as Malaysia.

Through analysis of the literature, it is clear that environmental, social, economic, and health aspects form the basis of the sustainable development of the palm oil industry. These aspects are diverse and are intertwined with one another. Supporting the sustainable production of palm oil is a constructive approach compared to limiting palm oil development and shunning oil palm plantations. Due to its global impact, the palm oil sector supports multiple SDGs, such as the SDG 2 on ending hunger, SDG 3 by promoting good health, SDG 7 through the development of renewable energy and SDG 12 on sustainable consumption and production patterns. As such, the increase in accountable and responsible

practices in the palm oil industry should be strategically and consistently encouraged. This would strengthen the industry's ability to address multi-dimensional challenges while remaining sustainable in order to enhance its international reputation.

This review also establishes that the discourse on sustainability provides various insights. The interpretation of sustainability is contested according to the views and contexts of the stakeholders. There exist patterns of differing perspectives in the palm oil industry where palm oil-producing countries focus on the economic benefits while palm oil-consuming countries stress social, environmental, and safety concerns. Furthermore, although sustainability standards have been asserted regarding palm oil, other vegetable oils have been subjected to fewer requirements for compliance. This has led to the perception that the regulation on sustainability has been targeted at disqualifying palm oil, especially as the feedstock for biodiesel production. The regulation can be interpreted as a form of trade protectionism with a strong political clout that specifically targets palm oil. As such, issues on palm oil sustainability need to be addressed effectively through comprehensive policies to ensure its competitiveness and acceptance in the international market.

### **5. Conclusions**

It is evident that the oil palm tree is a highly productive and versatile crop that produces competitively priced oil. The broad functionality of the oil seed crop, which is achieved through the chemical modification process, has widened the application of palm oil in diverse industries. As such, the production and trade of palm oil are of great concern due to their socio-economic significance in national development. Hence, the sustainability issues need to be addressed effectively for the industry to remain resilient in the future.

Based on the discussion, concerns about palm oil sustainability, such as commoditydriven deforestation, which results in a loss of biodiversity and induces climate change, continue to dominate the discourse on palm oil cultivation. The increase in the crime rate, the high employment of foreign labor, and the violation of indigenous community land rights are some of the social issues that further place the entire industry in a bad light. On the economic front, the commodity has faced immense scrutiny from the international community despite its price competitiveness and high productivity rate. Emerging issues concerning palm oil quality and nutritional value have been extensively researched compared to other vegetable oils by scientists around the world. These challenges that confront the palm oil industry are multi-dimensional and involve numerous stakeholders at the national and international levels. The way forward to enhance the image and increase the competitiveness of Malaysian palm oil in the global market is through compliance with sustainability certification. The adherence will assist in positioning Malaysia as the preferred global supplier of CSPO.

This review shows that the palm oil industry constantly faces pressure and challenges in conforming to sustainable production and development. This warrants further research using empirical data as issues on sustainability continue to evolve in the palm oil supply chain. Other emerging factors, such as advancement in the research, innovation, and development of new products and technologies and the requirements of importing countries can influence the criteria and principles of sustainability imposed on palm oil. Although this article provides an overall review of the palm oil sector, the study utilizes sources from the Scopus database only. Nevertheless, this research area on palm oil sustainability can provide valuable preliminary insights to policymakers in developing strategies to manage disruption in the global palm oil trade. These strategies serve to safeguard the country's economic interest in the export market and protect the environment and ensure societal wellbeing.

**Author Contributions:** Conceptualization, L.N.; methodology, L.N.; resources, L.N.; writing original draft preparation, L.N.; writing—review and editing, R.M.; supervision, R.M.; project administration, R.M.; funding acquisition, R.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** The APC was funded by Geran Kursi Endowmen MPOB-UKM (EP-2019-047).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


### *Review* **What Nudge Techniques Work for Food Waste Behaviour Change at the Consumer Level? A Systematic Review**

**Hannah Barker 1,\*, Peter J. Shaw 1, Beth Richards 2, Zoe Clegg <sup>2</sup> and Dianna Smith <sup>1</sup>**


**Abstract:** In European countries over 40% of food loss and waste occurs at the retail and consumer stages; this situation cannot be sustained and remediation is urgently needed; opportunities for change must be created. "Nudge" techniques have been shown to be effective in changing behaviour in areas related to food consumption (e.g., healthy diet), but the effectiveness of interventions using nudge techniques to change food waste behaviours remains unclear, despite a growing body of research. The aim of this review is to elucidate means to change household food waste behaviour using nudge approaches and identify priority needs for further research. Four databases, grey literature and reference lists were searched systematically to identify relevant research on nudges to change food waste behaviours. This search identified sixteen peer-reviewed research articles and two grey literature reports that were critically appraised using a critical appraisal checklist framework for descriptive/case series. Four studies deemed reliable show interventions using nudges of social norms, reminders or social norms with disclosure were effective in changing food waste behaviours at the household level, while disclosure alone, i.e., revealing environmental costs of food waste, was not. This review, unique in the application of a critical appraisal, suggests there is reliable information on the effectiveness of nudge for food waste recycling interventions when incorporating nudges of social norms, reminders or disclosure alongside use of social norms. If food waste recycling behaviour is considered an upstream measure to raise consumers' consciousness on the amount of food waste they produce, this may have a positive impact on food waste reduction and therefore has important policy implications for food waste behaviour change at the household level.

**Keywords:** food waste; behaviour change; consumer; household; nudge

### **1. Introduction**

Sustainability aims to protect the natural environment specifically human and environmental health, while compelling innovation so as not to compromise lifestyle for future generations [1]. Approaches to sustainability intend to maintain the delicate ecosystems of earth in balance, usually through encouraging renewable fuel sources, protecting physical environments and decreasing carbon emissions. At present, poor sustainability is a key concern affecting the global food system [2]. This situation is a significant problem globally for societies and governments. Food processing and production create environmental problems along the entire food supply chain [3,4] with direct effects on environmental resources through use of fertile soils, fresh water, energy, fertilisers and release of carbon emissions in the production and transport of food [5–9]. Globally, approximately a third of total food produced is wasted or lost [7] along the food supply chain [10]. In European countries 21–33% of food is lost during agricultural production, 21–25% during manufacturing, storage, processing and distribution, and over 40% at retail and consumer stage [11–13]. In the UK, more than £19 billion worth of food is lost or wasted annually [14]. A focus on tackling this complex problem at consumer level is an essential part of the multifarious

**Citation:** Barker, H.; Shaw, P.J.; Richards, B.; Clegg, Z.; Smith, D. What Nudge Techniques Work for Food Waste Behaviour Change at the Consumer Level? A Systematic Review. *Sustainability* **2021**, *13*, 11099. https://doi.org/10.3390/ su131911099

Academic Editors: Jun Yang, Ayyoob Sharifi, Baojie He and Chi Feng

Received: 4 September 2021 Accepted: 30 September 2021 Published: 8 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

puzzle. The targets set under the United Nations' Sustainable Development Goals call for halving per capita food waste at the consumer level by 2030 [15].

In order to shift from the current situation of high household food waste to a more sustainable future for food waste, behaviour change will be necessary. General information can affect the motivation for and ability to change behaviour [16,17], for example information awareness campaigns [18,19] as frequently individuals lack awareness of environmental sustainability issues relating to food [20]. However, to create change at appropriate scale and speed, additional approaches to awareness campaigns would be required [21]. Research shows that although information is valuable, when offered alone this is not where the key to motivating change lies [19]. Opportunities for change must be created. Constructing opportunities to change household food waste behaviour can be simple, e.g., making preferred choices more accessible. Examples include positioning food waste caddies in households in easy reach to support recycling of food waste, offering household food deliveries containing optimal food amounts to avoid surplus, or encouraging food portioning tools that help to avoid over-portioning and thus reduce plate waste.

Psychologists and neuroscientists have developed a description of brain function based on two systems, system 1—processes that are automatic, unconscious and fast and system 2—reflective, controlled, slow and effortful [22–24]. This dual process is a theoretical basis for nudge theory, with nudge proposing that system 1, automatic decisions, can be systematically triggered to change behaviours and improve outcomes going with the flow of human nature [22,25].

Nudging [26] behaviours in this way, has considerable merit in this context. Nudging was developed from ideas advanced by Daniel Kahneman [27]. It is challenging to offer a universal definition of the term nudge as understandings of nudge can vary broadly [22]. Economist Thaler and legal scholar Sunstein convey the concept of "nudging," defined as "any aspect of the choice architecture that alters people's behaviour in a predictable way without forbidding any options or significantly changing their economic incentives" [28]. There is a growing interest in nudging as despite usually incurring low cost, they can deliver results and be highly effective all the while negating unpopular rule setting [29]. Subsequently, Bornemann and Smeddinck [30] identify five criticisms of nudge: conceptual, normative, functional, empiric and practical [31]. Conceptual criticisms question the reach of nudge and boundary between nudges and other behavioural influences, normative criticisms express concern over potential manipulation of moral concepts relevant to freedom, independence and objective information [31]. Functionality concerns the effectiveness of the nudge approach, while empiric broaches the efficiency of the method and long-term success [31]. Moreover, practical issues concerning knowledge on the decision context reflect resource demand and cost of implementation of nudge approaches [32]. These five concerns mainly centre around hard-to-avoid hidden automatic defaults, which is inconsistent with the definition offered by Thaler and Sunstein [28], thus most normative, functional and empiric criticisms may be overcome with judicious planning and implementation [31]. If resources are available then practical issues may be overcome; this is the same as with countless other interventions. Conceptual concerns are reflective of the wide-reaching applications and understandings possible to the nudge approach and will always provoke discussion due to individual perspectives of nudge [31].

In the food domain, nudging has been applied largely in response to the obesity epidemic [33] and, to a lesser extent to encourage environmental initiatives affected by food consumption, e.g., reducing red meat consumption [34]. For information on nudge interventions more broadly related to the food supply chain a systematic review shows that there is evidence to support "green nudging" as effective in leveraging more sustainable practices for farmers and consumers [35]. There remains minimal application of nudges in the context of household food waste behaviours, and those published tend to focus on eating-out options [36–38]. However, research on food waste behaviours has expanded in recent years, leading to a requirement for a systematic review to appraise critically the

body of research evidence and to understand what works to change household food waste behaviours and what are the priority needs for further research. A recent review developed a systematic map of existing research on behaviourally informed interventions targeting changes in consumer food waste and consumption behaviour [39]. However, a feature missing was 'a critical appraisal of each individual study ... (as) ... this is not a common standard for systematic maps' [39]. Previous reviews on food waste in households have not included a quality assessment of the studies included and have focused on policy actions, interventions for food waste reduction, food waste drivers, causal mechanisms for food waste behaviour, comparison of food waste amounts or avenues for future research. This review addresses the gap in critical assessment.

Frequently applied theoretical bases for behaviour change interventions include the transtheoretical model, social cognitive theory and the theory of planned behaviour [40]. This present review acknowledges that often these theoretical frameworks can work in parallel with nudges [41] and these theories explain why various nudges may be effective. Self-monitoring and other self-regulatory techniques (goal-setting, prompting, selfmonitoring, providing feedback on performance, goal review) are consistently reported as effective behaviour change tools [42,43]. Some of these elements also constitute nudges per se. From a policy perspective, nudge interventions have advantages. Firstly, relative affordability, and secondly, ease of implementation and scope for adaptation in different contexts [28,29,44]. As shown [35,44–46] nudges can be considered not as a replacement to firmer environmental and food policies, but rather as a complement.

Ten optimal nudges have been identified with examples to define the scope of nudge in this review as shown in Table 1. They are in line with definitions described by originators of the nudge concept [29], these definitions are also used specifically in the context of food waste in a peer-reviewed primary data study [31].

**Table 1.** Identification of Nudges.

A. Default rules, e.g., automatic enrolment in programs such as external meal planning and fee-based strategically portioned food ingredient delivery

B. Simplification, e.g., reducing barriers of target behaviour

C. Use of social norms, e.g., Regular exchange about personal experiences on the reduction in food waste with friends and neighbours

D. Increase in ease and convenience, e.g., making low-waste food options visible

E. Disclosure, e.g., revealing environmental costs associated with food waste

F. Warnings, graphic, or otherwise, e.g., Pictures that demonstrate how food waste damages the environment

G. Pre-commitment strategies, e.g., A challenge on household food waste reduction with a friend

H. Reminders, e.g., Tips on shopping planning via email

I. Eliciting implementation intentions, e.g., asking "do you plan to reduce food waste?"

J. Informing people of the nature and consequences of their own past choices, e.g., Feedback on financial costs of an individual's food waste

The aim of this review is to determine what nudge techniques work for food waste behaviour change through comprehensive literature search, review, critical appraisal and discussion of relevant papers.

### **2. Materials and Methods**

To identify the peer-reviewed literature on this topic, four databases were searched: Scopus, IBSS, Web of Science and Psych Info in March 2021. The search terms used were (Nudg\* OR "Architect\* OR Choice Architect\* OR "Behavioural insights") AND ("Food Waste" OR "Food Loss") AND (Consumer\* OR domest\* OR Household\*). Studies were identified on the basis of inclusion (Table 2) and exclusion criteria and then assessed as

full text articles. Figure 1 shows the search process as conducted. For the initial screening of titles and abstracts the free website Rayyan, developed by Mourad Ouzzani, Hossam Hammady and Ahmed Elmagarmid, was used to sort and organise the literature; articles were included or excluded on the basis of the title and abstract fitting the inclusion criteria (Table 2) and exclusion criteria, i.e., not inclusion criteria and no review study designs. Next a file was created on Elsevier Mendeley reference manager for full text PDFs identified. The inclusion and exclusion criteria (Table 2) were applied again to these texts on full reading.


**Table 2.** Inclusion criteria to select relevant papers.

Reference lists were searched in all papers identified for full text articles. The grey literature search was carried out in UK-only institutions due to limitations of resource and English language inclusion. In this review eight major UK supermarket websites (Tesco, Sainsbury's, ASDA, ALDI, LIDL, Morrisons, Waitrose and Coop) and the UK's Waste and Resources Action Programme (WRAP) webpages were searched.

When all papers were collated and those meeting the criteria selected, a critical appraisal of the studies was completed. Previous systematic reviews on food waste have not included a critical appraisal [39,41,47–49].

The quality appraisal is therefore a novel contribution and focused on aspects likely to affect the validity of the results including design, the methods of observation, adequate reporting, statistical analysis, sample sizes and allocation. A framework based on the Joanna Briggs Institute Critical Appraisal checklist for descriptive/case series [50] was applied. No meta-analysis was carried out as the identified studies were heterogeneous in type of design and results. A process of assessment was carried out to determine quality studies, following formalised rules detailed in Table 3:

**Figure 1.** Search strategy and results to select relevant papers.


**Table 3.** Inclusion and exclusion criteria for critical appraisal of relevant papers.

### **3. Results**

Following the search strategy described above, a total of 291 potential articles were identified (Figure 1). The initial screening identified 78 research papers, of which 42 were duplicates (duplicated two, three or four times over the four databases), leaving 36 for full paper search. On reading the 36 papers fully the database search produced 7 papers for consideration. Reference list searching of the 36 papers for full paper search identified another 8 papers for data extraction. One extra paper was identified by an expert. Grey literature studies on food waste were discovered for three UK supermarkets: Tesco, Sainsburys and ASDA. The ASDA study was already captured in two journal publications. Thus, the grey literature search brought 2 extra studies for data extraction. The total number of papers identified for data extraction, from the database search, reference list search, expert advice and grey literature search was 18. Supplementary material Table S1 shows the critical appraisal applied to the eighteen selected studies. Table 4 shows the key results of papers deemed higher quality from the review and illustrates good practice to be replicated. Supplementary material Tables S2–S4 show the summarised results of lower quality papers in the review. Table S1 shows changes to food waste behaviours or perceptions of food waste behaviours in relation to food waste behaviour interventions using nudge techniques. Six studies were published between 2011 and 2016, and twelve studies were published from 2017 to 2020, illustrating the increase in research on food waste in recent years [49]. Ten of the studies were UK based, three were from Sweden, two from Germany, two from The Netherlands and one from Denmark.

### *3.1. Nudge Interventions*

Nudge interventions and associated research were wide–ranging. Three studies did not run an intervention and instead discussed consumer perception(s) of food waste behaviour interventions [31,51,52]. Eight studies used more than one key intervention [53–60]. Most frequently interventions used written information. Five studies used written information interventions incorporating a variety of nudges ranging from disclosure, e.g., environmental impact of food waste from an average household, to individualized consequences to the environment or financial impacts, to reminders, to descriptions of food waste behaviours of other people in the same community, i.e., social norms [53,54,57,60,61]. Four studies with supermarket awareness campaigns all incorporated social norm and reminder nudges [55,56,58,59]. One campaign also included pre-commitment strategies, e.g., making pledges on food waste behaviour [59] and another included a number of tools to increase the convenience of behaviour change e.g. food bag clips [56]. Another intervention included, in addition to nudge, economic marketing techniques, i.e., incentives

and positive communication [55]. Three studies used a food waste sorting bin, arguably a visual prompt or reminder nudge for food waste behaviour [53,62,63]. Other interventions included: sticker prompts for a food waste bin, i.e., a visual reminder nudge [60]; social recipes whereby participants shared ingredients to make recipes together to reduce waste incorporating social norm nudges [57]; verbal information with environmental disclosure [54]; written social comparison feedback on food waste behaviours of nearby streets, i.e., social norms nudging [64]; written reminders and recommendations to change food waste behaviours, i.e., including nudges of pre-commitment strategies, reminders and setting implementation intentions [65]; a measuring tool for portioning of rice and pasta, i.e., increase in ease of convenience [66]; and environmental impact feedback on food waste habits, i.e., a nudge informing people of their individual consequences [57].

**Table 4.** Results following application of inclusion and exclusion criteria for critical appraisal of relevant papers.



**Table 4.** *Cont.*

### *3.2. Study Design and Samples*

Of the eighteen studies selected (Table S1), fifteen were quantitative and three were qualitative. The most common sampling strategy used was convenience sampling (thirteen studies); however, purposive sampling [63], ad hoc sampling [31], random sampling [64,67] and unclear or unstated methods were also used [53,56]. Sixteen studies included participants who were self-selected. Total sample sizes of all eighteen studies ranged from 15 to 64,284, 9 studies used individuals [31,51,52,57–59,62,65,66] and 8 studies used households as the sampling unit [53–55,60,61,63,64,67] and 1 study was unclear [56]. In one study the sample sizes were unclear [56]. Five studies had sample sizes of under 100 [55,57,60,62,63]. Five studies had sample sizes between 100 and 500 [31,52,53,61,65]. Seven studies had total sample sizes of over 500 [51,54,58,59,64,66,67].

Regarding the allocation of the interventions, nine studies used geographical area [53,54,56,60,61,63–66]. Once the geographical area was selected, two studies used random sampling methods [64,67]. Four studies used convenience sampling [54,61,62,65]. Two studies used purposive sampling [60,63] and two studies were unclear on sampling methods [53,56]. Five studies recruited from a supermarket customer base [51,55,58,59,66]. One study [66] included random selection of customer base—all others used convenience sampling. Two studies allocated their sampling strategy from specific locations, i.e., fair or shop [31,66] both studies used convenience sampling and ad hoc sampling, respectively. Two studies recruited from social media [52,57], both used convenience sampling. Two studies recruited from personal contacts [57,62], both used convenience sampling.

The studies largely did not have samples representative of the country's population in which they took place. Eight studies included no statement on how representative the sample was [52,55–59,62,63]. Six studies included a detailed description on demographics indicating how representative the sample was in relation to the local town or area [31,53,54,60,61,65]. Three were representative [51,64,67]. Three studies had randomised samples [64,66,67]. Two studies were random and representative [64,67].

### *3.3. Methods of Assessment*

The majority of studies were clear and transparent concerning their methods of assessment and inherent limitations. In Bernstad [53], which measured food waste weight at multiple time points with comparisons taken over 10 weeks before and 33 weeks after campaigns A and B, the method description of how many households per intervention were included in the food waste weight measurement (written information or bin equipment) was ambiguous. Three studies had two methods of measurement clearly indicated [54,55,57]. Eleven studies used questionnaire methods [31,51,52,54–59,65,66]. Eight studies used weights of food waste as measurement to varying degrees of accuracy [53–57,60,61,67]. Of these studies only one or two of five recommended methods for optimal physical measurement of food waste were used [68]. One study used observations of food caddy placement [64] and three studies used qualitative responses [52,62,63]. Nine

studies relied solely on self-reported data [31,51,52,58,59,62,63,65,66] and nine used an objective measure [53–57,60,61,64,67].

The majority of studies did not specify whether the food waste measured was inedible, or edible-avoidable-food waste; a crucial oversight given the importance of these definitions in practically informing solutions to the issue of food waste. Seven studies did specify the food waste measured was either edible food waste or avoidable food waste [55,57–60,65,66]. Five studies referred to the type of food that was wasted [57–60,66]. Hubbard and Tesco [55] used a photo diary to measure food waste however these findings were not reported in the study summary accessed through grey literature searches.

In all fifteen quantitative studies drop-out rates from interventions were not indicated. It is unclear how easy interventions were for participants to complete. Four studies did report on missing data, i.e., missed bin collections [61,67] and missed responses on online surveys [58,59].

### *3.4. Reliability and Precision*

For the majority of studies, whether or not the assessment was reliable was unclear. It is established from van Herpen and van der Lans [69] and van Herpen et al. [70] that self-reported measures of food waste behaviour change via questionnaire are not reliably accurate as a measure of food waste unless used purely for comparative methods to assess differences between households and ideally within a specified recent timeframe, i.e., the last week. Hence, all studies that used this method of measurement via questionnaire or interview or focus group were classified as "unclear" regarding reliability. The results of the self-reported studies that used quantitative methods are summarised (Table S2–S4). These studies all had relatively small sample sizes of approximately 0–500 and did tend to report positively with regard to the effect of intervention on food waste reduction, however it may be that the positive results from these smaller studies were subject to publication bias.

There were nine studies that used objective measures [53–57,60,61,64,67]. Sainsburys et al. was not considered reliable as sample sizes, methods of recruitment and analysis were consistently unclear [56]. In Bernstad [53] the methods description of how many households per intervention were included in the food waste weight measurement (written information or bin equipment) was ambiguous, hence reliability was classified as unclear, although this study had merit in weighing food waste objectively at multiple time points before and after the intervention. The Hubbard and Tesco study [55] was classified unreliable and imprecise as the report did not clearly describe the statistical analysis for the results and there were no confidence intervals, standard deviations or p values indicated. There was a summary comparison of the average waste from the first week compared to the final week and no other data available. The Lim et al. study [57] was marked as unreliable because the sample size was only fifteen and unrepresentative (all university students between the ages of 20–28). Furthermore, the study did not take travel into account for logistics involved with the social recipe intervention.

Bernstad et al. [54] in Sweden split their sample into two intervention groups. One group consisted of 420 households and included an intervention using nudges of disclosure in written and oral information and nudges of increased ease and convenience, i.e., being given food waste recycling bags. The second group consisted of 210 households and the intervention included written information using disclosure nudges. Food waste weights were recorded at multiple time points over 24 months. P values were included along with clear details of statistical analysis. The results showed overall that there was no change in either group for food waste recycling. This study also included a clear description of the population and discussed how representative the sample was in comparison with the population average of the City of Malmo. However, the study did not include a control group in the design hence it was excluded on the application of quality rules (Table 3). Despite this exclusion, it was noted that there was no change in either group as both received nudge interventions, however without a control group it is difficult to know

whether this would have been replicated or different with other households in the same community without any intervention.

Four studies were considered reliable assessments on application of the quality rules applied (Table 3). Their results were summarised in Table 4 [60,61,64,67]. All four studies addressed food waste recycling or food waste reduction, their interventions were simple and well-articulated, sample sizes adequate or large and precision of results calculated. Three of these studies were based in the UK [60,64,67] and one in Sweden [61].

Nomura et al. [64] in a UK study incorporated a randomised control trial design with two groups, treatment and control, of 5009 and 4073, respectively. The intervention used social norms nudges by applying social feedback on local recycling rates. Regression analysis, standard errors and level of significance (p values) were calculated and showed a statistically significant positive effect of the intervention on household food waste recycling. Mode of measurement was one of observation of recycling food bin to indicate participation with food waste recycling. There were, however, no weight measures or compositional measures of the food waste. Whether the food waste was properly separated or what amount of food waste was to be recycled was not therefore specified.

Shearer et al. [67] in their UK study included a randomised control trial design with treatment and control group of 33,716 and 30,568 participants, respectively. The treatment group received a visual prompt nudge reminder as a sticker on their food waste caddy. Weights of food waste were measured for both groups at multiple time points pre and post intervention and standard deviation and p values calculated, with statistically significant changes in food waste recycling observed.

Linder et al. [61] in Sweden sent an information leaflet with nudges of social norms and disclosure for food waste recycling to the treatment group. Both treatment and control groups were subject to food waste recycling stations near their homes, i.e., arguably a nudge of increase in ease and convenience. Treatment and control groups (264 and 210, respectively) had food waste weighed pre- and post-intervention. Standard deviation was indicated and level of statistical significance (p values) calculated. A positive and statistically significant change in food waste recycling was noted in the treatment group.

The study by Shaw et al. [60] in the UK comprised a sample size of 60, including 3 groups (n = 20 for leaflet using the nudge disclosure for environmental impact, n = 20 for leaflet using the nudge disclosure for economic impact and n = 20 for control). Food waste was measured via compositional analysis and weight. It was the only study, of the four studies (Table 4), that differentiated between avoidable and unavoidable food waste and that separated food waste by food type. A standard error was included in the results. The results showed a lack of differences between the three groups which negated the need to fully conduct statistical analyses usually involved with a before-after-control-impact experimental design.

### **4. Discussion**

This systematic review aimed to gather and appraise the evidence around interventions using nudge for food waste behaviour change. The results contribute to this field of research by identifying the most effective nudge interventions for altering food waste behaviour in households in Europe, providing insights for future policy formation and nudge applications.

There were four studies that were determined to be of higher quality that showed reliable results with three nudges used: use of social norms, reminders and disclosure. The use of social norms and reminders were both shown to have positive influence on change in food waste behaviours [61,64,67]. Disclosure was shown to have a positive influence when incorporated in an intervention for food waste recycling [61]. However, disclosure showed no change for an intervention to reduce food waste [60]. Despite these interventions all using objective measures, optimal methods for physical measurement of food waste as outlined by Elimelech et al. (2018) [68] were not used in any of the studies indicating that although the results have rigorous elements there is room to increase rigor in the methods used to obtain a more robust result.

The outcomes of the present study provide some insight into the application of nudges in changing food waste behaviour, particularly in relation to food waste recycling. If food waste recycling is considered an upstream nudge (visual reminder) that increases awareness of food waste for the consumer the outcome could arguably be a reduction in food waste in households. There exist implications to local government and individuals, and for the practical application of the findings.

### *4.1. Explanation*

Other reviews support the use of social norms as being one of the most influential elements affecting sustainable consumer behaviour [41,47]. It is well documented that consumer behaviour in relation to food is affected by a wide range of personal, social and environmental factors, i.e., personal beliefs, attitudes, knowledge and genetics; social interaction with friends, family and community and the environment—shops, schools, work place, facilities, the economy and technology [31]. The theory of planned behaviour explains this phenomenon as it indicates that attitudes, social norms and perceived behavioural control influence intentions which predict behaviour [40]. Despite good intentions the value-action gap is well documented and it is broadly understood that behavioural nudges may help to bridge this gap. Social cognitive theory also explains why the use of social norms in nudging for food waste behaviour change is effective as it suggests that a focus on observing and learning from others has influence on positive and negative reinforcement of behaviour [40]. This also suggests that social norms should be used with care as social norms have the potential to reinforce negative behaviour [71].

Shearer et al. [67] (Table 4) showed food waste behaviour was changed by the use of nudge reminders. This outcome was supported by other studies [53,55–59,62,65,66] that were not considered for the purpose of this review as there were no objective measures of change in food waste. However, the methods were clear and the sample sizes adequate, thus it was useful to understand the perspectives of consumers towards nudges for food waste reduction to explain the findings (Tables S1 and S4). Aschemann-Witzel [51] used a Likert scale of 1–7 (with 1 being least agreeable and 7 most agreeable) for four demographically different sample groups, the combined total N = 826. The fourth most accepted nudge interventions by all four groups out of thirteen nudges was: "I would like to avoid that food goes bad while stored at my home with the help of very easy tricks and tips" [51]. Von Kameke and Fischer [31] used a 1–5 Likert scale (1 = great support; 5 = no support at all) N = 101. Participants were recruited by ad hoc sampling outside one organic and one discount food store in the City of Lüneburg in Germany. In contrast to Aschemann-Witzel [51] one of the nudges that received the least support was: tips on shopping planning via mail/post (median = 4.49, standard deviation = 1.09); though it was better received online (median = 3.49, standard deviation = 1.69) but support was still lacking [31]. It appears perceptions on nudge reminders are divided but certainly for some groups of people it is perceived as effective in changing behaviour.

This pattern can be explained by the transtheoretical model of behaviour change which splits receptivity to behaviour change into stages: precontemplation, contemplation, preparation, action, maintenance and termination [72,73]. This model outlines that an intervention may be successful—or not—depending on the stage in which an individual is at the time. If an individual is at the action stage a reminder may be well received and effective, but, if an individual is at the precontemplation stage, they may not be interested. Equally some individuals may have more pressures from their environment and social background that may influence their response [47,48]. There would also be a difference between individuals which may be explained by the self-determination theory which references motivations and aspects required for lasting change. The theory suggests motivation 'exists in the individual and is driven by interest or enjoyment of the task itself'. The individual must believe the behaviour is enjoyable or compatible with their 'sense of self',

values and life goals [74]. This is also compatible with the SHIFT (Social influence, Habit formation, Individual self, Feelings and cognition, and Tangibility) framework of sustainable behaviour change whereby there is focus on the individual self, having powerful influences on consumer behaviour, i.e., positivity of self, self-interest, self-efficacy [71].

Two of the studies in Table 4 used disclosure. Linder et al. [61] used disclosure and social norms and did show change in food waste behaviours while Shaw et al. [60] only used disclosure as a nudge and showed no change. This difference may be due to the type of food behaviour change the intervention aimed to disrupt, the former pertained to food waste recycling, while the latter pertained to food waste reduction behaviours. Alternatively, the dual use of social norms and disclosure may be more compelling than disclosure alone. One reason for this outcome may be that the use of social norms lends a positivity to the intervention that offsets the negativity often associated with disclosure. Disclosure may reveal environmental costs associated with food waste. This may not be effective due to the problem of abstractness, information on climate change can be fearprovoking and vague with overwhelmingly large-scale consequences making individual acts feel inconsequential which may lead to green fatigue or demotivate as a result of information overload [71].

In one study individuals' perceptions of nudges of warnings, i.e., pictures that demonstrate the extent of the food waste amounts were collected. Overall the rating offered was 2.91 by 101 participants, the scale ranged from 1 ("great support") to 5 ("no support at all"). This was the only mention of the nudge warning within the review. It is unclear why this has not been used more frequently and whether it is a nudge that could be effective. The WRAP "love food hate waste" campaign commenced this line of engagement in social media campaigns [75]. In other areas, i.e., cigarette smoking, the impact of pictures of tobacco health warnings is shown to have an effect [76]. As food waste connects to a lesser degree immediately with the individual it may be that this approach is less effective due to its relative abstractness. Highlighting minimisation of food waste as a way of boosting nutrition and saving money may be more immediately beneficial to the individual and therefore a useful angle to exploit for mutual benefit of changing food waste behaviours and improving health and food security. Other nudge techniques only used once in the studies reviewed include: pre-commitment strategies, eliciting implementation intentions and simplification. It may be that pre-commitment strategies and eliciting implementation intentions are less used as these nudges require that participants already wish to change their food waste behaviours which may not be the case at all. Simplification by removing barriers is little used as it may be that barriers to food waste behaviour are frequently aspects outside of the consumer's control.

Further explanation for why social norms and reminders can be effective relate to tangibility, that is bringing sustainable behaviour to the personal human level. Often green actions can seem vague, distant from the self or abstract [71,77], only for realisation in the future or not feasible in the face of daily challenges [71]. Changes slowly emerge and uncertainty surrounds problems, solutions and outcomes. Social norms and simple reminders prompting actions at the individual and social level are tangible and key to individuals paying attention and taking part [71]. There is much long-term thinking associated with sustainable behaviours regarding cost to current pleasure to promote a sustainable result in the future. This poses problems as people are often hesitant to sacrifice their own benefit [71]. Yet, carrying out actions with others that help others can offer a positive feeling occasionally described the 'warm glow' effect [78], focusing on these kinds of benefits to the self in the present may increase sustainable behaviours [71]. Framing social norms or reminders as nudges for food waste reduction in this way, e.g., 'reducing food waste will benefit your children's future' may improve their effectiveness.

### *4.2. Implications for Policy*

Nudges of social norms and reminders could be useful policy actions for changing food waste behaviours, particularly because they are inexpensive and adaptable approaches. Such approaches should not replace stricter policy measures for food waste reduction at the household level, but as a complement [35,44]. This discussion will consider usage of these identified nudges for food waste behaviour change in local government contexts.

Overwhelmingly the sub context of policy around food waste is one of inefficient legislation. Filippini et al. [79] state that generally policies underlining production, social equity and governance closely connected with social and economic dimensions of food production and consumption are prioritised and food waste actions are poorly represented. The cost of food is clearly prohibitive as in 2018 the UN FAO data showed approximately 2.2 million people in the UK were severely food insecure [80]. If nudges facilitate better use of purchased food, that could also help to reduce food waste, we may positively impact on food security and diet quality (as often fresh produce is most wasted) in households [81]. Policy is often made without full understanding of the benefits and costs to society [82]. Policy measures of fields connected to food waste such as food (in)security, food safety and low cost of waste disposal may take priority [83]. It is worth acknowledging this and considering how best to frame a policy on food waste so it has more backing and traction. Framing actions for food waste within a food security policy may serve both goals as the two, although distinct, are deeply connected along the food supply chain [11–13].

Governments and stakeholders are keen to find ways to effectively improve healthier food behaviours to encourage improvement in public health [45]. The WHO recently asked for retail environments to encourage consumers in this way [84]. Interventions using nudges have gained increased attention in the international policy debate, particularly in the food context they have been applied to promote healthier food patterns of consumption such as increased fruits and vegetables [44,85–88]. Food waste is a point of intersection between these key issues. As nudges have been shown useful to both these issues, an intervention using nudge to encourage consumers towards plant-based diets could also incorporate nudges towards food waste behaviour change. This is especially key as it has been commonly shown that an increase in fresh fruit and vegetable consumption can lead to an increase in food waste [89]. A suggestion on how this could work would be to nudge consumers to buy fruit and vegetables in forms such as canned or frozen—items that are less often wasted compared with fresh produce but offer nutritional gains [90]. Alternatively, nudging storage of apples in the fridge rather than a fruit bowl would increase their shelf life. Discussion about the approach of linking healthy nutrition and food waste awareness is often neglected in the discussion around food waste. It is an approach that may benefit health and environmental outcomes for local government.

### *4.3. Limitations and Priorities for Future Research*

Overall there is no assessment of study quality and robustness in previous reviews of food waste behaviour interventions. This review adds to the literature by indicating the paucity of quality primary studies using interventions with nudge for food waste behaviour change. This review indicates there is some information on the benefit of nudges (namely use of social norms, reminders or disclosure alongside use of social norms) for food waste recycling interventions, which as an upstream measure may have a positive impact on food waste reduction. However, there is currently limited information on the benefit of nudge for food waste reduction interventions.

There was lack of distinction, in the included studies, between whether or not food waste was edible or inedible, which is key information when considering the effectiveness of edible food waste reduction interventions. It is also key information for food waste recycling interventions as it is helpful to understand whether the increase in food waste recycling is due to edible or inedible food waste as this gives an indication of how to target food waste reduction interventions. Another key limitation included understanding the duration of effectiveness of nudge interventions as studies rarely evaluate long term outcomes [91]; some research articles state nudge may only have short term effects [92]. Thus, methods to attempt to measure the longer-term effect of interventions using nudge should

be incorporated in future studies, studies in other areas, health not food consumption, have achieved this [93] and could thus help to inform this methodology.

Regarding limitations to the current review, qualitative studies hold strength to uncover subject matter and anomalies to add to the body of research, however, they are not a reliable method to uncover whether or not nudge interventions are effective for food waste behaviour change in a generalisable sample. Thus, despite robust qualitative methods from studies reviewed in this paper, we have not deemed them reliable for the purpose of this review and research question to hand [52,62,63]. Some studies included in the present study carried out mixed methods and not all parts of the study were relevant to this review [31,46,51,56,62,66]. In these instances, the parts of the studies that did adhere to the inclusion criteria were included.

In the future we need more food waste behaviour studies that use nudge interventions and measure changes in food waste before and after the intervention using either physical weight measurements using robust methods [68], or photo diary studies using appropriate methods [94] to identify edible and inedible food waste, and capture more data on food groups wasted. There is also a need for studies to use representative samples and control groups when testing the effectiveness of a nudge intervention to change food waste behaviours as well as precision in statistical analysis. There are different outcome measures and effect sizes in almost every paper included in the review; future research could work to overcome these challenges which a more standardised approach so that a synthesis of results could be undertaken with meta-analysis. Future research could also assess effectiveness of nudges to change food waste behaviours in different demographics to find out whether there are differences in the kinds of approaches that work depending on demographics. Another area for future research would be to explore the effectiveness of interventions incorporating social norms and reminder nudges to change food waste recycling behaviours and public health nutrition given that both are shown to be effective in each area and population.

### **5. Conclusions**

In conclusion there is no assessment of study quality in previous reviews of food waste behaviour interventions, thus this review indicates a lack of quality primary studies using interventions with nudge for food waste behaviour change. This review suggests there is reliable information on the effectiveness of nudge for food waste recycling interventions when incorporating nudges of social norms, reminders or disclosure alongside use of social norms. If food waste recycling behaviour is considered an upstream measure to raise consumer consciousness on the topic of food waste this may have a positive impact on food waste reduction. This review illustrates the limited information on the effectiveness of nudge for food waste reduction interventions. Behaviour change models and frameworks indicate nudges work when they are tangible, relevant and beneficial to the individual and their lifestyle. Nudges are inherently flexible and adaptable which lends them to policy implementation in different contexts. Incorporating policy on food waste within policy for food security and public health nutrition may maximise impact.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/ 10.3390/su131911099/s1, Table S1: Critical Appraisal; Table S2: Food waste weight/household or individual/timeframe reported for food waste reduction; Table S3: Percentages reported for food waste reduction; Table S4: Likert scales measuring nudge intervention on food waste behaviour change.

**Author Contributions:** Conceptualization, H.B.; methodology, H.B., D.S. and P.J.S.; writing—original draft preparation, H.B.; writing—review and editing, H.B., D.S., P.J.S., B.R. and Z.C.; supervision, D.S., P.J.S., B.R. and Z.C.; project administration, H.B.; funding acquisition, H.B. and D.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** The research for this article was funded by the Economic and Social Research Council South Coast Doctoral Training Partnership (Grant Number ES/P000673/1).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Supporting data are available in the Supplementary Materials.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


### *Article* **When Harmful Tax Expenditure Prevails over Environmental Tax: An Assessment on the 2014 Mexican Fiscal Reform**

**Sugey de Jesús López Pérez \* and Xavier Vence \***

Department of Applied Economics, ICEDE, University of Santiago de Compostela, 15701 Galicia, Spain **\*** Correspondence: sugeydejesus.lopez@usc.es (S.d.J.L.P.); xavier.vence@usc.es (X.V.)

**Abstract:** This article examines the role of environmental taxation in mitigating environmental problems and contributing to sustainability in Mexico. It focuses on environmental tax revenues and tax expenditures since the 2014 Public Financial Reform (PFR), according to pro- or anti-environmental orientation. The research carried out combines the study of the regulation of the selected tax instruments, their classification and the empirical analysis of the tax revenues and tax expenditures associated with the different taxes over the periods of validity of the taxes and benefits studied, using the databases of the CIAT and the Mexican SHCP. A critical analysis addresses the weak environmental function of environment-related taxes (IEPS, ISAN ... ), as well as the late implementation and reduced impact of the carbon and pesticide taxes introduced in 2014. The evolution of tax incentives and expenditure is thoroughly examined by examining both environmental measures, which have evolved positively but within a very reduced level, and the most prevalent tax expenditure measures, with harmful impacts to the environment. Based on the results obtained, long-term structural changes in the Mexican tax system are suggested. As for the short to medium term, profound changes in tax expenditure are proposed to eliminate of those tax benefits harmful to the environment, introduce of tax benefits for circular activities (e.g., repairing, reusing and remanufacturing) and broaden the carbon tax base and rates. The conclusions include recommendations for moving towards a systemic green tax reform that assists the transformation towards a sustainable economy.

**Keywords:** environmental taxation; tax benefits; tax expenditure; carbon tax; harmful tax expenditure

**JEL Classification:** E62; H22; H23; H24; H25

### **1. Introduction**

Environmental problems are increasingly prominent and currently figure among the most important and urgent items on the global agenda, and consequently in Mexico also [1–7]. In fact, in successive reports by the Intergovernmental Panel on Climate Change [5] scientific teams warn that both greenhouse gas (GHG) emissions and climate change have been accelerating at a strong pace in recent years. An exhaustive analysis by Rockström et al. [2] of the nine main planetary boundaries, indicates that we have already exceeded three of those boundaries (rate of biodiversity loss, nitrogen cycle and climate change) and are approaching the point of no return on several others (phosphorus cycle, ocean acidification ... ). Meanwhile, studies on the evolution of the circularity gap estimate that the circular economy will account for merely 8.6% of the global economy in 2019 [8].

The market does not generate the appropriate corrective mechanisms, nor do policies subordinated to market imperatives [9]. In fact, a recent study by the International Monetary Fund clearly warns that "markets alone cannot provide sufficient mitigation. Market failures, unaddressed and exacerbated by government failures, prevent an adequate market response to the challenge of climate change mitigation" [7]. The problem lies in the essential characteristics of the current economic system. The very dynamic capitalist production model also turned out to be extremely intensive in the use of raw materials, energy and oil, without internalizing the environmental costs derived from them [3,10–14].

**Citation:** López Pérez, S.d.J.; Vence, X. When Harmful Tax Expenditure Prevails over Environmental Tax: An Assessment on the 2014 Mexican Fiscal Reform. *Sustainability* **2021**, *13*, 11269. https://doi.org/10.3390/ su132011269

Academic Editors: Baojie He, Ayyoob Sharifi, Chi Feng and Jun Yang

Received: 8 September 2021 Accepted: 9 October 2021 Published: 13 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

While developed countries are certainly at the origin of most environmental problems, all countries have seen increases in their own environmental issues and their contribution to global problems. Within this dynamic, Mexico has also become a country with serious environmental challenges [4,15]. In fact, Mexico is in 12th place in CO2 emissions in the world (significantly lower in per capita terms), and emissions of CO2 have increased by almost 74.5% since 1990, although moderating in pace in recent years [15]. Environmental policy in Mexico has gone through different transformations that began in the 1980s and generally correspond to global trends and the successive international agreements signed by Mexico (Rio Agreement, Kyoto Protocol, Paris Agreement ... ). The aim has been to combat environmental pollution through direct regulation, soft technological and ecological measures, voluntary instruments (environmental certifications), and some fiscal measures (mainly, subsidies and benefits) [16–18]. A new step was taken with the 2014 Public Finance Reform (PFR), when environmental taxes on fossil fuels (CO2) and pesticides were set up for the first time [19].

This paper seeks to contribute to the existing literature on environmental taxation in two directions: on one hand, by analyzing the situation and the changes introduced in a large country with an intermediate level of development; on the other hand, it tries to balance the efforts for the introduction of new environmental taxes (carbon tax and pesticides) and those related to the environment against the measures focused on tax benefits on the major taxes of the tax system (VAT, corporate, income, etc.). The aim is to examine particularly the latter in order to test the hypothesis that these tax benefits have an enormously greater weight in the revenue of public finances than environmental and environment-related taxes, with the aggravating factor, moreover, that the former are for the most part tax benefits that are harmful to the environment.

This article provides an analysis of the more recent changes. Following the introduction in Section 1, we review the theoretical and empirical literature on environmental taxation in Section 2, highlighting the advantages and limitations of these environmental policy instruments. Section 3 gives information on the databases used for the empirical study on Mexico. In Section 4, we analyse the implementation of environmental taxation in Mexico, particularly from 2014 onwards, giving attention to tax revenues and tax expenditure. In Section 5, we summarize the main results, and Section 6 offers some conclusions and recommendations.

### **2. Literature Review on Environmental Fiscal Policy**

Environmental fiscal policy is directly aiming at creating economic incentives to promote positive environmental behaviour by the various economic actors (producers, dealers, retailers, consumers, public institutions, finance ... ). It is transmitted to economic agents through tax instruments, public expenditure and fiscal or tax expenditure, including subsidies, reductions and benefits [7,20–22]. Tax expenditure differs from public spending in that the former involves the renunciation of public revenue that is made operative through the existence of incentives or benefits that reduce the direct or indirect tax burden of certain taxpayers in relation to a reference tax system or 'benchmark' [23].

Most of the literature analysed on fiscal policy and environmental problems starts with the so-called Pigouvian taxes [3,24–33]. To Pigou and the environmental fiscal literature, taxation by the public sector is the best way of internalizing, through prices, the social cost of negative externalities that were not reflected in the market price. This idea was firstly developed in the theoretical framework of welfare economics [24], which, though often overlooked, poses analytical conditions that are difficult problems to solve in real implementation and practice. A precise calculation of externality would be required to establish an optimal tax, one equivalent to the social cost and added private benefit. The idea has been transferred to the debate on environmental policies in a more general and pragmatic formula: the polluter pays principle. One way of making this happen is through environmental taxation [26,32].

Environmental taxes are defined in the literature as compulsory tax payments at a fixed or variable rate, which must be paid by polluting agents based on facts related to the pollution caused by their production or consumption. These facts set the tax base for using the mechanism of environmental taxation to address pollution problems and social costs. While there is no single definition, the environmental tax is expected to reorient production patterns and alter consumption patterns, regardless of the destination of the revenue obtained [3,21,32,34–37]. Environmental taxes are economic instruments to the extent that they affect the cost and price of goods, and by that means influence their consumption. In other words, the environmental tax works as a price altering agent to discourage uses and consumption that pollute the environment and encourages innovations in a more environmentally friendly direction. Environment tax instruments can create market incentives to develop and invest in emission-reduction technologies, to encourage behavioural changes in consumption and production and to achieve least-cost solutions. Moreover, according to the double dividend argument, the tax instruments can generate revenue that could be used to finance environmental expenditures or to mitigate adverse impacts on the diverse social groups [27,28,36,38].

Although the environmental taxes are preferable to pollution market [3,29,38–44], taxes also have some critical flaws. Current environmental taxation focuses on penalizing pollution through price, but does not prevent it. The implementation of the Pigouvian approach resulted in the first generation of environmental taxes, leading to set a tax for each specific environmental problem (different chemical pollutions, energy, NOx, CO2 ... ). Such an approach generated a multitude of scope-limited proposals that eventually proved ineffective in addressing growing, complex and interrelated environmental problems [5,6,28,45,46].

Though taxation is increasingly recognized as an important environmental policy instrument, real progress in implementing environmental fiscal policies have been modest in Mexico and around the world [6,20,28,47,48]. Indeed, the numerous environmental taxes that have been implemented in recent decades—probably more than a hundred—have not met their environmental objectives [5], and their use is currently limited. According to Groothuis [45], "over the past 15 years, environmental tax as a share of GDP has declined in 52 out of 79 countries in the Organization for Economic Co-operation and Development (OECD) database. In addition to relatively low green tax levels, global fossil fuel subsidies amounted to \$373 billion in 2015."

The real evolution of the most serious environmental problems, such as climate change or urban and ocean pollution, show the insufficiency or failure of the measures that have been adopted in the last three decades. This applies to emissions market mechanisms especially, but also to CO2 taxes insofar as they have been implemented.

A recent paper issued by Best et al. [49] ran a cross-country empirical study on the efficacy of carbon pricing, showing that "a negative association between carbon pricing and the subsequent CO2 emissions growth rate, with a one euro increase in the effective carbon price rate per tonne of CO2 emissions being associated with a 0.3 percentage point reduction in the annual rate of emissions growth." At the same time, the most recent scientific reports unequivocally indicate that CO2 and other GHG emissions have increased and even accelerated in the years since the 1997 Kyoto Protocol, and despite the launching of the EU Emissions Trading Scheme (EU ETS) in Europe and other countries, with only a brief and apparent respite caused by the Great Recession at the end of the last decade [5]. This disappointing result could suggest that carbon taxes (and other market solutions) run the risk of failing if the taxable events are narrow and the tax rates very low. Through the influence of companies and interest groups, carbon prices end up being set so low that they are ineffective in curbing emissions [13,46]. "The problem is in the economy: if the tax is very moderate, it fails to remove enough fossil fuel to help the climate; but if it is high enough to actually reduce it, then business and consumers resist the tax—because without some safety cushion for business and consumers, the whole problem falls on them and they rationally resist—to save profits and jobs" [46].

As previously stated, the environmental tax policy includes other relevant instruments, particularly public and tax expenditures. Tax benefits (or tax expenditure) materialize into government fiscal waivers [50–53] which can be granted to economic agents in very diverse ways (incentives, tax relief, deductions, accelerated depreciation, etc.). From an economic point of view, justification for these tax benefits may be linked to industrial and trade policies favouring certain economic activities or supporting some consumption patterns, as part of a social or redistributive policy [52,54]. The usage of tax benefits as an incentive—a carrot—to promote environmental objectives is well established, in fact, is one of the first instruments of environmental tax policy [52,55,56].

Tax expenditure is an opaque and little-studied subject, instead of existing estimates placing tax expenditure at between 14–24% of total revenue in most countries and, in some cases (e.g., US, UK) that proportion exceeds 30%; as a proportion of GDP, the available estimations go from 3.7% in the Latin American countries to 8% in the US. Mexico also makes extensive use of tax expenditure measures, reaching up to 23.8% of total revenues and 3.15% of GDP in 2019 [23,57]. The question here concerns whether these instruments have been used in environmental policy to correct the negative environmental impacts caused by the various economic sectors, or the other way around. The literature on this issue is scarce, is mostly focused on discussing which tax expenditure instruments have an environmental purpose and, where appropriate, is concerned with assessing their effectiveness, either theoretically or empirically. However, the above-mentioned literature does not take stock of tax expenditures that serve other purposes and have a harmful impact on the environment.

This article attempts to fill this gap and provides empirical evidence of the evolution of the environment tax policy in Mexico in recent years, balancing the evolution of taxes and the benefits, including both environmental-friendly benefits and harmful ones.

### **3. Databases**

The database used for this empirical study includes official sources such as the Inter-American Center of Tax Administrations (CIAT) [58,59] for revenues (1990 to 2018) and tax expenditure (2014 to 2018), updated with 2019 information from the SHCP and the SAT (Tax Administration Service-Government of Mexico). In addition, the statistics corresponding to the IEPS on fossil fuels (CO2) (2016 and 2017) were adapted according to the SHCP methodological note on carbon [60]. We focused on environmental fiscal instruments (taxes, incentives and tax benefits). An overview of environment-related taxes (1990–2019) is also developed, including information on six different taxes collected at the federal level (Figure 1).

The tax expenditure statistics (2014–2018) were compiled from CIAT data using dynamic tables. The incentives and benefits were classified by type (exemptions, reduced rates, depreciation, deferral, deductions, etc.). To identify the categories in the working database, we applied the following process: the filtering of the data group of environmentally friendly and environmentally harmful tax measures was done based on the description of the regulatory source (tax law for each tax instrument and Mexican federal income law) and according to the tax measure included in the tax expenditure database for each indicator or tax (VAT, ISR, ISAN and IEPS). Thus, the categories (a) environmentally friendly (including all measures with clear environmental characteristics and/or renewable components) and (b) environmentally harmful (fiscal measures that stimulate non-environmental investment opposed to the purposes of environmental policy and incentives that generate stimuli for consumption of products such diesel and fossil fuels) were obtained. The classification of the category by type of tax expenditure was made according to [23,62,63].

**Figure 1.** Evolution of environment-related taxes in Mexico, 1990–2019 (IEPS-G&D right scale) (Units: Percentage of GDP). Source: prepared by the authors, based on [58,61].

### **4. Implementation of Environmental Taxation in Mexico**

### *4.1. Environmental Tax and Environmental-Related Tax in Mexico*

Environmental fiscal policy took its very first steps in the form of tax expenditure (stimulus, incentives and tax benefits) in the early 1980s, as a second-order instrument to reinforce the environmental objectives being implemented through national development programs. In 1981, a government decree established the Fiscal Incentives for the Promotion of the Preventive Activity of Environmental Pollution [64]. In addition to direct regulation measures and voluntary instruments (ecological certifications), fiscal instruments were introduced to provide incentives for the immediate depreciation of investments in equipment for controlling and preventing pollution, along with tariff reductions on imported industrial equipment of this type. Some taxes and fiscal instruments concerning the use of energy resources or specific consumption, which are sometimes considered environmentally-related taxes (e.g., IEPS-G&D, ISAN), have been around for decades. However, environmental taxes that were explicitly designed as such appeared in 2014, resulting from the country's commitments to international agreements on climate change. The 2014 Public Finance Reform (PFR) established nine major objectives for improving the state's income capacity, promoting equity and reducing tax evasion and abuse by reducing existing fiscal incentives and expenditures and creating conditions for the liberalization and privatization of the energy and hydrocarbon sector. The environmental goals behind the measures included "fighting obesity and protecting the environment: fiscal provisions are established to discourage the consumption of goods that are harmful to health and the environment." In particular, two new green taxes were introduced: a tax on fossil fuels, or a carbon tax (quota per CO2 content of various energy products), and a tax on pesticides (according to the level of toxicity), categorised as IEPS-Other consumption [16,19,65].

A conceptual and terminological clarification is needed before entering into data analysis. The OECD and the European Union have agreed to distinguish environmental taxes from so-called environment-related taxes, as a broader category that includes taxes "whose basis is a physical unit (or a proxy of a physical unit) of something that has a specific and proven harmful impact on the environment [66], in accordance with the tax base, regardless of whether the tax is intended to change behaviour or is imposed for another reason." In practice, these two categories are used interchangeably, which undoubtedly causes some confusion. Arlinghaus and van Dender [16] distinguish four subsets of environment-related taxes: energy taxes, transport taxes, pollution taxes and resource taxes. For its part, the CEPAL [20] distinguishes three conventional categories for classifying all taxes as environmentally related according to the tax base under consideration: (i) energy taxes, which include taxes on products related to energy generation with polluting effects, such as fossil fuels and electricity, in addition to those used in transport, such as gasoline and diesel; (ii) taxes on transport, which includes the full range of taxes on motor vehicles and other motorised means of transport by virtue of their marketing (domestic or imported), ownership (recurrent taxes), registration and circulation permits or road use; and (iii) other taxes on pollution and the use of natural resources. The latter include many of the least developed, least used instruments, regionally and internationally, along with taxes on gaseous substances, water extraction and disposal, extraction of natural resources, and pesticides and fertilizers, among others.

According to this definition, the following environment-related taxes exist in Mexico: gasoline and diesel tax (IEPS G&D), tax on petroleum income (Legal Entities), tax on the activity of exploration and extraction of hydrocarbons (Legal Entities), new car tax (ISAN), and of course, the carbon tax (CO2) and pesticides tax (Figure 1). However, a more detailed analysis reveals that some of them (the first four) are not environmental taxes, since they were designed for increasing public revenues or to adjust the price of certain resources to scarcity perspectives and not to reduce pollution or other harmful effects. Their taxable fact and taxable base do not include explicit incentives to modify the agent's economic behaviour. Moreover, some benefits applied to these taxes are clearly anti-environmental. Consequently, they do not really work as environmental taxes.

In this section, we will review the entire family of taxes in force in Mexico, starting with those which are environment-related (IEPS-G&D, ISAN, Petroleum and Hydrocarbons), and then the environmental ones (IEPS-Pesticides and CO2) (See Figure 1).

The IEPS G&D was introduced in 1980. Despite being a tax on a highly polluting products—with CO2 and other GHG emissions and effects on urban pollution and human health—its specific design clearly does not present environmental tax features and the evolution of revenues does not fit with the environmental policy objectives. The IEPS rate would be calculated according to the following formula as presented by Hernández and Antón (2014): IEPS rate = (αi,j PVP − C − F − PP)/PP, where αi = 0.9091 when VAT = 11% and αj = 0.8696 when VAT = 16%. PVP is the public price of gasoline, F is the freight and transport cost, and C is the commission to distributors, PP is the final producer price (PP = PS + AC + CT + CM, where PP is the final producer price, PS is the spot reference price (average US Gulf Coast gasoline price), AC is the quality adjustment, CT is the transport cost and CM is the handling cost). The saw shape of the curve over the period analysed—including a decade of negative revenues—is conditioned by the international crude oil reference prices, since the IEPS-G&D was designed with this complex formula that seeks to adjust the differences between international and national prices but does not really seek to reduce consumption [67].

There are also a number of subsidies that are implicit in the consumption of these products and clearly encourage pollution [16,48]. These strong subsidies explain the negative fiscal revenues from 2005 to 2014 (see Figure 1). Therefore, the reduced price does not provide sufficient signals for a reorientation of the consumption and production patterns that would lead to reductions in the pollution generated by these activities. In fact, the main specific changes of the 2014 Energy and Tax Reform were the gradual elimination of diesel subsidies in the transport sector [68]. These changes partially explain the evolution of the fiscal revenue curve, which takes positive values after 2014, reaching a maximum of 1.4% of GDP in 2016 (right axis), then falls in 2017–2018 and rises again in 2019 up to 1.2%.

This analysis of the characteristics of the IEPS-G&D and its revenue trend, including a decade of negative results, lead us to question its classification as an environmental or environment-related tax by the OECD, CEPAL or the World Bank. In fact, it does not really even qualify as an actual tax, as its special design makes it a pseudo-tax [57]. This reconsideration would imply that the implementation of environmental taxation in Mexico is still in a very incipient state. If this tax is excluded from the group of environmental taxes, an evaluation of the degree of compliance with international environmental commitments would reveal a much more critical state than that indicated by the OECD report on the carbon pricing gap (CPG) [69]. In fact, the estimated CPG for Mexico indicates that 69% of the parameters established for the 2030 scenario have not been met. Obviously, if we exclude this tax, which brings in the highest revenues of all those considered "environmental" by that organization, then the carbon pricing gap for that year would be much higher than the current OECD estimate [69].

ISAN is a tax on the purchase of new cars and applies a progressive rate to the purchase of luxury cars. Revenue from this tax reached 0.04% of GDP in 2019. As a tax that implies purchasing power, economic growth informs the evolution of fiscal revenues (falling down during the tequila crisis of 1994–1996 and the financial crisis or Great Recession of 2008). This instrument mainly taxes luxury goods, but does not incorporate the achievement of environmental objectives.

The tax on Petrol Rent and the Exploration and Extraction of Hydrocarbons is a unique case in Mexico, a nationalized producer of crude oil and hydrocarbons that is organized through the public company Pemex. This tax is not related to conventional income tax (since Pemex is exempt from it) and revenues are linked to Pemex control mechanisms and operational management [70]. Environmental motivation in the resource appropriation policy still very marginal, given that Mexico has prioritized a policy of economic growth that has always been supported in this sector in times of boom [71].

In any case, all environmental and environment-related taxes show a very limited revenue-raising capacity. Recent CEPAL [20] and OECD et al., [72] reports show that revenues from environmental taxes in the 23 Latin America and the Caribbean countries averaged 1.1% of GDP in 2018 (and 1.3% since 2006), which is considerably lower than the also-modest 2.3% average for OECD nations as a whole for the same year. Mexico is well below the OECD average, slightly below the Latin American Countries (LAC) average, and behind countries such as Costa Rica, Chile, Uruguay or Argentina. Even so, energy taxation is the most prominent component in the Mexican case, given the minimal presence of the other three tax bases (pollution, resources and transport).

IEPS-Other consumption includes a conglomerate of specific taxes applied to different products that are intended to provide a price disincentive to harmful consumption. With PFR 2014, two environmental taxes were created in this group: the tax on fossil fuels, or carbon tax (CO2) and the tax on pesticides. The first was determined following the IPCC Guidelines for National GHG Inventories, which is part of the strategy to reduce CO2 and other greenhouse gas emissions. The second, the tax on pesticides, was determined using rates ranging from 0 to 9 percent, according to the level of acute toxicity from exposure to these products, and is subject to the parameters established by the Official Mexican Standard NOM-232-SSA1-2009 for pesticides, "which establishes the requirements for the packaging, packing and labelling of technical grade products and for agricultural, forestry, livestock, gardening, urban, industrial and domestic use" [73]. This proposal will induce the replacement of more toxic agrochemicals with others that are less harmful.

The environmental taxes on CO2 and pesticides were established with two central objectives: (1) to reduce pollution levels (of GHG emissions and harmful products, respectively), and (2) to increase tax revenues (designated to the general budget). Here we will look at the extent to which these objectives have been met. The carbon tax (CO2) is levied on a quota linked to the amount of CO2 in each fossil fuel and based on exchange quotes for the main carbon markets. The petrol tax, for example, was set in 2014 at 10.3 cents peso per liter, diesel at 12.6, fuel oil at 13.4 and coal at 27.5 per ton. This tax amount will remain

constant in real terms, as it is adjusted annually by the variation in the National Consumer Price Index (INPC in Mexico). However, natural gas and turbosine were exempted from the tax at the express request of the private sector, which effectively undermines emissions reductions in large sectors of the economy. The CO2 tax is also not applied when oil is used for manufacturing, e.g., for the production of plastics, rather than combustion [74]. In fact, Mexico shows a low effective carbon rate, taxing above EUR 30 per tonne of CO2 for only the 30% of emissions from energy use (mainly from road transport sector), with most of emissions (68%) from industry, electricity, and the residential and commercial sectors remaining unpriced [68].

Environmental taxes are barely significant in collecting revenues and even show decreasing performance (Table 1). Revenues from these taxes represented a meagre 0.024% of GDP in 2019 (0.003% from pesticides and 0.021% from the CO2 tax). Even more relevant is the continuous decrease in the capacity of CO2 tax to create resources over time (Figure 2). Revenue in 2019 amounted to 53.3% of what was obtained in its first year. Meanwhile, the pesticide tax has been basically irrelevant as far as revenue is concerned. Arlinghaus et al. [16] observed that the effectiveness of pesticide taxes in reducing the use of harmful products is hard to establish due to a relatively dynamic market, the wide variety of products, storage behaviour prior to tax increases, large seasonal and geographical variations in the intensity and frequency of treatments, and the effects of competing regulations.

**Table 1.** Annual revenues from environmental taxes (CO2 and pesticides) in Mexico, 2014–2019. Source: Prepared by the authors, based on [58,61,75].


**Figure 2.** Trend of environmental taxes in relation to total revenues and GDP (left-CO2 and right-Pesticides) in Mexico, 2014–2019. Units: percentage; source: prepared by the authors, based on [58,61].

Regarding the actual evolution of the emissions burden, long-term data series show that CO2 emissions in Mexico have increased by almost 74.5% since 1990, albeit moderating

in recent years [15]. It is interesting to observe the parallel evolution between the drop in CO2 tax revenue both in absolute terms and as a proportion of GDP and the drop in the GDP growth rate over this period. Indeed, there is a very close statistical correlation of R2 = 0.446 between the two variables, which suggests several hypotheses (Figure 3). First of all, the design of the CO2 tax in Mexico applies to a reduced tax base (a limited number of activities) and has a very low price per ton of CO2. Second, its evolution seems to be directly dependent on the growth rate, since growth deceleration is most strongly reflected in CO2 emissions and in tax revenues. This also reveals how fossil fuel consumption is extremely sensitive to the country's economic progress. Thirdly, the evidence suggests that the implementation of a carbon tax with this specific design has had no significant effect on the evolution of emissions or economic activity.

**Figure 3.** CO2 tax revenue (% GDP) and annual GDP growth (right axis) in Mexico, 2014–2019. Source: prepared by the authors, based on [58,75].

These hypotheses suggest some nuances relative to the empirical literature on the topic. The meta-regression study conducted by Galindo et al. [76] on the potential effects of a CO2 tax in Latin American countries shows that the impact of implementing a CO2 tax on GDP will depend on the structural conditions of each country. For Mexico, all the estimated long-term results show negative impacts [48]. However, these results seem to be associated with being evaluated as an OECD member country and applying the OECD coefficients (the same happens for Chile). Alatorre et al. [77] conducted a study to determine the effects of the CO2 tax on economic, social and technology transfer variables through a recursive system of equations. That author suggests that for LAC, the effect of the tax on these three variables depends largely on the non-linear relationship between the increase in relative energy prices and per capita GDP. Thus, in a context of low prices, economic and social policy would have negative effects, while a context of high prices would result in greater environmental benefits. In order to reduce the negative effects associated with the tax, both studies [76,77] suggest a combination of mitigating policies (e.g., tax reductions on the labour factor). They also highlight the importance of applying a CO2 tax in order to encourage the implementation of new technologies (energy efficiency), the development of less-energy-intensive sectors and the generation of jobs that improve environmental conditions.

To sum up, the introduction of the carbon tax in 2014 made Mexico the 27th world jurisdiction to implement a CO2 tax in accordance with the Kyoto Protocol and the Paris Agreement, both signed by Mexico. Mexico was the pioneer in the LAC region, followed by Chile (in 2017), Colombia (in 2017) and Argentina (in 2019) [78]. However, the price established per type of fossil fuel was too marginal to influence the behaviour of economic agents (the minimum threshold is suggested around EUR 30 per ton of CO2, including all activities, and preferably above 100 in the rich countries). After 6 years of implementing both environmental taxes, the initial objectives are very far from being met. The taxes have been ineffective in both reducing pollution levels (GHG emissions and harmful products) and increasing tax revenue beyond a symbolic level which; it is actually declining. So, the empirical results show that environmental taxation in Mexico has not achieved any of the objectives that were established in the 2014 General Economic Policy Criteria [19] and still has a very wide gap to reduce in terms of pollution levels.

However, this problem is not exclusive to Mexico; it is general and global, as emphasized by different global reports. "After a quarter century of academic debate and experimentation, a gap persists with respect to the 'carbon price change' needed to trigger rapid changes ( ... ) The Mexico performance is relatively poorer as a result of the low-price set for each Tm of CO2 emissions and the reduced range of sectors affected" [5].

Due to the environment taxation is failing in promoting a real change towards sustainability, we are dealing with other fiscal instruments such as tax expenditures in order to assess real performance.

### *4.2. Environmental-Frienly and Environmental-Harmful Tax Expenditure in Mexico*

The many existing tax benefits can be classified from different perspectives [52,55]. Here we will focus on three main categories: the objectives pursued, their environmental effects (positive or negative) and all other objectives, be they social, cultural, employment, competitiveness, etc. The last categories were grouped into a single block and isolated them from the analysis, though they are more abundant in Mexico and in many other countries [23].

We are focusing here on incentives and benefits that affect consumption, investments that favour the environment or reduce environmental impacts, whether explicitly established or not. Meanwhile, we will also examine benefits granted for consumption and investments favouring activities harmful for the environment. By doing so for the period 2014–2019, we are providing an assessment of the real environmental content of the 2014 Public Financial Reform.

The benefits and incentives related to environmental protection apply, above all, to the investment cost (deductions and accelerated depreciation) for investments in fixed assets of electric vehicles, vehicles powered by rechargeable electric batteries, electric bicycles and machinery and equipment for generating energy from renewable sources. Together, these accounted for only 0.07% of tax revenue and 0.01% of GDP in 2018 (Table 2 and Figure 4). In examining the magnitude of the incentives and benefits at the level of each tax for the last available year, it can be seen that the environmental cut was granted mainly through the ISR (0.31% of the revenue from this tax) and, to a lesser extent, the ISAN (which represents 0.45% of revenue). Most of these were reductions for qualifying investments (0.05% of total revenue) and other deductions (0.02% of total revenue). For income tax purposes, the incentive consists of a 100% deduction of the investment in clean technology and the depreciation of renewable assets. This reduces the taxable base for determining profit, allowing companies to declare high investment costs, which could even be reflected as losses and thus be tax-free. Direct benefits are also granted through an ISAN exemption (it is not included in the payment) for the purchase of ecological cars.

On the opposite side is the group of fiscal measures promoting productive activities and consumption practices with clearly unsustainable components that contradict the objectives of environmental policy in Mexico [79–81]. These environmentally harmful benefits include incentives for diesel and fuel consumption, which generate CO2 emissions, or benefits for resource extractive industries and the consumption of materials (Table 2). This relatively more substantial group accounted for 1.66% of total tax revenue and 0.26% of GDP in 2018. Most represent IEPS benefits (5.02% of this tax) from the consumption of diesel, fossil fuels and gasoline in the agricultural and transportation sectors, corporate income tax (3.14% of this tax) from reductions on investments and other deductions, and ISAN exemptions that are applied to the purchase of high-priced vehicles (1.48% of this tax). Similarly, the reduced VAT rate (1.41% of this tax) related to drinking water supply services is understood as a social-economic policy, but has been categorized as anti-environmental, as it is equivalent to tax-free use of a natural resource.

**Table 2.** Tax revenues and environmental-friendly and environmental-harmful tax expenditure in Mexico, 2018. Source: prepared by the authors, based on [59].


**Figure 4.** Environmental-friendly and environmental-harmful tax benefits as a percentage of total revenue in Mexico, 2014–2018. Units: percentage of total tax revenues; source: authors, based on [59].

> Incentives for the consumption of new products, such as investment allowances or investment exemptions, especially when such benefits are not conditional on the use of the most environmentally friendly technologies, could encourage consumption practices that are undesirable from an environmental point of view. Such incentives encourage the consumption (and waste) of new materials, components and inputs from non-renewable resources. Accordingly, they are contradictory to the objectives of the circular economy, which seeks to extend the useful life of goods that can be repaired or maintained as long as possible. Although the investment incentives have declined significantly since the 2014 Reform in terms of GDP (from 0.24% in 2014 to 0.08% in 2018), more decisive cuts are

desirable. Furthermore, even when the 2014 Reform introduced the tax on fossil fuels (CO2 tax), the benefits of this tax to the transportation and agricultural sectors remained in place at least until 2018 (reaching a peak in 2017: 2.96% of total revenue and 0.46% of GDP), only starting to fall after that year from the definitive withdrawal of diesel benefits for the transport sector.

In short, evolution in the period since the reform (Figure 4) shows a slow, partial, but progressive reduction of incentives and tax benefits that have a clear anti-environmental bias. Though slimmer than before, they continue to be very broad and carry much more weight than environmental incentives and benefits. The latter have quadrupled since the reform (with a very striking jump in 2017 when deductions increased for electric vehicles and bicycles, and clean technologies), but the absolute and relative levels of these incentives are still extremely low (0.01% of total revenue).

As we have already pointed out, most of the incentives and benefits granted for other purposes (social, culture, foreign trade, etc.) are difficult to classify using environmental criteria. These tax expenditures represent 18.97% of total tax revenues and 2.97% of GDP.

To summarize, the reality of tax expenditure in Mexico still disappointing. Tax benefits and subsidies have been used extensively to favour industrial activities with high consumption of material inputs and non-renewable resources, or even natural resource extraction activities (hydrocarbons and mining). Also, many subsidies to the consumption of fuel, diesel, energy use and water in homes and industries have been applied in Mexico before and after the 2014 Reform [23].

Therefore, most of the subsidies and tax benefits presently in use do not qualify as environmental policy, but on the contrary benefit activities harmful to the environment. This increases the urgency of the challenge to redirect or drop these incentives, especially because of their harmful effects on the environment, as suggested by Stahel [82] and Martínez and Roca [3]. This argument should be distinguished from other questionings of subsidies and benefits. In fact, the OECD [48] recommends they be eliminated because international experience shows that such instruments diminish tax revenues and the effective reallocation of resources, while their effective impact on growth, productivity and investment remains unclear. Anyway, the redistributive effect of eliminating subsidies and implementing carbon taxes on household income in Mexico should be considered (some studies suggest a progressive impact, e.g., Rosas-Flores et al., [83]).

### **5. Results**

The examination of the available information regarding environmental and environmentalrelated fiscal instruments in Mexico since 1990, and particularly since the 2014 Public Financial Reform, allows us to highlight the following results:

Environmental taxes in Mexico are still underdeveloped when compared to European countries and other OECD countries, which also show a poor performance. The environment-related taxes are also quite lower than in European countries and, for years, the OECD countries. It is worth mentioning that Mexico is reaching the mean OECD countries in the very last years, but its evolution is highly dependent on the variability of the main tax, IEPS-G&D, whose classification as an environmental-related tax is highly debatable.

Revenue volatility in IEPS-G&D is caused by its very design as an external-internal price adjustment mechanism for hydrocarbons. Such a sui generis design converts this tax into a pseudo-tax and, as a consequence of a political price strategy in the hydrocarbon sector, it includes implicit subsidies to reduce the effects of high prices for consumption. It is characterized as an environment-related tax because the product being taxed (fuel) has clear environmental impacts involving the use of natural resources and the polluting emissions resulting from its consumption. However, this tax does not incorporate explicit environmental criteria or objectives (concerning emissions or other impacts); neither its design nor its implementation respond to an environmental purpose.

Since the 2014 Reform, steps have been taken in the implementation of environmental fiscal instruments, both on the tax side and on the fiscal expenditure side, but they are modest in scope and reflect the approach adopted in the first generation of environmental taxation. Tax revenues from environmental taxes (CO2 and pesticides) reveal a very limited tax base and extremely low prices that barely reached 0.036% of GDP on average between 2014 and 2019. In current currency, the carbon tax has evolved negatively (revenues in 2019 were only 53.3% of those obtained in its first year), while the pesticides tax has shown a positive but insignificant trend.

There is a positive correlation (R2 = 0.446) between the CO2 tax/GDP ratio and annual GDP growth, with a stronger fall in GDP growth than in the CO2 tax, suggesting that economic growth is the main variable explaining the pace of emissions (instead of the tax).

Since the 2014 reform, some anti-environmental incentives and tax benefits have diminished, but they are still very broad and outweigh environmental incentives and benefits, which, while growing, remain at very low levels. Incentives and benefits which promote practices harmful to environment represented 1.66% of total revenue and 0.26% of GDP in 2018, of which almost half (0.12% of GDP) corresponded to activities with high CO2 emissions. Meanwhile, environmental-oriented benefits scarcely accounted for 0.07% of revenues and 0.01% of GDP in 2018, which makes their positive evolution practically irrelevant.

The highest percentage of ecologically harmful tax waivers were granted under ISRbusiness (0.11% of GDP in 2018), mainly in the form of investment deductions (0.06%) or tax credits, or other credits and reductions (0.04%).

### **6. Conclusions and Future Research**

Environmental fiscal policy is a potentially effective and fundamental instrument for achieving environmental objectives. Beyond public spending (green public investments, green public procurement, subsidies, etc) to finance actions to mitigate or repair environmental damage, we are focusing on the importance of other fiscal instruments such as taxation and tax expenditure (a large range of tax benefits as exemptions, reduced rates, depreciation, deferral, deductions, etc.). Taxes and tax benefits are flexible instruments that can alter the prices of goods or services, generating price signals and incentives to reorient the behaviour of economic agents (producer and consumers) towards sustainable patterns. They have the potential for transversal impact along the economy and they can generate resources for financing public environmental spending or compensating the negative redistributive effects of such taxes. Moreover, they are relatively easy to manage by the tax administration.

The analysis of the tax regulation shows that, since the 2014 Public Finance Reform, Mexico has taken steps in using fiscal instruments to respond to environmental policy objectives. Environmental taxes (CO2 and pesticides) have been incorporated and tax benefits and incentives have been reduced for some specific polluting and harmful activities and consumption. However, the empirical evidence provided in this paper shows that the modest aims and commitments expressed in the 2014 Fiscal Reform are far from being substantiated. The detailed analysis of measures and results suggest some explanatory factors to the modest performance. On the one hand, environmental taxes have minimal incidence and revenue capacity due to the narrow range of activities subject to the tax and the extremely low prices of carbon emissions. To reach some effectiveness it should be necessary to expand substantially the tax base of the carbon tax, including all relevant emitters, and increase the tax rate at an accelerated pace (taking as reference some successful countries, e.g., Sweden (see Sterner [84]), or the expert recommendation to start at about EUR 50 per ton of CO2 (see Edenhofer [85])). On the other hand, the anti-environmental incentives and benefits still outpace those designed to safeguard the environment. These are key results that allow us to understand the clearly negative environmental bias of the current tax system and, at the same time, to identify the type of problems and instruments on which to focus efforts to change tax policy in an environmentally friendly direction.

There are many reasons why Mexico and other developing countries have not given the same level of priority to the environmental agenda and to implementing the same advanced, systemic environmental fiscal policies as some developed countries (e.g., the urgent need to address the problems of poverty, welfare and job creation, or the awareness that the major global environmental problems are primarily the responsibility of the growth and consumption patterns of the more developed economies). However, it is increasingly evident that past and current fiscal architecture reinforces the unsustainability of the economic model, everywhere. Despite the progress of last three decades, the global balance sheet remains openly unsatisfactory, even for official international bodies [86]. A key conclusion of this work is that particular attention should be paid to the relevant environmental harmful benefits, a huge and hidden share of the current tax system. The problem is not exclusive to Mexico, but general in nature.

Significant progress in this direction requires taking firm steps towards systemic green tax reform that can align fiscal policy design with environmental and climate challenges. It will be necessary to focus more on taxation than on expenditure, because taxes (and tax expenditures) are among the most cost-effective environmental policy instruments [24,28,41,87]. The point is that taxes have the potential to change the relative prices of goods and services, but to do so changes must be significant in order to change the cost structure of the economy. Otherwise, the success in reshaping behaviours of economic agents towards sustainability will be very unlikely. Far from neutral, the tax shifts should have an uneven impact on different sectors, depending on the elasticity of supply and demand and the capacity of companies to transfer the tax impact to suppliers, customers or consumers.

The analysis of environmental taxes and tax expenditures in Mexico suggests the basis for an environmental tax reform in the short term: (a) significantly increase the tax base of the carbon tax, extending its application to all activities that are major emitters of CO2, (b) significantly raise the tax levied on tonnes of CO2 emitted, (c) eliminate in the short term all tax benefits for activities or products with clearly negative effects on the environment, and, (d) establish fair and generous tax benefits in VAT or corporate income tax for all labour-intensive circular economy activities such as repair, reuse, remanufacturing or remediation.

Of course, further research is needed to address the issue of the negative effects on income distribution and the design of the appropriate instruments to cushion it [88]. If the carbon tax were able to generate significant resources, these could be used to implement redistributive policies to benefit the poorest households or to balance significant VAT benefits for basic consumption. Going even further, the transition towards a sustainable circular economy requires a deeper study of the most profound changes needed in the architecture of the current tax system, which is directly or indirectly supported by the taxation of labour [89].

**Author Contributions:** Both authors contributed equally to the paper. All authors have read and agreed to the published version of the manuscript.

**Funding:** The authors belong to the ICEDE-Galician Competitive Research Group GRC ED431C2018/ 23 and to the "R2π: Transition from linear 2 circular: Policy and Innovation" H2020 Project. This research was supported by the Spanish Innovation Agency (AEI) through the project ECO-CIRCULAR: "La estrategia europea de transición a la economía circular: un análisis jurídico prospec-tivo y cambios en las cadenas globales de valor"—ECO2017-87142-C2-1-R. All these projects are co-funded by FEDER (UE). This research also received funding support from the National Council for Science and Research (CONACYT) México, PNPC 000625.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data Availability Statement:Publicly available datasets were analyzed in this study. This data canbe found here: CIATData Tax Expenditures "TEDLAC 2018 by Country". 2020. Available online: https://www.ciat.org/gastos-tributarios/ (accessed on 4 May 2020).

**Acknowledgments:** The authors are grateful to the comments and suggestions made by the three anonymous reviewers.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


## *Article* **A Future Study of an Environment Driving Force (EDR): The Impacts of Urmia Lake Water-Level Fluctuations on Human Settlements**

**Somayeh Mohammadi Hamidi 1,2,\*, Christine Fürst 1, Hossein Nazmfar 2, Ahad Rezayan <sup>3</sup> and Mohammad Hassan Yazdani <sup>2</sup>**


**Abstract:** Lake Urmia, one of the world's largest salt lakes, is rapidly losing water and drying up. This environmental hazard has raised concerns about the consequences and impact on the surrounding communities. In this paper, we use a futuristic view (horizon of 10 years based on medium-term planning) to identify the main environmental drivers in the surrounding settlements of the Urmia Lake basin. A qualitative method, based on cross-impact analysis, was used as a means of future research. We also used a Delphi-based expert panel method to collect data and extract the environmental impacts of Urmia Lake. After the three rounds of the Delphi process, the expert panel reached a high level of agreement (100%) on the top 17 environmental consequences. Then, these consequences were classified by driving force and dependency using the MICMAC method. The results show that reducing pasture area, soil and water salinity, groundwater decline and depletion, and destruction of surrounding agricultural lands play a significant role in environmental change in Urmia Lake. Overall, any small change in these variables may lead to fundamental changes in the entire system.

**Keywords:** future study; driving force; MICMAC; Delphi; Urmia Lake

### **1. Introduction**

Salt lakes are a significant part of the earth's inland aquatic ecosystems and are found worldwide. These lakes play a pivotal role in determining regional climate patterns, maintaining biotic productivity and diversity, sustaining ecological and human health, and providing recreational services, minerals, and other resources [1]. Therefore, the dynamics of salt lakes are of great importance to a wide range of stakeholders [2].

The Caspian Sea is the world's largest salt lake, accounting for 41% of all salt lake volume, and is home to a robust fishing, shipping, and mining sector. Other major hypersaline systems, such as the Great Salt Lake, offer various waterfowl habitats that can be exploited for minerals. Flamingos and other birds live in small Andean salars and lakes in the Middle East and Africa [3]. Around the world, we witness that salt lakes are shrinking. On closer inspection, environmental factors, namely almost all human activities, such as diversion of surface inflows, salinisation, mining activities, pollution, and climate change, threaten salt lakes [4]. Humans' increased water use, particularly for agricultural irrigation, is a major contributor to lakes drying up. Agricultural water consumption in the Aral Sea 2 watershed, for example, has reduced the lake's surface area by 74% and volume by 90% [5]. There is little doubt that, by 2025, the natural character of most of the world's salt lakes will have changed. Several recent 'vision' statements clearly point in this direction.

Nazmfar, H.; Rezayan, A.; Yazdani, M.H. A Future Study of an Environment Driving Force (EDR): The Impacts of Urmia Lake Water-Level Fluctuations on Human Settlements. *Sustainability* **2021**, *13*, 11495. https://doi.org/10.3390/ su132011495

**Citation:** Hamidi, S.M.; Fürst, C.;

Academic Editors: Jun Yang, Ayyoob Sharifi, Baojie He and Chi Feng

Received: 22 August 2021 Accepted: 9 October 2021 Published: 18 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

In central Asia, for example, the 'vision' proposed by UNESCO (2000) for the Aral Sea basin involves almost complete desiccation of the lake itself, and greatly increased 'development' of its catchment to support the growing populations of Kazahkstan, Uzbekistan, Turkmenistan, and other smaller states in the region [1]. Figure 1 shows the largest salt lakes in the world which are losing their water and drying up.

**Figure 1.** Examples of the observed decrease in surface water area in major salt lakes worldwide over the period 1984–2015 [6].

The drying up of salt lakes is not a new occurrence, and scientists have observed an alarming and accelerated loss in many of these essential ecosystems [7]. The Evans River diversion, for example, became the major source of pollution and particle matter in that section of the lake in 1926, creating asthma and other health concerns for nearly 40,000 people living there [8]. In a similar vein, the Utah Lake in the United States and the Aral Sea in Kazakhstan and Uzbekistan [9] have caused many disasters for the residents of these areas. The Tarim Basin, which resulted in the collapse of the Loulan Kingdom in 645 environmental consequences, is likely the oldest known direct human action that caused salt lakes to dry up [10]. Other consequences of increasing water consumption are more recent. Since 2000, the Salton Sea in California has dropped by over 7 m due to management measures that restricted water input to the lake [11].

Lake Urmia in Iran, like many other salt lakes on the continent except Antarctica, has experienced a similar fate. Nowadays, Lake Urmia exposes a salt desert that produces dust harmful to health and threatens crops and people [12]. According to similar occurrences involving Lake Aral, Lake Ebinur, and Lake Chad, the continued drying process of Urmia Lake would bring multiple social, environmental, and ecological disastrous consequences to the region [4] and pose a threat to the regional economy and human wellbeing.

The major goal of this paper is to (1) determine the environmental impacts of waterlevel fluctuations of Urmia Lake; (2) to identify the environmental drivers (10-year horizon) in the study area; (3) analyse the impacts of these drivers on human settlements around the lake that are directly and indirectly affected by these impacts.

### **2. Study Design and Methods**

### *2.1. Model Region: Lake Urmia Basin, Iran*

The largest hyper-saline water body in Iran, Lake Urmia, is located in northwestern Iran and encompasses around 52,000 square kilometres (Figure 2).

**Figure 2.** Location of the study area (Urmia Lake basin) and settlements.

The lake is located in a mountainous region between three provinces: East Azerbaijan (39 percent), West Azerbaijan (51 percent), and the Southern Province of Kurdistan (10 percent). The lake's mean annual precipitation ranges between 240 and 272 mm [13], with approximately 77 percent of the precipitation falling between December and May each year. Precisely speaking, it is the largest lake in the Middle East and the third largest salt lake globally [14], at 140 km long, 55 km wide, and 16 m deep [15].

After two decades (2000–2020) of intensive agricultural development, construction of reservoirs, and reduced river flows, the water level of Lake Urmia in northwestern Iran decreased by 6.5 m from a historical maximum of 1278 m in 2000 to 1271.5 m in July 2020, during which time the lake lost almost 45% of its area and 85% of its volume [16]. Figure 3 shows the trend of lake water depletion from 1975 to 2018.

The water area of Urmia Lake was estimated to be 5412 km2 in 1975. UL's surface area decreased by more than 5000 km<sup>2</sup> between 1995 and 2014, despite the lake's water surface area growing after that. According to the Urmia Lake Restoration Program report (ULRP), in 2016, the increase in rainfall in the whole basin, especially in some stations, was about 43% compared to the long-term average) [17]. The water surface area increased by 1490 km2 in 2016, whereas the salt body shrank by 348 km2. In August 2018, the lake's water surface area was estimated to be 1624 km2 (1852 km2) [18].

**Figure 3.** From 1975 to 2018, the salinisation and desertification of Urmia Lake progressed.

In the Urmia lake basin (Refare to Figure 2), more than 36 cities and 3150 villages have more than 5 million inhabitants (in this study, we have assumed these settlements to be an integrated system, whether or not environmental impacts have disrupted the system). Important demographic characteristics of the population within the ecological zone (a zone or area characterized by extensive but relatively homogeneous natural vegetation formations) [1], which is subject to an increasing old-age dependency ratio, a decreasing gender dependency ratio as a result of male labor migration, rising unemployment, and rapidly rising literacy [19]. Table 1 shows Urmia Lake's key ecological, hydrological, climate, social, and economic data characteristics.

One of the most important government programs to address the declining lake levels was the Urmia Lake Restoration Program. ULRP was launched in 2013 under the responsibility of the Ministry of Energy and in cooperation with the Ministry of Agriculture and the Iranian Ministry of Planning and Budget Organization to apply integrated solutions in the field of lake basin management and present several solutions at Sharif University of Technology. As Figure 4 shows, the vision of the ULRP is to reach ecological level (1274MCM).


**Figure 4.** Lake Urmia restoration road map.

According to Figure 4, Lake Urmia restoration program has been planned in three steps, as follows: (1) the stabilisation period (2014–2016)—the main goal in this period is to stabilise Lake Urmia's level and implement projects to reduce the potential impacts of the persistence of Lake Urmia; (2) the restoration period (2016–2022)—the main objective of this period is to implement all solutions for the lake's water supply and gradually increase its level (1273MCM); and (3) final restoration (2023)—the expected purpose of this period is to stabilise the lake's restoration conditions and create the necessary conditions for the final restoration and sustainable maintenance of lake conditions (1274.1 MCM).

### *2.2. Methodology*

This study is based on cross-impact analysis, which falls under the category of crossimpact analysis approaches. The most popular methods are Gordon's cross-impact method, SMIC, BASICS, MICMAC, KSIM, and the cross-impact balance approach [20]. Accordingly, this study used the Delphi Survey for data collection and MICMAC for analysis due to the qualitative database.

### 2.2.1. Delphi Survey: Foresight Desirability

Foresight departs from probabilistic prediction by fostering a notion of the future as variously imagined, shaped, and influenced through strategic planning and intervention [21]. Opening up foresight processes implies the advantage of integrating valuable sources of knowledge more effectively and systematically to make future decisions based on comprehensible facts [22]. Foresight methods allow individuals and organisations to imagine different future scenarios and plan for greater future resilience [23]. Consequently, foresight can identify benefits and constraints in terms of innovative solutions [24]. There are many methods of foresight or predicting the future. One of the leading tools in foresight is the Delphi method. Delphi is defined as a means of organizing a group communication process to allow a group of people to address a difficult problem as a whole. Through the four features of Delphi: (1) anonymity; (2) feedback; (3) statistical aggregation of group responses; and (4) iteration, experts can achieve more objective results, such as more objective probability estimates [23]. Delphi and other expert panel methods for planning the future are also widely used in environmental studies [25]. We performed a three-round Delphi study between November 2019 and June 2020, comprising three overarching phases, as follows:

**Round 1:** In the first Delphi round, key experts were identified. There are no clear guidelines on the number of participants included in studies that use Delphi questioning as the sample is purposively selected, depending on the problem being studied [26]. Some studies have used 15 participants, while others have used 60 [27]. In this paper, the target panel of experts included 35 people (Table 2). They were selected based on published papers (also five people who have worked on the Aral Sea); recommendations by other scientists; and consultations with directors of scientific institutions, such as the Lake Urmia restoration program (ULRP), the Urmia Lake research Institute, and Urmia and Tabriz Universities. Except for five people (X11, 12, 13, 15, and 17), all experts had local knowledge (village level) and regional knowledge (research across districts).


**Table 2.** Expert panel characteristics.


**Table 2.** *Cont.*

XP: An expert panel (official meeting) was held in the office of the Lake Urmia Restoration Center in East Azerbaijan Province. Led by a futurist researcher (Fellow at the National Research Institute for Science Policy. Futures Studies Postdoctoral Researcher, University of Tehran, Iran). I: Interview. Q: Questionnaire.

> Due to the dispersion of experts, at first, a meeting was held in cooperation with the Lake Urmia Restoration Centre with committee members led by a futurist. In this meeting, members discussed the environmental consequences of Lake Urmia in the future. Then, a questionnaire was sent to other experts. The questionnaire included an openended question that was in the spirit of brainstorming. In this question, the experts were asked, "What will be the environmental impact of drying up Lake Urmia? (with a 10-year horizon)". A total of 35 (100% response rate) completed the first round. Subsequently, all responses were coded using the qualitative technique of grounded theory (open and axial coding). Coding of the open-ended questions of the questionnaire resulted in the identification of 80 general concepts, and axial coding took the form of 62 categories and 35 main outcome indicators.

> **Round 2:** The second round was conducted with experts at Urmia Lake Restoration Centre. Nine experts (25.71% response rate) participated in the second round. In this round, the results of the first round were discussed. Finally, 17 environmental consequences were selected, which were agreed upon by the panel members.

**Round 3:** The third round was also via email. We emailed 17 environmental consequences (which had been agreed upon in Round 2) to all 35 participants, and 27 experts participated in Round 3 (77.14% response rate). After applying some changes by the respondents, the results of the expert responses were summarized in Table 3.


**Table 3.** Results of Round 3 of the Delphi survey (final environmental consequences).

### 2.2.2. Cross-Impact Analysis: MICMAC

From a collection of variables first determined by a committee of experts, the MICMAC technique of structural analysis tries to find the most essential variables inside a system and their role [28]. The three steps of MICMAC are as follows:


Influences range from 0 to 3, with the possibility to identify potential influences: 0: No influence;



**Table 4.** Direct influences matrix sample.

**Figure 5.** MICMAC influence–dependence map.

The MICMAC analysis is used to cluster consequences based on their type. Clustering is performed based on the driving and dependency power of each barrier [30]. Figure 5 shows the output of MICMAC and how to distribute the variables.

The results are discussed in the following section.

### **3. Results**

Figure 6 shows a MICMAC diagram that was created by changing the reachability matrix. A MICMAC diagram is useful for determining driving and dependency power. If the value of dependence power is higher, other barriers should be addressed before the highest dependence power barrier is removed. With more driving power, other hurdles can be readily overcome after this one is removed.

**Figure 6.** Direct influence–dependence map.

The grid is divided into four quadrants, each representing one of four different categories of variables. The differences between the variables are in the values for influence and dependence. Decreased livability in surrounding cities (X17) will have the largest dependence, and groundwater decline and depletion (X8) will have the most significant influence on the system (see page 4, row 103). Moreover, the other variables are as follows:

**Autonomous barriers (I):** Quadrant I represents autonomous environmental consequences and consists of increased temperature (X1), air pollution (X9), plant extinction species (X10), threats to the food chain (X14), flooding (X3), and incidence of different types of diseases (respiratory, skin diseases, and various cancers) (X13). Dependence and conductivity are low, and a change in these variables does not lead to a serious change in the system.

**Dependent barriers (II):** Quadrant II shows the dependent barriers, which are disturbance in the ecosystem structure of the lake (X15), extinction of wildlife (X2), decreased viability in surrounding cities (X17), and changes in cultivation patterns (X4). They have strong dependence and poor conductivity. These variables basically have high influence, but less impact on the system.

**Linkage barriers (III):** Quadrant III shows the linkage barriers, including dust storm (X6), drought (X11), and desertification (X16), which are all possible.

**Independent barriers (IV):** Quadrant IV consists of driving factors with a strong driving force but a weak dependent force. In these barriers, four environmental consequences reducing pasture area (X5), soil and water salinity (X7), and groundwater decline and depletion (X8), and destruction of surrounding agricultural lands (X12)—were found to play a controlling role in environmental changes in the case study area. Overall, any small change in these variables leads to fundamental changes in the entire system as Figure 7 shows. We will continue research into these variables.

**Figure 7.** Direct influence graph.

Groundwater decline (X8) and destruction of surrounding agricultural lands (X12) are the most important and effective consequences in this system. Desertification (X16) and groundwater decline (X8) also have the strongest influence on the quality of life in cities (X17).

### **4. Discussion**

Planning for the future predicted by our current data, accepting uncertainty, and preparing for the improbable gives us a chance to select a better plan. According to the results of this study, four driving forces were identified, which means that by controlling and monitoring these driving forces, other consequences of the drying up of Lake Urmia can be counteracted in the future. These driving forces are the driving factors that will disrupt the ecosystem of Lake Urmia.

**The environmental driving force of Urmia lake dessication:**

Seventeen significant factors were identified in this paper. Among the 17 factors, four factors were identified as driving forces, as seen in Figure 6.

**Groundwater decline and depletion:** The construction of numerous dams on rivers flowing into Lake Urmia has led to a shortage of water for agriculture, and farmers have extracted groundwater by digging deep and semi-deep wells (Figure 8) to provide water for agriculture.

**Figure 8.** The amount of groundwater extracted in the Urmia Lake basin has changed dramatically over the last four decades.

According to ULRP statistics (2019), the number of legal wells in the Lake Basin is more than 40,000. Figure 9 shows the distribution of wells in Lake Urmia's basin. The number of semi-deep wells in the basin more than quadrupled in 2012 compared to 1982, and their respective discharges increased about sixfold. It should be mentioned that the majority of these wells are illegally drilled and used by garden and farm owners [17]. Furthermore, because there are many illegal wells, the government has limited control over groundwater abstraction, indicating a major data gap. Field-level water use is not included in current data collected near ULRP, and the existing data are either obsolete or held in inaccessible archives.

As a result, we are skeptical of the ULRP's target of reducing agricultural water use by 40%, as we believe it will be challenging to accomplish and will have severe consequences for farmers and people in the basin. The idea could even exacerbate societal tensions; for

example, a proposal to limit or restrict local farmers' illicit use of groundwater would be met with criticism.

**Reducing Pasture area:** The second and strongest driving force is the loss of a large area of pasture and vegetation. Furthermore, because of the diversity of soils and topography, the Urmia Lake islands possess a diverse flora. The first report on the flora of Lake Urmia was carried out in 1990–1994 (1998) (when the lake level was at the highest water level), and the second report was in 2016–2017 [31], but when the lake level was at its lowest, this rate was reached). During this period, various events caused many changes in flora, plant communities and vegetation levels. Comparing the results of these two reports shows that the number of plant species in these salt marshes decreased from 201 species to 93 species during these two decades. Out of 28 plant genera, only 24 members remain [30]. The majority of the plants in this genus are salinity-sensitive and have been wiped out due to habitat shifts away from the lake's edge. They play an essential role in the ecosystem's food chain. Therefore, this consequence does not directly affect human settlements, but is the root of other natural consequences in the field.

**Soil and water salinity:** One of the most major concerns promoting desertification in Iran's northwest is soil salinisation caused by the drying of Urmia Lake and dust produced from the lake's dried bottom (Figure 10). It results in a decrease in biomass output and a decrease in soil productivity.

**Figure 10.** Monthly variations in the average and maximum suspended particle concentrations less than 10 microns at the Islamic Island (the largest island in Urmia Lake) station.

As a result, soil salinity looks to be the most serious hazard to this region's agricultural areas. As a result, the majority of the lake has been transformed into useless land in recent years. As the drought continues and the lake's salt concentration increases, the extinction of the lake's wildlife has increased (http://www.iew.ir/ accessed on 22 August 2021). Nikjoo et al.'s (2017) findings show that the environmental situation was better before 2009 when approaching the shore of Lake Urmia; however, after 2009, it has improved by moving away from the shore, and the situation at the lakeshore has become much worse [32].

**Destruction of surrounding agricultural lands:** Agriculture is a major source of employment in the region. However, it has been particularly hard hit by the drying up of the lake, a major water deficit, and climate change. Salt deposits, dry wells, extreme spring rains, and long droughts have negatively impacted agricultural production.

Several dams and dikes have been built in the Lake Urmia Basin, primarily to divert and storage for agricultural purposes. Although the first and second downstream canals are modern, old irrigation systems are still used in the region and water is distributed through open canals constructed by farmers. The number of farms around Lake Urmia is shown in Figure 11.

**Figure 11.** The number of farms with horticultural activity in three provinces around Lake Urmia.

From 2007 to 2018, 38,635 farmers lost their jobs (majority in the agriculture sector) in the province of East Azerbaijan, 2277 in West Azerbaijan, and 14,722 in Kurdistan. Furthermore, according to Soleimani Zeyveh's (2010) results, the drying up of Urmia Lake is damaging thousands of hectares of agricultural land in the lake area, resulting in a rise in unemployment and migration of roughly 3 million people to the neighbouring areas [33].

According to Mohammadi Hamidi et al. (2019), declining production and changing the type of some agricultural products are the main negative consequences that will lead to the migration of people from this region. Additionally, Ebrahimzadeh et al. (2014) argued that the least important consequence of the drying up of Lake Urmia would be the unemployment of more than three million people in the surrounding provinces. The research findings are summarised as follows [34]. Figure 12. shows an summarizes of research findings and the cause-and-effect relationships between outcomes and ultimately their impact on human habitats, which were the focus point of this paper.

Increasing unemployment due to loss of agricultural land and the related social and economic problems [35], the creation of salt winds as a threat to agricultural lands on the shores of the lake, and the inhalation of salt dust [36], which could cause cancer and lung problems [37], are some of the greatest environmental disasters, resulting in the emigration of tens of thousands of people from the region. For example, in the past 50 years, East Azerbaijan province (located on the eastern side of Lake Urmia and directly influenced by the water fluctuations of Lake Urmia due to western winds) has experienced a surge in emigration and is ranked top among Iran's other immigrant sending provinces, as seen in Figure 13.

According to the census results, during the 2011 to 2016 census, a total of 171,892 people left the province. Currently, East Azerbaijan province has 3083 villages, 138 of which have been depleted and are void of inhabitants compared to the 1996 census. In 2006, the number of depleted villages reached 378 (Figure 14). Finally, according to the last census in 2016, 494 villages in this province were depopulated and multiplied since 1996.

Considering the prevailing wind direction in the region, coming from the west, the probability of damage to the settlements of eastern and northeastern Urmia due to salt storms is very high. Figure 15 shows that most of the settlements in this area are depopulated.

**Figure 12.** The decreased water level of Lake Urmia and its consequences in the surrounding settlements.

**Figure 13.** Migrants entered and exited during consecutive census (1986–2006, the Iran–Iraq war caused people to migrate from this border province) (Iran Statistical Yearbook).

**Figure 14.** The populated and depopulated villages in the East Azerbaijan province according to the consecutive census.

**Figure 15.** Distribution of settlements in the Lake Urmia basin.

According to Figure 15, most of the settlements are on the western and southern shores of the lake. Most of the settlements on the lake's eastern shore (East Azerbaijan province) were abandoned by the inhabitants in recent decades.

In response to those threats, the government's action was the formation of the Lake Urmia restoration centre, which, despite the passage of more than 80% of the program, has not yet achieved its goals, and this lake is still considered a dangerous threat to this region. Thus, the wave of migration that began in the region in recent decades will continue gradually, causing problems for cities and political conflicts in the region.

### **5. Conclusions**

In this paper, the consequences of the drying of Lake Urmia on human settlements were investigated. The results of this study are as follows.

(1) There were 17 significant environmental consequences of lake water fluctuation identified. (2) Among the 17 consequences, decreased livability in surrounding cities was identified as the most dependent, and groundwater decline and depletion were identified as the most significant and effective environmental consequences in the system. (3) Finally, four environmental consequences— reducing pasture area, soil and water salinity, and groundwater decline and depletion, and destruction of surrounding agricultural lands were identified.

Furthermore, this effort emphasized the importance of finding long-term solutions to mitigate environmental disasters and provide opportunities for local residents as soon as possible. However, many of the present efforts to restore the lake's natural equilibrium have proven to be ineffective.

The northwestern regions of Iran (around Lake Urmia) have strategic conditions due to the border. In addition, due to the impact of water scarcity in Lake Urmia Basin, the migration process from these areas has accelerated in recent years. Given the region's geopolitical status, this population loss will have long-term security ramifications. As a result, it appears that planners and policymakers must address the climatic, sociological, demographic, and political consequences of this change. Although programs and actions have been undertaken in recent years to revitalise the lake and take steps toward environmental sustainability, these trends are so slow that they are far off from achieving the desired sustainability conditions.

According to the Department of Energy, the basin is home to five million people, with average daily water use of 220 litres per person per day. With appropriate water management, the average water use in many urban areas (particularly in Europe) is around 150 litres per person per day. The urban population must play a role in the solution. Therefore, a public education program to reduce urban water use is needed. Some initiatives, such as reducing water pressure within the system, introducing a rebate program for low flush toilets, and installing water-efficient appliances in all new homes, could help to maintain the lake's water level. Agricultural growth has had a significant influence on the marshes surrounding the lake (cultivation and grazing). Wetlands are excellent water storage and filtration systems that can help the lake recover. They need to be better preserved and maintained. While technological capability is strong, there appears to be a lack of coordination among government agencies. ULRP staff work only in the headwaters but should be able to work in the agricultural area. To launch a collaborative plan, all concerned agencies must work together. The ULRP appears to have been tasked with taking the lead, promoting cooperation and integration, and putting together an action plan.

Conducting a groundwater resource analysis includes mapping aquifers and well inventories, groundwater use, extraction vs recharge rates, and groundwater and surface water interactions, which are all crucial for drought management.

There appears to be much disagreement among specialists about how much groundwater contributes to the lake, and there is a lot of information on how it is used in agriculture. Identification and mapping of key aquifers, an inventory of groundwater wells, estimates

of groundwater use, determining if extraction rates are in balance with recharge rates, and research to determine where groundwater–surface water interactions are occurring within the watershed should all be undertaken. Considering all of these, it should be emphasized that these solutions represent crucial information as some of the few solutions that are accessible and executable during droughts.

**Author Contributions:** S.M.H., C.F., H.N., A.R. and M.H.Y., participated in the writing and editing of the manuscript and the preparation of the figures. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** All data generated or analysed during this study are included in this published article.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


### *Article* **The Effects of the Spatial Extent on Modelling Giant Panda Distributions Using Ecological Niche Models**

**Ziye Huang 1, Anmin Huang 1,\*, Terence P. Dawson <sup>2</sup> and Li Cong <sup>3</sup>**


**Abstract:** Climate change and biodiversity loss have become increasingly prominent in recent years. To evaluate these two issues, prediction models have been developed on the basis of ecological-niche (or climate-envelope) models. However, the spatial scale and extent of the underlying environmental data are known to affect results. To verify whether the difference in the modelled spatial extent will affect model results, this study uses the MaxEnt model to predict the suitability range of giant pandas in the Min Mountain System (MMS) area through modelling performed (1) at a nationwide scale and (2) at a restricted MMS extent. The results show that, firstly, both models performed well in terms of accuracy. Secondly, extending the modelling extent does help improve the modelling results when the distribution data is incomplete. Thirdly, when environmental information is insufficient, the qualitative analysis should be combined with quantitative analysis to ensure the accuracy and practicality of the research. Finally, when predicting a suitability distribution of giant pandas, the modelling results under different spatial extents can provide management agencies at the various administrative levels with more targeted giant panda protective measures.

**Keywords:** MaxEnt model; spatial extent; ecological niche; *Ailuropoda melanoleuca*; Min Mountain System

### **1. Introduction**

Climate change and human activities have caused massive loss and fragmentation of animal and plant habitats and a sharp decline in the numbers of many rare species [1]. The wild giant panda (*Ailuropoda melanoleuca)* is one of the most globally endangered species, and its survival is uncertain. In 1869, the French priest Pere Armand David first discovered the giant panda in Sichuan province, China, and it attracted scholars' attention worldwide [2]. According to fossil evidence, documentary records, and quantitative surveys, the distribution of wild giant pandas has gradually shrunk; they have disappeared from most parts of China and from Vietnam, Laos, Myanmar, and other regions, and now remain only in parts of southwestern China and northwestern China [3,4]. Today, they are an endangered species unique to China and only distributed in six major Chinese mountain systems: the Qinling Mountain System, the Min Mountain System (MMS), the Qionglai Mountain System, the Daxiangling Mountain System, the Xiaoxiangling Mountain System, and the Liang Mountain System. These mountains are located in southwestern China, crossing Shaanxi, Gansu, and Sichuan provinces [5].

The Fourth National Giant Panda Survey Report (2015) reports the status of wild giant pandas in recent years [6]. The number of wild giant pandas increased by 268 individuals compared with the previous survey, reaching 1864 [7]. Habitats and potential habitats rose by 11.8% and 6.3%, respectively [6]. Even though the current survival situation of giant pandas has not continued to deteriorate based on data, the fragmentation of the habitat is

**Citation:** Huang, Z.; Huang, A.; Dawson, T.P.; Cong, L. The Effects of the Spatial Extent on Modelling Giant Panda Distributions Using Ecological Niche Models. *Sustainability* **2021**, *13*, 11707. https://doi.org/10.3390/ su132111707

Academic Editors: Baojie He, Ayyoob Sharifi, Chi Feng and Jun Yang

Received: 16 September 2021 Accepted: 20 October 2021 Published: 22 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

a hidden danger that needs continuous attention. The existing giant pandas are divided into 33 local populations, and there has been a decrease in connectivity between these populations and some of the habitats are of poor quality. Therefore, although the number of wild giant pandas has increased, the ecological isolation between local populations and poor quality of some habitats still poses a significant threat to the giant pandas' future survival [3,4]. According to Linderman et al. (2004), human influence in the next 30 years will lead to a 16% loss of giant panda habitat [8,9].

Therefore, it is essential to figure out what has caused this species to become endangered and to understand the factors that interfere with their natural environment to ensure the sustainable development of this endangered species; moreover, such issues need to be resolved urgently [10]. To ensure the suitability of the wild giant pandas' habitat and promote the formulation of protection measures, many scholars are devoted to studying the factors that give rise to the fragmentation of habitats and cause other threats to the survival of the population. The conclusion of these research results identify four categories of factors that primarily affect the giant pandas' survival and habitat fragmentation: (1) human activities, logging, reclamation, construction of cities and roads, grazing, and poaching; (2) ecological limitations of giant pandas, such as the special requirement for food and low reproduction rate [11]; (3) natural disturbances, such as earthquakes, and bamboo blossoming [12,13]; (4) climate change, the changes of plants' growth structure, temperature, rainfall, and other factors affected by climate warming.

However, due to the imbalance of scientific research structure, most scientific research work only focus on the aspect of phenomenon analysis, and relatively little work has been carried out from the perspective of sustainable development [10]. Presently, to increase the population and prevent the reduction of giant pandas' habitat, there is an urgent need for quantifying how different populations of giant pandas are predicted to respond to different climate change scenarios. At the same time, corresponding response measures should be made based on the model results to ensure the sustainable survival of giant pandas. Ecological niche models (ENMs) are currently used in geo-biological research, and are also the main modelling tool for predicting and solving sustainable ecological development [11]. Due to the practicality and accuracy of such models in solving ecological problems, ENMs have been widely used in environmental research, including issues such as climate change, the impact of species invasion on another species, and for predicting the potential distribution areas of different species [14,15]. The popular ENMs in recent years include: Genetic Algorithm for Rule-Set Prediction (GARP), Ecological-Niche Factor Analysis (ENFA), Bioclimate Analysis and Prediction System (BIOCLIM), and MaxEnt [16]. Of these, the MaxEnt model is the most widely used in studying the geographic-biological connection.

Few studies have examined the influence of spatial scale on the predictions generated by MaxEnt modelling. Of these, some studies have shown that different scales have minimal influence on model predictions [17], whereas other studies have concluded that variation in scale-dependent effects is influenced by environmental variable selection [5,18]. However, there remain few concrete recommendations on how to deal with these issues in order to improve MaxEnt modelling.

Based on the concern about the development of the giant pandas' living conditions and interest in further research on modelling extents, this study aims to examine how the spatial extent of the modelling affects the niche model result. Taking spatial extent as a breakthrough point to technically reduce the errors in the study of endangered species models and will make it possible to provide suggestions for future monitoring and protection methods for giant pandas and thereby ensure the survival of giant pandas and alleviate the conditions that threaten their survival. To accomplish this aim, the research has the following objectives: (1) to quantify some significant human disturbances and incorporate them into the environmental variables in the MaxEnt model; (2) to predict the accuracy of the model to ensure the reliability of follow-up research; (3) to model suitable habitats for giant pandas in different spatial ranges and identify some differences between them; (4) to

follow the results with further discussion and suggest how this research provides practical help for the future.

### **2. Materials and Methods**

*2.1. Introduction to Model*

The MaxEnt model (Philips et al., 2006) takes the Maximum Entropy Theory proposed by Jaynes in 1957 as theoretical guidance [19]. It is a widely used method with high performance and is commonly used in climate prediction, biological protection, species invasion, and other biological research [5,20,21]. The principle of MaxEnt is that the 'dissipation' of the system increases the entropy. Until the species and the environment reach the maximum entropy, the environment will be in equilibrium [22]. The model predicts species' distribution by calculating the state parameters when the entropy is maximum [22]. The MaxEnt models exhibit little sensitivity or change to model accuracy with significant changes in data quantity, although processing times tend to be accelerated and model accuracy is higher when models are based on presence-only data [23]. Thus, follow-up research is based on the MaxEnt model with high accuracy, easy operation, and low data quantity requirements.

### *2.2. Case Study Area and Research Extent*

Wild Chinese giant pandas are distributed in six major mountain systems of the Sichuan, Shaanxi, and Gansu provinces. The area and habitat ratio of each mountain system are shown in Table 1 [13]. The MMS is located in southwestern China, running through northern Sichuan province and southern Gansu province. The geographical position is 102◦ 70 –105.60◦ E and 31◦40 –33.70 N, transitioning from the Yangtze River's upper reaches to the Tibetan Plateau [24]. The giant pandas in the MMS are mainly distributed in the mountainous dark coniferous forest belt at 2100 to 3400 m. The subalpine dark coniferous forest belt is at a height of 3000 to 3900 m in the alpine valley [13]. There are 27 established nature reserves in the MMS, where 44% of China's pandas are distributed [21].


**Table 1.** Statistics of the habitat area of giant pandas in each mountain system.

Source from: Zhang, Q., 2009. Study on Habitat Selection of Giant Panda in the Min Mountain, Gansu. Master. Northwest Normal University.

Since the MMS region is the most critical habitat for giant pandas among the six mountain systems, this area was selected as the target area for modelling. A national extent is selected as a control group, which contains all the known global distributional data for the giant panda. Comparing the model result under the MMS region and the nationwide extent (subsequently clipped to the MMS range) enabled us to evaluate variation in predicted suitable habitat areas. The specific modelling area comparison shows in Figure 1.

### *2.3. Data Collection*

Species distribution data were derived from two sources; firstly the Global Biodiversity Information Faculty [25] and National Specimen Information Infrastructure [26]; and secondly, from all relevant peer-review literature records of giant pandas' occurrence. A total of 260 occurrence sites were obtained from these two sources. Google Earth was used to filter the distribution information to gain the relevant latitude and longitude coordinates,

as well as helping to remove 'double records' and locations with unknown geographic coordinates. This resulted in 89 occurrence records, of which 65 were from the MMS.

**Figure 1.** Modelling areas of spatial extent and distribution sites of the wild giant panda.

All climate and environmental data were obtained from WORLDCLIM [27]–a global database based on meteorological information recorded by weather stations worldwide from 1970 to 2000 (Table 2). It uses interpolation to generate global climate raster data with a spatial resolution of 30 arc-seconds (1 km2) [17]. The roads, rivers, and residential areas data were derived from OpenStreetMap then calculated by the Euclidean distance tool in ArcGIS to acquire the distance to the residential area, the distance to the road, and the distance to the river, with resampling of 30 arc-seconds (1 km2). The data of land use types in 2018 came from the Resource and Environment Data Center of the Chinese Academy of Sciences [28], with a resolution of 30 arc-seconds (1 km2).

However, the redundancy and overfitting of variables can bias the data results [29]. To avoid overfitting, only when the environmental variables are reduced to a reasonable number can the accuracy and predictive ability of the model be improved [30,31]. Therefore, the environmental variable data values of 89 sample points were extracted by DIVA-GIS software. The Pearson correlation coefficient was used to test the multicollinearity between climate variables in a set of climate variables with high correlation (r > 0.8). According to the contribution rate, only one variable closely related to the species distribution or convenient for model interpretation was selected for model prediction. In this case, 11 environmental variables were ultimately screened at two extents, respectively, for model construction.

### *2.4. Model Construction*

Firstly, giant pandas' distribution and environmental data were imported into the MaxEnt version 3.4.1 to model potential distributions of giant pandas at both spatial extents [32]. The maximum number of iterations (500 times) and the maximum number of background points (10,000) were kept recommended default settings. The cross-validation method was used to test the robustness of each model. The study used 25% distribution points as the test set and 75% distribution points as the training set and ran these 10 times repeatedly to make the data utilization rate higher [33]. Jackknife and percentage rate were selected to evaluate the relative importance of each environmental factor on the giant pandas' distribution. The response curve was chosen to obtain specific values of environmental variables most suitable for the survival of giant pandas.



The area under the curve (AUC) of receiver operating characteristics (ROC) was used to predict the model performance. When the ROC curve cannot indicate which classifier performs better, AUC provides a more intuitive way to predict the accuracy of models, which is a single measure of overall accuracy that does not depend on a particular threshold [34]. The value of AUC is between 0 and 1. The larger the AUC value, the better the prediction effect, and the higher the prediction accuracy. The general evaluation standard is less than 0.5 is model meaningless, 0.5–0.6 is poor, 0.6–0.7 is fair, 0.7–0.8 is more accurate, 0.8–0.9 is very accurate, and 0.9–1.0 is extremely accurate [35]. When the AUC value is more than 0.7, the prediction result of the MaxEnt model can be credible.

### **3. Results**

### *3.1. Model Prediction Accuracy Test*

The ROC curve was obtained after running the MaxEnt model 20 times repeatedly to ensure the model's prediction results' stability. AUC values of the MMS region (Figure 2) and the national extent (Figure 3) were 0.879 and 0.987, respectively, suggesting that the MaxEnt models for both scales performed well, and the national extent scaled model is more accurate than the MMS regional model.

### *3.2. Distribution of Predicted Suitable Areas for the Giant Panda*

The suitable habitat distribution map under the MMS (Figure 4) and the country extent (Figure 5) was directly obtained through modelling. To make the information in the map of the national range clearer, it was clipped to the range of the MMS (Figure 6). There is a big difference in the suitability distribution map under the two areas modelling. Geographically, modelling at the region of the MMS, the highly suitable areas are only scattered in the core area of the MMS, such as the Wanglang Reserve, Hulu ditch, Wenxian ditch, and Xi ditch. However, when modelling on a national size, most of the MMS areas are suitable for wild giant pandas to survive, and at least half of the regions are highly suitable. In the whole country, only the MMS area has a highly suitable area. Except for the Gansu and Sichuan provinces covered by the MMS, only a few areas at the junction of Tibet and Yunnan provinces have some suitable regions.

**Figure 3.** Receiver operating characteristic curve of country extent and area under the curve (AUC).

**Figure 4.** The distribution of suitable habitat of wild giant panda in the Min Mountain System obtained by modelling on the Min Mountain extent.

**Figure 5.** Prediction of the distribution probability of wild giant pandas in the Min Mountain System obtained by modelling on the country extent (not clipped).

**Figure 6.** Prediction of the distribution probability of wild giant pandas in the Min Mountain System obtained by modelling on the country extent (clipped).

### *3.3. Contribution and Importance of Environmental Predictors*

The result of jackknife under the MMS area (Figure 7) and the national range (Figure 8), as well as the percentage contribution rate under two areas (Table 3), are used to test the contribution rate of environmental variables. According to the results of the two methods, under the MMS area, bio15 (coefficient of variation of precipitation seasonality), bio2 (mean diurnal air temperature range), bio14 (precipitation of the driest period) and distance to a stream are the dominant factors affecting the spatial distribution of giant pandas. Under country area, bio12 (annual precipitation), bio3 (isothermality Bio2/Bio7 × 100), bio4 (standard deviation of temperature seasonality) and slope are the dominant factors affecting the geographic distribution of giant pandas.



Slo = slope; Alt = altitude; DTC = distance to community; DTS= distance to a stream; DTR= distance to road.

**Figure 7.** Results of jackknife evaluation of the environmental variables concerning regularized training gain for the Min Mountain System.

**Figure 8.** Results of jackknife evaluation of the environmental variables concerning regularized training gain for the country extent.

### *3.4. Response Curve*

Four factors were selected from the percentage rate and jackknife on the MMS region (Figure 9) and the national region (Figure 10) to conduct the response curve further. The highest point in the figure represents the variable value that is most suitable for the survival of giant pandas. On the one hand, under the MMS area, the results show that the most suitable Mean diurnal air temperature range is about 9.5 ◦C, and 6–8 mm is the best range of precipitation of the driest month. In addition, 0.15–0.16 (unit: 10 km) is the most suitable

distance from the giant pandas' habitat to a stream, and 65–72 mm is the best range of the value of the precipitation seasonality.

**Figure 9.** The response curve of the Min Mountain extent with the variables bio15, bio2, bio14, and DTS.

**Figure 10.** The response curve of the country extent with the variable bio12, bio3, bio4, and slope. 262

On the other hand, under the country extent, an ideal annual precipitation value for giant pandas' survival is about 800 mm, while isothermality (Bio2/Bio7 × 100) is about 32. The slope's value tends to be ideal at 17 ◦ and above, and the optimal value of the standard deviation of temperature seasonality is about 700.

### **4. Discussion**

### *4.1. Model Results Will Be Affected by Errors in the Data Collection*

From the process of the data collection and modelling, long data collection time and a lack of environmental and species location data will impact the model's accuracy. Many sites in the MMS areas are underdeveloped, the actual measurement points of the data are sparse, and the observation years of various types of data are not the same, which significantly increases the possibility of data errors [5]. For example, the climate data is from 1970 to 2000, the land type is based on the classification in 2018, and the species distribution data spans several decades. From the process of data collection and acquisition, time inequality is inevitable. Thus, timely updates and data supplements will be necessary to minimize the errors as much as possible and provide better data support for niche modelling.

However, this error has little effect on this study. In terms of the predicted suitable area, the research focus was not to obtain a specific geographical distribution area of giant pandas but to compare the differences in the distribution of the two regions. This error is simultaneously functioning on the two ranges. Therefore, the impact of the time inequality of the data on the research is reduced to some extent.

### *4.2. The Predicted Suitable Areas Are Different at Different Modelling Extents*

The suitable habitat maps are quite different at the two spatial extents. However, the highly suitable area in a more extensive range of modelling results almost overlaps with the suitable area (highly suitable area plus moderately suitable area) in the target area modelling results. Thus, the suitable range under larger-extent modelling is more extensive than that under smaller-extent modelling for some reason.

Combined with the accuracy-test results, the model result on a national scale in this study shows a higher prediction accuracy. However, there is an assumption in extreme cases: assuming that the occurrence site data and environmental data are complete and accurate, there is no doubt that the most accurate results will be obtained when the target area is the modelling area. In other words, if the site data and environmental data are very accurate, the model established under the MMS area should be the most accurate.

When the actual results of the model do not match the assumptions, reviewing some phenomena that have not been taken into account in the modelling process can help explain these unexpected results. When the research object is a kind of animal, for example, the giant panda, their mobility and rarity often lead to a lack of data and uncertainty in obtaining distribution sites [5]. Furthermore, giant pandas are sensitive to humans, and many distribution records do not entirely accurately represent the areas where pandas have lived for a long time [5]. Therefore, the lack of distribution sites under the MMS modelling area will underestimate the range of potentially suitable distribution areas [17]. At the same time, modelling at a range more extensive than the target area will overestimate the suitable spatial range. It is easier to obtain more distribution sites and environmental data under a larger spatial scale, which will provide systematic and comprehensive information for establishing a species–environment relationship [17]. Thus, if there is a lack of species distribution sites, appropriately expanding the modelling scope will help offset the possible deviation in modelling at the target area scale.

However, when the research object is a plant, its distribution site will not move, so it is relatively easy to collect. The suitable distribution map from different spatial scales should be identical [17]. Therefore, the degree of movement of the species and the difficulty of data collection is different, and the range that may need to be expanded during modelling is also different.

Based on the above analysis, the MaxEnt model can provide adequate technical support for endangered or difficult-to-track species. For one, the MaxEnt model has low requirements for the number of data. For the other, this study concluded that appropriately expanding the modelling range and clipping out the appropriate scope of the target area may offset the error caused by the lack of distribution sites or environmental variables. The MaxEnt model makes it possible to predict the living conditions of endangered species, and the study of the spatial extent of modelling reduces the possible errors of the model. This provides technical support and guarantees that relevant personnel and agencies can monitor the living conditions of endangered species and intervene if necessary. However, how to expand the scope of the modelling to compensate for the errors caused by the missing sites and variables and to achieve the most accurate results further studies are needed.

### *4.3. Human Interference Factors Need to Be Considered*

None of the human interference factors quantified in this study is significant under contribution rate analysis. For one, it may be because human interference shows a gradual weakening trend as the extent increases. Connor et al. (2019) mentioned that human interference, such as the negative impact of roads, is evident at more minor scales but becomes smoother as the total range increases to more than 500 km2 [18]. The target area in this study is much larger than 500 km2, so the influence of human interference is weakened. For the other, the MMS area has more than 30 protected reserves, and there are fewer permanent residents than in other regions [13]. In addition, because fewer roads are connected to the outside in each nature reserve, human disturbance caused by residents' activities and road traffic is not substantial [36].

However, on a more microscopic scale, there are still many human variables that should be considered. For example, the MMS area is the settlement of the Baima Tibetan ethnic minority in China. Grazing horses and yaks have been the traditional production method of this ethnic group for a long time. In recent years, the grazing range has expanded, causing domestic animals and giant pandas to compete for bamboo as a food source, further compressing wild pandas' food and living space [36]. Therefore, grazing activities have become the most influential human disturbance in the MMS [5].

Many empirical studies have mentioned the impact of human interference [8,9]. In order to more accurately predict the habitat area of the giant panda, a standardized quantification system of human interference should be established in the future, or at least combined qualitative analysis to ensure the comprehensiveness of research.

### *4.4. The Contribution Rate of Environmental Variables and Response Curves under Different Modelling Spatial Extents Can Provide New Ideas for Protecting Giant Pandas*

Together with geographic extents, the response curve can provide data support for the cross-scale protection of wild giant pandas. It provides references for protection departments of different regions and levels when implementing the protection of giant pandas:


points can be set up in areas that are within the pandas' suitable distance to a water source to improve wild panda investigation and monitoring efficiency.

(3) From a national macro-scale, national environmental protection organisations and national core ecological protection agencies should adopt macro-monitoring of factors with high contribution rates at large scales to control the fragmentation and degradation of the giant pandas' habitat, for example, monitoring annual precipitation and annual temperature differences.

In future relevant research, if protection or management suggestions need to be made for a certain level of administrative agencies, some data can be collected within their corresponding geographic level. This will help clarify the administrative divisions of giant panda protection agencies and make the division of labor between various administrative groups clearer [10].

### **5. Conclusions**

Due to climate change and human disturbance activities, the habitats of endangered wild animals continue to be degraded and fragmented, so their survival is threatened. The research of endangered species can be effectively supported by ENMs, especially the MaxEnt model, which can maintain the stability of model results even when sample data is extremely scarce. At present, the application of the MaxEnt model is mature, but there is still a lack of research on its modelling extent. This study attempts to analyze how changes in the modelling spatial range affect the results of the MaxEnt model, with a view to reducing errors in model research and providing technical support and management suggestions for the sustainable development of giant pandas and their habitats. Taking the MMS region and the MMS region modelled on a national scale as a mutual control group, the main conclusions are as follow: (1) When an endangered species that difficult in data collection of sites is the research object, the modelling range should be appropriately expanded and clipped into the target area to offset errors caused by missing sites; (2) Human factors are generally more significant at a small scale, but human interference at any scale is a factor that cannot be ignored; (3) The contribution rates of environmental variables at different scales are different. Ecological protection departments at different levels should pay attention to changes in the most significant environmental factors at corresponding scales that help control and intervene in potential negative impacts to achieve efficient protection.

The study only suggests that appropriately expanding the modelling scope can help offset the errors caused by the lack of data, but how to accurately offset the errors caused by missing data by changing the extent of modelling requires further research. In addition, the supervision and intervention of species protection agencies are vital to ensure the sustainable development of endangered species, such as giant pandas, and maintain the stability of their habitat. Thus, the idea that administrative agencies at all levels distinguish the management priorities of endangered species through the spatial scope can be further applied to practice to make the effects of supervision and intervention more efficient.

**Author Contributions:** Z.H., conceptualization, data Curation, writing—original draft preparation, validation, formal analysis, visualization; T.P.D., methodology, data curation, supervision, writing review and editing; L.C., formal analysis, writing—review and editing; A.H., writing—review and editing. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** Thanks to Harriet Dawson (University of Edinburgh) for her review and suggestions for improvement on earlier drafts of this manuscript.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


## *Article* **Public's Intention and Influencing Factors of Dockless Bike-Sharing in Central Urban Areas: A Case Study of Lanzhou City, China**

**Wei Ji 1,2, Chengpeng Lu 1,2,\*, Jinhuang Mao 1,2, Yiping Liu 1, Muchen Hou 1,2 and Xiaoli Pan 1,2**


**\*** Correspondence: lcp@lzu.edu.cn

**Abstract:** Taking the main district in Lanzhou city of China as an example, the questionnaires were designed and distributed, and then the effects of five factors, i.e., behavioral attitude, subjective norm, perceived behavioral control, perceived ease of use and perceived usefulness, on the behavioral intention of dockless bike-sharing (DBS) use were empirically analyzed based on the integrated model of technology acceptance model (TAM) and the theory of planned behavior (TPB) as well as the structural equation model. Results show that the five factors all impose significantly positive effects on the public's behavioral intention of DBS use but differ in influencing degrees. Behavioral attitude, subjective norm and perceived behavioral control can all directly affect the public's behavioral intention of DBS use, with direct influence coefficients of 0.691, 0.257 and 0.198, while perceived ease of use and perceived usefulness impose indirectly effects on behavioral intention, with indirect influence coefficients of 0.372 and 0.396. Overall, behavioral attitude imposes the most significant effect, followed by perceived ease of use, perceived usefulness and subjective norm, and finally perceived behavioral control. This indicates that the public's behavioral intention of DBS use depends heavily on their behavioral attitude towards the shared bikes. In view of the limited open space of the main district in Lanzhou, the explosive growth of shared bikes, oversaturated arrangements, disordered competition, unclear and unscientific divisions of parking regions, and hindrance of traffic, this study proposes a lot of policy suggestions from the research results. A series of supporting service systems related to DBS should be formulated. The shared bikes with different characteristics should be launched for different age groups, gender groups and work groups. The corresponding feedback platform for realtime acquisition, organization, analysis and solution of data information, as well as the adequate platform feedback mechanism, should be established.

**Keywords:** dockless bike-sharing (DBS); behavioral intention; influencing factors; theory of planned behavior (TPB); Lanzhou City

### **1. Introduction**

Urban traffic problem now has gradually become one of the main factors that affect the improvement of urban built environment in China [1]. Green travel is exactly the reflection and reformation of current urban traffic development [2]. Bike sharing, as a novel travel mode featured by the Internet and sharing, offers a new transport mode for short trips and enhances the connections with other modes such as bus and subway [3–6]. Bike sharing can not only satisfy the heavy demand for short trips but also contribute to solving a lot of urban stubborn diseases such as traffic jams and environmental pollution [7], thereby injecting new vitality into the urban traffic system. Bike sharing can make urban traffic services more diversified and intelligent [8] and play quite an active role in establishing a green travel system. However, with the rapid development of mobile information

Y.; Hou, M.; Pan, X. Public's Intention and Influencing Factors of Dockless Bike-Sharing in Central Urban Areas: A Case Study of Lanzhou City, China. *Sustainability* **2021**, *13*, 9265. https://doi.org/10.3390/su13169265

**Citation:** Ji, W.; Lu, C.; Mao, J.; Liu,

Academic Editors: Baojie He, Ayyoob Sharifi, Chi Feng and Jun Yang

Received: 26 July 2021 Accepted: 17 August 2021 Published: 18 August 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

communication technology and full penetration into social life, time, space and distance have been highly compressed, leading to the transformation of daily lives of residents [9]. Meanwhile, temporal-spatial elasticity and flexibility of some activities such as shopping, leisure and commuting have been enhanced [10], and the public's intention and choice to use shared bike services also show some new features [11].

Over the last several years, China has experienced a fast development in DBS, which provides great convenience for travelers [12]. It is a crucial supplement for other traffic modes in cities. Understanding the travel mobility and traffic demand of DBS is conducive to many urban problems such as urban development and traffic management [13–15]. Currently, scholars have mainly investigated cycling trips on the planning of public bike docks and the arrangement model [16,17] and explored the influencing factors of public bike trips from the perspectives of the built environment, land utilization and user characteristics [18–24]. The DBS has been poorly investigated to date. Most existing studies have laid the research emphasis on temporal-spatial distribution characteristics [25–27], use travel features [28] and the influencing factors based on the spatial scale [29]. Accompanied with increasing urban traffic pressure and environmental pressure, bike sharing has become one of the most basic daily living consumptions for the public. Investigating the use of shared bikes by residents should include both in-depth studies at a macro spatial scale and the discussions of transportation behavioral intentions of urban residents and the related influencing factors from the micro individual perspective.

Currently, some studies have revealed that individual psychological factors imposed significant effects on the residents' intention to use public bikes [30,31]. Accordingly, whether individual psychological factors also significantly affect the use of DBS. In view of this, this study selected the main urban zone in Lanzhou, China, i.e., Chengguan District, Qilihe District, Xigu District and Anning District as research areas. By taking DBS as an example, the integrated model of the public's intentions of DBS use was established based on the technology acceptance model and the theory of planned behavior. In combination with the structural equation model, the subjective psychological factors that affect the public's intentions of DBS use were explored so as to make up for the shortcomings of investigating travel behaviors in geographic space. Meanwhile, this study can provide decision-making basis for the governmental administration departments to adequately guide green travel.

### **2. Study Area and Methods**

### *2.1. Study Area*

Lanzhou city is located at the geometric center of China's land territory, which is also an important center city, industrial base and comprehensive transportation junction in the northwest of China. As also shown in Figure 1, Lanzhou is a key node city in Silk Road Economic Belt. Five districts (Chengguan, Qilihe, Xigu, Anning and Honggu), five counties (Yongdeng, Yuzhong and Gaolan), 1 national-level new district (Lanzhou New District), and two national-level development zones (Lanzhou High-Tech Development Zone and Lanzhou Economic and Technological Development Zone) are under the administration. Lanzhou covers a total area of 13,100 km2 and an urban area of 1631.6 km2. According to the 7th National Census data, the permanent resident pollution in Lanzhou reaches up to 435.94 million. DBS has successively appeared in Lanzhou since March 2017. Currently, over 300,000 shared bikes and electrical bikes owned by different companies such as Hellobike, Qingju and Mobike were put on the market in Lanzhou. Additionally, 1 subway line, 27 intercity bus lines, 52 urban-rural public bus lines and 11 micro-bus lines operate normally in Lanzhou. According to The Investigation Report of Xinhua Green Travel Index (2017), Lanzhou ranks first in terms of green travel satisfaction degree [32]. The respondents are satisfied with the infrastructure construction, service and policies of urban green travel. The shared bikes can provide the residents with great convenience owing to the design concept of go and stop at any time. However, due to limited urban spatial resources, unclear and unscientific division of bike parking zones can not only cause the

wasting of resources and affect urban appearance but also result in the congestion on sidewalk road and hinder the traffic [33].

**Figure 1.** Location of the Lanzhou City.

### *2.2. Integrated Model*

Theory of reasoned action (TRA) is one of the most basic theories to investigate cognitive behaviors, which can be used for predicting behaviors and behavioral intention including both general social activities and consumption activities [34]. TRA advocates that individual behaviors are subjected to behavioral intention while the behavior intention is codetermined by attitude and subjective norms. However, TRA is only applicable to predicting the behaviors fully controlled by the mind. Therefore, on the basis of TRA, Ajzen added a new predictive variable, i.e., perceived behavioral control, and established the Theory of Planned Behavior (TPB) [35]. Nevertheless, TPB still exists limitations in the adoption of new technologies. For this reason, Davis added two factors, i.e., perceived ease of use and perceived usefulness, to investigate users' using behaviors of information system, and proposed the technology acceptance model (TAM), which holds the opinion that both perceived case of use and perceived usefulness can affect the behavioral attitude and thereby affect the behavioral intention [36]. On the basis of TRA, the integrated model organically combines TPB and TAM, in which behavioral intention is regarded as the outcome variable, and behavioral attitude, subjective norm, perceived behavioral control, perceived case of use and perceived usefulness are five causal variables affecting the behavioral intention [37].

### *2.3. Structural Equation Model (SEM)*

The structural equation model, integrating factor analysis and path analysis, can be used for analyzing both direct and indirect relations among variables. A complete structural equation includes two equations, i.e., the measurement equation that describes the relation between the latent variable and measured variable, and the structural equation that describes the relationship between latent variables [38,39]. According to the structural relationship among variables and the definitions of variables as shown in Table 1, the model including 6 latent variables and 23 measured variables was established, as shown in Figure 2.


**Table 1.** Variable definition.

Conclusively, based on TAM and TPB, this study employed SEM for empirically analyzing the effects of five factors, i.e., behavioral attitude, subjective norm, perceived behavioral control, perceived ease of use and perceived usefulness, on the public's behavioral intention of DBS use. The whole research process is shown in Figure 3.

**Figure 2.** Path analysis results of structure equation model.

**Figure 3.** Technology roadmap.

### *2.4. Data Sources*

Based on TPB and the scale of TAM [40,41], the questionnaire was designed by consulting the related references [42–45]. The present questionnaire was implemented by an authoritative survey company in China, WJX.cn. In order to ensure the respondents have ever used the shared bikes, the jumping question was set in the questionnaire. For example, the respondent can end the questionnaire if he or she gives the answer 'Never use the shared bike'. In other words, the users that never use the shared bike can jump to the end of the questionnaire after filling in the basic information. The detailed contents of the questionnaire are shown in Table 2. The distribution and collection of the questionnaires were performed from May to July 2021. To be specific, 700 questionnaires were distributed, and 680 questionnaires were collected, of which 52 questionnaires were judged as invalid. Therefore, 628 valid questionnaires were collected, while 52 questionnaires from the respondents who gave the answer 'Never use the shared bike' or answered the question within a short time were considered invalid. Therefore, 628 valid questionnaires were collected. Table 3 shows the basic characteristics of the sample data. Among all respondents, 54% were female, that is, the female respondents slightly exceeded male respondents. In terms of age, 14.04% of the respondents were within the age range of 12~18. According to the Regulations for the Implementation of Road Traffic Safety Law, the cyclists should be aged 18 or above. The respondents under the age of 12 were rejected with the answer 'Never use the shared bike'. An overwhelming majority of respondents (with a proportion of 81.21%) were 19~50 years old, which may be due to the fact that the residents in Lanzhou with an age of over 50 mainly use non-smart phones. Since the survey was performed in central downtown, the respondents showed favorable education degrees. A total of 66.08% of the respondents had a Bachelor degree or above, while 55.09% had a monthly income of below CNY 4000. This suggests that DBS is more preferred by low-income groups. In terms of the frequency of utilization, approximately 57.97% of users used the shared bikes 1~4 times in one week, while 7.96% of users used the shared bikes over 16 times in one week; approximately 38.06% of users selected to use the shared bikes to reach railway stations, motor stations or subway stations. Obviously, DBS played an indispensable role in public transport connection. Overall, the structural characteristics of the respondents in this study fit well with the sociological population characteristics of the residents who use the shared bikes in previous studies [46,47], which confirms the favorable representativeness of the samples in this study.


**Table 2.** Questionnaire design.

#### **Table 2.** *Cont.*



### **3. Results**

*3.1. Factor Analysis of Residents' Behavioral Intention of Using the Shared Bikes*

The reliability and validity of the questionnaire were analyzed by SPSS23.0. The Cronbach's α coefficient was selected as the measuring index of reliability [47]. Through measurements, the Cronbach's α coefficients of behavioral attitude, subjective norm, perceived behavioral control, perceived usefulness, perceived ease of use, and behavioral intention were 0.742, 0.765, 0.801, 0.831, 0.859 and 0.728, respectively, while the overall Cronbach's α coefficient was 0.907, indicating the high reliability of both the whole questionnaire and various dimensions. The validity was measured via the KMO test and Bartlett sphericity test [48]. The overall KMO value of six latent variables was 0.907, which has also passed the Bartlett sphericity test. Accordingly, factory analysis can be performed on the questionnaire. The correlation coefficient matrix was then constructed by Amos23.0, and the path coefficient in the model was estimated via maximum likelihood estimation. Before parameter estimation, the model was first fitted. The goodness of fit of the model was evaluated by 11 specific indexes. In the initial model fitting, the Chi-square freedom degree ratio (c2/df) was 4.563, suggesting the model could not adequately reflect the

questionnaire and needed to be improved [49]. Based on the principle of reasonableness and specification, the model was modified in accordance with the Amos correction index so that various fitting indexes can finally satisfy the test requirements (see Table 4). Overall, the fitting was favorable, suggesting high compatibility between the model and the present questionnaire. The integrated model based on TAM and TPB is applicable to the studies in the traffic domain, which can well explain the public's behavioral intentions of using the shared bikes.


**Table 4.** Model fitting indexes and the fitting results.

The standard loading factors between 23 observable variables in parameter test results and the corresponding latent variables ranged from 0.545 to 0.891, which all exceeded 0.5, suggesting a favorable basic fit measure. The values of C.R all exceeded 1.96, that is, the parameter estimated values all reached the significance level of 1% (Table 5). The standard factor loads of four measured variables regarding behavioral attitude were 0.690, 0.776, 0.635 and 0.862, respectively, suggesting the convenience, comfort, interesting degree and value of using the shared bikes can significantly affect the public's opinions and behavioral intentions of DBS use. The standard factor loads of three measured variables regarding subjective norm were 0.651, 0.891 and 0.862, respectively, suggesting that residents are subjected to positive influences from family members, friends and schoolmates (workmates) when using the shared bikes. The standard factor loads of four measured variables regarding perceived behavioral control were 0.545, 0.710, 0.765 and 0.621, respectively, suggesting positive effects of the mobile skill, riding skill, physical fitness and psychological quality, especially physical fitness, on the use of shared bikes. The standard factor loads of five measured variables regarding perceived usefulness were 0.767, 0.679, 0.701, 0.595 and 0.644, respectively, suggesting that DBS use can bring about useful experiences such as environmental protection, enhancement of efficiency, bodybuilding and energy conservation, among which environmental protection was most remarkable. The standard factor loads of four measured variables regarding perceived ease of use were 0.690, 0.801, 0.778 and 0.791, respectively, suggesting that the registration program, the parking convenience, payment and program and the performance of the shared bikes affect the public's perceiving ease degree of DBS use; in particular, the parking convenience imposes most significant effect. The standard factor loads of three measured variables regarding behavioral intention were 0.596, 0.674 and 0.713, respectively, indicating that the residents not only are willing to use the shared bikes but also encourage and advocate the other people to use the shared bikes.


**Table 5.** Model fitting indexes and the fitting results.

Note: \*\*\* suggests the significance at the level of 1%, and C.R value equals the t value (the same below).

### *3.2. Analysis of the Influencing Factors of the Public's Behavioral Intention of DBS Use*

Figure 4 shows the relations among six latent variables, i.e., behavioral attitude, subjective norm, perceived behavioral control, perceived usefulness, perceived ease of use and behavioral intention. Table 6 displays the direct, indirect and total effects of the former five factors on behavioral intention. The following conclusions can be drawn.

**Figure 4.** Path analysis results of structure equation model. Note: \*\*\* suggests the significance at the level of 1%.


**Table 6.** Effects of different variables on the willingness to use shared bikes.


### **4. Conclusions and Discussion**

By taking the use of shared bikes in the main distribution of Lanzhou, China as an example, this study established the integrated model based on TAM, distributed 628 valid questionnaires and systematically analyzed the effects and the related influencing mechanism of five factors (i.e., behavioral attitude, subjective norm, perceived behavioral control, perceived ease of use and perceived usefulness) on the residents' behavioral intention of using shared bikes. The main conclusions are given in detail below.

Firstly, these five factors, i.e., behavioral attitude, subjective norm, perceived behavioral control, perceived ease of use and perceived usefulness, all have significantly positive effects on the public's behavioral intention of DBS use. To be specific, behavioral attitude, subjective norm and perceived behavioral control can directly affect the residents' behavioral intention, with direct influence coefficients of 0.691, 0.257 and 0.198, respectively, while perceived usefulness and perceived ease of use impose indirect effects on behavioral intention, with indirect influence coefficients of 0.372 and 0.396, respectively. Secondly, different subjective psychological factors affect behavioral intention to varying degrees. Overall, behavioral attitude imposes most significant effect, followed by perceived ease of use, perceived usefulness and subjective norm, while perceived behavioral control imposes the least effect. The public's behavioral intention of DBS use depends greatly on their behavioral attitude towards the shared bikes. Finally, behavioral attitude is an important bridge that integrating both TAM and TPB since perceived usefulness and perceived ease of use can indirectly affect the public's behavioral intention of DBS use via the mediating effect of behavioral attitude. Meanwhile, perceived ease of use can also indirectly affect the public's behavioral intention of DBS use via two mediating variables, namely, perceived usefulness and behavioral attitude.

This study focused on the behavioral intention of DBS use and analyzed the main influencing factors, which is expected to provide theoretical guidance for DBS enterprises in product design and marketing plans. The combination of TPB and SEM in urban sharing economy study is a beneficial attempt, which shows stronger explanatory power than a single use of TPB and SEM. An in-depth investigation of the main distribution of Lanzhou can reflect the overall characteristics of urban residents in DBS use to a certain degree but still needs to be improved. In future studies, we can expand the research range, prolong the research period and perform multi-scale and dynamic analysis. Next, this study laid the emphasis on the effects of the public's psychological factors on behavioral intention. By taking into account the built environment [50,51], topographic features [52] and climate factors [53], future studies will overall evaluate the effects of both subjective and objective factors on the public's behavioral intention of DBS use so as to obtain more strict and abundant conclusions.

Considering the limited open space of the main district in Lanzhou, the explosive growth of shared bikes, oversaturated arrangements, disordered competition, unclear and unscientific divisions of parking regions and hindrance of traffic, the following policy suggestions were proposed in combination with the present conclusions. Firstly, enterprises should work on technological innovation, optimization and upgrade, enhance the fit measure between software and hardware, and formulate a series of supporting service systems related to shared bikes. To be specific, the shared bikes can be equipped with some devices such as speed variators, shock attenuation devices, shelves and back seats, and some protective devices such as helmets and gloves to provide better riding experiences under the premise of ensuring bike quality. Secondly, the shared bikes with different characteristics should be launched for different age groups, gender groups and work groups. For example, pink bikes can be designed for women, the bikes with animal themes can be designed for students, while the elderly electric shared bikes should be added for the middle and old people. Thirdly, enterprises should set the corresponding feedback platform for real-time acquisition, organization, analysis and solution of data information, establish adequate platform feedback mechanism, build their WeChat official accounts and official websites and achieve good management, so as to reinforce the communication with

users and the interaction between enterprises and users. Fourthly, the government should perfect the social credit system and incorporate DBS into the individual and enterprise credit system. On one hand, the enterprise credit system should be strengthened, that is, the credit system should be formulated to govern vicious competition among enterprises so as to encourage fair competition among enterprises, ensure the consumers' usage experiences and protect the related rights and interests of consumers. On the other hand, individual credit system should be established. For example, some improper illegal behaviors should be integrated into individual credit system via Internet, and then the users with insufficient credit scores will be restricted to a certain degree, which can better restrain user's DBS use behaviors and achieve civilized DBS use. Finally, as regard to the parking of shared bikes, the enterprise dominants should take reasonable distribution and grid administration. Various operators should reasonably allocate and distribute the shared bikes in accordance with usage density of shared bikes and population density, and increase the number of shared bikes at some public regions such as railway stations, subway stations, bus stations, schools, shopping malls and entertainment venues, which can facilitate connection with other transportation tools. Some supervision departments, such as the Bureau of Transport, the Bureau of Urban Administration and the Bureau of Market Supervision and Administration, should act in close coordination and strengthen communication and coordination to make concerted efforts and further standardize the parking regions of shared bikes. The parking system should set the function of temporary parking to provide the users with great convenience. Moreover, the convenient query function whether the destination can be parked or not should be added to avoid the extra dispatch fee for exceeding the designated parking area.

**Author Contributions:** Conceptualization, C.L.; methodology, W.J. and J.M.; formal analysis, W.J., Y.L.; investigation, W.J., J.M., Y.L., M.H., X.P. and C.L.; writing—original draft preparation, W.J. and C.L.; writing—review and editing, C.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by National Natural Science Foundation of China (41701142), and Fundamental Research Funds for the Central Universities (2020jbkyjc002).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data used to support the findings of this study are available from the corresponding author upon reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Sustainability* Editorial Office E-mail: sustainability@mdpi.com www.mdpi.com/journal/sustainability

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34 Fax: +41 61 302 89 18

www.mdpi.com

ISBN 978-3-0365-2671-3