1. Introduction
This study aims to provide a tool to analyze how European banks’ sustainability reports disclose their commitment to addressing the Sustainable Development Goals (SDGs) established by the United Nations (UN) in 2015 [
1]. For the reader’s convenience,
Table A1 provides in its first two columns the objective of each SDG and its number. Sustainability per se is a broadly defined term (see [
2]), and focusing on the SDGs provides guidelines on how to structure our research. In this research, we focus on a panel of European banks for various reasons. It is well known that European banks play a different role in economic growth than American banks since European non-financial companies rely mostly on banks for financing. In the U.S., in contrast, firms tend to rely on competitive stock markets for financing [
3]. Since banks can orient non-financial firms’ projects, they can also play a directing role in guiding firms toward adopting sustainability-enhancing projects in addition to what the regulatory framework may impose. They can also direct households in their investment decisions and help authorities develop new projects. Since banks play such an important role, understanding their stance on SDGs is therefore a systemically relevant question. By focusing on a subset of possible countries, we also generate homogeneity in the reports. As shown in [
4], the countries’ legal origin strongly affects the firms’ interest in CSR and, by extension, in the sustainable development goals.
Analysts who seek to rapidly compare reports from different companies are confronted with a large volume of information, which slows their decision-making. Our research, which is methodological in nature, aims to develop a solution for analysts and researchers in general to automate this task. Our methodology also applies to firms at large.
We contribute not only to the field at the methodological level but also to the field at the economic level. A survey paper [
5] on banking reporting on sustainability asked how the introduction of Directive 2022/2464 in Europe, which requires mandatory sustainability disclosure, could affect the disclosure of banks. In 2022, the United Kingdom enacted two mandatory climate-related financial disclosure laws. Interestingly, in our research, we cover the period from 2010 to 2022, which saw the enactment of Directive 2014/95/EU in 2017 to introduce mandatory non-financial disclosure. Therefore, with our methodology, we can begin to grasp the possible consequences of the introduction of new regulations. Our findings suggest that more banks will produce specific sustainability reports. Those banks that already have nonmandatory financial disclosure on sustainability will report more. We do not anticipate, however, a change in terms of what will be disclosed. Understanding the urgency of climate-related action may, however, change the emphasis of the discussions and the policies of banks. We will not only be able to classify the various sentences according to their SDG-related content but also analyze the texts in depth to gain a better understanding of what is said.
The most closely related research to ours is [
6]. These authors gathered various financial statements, such as management reports from annual reports, standalone documents, integrated reports, and other reports, for the 2017 accounting period from BankFocus. They obtained textual content for 262 banks that they analyzed manually, i.e., without the help of content analysis software. In a painstaking effort, they gathered information on the SDGs discussed and other bank-specific variables. The authors justified the necessity of manual classification by arguing that statements can be introduced by one of the 17 SDG icons rather than text. As a consequence, they claimed that their document represents the only methodological reference for combining sustainability disclosure with the SDGs. We seek to improve upon this claim by introducing a state-of-the-art text-analysis system that overcomes the challenge of automatically classifying sentences according to their SDG-related wording. This new methodology, its validation, and its first application are summarized in the next sections. By applying our methodology to sustainability reports, we can discuss additional dimensions such as changes over time regarding the emphasis of certain SDGs. To demonstrate that our methodology confirms and simplifies their approach, it should be mentioned that, by comparing the rankings of SDG popularity for our reports and for the accounting period 2017 with theirs, we obtained a highly significant correlation of 0.55. The correlation with findings from the consultancy industry is somewhat different regarding issues such as workplace, gender, and responsible consumption [
7,
8]. We explain this difference by the fact that the consultancy industry considers all industrial sectors and not specifically the banking sector, as we do here.
Furthermore, our ranking is in line with SDG ratings of firms in general and with ratings of banks [
9]. Note that, in this latter paper, sustainability-related text is also classified manually.
In this contribution, we introduce an advanced text analysis method that is also used by Internet search engines. Before discussing its function, it is useful to briefly retrace the history of text analysis. This is not a new method; however, it has experienced explosive development in recent years due to the availability of digitized text, faster computers, and theoretical development on how to estimate neural networks (NNs), which currently provide the most powerful approach. The textbook by [
10] provides a good idea of how text analysis has evolved over the years. In the early days, sentences were decomposed into their words, and those words were then counted, leading to summary statistics. In addition, thematic dictionaries with predefined wordings capturing certain themes were developed (initially, mostly in sociology). Many of the earlier applications focused on the question of whether a sentence or a text were expressed in a positive or negative tone. It became possible to construct scores to measure the relevance of certain themes. Methods for querying texts to find themes in the text were also developed. The term frequency-inverse document frequency (tf-idf) approach introduced by [
11] allowed us to down-weigh overall frequent words but over-weigh words that appear in a given text. For a modern discussion, see [
12,
13]. Such weighted wordings are found to provide simple ways to query a text. These early approaches, also called bag-of-words methods, have several limitations. The understanding of text is very limited because the order in which words are treated is as if they were thrown in a bag unrelated to their position in a sentence. Additionally, they cannot capture groups of words such as New York. Furthermore, they cannot understand more general concepts since the order of the words does not play a role.
Modern natural language processing (NLP) techniques, notably the bidirectional encoder representations from transformers (BERT) model [
14], eventually emerged. After training a model on a specific dataset, such techniques allow for the classification of sentences with great precision according to their content. The idea of those methods is to expose NNs to sentences where words are missing and to train the NN to learn the probability of missing words. In this paper, we train an existing BERT-type NN on a dataset filled with classified SDG-related topics. These training sentences are either open-source or hand-classified by us. This approach results in a cutting-edge classifier based on BERT to enhance the semantic mapping of wording within the SDG context. Our novel BERT model proves to be a valuable tool for classifying SDG-related texts in the banking sector, with an overall F1 score of 0.93. Once we have obtained a satisfactory classifier, we apply the methodology to bank sustainability reports covering the period from 2010 to 2022. This allows us to make statements about the prominence of certain SDG topics over time and over geographic regions. By investigating the content of the sentences according to the SDG classification, we can glean some understanding of how banks approach the various SDGs.
The structure of this paper is as follows. In the following section, we discuss both the relevant literature on sustainability research and the related state-of-the-art text analysis. In
Section 3, we discuss the data that we use to train BERT and the data that we use to analyze our classifications. In
Section 4, we discuss how we train the neural network. In
Section 5, we demonstrate that our network is well trained. We then discuss which SDGs banks emphasize over time across countries, and we demonstrate that the topics discussed are related to macroeconomic variables. In
Section 6, we investigate the themes that are actually discussed in the various SDGs. This further validates our methodology. Finally, we conclude in
Section 7. A technical appendix contains tables of descriptive statistics. The
Supplementary Materials contains detailed figures with more details related to the main tables of this paper.
5. Results
5.1. Performance of Our Novel RoBERTa Model
The performance of the classification task on each SDG was evaluated using precision, recall, and the F1 score. Precision measures the number of true positives relative to the total number of predicted positives. Recall measures the number of true positives in the total number of true positives. F1 is the harmonic mean between precision and recall scaled in such a manner that this statistic ranges from 0 to 1. A higher score corresponds to a better classification. Accuracy measures the number of correct predictions out of the total number of predictions. The macroaverage is the average accuracy across all possible classes. As shown in
Table 3, the accuracy (accu.), and the macro average (m.avg.) all reached 0.93, demonstrating that our model can differentiate texts across various SDGs effectively.
The best F1 score is obtained for SDGs 2, 6, and 13, followed by marginally lower scores for SDGs 1, 3, and 9. Overall, the classification is very satisfactory for all SDGs. The worst performer is SDG 10, with a score of 0.81, which is still respectable. We verified that the quality of the prediction was unrelated to the training sample.
5.2. Evolution of SDG Prevalence over Time
To gain insights into the prevalence and evolution of each SDG, for each report, we sum the total number of words for all sentences classified under a given SDG. This provides a score of the degree to which each SDG is discussed. For a given report, all the SDGs are then ranked according to this score. This ranking serves as an indicator of the incidence of each SDG, with a ranking of 1 denoting the most prevalent topic.
The first exercise is to investigate how often a given SDG has been discussed over time. To do so, we consider the matrix indicating for each report the ranks for all SDGs over time. We then obtain the median of all the reports for a given year. We focus here on the median, which is a robust measure. In the
Supplementary Materials, we provide box plots for each SDG and its temporal evolution. This allows for a more detailed analysis of the minimum, maximum, and interquartile frequency of each SDG across reports.
Table 4 shows how the 17 SDGs evolved from 2010 to 2022. The most discussed SDGs are displayed in red, and the least discussed ones are in green. There are several striking features. We notice very homogenous patterns over time for a few of the SDGs and structural changes around the fiscal years 2019 and 2020 for others. As mentioned in the literature, for 2017, the hierarchy of relevance was significantly correlated with the results found in the literature [
6]. Here, we are able to extend those results to the entire time period of 2010–2022.
If we focus first on 2010, the most frequently discussed SDGs are 8 (work), 12 (responsible consumption), 17 (partnerships), 9 (industry), and 16 (peace, justice). However, by 2022, the most discussed SDGs were 13 (climate action), 12, 7 (clean energy), 17, and 16, which reveals, as expected, the relevance of environmental issues. The consistently high ranking of SDG 17 aligns with the insights provided by [
23]. They suggest that shareholder and lender engagement, which are often encapsulated within the framework of SDG 17, play a crucial role in steering industries toward a more sustainable trajectory. The least frequently mentioned SDGs in 2010 were 10 (inequalities), 14 (life below water), 1 (poverty), 6 (clean water), and 2 (hunger). Interestingly, in 2022, according to our methodology, those SDGs remain the least mentioned ones.
We can now focus on possible structural changes over time. We notice that the ranking (and therefore also the colors) changes very little for the fiscal year 2017 when the EU Directive 2017/95 became mandatory. Since we are investigating the sustainability reports of listed banks, we interpret those findings to mean that banks produced more reports from that year on; see
Table 2. The content reported, however, remained homogeneous. Banks that started producing reports likely found inspiration in content from their peers. An investigation of the shape of the entire distribution of ranks for a given SDG is provided in
Supplementary Materials, confirming our findings. In particular, the standard deviation of ranks did not change over time, demonstrating that banks did not report in a more homogenous manner.
Our prediction concerning the mandatory sustainability disclosure Directive 2022/2464 is that banks will produce more reports, possibly longer ones, but that the content of the discussion will not necessarily be affected.
As shown in
Table 4, between 2018 and 2019, there was a jump in the ranking of SDG 13 related to climate change. Between 2019 and 2022, this topic became even more discussed. This increase in discussion of SDG 13 coincides with the publication by the EU of a supplement to the 2017 guidelines on nonfinancial reporting on climate-related information. This applies to financial institutions with more than 500 employees, which is the case for our listed banks.
We also note that, since 2019, with the unprecedented health crisis due to the COVID-19 pandemic, structural changes have occurred among several SDGs. As the rank of some of the SDGs improved, in parallel, the rank of others decreased. The SDGs gaining in popularity are 3 (health), 7 (energy), 13 (climate), and 15 (life on land), whereas SDGs 4 (education), 8 (work), 9 (industry), and 17 (partnership) dropped in ranking. These findings show that not only the relevance of climate- and energy-related issues increased but also health concerns as individuals shifted to working from home to adapting to the new situation. These findings are consistent with the studies by [
6,
54]. They also match [
55,
56,
57], who suggest that the unprecedented health crisis due to the COVID-19 pandemic and its economic effects necessitate a reevaluation of banks’ priorities concerning the SDGs in response to urgent health needs (SDG 3) and economic disruptions (SDGs 8 and 9). Ref. [
55] address the impact of the COVID-19 pandemic on GDP, income, and exports, pointing out the critical role of banks in supporting healthcare and economic recovery.
To strengthen the validity of these arguments, we carry out regression analysis to explore the relationships between the prevalence of SDGs and various indicators.
5.3. Regression Analysis
Encouraged by these reasonable findings, we propose that one could attempt to further validate our methodology by relating the changes in SDG levels that indeed change over time with some macroeconomic variables. Since some of the SDGs, such as SDG 6, remain constant, it appears futile to try to explain them by some variables. For this purpose, we decided to carry out regression analysis to explore the relationship between the prevalence of SDGs and various indicators. In those OLS regressions, the incidence of an SDG is the dependent variable, and variables such as health status and climate-related indicators are the left-hand variables. Note that we do not imply causal direction in our regressions. We view those regressions more like correlations with statistical significance.
To enable the reader to perform an easier comparison of the impact of a unit change in an explanatory variable, we studentize the dependent variable by subtracting the mean and dividing it by the standard deviation.
The results are presented in
Table 5. Current health expenditure has a significant direct correlation with the prevalence of SDG 3, aligning with the increasing focus on SDG 3 to support public health during the pandemic. In
Table 5, a positive coefficient indicates that an increase in a right-hand-side variable leads to a greater prevalence of the SDG, i.e., a better rank (1 being the best).
The next regression relates SDG 7 (energy) to the energy intensity level, which is a measure of how much energy is required to produce a unit of output (measured by GDP). As
Table A4 from the
Appendix A demonstrates, due to energy-improving technology, this measure decreased between 2010 and 2020. Another opportunity for banks to contribute to green industries is through supporting large-scale renewable energy projects addressed by SDG 7 [
58]. This also corresponds to a rising emphasis on renewable energy, as shown in
Table A4, which indicates that the growth rate for the renewable energy share in the total final energy consumption has more than doubled in 2020. The parameter estimator of the second regression in
Table 5 confirms that banks also talked much more about this topic.
Next, in this table, we find the regression concerning SDG 8 (work and growth). SDG 8 fell from being top-ranked in 2010 to being 7th-ranked in 2022. For these SDGs, we were able to find three potential explanatory variables: a measure of CO
emissions, one on employment, and one on low carbon technology. Since CO
emissions fell and unemployment decreased during the period of 2010–2020, the prevalence of SDG8 also decreased. For instance, as
Table A5 reveals, unemployment at the European level fell from 8.9% to 5.7% by 2020. We are also able to corroborate the evolution of SDG 8 by finding that an increase in the trade of low-carbon technology products leads to a decrease in the focus on SDG 8.
In Columns 6 and 7 of
Table 5, we relate SDG 9 (industry), a decreasingly popular SDG, with expenditures on economic affairs and expenditures on housing and community amenities. SDG 9 is negatively influenced by expenditures on economic affairs but positively influenced by expenditures on housing and community amenities. The outbreak of the pandemic led banks and governments to prioritize economic affairs to address immediate financial crises. Conversely, funds allocated for housing and community amenities were redirected to urgent pandemic-related needs.
In the last column, we find the relation between SDG 13 (climate action) and greenhouse gas emissions. SDG 13 represents the strategic effort of banks to mitigate climate-related financial risks [
23].
Table 5 shows that a reduction in GHG emissions is accompanied by a heightened focus on SDG 13.
Overall, the high statistical significance of the parameters appears to validate our classification methodology.
5.4. Regional Differences
To delve into sustainable progress within the European Union, it is crucial to highlight regional disparities. To provide a more comprehensive understanding of the significant shifts in key SDGs over time, we further segmented all the reports into four geographic groups: western, northern, southern, and central and eastern Europe. In
Table 6, we follow the same color code as in
Table 4, with red (green) being the most (least) mentioned SDG. As we immediately notice, within some of the categories, there exist different regional nuances. Western, southern, and central and eastern Europe witnessed a remarkable surge in the ranking of SDG 3 (health). This increase suggests that heightened attention should be given to healthcare concerns during the outbreak of the COVID-19 pandemic in 2020. However, northern Europe displayed a conservative increase. This observation aligns with the level of healthcare services across European regions, as evidenced by the data presented in
Table A3. Northern European countries have demonstrated extensive public health coverage. Such well-established healthcare conditions likely mitigated the potentially disruptive effects of the outbreak of the COVID-19 pandemic at the start of 2020, thereby maintaining the relative stability of the focus on SDG 3 in Northern Europe. SDG 13 (climate) demonstrated a continuous upward trend from 2019 onward. The most substantial increase was observed in Central and Eastern Europe. This increase can be related to the changes in greenhouse gas (GHG) emissions shown in
Table A6. Over the period from 2017 to 2021, GHG emissions decreased by 22% for Northern European countries which had already a low level of GHG emissions. For western and southern Europe, the corresponding variations are −11% and −13%. Unfortunately, for central and eastern countries, the change is +9%. Also, this is starting from the highest level in Europe. The year 2021 marked a distinct divergence in government expenditures on environmental protection within the EU. While other EU regions cut back on their environmental spending, central and eastern Europe was the only region in which spending was bolstered, denoting an enhanced commitment to climate action within this region. This coincides with a sharp increase in the prevalence of SDG 13 for those countries.
7. Conclusions
In this study, we examine the European banks’ commitment to the SDGs as presented in their sustainability reports. We leverage a state-of-the-art BERT model to classify the content of 816 bank sustainability reports into 17 SDGs. Our novel BERT-based classifier, which achieved an overall F1 score of 0.93, proves to be a reliable tool for our research.
After classifying each sentence of each bank sustainability report over the 2010–2022 accounting period, we first investigate the temporal evolution of the prevalence of the 17 SDGs. The first step is to confirm the ranking of SDG prevalence previously generated in the literature [
6] who examine, however, only a single year of reports and use manual classification. We find that banks mention certain SDGs relatively infrequently; see SDG 1 (poverty), 2 (hunger), 6 (water), 10 (inequality), 14 (life below water), and 15 (life on land). The consequence of neglecting those factors has already been discussed [
23]. Some of the SDGs are often mentioned throughout the period; see SDG 8 (work), 9 (industry), 12 (responsible consumption), 16 (strong institutions), and 17 (partnership for the goals). We also notice the emergence of certain themes, particularly those related to the individual, the environment, and the climate. We find that SDG 3 (health) experienced a notable increase in rank in 2020 and then returned to its prepandemic level by 2022, indicating a significant but transient impact of the pandemic on banks’ attention on health issues. Additionally, we note that SDG 7 (clean energy) and 13 (climate action) experienced a jump in 2019.
Then, by using regression analysis, we confirm for those SDGs that experienced temporal evolution that macroeconomic indicators are correlated with this evolution. Next, we consider again the classification of the SDGs, but in addition, we introduce a regional distinction. This new dimension reveals that, for climate action, southern as well as central and eastern European countries lagged behind western and northern Europe. However, since 2019, these countries have undergone remarkable advances in this SDG. This coincided with particular financial campaigns at the EU level.
To glean an understanding of the topics that banks discuss within each SDG category, we apply word counts and a clustering algorithm to the bank reports year after year. The cluster analysis (k-means) provides the grouping of themes that we further flesh out with the word counts. Focusing on those SDGs with the strongest temporal evolution, we can observe thematic changes within each SDG. For SDG 3, the COVID-19 pandemic temporarily intensified the emphasis on financial partnerships and employee well-being for combating the virus. However, as the pandemic’s impact diminished, banks’ attention shifted back to prepandemic patterns, indicating the resilience and adaptability of banks in the face of external challenges. SDG 7 has shown banks’ increasing support for sustainable projects. Notably, banks have integrated detailed energy consumption and emissions reporting, reflecting a trend toward transparency and accountability in addressing environmental challenges. In terms of SDG 8, discussions evolved from basic employment metrics to a more comprehensive view, such as work-life balance, employer reputation, and a sense of belonging. Upskilling and reskilling in the digital domain have also gained attention. Starting in 2020, however, the focus shifted toward job creation and unemployment along with the impact of the COVID-19 pandemic on the labor market. With respect to SDG 9, technological developments have transitioned banking operations from traditional methods to digital solutions and from foundational online features to advanced technologies such as blockchain and AI. Such technological advancements have accelerated data generation and improved customer experience. For SDG 13, banks progressed from the early recognition of climate change and collaborative efforts to integrating climate risk into core business operations. These shifts indicate that global events, technological advancements, and societal values influence discussions of SDGs.
These findings have timely implications. We believe our study provides a strong starting point for further exploration of the textual analysis of sustainability-related issues. Our automated tool simplifies the often tedious job of accessing lengthy reports from various companies. This innovation holds promise for transforming practices for both analysts and industry professionals alike.