Towards Automating the Identification of Sustainable Projects Seeking Financial Support: An AI-Powered Approach

Abstract

The criticality of sustainable development to control the unprecedented consequences of climate change is clear. A vital element in launching sustainability projects is financing, especially for projects by small and medium enterprises. The first and crucial step to offering financing services for sustainable development is to identify and evaluate promising projects. The current practice to accomplish this step heavily depends on subject-matter expertise and professional networks. The current practice also involves extensive manual document reviews and subjective decisions. Therefore, existing methods are time-consuming, inefficient, and not scalable. This study proposes an automated system to identify potential sustainability projects for financing services using Artificial Intelligence (AI). The proposed method uses web crawlers and text mining solutions, including Natural Language Processing (NLP), to search the Internet, analyze text data, evaluate the information quantitatively, and identify potential sustainability projects for financing services. The proposed method was implemented and empirically assessed. The results indicate that the AI-enhanced system is able to identify and prioritize potential sustainability projects with 87% accuracy. The outcomes of this study will help financial experts and decision-makers take advantage of the information available on the Internet efficiently to improve the existing methods for identifying potential projects for financing services.

1. Introduction

Climate change and its unprecedented consequences are impacting every aspect of human life. The criticality of sustainable development is evident more than ever. For years, climate change was only a scientific discussion. Now, it is a central part of the political agenda and industrial evolution. A successful and effective shift towards sustainable development is a multidimensional challenge that requires investments in novel and innovative solutions to address different aspects of the problem, including scientific and technological advancements, new policies and regulations, and transformation of the industrial base to the reality of a circular economy through novel financing mechanisms. There have been an extensive number of studies in recent years that proposed novel scientific solutions for the technical aspects of sustainable development. Examples include efforts to understand and model earth systems and climate change (e.g., [1,2,3,4]), developing new technologies that offer alternative sources of energy such as solar electric panels [5], wind turbines [6], hydroelectric turbines [7], creating new materials with lower adverse environmental impacts (e.g., [8,9,10]), novel solutions for recycling and waste management (e.g., [11,12,13]), and new methods for lifecycle assessment in development projects (e.g., [14,15,16]). In addition, many studies focused on evaluating the design and analyzing the effectiveness of policies and regulations for sustainable development, such as the Building Research Establishment Environmental Assessment Method (BREEM) program (e.g., [17,18]), Leadership in Energy and Environmental Design (LEED) program (e.g., [19,20]), and the Environmental, Social, and Governance (ESG) rating system (e.g., [21]).

Financing is a vital aspect of sustainable development that needs further investigation to address emerging needs and challenges. Recent projections indicate a significant increase in investments in sustainable development projects. For example, Bloomberg Intelligence [22] estimated that the total financial value of sustainable developments will reach $53 trillion by 2025, doubled from 2018. Financial institutions play a critical role in supporting initiatives and projects for sustainable development. Ideas for green and sustainable solutions may come from various entities, such as industrial sectors, universities, research centers, research and development initiatives, government agencies, municipalities, and community organizations. The size of projects may range from small to mega projects. To effectively support evolution toward developing a more sustainable world and to encourage industrial entities to adopt sustainable solutions, the role of financial institutions is crucial. The first and essential step to offering financing services to sustainability plans is to identify and evaluate promising projects. This is specifically vital for projects by Small and Medium Enterprises (SMEs) that may have limited access to financial resources for their sustainability projects. To address this need, the current state of practice revolves around subject-matter experts’ capabilities to identify potential projects using their experience, professional networks, and subscription-based databases. This process heavily depends on experts’ knowledge, expertise, and experience. It may also involve extensive manual document assessment and subjective decisions. Therefore, the existing methods are time-consuming, inefficient, and not scalable.

The Internet is a significant source of information that is not properly used in the existing processes. Many agencies, including SMEs, share information about their plans and projects on their websites. These websites can be a great source of information for financial institutions looking for promising sustainability projects. However, the current manual process is not able to review millions of online pages.

In this study, we hypothesize that, if financial institutions were able to augment their current processes with intelligent systems that are able to automatically review and assess available information on the web, they would be able to expand their targets and find projects that may be missed in the existing manual processes.

Motivated by this notion, the overarching objective of this study is to create and evaluate the performance of a fully automated system using Artificial Intelligence (AI) that can search the Internet, analyze text data, evaluate the information quantitatively, and identify potential sustainability projects for financing services. More specifically, this study empirically addresses the question of whether AI-enhanced solutions can effectively facilitate the process of identifying potential sustainability projects for financing services.

The created AI-enhanced system, called “Green-Connect”, first crawls the Internet to find potential sources of information using a set of predefined criteria presented in a set of keywords. It then uses Natural Language Processing (NLP) to analyze the text data in the detected websites to find potential financing opportunities for sustainable development projects. Finally, it determines a relevancy score for each text document that indicates the value of information in the document.

The outcomes of this study will help financial institutions, policymakers, researchers, and decision-makers devise more efficient and effective ways to manage their financial investments in sustainable development projects.

The remainder of this paper is structured as follows. In Section 2, we review existing studies that aimed to facilitate the financing process for sustainability projects. In Section 3, we articulate the knowledge gap and the research objective of this study. In Section 4, we introduce methodology to create the AI-enhanced system and present the implementation outcomes of the proposed system, discuss the results, and assess the system’s performance. Finally, we summarize the contributions of this study to the existing body of knowledge and state of practice, limitations of this study, and potential directions for future studies in Section 5.

2. Literature Review

In the area of facilitating financing processes for sustainability projects, a considerable number of studies focused on the parameters related to investment characteristics and economic attractiveness. A group of studies explored the key barriers to financing sustainable development projects and provided potential strategies to address those barriers. For example, Baru [23] conducted a systematic review of the literature to identify prevailing challenges in the process of financing sustainable development projects. The identified challenges include a lack of stakeholder cooperation, low private sector engagement, lack of stakeholder willingness, poor governance and political stability, and unavailability of reliable information. Hafner et al. [24] identified policy uncertainty and short-termism in the financial system as the two main barriers to investment in green energy projects. The authors suggested expanding sustainable investment mediums, such as green bonds, as a useful short-term solution. For long-term policy intervention to tackle the identified barriers, they proposed adopting a systems perspective as an analytical framework to identify key leverage points. Another study [25] listed strict policies and guidelines, capital constraints of financial institutions, low rate of return, high risk of green investment, and lack of access to sufficient information as the major barriers currently impeding investors’ engagement in sustainability initiatives. To overcome the existing obstacles and mobilize private financing, Clark et al. [26] proposed solutions, such as institutional and policy reforms, providing a centralized information hub, thus enhancing the cost-effectiveness of projects.

Another group of studies aimed to facilitate financing sustainability projects by designing applied financing mechanisms. For example, Yoshino et al. [27] introduced a community-based funding scheme that utilizes the spillover effects of financing green projects in the form of tax returns to increase the rate of return and incentivize private investors. In another study, Taghizadeh-Hesary and Yoshino [28] proposed a funding framework based on Distributed Ledger Technologies (DLTs) that integrates credit guarantee schemes and tax-based incentives to manage risks and induce the private sector’s engagement in financing green energy projects. To promote financing sustainable infrastructure systems, a study [29] proposed a financial framework based on mezzanine-type debt that provides the chance of exchanging debts for equity shares if sustainability criteria are reached. Their proposed framework enables the lenders to capture some of the corporate value created by the project.

A limited number of studies explored the factors that may influence investors’ intention and motivation to finance sustainable development. For example, Reyhanloo et al. [30] surveyed the attitude of 68 private investors toward Land Degradation Neutrality (LDN). Their findings indicate that the motivation to invest in LDN can be triggered by promoting a consciousness of sustainability and the prospect of long-term financial returns. At the same time, short-term profit maximization plays a subordinate role. In another study [31], the outcomes of a survey revealed that younger individuals, individuals with pro-environmental attitudes, and those with greater financial resources are more likely to invest directly in renewable energy projects and even seem to accept some levels of financial risks in such environmentally-friendly projects. Another study [32] found that self-transcendent values (e.g., social and environmental values) drive private and institutional investors to consider social and environmental factors in their investment decisions. Their findings also indicate that the main obstacle toward Socially Responsible Investment (SRI) is a negative belief among investors about the financial returns of SRI.

Another group of studies attempted to analyze the financial performance and economic attractiveness of sustainable businesses based on their ESG properties. For example, a study [33] proposed a machine-learning approach that quantifies startups’ ESG properties by extracting information from textual data. This approach explored the economic attractiveness of small- and medium-sized enterprises. It found that startups with high ESG scores benefit from substantially higher valuations, while they have lower financial returns in the first year of their business. Friede et al. [34] reviewed the findings of around 2200 empirical studies to explore the link between ESG criteria and Corporate Financial Performance (CFP). They observed that a significant number of studies reported a stable positive ESG impact of CFP over time. A more recent study [35] examined the relationship between ESG and financial performance in more than 1000 research articles published after 2015. The study reported a growing consensus that proper management of ESG criteria typically results in improved operational metrics, such as Return On Equity (ROE), Return On Assets (ROA), and stock prices. The authors also noted that ESG investing appears to generate market rate and excess returns, especially for long-term investors, and it provides downside protection during an economic or social crisis.

3. Knowledge Gap and Research Objective

As briefly assessed in Section 2, many previous studies have contributed to facilitating the financing of sustainability projects from various aspects. However, little is known about developing systematic and data-driven methods to identify potential sustainability projects for financing services. An essential key to addressing this need that motivated this study is to create technology-mediated solutions that are able to automatically look for information, analyze it, and suggest potential target projects to subject-matter experts.

The overarching objective of this study is to create and empirically assess the performance of a novel approach for identifying and linking sustainability projects with financial resources using AI and Machine Learning (ML)-based solutions. To achieve this objective, we created an AI-enhanced digital platform, Green-Connect, that can automatically harvest and analyze online textual information. We then empirically assessed and quantified the proposed solution’s performance.

The primary rationale behind the proposed approach in this study revolves around AI capabilities as they pertain to developing automated and data-driven methods. AI capabilities can be integrated with human judgement to increase efficiency and accuracy in repetitive activities that are traditionally completed manually. Considering the dynamic nature of the Internet and constant generation of information sources, leveraging AI and ML capabilities allows for continued and prompt collection and analysis of new information. Among various AI and ML abilities, NLP-based methods offer opportunities for creating automated systems that can analyze a significant amount of text data in a timely manner. These solutions can be tailored and customized to specific requirements and needs. More specifically, NLP-based solutions have been implemented in various domains, such as healthcare [36,37,38], cybersecurity [39,40,41], and marketing [42,43,44], and they demonstrated reliable performance in analyzing a large amount of text data efficiently and accurately.

4. Method

The proposed solution to address the objective of this study uses advanced AI and machine learning techniques, including NLP, for data acquisition and manipulation. The Green-Connect platform consists of two main components: (1) a web crawler and (2) a text mining system. Using these two components, the platform can automatically harvest information on the Internet, identify sustainability projects, and rank them based on customized criteria defined by subject-matter experts. Figure 1 shows the overall structure of the proposed method.

In the remainder of this section, first, we explain the design of the two main components. Next, we present the platform’s structure and how it integrates these two components with the input and output data.

4.1. The Web Crawler Component

A web crawler is a program that automatically retrieves and processes data from the Internet [45]. Web crawlers commonly use online search engines to discover websites containing information about a specific topic, and they harvest information [46]. Web crawlers operate by starting at a seed URL and then following links on that page to other pages within the same website or other websites. As the crawler visits each page, it retrieves the content of the page and adds it to an index. It also follows any links on the page to other pages and continues this process recursively until it has retrieved and indexed all the pages it is programmed to visit. Web crawlers can be customized to perform certain tasks, such as searching for specific data types or following links within specific websites or domains. They can also be programmed to follow defined rules or guidelines.

The web crawler we designed and developed for the Green-Connect platform uses three sets of criteria to explore and exploit the Internet. The first set focuses on keywords related to the context of sustainability. The second set concentrates on keywords related to potential development projects. Additionally, the third set contains keywords related to financing needs and opportunities. A website should contain information related to all three sets of criteria to be shortlisted by the crawler system in the Green-Connect platform.

Although the web crawler can search the Internet automatically to identify web pages with potentially related information using a set of keywords, it is not able to rigorously analyze and evaluate the textual information in the shortlisted web pages. The rigorous analysis is conducted by the text mining system, which is explained in the next section. The shortlisted documents retrieved from the explored websites are the inputs to the text mining component.

4.2. Text Mining Component

The text mining component is designed based on an NLP modeling framework proposed by Lipizzi et al. [47]. The framework is based on the domain specificity theories indicating that elements in a natural language document, including its wordings, phrases, sentences, and paragraphs and their patterns, and they are linked to the context and domain-specific information the document delivers [48]. Accordingly, a domain-specific document, such as a text related to a sustainability project, has a particular syntaxial and semantical frame that distinguishes it from other documents. This intuition is the key to the proposed framework and its mechanisms in using NLP. The framework consists of three main components: (1) Room, (2) Benchmark, and (3) Incoming Documents. The Room is a computational representation of the knowledge in the area of financing sustainability projects. It serves as the cornerstone in the search and evaluation processes. The Benchmark is a list of keywords defining the evaluation criteria. Keywords are defined and weighted by domain-specific subject-matter experts. The Incoming Documents are sources of information in a text format that may include evidence related to financing opportunities for sustainability projects. The Incoming Documents provide information that should be evaluated with the weighted criteria defined in the Benchmark using the domain knowledge from the Room. The Room, Benchmark, and Incoming Documents are linked and utilized by an NLP system.

The Room, which represents the domain-specific knowledge base, is created using a large number of raw textual documents related to the context identified by subject-matter experts. In this study, through collaboration with professionals in sustainability project financing, we identified and incorporated over 10,000 pages of documents that provide a rich knowledge base for different aspects of the topic, including sustainability, development, and financial needs and mechanisms. In addition, subject matter experts provided Benchmarks containing a list of keywords. The keywords are categorized into three main groups: (1) keywords linked to the concept of sustainability, (2) keywords indicating the existence of development projects, and (3) keywords related to opportunities for investment and financing. The Benchmarks created in this study contained 460 keywords.

In addition, further criteria can be defined to direct the search to a specific geographical location or add other types of restrictions to the search process.

4.3. Natural Language Processing

The NLP operation begins with data preprocessing and cleaning. The preprocessing step includes two tasks: (1) removing stop words and (2) n-gramming. Stop words are common phrases and symbols that are not closely linked to the main concepts presented in a text document. For example, verbs such as “am”, “is”, “are”, and special characters, such as “?”, are excluded from the text to let the NLP system focus on phrases related to the concepts presented in the document. The n-gramming task transforms common multi-word phrases into a single-word format to help the NLP system recognize multi-word phrases that deliver a particular meaning. For example, it changes “information technology” to “information-technology”.

The first step after preprocessing is to train the NLP model to create the Room. Throughout this step, a large enough number of text documents are used to create a knowledge base that represents phrases and descriptions related to the target concept. The NLP model training utilizes the Global Vectors (GloVe) method [46] for word representation embedding. This embedding method transforms the words into multidimensional numerical vectors that quantify the co-occurrence probability between each pair of words or phrases. A significant advantage of the GloVe embedding method is its capability to keep updating the vectors when it receives new data. In other words, when new documents become available, we do not need to recreate the NLP training model from the beginning, and new data can be incorporated into the existing model to improve its performance. Using this capability, the model training procedure is an iterative process in which the model is trained using a series of text data, and its performance is assessed using a series of pre-labeled documents. The pre-labeled documents include a series of text documents with different levels of relevancy to the target concept that are labeled by subject-matter experts. Depending on the model assessment and evaluation outcomes, more input data may be fed into the training model, or benchmarks might be updated. This process is iteratively repeated until the NLP model achieves the desired accuracy and performance.

The training process is conducted in two steps. First, the model is trained using common phrases and vocabulary captured from online encyclopedias, such as Wikipedia, to prepare the model to understand the relationship between words and phrases that are repeated in many documents. For that purpose, we used a publicly available dataset [49] that contains more than 400,000 vectorized vocabulary. At the end of this step, the model is able to capture the relationship between frequent words that typically appear in many documents regardless of their topic. Second, we incorporated 2300 documents related to financing sustainability projects to train the model for the specific context that is the target of the search. These documents contain over 10,000 pages, containing more than 2.6 million words. This step eventually added more than 50,000 vectorized phrases specifically captured from documents closely related to the topic of financing sustainability projects. Figure 2 shows the steps conducted during the NLP operation.

The output of the NLP system is a score that shows the relevancy of a document to the target concept, i.e., financing sustainability projects. The score is calculated based on a cosine value between the vector of each word in the documents with each word in the Benchmark. More specifically, cos(W_i, B_j) shows the cosine between the ith word of the input document and the jth word in the Benchmark. The cosine value ranges from −1 to +1, where +1 indicates a perfect similarity, and −1 shows a complete dissimilarity. To improve the robustness of the model, the similarity of two words is only calculated for a cosine value greater than a threshold (e.g., 0.8). The similarity of a word to the Benchmark is calculated as follows:

$$sim\left(W_i,B_j\right)=\left\{\begin{array}{c}cos\left(W_i,B_j\right), cos\left(W_i,B_j\right)>T\\ 0 , elsewhere\end{array}\right.$$

(1)

where:

• T is the similarity threshold (e.g., 0.8).
• W_i is the ith word in the input document.
• B_j is the jth word in The Benchmark.
• cos(W_i, B_j) is the cosine similarity value between the vectors of W_i and B_j.

The relevancy of an input document is measured based on the weighted average of maximum similarity between each word in the input document and Benchmark, as follows:

$$R=\frac{{{\displaystyle \sum }}_{i=1}^m\underset{1\le j\le n}{\max }{S}_j\ast sim\left(W_i,B_j\right) }{{{\displaystyle \sum }}_{j=1}^n{S}_j}$$

(2)

where:

• m is the number of words in the input document.
• n is the number of words in the Benchmark.
• S_j is the importance weight for B_j.
• R is the total relevancy of the input document.

As noted before, the Benchmark consists of keywords categorized into three groups, sustainability, project, and finance, to capture all three aspects of the target concept (i.e., financing sustainability projects). To make sure that the relevancy of a document is measured from all three aspects collectively, first, we measure the relevancy of a document to each group of benchmarks separately. Next, we normalize each relevancy measure and then calculate the total relevancy as follows:

$$R_{total}=R_{sustainablity}\ast R_{project}\ast R_{finance}$$

(3)

Based on the above formulation, an input document will have a R_total greater than zero only if its relevance to sustainability (i.e., R_{sustainablity}), project (i.e., R_project), and financing (i.e., R_finance) are greater than zero.

4.4. Implementation and Outcomes

To empirically assess the performance of the proposed method, we created the envisioned AI-enhanced platform to examine its capabilities in identifying potential sustainability projects for financing via information collected on the Internet. As noted before, in the first step, a series of keywords categorized into three groups (i.e., sustainability, projects, and finance) were identified and listed by subject-matter experts. The keywords and their importance weights were updated iteratively throughout the model training process. The final list of keywords for the Benchmark included 460 phrases. It is expected that a relevant document contains phrases from all three groups of keywords (i.e., sustainability, project, and finance). The process to train the model (i.e., Room) took around 30 h using a workstation equipped with a 12th Gen. Intel Core i9 and 64 GB memory.

The developed web crawler used the duckduckgo.com search engine and its APIs for Internet exploration and exploitation. We limited the search to only pdf files, as most of the official announcements related to a project are in this format. The crawler assessed over a billion online pages using various combinations of the keywords listed in the three categories (i.e., sustainability, project, and finance) and eventually shortlisted 30,900 pdf documents. The entire process took about six hours, most of which was the imposed wait time by the search engine due to the bandwidth limitations (i.e., a two-second mandatory wait time after each search for a specific combination of keywords).

In the case of full implementation of the proposed method, the 30,900 shortlisted documents are the inputs for the text mining component. However, for the sake of feasibility in this study, we randomly selected 70 documents from the 30,900 documents and used the text mining component to analyze them. As described before, the NLP system quantifies the relevancy of each document by calculating R_total.

To evaluate the performance of the AI-enhanced platform, subject-matter experts also reviewed 70 documents and assessed them in terms of their relevancy to the topic. They assigned a score from 0 to 10, where 0 indicates not related at all, and 10 shows a perfect match. Figure 3 visualizes the correlation between the total relevancy score calculated by the NLP system (i.e., R_total) and the subject-matter experts’ score. The figure clearly shows a strong monotonic relationship between the two sets of scores. To further assess this relationship, we evaluate the rank correlation between the two variables (i.e., R_total by the NLP system and the subject-matter experts’ score) using Spearman’s rank correlation coefficient [50]. This coefficient is a nonparametric measure using a monotonic function to evaluate statistical dependence between the ranking of two variables. Spearman’s coefficient is appropriate for both continuous and discrete ordinal variables [51]. The coefficient is calculated as follows:

$$r_s=1-\frac{cov\left(R\left(X\right),R\left(Y\right)\right)}{{\sigma }_{R\left(X\right)}{\sigma }_{R\left(Y\right)}}$$

(4)

where:

• R(X) is the rank variable for the first set of observations.
• R(Y) is the rank variable for the second set of observations.
• cov(R(X),R(Y)) is the covariance Of the rank variables.
• σ_R(X) is the standard deviation of the first rank variable.
• σ_R(Y) is the standard deviation of the second rank variable.

Figure 3. Correlation between subject-matter expert and the NLP-generated scores.

Figure 3. Correlation between subject-matter expert and the NLP-generated scores.

The Spearman’s correlation coefficient ranges from −1 to +1. The coefficient will be higher when the two sets of variables have a more similar rank and lower when the two sets of variables have a more dissimilar rank.

The results indicate that Spearman’s correlation coefficient between the NLP relevancy score and the subject-matter experts’ score is 0.87 with a p-value of 4.45 × 10⁻²³. The very small p-value suggests that the correlation between the two variables is statistically significant, and the relatively high coefficient confirms the capability of the NLP system to analyze and rank the documents reasonably close to human experts.

5. Conclusions

This study proposed an AI-enhanced solution to collect and analyze information about potential sustainability projects for financing services. The primary contribution of this study to the existing body of knowledge is to develop and empirically evaluate the performance of an automated system that is able to search the Internet, analyze text data, evaluate the information quantitatively, and identify potential sustainability projects for financing services. The proposed AI-enhanced solution finds potential sources of information by crawling the Internet based on a set of predefined criteria presented in the form of keywords. It then analyzes the collected text data from potentially related websites using NLP solutions. The output of the system is a list of text documents consisting of information related to potential sustainability projects for financing services. The list of the projects is prioritized based on their collective relevance that is quantified based on the three main aspects of the target: sustainability, projects, and financing.

The outcomes of this study can transform the existing practices for identifying and evaluating potential sustainability projects that heavily depend on manual document assessments and subject-matter experts’ professional networks. Although the proposed AI-enhanced platform is not able and does not intend to replace human experts, it will help them identify potential projects more efficiently and effectively.

While the implementation and evaluation of the proposed system showed promising outcomes, the proposed solution is subject to some inherent limitations. One of the important limitations is the dependency on subject-matter experts for defining the benchmark keywords. More advanced NLP systems that are able to automatically identify keywords from relevant text documents or through large language models, such as Generative Pretrained Transformers (GPT), can be a basis for future studies. The current version of the Green-Connect platform is not able to assess the documents based on the location and timing of the potential projects. More advanced crawling and text mining systems that are able to recognize spatiotemporal information can be a target for further investigations to improve the performance of the proposed system. Another limitation of the presented solution in this study is related to its capability to remember the websites that have visited in the previous rounds. Currently, each time that the system runs, it begins the entire process again and visits all possible websites using the keywords. During full-scale implementation, experts may use the systems routinely to identify new projects. A web crawler equipped with a memory that can recognize web pages that it has visited in the past and have not updated yet can significantly improve the efficiency of the system for continuous utilization. This can be a direction for future works too. Finally, developing more advanced systems that are able to match the identified potential projects with proper financing mechanisms would be another direction for potential future studies.