Impact Assessment in Not-for-Profit Organizations: The Case of a Foundation for the Development of the Territory

Abstract

The paper describes the impact assessment method of new projects and investments in a foundation for the development of the territory based on a venture philanthropy approach. It compares the method identified with the main procedures included in the scientific literature. The paper highlights a qualitative case study carried out through three steps: (a) case study selection, (b) data collection, and (c) data analysis. Data were analyzed by three techniques: understanding the context, within-case study, and cross-case study. The result identifies an impact assessment method of new projects and investments used by a foundation for the development of the territory. It highlights a qualitative impact assessment method used for internal reporting purposes. This method is based on an ex-post evaluation with high feasibility, which allows for significant time saving. However, it does not allow for an in-depth cost analysis and presents low credibility. This assessment method can be used to justify contributions to projects and investments.

1. Introduction

The growing interest in social and environmental impact assessment represents a major opportunity to make the economic and financial sector more ethically sustainable. More and more investors are choosing to finance projects that not only provide high financial returns but also generate extra-financial benefits. As reported by the OECD report [1], the number of international investors focused on increasing social impact has risen from fewer than 50 in 1997 to more than 200 in 2017, and in 2019, assets managed by social impact investments accounted for around $228.1 trillion.

Financing more socially and environmentally conscious investments is not only a conscious choice but above all a response to the 2030 Agenda’s European Sustainable Goals, which will dictate the development agenda of cities in the short term. To this end, in recent times, academics, together with the world of finance and non-profit organizations, have begun to question themselves on two main points. Firstly, as stated by a report supported by the Rockefeller Foundation [2], it is important to assess scientifically and objectively whether the project being funded is hindering or promoting the development of a more sustainable economy following the goals of the 2030 Agenda. The second issue concerns the choice of the best project evaluation tools that can consider, at the same time, the risks, returns, and extra-financial (social and environmental) impacts generated in the territory [3,4,5]. What approach can be pursued and employed in order to convince an investor that it is worthwhile?

Social impact investment is a strategy of asset allocation, which combines financial profitability with a measurable social and environmental impact. The social impact investment is evolving in two ways. On the one hand, social impact investment activities are becoming institutionalized; on the other hand, social impact investment activities infrastructures lack any systematization [2].

The venture philanthropy approach represents a useful approach to promote social, cultural, and environmental initiatives. The term was first adopted by the Rockefeller Foundation in 1969. From the mid-to-late 1990s, the venture philanthropy approach has revolutionized grant making through various instruments [6,7]. It supports social enterprises to achieve social and economic benefits [8]. Today, the venture philanthropy is considered as a “new” organizational field and “new” professional culture [7]. OECD defines venture philanthropy as “an entrepreneurial approach to philanthropy that combines a variety of financial and non-financial resources to identify, analyze, coordinate and support self-sustaining, systemic and scalable (for and not-for-profit) solutions to development challenges aimed at achieving the greatest impact” [9], whereas the European Venture Philanthropy Association defines venture philanthropy as “a high-engagement and long-term approach whereby an investor for impact supports a social purpose organization (e.g., supports social enterprises, charities, and not-for-profit organizations) to help it maximize its social impact” [10].

Investors for impact can be actors, institutions, and organizations; for instance, foundations and social impact funds. These investors often take risks that other investors are not prepared to take in order to support innovative initiatives of social organizations. They use the venture philanthropy approach to support social purpose organizations, maximizing their social impact, and social innovations [10] that are also sponsored by central governments [11]. As a result of funding and human resources, they resolve social–environmental problems. The “final beneficiaries” are those benefitting from the activities of social purpose organizations; e.g., people in poverty, migrants, or environmental areas [10].

Venture philanthropy adopts an investment methodology that improves the reference area without neglecting the financial returns deriving from the investment. This approach puts human capital at the center through the provision of financial and non-financial support [9].

The literature suggests a venture philanthropy model based on eight distinct practices, including deal sourcing, relationship building, screening and information gathering, co-creation, early decision making, circular reasoning, deal structuring, post-investment aftercare, disengagement, and return [12]. This model highlights a hybrid model that includes elements of venture capital, developmental venture capital, and business angel investment approaches [12]. The literature also suggests performance measurement and management as key practices [10]. They allow measuring the social, economic, and environmental impact of the project and investment financed by the investor. Furthermore, they allow demonstrating the impact of actions to stakeholders. To summarize, on the one hand, performance measurement and management practices favor the internal control of projects and investments; on the other hand, they support the output and outcome reporting of activities, encouraging the dialog with the stakeholders. The main output and outcome that venture philanthropy organizations seek when selecting their investments are the social value proposition of the social enterprises, key resources, and processes of both social enterprises and venture philanthropy organizations and the synergies between them. The use of social return investment measures would support and help the field of philanthropy. Consequently, it may allow for the development of a performance measurement system of the venture philanthropy model. There is little evidence to support this assumption [13].

Since 1990, measuring organizational performance has been a great challenge [14,15,16,17,18,19], especially in public organizations and not-for-profit organizations [20,21,22,23]. Furthermore, the sustainability issue has dramatically evolved the purpose of measurement options, and leading organizations have proposed innovative sustainability reporting [16,24,25], but there are not enough common reporting standards [16]. The literature highlights that measuring the sustainable performance has to be conceptually based, but simplified, in order to be practically useful. Hubbard (20009) proposes a sustainable balanced scorecard conceptual framework, including sustainable performance measures useful for reporting to stakeholders.

Researches underline the impediments of the emerging social impact investment field [26]. According to Glanzel and Scheuerle [26], the major needs for social impact investing in Germany have been arranged along three dimensions: financial returns, social returns, and relationships and infrastructure. They also highlight a friction area between the logic of the market and civil society. These needs may be bridged through the use of impact assessment tools. While the approach to impact assessment is well known and officially recognized, less evident are the impact assessment tools that various companies and nonprofit organizations employ in their projects [27,28,29,30,31,32,33]. Due to the profoundly heterogeneous nature of investments, together with the multiple goals and desires of investors and stakeholders, there is no single, officially recognized social impact assessment tool to date. On the one hand, the multiplicity of tools can be a strength for project evaluation because they are created ad hoc for each activity; on the other hand, there are no transparent and officially recognized tools that can compare and put different experiences on the same level [2,27,28,31].

A second gap that has emerged from the analysis of the literature concerns the subject of impact assessment, which is usually focused more on the analysis of the performance of the company or its stakeholders and less on the impacts generated in the territory [2,34,35,36,37]. This is due to the greater uncertainty and difficulty in accurately measuring the impacts generated in the reference context. Nevertheless, the risk is to lose many of the benefits that the project can generate indirectly.

Adopting a qualitative case study methodology, this paper, which may be considered a “technical report”, aims to describe how an Italian not-for-profit organization assesses the social and economic impact of new projects and investments. To contribute to the above research gaps, the paper answers the following research question:

“How do the Foundations for the development of the territory based on a venture philanthropy approach assess the impact of new projects and investments?”

The Section 2 introduces the materials and methods used for this research. Section 3 highlights the main findings concerning the context and features of the case study related to the method used for impact evaluation by a not-for-profit organization. Section 4 bridges the gap between theory and practice. Finally, Section 5 identifies any implications for research, practice, and society.

2. Materials and Methods

The research used a qualitative case study methodology which favors detailed research of a specific subject [38,39]. This methodology is commonly adopted in business, social, and educational research [40]. It involves qualitative methods for describing, comparing, and evaluating different aspects of a research problem [41]. A qualitative case study methodology was selected due to the fact that it is recognized as a useful research design to gain concrete, contextual, and in-depth knowledge about a specific problem. It allows identifying the characteristics of a specific context through an in-depth investigation [38,42].

This paper presents a qualitative case study based on real-time investigations in order to describe the impact assessment of new projects and investments to a foundation focused on the development of the territory.

The main research steps were: (a) case study selection, (b) data collection, and (c) data analysis.

(a) Case study selection. The first research step defines the features of the research population from which the research sample was drawn [38,43]. The features of this organization were to be a not-for-profit organization—NPO, and to be a foundation focused on the development of the territory. The case studies were selected from a pool of well-known leading NPOs [38]. To ensure anonymity, the selection organization was called here “the XXX Foundation”.

(b) Data collection. A large amount of empirical data was collected through different methods. Four researchers interviewed the chief executive officer, director, control managers, and human resources managers. The methods used were:

• Semi-structured interviews with a cross-section of key employees. Four researchers carried out 8 semi-structured interviews with five people [38], i.e., director, president, control manager, project manager, and administrative manager. The semi-structured interviews were based on open questions to respond to the research question, i.e., “How do the Foundations for the development of the territory assess the impact of new projects and investments?” To better understand their evaluation, researchers asked what the evaluation does, what tools, procedures, and/or methods it adopts, what the assessment purpose is, etc;
• Company documents, such as financial statements, project management reports, research papers, videos, conference proceedings, and official websites. The researchers collected 23 documents;
• Direct observations of the management practices.

As per Yin’s (2018) recommendation, data triangulation is important to strengthen its validity. As a result, all three sources of data collection were used to favor data triangulation. The identified information was the basis for developing the data analysis and highlights how the foundation assesses the socioeconomic impact of new projects and investments.

(c) Data analysis. As already used by other empirical researches [18,40], data were analyzed through:

• Pre-understanding the context. This analysis aims at highlighting the main information about the case study, including governance information, asset management, and main activities [38];
• Within-case study. This analysis aims at searching the explanations and causality within the case study and identifying the characteristics of the emerging evaluation model [38,43,44]. This analysis is conducted through an accurate description of the cases that highlight the main features of their impact assessment system [38];
• Cross-case study. This analysis compares the methods known in the literature with the method used by the XXX Foundation. It adopts the criteria used to define the characteristics of the assessment methods in order to compare these methods. The adoption of these characteristics supports the development of the cross-analysis and the identification of the new features, with respect to the well-known assessment methods. This analysis improves the internal validity and favors generalizing research results [38]. Furthermore, it supports the model creation deriving from the research [43,45].

3. Results

This section provides a concise and precise description of the results.

3.1. Pre-Understanding the Context

Governance information. The XXX Foundation was created by a founder (Banking Group) in 2007 who has allocated the assets for specific purposes, i.e., it aims to develop its founder’s territory, to which its institutional purposes are linked. The XXX Foundation is a not-for-profit organization recognized as a foundation for the development of the territory. The foundation governance is based on its current statute. It defines the following bodies and positions:

• The Board of Directors is made up of seven members with experience in the not-for-profit sector and the field of venture philanthropy. The board of directors remains in office for four years and its members can be confirmed. The board elects a president and two vice presidents. The board is responsible for the administration of the foundation, except for the tasks attributed by the law and/or the statute to the founder;
• The President chairs the board of directors; he/she is the legal representative of the foundation. He/she holds the functions delegated by the board of directors;
• The Board of Auditors is made up of three members and two alternates appointed by the founder, who chooses them from among those enrolled in the register of statutory auditors. The mandate lasts four years and can be reconfirmed once. The board is the supervisory body of the foundation; it checks the financial management, ascertains the regular keeping of the accounting records, and approves the financial statements. The board of auditors carries out the legal control of the accounts required by law;
• The Secretary-General heads the offices and staff of the foundation. The general secretary is appointed by the board of directors, with a term of office equal to that of the board of directors;
• The Treasury Committee has the task of researching and analyzing the most efficient solutions for asset management.

The XXX Foundation operates mainly through a venture philanthropy approach, utilizing a variety of financial instruments managed by professional operators (preferably real estate and/or securities investment funds and, depending on the type of investment, bonds and equity investments, including a majority, in dedicated companies). The XXX Foundation also pursues its aims through: (a) instrumental companies, of which it can also hold the entire share capital, and (b) public–private partnership operations, consistent with the achievement of its statutory purposes.

Asset management. The year 2020 represents the 12th year of activity of the XXX Foundation. In 2020, it continued the initiatives already initiated and undertook new investment initiatives to support local development. The main project regards the start-up of a multi-year project that aims to create an international development center for local companies. In the last year, the XXX Foundation paid attention to the integration with the founder’s programs; it evaluates the activities and can capture significant synergies to both actors. The XXX Foundation carries out its institutional interventions mainly through the contributions deliberated in its favor by the founder. The resources approved by the founder of the XXX Foundation amounted to EUR 40 million in the last financial year. The XXX Foundation has a low-risk profile concerning financial investments; its investment portfolio consists of government bonds and investment funds with a book value of approximately EUR 6 million.

Institutional activity. In order to achieve its objectives, the XXX Foundation can, either alone or in collaboration with other public and private institutions, carry out activities such as:

• Scientific and technological dissemination and transfer;
• People talent enhancement;
• Landscape, artistic, cultural, and food and wine heritage enhancement.

The projects and investments of the XXX Foundation regard the following areas:

• Social public housing—properties for collective social use: these projects and investments areas are characterized by medium/long-term real estate investments, which are to be made through a closed real estate fund with social purposes. The interventions concern the retirement residence, nursery schools, libraries, museums, and other initiatives related to the development of human capital through culture and training. It also operates on goods of collective interest, such as goods of local authorities and institutions for public utility purposes;
• Social private housing—properties for private use: these projects and investments are focused on the construction, renovation, and functional recovery of buildings for residential use, to be allocated to socially and/or economically weak categories. It aims at providing a concrete response to housing problems by creating good quality structures and services at controlled costs;
• Territorial development: this area is characterized by the attention paid to the issues of the environment, life quality, economic and social development, renewable energy, and services to citizens;
• Entrepreneurial development: this area is characterized by the initiatives to promote technology transfer also at the international level, the establishment of new businesses, the growth of small and medium enterprises, and the development of managerial skills, also in connection with other initiatives at a regional and national level.

3.2. Within-Case Study

The XXX Foundation independently assesses the impact of the projects that follow directly. It has implemented an impact assessment system to report and demonstrate to investors the results that the activities generate at an economic and social level. The foundation’s staff evaluate the projects and investments through a specific form. Below, (a) the main information collected (

Table 1. Main information on projects and investments.

Project Information
Project title
Coordinating organization
Project supervisor
Sector: profit or not-for-profit
Project budget   1. Available funds   2. Resources required
Thematic areas:   Socal and human purpose   Social housing   Territorial development   Entrepreneurial development
Project description
Market and competitors
Entrepreneurial team background
Type of investment
Purpose of the project and consistency with the foundation’s statute
Strengths and opportunities of the project/investment
Weaknesses and risk points of the project/investment
Further information

) and (b) the assessment grid used by the XXX Foundation are shown (

Table 2. Project and investment assessment grid.

Criteria	Description	Score	Weight
Business model			10%
Market area			10%
Team			10%
Governance			10%
Organizational structure			10%
Financial aspects			10%
Income aspects			10%
Reputational risks			10%
Project stakeholders			5%
Project outcome—Social impact			5%
Project outcome—Employment impact			5%
Project outcome—Indirect impact			5%
Total score			100%

).

The impact assessment system adopted by the XXX Foundation is easy to compile and does not require any particular statistical or economic and financial analysis. The adopted evaluation of the XXX Foundation is similar to other assessment methods described in the literature; to compare their method with other procedures, it is necessary to identify the main characteristics of the evaluation system used by the foundation.

First, the system used by the foundation distinguishes between for-profit and not-for-profit organizations and activities ex-post. Secondly, the purpose of their analysis is to evaluate the economic and financial performance of the activities and projects from a qualitative point of view. Thirdly, the method used by the foundation is simple, clear, and exhaustive; the judgments are taken through a subjective evaluation based on the know-how of the team members. Finally, the last feature is given by the evaluation output: the final result is represented by a number that allows for an easy comparison with other impact assessments performed on other activities and projects.

3.3. Cross-Case Study

The cross-case analysis compares the XXX Foundation’s impact assessment with other main impact assessment methods known in scientific literature. In order to compare this impact assessment,

Table 3. The criteria used to define the methods’ characteristics.

Criteria	Characteristics
A. Methods	Ex-post evaluation Ex-ante evaluation Ongoing evaluation
B. Purpose	Internal reporting Constructive evaluation to implement available policies
C. Costs	Low Medium-Low Medium-High High
D. Feasibility	Low Medium-Low Medium-High High
E. Credibility	Low Medium-Low Medium-High High
F. Measurement method	Qualitative Experimental and non-experimental quantity
G. Investigation perspective	Internal evaluation Internal and external evaluation impacts
H. Approach for indicators	Differentiated Undifferentiated
I. Category	Economic Social Environmental

highlights the criteria used to define the characteristics of the assessment methods [27,29,34,35,36,37,46,47,48,49,50,51] shown in

Table 4. Cross-case analysis of the assessment methods’ criteria and characteristics reported in Table 3.

	Criteria & Characteristics
Assessment Method	Methodology	Purpose	Cost	Feasibility	Credibility	Measurement Method	Survey Perspective	Approach for Indicators	Category
	A	B	C	D	E	F	G	H	I
Balanced scorecard	1	1, 2	2	3	3	1, 3	1, 2	1	1, 2
AtKisson compass assessment	1, 3	1	3	4	2	1, 3	1, 2	2	1, 2, 3
B rating system	1	1, 1	2	4	2	1, 2	1, 2	2	1, 2, 3
Global reporting initiatives	1, 3	1	2	4	3	1	1, 2	2	1, 2, 3
Impact assessment framework	1	1, 2	1	4	2	1, 3	1, 2	2	1, 2, 3
Volunteering impact assessment	1	1, 2	2	4	2	1	1	2	1, 2, 3
AA1000 Assurance Standard	1	1	2	3	3	1	1	2	1, 2, 3
Multicriteria decision analysis	1	1, 2	2	3	3	1, 2	1	2	1, 2, 3
Measuring impact framework	1	1, 2	3	3	4	1, 2, 3	1, 2	1, 2	1, 2, 3
Public value scorecard	1	1, 2	2	3	3	1, 3	1, 2	1	1, 2
SCALERS	1	1, 2	1	3	2	1, 3	1	1	1, 2
Triple bottom line scorecard	1	1, 2	2	3	3	1, 3	1, 2	1	1, 2
XXX Foundation	1	1	1	4	2	1	1, 2	1	1

.

The cross-case analysis highlights the main features of the XXX Foundation’s impact assessment. In order to compare the main features of the XXX Foundation’s impact assessment, this analysis also compares 11 assessment methods [48,52] (Table 4).

The XXX Foundation presents an ex-post method with internal reporting purposes that is relatively little focused on costs. The methodology is highly feasible and has low to medium credibility. As defined by the literature [11], the feasibility and credibility variables are usually inversely proportional: the more feasible and easier an evaluation procedure is to carry out, the lower its external credibility will be, because it is assumed that the approach employed is too simplistic and not very objective. The measurement methodology has a qualitative approach oriented both to the internal and external evaluation of the benefits and economic impacts of projects and investments.

4. Discussion

Most of the methodologies identified are ex-post for internal reporting purposes. As for the XXX Foundation, evaluation procedures are used to justify contributions to projects and investments. The “costs” criterion is low-medium; this poor application is due to the type of qualitative analysis implemented. As for the XXX Foundation, the analysis procedures involve judgments through the use of a scale of values; therefore, statistical analyses or specific quantitative indicators are not necessary. A low-cost analysis corresponds to high feasibility because the judgments based on the variables shown in Table 4 can be given to most people who know the object of the evaluation. For this reason, the credibility criterion is usually low: the stakeholders and people external to the evaluation recognize that this evaluation system based on a subjective formulation of judgments can be rather generic; it does not fully highlight all the positive aspects and negatives of the activity that an expert evaluator can highlight with more in-depth and detailed analyses. The survey perspective is mostly internal and, consequently, the approaches to the indicators are, for the most part, undifferentiated: the ultimate goal of the evaluation in these cases is the comparison between various projects with the need to have an objective value in order to be used. Finally, the category investigated is usually at an economic, social, and sometimes also environmental level. In contrast, for the foundation, the main aspects refer to the economic impacts generated by the activity.

The comparison between methods identified highlights the strengths and weaknesses of the impact assessment method used by the XXX Foundation.

The assessment method adopted by the XXX Foundation is cheap, quick, simple, and, thanks to the use of undifferentiated indicators, the results are easily comparable; however, some critical issues are observed. Firstly, the credibility of such instruments is medium-low, especially at the international level. From the perspective of potential investors, it is important to access adequate and detailed information on the projects and activities to be financed; currently, the valuation table provides only a summary, and sometimes insufficient indications for investors. Secondly, the indicators used by the XXX Foundation analyze the aspects related to the performance of both the activity and economic and financial issues, leaving out the positive and negative effects, direct and indirect, that the activities generate at social and territorial levels. For example, the AtKisson Storecard method is similar to that employed by the foundation, but at the same time, is more comprehensive. This method analyzes four key points related to sustainability: nature, well-being, society, and economy. Starting from these points, numerous sub-categories have been analyzed that detail the sustainability of the project. As the multi-criteria evaluation methodologies, the foundation adopts the use of different weights for each variable in order to customize the evaluation system while using the same indicators as the other analyses.

In order to better support the consultants and practitioners on how to learn from this extensive assessment of social enterprises and non-profit organizations, the paper summarizes the main steps followed by the XXX Foundation to assess the new projects and investments (Figure 1):

• Step 1: Data collection on new projects and investments (Table 1);
• Step 2: Initial project and investment assessment (based on a grid—Table 2);
• Step 3: Ranking and financing projects and investments;
• Step 4: Middle project—investment assessment (based on a grid—Table 2);
• Step 5: Final project—investment assessment (based on a grid—Table 2).

5. Conclusions

The paper highlights the impact assessment method of new projects and investments adopted by a foundation for the development of the territory; it has been identified through a qualitative case study. Furthermore, the paper compares the method identified with the comparable procedures included in the scientific literature.

The findings highlight a qualitative impact assessment method for internal reporting purposes. It is based on ex-post evaluation with high feasibility. In contrast, it is not based on in-depth cost analysis and minimum credibility. This assessment method has been mainly developed to justify the financing of projects and investments. It introduces a method that allows time-saving, which is simple and easy to apply. Furthermore, the results of its application are easily comparable.

The main research limitation is that only one foundation was investigated. However, its strength is that it permits an in-depth understanding of the assessment method adopted by the XXX Foundation and compares it with other assessment methods included in the literature.

Findings give opportunities for future research through more case studies on this research question; it also encourages testing, validating, or improving this assessment method. Further research is needed to explain how impact assessment methods are changing in foundations for the development of the territory.

This research gives academic and practical contributions in the form of a longitudinal, qualitative, and in-depth assessment of the impact assessment method adopted by leading foundations. The academic contributions define the understanding of the assessment method of social and economic impacts in a foundation for the development of the territory, according to the main method known in the literature. The practical contributions highlight an economical, quick, and user-friendly assessment method. These findings will hopefully lead to the design of innovative and simple assessment methods in foundations. However, the main result of the research is the recognition of the increasing role of customized assessment activities in order to respond to the founders’ needs.

Although the results of the research presents an overly simplistic system, it may be considered a springboard for further research of this key topic for venture philanthropy applied to a foundation for the development of the territory. Furthermore, it may be the first stage to understand the assessment practice realty adopted by this typology of foundation.