Ambiguity in the Attribution of Social Impact: A Study of the Difficulties of Calculating Filter Coefficients in the SROI Method

Marta Solórzano-García; Julio Navío-Marco; Luis Manuel Ruiz-Gómez

doi:10.3390/su11020386

Abstract

In order to analyse, manage, and compare social projects we need, among other things, to be able to measure their impact. One of the methodologies currently used to measure and manage social impact is Social Return on Investment (SROI). However, not all the results calculated by the SROI method are directly attributable to the project, and, therefore, to determine the real impact it is necessary to filter out the changes that the project has not produced. Filter coefficients perform this function. However, the theoretical logic on which the chain is constructed that converts the outputs into impacts is ambiguous. In this study, we will analyse twenty-five real cases where SROI was used to measure social projects. We will identify the difficulties of isolating and measuring impacts by performing a comparative study of the procedures that entities develop to calculate the filters. This allows us to calculate the impacts from the outputs. We will then propose the improvements needed to overcome these shortcomings.

Keywords:

social return on investment (SROI); social impact measurement; accountability

1. Introduction

In recent years, Third Sector of Social Action organisations are facing ever-increasing demands for information about the social and environmental impact they generate. The various groups affected by the activity of these organisations, including investors, consumers, funders, employees and the public administration, are aware that in order to assess entities in the Third Sector of Social Action (TSSA from now on), they need information that provides a holistic picture of their activity that is not limited to financial indicators. These interested parties have started to put pressure on the entities to disclose information about their social and environmental performance [1,2].

The various interest groups of TSSA organisations can exercise this control in a formal and direct way, through evaluation requirements included in contracts, or indirectly, through rules that individuals and organisations must follow in order to gain approval and the verification that their activity meets the established requirements [3]. Therefore, TSSA organisations are conditioned by the influence of the logics and rules of other groups [4].

These demands have led to an increase in sustainability reports by TSSA organisations. Therefore, does this increase mean an improvement in the transparency of TSSA entities? Previous research has concluded that the content and depth of sustainability reports varies significantly between entities [5]. There are even studies that reflect a lack of quality in these reports [6,7,8,9].

There is a need for a standardised methodology for monitoring and evaluation that integrates, in plain and easily understood language, the internal information needs for learning and development in the organisation, with the information demanded by external interest groups to account for their performance [10,11,12].

One of the external groups that applies the most pressure is the funders [13], meaning that in order to obtain the necessary resources to develop their activity, TSSA organisations need to communicate and account for what they do and how they do it [14,15]. It is not enough to show information that is favourable to the organisation or project; the institutional pressures exerted on the third sector mean that the demand for transparency is very high [15,16,17].

In parallel, and at an internal level, these entities, in addition to meeting the economic, social and environmental objectives required of them, must prepare, process and communicate a large quantity of information that meets specific requirements, both to practise effective management, and to control their results and possible deviations [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18].

This explains the relevance of the dependence of TSSA entities on their main financial backers, and how this dependence entails providing information to backers on their activities and results in a particular format. Increasingly, funders require formal methodologies for measuring social impact that generate objective metrics with an empirical basis, with the aim of improving the standardisation, verifiability and accountability of these metrics [19,20,21,22].

However, the measurement of social impact is an extensively questioned practice, owing to, among other reasons, the ambiguity that methodologies for measuring impact suffer from. A frequent object of criticism is the lack of logic of the underlying assumption within most methodologies of a causal chain or logic model through which the activities of the entity are translated into results, changes and impacts for each of the stakeholders [2,23]. In contrast, practitioners with experience in implementing such formal methodologies often stress the causal ambiguity of this chain; they contend that impacts are difficult to understand with precision, much less calculate [2,24,25].

One of the methodologies currently used to measure and manage social impact is the Social Return on Investment method, or SROI, which, based on traditional cost-benefit analysis, quantifies social value through indicators associated with the results achieved. SROI is a process of understanding, measurement, management and communication of the social, environmental and economic values generated by an organisation. Its purpose is to examine, qualitatively and quantitatively, the project’s process of generating social value.

Where impact is caused by a range of factors, it is difficult to determine the amount of social value that can be attributed to the activity of the project that is being measured. To cope with this and determine the real impact of the project, filter coefficients are introduced. However, their estimation has some shortcomings [4,26,27]. Even though the filters are perfectly defined within the SROI methodology, when we fix them at a certain percentage, we cannot find a method or reference that enables these filters to be calculated with precision and ease [28,29,30]. We can see that these coefficients have a different accuracy depending on the person who conducts the study of measurement, their interests and ability to access the information. Since the function of these filter indexes is the correction of the result to reach the impact, their effect on the impact is key. This is why improving the construction of the coefficients that filter the outcomes to calculate the impact is essential for accuracy in the measurement of social value.

In this study we investigate the following research questions: what are the difficulties in calculating the filter coefficients of the SROI and how to develop the necessary improvements to overcome these difficulties. We explore these questions through a comparative case of real SROI analysis of organisations from different sectors and geographical areas.

This article critiques the legitimacy of the SROI method to measure the social impact created by organisations, calling into question the underlying theoretical logic between outputs and impacts. While outputs can be managed by organisations, exogenous factors that escape their control influence results and impacts. Therefore, the attribution of results to a particular actor can be very difficult [2,24,31]. Through the analysis of the reports of SROI measurement of twenty-three social projects, and the identification of the gaps in the underlying theoretical logic between outputs and impacts, we will study the calculation of the filter coefficients of social impact and identify their shortcomings.

We will consider an impact to be those results that can be attributed directly to the organisation or project in question. To arrive at this concept, it is necessary to filter the changes, taking away that which has not been produced by this project, so that the organisation only claims what it has created itself as a social impact. To do this, we must introduce these elements: deadweight, displacement, attribution and drop-off [27] (p. 54).

Deadweight deals with quantifying whether the project has really influenced society or whether, on the contrary, the result we see is a product of other external interventions. In other words, this coefficient responds to the question of the extent to which the result would have been achieved if the project being evaluated had not taken place [27] (p. 56). Displacement studies what percentage of the change achieved has displaced other changes. If the project consisted in helping people at risk of social exclusion to get a job, displacement would seek to quantify whether these jobs were obtained to the detriment of another worker. If so, the quantification of this circumstance has to be removed, which will diminish the impact of our initiative [27] (p. 57). Attribution measures the percentage of changes that is not attributable to carrying out the project. It indicates the need to deduct, in the calculation of the impact, the percentage of the result that has been caused by the contribution of other organisations or people [27] (p. 59). Drop-off is the decline in the results of a project over time, which could be compared to the depreciation on a fixed asset due to use. In other words, this coefficient analyses how long the results have lasted [27] (p. 61). It is logical to think that in the years after completing the project, the quantity of the result obtained will diminish, and therefore the attribution to this project will diminish [18].

The next section outlines the methodology that was followed throughout the study. The third section is a comparative report of the cases studied during the research. The fourth and last section identifies the deficiencies detected in the study. This is the central body of our research because it allows us, through the above analysis, to draw conclusions on the results. We will warn of the shortcomings that must be addressed in order to eliminate the ambiguity of the causal chain of logic between outputs and impacts, and also provide a measurement that contains the real value that a project generates, integrating both social and economic impacts to give a result that can be standardised, verifiable, accountable, comparable and understandable to others.

2. Materials and Methods

An ISI web of knowledge and Scopus online literature search was conducted for the years 2007–2016 for articles and conference papers on SROI analysis. Key words used included: “SROI”, “Social return on investment” in the subject area “Social Sciences”. We searched in all fields of the articles using the above-mentioned key words separately on the date of 31st October 2016. We obtained the following results: 23 results for “SROI”; 36 results for “Social Return on Investment”.

In addition, we conducted a search by entering the above terms in “Google Scholar” and the documents, reports, book chapters, articles and practical guides were also checked, and relevant documents included in the review. We decided to include non-academic material in the study material, as SROI is a methodology that is debated more frequently in professional environments than in academic ones. Where relevant to the discussion on suitable theories, articles and papers based on the methodological approach of literature review [32,33,34,35] were also added, given that the empirical studies and surveys are still limited, especially in the scientific literature.

The most relevant documents were examined and on that basis we chose the most important ones for the study. In particular, with reference to the selection process, we verified the presence of a consideration of the filter coefficients of the SROI method and its importance for the objective of the analysis. In total, twenty-three case studies from this set have been analysed to identify possible irregularities in the calculation of the filter coefficients:

A program aimed at combating loneliness and isolation of the elderly in Scotland.
Conservation of the natural environment surrounding the Greenlink area in Scotland.
A project providing access to water to families who do not have access to other water sources in Stockholm.
Promotion of healthy lifestyle habits for overweight people in Bristol.
The Solvatten project provides household water treatment in developing countries (Kenya).
Benefits of interaction with the rural environment for people with problems of socialisation or behaviour in England.
Benefits of Tai Chi, in the forest, in patients of the Hospital Firth Valley in Scotland.
A rehabilitation program based on an intensive program of activities for people with bone marrow injury in Australia.
A programme for the development of adolescent women in areas where unemployment, poverty and lack of education are rife in Northern Ireland.
A program that informs local women about health issues in an informal way in Northern Ireland.
A project to provide a minimum income for rural families living in semi-arid areas in Ghana.
A project to support, through job creation, people with mental health problems in Glasgow.
Helping parents who care for children with disabilities develop their skills, endurance, and confidence in the UK.
Make organically grown food accessible to the citizens of Sydney.
Use art as a catalyst to bring about positive and lasting change in people’s lives in North Ayrshire.
A family intervention project that works with families with children and in situations of high vulnerability in Northamptonshire (England).
Evaluation of the social impact of a “social initiative protected employment centre”.
A project to create a cafeteria for 30 elderly or disabled persons who are provided with transportation to the cafeteria in England.
The Oxford Castle Renovation Project.
Respond to complaints about antisocial behaviour in the area caused by the increasing volume of unsupervised and unauthorised bicycles in Scotland.
Restoration of gardens, with the participation of volunteers from the municipality and students in Scotland.
Use of a natural space for outdoor activities with the mentally ill in Scotland.
Evaluation of the social return of investment in the creation of the Ecoclub school in Scotland.

The cases belong to different sectors and geographical areas to avoid the inaccuracies that the sample selection bias could generate in the conclusions. They are briefly described in Appendix A. Through a comparative study of the procedures followed to calculate the coefficients in each one of the projects, we have identified the shortcomings of SROI methodology in this area. The study ends with a proposal for improving the construction of these filter coefficients that make up for the shortcomings identified and respond to the needs for accuracy in the measurement.

3. Results

Before the SROI calculation can be finalised a decision has to be made as to how long the changes produced by the analysed project will last. In an SROI analysis the length of time changes endure is considered so that their future value can be assessed. The question to be answered is “if the activity stopped tomorrow, how much of the value would still be there?” This is dealt with by assuming that the value will reduce or drop off a percentage each year. As well as considering how long changes will last, it is necessary to take account of other factors that may also have impacts. The recorded change might have happened regardless of the activity, something else may have made a contribution to it or the activity may have displaced changes taking place elsewhere. In considering the influence these factors exert, a realistic approach should be adopted. The aim is to be pragmatic about the benefits actually provided by the project and to recognise that the value it creates is influenced by other factors. The SROI methodology does this by reducing the value calculated for each outcome on a percentage basis, applying several filter coefficients: deadweight, attribution and displacement.

A reduction for deadweight reflects the fact that a proportion of an outcome might have happened without any intervention. Attribution recognises that external factors, or the contribution of others, may have a part to play in the changes that are identified and displacement applies when one outcome is achieved but at the expense of another outcome, or another stakeholder is adversely affected.

This section presents an analysis that compares each of the filter coefficients of the social projects that, when analysed, produced the most significant results. There are four tables in total: deadweight (Table 1), displacement (Table 2), attribution (Table 3) and drop-off (Table 4). Our objective is to make the study of the various ways of calculating these coefficients more visual, and to allow for a more exhaustive comparison.

Table 1. Deadweight.

Table 2. Displacement.

Table 3. Attribution.

Table 4. Drop-off.

4. Discussion

The analysis allows us to observe that the main shortcomings of the filter coefficients are related to the heterogeneity in the estimation. Even though all the cases studied share the same conception of these filters, the way in which the percentage is obtained in each case is very different, as summarised in Table 5.

Table 5. Filters coefficients comparison (I: Collection of information C: Calculation of filters).

One of the obstacles we encountered is that the influence of the subjectivity of the people responsible for the identification and subsequent quantification of information is very high. The estimates are approximations, which calls into question the accuracy of the evaluation of impact. Accuracy in measurement is one of the aims of the social impact evaluations. However, the estimates of the filter coefficients involve a wide margin of error and can produce erroneous interpretations and uses of the ratios.

In most of the cases considered in this analysis, the information that is used to make these estimates comes from interviews with the project’s interest groups. In these interviews, which normally use questionnaires, the participants report on an individual basis, in accordance with their personal assessment. Therefore, the information that is incorporated into the measurement is fundamentally based on value judgements and not on objective data.

The calculation of the coefficients also supports various possibilities. Some cases associate, to each result for each interest group, a percentage of deadweight; other cases associate a percentage for all the results of each group, and others make a general estimate of the whole project. However, the majority of the cases arrive at these percentages through logical deduction or reasoning, which generates an added difficulty, since this reasoning does not always follow the same pattern.

If the quantity of the filter coefficients is high, it is assumed that the project is not of great benefit on its own, which is why, in the majority of cases, analysts set small percentages. Several of the cases studied allocate a deadweight figure (Table 1 and Table 5) of under 10%; one of them (case 5) even considers that there is no deadweight for any of its results. This means that the project recognises that without its activity, none of the results that participants have experienced would have been achieved. On the other hand, given the absence of certainty, some projects try to avoid this problem and establish a representative percentage. Cases 6 and 11, for example, use this methodology. Some projects consider that the changes observed would not have been possible if the project had not taken place, but assign a 5% deadweight figure to avoid an overestimation of the impact.

Other cases follow a different methodology to perform these calculations and establish references to comparison groups or benchmarks. For example, case 2 bases its estimates on official surveys. In this case, the calculations offer greater objectivity, since the same method could be applied for similar programs. This would make such projects directly comparable with each other.

In the analysis of deadweight there is another estimation system that consists of calculating an average of the information obtained in individual interviews with the interest groups (case 15 as example). That is to say, the individual results are added up and then divided by the number of participants. This system is based on the assumption that the groups that participate in the same programme tend to have similar personality profiles, which is why, to avoid errors of subjectivity, an average of all the information obtained is taken.

Another important element in the calculation of impact is displacement (Table 2 and Table 5). Again, the main shortcoming that this coefficient presents is that it is based on estimates. Most of the projects do not take into account the displacement that their activity can generate. This claim is based on the idea that from among the twenty-three cases analysed, only twelve of them consider a certain percentage of this indicator, while the remaining eleven do not identify any type of displacement.

The fact that this coefficient does not exist in most of the projects is a limitation to homogenizing the calculation of impact. Of the cases that contemplate the existence of displacement, a small portion of them consider its effects to be minimal, and use a percentage of below 20%. Cases 2 and 4 go over a 50% displacement rate in some of their results; however, their estimate is based on the discussions held with participating agents. Considering whether a project displaces its results to other activities is not very complex; the problem lies in what percentage it is best to assign to this displacement.

Thirdly, we analysed attribution (Table 3 and Table 5). All the cases studied allocated a percentage for each of the results, and estimates in the twenty-three cases were based on the declarations of the interest groups.

The last correction factor is drop-off (Table 4 and Table 5), although some projects do not regard it as such. The drop-off indicator is used to record whether the result obtained is maintained, or not, over time, and is only calculated for results that last for more than a year. In the study we observe a number of different procedures to calculate it. One of them is to deduct a fixed percentage each year after the completion of the project. Another procedure used is to arrange an interview with the interest groups some time after the completion of the project. However, this system can imply a self-selection bias because the people who will be willing to be interviewed are likely to be the people who are the most satisfied with the results of the project.

Within the cases analysed, we would like to highlight, for their originality, cases 8 and 14. Both cases unify their assessment criterion of the filter coefficients, dividing each result into five categories. In this way they get the interest groups to identify more clearly which percentage of deadweight, displacement, attribution or drop-off is more in line with the results that they perceive.

5. Conclusions

In our analysis we have highlighted some of the difficulties in standardising calculations of filter coefficients, since the social benefits generated by projects are primarily intangible, and difficult to estimate. The main objective of this study has been to reflect the differences in unifying criteria, the shortcomings of the methodology and the difficulty in achieving more consistent estimates. If we could obtain more homogeneous filter coefficients, it would be possible to compare the return on investment of several social projects and know which of them generates greater value.

Along these lines, we perceive clearly that an effort is required to homogenise the calculation of filter coefficients in order to: (1) Minimise, as far as possible, the degree of subjectivity that they support; (2) reinforce their quantitative character against other qualitative interpretations; (3) provide them with a universal character; by standardising them they can be used in a general way and similarly in other projects and (4) incorporate the time value in their estimation and analysis to check their development, make comparisons over time and corroborate their diagnosis. In this way they would become even more useful tools for making financial and social decisions.

In view of the results obtained, we propose some ways forward or recommendations that, as well as improving measurement and evaluation, constitute new avenues of research, to continue refining and improving these coefficients and thus obtain more valuable measurements:

(1): Create evaluation bands or quantification stages, forcing the filters to take discrete values (in bands or stages). A good example is case 8, which, as we have seen, structures its percentages on a scale that goes from lowest to highest in terms of the influence of the project on the result obtained. By establishing standard bands, we can reduce variation and subjectivity and get more easily comparable values. Intermediate values can be included within the bands, establishing the characteristic of the band, and adjusting once within it.
(2): Stipulate some standard or average values for certain common or standard situations. This would help spread uniform and unambiguous criteria to be used in the various analyses. For example, adjust deadweight to a certain percentage when dealing with individual exceptions (case 9 is a good example, where individual recoveries, not attributable to the project, were given a deadweight figure of 5%).
(3): Identify comparison populations, like in case 11, where a comparison group or control group was introduced. The data collected for these groups represent the deadweight for the SROI. It was a set of stakeholders outside the project’s area of influence, since they did not participate in the project’s activities, but had equivalent characteristics.
(4): Homogenise, as far as possible, the timeline for the project. Projects with a duration of under six months make it very difficult to quantify the coefficients appropriately. Durations of over three years are equally difficult to quantify, given the high probability of exogenous events and spurious variations beyond the control of the analysis. Setting durations or timelines for analysis of 6, 12, 24 and 36 months (allowing intermediate periods of 6 months) seems a sensible approach, with time periods that allow results to be compared and analysed better. In this way we will get filters with more accurate and comparable results, and in particular, the calculation of the drop-off will have much greater value and significance.
(5): Identify reference estimates, benchmarks, of universal validity as the basis for calculating the filters. This happened in case 2, where to filter the result of the project for the volunteers, they used the rate of volunteers in North Lanarkshire as a deadweight percentage according to a survey of households in Scotland.
(6): Define standard rules in the valuation of the coefficients of certain interest groups. For example, groups that provide funding. Establishing certain rules in the quantification of the filters and their importance in these groups, not being the main stakeholders, can facilitate their homogeneous, common quantification.

Applying these recommendations to one of the analysed cases, the Craft Café case, results that allow a proper comparation with to other SROI cases can be obtained. The recommendations guarantee a unified assessment criterion of the filter coefficients, and the percentages of deadweight, displacement, attribution or drop-off are more in line with other analysis, as detailed in Table 6. In any case, this re-evaluation has only illustrative purposes, as the case does not provide all the information from the interviews and the whole process should be re-done to guarantee the accuracy of the re-evaluation. In our analysis we have taken an in-depth look at filter coefficients in the calculation of SROI through an exhaustive review of twenty-three relevant projects in the social sphere.

Table 6. Craft Café case. Re-evaluation.

For the drop-off, the case recognised that they do not have research data available to establish the drop-off rate. Therefore, they have assumed a drop off percentage of 33% for this SROI analysis. The drop-off should be re-evaluated as 50%. Regarding the displacement, there is no evidence for this that justifies an inclusion of displacement in the impact results. As conclusion. the level of standardisation in the evaluation has been increased.

By virtue of its analysis, this article is a contribution to research in this field, and provides different contributions for both scholars and practitioners. Specifically, we have managed to: (1) make known the cases themselves, presenting a panoply of examples of successful social projects that can serve as models of good practice and inspiration for starting similar projects promoting the social economy; (2) study these projects in a systematic way, considering the methodology, results and study of the filter coefficients, in a concise and accessible way; (3) systematise the review of the calculation and comparison of the filter coefficients, allowing a comprehensive view, through the summary and analysis of these results; (4) observe some relevant facts and significant trends in the form and results of the calculation of these filter coefficients, identifying gaps and problems in their evaluation, and, finally, (5) suggest, in view of the situation observed, some recommendations and improvements in the road towards homogenising the method of calculation of the filters, as a necessary step for their implementation and success, which in turn open up new avenues of research.

The quantitative assessment of social impact, despite the conceptual and methodological difficulties that we have seen, is of great value for society. Its fixing and dissemination are important tasks, but there is still a long way to go. This work hopes to make a small contribution to this field by taking an in-depth look at the calculation and improvement of filters, which are, ultimately, coefficients defined to make the calculation of social value more accurate, precise, credible and usable.

Author Contributions

Conceptualisation, M.S.-G.; Data curation, J.N.-M.; Investigation, M.S.-G.; Methodology, M.S.-G. and J.N.-M.; Supervision, M.S.-G.; Writing—original draft, M.S.-G.; Writing—review & editing, L.M.R.-G.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Caption.

Case	Title	Summary	Link
1	CraftCafé, Social return on investment evaluation. Report for impact arts, Scotland.	Craft Café is a programme that develops support activities targeted at older people to reduce their isolation and loneliness, and improve their physical and mental well-being. In the Arts Café they are taught arts and craft activities to, through these activities, learn new skills, meet new people, and reconnect with their communities, in addition to boosting their creativity, artistic expression, and confidence.	http://www.socialimpactscotland.org.uk/case-studies/
2	Social Return on Investment (SROI): Analysis of the Greenlink, a partnership project managed by the Central Scotland Forest Trust (CSFT).	The objective of the Greenlink project is to conserve the woodland that surrounds the Greenlink path. On the basis of the importance of nature and green spaces in the promotion of mental health and well-being, the project carries out a wide range of activities, whose goals are to provide an opportunity for exercise, increase biodiversity in woodland and offer informal training and experience of forest management.	http://1068899683.n263075.test.prositehosting.co.uk/wp-content/uploads/2013/04/Greenlink-SROI-Final-report-5-October-2009.pdf
3	Using social return on investment (SROI) to measure the value created by CSR initiatives, Stockholm resilience centre, Stockholm.	The project consists in the promotion of a shared water-collection structure in the Thar Desert in Rajasthan, called taanka. Due to the scarcity of water in this area, it is necessary to install mechanisms that facilitate the collection and storage of water. The objective of the taankas is to provide water to families who do not have access to other sources nearby.	http://www.diva-portal.org/smash/get/diva2:539650/FULLTEXT01.pdf
4	The social value of a community-based health project: Healthy Living Wessex Social Return on Investment report, project report University of the West of England, Bristol.	Healthy Living Wessex is a project designed to promote healthy lifestyles in people at risk of health problems because of being overweight. The project operates through two centres, offering their services in the most disadvantaged communities in these areas. These centres are responsible for attracting hard-to-reach groups by helping them to improve their own physical and mental health through fun, accessible activities.	http://eprints.uwe.ac.uk/16589/14/Jones_2012_HLW_Social_Value_Report_.pdf
5	Social Return on Investment (SROI), the value added for families before and after using Solvatten in the Bungoma district in western Kenya.	Solvatten is a Swedish product designed as a low-cost solution to the serious health problems that exist in most developing countries, arising from the lack of fresh, clean water. The product, Solvatten, is a water-treatment unit for the home that can treat water with viruses, parasites and bacteria, without needing any chemical product or power source except the sun. These features make this product easy to use.	Economics and Management, Vol.15, N°699, (1-91)
6	The Houghton Project: Social Return on Investment (SROI), Worcester.	The Houghton Project is responsible for promoting the positive benefits that interaction with the rural environment generates in the health of people, from a farm in rural Herefordshire (England). This farm accommodates people who suffer from some type of physical or mental illness, or any type of socialisation or behavioural problem. The project is responsible for providing them with therapy, education, training, work and friendship through rural activities such as harvesting fruits and vegetables, looking after plants and animals, and developing new skills in the environment.	http://www.futureroots.net/wp-content/uploads/2013/01/Houghton-Project-SROI-for-assurance-03.09.12.pdf
7	Evaluating and measuring the impact of health promotion activities in hospital grounds, Centre for Rural Health, University of the Highlands and Islands.	The programme, offered to staff and patients of the Forth Valley Hospital, consists of a weekly Tai Chi session in the woods for five weeks, with the aim that the participants carry out low-intensity exercise and spend time in the woods. This outdoor activity is anticipated to have a positive influence on health as a result of physical activity, social relations and the well-being associated with being in touch with nature.	http://www.uhi.ac.uk/en/research-enterprise/res-themes/health/centre-for-rural-health/AGuideforDrawingonthePrinciplesofSocialReturnonInvestment.pdf
8	Social Ventures Australia: Spinal cord injuries Australia’s national Walk on program.	Walk on is a rehabilitation programme based on an intensive programme of activities. Through an individual plan, people with spinal cord injury can improve and maximize their functional capacity and lead a more independent life.	http://scia.org.au/images/SCIA-media/Services/Walkon/Walk%20On%20National%20SROI%20-%202013%20-%20Full%20Report.pdf
9	SROI analysis on the Ardoyne Women’s Group (AWG) Young Women’s Programme.	The Young Women’s programme was set up by AWG to engage with women during adolescence. Prevention and building up the network of support for these young women will help them to overcome the cyclical patterns that plague the area in which they live such as unemployment, poverty, lack of education and low levels of self-esteem. The Programme works with the young women to develop them as an individual. Building their self confidence and self-esteem while also providing opportunities to engage with other young people and be a part of programmes that will encourage them to learn new skills.	http://www.wsn.org.uk/sites/default/files/publications/WSN%20SROI%20Reports_AWG_WINI_FINAL_Aug11.pdf
10	SROI analysis on Women’s Information Northern Ireland (WINI) Champion’s Programme.	CHIW is based on the rationale that ‘information is power’ and that through the provision of health information by local women in an informal setting, individuals are empowered to improve their own health and well-being, and that of their local communities. The Programme recognises the pivotal role women in local communities can have in strengthening social and community networks, and the significance of such networks in the delivery of health services to people living in neighbourhoods of high social and economic need.	http://www.wsn.org.uk/sites/default/files/publications/WSN%20SROI%20Reports_AWG_WINI_FINAL_Aug11.pdf
11	Talensi Farmer-Managed Natural Regeneration Project, Ghana. Social Return on Investment report.	The main objective of the Talensi Natural Regeneration Project, managed by farmers, was to provide a subsistence income for rural families who live in a semi-arid area. Fires and the consumption of forest resources prevented the recovery of the natural environment. To reverse the deterioration, the project promoted the mobilisation of the community around natural regeneration and agroecology assisted by farmers.	http://fmnrhub.com.au/wp-content/uploads/2013/09/SROI-Report_High-Resolution.pdf
12	SROI-Forecast of Forth Sector Business Development Plan.	The Forth Sector company aims to support, through the creation of employment, people with mental health problems. It sets up socially oriented companies that operate sectors in a large quantity: from laundry services to manufacturing for retail. The SROI report presents a forecast of the social return created by the Forth Sector’s Business Development Plan that forecasts substantial growth over the next five years.	http://1068899683.n263075.test.prositehosting.co.uk/wp-content/uploads/2013/04/SROI-Forecast-Forth-Sector-Final-130410.pdf
13	Adult and community learning fund Forecast of Social Return on Investment of Insiders’ Guide ‘Journeys for Living and Learning’.	The support course for caregiver parents Insiders’ Guide (IG) aims to help caregiver parents of disabled children to develop their skills, resilience and confidence so that they can face, in the best possible way, the challenge of bringing up children with special needs, emotional pain and adverse situations.	http://www.parents-and-carers.org.uk/blogimages/SROI%20FINAL%20REPORT%20August%202012%20Insiders%20Guide%20to%20Living%20and%20Learning%20ACLF%20Project%205592.pdf
14	Food Connect Sydney. Forecast SROI Report.	An Australian consultancy firm, Social Ventures Australia Consulting, published a predictive SROI analysis on a project that consists in making food grown according to ecological criteria accessible to the citizens of Sydney. These foods are produced by local farmers.	http://www.socialvalueuk.org/app/uploads/2016/03/Food%20Connect%20Sydney_Forecast%20SROI%20Report_Mar%202011.pdf
15	SROI analysis of the “FAB PAD” project of impact arts in North Ayrshire.	The main objective of Impact Arts is to use art as a catalyst to produce lasting positive changes in people’s lives. The Fab Pad concept was developed to work with vulnerable people who lead disorganised lives. Fab Pad offers people training in art, design and practical skills to transform their houses into homes, and supports them afterwards when finding their way in education, learning and employment.	http://www.socialvalueuk.org/app/uploads/2016/03/Impact%20Arts%20SROI%20report.pdf
16	The economic and social return of Action for Children’s Family Intervention Project, Northamptonshire.	This analysis is audited by the SROI Network. It evaluates the social value generated by a family intervention project that works with families with children and in highly vulnerable situations (alcohol and drug problems, anti-social behaviour, domestic violence, risk of loss of housing, neglected children, etc.)	http://www.socialvalueuk.org/app/uploads/2016/03/assurance%20submission%20final%20TVB.pdf
17	Analysis of the social return on public investment in a social initiative sheltered employment centre through the implementation of SROI (Social return on investment) methodology.	The analysis is carried out by Ecodes and a social consultancy called Altercivitas. It evaluates the social impact of a “social initiative sheltered employment centre”. Sheltered employment centres dealt with the work integration of disabled people and gained legal recognition in 1982 (Law 13/1982 for the Social Integration of Disabled Persons),	http://www.ecodes.org/phocadownload/Informe_SROI_CEE_2013.pdf
18	Analysis of the social return of a cafeteria for the elderly in England.	This is the example that is worked on in “A Guide to Social Return on Investment”, published by the Office of the Third Sector of the Cabinet Office. It looks at “Wheel to meals”, a project to create a cafeteria for 30 elderly or disabled people who were provided with transport to the cafeteria.	http://www.socialvalueuk.org/app/uploads/2016/03/The%20Guide%20to%20Social%20Return%20on%20Investment%202015.pdf
19	Oxford Castle Renovation.	This is an example of a simplified SROI analysis of the renovation of Oxford Castle.	http://www.princesregeneration.org/sustainableheritage/sites/all/themes/prtrust/files/prt-oxford-castle-sroi.pdf
20	Urban mountain bike trail in Glen Mile Mountain.	The mountain bike trail was initially developed as a partnership between SWT, Forestry Commission Scotland (FCS), Cycling Scotland and high schools in the Cumbernauld area. The aim was to respond to complaints about anti-social behaviour in the area caused by the increasing volume of unsupervised and unauthorised biking in the reserve. This was causing concern to local residents who used the reserve for walking and other legitimate purposes.	http://www.socialvalueuk.org/app/uploads/2016/09/SROIurbannature_Glen20Mile1GreenspaceJuly-2011.pdf
21	Restoration of a garden in the grounds of the old Speir’s school in Beith (Scotland). July 2011.	The project consisted of the restoration of gardens, in which volunteers from the municipality and students got involved. SROI analysis focussed on a single activity, although various activities were being carried out at the same time, and a number of organisations were involved.	http://www.socialvalueuk.org/app/uploads/2016/09/SROIurbannature_Spiers1greenspace-july-2011.pdf
22	Activities for people with mental illnesses in Kinnoull Hill Woodland Park, in Perth. July 2011.	The project consisted of using a natural area for outdoor activities with the mentally ill, with the aim of improving their well-being and fitness, and developing their skills.	http://www.socialvalueuk.org/app/uploads/2016/09/SROIurbannature_woodsforhealth1-July-2011.pdf
23	Creation of a school ecoclub in the Dumbreck marsh local nature reserve. July 2011.	Drumbreck Marsh is a wetland near an urban population. The SROI analysis evaluates the social return on the investment in the creation of the school Ecoclub. It is a partial analysis because, for various reasons, not all of the stakeholders could be contacted.	http://www.socialvalueuk.org/app/uploads/2016/09/SROIurbannature_Dumbreck1july-2011.pdf

References

Costa, E.; Pesci, C. Social impact measurement: Why do stakeholders matter? Sustain. Account. Manag. Policy J. 2016, 7, 99–124. [Google Scholar] [CrossRef] [Scilit]
Ebrahim, A.; Rangan, V.K. What impact? A framework for measuring the scale and scope of social performance. Calif. Manag. Rev. 2014, 56, 118–141. [Google Scholar] [CrossRef] [Scilit]
Power, M. The Audit Society: Rituals of Verification, 2nd ed.; Oxford University Press: Oxford, UK, 1999. [Google Scholar]
Arvidson, M.; Lyon, F.; McKay, S.; Moro, D. The Ambitions and Challenges of SROI; Working Paper; TSRC Third Sector Research Centre: University of Birmingham, Birmingham, 2010. [Google Scholar]
Kolk, A. A decade of sustainability reporting: Developments and significance. Int. J. Environ. Sustain. Dev. 2004, 3, 51–64. [Google Scholar] [CrossRef] [Scilit]
Morhardt, J.E. Corporate social responsibility and sustainability reporting on the internet. Bus. Strategy Environ. 2010, 19, 436–452. [Google Scholar] [CrossRef] [Scilit]
Milne, M.J.; Tregidga, H.M.; Walton, S. Words not actions! The ideological role of sustainable development reporting. Account. Audit. Account. J. 2009, 22, 1211–1257. [Google Scholar] [CrossRef] [Scilit]
Cooper, S.M.; Owen, D.L. Corporate social reporting and stakeholder accountability: The missing link. Account. Organ. Soc. 2007, 32, 649–667. [Google Scholar] [CrossRef] [Scilit]
Sinclair, P.; Walton, J. Environmental reporting within the forest and paper industry. Bus. Strategy Environ. 2003, 12, 326–337. [Google Scholar] [CrossRef] [Scilit]
Maas, K.; Schaltegger, S.; Crutzen, N. Integrating corporate sustainability assessment, management accounting, control, and reporting. J. Clean. Prod. 2016, 136, 237–248. [Google Scholar] [CrossRef] [Scilit]
Vigneau, L.; Humphreys, M.; Moon, J. How do firms comply with international sustainability standards? Processes and consequences of adopting the global reporting initiative. J. Bus. Ethics 2015, 131, 469–486. [Google Scholar] [CrossRef] [Scilit]
Bebbington, J.; Brown, J.; Frame, B. Accounting technologies and sustainability assessment models. Ecol. Econ. 2007, 61, 224–236. [Google Scholar] [CrossRef] [Scilit]
Arvidson, M.; Lyon, F. Social impact measurement and non-profit organisations: Compliance, resistance, and promotion. VOLUNTAS Int. J. Volunt. Nonprofit Organ. 2014, 25, 869–886. [Google Scholar] [CrossRef] [Scilit]
Moxham, C. Help or hindrance? Examining the role of performance measurement in UK nonprofit organizations. Public Perform. Manag. Rev. 2010, 33, 342–354. [Google Scholar] [CrossRef] [Scilit]
Nicholls, A. The functions of performance measurement in social entrepreneurship: Control, planning and accountability. In Values and Opportunities in Social Entrepreneurship; Palgrave Macmillan: London, UK, 2010; pp. 241–272. [Google Scholar]
Ebrahim, A.; Weisband, E. Global Accountabilities: Participation, Pluralism, and Public Ethics; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
Gibelman, M.; Gelman, S.R. A loss of credibility: Patterns of wrongdoing among nongovernmental organizations. VOLUNTAS Int. J. Volunt. Nonprofit Organ. 2004, 15, 355–381. [Google Scholar] [CrossRef] [Scilit]
Solórzano-García, M.; Navío-Marco, J.; Valcárcel-Dueñas, M. Impact measurement for social innovation: Analysis of the Spanish third sector. In Social Impact Investing Beyond the SIB; Palgrave Macmillan: Cham, Switzerland, 2018; pp. 195–209. [Google Scholar]
Lall, S. Measuring to Improve Versus Measuring to Prove: Understanding the Adoption of Social Performance Measurement Practices in Nascent Social Enterprises. VOLUNTAS Int. J. Volunt. Nonprofit Organ. 2017, 28, 2633–2657. [Google Scholar] [CrossRef] [Scilit]
Arena, M.; Azzone, G.; Bengo, I. Performance measurement for social enterprises. VOLUNTAS Int. J. Volunt. Nonprofit Organ. 2015, 26, 649–672. [Google Scholar] [CrossRef] [Scilit]
Barman, E. Of principle and principal: Value plurality in the market of impact investing. Valuat. Stud. 2015, 3, 9–44. [Google Scholar] [CrossRef] [Scilit]
Nicholls, A. We do good things, don’t we?: Blended value accounting in social entrepreneurship. Account. Organ. Soc. 2009, 34, 755–769. [Google Scholar] [CrossRef] [Scilit]
Liket, K.C.; Rey-Garcia, M.; Maas, K.E. Why aren’t evaluations working and what to do about it: A framework for negotiating meaningful evaluation in nonprofits. Am. J. Eval. 2014, 35, 171–188. [Google Scholar] [CrossRef] [Scilit]
Molecke, G.; Pinkse, J. Accountability for social impact: A bricolage perspective on impact measurement in social enterprises. J. Bus. Ventur. 2017, 32, 550–568. [Google Scholar] [CrossRef] [Scilit]
Emerson, J. The blended value proposition: Integrating social and financial returns. Calif. Manag. Rev. 2003, 45, 35–51. [Google Scholar] [CrossRef] [Scilit]
Maier, F.; Schober, C.; Simsa, R.; Millner, R. SROI as a method for evaluation research: Understanding merits and limitations. VOLUNTAS Int. J. Volunt. Nonprofit Organ. 2015, 26, 1805–1830. [Google Scholar] [CrossRef] [Scilit]
Nicholls, J.; Lawlor, E.; Neitzert, E.; Goodspeed, T. A Guide to Social Return on Investment; The SROI network in association with the Scottish Government; Available online: https://iris.thegiin.org/guide/a-guide-to-social-return-on-investment/summary (accessed on 6 August 2018).
Pathak, P.; Dattani, P. Social return on investment: Three technical challenges. Soc. Enterp. J. 2014, 10, 91–104. [Google Scholar] [CrossRef] [Scilit]
Jardine, C.; Whyte, B. Valuing desistence? A social return on investment case study of a Throughcare Project for short-term prisoners. Soc. Environ. Account. J. 2013, 33, 20–32. [Google Scholar] [CrossRef] [Scilit]
Steed, S.; Nicholles, N. Small Slices of a Bigger Pie: Attribution in SROI; New Economics Foundation: London, UK, 2011. [Google Scholar]
Roche, C.J. Impact Assessment for Development Agencies: Learning to Value Change; Oxfam: Oxford, UK, 1999. [Google Scholar]
Solórzano-García, M.; González-García, N.; Contreras-Comeche, R.; Navío-Marco, J. Del Valor Social al Impacto Social Aplicación de Coeficientes de Depuración en el Método SROI; Instituto Nacional de Administración Pública (INAP): Madrid, Spain, 2015. [Google Scholar]
Eriksson, P.; Kovalainen, A. Qualitative Methods in Business Research: A Practical Guide to Social Research; Sage: London, UK, 2015. [Google Scholar]
Jesson, J.K.; Matheson, L.; Lacey, F.M. Doing Your Literature Review: Traditional and Systematic Techniques; Sage: London, UK, 2011. [Google Scholar]
Silverman, D. Qualitative Research; Sage: London, UK, 2016. [Google Scholar]

Table 1. Deadweight.

Case	Calculation of the deadweight
1	The method for choosing the appropriate percentage was carried out through interviews. The project analysed the percentages of each person and then they made an average to obtain a more robust figure. The family members group did not recognise the existence of deadweight; however, the evaluators felt the need to incorporate 5% to reflect a small sample size.
2	The method used in the calculation was based on a survey of households in Scotland. The results show that the volunteering rate in general in North Lanarkshire is 13.7%. Therefore, it is estimated that 13.7% of the Greenlink volunteers would be volunteers whether the Greenlink was there or not. The rest of the data is obtained through statistical data.
3	Deadweight is divided into three areas of results, but the estimates made are not specified.
4	A minimum deadweight percentage for all the results is established, which ranges from 5% to 10%.
5	Deadweight is not taken into account due to the fact that all the results are caused solely by the effect of the product.
6	The lack of absolute certainty meant that a deadweight figure of 5% was assigned to all results.
7	The percentages are obtained through estimates for each result, but there are no details on the procedure.
8	Percentages are structured on a scale that goes from lowest to highest in terms of the influence of the project on the result obtained (0% where the result would not have been achieved had the activity not been carried out; 25% where the result would have been achieved but to a limited degree; 50% where the result would have occurred, in part, in any case; 75% where the result would have occurred, for the most part, in any case; and 100% where the result would have occurred anyway).
9	For the young women group, 5% was set as the value of the deadweight. In general, it was quantified at 5%, because of a previous example in which the hospitalisation of one person was prevented. For the volunteers/employees group the deadweight figure was fixed as non-existent (0%), since they would not have developed certain skills if they had not worked on the project. For the funding entities, deadweight was set at 0%.
10	For the group formed by the elderly a deadweight value of 5% was established. In the case of the Champion, i.e., the volunteer in the Day Centre, 5% was set. For the CHIW and Ardoyne Healthy Living, deadweight was estimated as non-existent. It was understood that without the Champions Programme, the results would not have been produced. Finally, for the families of the elderly group, deadweight was set at 5%.
11	In most cases, the deadweight values were calculated on the basis of data collected in the comparison group. The percentages to be applied ranged from 0% to 10%.
12	The deadweight percentages applied in this case ranged from 0% (used, for example, in the case of the result “increase in the efficiency of the therapy” for the interest group formed by support workers and occupational therapists) to 50% (set for the result “increased workload” of the interest group “support workers”).
13	There are no details on the manner in which they have come to those estimates.
14	Calculation is standardised (0% = The result would not have been achieved without the intervention; 25% = The result would have been achieved but to a very limited degree; 50% = The result would have occurred, partially, in any case; 75% = The result would have occurred, for the most part, in any case; 100% = The result would have occurred anyway).
15	Of the eight results analysed, in five cases statistical data is used to evaluate deadweight. In two cases, information taken from surveys of participants is used. In one case, the estimate is not justified.
16	Although the consultant’s opinion is that without this intervention the results evaluated would not have occurred, the evaluation attributes 15% in one of the results (the most emotional, the most difficult to quantify) and 0% in the other two results.
17	The evaluators quantify it mostly as 0% because without this project, the employability of the severely disabled is practically non-existent. Only in the case of disabled people without special problems of employability do they consider deadweight to be the percentage of people with a job in the general population (statistic).
18	The volunteers identified that the luncheon club involved more physical exercise than they might have otherwise sought. The average was around 45% more. As such, if the benchmark is 100%, since all of them would have done some other exercise anyway, the increase is therefore 145%. The estimate of deadweight is 100%/145% or 70%. For the outcome of ‘residents having nutritious meals’, the local government organisation could have obtained another provider to deliver it, so the deadweight is 100%.
19	They calculate the deadweight as the percentage of the result that would have occurred anyway, but there is no indication of how they calculated it.
20	It is calculated by reasoning and logical estimates, without any systematics.
21	For most of the results, statistical data is used. In other cases, it can be concluded logically that there is no deadweight. In certain other cases, a value is estimated in the absence of statistics.
22	It is calculated by means of estimates.
23	It is considered non-existent.

Table 2. Displacement.

Case	Calculation of displacement
1	It is not considered.
2	The only result that always involves some displacement is getting a job. The job density in Scotland is 0.82, meaning that for each applicant there are 0.82 jobs, which is equivalent to 18% displacement. That is to say, the possibility of getting a job is now 18% lower.
3	No displacement has been identified in this study.
4	The effects of displacement are considered to be minimal and range from 0% to 10%, depending on the nature of the result.
5	It is not considered that there is displacement in this case.
6	Displacement was not considered by the vast majority of the project’s stakeholders. However, one of the volunteers suggested that they could have opted for helping elsewhere if they had not decided to spend their time on the project, and therefore a displacement rate of 20% has been assigned to the project.
7	A displacement rate of 75% is applied to the Tai Chi intervention.
8	Standardisation is as follows: 0% if no displacement; 25% if it displaces another result in a limited way; 50% if the result displaces another result partially; 75% if the result displaces another result significantly; and, finally, 100% displacement if the result displaces another result completely.
9	The same values are applied as for deadweight: young women 5%, parents and caregivers 5%, volunteers/employees 0% and funding entities 0%.
10	In this programme only 5% was applied, in the case of the Champions group.
11	In this case, due to the various contradictions in the information collected for the different groups, a decision was taken to add the speculative effect as 50% of displacement.
12	In the case of this Plan, no displacements of other services or activities were identified.
13	There are no details on the manner in which they have come to those estimates.
14	The calculation is standardised 0% = The result has not displaced any other result; 25% = The result has displaced another result, in a limited way; 50% = The result has displaced another result partially; 75% = The result has displaced another result to a significant degree; 100% = The result has displaced another result completely.
15	It is not considered.
16	It is considered non-existent, justifiably.
17	It is considered non-existent, justifiably.
18	Displacement has not been considered.
19	It is not considered, without being justified.
20	Two possible effects are considered; one is discarded after an interview with the stakeholder, and the second is seen as a negative effect on the results (it is counted).
21	The percentages of displacement reflect the time that some stakeholders stopped devoting to other possible activities because they were carrying out this initiative. The value is estimated in one case. In another, it is based on surveys.
22	The extra work caused to third parties is considered to be displacement. The figure is estimated.
23	The evaluators consider it non-existent, but the reason provided is doubtful. It is argued that since this is an analysis of a project that is taking place in the present, there is no displacement.

Table 3. Attribution.

Case	Calculation of attribution
1	The attribution rates assigned were: 19% for participants in the programme; 5% for family members; 15% for Housing Associations, and 18% for the NHS. The calculations are not specified.
2	Based on their experience, Greenlink establishes a 33% attribution rate for volunteers, since 33% of the volunteers had arrived through references from other agencies.
3	A 100% attribution coefficient is used.
4	It is calculated by reasoning and logical estimates, but the procedure is not specified.
5	It is obtained through statistics. An attribution rate of 30% is considered, since, as noted in the interviews, only 60% of patients listen to the doctor’s recommendations, and 50% of these do not follow the recommendations in their homes.
6	It is calculated by reasoning and logical estimates, but the procedure is not specified.
7	The participants were asked to assign a percentage of attribution to any change in physical or mental health that could have occurred during the course of the intervention. The average percentage of these results was 62%.
8	The calculation is standardised (0% attribution rate if no external input; if there are other organisms that had some minor role in the results 25% would be established; if these organisms had a more important role in getting the results 50% would be assigned; in the event that the role of the organisms external to the project was important it would be a 75% attribution rate; and if there was full external input 100% attribution rate would be used).
9	We applied the same values as for the deadweight: young women 5%, parents and caregivers 5%, volunteers/employees 0%. Finally, for the interest group formed by the entities that participated in funding, a figure of 10% was established for attribution calculations in the case of theatre activities, since these could have been carried out by another company.
10	In the case of the Champion group, i.e., the volunteer in the Day Centre, 5% was set. 5% was also set for the group of families of the elderly.
11	Based on the responses obtained in forums of local groups, and interviews with those who were key to the project, the overall attribution rates were set between 0% and 20%. In the individual case of a participating community that already owned an area of forest reserve, it was understood that it was already a leader in nature conservation. An attribution rate of 12% was applied, a result of rounding off the maximum and minimum values considered.
12	On many occasions the changes are not due to an activity; they are the result of more than one service or set of people working together. The attribution contemplates this overvaluation, and it was fixed as a percentage (varying between 0% and 30%), which was subsequently deducted from the total impact. In the case of the customers it was 30%.
13	The report analyses the four filter coefficients for each of the results. However, it does not specify the way in which it has estimated these percentages.
14	The calculation is standardised (0% = The result is wholly the result of the intervention and no other activity, programme or organisation has contributed; 25% = Other people or organisations have played a minor role in generating the results; 50% = Other people or organisations have played an important role in generating the results; 75% = Other people or organisations have played a significant role in generating the results; 100% = The result is entirely the result of other people or organisations).
15	In the calculation they attribute all the results to themselves, and obtain a SROI greater than 8. Then they make the assumption considering themselves responsible for the results in 75% of cases, and the SROI is still greater than 6. Then they make the same assumption, but with 50%, and the SROI is still greater than 3. Conclusion: it is still a high SROI.
16	Attribution is calculated by asking everyone involved in the project whether they believe that other organisations have also contributed to the result. The values range from 0% to 25%, although the consultant’s comments emphasise that, despite the fact that there were other organisations involved, no changes were occurring until this intervention began.
17	In general, they consider a 100% attribution rate of the results, because if there were no project, the chances of achieving the result are practically non-existent.
18	Based on the results in the questionnaire it was possible to estimate that 35% of the outcome was the result of the contributions of others.
19	The attribution rate for each of the results is different, but they do not explain how they arrive at those figures.
20	It is calculated by reasoning and logical estimates, without any systematics.
21	In most cases, attribution has been calculated by direct observation of the progress of the project, by logical causes and also by interviews/questionnaires.
22	In a more or less arbitrary way. In some cases, attribution is based on questionnaires prior to the activity, but in others, it is only based on estimates.
23	Based on estimates and questionnaires to stakeholders.

Table 4. Drop-off.

Case	Calculation of drop-off
1	A percentage drop of 33% is assumed for this SROI analysis, based on the experience of the Arts Impact team in the delivery of the programme for older people.
2	In terms of the Greenlink, we have no historical data on the extent to which the result reduces with time, which is why for most of the results (which last more than a year), a 15% drop-off rate has been used as the standard percentage.
3	The percentage is calculated by dividing the value created by the increased school attendance by the annual value of 2.5 of all the results during the first two and a half years.
4	SROI limits the duration of all results to a maximum of three years, and estimates a fall of up to 50% for many results.
5	The beneficiaries experience results that will continue during the lifespan of the product (which coincides with the period of analysis), and for this reason drop-off is considered non-existent.
6	Drop-off is estimated at 0% except for the results for young people and adults trained as users of the service, for which a drop-off rate of 20% is estimated.
7	It was considered that the impact of the intervention would not last more than six months after the pilot project’s date of completion. After this time, without continuous learning sessions, it would probably decrease, and the future benefits could be attributed to other factors. A drop-off rate of 50% and 70% is assigned.
8	The calculation is standardised: the result lasts for the entire period of time it is allocated (0%); the result falls by 25% yearly from the second year (25%); the result falls by 50% yearly from the second year (50%); the result falls by 75% yearly from the second year (75%); and the result falls completely at the end of the time period (100%).
9	Drop-off was not applied in this case. SROI was only measured for the year in which the Programme was carried out. The report stressed the difficulty of measuring the long-term effects. An adjustment was made of some of the results obtained for the calculation of the SROI, with the aim of bringing it closer to reality. To do this, a sensitivity analysis was performed.
10	For the group formed by the elderly, a variable drop-off rate was established, with a generic value of 10%, 30% for the care homes, since there may be a decrease in the impact of the Programme after the first year, and 50% for a smaller number of home-care visits. For the families of the elderly group, a drop-off rate of 15% was determined, to consider the possibility that the activity could decrease, due to its ceasing to be a novelty. For the rest 0% was valued.
11	This percentage of deduction is applied when the changes last longer than the reference period and when it determines the percentage of annual loss in the benefits generated by the project. The drop-off values applied, which were generally low, were 0%, 10% or 20%.
12	A percentage drop of a 33% was considered for outcomes with a physical component (skills to cope with situations, life skills) and 66% for results with a predominantly mental component (confidence, self-esteem, personal skills and relationships).
13	Not specified.
14	The calculation is standardised (0% = the result lasts for the entire period of time it is allocated; 25% = The result falls by 25% from the second year; 50% = The result falls by 50% from the second year; 75% = The result falls by 75% from the second year; 100% = The result disappears completely from the second year).
15	In the majority of the results, drop-off is calculated by extrapolations based on some data resulting from the observation of the project. In two cases, drop-off is considered to be 0%, because during the programme there was no reduction in the obtaining of the result (a dubious interpretation of the parameter). In the end, in the sensitivity analysis they calculate that the drop-off value would have to be above 90% in order for the SROI to approach the value of 1, which is very unlikely.
16	They calculate drop-off based on the opinion of those involved as to whether, once the intervention is finished, the results obtained will be maintained, or whether the situation will worsen.
17	Drop-off is not calculated. This is a type of project that, if the intervention stops, the drop-off rate would be 100%.
18	10% is an estimation of the likelihood that residents will use the knowledge they gain less as time goes on, as they forget the sessions.
19	Neither displacement nor drop-off is considered, without explaining the reasons.
20	It is considered that the infrastructure created will remain in a good condition for about five years. An estimate is made (with logical reasoning, without systematics) of how much each of the results will decrease each year, in the next five years, if the associated activities cease to be carried out.
21	The skills developed by the project have a duration of two or three years (estimate) and the annual drop-off depends on the assumption of whether they will be used again or not (estimates/assumptions). It is considered that the infrastructure will be maintained for five years with an annual decline of 30% (estimate).
22	The skills developed by the project have a duration of five years (estimate) and the annual drop-off depends on the assumption of whether they will be used again or not (estimates/assumptions).
23	The knowledge and behaviour developed by the project have a duration of five years, and evenly the evaluators distribute the decline in the five years until its expiry.

Table 5. Filters coefficients comparison (I: Collection of information C: Calculation of filters).

Table 6. Craft Café case. Re-evaluation.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Article Metrics

Citations

Article Access Statistics

Journal Statistics

Multiple requests from the same IP address are counted as one view.

	Deadweight	Displacement	Attribution	Detriment
1.	I: Stakeholder feedback C: Argument and justification.	Not considered	I: Stakeholder feedback C: Percentages based on stakeholder’s judgements.	No research data available. Percentage based on the past experience of the programme team in similar cases
2.	I + C: Benchmark indicators (public statistics)+argument and justification	I: Stakeholder feedback C: Argument and justification	I: Stakeholder feedback+SROI team experience C: Argument and justification	No historical data. Used 15% as a standard percentage
3.	I: Arbitrary estimation C: Neither argument nor justification	Not considered	I: Stakeholder feedback	Benchmark indicator Neither argument nor justification
4.	I: Research evidence+stakeholder feedback C: Neither argument nor justification	Neither argument nor justification	I: Stakeholder feedback C: Argument not justification	I: Research evidence+stakeholder feedback C: Argument not justification
5.	I+C: Control group	I+C: Control group	I+C: Control group	No drop-off. No argument
6.	I: Stakeholder feedback. No methodologies specified C: Neither argument nor justification	I: Stakeholder feedback. No methodologies specified C: Neither argument nor justification	I: Stakeholder feedback. No methodologies specified C: Neither argument nor justification	I: NA C: NA
7.	I: Arbitrary estimation C: Neither argument nor justification	I: NA C: Argument not justification	I: Stakeholder feedback C: Argument and justification	I: Arbitrary estimation C: Neither argument nor justification
8.	I: Stakeholder feedback C: Structured on a five standardised intervals scale Argument+justification	I: Stakeholder feedback C: Structured on a five standardised intervals scale Argument+justification	I: Stakeholder feedback C: Structured on a five standardised intervals scale Argument+justification	I: Stakeholder feedback C: Structured on a five standardised intervals scale Argument+justification
9.	I: Research evidence+stakeholder feedback C: Neither argument nor justification	I: Research evidence+stakeholder feedback C: Neither argument nor justification	I: Research evidence+stakeholder feedback C: Neither argument nor justification	No drop-off applied.
10.	I: Research evidence+stakeholder feedback C: Neither argument nor justification	I: Research evidence+stakeholder feedback C: Neither argument nor justification	I: Research evidence+stakeholder feedback C: Neither argument nor justification	I: Research evidence+stakeholder feedback C: Neither argument nor justification
11.	I+C: Control group	I: Stakeholders and key informant feedback C: Neither argument nor justification	I: Stakeholders and key informant feedback C: Neither argument nor justification	I: Stakeholders and key informant feedback+research information
12.	I: Stakeholder feedback. Not representative sample. C: Neither argument nor justification	Not considered	I: Stakeholder feedback. Not representative sample. C: Neither argument nor justification	I: Arbitrary estimation C: Neither argument nor justification
13.	I: Stakeholder feedback. C: Neither argument nor justification	I: Stakeholder feedback. C: Neither argument nor justification	I: Stakeholder feedback. C: Neither argument nor justification	I: Stakeholder feedback. C: Neither argument nor justification One percentage for every outcome.
14.	I: Stakeholder feedback C: Structured on a five standardised intervals scale. Argument+justification	Not considered	Not considered	No drop off applied
15.	I+C: Stakeholder feedback+benchmark indicators (public statistics+research evidence)	Not considered	No attribution	I: Research evidence C: Argument not justification
16.	I: Stakeholder feedback+research evidence. C: Argument not justification	Not considered	I: Stakeholder feedback+research evidence. C: Argument not justification	I: Stakeholder feedback C: Argument not justification
17.	I: Stakeholder feedback+research evidence+benchmark indicators (public statistics) C: Neither argument nor justification	Not considered	I: Arbitrary estimation C: Neither argument nor justification	Not considered
18.	I+C: Neither argument nor justification	I+C: Neither argument nor justification	I+C: Neither argument nor justification	I+C: Neither argument nor justification
19.	I+C: Neither argument nor justification	Not considered	I+C: Neither argument nor justification	Not considered
20.	I: Stakeholder feedback C: Argument+justification	I: Stakeholder feedback C: Neither argument nor justification	I: Stakeholder feedback C: Argument+justification	I: Stakeholder feedback C: Argument not justification
21.	I: Stakeholder feedback+research evidence+benchmark indicators (public statistics) C: Argument+justification	I: Stakeholder feedback C: Argument+justification	I: Stakeholder feedback+research evidence C: Argument+justification	I: Stakeholder feedback C: Argument not justification
22.	I: Stakeholder feedback C: Argument not justification	I: Stakeholder feedback C: Argument not justification	I: Stakeholder feedback C: Argument not justification	I: Research evidence C: Argument not justification
23.	I: Stakeholder feedback C: Neither argument nor justification	I: Stakeholder feedback C: Neither argument nor justification	I: Stakeholder feedback C: Neither argument nor justification	I: Arbitrary estimation C: Neither argument nor justification

Stakeholder	Attribution	Attribution Re-Evaluated	Deadweight	Deadweight Re-Evaluated
Older people that participate	19%	25%	17%	25%
Family members of participants	5%	0%	5%	0%
Partner Housing associations	15%	25%	13%	25%
The National Health Service	18%	25%	15%	25%