Benefit Transfer: A Review of Methodologies and Challenges

Abstract

For policy makers, regulators and natural resource managers, the resources necessary for original empirical resource valuations are often unavailable. A common alternative to original valuation studies is the practice of benefit transfer—the use of an empirical value estimate or estimates from a previous study or studies for application in a similar context. In order to reduce the error inherent in applying values from one parcel of land to another, researchers commonly use meta-analysis, or the “study of studies”, to provide a more thorough and statistically valid value estimate for use in a benefit transfer. In the practice of benefit transfer, much emphasis has been placed on improving the validity of values for transfer, but fewer studies have focused on the appropriate application of the established estimates. In this article, several often disregarded concerns that should be addressed when practicing benefit transfer are identified. A special focus is placed on spatial considerations and the recent progress that has been made to incorporate spatial trends. Geographic information systems (GIS) are advocated as a useful tool for incorporating the spatial aspects of benefit transfer. Consensuses and trends in the literature are acknowledged, and areas of potential improvement are highlighted.

1. Introduction

It is common for federal and state agencies to analyze the costs and benefits that result from a given policy initiative. In fact, for federally implemented policies that are expected to have large economic impacts (more than $100 million dollars annually), impact analyses are required [1]. For the evaluation of environmental policies, estimation of a policy’s economic impact can be challenging. The estimation of value for marketable environmental resources, such as timber or fisheries, is relatively simple, but the estimation of non-market resources can be problematic.

Non-market resources are those resources that are not bought and sold in markets, usually because these goods or services are not partitionable or excludable. Examples of these goods and services are water/air quality provision, erosion control, carbon sequestration, etc. For the estimation of non-market environmental goods and services, it is desirable to perform original empirical research with regard to the value of the exact resource in question. Methods for this type of estimation include the contingent valuation method [2] and hedonic price method [3]. Original research is generally very costly in terms of the amount of time and the financial resources necessary to conduct defensible empirical estimations. However, because of the growth and availability of empirical valuation studies, a commonly used alternative to original valuation attempts—benefit transfer—has been increasingly utilized [4].

2. Benefit Transfer

Benefit transfer is said to be the “…practice of adapting value estimates from past research to assess the value of a similar, but separate, change in a different resource” [5]. In other words, benefit transfer uses previously established values that were estimated for sites with similar characteristics and in similar contexts for application into existing policy decisions. Although benefit transfers have been performed for decades, increased attention has been observed since the method began to become more recognized after a Special Issue of Water Resources Research was published in 1992, which was devoted entirely to the method and sponsored by the Association of Environmental and Resource Economics and the U.S. Environmental Protection Agency [6]. There are many economists who have doubts about the reliability of benefit transfers [7,8], but the consensus remains that benefit transfer will continue to play a role in environmental policy analysis because of the resource availability, or lack thereof, in governmental land management agencies.

While varying classifications exist [9], the literature generally characterizes benefit transfers in one of two ways. First, value transfers (AKA unit value transfers or fixed value transfers) refer to benefit transfers of two types: transfers which use a single value from a similar and previously valued site (study site) and transfers that use an aggregation or average of values from previous studies and apply them to the resource for which values are to be estimated (policy site). The former requires that the policy site and the study site be very similar with respect to demographic and physical characteristics of the land. The latter allows for transfers to less demographically similar sites by calibrating values according to features of the population or physical site characteristics attributed to that site [8].

Although value transfer methodologies continue to be developed [8] and no definitive conclusion exists as to whether value transfer or function transfer, the second classification, performs better [10], recent emphasis has been placed on function transfer as the more statistically rigorous and promising of the two transfer types [6]. Function transfer involves the use of a valuation function to calibrate the value being transferred from the study site(s) according to specific physical and demographic characteristics of the policy site. Though there is consensus that site similarity is an important factor in assuring transfer validity [6,11], function transfer has the potential to relax the degree of similarity necessary for transfer of benefits by allowing for differences in characteristics to be taken into account by a valuation function.

3. Reliability Concerns

There are errors inherent in any function transfer. Typically, these errors are categorized as one of two types: measurement errors and transfer error (AKA generalization error). Measurement errors are those errors that are manifest in the research related to the study site from which value is to be transferred. In other words, this error is associated with the assumptions made by practitioners of the primary research and the degree to which those assumptions generate a divergence between the actual value of a change of the resource in question and the estimate of that value by the primary study [6,12]. Some, but not all, of measurement error can be reduced by appropriate study selection, which has become a matter of some controversy as economists disagree about how inclusive one should be with respect to which value estimates should be permissible for transfer. Selection bias refers to the errors resulting from the inclusion or exclusion of primary studies originating from four general sources [6].

The first source of selection bias, research priority selection bias, is the bias that comes from choosing only studies that meet narrow criteria related to your research question. These biases could come from origins other than the benefit transfer practitioner such as pressure to cater to an intended audience or pressure from a funding institution. Methodology selection bias refers to the bias that stems from selection of studies in fields that may over- or under-utilize certain methodologies for reasons other than their being the best available (i.e., historical tendency, lack of exposure to alternative methodologies, etc.). This is problematic, especially if methodology is found to significantly determine variations in WTP [13]. Publication selection bias occurs when research dissemination media are preferential toward a particular subject or methodology and that preference results in useful literature being omitted or certain types of literature to be over-represented [14]. Finally, sample selection bias occurs when researchers performing a benefit transfer select only those studies that have sufficient metadata so that they can be incorporated into a valuation function with limited additional research. Though these biases have the potential to be very large and because selection effects are difficult to observe, it is not common that they are explicitly addressed in empirical research. Rosenberger and Stanley (2006) [7] offer a comprehensive discussion of these topics.

In contrast, transfer errors are routinely assessed in benefit transfer studies [6,7,12,15]. Transfer errors are errors that are a result of the actual act of transferring values. These errors can be associated with dissimilarities between the study site and the policy site, the method used to transfer values, or from a lack of consistency with respect to the economic construct under investigation [9]. It is in anticipation of transfer errors that economists put such emphasis on site selection and study selection. The measurement of transfer errors generally is performed using convergent validity testing, which compares an out-of-sample prediction of a transferable value to a primary empirical valuation of the (hypothetical) policy site [7,11].

It has also been asserted that the minimization of transfer errors is an academic venture and that transfer validity is a statistical issue [16]. The necessary precision of an estimate is contingent on the context of the estimation (the use of the value estimate). Higher degrees of precision are necessary for legal settlements and litigation than for preliminary policy impact estimations. This is suggested to be the reason that no consensus has been reached as to an acceptable minimum standard for reliability [6]. Nevertheless, the reduction of transfer errors will be critical in promoting the acceptance of benefit transfer as a viable method of policy evaluation.

4. Meta-Analysis for Benefits Transfer

A commonly used method for compiling and analyzing the data from studies toward the creation of a valuation function is a meta-analysis (MA). A meta-analysis is “…the statistical analysis of a large collection of results from individual studies for the purposes of integrating the findings. It provides a rigorous alternative to the casual, narrative discussion of research studies which is commonly used to make some sense of the rapidly expanding research literature” [17]. MA is used to promote progress in research fields by illuminating sources of variance between studies, thereby identifying areas of future research and technical improvement. In the realm of environmental resource valuation, MA is commonly used in benefit transfer endeavors because of its usefulness with regard to incorporating a structural utility framework with less strictly-economic information [18,19].

MAs can be performed for reasons as simple as to acquire a central tendency or mean value for certain ecosystem types, but usually practitioners conduct a MA to establish a model for out-of-sample value estimation [18]. Value estimators derived from MA can incorporate variables describing a myriad of site characteristics, demographics and economic characteristics. The only consensus is that, at a minimum, the economic variables used in theoretical utility functions should be present in the valuation function [8,19]. Although, outside of scholarly benefit transfer, this is often not the case. Smith et al. (2002) advocate strictly defining the structure of the core economic variables from an unobserved utility function a priori [19]. While this approach, which they call “preference calibration”, is more economically defensible and provides theoretical uniformity, there is no evidence that it performs better for benefit transfer [8]. It has also been noted that almost all of the research in this area has come from the economists who initially advanced the approach, perhaps because of the reluctance to accept the associated assumptions [6].

There are conflicting ideas about the most appropriate approach to establishing a valuation function for value estimation. Experts disagree about the structure and content necessary to provide a reliable estimation of value and the acceptability of the associated assumptions. A “strong structural utility theoretic” approach (like that advocated by [5]) implies that WTP for an environmental good is derived from a function of income, a vector of market prices, the quantity of the good available to the individual, the price of substitutes, the quality of the good, non-income characteristics of the individuals household, and a measure of the information available to the individual (under the common assumption that there is imperfect information when valuing environmental commodities). This approach follows most closely with economic theory and imposes a high degree of standardization.

Alternatively, loosening the standards of the strong structural utility theoretic approach to allow for other explanatory variables to enter the valuation function that do not necessarily follow economic theory, but contain useful information regarding sources of variance, allows a valuation practitioner to account for differences in study characteristics when multiple studies are being analyzed. Bergstrom and Taylor (2006) call this approach the “weak structural utility theoretic” approach [18]. As long as all of the core economic variables are included in a valuation function, including other explanatory variables in the model allows for greater flexibility in establishing the specification of the economic variables [5]. So, selecting studies that disclose the information related to the measurement of the variables mentioned above and assuring that they are measured consistently is critical in building valid valuation functions. The remaining sections will focus on function transfer using a weak structural utility theoretic approach to establishing a valuation function from meta-analysis.

5. Commodity Consistency

One consistently emphasized necessary condition for valid benefit transfer is commodity consistency. Intuitively, a transfer of benefits cannot be valid if the construct under investigation differs from study site to policy site. A seemingly simple requisite for valid transfer, securing commodity consistency can become problematic because of the nature of non-market environmental goods. The heterogeneity of some environmental goods, such as wetlands, presents a challenge with respect to acquiring sufficient primary study data that satisfies the requirement for commodity consistency. The following example is illustrative of this challenge.

5.1. Consistency between the Objects of Value

There is a wealth of valuation studies that focus on wetlands [20], many of which estimate the value of wetlands with respect to water quality provision, a seemingly distinct ecosystem service. However, water “quality” could be defined differently by different studies. When collecting valuation data for a MA, the practitioner should be conscious of whether a primary study values one of three things: the unbundled end product of an ecosystem service, the value of all ecosystem services contributed by an ecosystem, or the actual physical ecosystem.

Some studies attempt to estimate the value of wetlands with respect to their effect on the unbundled end-product of a water quality improvement like recreational swimming or recreational fishing [21]. This represents an attempt to estimate the value of the contribution of wetlands to the provision of a greater quality or quantity of purified water as the increased in value of an end product, in this case recreation in the form of swimming or fishing. This is difficult because a number of components, besides the effects of wetlands, define the quality of water. Also, in the case of increased quality of aquatic habitat, several features (food chain dynamics, climate conditions, etc.) of such a habitat affect its quality besides the nature of its water. So, in a judgment of this type, a value is estimated for wetlands ecosystem service (water quality provision) as a factor (water quality) influencing an input (aquatic habitat/water swimability) into production of an output (recreation).

As an alternative, the value of wetlands can be estimated as the value of all of the actual functions wetlands perform. These would include water purification, food chain facilitation, geochemical regulation, flow regulation, storm surge protection, etc. In estimating and aggregating the value of all of these services, it would be impossible to comprehensively estimate all functions without double accounting or making undue assumptions about the relationship between wetlands and their services. This is because wetlands have a high degree of heterogeneity between sites (and even within sites) that affects the characteristics of the services provided. Also, many services are being performed concurrently and the performance of one service may be affected by the performance of another, making it difficult to definitively distinguish which function is the actual good or service that should be valued. The relationship between these functions is challenging to observe and even more arduous to quantify.

One could also estimate the value of the feature itself. This estimate would be a measure of the utility derived from the wetlands distinct from its consumptive uses. Cultural significance and biological diversity are valuable commodities associated with wetlands, and while these are not directly consumed, they are valuable nonetheless because of the intrinsic interconnection between wetlands existence and utility of those who enjoy them (even if they enjoy them remotely). The value of wetlands for the provision of quality water could also be estimated in this manner.

5.2. Accounting for Methodological Inconsistencies

Even if the designation problems described above are reconciled by careful description of criteria for inclusion into a MA, there are other conceptual challenges related to commodity consistency that must be addressed during study selection. A households’ WTP for an environmental good or service could be affected by factors other than the differences noted in the structure of the commodity in question. Two major concerns should be addressed—first, consistency with respect to the degree of change, payment mechanism or policy process, and population specific contexts, and second, consistency with respect to the actual economic construct being estimated.

The first concern is primarily an issue of study design and research context. Methodology is known to affect valuation outcomes. Even within methodological contexts, different treatments can cause divergence in estimation results. For example, Brouwer et al. (1999) show that contingent valuation studies estimating wetland values achieve different estimation outcomes based on the elicitation format and the payment mode [10]. It is also suggested that a respondents exhibit different WTP according to their perception as a citizen or consumer [22]. So, even if a MA for benefit transfer is conducted using only one methodology, a valuation function must account for these kinds of differences.

The differences present between methodologies are more than statistical artifacts, alluding to the second concern. Even if valuation studies are seeking to value the same object, methodological differences could cause them to value separate welfare measures. For example, stated preference methods (such as contingent valuation) and revealed preference methods (such as hedonic pricing or travel cost method) are estimating theoretically distinct constructs. A MA should not use all methods indiscriminately, especially when the MA is to be used to transfer values [19]. There are empirical approaches to rectifying some of the discrepancies [23], but a meta-analysis should account for distinctions with respect to the economic construct under investigation, or only consider one construct. Specifically, a MA should distinguish between consumer surplus and producer surplus.

6. Spatial Considerations

Perhaps the largest deficiency in the benefit transfer literature is with respect to the spatial aspects of valuation. Small changes in assumptions regarding the spatial patterns associated with the transfer of value estimates have potentially enormous consequences for the reported value [24,25,26]. Of course, like many issues with benefit transfer, the assumptions that are appropriate for a benefit transfer are context dependent and must be made on an ad hoc basis. However, when collecting data for a MA, careful consideration should be taken regarding the assumptions of the purveyors of the primary research so that it is possible to embody those assumptions in the transfer of benefits. Specifically, benefit transfer practitioners should be concerned with what is assumed to be the spatial extent of the market in the primary research and the degree to which WTP is assumed to change spatially [27].

The spatial extent of the market is difficult to establish for a MA for benefit transfer because of the differences in market extent according to the economic concept of interest. For example, when estimating a resources bequest value or existence value, it is reasonable to incorporate the WTP of people who are relatively remote from that resource. Conversely, when estimating the value of water quality improvements for municipal water consumption, there is an explicitly defined local market (the costumers of the municipal water supply). Beyond delineating the market, there are patterns in the way individuals value certain environmental commodities that are exhibited spatially.

Conscious of these concerns, benefit transfer practitioners have employed geographic information systems (GIS). GIS technology facilitates the incorporation of socio-economic and environmental characteristics that are known to affect the value of land into a model to calibrate the value estimate accordingly. This can be done without GIS methodology. However, using GIS enables the analysis of data that describes land, ecosystems, populations, economies and discrete points that do not share boundaries or jurisdictions, allowing for a spatially explicit mosaic-like representation of the value of an ecosystem or any of the services provided by that ecosystem. Also important is the ability to simulate scenarios that involve explicitly spatial components (i.e., demand, climate change, development scenarios) across and between scales. GIS methodology provides the most promising platform for building and increasingly standardized methodology for benefit transfer.

In general, those using GIS for a benefit transfer will proceed using some variation of the following process: define the geographic extent of the study, determine how the ecosystem is classified spatially (based on the characteristics that define the flow of services from that ecosystem), perform a meta-analysis to establish effect for the characteristics of interest on the estimated value (again, based on the characteristics that define the flow of services from that ecosystem) and apply the resulting valuation function to a land typology dataset in a GIS that would distinguish between ecosystem types and incorporate other variables (i.e., income, substitutes, etc.). This process alone provides an excellent platform for calibrating transferred values, but there are other spatial phenomena that can be incorporated into a GIS in order to more accurately estimate the value of ecosystems services.

Many concerns regarding benefit transfer involve issues of scale. As has been mentioned, the value of ecosystem services is dependent on characteristics of the population to which the services accrue. The size (or scarcity) of the ecosystem and the presence of substitutes impacts the value of that ecosystem or service. This can create challenges when estimating the value of ecosystem services at multiple scales or using policy, planning or environmental change scenarios that have varying impacts between scales. The issue of scaling is an important one, particularly when analyzing scenarios such as the impacts of climate change. Subjects, such as climate change, that have impacts that are local, regional and global require the analysis of changes within and between scales. For example, when evaluating the impact of a climate change scenario, it is not adequate (in most cases) to estimate the value of a change by measuring the independent change in a service flow that could be expected under a given set of changes. Changes that occur about a stock of resources must be considered. This would involve accounting for changes in proximal and substitute resources that would presumably be occurring simultaneously.

Brander et al. (2012) provides useful framework for (what they call) “scaling-up” value changes resulting from climate changes. They propose and employ a framework that incorporates the presence and extent of similar ecosystem sites (substitutes) into a meta-analysis and resulting valuation function. This allows the practitioner to account for expected changes in a stock of resources that are expected to change simultaneously [28]. If details regarding the availability of substitutes can be acquired from the primary studies and incorporated into the value function, these variables are simply incorporated into models using GIS, and change scenarios can be applied that will account for changes in resource of interest based on other relevant environmental changes. In this way, localized changes in ecosystem service values can be modeled in the context of changes at larger scales.

There are other spatial considerations that GIS is particularly well-suited to include in transfer calculations. Much of the early application of GIS for benefit transfer focused on transferring recreation values [29,30]. The inherently spatial aspects of the travel cost method are relatively simple to account for in a spatial data environment, such as GIS, because transportation systems, population characteristics and distributions, and spatially explicit environmental attributes can be included in the same geographic framework.

Other, less often incorporated spatial concerns can be modeled to some degree using GIS. Beyond the usefulness of spatially explicit models for incorporating environmental and transportation data, GIS frameworks can model the concentration and non-linear distribution of populations and the corresponding willingness to pay according to their geographic proximity to the resource and household characteristics. It stands to reason that where an ecosystem produces or facilitates a good or service that is consumed, the consumers of that good or service will be proximate to that ecosystem (in the case of use values). With fewer users as distance from the site increases, willingness to pay decreases. This is a phenomenon known as distance decay, and can be incorporated into value transfers using GIS [31].

The most common approach to dealing with non-constant willingness to pay through space is to incorporate a decay mechanism for demand that is controlled by distance from the site. These mechanisms generally apply a linear relationship between distance and willingness to pay [27]. However, the rate of decay for willingness to pay is non-linear [32]. The estimation errors that result from assuming homogeneity within a market have been shown to be larger than errors that result from methodological concerns which are addressed far more frequently by resource economists [27]. The idea of patchiness in willingness to pay or non-continuous change in willingness to pay is almost absent from the benefit transfer literature. A few exceptions exist.

Campbell et al. (2009) use kriging, a spatial method of interpolation that is facilitated by GIS (for more information on kriging, see [33]), to model how willingness to pay changes based on population concentration and jurisdictional residence (in addition to more conventional variables). They use a survey to obtain data regarding how willingness to pay is associated with the preferences of urban and rural populations and between electoral jurisdictions to produce a spatially continuous willingness to pay estimate for populations across political boundaries in the UK. The estimates suggest that spatial heterogeneity in willingness to pay is considerable between rural and urban landscapes and between regions and that these trends are obscured by traditional distance decay accounting [34].

Johnston et al. (2011) also used stated preference methodology to explore how hot-spots in willingness to pay may result in a mosaic-like demand surface with regional disparities evident [35]. It is suggested that hot spots for willingness to pay for an environmental improvement (in this case, fish habitat) can occur distant from the resource and that simple decay functions fail to accurately represent the true distribution of willingness to pay. The authors advocate for accounting for local differences in preferences for environmental goods (in addition to simple distance decay functions) when performing benefit transfer, which they demonstrate can be accommodated using GIS. However, the use of stated preference methodology may oppose the impetus of benefit transfer. That is, because benefit transfer is commonly used to avoid costly data collection or to complete a valuation in limited time, using survey methods may defeat the purpose of a benefit transfer [35].

This is indicative of a broader concern regarding benefit transfer. There is no standardized method for dealing with the more nuanced issue pertaining to the practice. GIS has the potential make methodology and estimations more consistent because of the widespread availability of spatial data with extensive geographic coverage. The use of a framework for function transfers using standardized data and best practices could produce dramatic improvements in tools for non-economist who are involved in resource management. Of course, because of the drastic deficiencies and lack of consensus in the literature today, any estimates would be useful only for preliminary analysis or project prioritization. But, transferable, semi-automatic decision support systems that use spatially explicit data to calibrate ecosystem service values according to many of the issues addressed in this review have the potential to increase the efficiency of natural resource management, especially for entities that do not have ready access to economic expertise. For this prospect, GIS is a promising resource.

7. Conclusions

It is clear that there is a great deal of progress to be made in the field of benefit transfer before it is widely accepted as a reliable method of resource valuation. Nevertheless, resource managers and policy analysts will continue to use this method because the method is minimally demanding from a resource standpoint. With the use of the method being widespread and the state of the science in its adolescence, the need for advancement in the method is seemingly more pressing than with other methods. While innovation that improves the precision of benefit transfers is of the first order of importance, equally imperative is the need to achieve advancement that can be integrated by policy professionals using broadly available data resources and agreed upon protocol regarding handling the aforementioned complications. This speaks to the need for non-market valuation research, outside the realm of benefit transfer, that is applied and for which data is well documented, so that the research can be adapted for MA and benefit transfer.