Stakeholder Perceptions about Incorporating Externalities and Vulnerability into Benefit–Cost Analysis Tools for Watershed Flood Risk Mitigation

Abstract

Multi-scalar climate hazards in watersheds and growing consideration regarding equity call for innovation in how agencies evaluate and prioritize mitigation and adaptation projects. Benefit–Cost Analysis (BCA) is one approach that is increasingly being applied to decision-making (i.e., FEMA BCA toolkit), but that has not been applied to watershed and equity-based flood management initiatives. This paper addresses this topic and presents a case study evaluating projects for watershed flood and climate mitigation projects by the Louisiana Watershed Initiative (Louisiana, USA). Through semi-structured interviews with stakeholders and practitioners, we found that BCA tool design must be embedded in the program and policy in order to be successfully applied and that equity has not traditionally been a core value of mitigation practice. Even though many stakeholders understand the need for incorporating environmental and social project consequences at a watershed scale, challenges to doing so include inequitable barriers to project design in competitive processes, the complexity of integrating modeling and environmental outcomes data, jurisdictional interests, and the need for better science communication with local decision-makers.

1. Introduction

Hazard exposure in the Gulf South region (LA, TX, FL, AL, and MS) of the United States of America (“United States”) is the product of climatic and geographical conditions interacting with planning decisions that induce growth in risky areas [1]. Socially vulnerable populations are disproportionately exposed to hazards [2], and face greater vulnerability in terms of economic damage, health outcomes, life interruptions, and access to recovery programs [3]. As one of the most socially unequal, flood-prone, and climate-vulnerable places in the United States, the Gulf South will need billions of dollars in flood mitigation and adaptation projects in the next decades [4].

Presently, most flood adaptation and mitigation projects are atomistically designed, conceived, and executed locally, or under state programs aimed at coordinating local efforts, even if funded competitively primarily with federal funds. In the United States, fully integrated planning and management at the regional watershed scale rarely occurs. Most hazard mitigation is driven by local governments, and the actions of neighboring jurisdictions are not always bound by consistency requirements. Due to the decentralization of mitigation policy, incomplete evaluation tools developed for stand-alone projects can systematically bias larger-scale project selection by allowing proponents to overlook externalities [5], such as localized changes to ecosystem services (e.g., water quality, and sedimentation), or other downstream impacts in the planning phase.

In response to problems with fragmented decision-making within inter-dependent hydrological socio-ecological systems, there has been a call for a shift to collaborative, watershed-scale management, and planning [6]. The watershed-scale paradigm in flood hazard mitigation requires coordinated mechanisms to address upstream and downstream consequences, as well as environmental and ecosystem integrity [7,8], beyond the limited localized target areas [9]. This raises governance issues of scale, and scope, beyond those of traditional flood hazard mitigation engineering. Some frame this as a return to older traditional approaches, but regardless of whether it is a novel concept or old wine in new bottles, it presents challenges to current decision-making processes [10].

Additionally, decision-making in current flood hazard mitigation practice is often engineering based and focuses on protecting structures rather than prioritizing social vulnerability, and wellbeing, leading to concerns about the social equity of mitigation practice [11,12]. If watershed management is to be equitable, it must have institutional rules that value addressing the distributional consequences of flood hazard exposure and mitigation decision-making. This is now part of US federal policy via the Justice 40 program, which calls for the consideration of environmental justice, and recognition of ecosystem co-benefits across multiple agencies (“Protecting Public Health and the Environment and Restoring Science to Tackle the Climate Crisis”, EO 13990) [13,14]) (e.g., FEMA and USACE) [15], and using alternative methodologies [16,17] to evaluate flood control and adaptation projects. This program has been accompanied by an accelerated effort to develop strategies and policies to advance nature-based solutions to address climate change, nature loss, and equity [15].

Policy demands for a socially cognizant landscape-level management approach to flood risk mitigation and social equity also imply the need for new integrated decision-making mechanisms. Challenges in socio-ecological fit are common to many policy processes aiming to build more robust and inclusive tools [18] and necessitate the development of tools that facilitate decision-making within large scales and that cross jurisdictional boundaries, which can also give meaningful feedback in potentially disconnected project design contacts. Incorporating these two perspectives requires an evolution in the institutions using decision-making tools and processes used in watershed management and flood mitigation planning, and in the tools themselves when viewed as part of the institutionalization of decision-making rules and processes [19].

There are many tools for watershed decision-making, and a rich literature on multi-criteria decision-making analysis [20,21]. However, this paper focuses on challenges for equitable watershed decision-making within the context of Benefit–Cost Analysis (or cost–benefit analysis, hereinafter, “BCA”), which is a formal economic means of assessing net benefits and can also be a means of ranking and comparing projects based on monetized, and thus normalized, values, making it a useful evaluative tool [22]. There are regulatory requirements that federally funded projects be cost-effective, meaning they have benefit–cost ratios greater than one (1). Proposed flood mitigation projects in several agencies (e.g., the United States Army Corps of Engineers (USACE) and the Federal Emergency Management Agency (FEMA), for example, as well as federal grant programs (e.g., Flood Mitigation Assistance (FMA) Grant, Building Resilient Infrastructure and Communities (BRIC), Hazard Mitigation Grant Program (HMGP), etc.) and recovery programs (e.g., Individual and Household Program Assistance) must adhere to these rules. These programs require the application of BCA, and often the default tool for local projects applied is the FEMA BCA toolkit; although, other agencies, e.g., USACE, have other BCA tools.

BCA is generally used as a threshold tool in the flood management realm. Given that it is used as a threshold tool for funding, it is used internally by project proponents, and the required and potential monetization of different classes of costs and benefits can powerfully guide project strategy and conceptualization. Part of the challenge for using current BCA tools (e.g., the FEMA BCA Toolkit 6.0) is that they are not specifically designed for watershed decision-making, nor to address disparate impacts of flood vulnerability or project consequences. The tools themselves unintentionally promote decision-making, focusing on the localized effects of flooding, prioritizing protecting structures, and moving water away from a narrowly defined target area that is relatively easy to calculate and incorporate into the analysis. Flood-risk BCA practice traditionally focuses on avoided damages, implicitly limiting consideration about whether assuming the distribution of the effects is equitable [23,24]. There have been recent changes in BCA policy meant to address Justices 40 in the processes used by agencies managing mitigation programs (e.g., FEMA), but these do not address the fundamental gap in current BCA tools addressing watershed mitigation contexts, nor do they fully address externalities, or distributional questions [25]. These changes are important because they recognize the importance of the evaluation tools for structuring the institutional context around adaptation decision-making, especially in the context of complex, socio-ecological problems, such as adaptation and mitigation of flood hazards.

1.1. Analysis and Diagnosis of BCA for Regional Watershed Hazards Mitigation within the Framework of Socio-Ecological Fit and Institutional Adaptation to Environmental Hazards

Adaptive management and governance recognize that questions of environmental resilience are context dependent and contingent, and involve interdependencies from many different stakeholder groups, and scales of government [6,26]. This is part of a larger focus on institutional factors within adaptation and mitigation research [27]. Flood hazard mitigation occurs within nested watershed scales, and across different governance and social contexts, often at scales that match those of metropolitan regions, and presents similar challenges to other regional interdependent urban common systems [28]. Seen from theories of collectively governed interdependent systems and commons, the Institutional Design Principles [29,30] and the Institutional Analysis and Development Framework (IAD) [29,31,32], a key means of understanding the role of decision tools is to understand how they act as part of an institutional system along with other contextual factors such as actors/stakeholders, rules, community attributes, types of environmental issue, decision-making criteria, influence action, and adaptation, within the evolution of collectively, or polycentric-governed, environmental systems.

These concepts are often paired with ideas about complexity and resilience [26,33], the literature of which has advanced the concept of institutional fit to socio-ecological fit (SES fit). They are important for BCA research, because the structure and use of the tool, and ultimately its suitability, are a question of its fit to a particular context. Here, we researched the decision tool and SES fit interface to provide an institutional perspective on BCA tools and their impact on decision-making for resilient watersheds [34]. SES fit builds on institutionalist theories of adaptive management and complexity thinking in resilience research and posits that effective practices should represent ‘response assemblages’ that build off the bio-physical, social, and geographical scales [35], where the IAD framework highlights the importance of embedded rules, relationships, and other institutional factors [36].

The socio-ecological fit of decision-making tools that can support an effective response assemblage in a flood hazard mitigation context, requires tools and institutional structures that can address issues of social values, biophysical systems, and economic mandates as a “response-in-context”. They are part of the rules-in-place and decision-making criteria for watershed governance institution building, where collaboration is important and depends on trust, access, and decentralized decision-making from a range of stakeholders about a common interdependent system [6,37]. BCA is not exempt from the need to provide a response-in-context that can assess costs with some recognition of the complexity of interactions and values at different scales [35]. This must be tempered by the realities of mandates for administrative efficiency and public accountability. Thus, a potential problem with large-scale BCA tools as discrete generalized instruments, is that they may not represent a good socio-ecological fit for the social, scalar, and institutional context of decision-making for which they are deployed. However, as they are deployed often as part of national programs, transaction costs for bespoke decision-making instruments are very high.

Challenges to the fit of the BCA instrument can present tensions in terms of divergent values, such as accessibility of use, and ability to address important questions for social welfare, such as distributional consequences and environmental externalities [38]. Part of the problem of BCA in a watershed context is the difficulty of addressing externalities, especially the environmental costs and the consequences of mitigation decisions that involve moving water downstream faster to reduce risk in target areas. Another issue is that technical evaluation and decision tools can represent a barrier to communicating key concepts, and to applying for funds, or otherwise participating. BCA requirements can shape project ideation, participation in the design process, cost of project design and applications, and the process of evaluating projects. This is important for flood hazard management because watersheds are a form of a shared interdependent socio-ecological resource system, a type of commons [39], and must be able to build institutions where many different actors can participate in common forums [6] to govern coupled natural and infrastructural systems [40].

1.2. Case Context

The Louisiana Watershed Initiative (LWI) is an ongoing restructuring of Louisiana’s flood hazard governance from a fragmented local context to one that considers the socio-ecological importance of watershed regions, and the interdependence of actors within them regardless of jurisdictional boundaries. In Louisiana, mitigation and planning actions across varying levels of government are largely uncoordinated and sometimes inconsistent with resilience best practices [8]. Moreover, the isolation of decision-making prevents the utilization of comprehensive floodplain management approaches to address downstream impacts [8].

LWI’s programmatic architects diagnosed that mitigation and planning actions across varying levels of government in Louisiana were uncoordinated and sometimes inconsistent with resilience best practices, and the isolation of decisions prevented the utilization of comprehensive floodplain management approaches to address downstream impacts [8]. LWI aims to build capacity at the regional level within a context of decentralized local government mitigation and land-use authorities in the State of Louisiana, USA, which is also one of the most flood and climate-vulnerable places in the United States [41]. LWI emerged after major flood events in Louisiana in 2016, just over 10 years after Hurricane Katrina destroyed many coastal areas and inspired a Coastal Master Plan. The 2016 flood caused more than 10 billion dollars in damage and brought the problem of flooding “inland from the coast” and into a wider context in the state [42]. These events not only exposed the key deficiencies in flood mitigation but also confirmed that Louisiana can no longer rely on isolated plans and policies that fail to operate at a larger scale.

The executive order creating LWI (EO JBE18-16, 2018) required five state agencies to participate in a Watershed Council tasked with coordinating statewide flood risk management efforts through a regional watershed approach, centering on cross-jurisdictional decision-making. The state’s flood mitigation strategy is now organized around and grouped into eight identified watershed regions, which should create regional governance models, watershed plans, and recommendations for selecting and prioritizing regional watershed management projects between 2021 and 2026. This process is initially funded by Community Development Block Grant Mitigation funds (CDBG–MIT), requires evaluation of cost-effectiveness, and funds benefit low and moderate-income communities. CDBG-MIT projects do not have to use a particular BCA tool, but the FEMA BCA Toolkit is the most common because localities often seek funding from a mix of FEMA and HUD (CDBG) sources [20]. From a practical standpoint, much of the interview process focuses on the FEMA BCA toolkit because of its key gatekeeping role within federally funded projects at the local level. LWI funds projects competitively, using different multi-criteria scoring systems, but these mechanisms are not capable of directly normalizing (via monetization) project impacts or making standardized comparisons among projects. The current research stems from collaboration among watershed managers, and academic researchers, aiming to build tools for decision-making that can address the need to rank and compare a range of mitigation projects at the regional level. These may range from traditional channel alteration to nature-based solutions, from rural to urban contexts, and greatly in their magnitude and scope.

1.3. Aims of Study

This paper aims to describe challenges in institutional fit related to developing and implementing more equitable and watershed-appropriate BCA tools for allocating funds and selecting hazard mitigation and adaptation projects. This is a qualitative paper intended to gain insight in terms of technical challenges for tool development, questions regarding how the administrative process and BCA rules affect the use of watershed project evaluation tools, and subsequently questions of resource management and planning. These insights should help explain to what extent existing BCA tools fit the policy goals of integrated watershed management, such as equitable distribution of costs and benefits, and preserve the integrity of natural functions. The research should also help answer questions of where there might be challenges in building better BCA tools within a complex regional decision-making framework.

BCA might be viewed very differently by local project proponents, engineering practitioners, policymakers, and members of project selection and evaluation teams. The challenges of building better tools may have complex tradeoffs in terms of watershed hazard mitigation, and tool design may require considerations across governance and stakeholder levels, as well as balancing questions of scientific uncertainty, user complexity, program timeframes, and project development and cost burden. We were unsure how LWI stakeholders viewed current BCA tools compared to their relative roles in the governance process, how the applicants view proposals for more watershed-specific tools, and how changing BCA might impact the institutional gatekeeping role of project design within larger complex watershed management processes.

In this study, we approach this research by framing these questions from an IAD framework and using its institutionalist approach to diagnose issues that will address decision-making tool design and rules development by examining the perceptions of stakeholders regarding the structure, form, content, requirements, scope, informational input, and implementation of BCA within a common watershed management context.

2. Materials and Methods

Our data collection approach focused on semi-structured interviews, open-ended questions, and conversation-specific follow-up probes, which provided the potential to explore specific concerns, areas of expertise, or perspectives of the interviewees [43]. The design of the semi-structured interviews aimed to gather introspective data on respondents’ personal experiences while also asking structured questions standardized across respondents. The research team focused on questions organized into two broader aspects: (a) Administration, concerning administrative and programmatic concerns involving BCAs in structured mitigation decision-making procedures, both for applicants and administrators; and (b) Tool Development, the practice-based and subject matter expert perspective on technical questions regarding incorporating externalities (such as water quality and downstream effects) and equity and broader distributional consequences of projects. These questions were organized and framed based on the institutional fit and IAD framework.

The project team conducted over fifteen (15) in-depth interviews, (

Table 1. Summary Table of Interviews.

Organization Type(s)	Duration	Number of Participants
Environmental and Engineering Consultants	1 h 17 min	2	Total Participants = 13 Total Duration = 6 h 22 min
	1 h 10 min	2
	1 h 25 min	5
	1 h 30 min	2
	60 min	2
Environmental Think Tank	24 min	1	Total Participants = 4 Total Duration = 1 h 50 min
	1 h 26 min	3
Federal Agency	55 min	1	Total Participants = 4 Total Duration = 1 h 55 min
	60 min	3
State Agency	1 h 27 min	3	Total Participants = 11 Total Duration = 3 h 3 min
	1 h 36 min	8
Regional Coordinators	1 h 14 min	3	Total Participants = 3 Total Duration = 1 h 14 min
Local Government	1 h 27 min	3	Total Participants = 3 Total Duration = 1 h 27 min
Academic	60 min	1	Total Participants = 3 Total Duration = 2 h 10 min
	1 h 10 min	2

), to better understand the perceptions, challenges, and opportunities. Among the fifteen (15) interviews, eight (8) were in-depth interviews and seven (7) were focus group interviews. (The in-depth interviews and focus group interviews were separated based on the number of participants. If the number of interviewees is one or two, it is considered to be an in-depth interview, if the number of interviewees is three or more it is considered to be a focus group interview.) Team members pilot-tested the semi-structured interviews with respondents before finalizing questions, which helped with important feedback on the length, format, meaning of the question, and responsiveness of participants [43]. These interviews constituted scoping discussions with key informants both in terms of LWI administration, and hazard-related BCA in the Gulf South.

2.1. Participant Selection

Semi-structured interviews often rely on nonprobability samples (a selection of people who are not randomly chosen) because the stakeholders included have a particular skill or point of view relevant to the research being conducted [43]. Accordingly, participants were recruited for this study due to their expertise and involvement with issues related to watershed management, flooding, floodplain management, risk mitigation, or community resilience. These included forty-two (42) participants ranging from federal and state agencies, local governments, environmental and engineering firms, think tanks, and academic institutions (Table 1). The research team used a purposive sampling strategy but aimed for heterogeneity in the sample. The interviews included between 1 and 8 participants and typically had lengths of between 60 and 90 min.

2.2. Interview Process

To conduct the semi-structured interview, an interview guide was developed that includes broader views of framing questions that emerged in the scoping discussion.

Table 2. Summary of Question Topics in Focus Group/Interview Process.

Administration	Tool Development
Experiences with the first round of LWI Proposals—Applications	Data needs and knowledge gaps while incorporating ecosystem and hydrological modeling
Experiences with the first round of LWI Proposals—Review	Demonstrating standard benefits and costs by project type
Challenges for rural and low-income communities	Documenting downstream and upstream water impacts
Impacts of program rules and guidelines	Incorporating negative consequences as costs in BCA (downstream communities, ecosystem function, and coastal resources)
Procedural issues in developing project proposals	Incorporating Social benefits and costs
Feasibility of use of the tool for applicants	Linking equity and EJ (Environmental Justice) goals to BCA and Hydrologic and Hydraulic (H&H) Modeling to BCA
Local dynamics and the process of project formulation	Other promising tools or approaches
The process of project development and its link to the BCA Tool	Quantifying the spatial distribution of impacts
Administering review of tool inputs	Structural elements of BCA that may disadvantage LMI populations
Addressing different capacities of applicants

summarizes the interview guide, which is the questions discussed in the fifteen (15) interviews and focus group sessions. The summary of the question topics was divided into two broader categories (1) Administration and (2) Tool development.

For those interviewees who had first-hand knowledge of the LWI’s First Round of funding or programmatic operations, specific questions regarding their experiences were asked about their perceptions of the program. Other interviews focused much more specifically on practitioner perspectives or experiences, which sometimes were more focused on the administration of BCA programs, questions regarding synthesizing modeling, methods, and data, and others on the practice of BCA development and experience with existing tools. Follow-up questions or probes were not predetermined; however, it was required for clarifications to stretch out details to the established topics of interview questions.

All these interviews were conducted online using Zoom software. A data protection plan was developed and maintained to keep personally identifying information separate from data responses. The recordings and transcripts of the interviews were saved followed by the data protection plan. This study received Institutional Review Board (IRB) approval and each interview was conducted providing an informed consent document to participants.

2.3. Interpretive Approach

Content analysis with thematic coding is widely used in qualitative research as it provides means for discovering the practical understanding of meaning and actions [44]. In this study, the team used two types of processes, the first is our major categories of inquiry, and questions were formed by an initial literature review and the institutional fit and IAD framework, (see Table 2). While we had general questions and assumptions, we lacked analytic questions about the data based on specific contextual details and perceptions from stakeholders. We used a grounded theory approach for developing structured codes based on interview notes, transcripts, and iterative analysis of results. The grounded theory approach is designed to iteratively guide interpretation and code the content as data or text in a form that can be used to address research questions [45], starting with open coding, and then building connections and meaning via axial coding of the first codes. Such active and analytic involvement of the researchers encourages the development of theoretical sensitivity as researchers begin to ask analytic questions about the data. Moreover, line-by-line coding prevents researchers from imposing their predetermined categories on the data. It is the detailed and systematic approach to the interpretation of content to identify patterns, themes, assumptions, and meanings [44]. This iterative approach helps to develop coding based on the similarities, dissimilarities, and patterns while going through the transcriptions. This approach provides a means for discovering the practical understanding of meanings and actions [44].

The categories the team developed from the interview questions incorporating a major portion of the conversation (Figure 1) are (1) Administration and (2) Tool development. We used a two-level coding approach (Figure 1). First, we found two major categories of discussion and content, open coding, and put them into a thematic structure, with four major themes, axial coding, under which we conducted more specific content coding and interpretation. These major themes included:

▪

Opportunities and challenges from practice regarding environmental co-benefits/costs and social co-benefits/costs;

▪

Opportunities and challenges from practice regarding equity and distributional consequences;

▪

Integrative or cross-cutting challenges;

▪

Administrative and programmatic challenges and recommendations.

We also divided our analysis again to reflect a specific discussion of tool development and the administration of benefit–cost analyses within LWI. Specific issues or content questions were then organized by theme to reflect the discussion framed under these conceptual categories. Specific codes were created that could be used in different categories, and reflect more specific content (e.g., Regulations, Property value, Disadvantaged community, Lack of quantifiable data, Environmental benefits, Externalities, etc.). Later, each code was mentioned based on how many times it has been addressed in the interviews. Appendix A 

Table A1. Summary of Common Interview Codes by Research Question Themes.

Key Takeaways and Recommendations		Interview Codes (N ≥ 4)	Interview Codes (N ≥ 2 and N < 4)	One-Offs.
Challenges from Practice Regarding Environmental Co-benefits/costs	Administration	1. Lack of Quantifiable Data (5) 2. Water Quality (4)	1. Channel Design (3) 2. Air Pollution (2) 3. Statewide Programs (2) 4. Ecological Impact (2)	1. Develop Guideline for Frameworks 2. Technical Assistance 3. Timescale 4. Downstream Impact 5. Benefit Area 6. Spillover 7. Externalities 8. Different Project Categories 9. Ecosystem Services 10. Project Evaluation/Reviewers 11. NBS 12. Regulations 13. Applicants need to know the overall condition of the project 14. Environmental benefits 15. How to aware applicants (Community meetings or scoping access)
	Tool Development		1. Tools Need to be Updated (2)	1. Modeling
Opportunities from Practice Regarding Environmental Co-benefits/costs	Administration	1. Channel Design (4)		1. Modeling 2. Design Improvements
	Tool Development	1. Tools Need to be Updated (5)	1. Upstream/ Downstream Area (3) 2. Water Quality (2) 3. Ecosystem Services (2) 4. Benefit Area (2)	1. Different Project Categories 2. Lack of Quantifiable Data 3. NBS 4. Project Evaluation/Reviewers
Challenges from Practice Regarding Equity	Administration		1. Property Value (3) 2. Income Level (3) 3. Rural Community (2) 4. Stakeholder Involvement (2)	1. Disadvantaged Community 2. Population Density 3. Equity 4. Renters 5. Lack of Quantifiable Data 6. Technical Assistance 7. Infrastructure Investment
	Tool Development
Opportunities From Practice Regarding Equity	Administration	1. Disadvantaged Community (4)	1. Structure Replacement Costs (2)	1. Justice40 2. Equity Weights
	Tool Development			1. Tools Need to be Updated
Challenges from Practice Regarding Social Co-Benefits Costs	Administration			1. FEMA BCA Only one time payment, and not annualized 2. FEMA BCA Misses other Factors (ability to recover, waiting for insurance, recovery of the locally owned business)
	Tool Development		1. FEMA BCA Mental health and Productivity (2)	1. FEMA BCA Threshold 0.75 to add social benefits
Opportunities from Practice Regarding Social Co-Benefits Costs	Administration			1. Quality of Life and Wider Community Benefits (2)
	Tool Development
Discussion of Integrative or Cross-Cutting Challenges	Administration		1. Benefit Area (3) 2. Project Evaluation/Reviewers (2) 3. Project Costs (2)	1. Externalities 2. Lack of Quantifiable Data 3. Regulations 4. Partnership among Cross Jurisdictions 5. Structure Replacement Costs 6. Different Project Categories 7. Rural Capacity 8. Local Match 9. Technical Assistance
	Tool Development		1. FEMA BCA Threshold 0.75 to add social benefits and environmental benefits (2)	1. Modeling
Administrative Recommendations	Administration	1. Partnership among Cross Jurisdiction (6) 2. Collection of Data (6) 3. Technical Assistance (5) 4. Application Process (4)	1. Specify the Criteria (2) 2. Project Evaluation/Reviewers (2) 3. Timescale (2) 4. Clear objective for Projects (2) 5. Benefit Area (2)	1. Regulations 2. Local Perspective 3. Financial Support 4. Scoring Criteria
	Tool Development	1. Tools Need to be Updated (5)	1. Modeling (3) 2. Simplify the Tool (2)
Administrative Challenges	Administration	1. Collection of Data (4)	1. Financial Support (3) 2. Lack of Quantifiable Data (2) 3. Application Process (2) 4. Clear objective for Projects (2) 5. Project Costs (2) 6. Awareness among Applicants (2) 7. Technical Assistance (2)
	Tool Development	1. Tools Need to be Updated (6)	1. Modeling (2)	1. Partnership among Cross Jurisdictions 2. Water Quality 3. Downstream Impact 4. Benefit Area 5. Local Match

summarizes all the coding from the interview where N represents the number of times the codes were mentioned in the interviews. In Appendix A Table A1, one-offs are the codes that have been mentioned only once. For example, under the theme Opportunities from Practice Regarding Equity, the code ‘Tools need to be updated’ was discussed once but under the theme Opportunities from Practice Regarding Environmental Co-benefits/costs; it has been discussed five times. The team has summarized the themes (Appendix A,

Table A2. Summary Matrix: Themes.

Key Takeaways	General Themes Environmental and Social Project Consequences
Challenges Ecosystem Benefits/Costs (Spillovers)	1. Available tools need to be standard to quantify ecosystem services. 2. No tools are available to quantify both grey infrastructure and green infrastructure. 3. Individual data are required for acquisition. 4. There are not enough quantifiable data within natural settings to account for water quality. 5. There are not enough data to correlate how flood control projects have water quality benefits. 6. A guideline is required to include the cost of carbon. 7. FEMA BCA only captures positive externalities. 8. FEMA BCA pulls in social and environmental benefits (floodplain, wetland, open park, etc.) only if the benefit–cost ratio is 0.75. 9. Target Area is required to quantify ecosystem services. 10. The value of restored wetland or habitat-specific values is hard to quantify. 11. Impacts do not become apparent for decades (e.g., species population, benefits are intangible). 12. No project sufficiently addresses downstream impacts. 13. A concrete line or channel might create an ecological nightmare. 14. Channel design is key for impacting water quality (positive or negative). 15. Channel work leads to a big loss of functions (e.g., floodplain connection, physical–chemical processes, natural functions). 16. Statewide modeling only addresses risk from a flood management perspective; additional efforts are required to evaluate NBS and their impacts on project development. 17. Statewide programs cannot measure the relative effects of different projects across states.
Opportunities Ecosystem Benefits/Costs (Spillovers)	1. FEMA included Ecosystem Service Benefits based on land use types in 2013. 2. USACE is developing tools to quantify the economic value of ecosystem services. 3. NBS group has put projects into different types of categories. 4. Heat Island, Air quality impacts, land-use environment, and area of benefits can be used to quantify NBS. 5. LWI models will evaluate downstream impacts using a functional assessment tool (such as for stream mitigation).
Challenges Social Benefits/Costs (Spillovers)	1. FEMA BCA threshold 0.75 to add social benefits. 2. FEMA BCA only one-time payment, and not annualized. 3. FEMA BCA misses other Factors (ability to recover, waiting for insurance, recovery of the locally owned business). 4. FEMA BCA only addresses mental health and Productivity.
Opportunities Social Benefits/Costs (Spillovers)
	General Equity Themes
Challenges—Accounting for Distributional Equity and Vulnerability	1. Infrastructure investment should prioritize poor and disadvantaged communities. 2. Population density does not favor rural communities. 3. Engineer firms focus on a higher density of population and do not want to invest in areas with lesser resources. 4. If the focus is on property value, anyone who does not own a property is left out. 5. There is a need to highlight those areas that have been underinvested in. 6. Equity is not static, think about how it is changing over time, and might be driven by amenities and past disasters. 7. Renters are a vulnerable group. The appraised value of a rental is not useful. 8. In FEMA BCA, the bigger the house, the more damage, and the more benefits one can obtain. 9. Smaller communities do not have the capacity/funding for BCA-based proposals.
Opportunities—Accounting for Distributional Equity and Vulnerability	1. NSF and NOAA, bringing the less privileged group or the disadvantaged group to discover the connection with projects. 2. Justice40, redefines a disadvantaged community very broadly, working on tools to identify these communities.
	Program Administration
Tools	1. The tool must be simple. 1. The tool should have a provision to link to the HECRAS 2 model. 3. FEMA may provide technical assistance to the community that will accelerate the application process. 4. BCA tool should be internalized at FEMA; it should not be a burden on communities. 5. When FEMA BCA reaches a value of 0.75, it will add ecosystem services, and social benefits. Once it reaches one (1), it is cost-effective.
Applicants	1. The approach to the application needs to be simple (e.g., spreadsheet with criteria ranking and scoring based on logic). 2. Be more transparent in terms of the types of impacts to account for during project application. 3. Changing the mindset of the applicants about the downstream effects: Not much discussion is carried out with applicants in this context.
Administrators/Reviewers	1. LWI’s Important role is to try and solve the problem of the data/modeling/technical gap when everything else does not stack up on a grant. 2. Application process needs to have instructions on model use, access, and incorporation. 3. Rounds of funding are tied to certain project types. This pot of money is for NS, this is for NBS, to force people to think about these project types. 4. Understand the project from the local perspective. 5. Difficult to obtain data to undertake a proper assessment of local information (assessment, parcel, structure) across parishes. 6. Access to relevant and good data is the key. 7. Change scoring criteria to reflect externalities. 8. During application, BCA may be only 25% of the project’s total score.
Communication and Collaboration	1. Qualitative Multi-Jurisdictional flood risk reduction would be helpful. 2. Communities do not want to spend money outside of their boundary. 3. Need a statewide regulatory framework. Need consistent definitions for FPM that are inclusive of NFIP, permitting, development, and models. 5. Encourage joint projects/collaborations between locals and provide a scoring bonus. 6. LWI Regional Modeling: Guidance may need to be more precisely defined (i.e., boundary conditions). 7. People are uncertain about the funding formula and losing control of watershed-level projects.

) to understand the different contexts of the conversations and to better reflect the perceptions of the interviewees.

Working within this structure, the team developed a specific understanding based on “coding and categories” from the transcriptions of the interviews. The first step in this process involved conceptual mapping and creating a SWOT table (Appendix A, Figure A1), as well as initial compiling of notes and discussion among interviewers in an initial summary table (Appendix A, Table A2) of results. Once this was completed, the team reviewed notes, answers, and key interviews and later conducted an interactive process of thematic organization, discussion, and analysis. (Additionally, the research team was simultaneously conducting stakeholder workshops regarding the use of benefit–cost analysis in watershed decision-making and results from feedback, and collaborative exercises in these workshops assisted in triangulating conclusions and interpretations of interview data.)

3. Results

This section reports results from interviews in terms of the key codes and themes identified in each major question section of our research, based on the question-related categories discussed in the methods.

Table 3. Summary of the most mentioned codes from the interviews.

Themes	Categories	Interview Codes (N ≥ 4)
Challenges from Practice Regarding Environmental Co-benefits/Costs	Administration	1. Lack of Quantifiable Data (5) 2. Water Quality (4)
	Tool Development
Opportunities from Practice Regarding Environmental Co-benefits/Costs	Administration	1. Channel Design (4)
	Tool Development	1. Tools Need to be Updated (5)
Challenges from Practice Regarding Equity	Administration
	Tool Development
Opportunities From Practice Regarding Equity	Administration	1. Disadvantaged Community (4)
	Tool Development
Challenges from Practice Regarding Social Co-Benefits Costs	Administration
	Tool Development
Opportunities from Practice Regarding Social Co-Benefits Costs	Administration
	Tool Development
Discussion of Integrative or Cross-Cutting Challenges	Administration
	Tool Development
Administrative Recommendations	Administration	1. Partnership among Cross Jurisdiction (6) 2. Collection of Data (6) 3. Technical Assistance (5) 4. Application Process (4)
	Tool Development	1. Tools Need to be Updated (5)
Administrative Challenges	Administration	1. Collection of Data (4)
	Tool Development	1. Tools Need to be Updated (6)

summarizes and represents the structure of the coding results by the four (4) major themes mentioned above, divided into two categories, ‘Administration’ of BCA programs and technical discussions regarding ‘Tool Development.’ The interview codes were italicized to emphasize them in the description. Table 3 also mentioned different interview codes developed while going through the interview transcripts, but it only represents the interview codes mentioned more than four (4) times. A complete list of the interview codes was included in Appendix A Table A1.

The following section describes the overall results from each of the sections delineated in Appendix A, Table A1; and Appendix A, Table A2.

3.1. Administration

In the administrative section, questions focused on the application requirements and preparation regarding flood mitigation projects, experience with the LWI project application and review, and impacts of program rules and guidelines on the application process and selection process.

3.1.1. Opportunities and Challenges from Practice Regarding Environmental Co-Benefits/Costs

Many administrators expressed their concerns about evaluating projects for impacts on environmental services due to a lack of clear cause-and-effect quantifiable data or established methods and practices. Lack of quantifiable data was mentioned five times among the interviewees, and they emphasized that due to a lack of data, applicants fail to quantify the benefits of Nature-based solution (NBS) compared to gray infrastructure projects. Moreover, the lack of quantifiable data impeded the application process for the disadvantaged community as they lack the resources (i.e., technical assistance and financial support) to apply for flood mitigation projects.

Several groups mentioned that traditional flood mitigation approaches such as channel alteration projects or other grey infrastructure projects seldom address issues such as water quality (mentioned four times) in the upstream and downstream areas or the major loss of functions (i.e., floodplain connection, physical–chemical processes, and wetland preservation), air pollution (mentioned two times), ecological impact (mentioned two times), etc. While discussing project selection, it was addressed that applicants must be able to quantify the impacts of the channelization project (mentioned three times) compared to NBS. This will help both the applicants and the reviewers to understand the benefit–cost ratio of each project and, thus, it will impact the decision-making process while considering the feasibility of the project. Showing environmental benefits requires extra effort by applicants and may not be within the expertise of existing consultant relationships, and many projects have been under development for many years without these considerations. For LWI’s decision-makers, who come from several agencies, there may be different perspectives on what should and should not be considered as a co-benefit or an impact. However, several interviewees in the BCA field noted that these issues come up during the environmental review and can cause project delays and the need for re-design for projects that have positive BCA ratios. The limitation of statewide programs was mentioned twice in terms of their inability to address the risk associated with flooding; it must be addressed in the watershed’s context. Apart from all these topics, different project categories, developing guidelines for frameworks, regulations, the timescale of the project, and design improvement as well as the spillover impact of the project were discussed among different interviewees.

3.1.2. Opportunities and Challenges from Practice Regarding Equity and Distributional Consequences

Equity comes into the discussion as a question of the distributional impacts and vulnerabilities of the disadvantaged communities who are marginalized, underserved, and overburdened by flood events’ impacts. Equity is a programmatic function in terms of how the program’s structure influences who and how a project may be proposed or ideated in response to different problems. Most of the time project conceptualization and ideation reflect local political power and its priorities regarding disadvantaged communities, low-income communities, and rural communities (mentioned two times). During the interview, disadvantaged communities were mentioned four times, and property value and income level were mentioned three times. It was mentioned that the FEMA BCA toolkit does not have the mechanism to address the property value (mentioned three times) or income level (mentioned three times) of the community accurately. For example, the impact of $100 flood damage is not equal for a person with a $1000 monthly income compared to a person with a $10,000 monthly income. Moreover, if the focus is only on property value or property damage anyone who does not own the property will be left out. Most infrastructure investments focus on a higher density of population and do not invest in areas with lesser resources. Thus, population density, property value, and local match play a significant role in the project application and sometimes work as hindrances for the smaller communities. While talking about FEMA BCA, it was mentioned that bigger houses could address more damage, which ensures benefits, whereas people without property such as renters often do not receive any incentives from these grant programs. Most of the time smaller communities do not have the technical assistance, quantifiable data, or local match to apply for a BCA-based proposal. However, it was also mentioned that NSF (National Science Foundation), NOAA (National Oceanic and Atmospheric), and USACE are bringing the less privileged group or the disadvantaged group to the discussion and trying to connect them with project proposals. Justice40 is a recent initiative, which helps redefine a disadvantaged community, working on tools to identify these communities and ensuring forty percent of the overall benefits of the federal investment flow to them.

3.1.3. Opportunities and Challenges from Practice Regarding Social Co-Benefits/Costs

While discussing social co-benefits/costs, most of the participants mentioned that the FEMA BCA toolkit only counts two aspects of social co-benefits, which are Mental health and Productivity (mentioned two times). Mental health benefits are based on the number of residents in a household and productivity is based on the number of days missed at the workplace during the flood event. All these benefits are provided one time (i.e., only one-time payment) during the flood event and are not annualized. It was evident from the discussion that, most of the time, the effect of the disaster is more crucial than the disaster itself. During this time people feel more vulnerable and need support to cope with the situation. While accounting for only two social co-benefits, FEMA BCA overlooks factors, such as, the ability to recover is different for different individuals, and the waiting period before insurance is hard to manage for low and moderate-income (LMI) communities. Moreover, aspects such as quality of life (mentioned two times) and health conditions are not incorporated.

3.1.4. Discussion of Integrative or Cross-Cutting Challenges

In this section, interviewees discussed the LWI’s key role to bridge the gap through collecting quantifiable data collection, incorporating models in the BCA toolkit, providing technical assistance to the applicants, implementing effective regulations, encouraging partnership among cross-jurisdictions while addressing local match, and addressing uncertainties regarding project cost (mentioned two times). It also addresses how LWI is more focused on implementing flood mitigation projects from a watershed perspective while identifying the benefits area (mentioned three times). It also mentioned that the application process needs to have instructions on model use, provide available sources to access the quantifiable data, and explain ways to develop operations from other stakeholders. It was suggested that a partnership across jurisdictions to implement flood risk reduction approaches would be more effective. However, interviewees also acknowledge that communities are reluctant to spend money outside their boundaries, stymieing joint projects by sharing project costs (mentioned two times) or collaborations between locals.

3.1.5. Administrative Challenges/Recommendations

Several interviewees mentioned that the key to quantifying benefits and costs is access to or collection of relevant data (mentioned six times) (i.e., data related to flood magnitude and occurrence, damage resulting from flood hazards, available alternatives to minimize flood risks, and the associated cost of these alternatives, environmental externalities associated with the selected mitigation approach and the distributional impact to different communities). When the interviewees mentioned administrative recommendations, they mentioned the collection of data six times, a partnership among cross-jurisdiction six times, and technical assistance five times. Often, a lack of data results in an improper assessment of the benefits and costs of the mitigation project. While addressing the administrative challenges, another topic discussed was the burden of the local match in terms of financial support (mentioned three times) for the smaller communities. The project evaluators/reviewers (mentioned two times) must understand the local perspective and provide sources with the required local information across parishes. It is recommended that evaluators should encourage NBS as a flood mitigation approach by introducing additional scoring criteria for selecting NBS projects. The application process (mentioned four times) needs to be simple and transparent in terms of what type of impacts to account for during project application.

Another pattern that the project team noticed related to the capacity building was the participants’ ability to be able to apply for these opportunities. This included:

• Not having the staff capacity to apply for grants.
• Lacking the resources to hire a consultant to assist with grant applications;
• Lacking the resources to hire a consultant to perform engineering and design work to be more competitive.

3.2. Tool Development

While asking questions related to tool development, codes such as ‘Tools need to be updated’ were frequently discussed. Apart from that, codes such as modeling, upstream/downstream areas, water quality, ecosystem services, benefit areas, and simplifying the tools were discussed by the interviewees. One major discussion from tool development was related to how FEMA BCA toolkit 6.0 only introduces social benefits if the threshold value of the Benefit–Cost ratio is greater than or equal to 0.75. However, recently, this has been changed and the BCA Toolkit has been updated to remove the requirement before Social Benefits are added to the benefits pool. Now, a mitigation action must have benefits of at least $1 before Social Benefits are added to the benefits pool.

In the context of LWI, the program includes regional flood models, but these face two challenges for incorporation into a model. The scale is too large for the typically more local hydrological and hydraulic studies, and applicants must invest to make their models consistent with LWI’s. Beyond the use of data, the burden of application and challenges with a local match for federal funding is a major challenge for jurisdictional watershed mitigation planning in the US.

3.2.1. Opportunities and Challenges from Practice Regarding Environmental Co-Benefits/Costs

While discussing tool development many participants mentioned that tools need to be updated (mentioned five times) so that they could connect to different other models (HEC-RAS, SWMM, etc.). It also needs to have the provision to count for ecosystem services (mentioned two times), upstream/downstream area (mentioned three times), and water quality (mentioned two times) from different flood mitigation projects. While considering the ecosystem services the following issues emerged:

◦

Most of the conversation was about the scarcity of tools that will include both gray infrastructures and green infrastructure (NBS).

◦

The inadequacy of quantifiable data for applicants.

◦

Addressing how to address the downstream impacts of projects in terms of environmental quality, and flood risk.

◦

The BCA Toolkit does include ecosystem services in a limited fashion, but in terms of making a more robust BCA, it faces the challenge of valuing the negative consequences of traditional projects, e.g., harm to water quality and riparian areas related to channel alteration or rip-wrap barriers.

3.2.2. Opportunities and Challenges from Practice Regarding Equity and Distributional Consequences

Several interviewees mentioned in the section that the updated FEMA BCA tool should have a provision to address the distributional impact or equity while addressing the flood risk for different communities. While discussing this, interviewees also mentioned that addressing equity should not be a burden for the community but should be engraved in the toolkit itself.

3.2.3. Opportunities and Challenges from Practice Regarding Social Co-Benefits/Costs

The conversation includes the limitation of FEMA BCA Toolkit 6.0, which is the principal tool used by local governments applying to LWI and includes very minimal community benefits related to interruptions to employment, and mental health damage. Other questions related to the quality of life and health are not incorporated. The issue that arose in that context was the lack of availability of data to measure the social co-benefits more robustly.

3.2.4. Discussion of Integrative or Cross-Cutting Challenges

Initial discussions highlighted the need to address the feasibility of tool administration, both in terms of the capacity for local governments and BCA practitioners to effectively use the BCA toolkit in development and for LWI administrators to interpret and compare the results in a way that would further accomplish programmatic goals. It also addresses issues such as how to incorporate different models into the BCA toolkit. There is a rapid expansion of environmental benefits tools, but few to no negative consequences are monetized. These often focus on nature-based solutions (NBS), which include engineered green infrastructure solutions and natural infrastructure that include protecting existing or restoring natural areas. Impacts of projects are relegated out of the design phase to environmental review.

3.2.5. Administrative Challenges/Recommendations

In this section, interviewees mentioned tools that needed to be updated six times. It shows how important it is to develop a toolkit that will be simple (mentioned two times) enough to be used by practitioners. It also mentioned modeling (mentioned three times), and how to incorporate models (HEC-RAS) into the BCA toolkit. Other topics that were highlighted during the administrative discussion were the partnership among cross-jurisdiction, how to improve water quality, how to quantify the downstream impact, and how to address the benefits area as well as how to ensure a local match for the smaller community.

3.3. Results Summary

We finalized our results section with a series of representative quotations identified in the interviews to provide greater context to the thematic results summaries (

Table 4. Representative Comments by Stakeholders.

Organization	Categories	Interview Codes	Exemplary Statements
Federal Agency	Administration	Property Value	“While focusing on Property value, anyone who doesn’t own a property is left out.”
Environmental and Engineering Consultants		Regulations	“Regulatory pressure is the only mechanism by which one can guarantee that environmental improvement happens.”
State Agency		Ecosystem Services and Social Benefits	“There is a lack of understanding in how a flood risk reduction project could be scoped to be more inclusive of social benefits and NBS overlaid.”
Environmental and Engineering Consultants		Data Collection	“Access to relevant and good data is the key.”
Environmental and Engineering Consultants		Cross-Jurisdictional Collaboration	“Encourage joint projects/collaborations between locals, provide a scoring bonus.”
State Agency		Downstream Impact	“Downstream impacts, no project sufficiently addressed that.”
Environmental and Engineering Consultants	Tool Development	Modeling	“Application process needs to have instruction on model use, access, and incorporation.”
Federal Agency		Tools Need to be Updated	“Current BCA toolkit is limited.”

). The seven quotations denoted below in Table 4, include both major structural issues practitioners face (e.g., prioritization of property protection over measures of human wellbeing, and a few mechanisms for collaboration at the watershed scale), and the importance of details, such as clear instructions that have been vetted with end-users. The comments also reflect common observations, that there are few explicit crossovers between flood models and environmental quality modeling, the existing evaluation processes prioritize property protection, a lack of rules to guide the integration of mitigation practice into wider planning and environmental regulation discussions, and applicants lack incentives to invest at the local level in inter-jurisdictional watershed collaborations or in evaluating non-localized impacts.

4. Discussion and Conclusions

4.1. BCA from the Perspective of Socio-Ecological Fit and Institutional Design

This study has documented questions of socio-ecological welfare, policy, and institutional fit for the use of BCA tools in watershed decision-making contexts. We started with a set of premises that related to the specific goals of LWI, a program that aims to shift management decisions to the watershed scale, by incorporating common hydrologic and hydraulic (H&H) models to promote multi-jurisdictional decision-making. LWI also aims to integrate natural functions into flood hazard mitigation and ensure that projects and plans benefit low and moderate-income communities. This shift in the management paradigm shines a light on the fact that current BCA tools have evolved in a much more piecemeal decision context, primarily focusing on avoided damage to properties rather than more holistic social welfare impacts. BCA has become a cornerstone to evaluate and justify federal funding allocations. Some may favor BCA due to its systematic quantitative analysis providing more objectivity [46].

Building more robust and inclusive tools, including BCA, in the field of sustainability is a larger challenge [27]. Our interviews documented that such challenges in tool development extend to watershed planning and flood mitigation [18]. One challenge is building a systematic quantitative method to prioritize and select projects for flood hazard mitigation based on watershed-level effects to better incorporate the risks associated with health, costs, environment, property, income, and the unequal social cost of flooding. This requires more resources and higher transaction costs due to additional coordination across entities. Our interviews suggested that in practice, the tool creates a process for measuring and demonstrating benefits while shaping the expertise and capacity needed for project design. This institutionalizes certain design practices, project types, and shapes, for better or for worse, for any program using BCA as a threshold or comparison tool in terms of project types and accessibility to communities of different capacities. The tradeoffs in BCA tool design can shape planning and project design around flood hazard mitigation.

4.1.1. Administration Conclusions

Even when scientific data exist, many barriers stymie its usefulness in decision-making [11,38]. The challenge for incorporating BCA into the decision-making process is the embeddedness of the administrative process, which relies on federal funding and is implemented with the governing authority of states and localities in the US Context. Resource-limited local governments often experience delays in funding, high costs of applications, and a lack of relevant data sources, which can pose challenges for state administrators. For the LWI and other similar programs, the need to use more precise data to guide project selection may raise the demand for skills in hazard modeling, GIS, and HEC-RAS. Many stakeholders already perceive that the process of demonstrating costs and benefits can be:

• Expensive: implementing a more sophisticated BCA requires a paid consultant, which many jurisdictions cannot afford.
• Convoluted: it is difficult to understand how/when it is appropriate to use the data.
• Inaccessible: the data sources exist across multiple platforms or agencies, and it can be challenging to know where to go to find the data, and how to integrate them to show the impact of projects or obtain a timely response to aid incorporation.
• Confusing: in some cases, many diverse types of datasets tell contradictory stories about these impacts.

The interviews highlight that developing a sophisticated model with more environmental effects while simultaneously making it easier to use among applicants are conflicting objectives. The issue of applying a standardized BCA framework to a particular program is challenging as it will facilitate the comparison of proposals from a broad group of users to a specific program representing a large diversity of perspectives and capacities.

Equity in BCA can be a product of valuation methods [11,46], but in a more procedural sense, many of our interviewees discussed barriers to inclusive participation in competitive processes. Several participants from different types of organizations note that the BCA phase of mitigation funding programs presents challenges for rural communities and low-income communities, which has also been documented in other recent studies and policy areas such as transportation and housing [38,46]. Additionally, within communities, mitigation projects often originate via discussions between engineering contractors, staff, and elected officials. These relationships often depend on trust and mutual understanding and may achieve de facto gatekeeping roles in terms of project initiation and ideas, refinement, and participation.

New tools and programs aiming to improve the integration of technical structured decision-making [47,48] risk alienating local governments with limited resources. The lack of resources for hazard response in local government exists outside of the US in the Gulf South [49], but our research suggests caution in using BCA within competitive grant programs where prospective applicants face large capacity gaps in terms of demonstrating costs and benefits.

If a program uses a competitive grants process, the evaluation tools, their accessibility, and their implementation inherently reflect programmatic values and policy. If these tools lack key components (e.g., monetizing downstream impacts in a watershed paradigm), or if the tools favor certain design solutions or applicants with greater capacity, the tools may unintentionally bias project selection contradictory to stated policy goals [50,51,52,53,54,55]. The interviews reinforced that addressing these challenges, equitably, and reliably, is difficult. Funding and policy around adaptation and mitigation should create and review processes that can make defendable decisions around limited funding (a strength of BCA), but also foster a design process that addresses and respects the perspectives, capacities, and constraints of applicants.

4.1.2. Tool Development Conclusions

Our literature review shows that even though BCA theory posits that evaluations should include environmental externalities and social welfare impacts, few BCA studies document the distributional consequences of flood hazard mitigation or the net value of positive and negative environmental impacts. Likewise, there is very little research on applying common hydrological models to varied decision-making contexts. That is, comparing the use of models, such as HEC-RAS and SWMM, from a decision-making point of view, outside of the engineering studies. Interviews with stakeholders suggest that BCA practice faces similar challenges due to the gaps in the academic literature. Notably, consideration of distributional consequences is new within the flood hazard mitigation BCA practice.

Regarding tool development, most interviews mentioned the need to develop a tool that will address both the upstream and downstream impacts, water quality and quantity issues, and provide clear measures of equity. Many interviewees mentioned the difficulty in identifying who receives benefits and costs from flood hazard mitigation aside from traditional property and structural damage values. Increasing complexity and modeling boundaries can impose additional participation costs. There may be wiliness to comply with these costs if explained, but the burden can seem punitive if the rationale for including environmental or equity goals is not transparently communicated and dialogued with applicants.

Documenting complexity, such as the environmental externalities and cumulative impacts of many different interconnected decisions, is a challenge for BCA. Firstly, understanding the interactions of outputs from different models, each examining different geographic scales, increases the burden of an applicant to provide a satisfactory analysis. For example, our interviewees mentioned that there is no standard for downstream negative impacts from increased water volume or positive impacts from increased detention. Using the appropriate spatial scale for analysis emerged as another related concern. Current models are limited to their boundary conditions, confining programmatic evaluation and BCA to that scale of impact. Because of this, programs and tools struggle with common regulatory and theoretical definitions and metrics to account for watershed externalities.

Many interviewees from the natural resources area desired more thorough evaluation techniques that take distributional impacts and externalities into account within a watershed. First, numerous competing water quality and quantity models decrease accessibility. (For example, the EPA Green Infrastructure Modeling Toolkit includes applications with different software and models, e.g., PC SWMM, SWC GIWiz, WMOST, VELMA, GIFMod, and CLASIC, i-DST. There are HEC-RAS models used for many flood hazard mitigation applications and PCSWMM models for sub-surface Stormwater. Regarding water quality, in the US, determinations of water quality total daily maximum loads under the Clean Water Act are conducted with another set of modeling tools that are totally decoupled from flood modeling. Integrative models are emerging, but, such as, e.g., RTI’s WaterFALL^®, they are still not widespread.) The same question occurs with water quality issues from sediment, nutrient export, and other issues. Furthermore, our interviewees noted that the impact of different project types is not fully integrated into a scenario process, where they can be more generally inserted into models. This problem exacerbates difficulties in comparing nature-based” projects versus more traditional “grey” infrastructure approaches. In this sense, BCA has not evolved around watershed externalities due to the lack of integrated, or coupled models, which can address the environmental effects of varied project types.

The interviews also highlighted the conflict between the need for administrative tractability for numerous users and the need for more detailed information about impacts on small geographic scales and in terms of how they may differ within groups that experience varying degrees of social vulnerability. Specifically, understanding the distribution of effects from mitigation requires the identification of those who receive benefits and costs, and how to set boundaries. It also requires understanding the way different groups are impacted by flood hazards and environmental externalities, which requires defining, modeling, and monetizing vulnerability for distinct groups, e.g., renters vs. owners, groups of different socio-economic status, etc. This is important because issues of housing precarity, employment precarity, and mental health often triangulate and can be a much greater disruptor for lower-income, underinsured populations [56].

4.2. Study Limitations and Larger Implications for Flood Hazard Mitigation BCA from a “Response-In-Context” Perspective

Our conclusions about flood hazard mitigation decision tools and BCA have limitations. First, we rely on one set of interviews surrounding a single region-specific program related to HUD and FEMA funds within the US. We did not have access to the range and breadth of local actors from local states and federal agencies for a larger review, and we have not focused on BCA programs related to the USACE; although, they were also mentioned. We still believe the documented issues are widely representative of processes within the context of “environmental federalism” in the US, especially for polycentric governance situations that involve a mix of local, state, and federal funding and decision-making. Despite these limitations, these interviews were essential to understanding the perceptions of stakeholders who are using the BCA toolkit as well as other evaluation tools available. These perceptions help clarify the institutional role and limitations of BCA in the LWI context focusing on environmental externalities and equity.

One representative conclusion is that stakeholders seek consistent tools and application processes that can be clear about what information is required, where to look for data, and where to obtain technical assistance for their implementation. If a program, such as LWI in our case, is based around a competitive grants process, the evaluation tools, their accessibility, and their implementation are principal vehicles of programmatic values and policy. If these tools lack key components (e.g., monetizing downstream impacts in a watershed paradigm) or if the tools favor certain design solutions or applicants with greater capacity, then those potential sources of bias will permeate as governing decision rules within the institutional structure of decision-making within the program. These processes may favor problem-solving for higher-status areas and pose challenges to equitable participation.

This project underscores the need for evaluation tools that consider externalities and distributional questions within a watershed, as part of larger “mission-oriented” governance projects around adaptation and hazard planning. In this sense, the major finding regarding BCA is the need for it to integrate with other models of evaluation, e.g., modeling, and multi-criteria approaches. BCA can play a significant role in this larger decision-making context, but it must be designed to fit the socio-ecological context, such that the relative costs and benefits can be demonstrated by planners and associated with the values and technical tools of management programs. Overlooking these contexts can lead to skewed institutional incentives and informational structures within flood hazard mitigation planning programs, especially those with integrative environmental or watershed objectives. This, however, involves potentially higher administration costs, a paradox that other studies have documented [38].

A major aspect of BCA currently overlooked in many flood hazard mitigation processes is the role of institutional rules and processes that shape programmatic communication and rulemaking. In this sense, BCA frameworks, when seen from a policy and design perspective, must go beyond considerations of providing better scores, and also consider the communicative value of the tool regarding planning and policy values. However, to serve this function, tools and competitive evaluation frameworks must be designed with sensitivity to governance capacity for evaluation, and local project proponents and their engineering partners, who develop and submit projects. All these approaches merit attention within future BCA policy and planning research, especially in the flood hazard mitigation and watershed governance context.

Our current research represents a diagnosis of issues for BCA in watershed management and planning. Solutions include greater research on capacity building and the BCA tool-program interface, and on building coupled flood hazard and environmental impact models and associating them with empirical valuation literature. Several initiatives are already underway to solve the capacity issue, focusing on both administration and tool development. Existing programs, (e.g., FEMA’s Building Resilient Infrastructure and Communities (BRIC)) provide design support, via a two-phase process, but they still can leave challenges for questions of “local match” on projects, and generally accept project framing by local officials, rather than emphasizing projects that reflect comprehensive and inclusive planning approaches. Special funds and program criteria have been a part of Justice 40, as have programs related to nature-based solutions [15,17]. These efforts, however, have not been explicitly designed around watershed hazards planning, and there is a need for research and funding for specific watershed BCA tools that can interface with the complex network of social, environmental, and structural data needed to consider watershed externalities and the distributional consequences of mitigation planning and decision-making.

Given that more descriptive BCA tools, which might address environmental externalities more robustly, could impose costs on participants, we suggest that programs internalize many of the data aggregation and modeling processes, and offer additional technical assistance as part of BCA program design. The Iowa Flood Center is a good example of this, where a state-designated and funded organization manages much of the modeling and assistance for watershed projects. However, more modeling and technical support are not a comprehensive BCA tool or package. In the US context, such a tool may need to originate in a federal agency and a program approved by Congress. The reason for this is that funding is often a mix of local, state, and federal sources. States respond to federal spending priorities, so the tools used by states and localities must interface well with agencies such as FEMA and USACE. This poses some challenges to socio-ecological fit in BCA tools in the US, where watersheds are not legally defined regulatory scales at the federal level [57,58]. Thus, building better BCA tools may require defining the watershed geography within federal programs, coupled with state-level integrated watershed statutes.