A Holistic Sustainability Assessment Framework for Evaluating Strategies to Prevent Nutrient Pollution

Abstract

Excessive nutrients from anthropogenic activities have caused eutrophication worldwide. While available assessment frameworks aid in selecting strategies to control nutrients, they often lack a holistic approach that considers social, ecological, and managerial categories to ensure long-term effectiveness for both point and non-point sources. This study addresses this gap by introducing a holistic sustainability framework with ten indicators derived from a literature review and stakeholder engagement. Quantitative and qualitative indicators were defined using either existing or newly designed methods. A weighted sum multi-criteria method was applied to calculate the total score for each strategy, considering indicator levels and weightings. The framework was applied to a case study in Hendry County, Florida, to demonstrate its applicability by assessing ten strategies. The same top-ranked strategy, alum treatment, was identified through this framework after applying 100 weighting scenarios. This demonstrates the robustness of strategy selection using the developed framework. This framework can be applied with limited data by prioritizing inputs related to five major contributors, effective cost, nutrient loading reduction, technology readiness level, benefit and direct impact, and labor operation and maintenance requirement, to the total score. These major indicators highlight the importance of considering social–ecological and managerial categories in addition to technological, environmental, and economic aspects in sustainability assessment of nutrient management strategies.

1. Introduction

Nitrogen and phosphorus are essential nutrients for human health and plant growth [1,2,3]. However, excessive nutrient loading to water bodies results in eutrophication, leading to an increase in the number of dead zones from 400 to 700 between 2008 and 2019 and an increase in the occurrence of harmful algal blooms from 26 events in 2003 to 43 events in 2023 [4,5]. Harmful algal blooms have far-reaching consequences on human health, the economy, and the environment [6,7]. Nutrient management strategies, including treatment technologies and best management practices (BMPs), are implemented to regulate nutrient sources in the U.S. In Europe and Asia, both traditional treatment technologies and emerging nature-based solutions have been adopted to prevent nutrient pollution [8,9,10]. Nutrient pollution includes both point and non-point sources. Point sources are effluent discharged from a single identifiable source, such as municipal wastewater treatment plants, while non-point sources encompass urban stormwater runoff, agricultural runoff, and onsite wastewater treatment systems.

Selecting nutrient management strategies for implementation typically requires a comprehensive evaluation of their performance across technological and economic categories. Technological performance focuses on the functionality and efficacy of a strategy and has been extensively examined in prior studies [11,12,13,14,15,16]. Most of the reviewed literature evaluates nutrient removal/reduction for different strategies, with a specific emphasis on total nitrogen (TN) and total phosphorus (TP) removal/reduction as the key technological indicators [13,17,18]. Many of these studies, however, overlook other technological category indicators, such as technology readiness level and scalability. Beyond the aforementioned technological indicators, economic indicators feature prominently in most reviewed articles, including cost-effectiveness, total cost, capital cost, and operation and maintenance (O&M) cost [12,13,15,16,19,20,21,22,23]. However, nutrient management strategies selected solely based on these common indicators often face long-term sustainability challenges, as their selection lacks holistic consideration of other factors, such as the impacts on ecological systems, managerial requirements, and public acceptance.

Hence, it is imperative to consider not only technological and economic aspects but also social, ecological, and managerial categories when evaluating nutrient management strategies [16]. Several assessment frameworks have been developed that include social indicators, such as public acceptance, number of affected persons, nuisance, and satisfaction of working conditions [13,16,19,20,21,22,23]. However, these social indicators primarily focus on the public or employees, lacking a broader consideration of key stakeholders, such as landowners/farmers. Given that certain nutrient management strategies necessitate the collaboration of various stakeholders, it is crucial to assess their willingness to adopt a strategy before implementation. Limited attention has been paid to managerial and ecological indicators in previous assessment frameworks, with a few exceptions. Notably, J-Tech considered both managerial and ecological categories in their assessment framework for the South Florida Water Management District (SFWMD) [16]. The managerial category assists in selecting strategies with low maintenance requirements, while the ecological category prioritizes strategies that benefit ecosystems. Instead of evaluating O&M from the economic standpoint, J-Tech integrates O&M as a managerial indicator, specifically selecting strategies with less complexity of operation and less operator involvement [16]. In terms of ecological indicators, the framework developed by J-Tech stands out as the only framework assessing the value of the habitat provided for fish and wildlife by the treatment area [16].

From the end user’s point of view, the majority of previous assessment frameworks are developed for government agencies, water utilities, and watershed management organizations [11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]. Some of the frameworks are developed for technology developers/designers to improve nutrient removal/reduction performance in the future [11,13,20,22,23], and a limited number of the frameworks are developed for community members, even though the community is considered as one group of the important stakeholders [11,13]. To the authors’ knowledge, there is no framework to assess nutrient management strategies that considers the concerns of various stakeholders, including both landowners/farmers and technology/BMP maintainers.

Furthermore, most reviewed literature developed conceptual assessment frameworks without practical application. Among the limited applications, most research adopted a fixed weighting scheme. For example, J-Tech evaluated ten nutrient management strategies for urban stormwater runoff and selected alum treatment using one set of weighting based on stakeholders’ preferences [16]. However, fixed weighting is limited in capturing diverse contexts and different stakeholder perspectives. Some studies have recognized the limitations associated with fixed weighting and addressed uncertainty by applying various weighting schemes [13,26,27]. Molinos-Senante et al. designed seven hypothetical weighting scenarios to assess seven secondary wastewater treatment technologies [13]. The authors found that constructed wetlands were the most sustainable technology in five out of seven scenarios when economic and social categories were preferred [13]. Lohman et al. designed 1000 hypothetical weighting scenarios to evaluate three sanitation alternatives and found that traditional design had the highest probability of ranking at the top [26]. Hall et al. applied four hypothetical weighting schemes to assess three onsite wastewater treatment systems, and they found that a passive system was the preferred alternative [27]. However, these weightings are hypothetical. Further work is needed to provide practical insights into nutrient management strategies by assigning weighting within a tangible range based on a literature review.

Therefore, this paper aims to develop a holistic sustainability assessment framework for various groups of stakeholders, and this is the first framework considering landowners/farmers and technology/BMP maintainers. This paper also aims to design a framework with tangible weighting schemes for evaluating nutrient management strategies while identifying the major contributor(s) among all indicators to simplify data collection efforts. The developed framework is applied to a case study in Florida to assess the nutrient management strategies and select the top-ranked one.

2. Materials and Methods

2.1. Framework Formulation

2.1.1. Category and Indicator Selection

Category and indicator selection was conducted based on a literature review and feedback from stakeholder meetings. The category and indicator selection procedure are available in Figure 1. An initial search was performed using Google Scholar and environmental-related government websites employing key terms such as “assessment framework”, “indicators”, “nutrient removal”, “technology”, “best management practices”, “stormwater”, “agricultural runoff”, and “wastewater”. It resulted in 980 articles and 232 reports. Rigorous screening, based on relevance to research objectives, was then applied to the titles and abstracts of these sources, with a dual emphasis on technologies or BMPs for nutrient reduction and conceptual or practical indicators for assessment. Following this initial screening, 282 articles and 81 reports were identified for further analysis. Selected articles were further screened by focusing on the availability of nutrient management, indicator quantification methods, and practical applications. Finally, 10 articles and 8 reports encompassing frameworks and indicators for assessing BMPs or technologies pertinent to stormwater, agricultural runoff, and wastewater were reviewed (details about selected articles/reports can be found in Tables S1 and S2 in Supplementary Materials). Common indicators were identified, and some were applied to develop composite indicators. For example, under the technological category, the predominant indicators selected were nutrient removal, including nitrogen removal and phosphorus removal, both were included in 9 out of 17 studies.

Following the initial selection of indicators, the research team organized two stakeholder meetings to gather feedback for refining indicators. The first stakeholder meeting occurred virtually on 21 October 2020, with a total of 31 participants representing diverse sectors, including state agencies (Florida Department of Environmental Protection Agency, Florida Department of Agriculture and Consumer Services, Florida Department of Health), county utilities (Hillsborough County Water Resources Department), regional water management districts (SFWMD), university researchers (University of Florida Institute of Food And Agricultural Sciences), foundations (Water Research Foundation, Everglades Foundation), organizations (Florida Onsite Wastewater Association, Florida Stormwater Association), intergovernmental partnerships (Tampa Bay Estuary Program), and consultants (Hazen and Sawyer). The second stakeholder meeting, conducted virtually on 30 March 2022, brought together 19 attendees from federal government agencies (Florida Farm Bureau Federation), state agencies, counties, and city utilities (City of Lakeland), regional water management districts (Southwest Florida Water Management District), university researchers, foundations, and consultants (Old Castle). The same sectors from the first meeting were not listed in parentheses. These meetings served as crucial platforms for engaging stakeholders and incorporating their insights into the ongoing indicator revision process. Engaging them in the process of the framework development and identification of sustainability indicators also ensures the applicability of the framework to broad groups of stakeholders since previous frameworks were developed for specific stakeholder groups. Details of stakeholders’ information in each reference are available in Table S1 in Supplementary Materials.

2.1.2. Indicator Quantification

The selected indicators were quantified using established methods found in the literature, while the developed indicators were quantified through newly designed methods. For example, nutrient loading reduction, as a selected indicator, was quantified using an existing method as the percentage of the total amount of nutrients removed relative to the total amount of nutrients in the system [11,12,16,17,18,24]. Scalability, recognized as a crucial indicator during the initial stakeholder meeting, was developed to highlight the preference for strategies that can be implemented across various scales. The category of scales was developed based on classifications used for wastewater systems based on the population served, i.e., small scale (less than 40 people), medium scale (40 to 9090 people), and large scale (more than 9090 people) [28]. Scale categorization of stormwater BMP was established using treated area acreage in the 2020 Lake Okeechobee watershed Basin Management Action Plan (BMAP) report, i.e., for stormwater, the three scales are less than 100 acres, 100 to less than 500 acres, and equal or larger than 500 acres [15]. Another significant indicator identified by stakeholders was the technology readiness level, which was quantified using a rating scheme adapted from the literature with an emphasis on actual implementation in the targeted location or state [16,29,30].

The selected and developed indicators included both quantitative and qualitative indicators. For quantitative indicators, such as nutrient loading reduction and effective cost, the indicator values were either directly obtained or calculated through equations based on required inputs. Indicator values were then used as inputs to determine an indicator level using the designated rating schemes outlined in Equation (1). This rating scheme is structured with five levels, with each level’s range defined by the worst and best values of the respective indicator. Qualitative indicators, such as scalability, were assessed using a 5-level rating scheme designed for each indicator where level 5 is the highest and level 1 is the lowest. Indicator levels were determined based on input information obtained from literature or surveys/interviews using the designed rating schemes. More details on indicator quantification are available in Tables S3 and S4 in the Supplementary Materials. For example, a level 5 scalability is assigned to a strategy that can be implemented at any scale (small, medium, and large), while a level 1 is assigned to a strategy that can be only implemented at a small scale.

$${\mathrm{L}}_{\mathrm{i}}=\mathrm{i}, \mathrm{x}\mathsf{ϵ}\left[{\mathrm{x}}_{\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{s}\mathrm{t}}+\left(\mathrm{i}-1\right)\ast \mathsf{\Delta }\mathrm{x},{\mathrm{x}}_{\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{s}\mathrm{t}}+\mathrm{i}\ast \mathsf{\Delta }\mathrm{x}\right],\hspace{1em}\mathsf{\Delta }\mathrm{x}=\frac{{\mathrm{x}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}-{\mathrm{x}}_{\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{s}\mathrm{t}}}{\mathrm{n}},\hspace{1em}\mathrm{n}=5$$

(1)

2.1.3. Total Score Calculation

The total score for a given nutrient management strategy was calculated using the weighted sum method by considering both the level of an indicator and its weighting. The indicator weighting, selected by stakeholders, was determined based on tangible ranges obtained through literature review or assumed based on similar indicators. The tangible range is determined by minimum and maximum weightings applied to each indicator in the literature. A comprehensive breakdown of the weighting range for each indicator can be found in

Table 1. Indicators and weightings in developed framework and SFWMD framework.

Developed Framework.		SFWMD Framework
Indicator	Weighting	Indicator	Weighting
Scalability	5 to 8.33%	Scalable	5
Nutrient loading reduction	3.57 to 25%	Confidence in performance estimates	5
Technology readiness level	5 to 6.67%	Available Florida case study	4
Residual management	5 to 6.67%	Residual production	4
Benefit and direct impact	4 to 21.53%	Habitat	3
Labor O&M requirement	2 to 10%	Ecosystem services	2
Maintenance complexity	3.33 to 5%	Energy efficiency	2
Public acceptance	1 to 8.81%	Land requirements	2
Social participation	1 to 8.81%	O&M	2
Effective cost	5 to 50%	Schedule of implementation	1
		Cost-effectiveness	30
Sum	100%	Sum	60

, and the relative references are available in Table S5. The weighted sum multi-criteria method was chosen due to its widespread use in multi-criteria evaluation. Since the developed framework was intended to be used by diverse stakeholders, this method was preferred due to its simplicity, as shown in Equation (2), and its adaptability in accommodating stakeholder preferences through weightings that signify the relative importance of the indicators [31,32,33].

$${\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}={\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{L}}_{\mathrm{i}}{\mathrm{W}}_{\mathrm{i}}$$

(2)

where S_total = total score of the assessed nutrient management strategy; L_i = level of assessed nutrient management strategy under ith indicator; W_i = the ith indicator weighting; n = the total number of indicators.

2.2. Framework Application

2.2.1. Application Process

Users are allowed to input their own data for indicator value since the framework is designed with flexibility and can be customized. The literature review can inform alternate indicators that may be adopted by future users based on regionally specific priorities for nutrient reduction. To apply the framework, users first input indicator values and then select the indicator weighting based on their preference. Indicator levels are automatically calculated in the framework. The output is the total scores and final ranks for all assessed strategies. To ensure its ease of use, the default data of indicator levels is provided in Table S9.

To facilitate the application of the developed framework, an Excel file is provided in Supplementary Materials. The procedure to apply the developed framework using Excel 2016 is shown in Figure 2. Users first come up with a list of potential strategies for their site after reading the title page and indicator definition tabs. For each strategy, they can either select an appropriate category for each indicator or enter the required input information to calculate the values. For example, for an urban stormwater BMP, users can select from the categories of “small scale (<100 acres)”, “medium scale (≥100 and <500 acres)”, and “large scale (≥500 acres)” based on treated area acreage. In the case that the information is limited for the evaluated strategies, users can identify the indicators of interest from the list in the framework. Once the indicator values are selected or calculated, they are then evaluated automatically in Excel using a 5-level scheme to determine the ratings. Users can assign weightings from 0 to 100% or based on the tangible range provided in Excel to the selected indicators, ensuring the total sums to 100%. All indicator levels and weightings are processed using the weighted sum multi-criteria method to derive the total scores and ranks for all assessed strategies. Although the weighted sum multi-criteria method is straightforward, it can be challenging when dealing with criteria/indicators that have different units. Users can choose to conduct uncertainty analysis by using the tangible range of weightings for each indicator (Table S5). The framework automatically generates 100 scenarios with different combinations of indicator weightings. The strategy that is consistently top-ranked is the best option.

2.2.2. Case Study

The case study aimed to demonstrate the applicability of the developed framework. The results were compared with those from an existing framework to highlight the advantages of the developed framework. The developed framework was applied for selecting stormwater treatment BMPs to enhance water quality in the Caloosahatchee River (C-43) West Basin Storage Reservoir, located in Hendry County, Florida. The results were compared with SFWMD’s framework developed by J-Tech since it encompassed all categories featured in the developed framework [13]. The ten BMPs evaluated are alum treatment, treatment wetland, hybrid wetland treatment technology, Bold&Gold^®, ElectroCoagulation, sand filtration, NutriGone^TM, AquaLutions^®TM, MPC-Buoy, and air diffusion system. An explanation of these ten BMPs is available in Table S6. Pertinent information regarding indicators, categories, and weightings (see Table 1) was sourced from the SFWMD website. SFWMD’s framework incorporates 11 indicators with a fixed weighting scheme. Notably, cost-effectiveness received the highest weight (30 out of 60), underscoring its paramount significance as an indicator for stakeholders.

2.2.3. Weighting Scenario Development

To understand the impact of weighting on strategy selection and ensure the robustness of applying the framework, an uncertainty analysis was carried out. Weighting settings were varied with 100 scenarios [34,35]. Within these scenarios, weightings for each indicator were randomly assigned within their respective ranges using Microsoft Excel 2016, ensuring that all weightings collectively summed up to 100%. A nutrient management strategy was selected based on its consistent top ranking across the 100 weighting scenarios.

3. Results and Discussion

3.1. Comprehensiveness of the Developed Framework

The developed framework incorporates categories from previously established frameworks and feedback from diverse stakeholder sectors. The literature review encompassed various categories shown in Table S2, with the technological category emerging as the most frequently employed (featured in 17 out of 18 studies). Within this category, nutrient removal stood out as a commonly utilized indicator. Following closely, economic and environmental categories were the second most prevalent (applied in 9 out of 18 studies each). Within the economic category, capital cost, total cost, cost-effectiveness, and O&M cost were commonly used indicators. Among nine articles focusing on the environmental category, energy consumption and solid/residual production were common indicators. Examining the social category (covered in 8 out of 18 studies), common indicators included public health, public acceptance, and nuisance, where nuisance includes noise, odors, and visual impact. Under the ecological category, the singularly employed indicator was ecosystem service, as noted in J-Tech’s framework [16]. In the managerial category, O&M in J-Tech’s framework was the only indicator, and it was referred to as a “preference for technologies with less complexity of the day-to-day operation and less operator involvement” [16]. Informed by insights from the reviewed assessment frameworks and indicators, the developed framework includes technological, environmental, economic, social–ecological, and managerial categories. The social–ecological category consolidates social and ecological considerations, recognizing that indicators within these two categories assess the interaction between society and ecosystems.

The initial framework developed after the literature review, consisting of 15 indicators, underwent revision following the initial stakeholder meeting [36]. The revised version 2 framework included ten indicators. The adjustments involved the replacement and removal of specific indicators. Notably, energy, water, and material usage under the environmental category were replaced with residual production because the latter more accurately reflects the environmental impact. Scalability and technology readiness levels were identified as crucial indicators during the stakeholder meeting. Adapted from the National Aeronautics and Space Administration (NASA), the technology readiness level was modified to assess both technology maturity and actual implementation in the targeted location [30]. Avoided cost and cost-effectiveness were replaced with effective cost (annualized total cost for removing a unit mass of nutrients over the lifespan of a system), synthesizing all cost-related indicators identified in the literature review (Table S2). Additionally, public acceptance and social participation were introduced under the social–ecological category to address two distinct user perspectives: local community and participants in strategy implementation. Version 2 of the framework was further refined following the second stakeholder meeting [37]. A noteworthy adjustment involved renaming “nutrient loading reduction”, previously termed “removal efficiency”. This modification aimed to distinguish between the reduction in nutrient concentration and reduction in nutrient mass that also addressed sources with different concentrations, such as wastewater and stormwater, providing greater clarity in the terminology employed within the framework. The developed framework, as illustrated in Figure 3, comprises ten indicators, including quantitative indicators of nutrient loading reduction and effective cost and qualitative indicators of scalability, technology readiness level, residual management, benefit and direct impact, public acceptance, social participation, labor O&M requirement, and maintenance complexity.

The framework was designed to empower different stakeholders to assess or enhance the performance and sustainability of nutrient management strategies for any site. For instance, the indicators of “technology readiness level” and “scalability” assist innovative technology developers in understanding how to advance their technologies in terms of maturity and scale for implementation. The framework addresses other types of stakeholders through indicators such as “social participation” for local farmers and “public acceptance” for residents within the social–ecological category, and “maintenance complexity” for operators within the managerial category. Stakeholders can also vary the weightings of different indicators based on their preferences. This flexibility allows users to tailor the framework to their needs, facilitating its application for the evaluation of different nutrient management strategies.

3.2. Robustness of Strategy Selection Using the Developed Framework

A comparison of strategy selection between the developed framework and the J-Tech framework is shown in Figure 4. Alum treatment, treatment wetland, hybrid wetland treatment, Bold&Gold^®, and ElectroCoagulation emerged as the top-ranked strategies across different weighting scenarios, with significantly higher total scores than the other five strategies evaluated using the developed framework. When evaluated using the J-Tech framework, alum treatment was ranked first, followed by treatment wetland, and ElectroCoagulation was positioned seventh. The rank improvement of ElectroCoagulation in the developed framework is due to the higher score under technology readiness level and effective cost. ElectroCoagulation achieved a level 5 in technology readiness level with existing implementation in Florida. It also achieved a higher effective cost level in the developed framework compared to J-Tech’s framework since J-Tech’s framework considered TN, TP, and total suspended solids (TSS) in cost evaluation, and ElectroCoagulation has a low TSS removal.

Alum treatment consistently ranked top among ten strategies in 95 out of 100 scenarios, demonstrating the robustness of strategy selection using the developed framework. Alum treatment exhibited relatively higher scores in effective cost, scalability, and benefit and direct impact than other strategies (Figure S8). This is attributed to the relatively higher levels of these indicators than others (Table S9) and the higher weightings assigned to effective cost and benefit and direct impact (Figure 5). Notably, alum treatment secures the highest average score under effective cost among the ten strategies, affirming its cost-effectiveness in the studied context. The effective cost for alum treatment is USD 102/kg, which is USD 62/kg lower than treatment wetland (USD 164/kg) [16,25]. This is because the cost of alum treatment mainly includes the cost of removing TN and TP, and it is very effective in removing TP (USD 176/kg) and TN (USD 28/kg) [13,34]. As a biological process, treatment wetlands are more effective in removing TN (USD 46/kg) than TP (USD 285/kg) [16,25]. Alum treatment, treatment wetland, and hybrid wetland treatment technology were rated as the highest level because they can be implemented at small, medium, and large scales. For benefit and direct impact, alum treatment and treatment wetlands were rated at the same level because both enhanced recreational value by creating wildlife habitat with minimal adverse impacts. Although alum treatment was also ranked first using the J-Tech framework, it only applies under the fixed weighting scenario. Therefore, the selection process using the developed framework is more robust than the J-Tech framework, as the selected strategies are applicable under multiple weighting scenarios.

3.3. Implementation Space Informed by the Developed Framework

The evaluation results using the development framework can inform not only the strategy selection (e.g., alum treatment in the case study) but also the implementation space for a strategy (e.g., treatment wetland is ranked top under scenarios 19, 30, 39, and 69 in the case study). As illustrated in Figure 5, treatment wetland emerges as the top-ranked strategy in scenarios where nutrient loading reduction, benefit and direct impact, and public acceptance carry higher weightings, underscoring a preference for treatment wetlands when considering both technological and social–ecological categories. Molinos-Senante et al. [13] also found that, when prioritizing the social category, treatment wetlands were the most sustainable technology among treatment technologies evaluated for a wastewater treatment plant serving 150 people. On the other hand, ElectroCoagulation took the lead in scenarios with higher weightings for the indicators of benefit and direct impact, public acceptance, and social participation. This strategy provided benefits by creating a habitat to attract animals and plants on the solids-drying bed [16], bringing nature into the community, promoting neighborhood interactions, and enhancing people’s sense of belonging. Therefore, this strategy is selected when higher weighting is assigned to public acceptance. With its minimal land footprint and reduced community disturbance, ElectroCoagulation enhances benefits and minimizes direct impacts, making it a preferred choice when prioritizing the social–ecological category.

3.4. Flexibility of Framework Application with Limited Data

The contribution of each indicator to the total score, as shown in Figure 6, ranges from 0.23% to 67%, with an average of 10%. Across all scenarios and evaluated nutrient management strategies, effective cost consistently emerged as the top contributor among the indicators (Figure 6), followed by nutrient loading reduction, technology readiness level, benefit and direct impact, and labor O&M requirement. The contribution of each indicator generally aligned with the weightings assigned to that indicator, except for the technology readiness level, since the indicator level for all ten assessed strategies was the same. The average contribution of effective cost and nutrient loading reduction is 35% and 15%, respectively. Effective cost and nutrient loading reduction were frequently applied in the reviewed literature and should be included in the assessment [11,12,16,17,18,24]. However, the average contributions of technology readiness level, benefit and direct impact, and labor O&M requirement are either higher than or close to 10% for the assessed strategies. Therefore, when confronted with the challenges of applying the developed framework due to limited data for strategy evaluation, it is important to focus on data collection for these five indicators. The developed framework is effectively represented by these five indicators, covering the economic, technological, social–ecological, and managerial categories. Additionally, these five indicators address the primary concerns of diverse stakeholders, such as government agencies, researchers, technology developers, technology/BMP maintainers, and the community/public. Application with these five dominant indicators streamlines the data collection process. However, it is important to note that employing only these five indicators may lead to variations in final scores, ranks, and the best option.

4. Conclusions

This paper presents a comprehensive sustainability assessment framework incorporating ten quantitative and qualitative indicators spanning technological, environmental, economic, social–ecological, and managerial categories. Each indicator was assigned a range of weightings obtained from a literature review. This is the first framework designed with consideration of the interests of different groups of stakeholders, and the comprehensive nature of the developed framework enables various stakeholders, such as technology developers, utility managers, landowners, and the public, to evaluate nutrient management strategies according to their specific preferences. To demonstrate the applicability of the developed framework, the results were compared with an existing framework developed by J-Tech for SFWMD to assess stormwater nutrient management strategies. The strategy of alum treatment ranked top in 95 out of 100 weighting scenarios using the developed framework. The application of the developed framework also informed the implementation space (i.e., the preferred weighting scheme) for a strategy under evaluation. Moreover, the consistently higher contribution of effective cost, nutrient loading reduction, technology readiness level, benefit and direct impact, and labor O&M requirement to the total score suggests their importance in strategy evaluation and priority for future data collection if limited data are available. To ensure the long-term effectiveness of the strategies for preventing nutrient pollution in achieving sustainable development goals (specifically Goal 6, Clean water and sanitation, and Goal 14, Life below water), it is necessary to consider not only technological, environmental, and economic aspects but also social–ecological and managerial categories in assessing the potential strategies. This research contributes to the field by providing a comprehensive and flexible framework for holistic sustainability assessment in selecting strategies for removing nutrients from urban stormwater, agricultural runoff, and wastewater.