Next Article in Journal
Offshore Deployments of Wave Energy Converters by Seabased Industry AB
Previous Article in Journal
Inspection-Class Remotely Operated Vehicles—A Review
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Application of State of the Art Modeling Techniques to Predict Flooding and Waves for a Coastal Area within a Protected Bay

1
Ocean Engineering, University of Rhode Island, Narragansett, RI 02882, USA
2
Environmental Data Center, University of Rhode Island, Kingston, RI 02881, USA
3
Rhode Island Coastal Resources Management Council, Wakefield, RI 02879, USA
4
RPS/ASA, South Kingstown, RI 02879, USA
*
Author to whom correspondence should be addressed.
J. Mar. Sci. Eng. 2017, 5(1), 14; https://doi.org/10.3390/jmse5010014
Submission received: 26 January 2017 / Revised: 6 March 2017 / Accepted: 14 March 2017 / Published: 20 March 2017
(This article belongs to the Section Ocean Engineering)

Abstract

:
Flood Insurance Rate Maps (FIRMs) are developed by the Federal Emergency Management Agency (FEMA) to provide guidance in establishing the risk to structures and infrastructure from storm surge sand associated waves in the coastal zone. The maps are used by state agencies and municipalities to help guide coastal planning and establish the minimum elevation and construction standards for new or substantially improved structures. A summary of the methods used and a comparison with the results of 2013 FIRM mapping are presented for Warwick, Rhode Island (RI), a coastal community located within Narragansett Bay. Because of its location, Warwick is protected from significant coastal erosion and wave attacks, but is subject to surge amplification. Concerns surrounding the FEMA methods used in the 2013 FIRM analysis are put in context with the National Research Council’s (NRC) 2009 review of the FEMA coastal mapping program. New mapping is then performed using state of the art, fully coupled surge and wave modeling, and data analysis methods, to address the NRC concerns. The new maps and methodologies are in compliance with FEMA regulations and guidelines. This new approach makes extensive use of the numerical modeling results from the recent US Army Corp of Engineers, North Atlantic Coast Comprehensive Study (NACCS, 2015). Revised flooding maps are presented and compared to the 2013 FIRM maps, to provide insight into the differences. The new maps highlight the importance of developing better estimates of surge dynamics and the advancement in nearshore mapping of waves in flood inundated areas by the use of state of the art, two-dimensional, wave transformation models.

1. Introduction

The Federal Emergency Management Agency (FEMA) leads the national effort for the development of Flood Insurance Rate Maps (FIRMS), for the purposes of establishing the risk that structures and infrastructure face from coastal inundation and wave attacks, and setting the associated insurance rates. Coastal municipalities often use these maps to guide coastal development and to establish the minimum elevation for construction standards. Additionally, the maps are regulatory, universally available, and developed within strict FEMA guidelines. This work is typically performed by contractors and organized by individual FEMA regions. The regions are given substantial latitude in establishing the methods used in the analysis, but all must follow the FEMA guidelines; for example, the Atlantic and Gulf of Mexico guidelines are provided in [1].
A Flood Insurance Study (FIS) was completed in 2012 and implemented in 2013 for Kent County, RI [2] (Figure 1). The storm surge levels were estimated by the linear interpolation of 100-year return period water levels from adjacent National Ocean and Atmospheric Administration (NOAA)-National Ocean Survey (NOS) water level stations [3]. For this study, data from Newport, RI, and Providence, RI, stations were used. The 10-, 2-, 1-, and 0.2-percent-annual-chance flood water levels were determined by the application of extreme value analysis (Generalized Extreme Value (GEV) using L Moments to estimate the distribution’s parameters) to the historical water level data at the two stations. Water levels corresponding to selected return periods were provided for each station. The water levels at intermediate locations, relative to these two reference stations, were determined by linear interpolation. No estimates of values for other confidence levels (e.g., 95%) were provided. The contractor used FEMA’s Coastal Hazard Analysis and Modeling Program (CHAMP)/Wave Height Analysis for Flood Insurance Studies (WHAFIS) [4] to estimate the waves in the flood inundated areas. This program computes the wave crest elevations along representative transects, using a 1-D wave action equation. Transects are selected by considering major topographic, vegetative, and cultural features, with variable spacing recommended to be tens to hundreds of meters, depending on the density of the development and nearshore topography. WHAFIS is forced by an appropriate depth-limited wave height at the seaward end of each transect. The 100-year Still Water Elevation Level data (SWEL), including, if applicable, the static wave setup calculated using Goda’s semi-empirical formulation [4], and the wave height estimates, are then assembled to provide predictions of the Base Flood Elevation (BFE) (Figure 2). The wave heights are then used to establish the flooding zones (VE, AE, and X, see Figure 2 for definitions of the zones). Data from each transect are laterally interpolated using engineering judgment, according to the FEMA guidelines [1], to develop a 2-D map of the flooding zones and associated BFEs.
The National Research Council (NRC) performed an in-depth review of the FEMA flood mapping program in 2009 [5]. Recommendations from their report on coastal flooding (Section 5 of their report) include the following: FEMA should use coupled 2-D surge and wave models to reduce uncertainties associated with the use of a 2-D surge model and the 1-D WHAFIS model. Wave crests calculated by CHAMP/WHAFIS have not been sufficiently validated, creating potentially significant uncertainties in BFE estimates. Problems identified in the NRC review, relevant to the Warwick, RI, study area, include: (1) wave transformation is a 2-D process that cannot be represented in a 1-D model; (2) surges and waves are completely decoupled, which may lead to over or underestimates of the BFE; (3) one-dimensional transects do not reflect 2-D terrain; and (4) the manual interpolation of 1-D results to two dimensions is subjective.
It is clear from analyzing the methods used in the FEMA 2013 flooding maps, performed for Kent County, RI, summarized above [2], that the NRC’s recommendations were not followed.
The goal of the present study is to develop new FIRMs for Kent County, using state of the art methods, following the NRC recommendations. The City of Warwick, RI (portion of Kent County on the coast), has been selected for this application. It is located inside Narragansett Bay and represents a protected coastal area. A companion paper, Spaulding et al. (2017) [7], provides an application of the same methods used here, but for a fully exposed coastal shoreline in Charlestown, RI. It should be noted that there are a variety of other models available in the international community that could potentially be used to address coastal flooding and the associated wave climate, including, but not limited to, FVCOM, FUNWAVE, TELEMAC, CH3D, Delft3D, SWASH, XBeach, and CMS Flow and Wave. These models might result in a more accurate prediction of coastal flooding, but unfortunately, they are not approved for use by FEMA for flood insurance studies in the US. In keeping with the FEMA guidelines, only models approved by FEMA for use in coastal flood mapping studies have been employed in the present effort [8]. FEMA guidelines for coastal flood mapping of the study area have also been followed [1]. While there are many alternatives in assembling a model system to perform the flooding study, we have used two well-known, widely used FEMA approved models in the current approach (ADCIRC and STWAVE). Each model is described in more detail below.
Section 2 provides an overview of the methods used in the analysis for estimating surge and waves. The results and discussion, including a validation of the estimates, are provided in Section 3. The conclusions are included in Section 4, followed by the references. Supplemental material is provided in Appendix A.

2. Methods

2.1. Surge and Tide Water Levels

The US Army Corp of Engineers (US/ACOE) completed a multi-year numerical modeling effort under the North Atlantic Coast Comprehensive Study in 2015 (NACCS) [9], with a focus on the area impacted by Hurricane Sandy. In the NACCS study, simulations were performed using fully coupled surge and wave models (the ADvanced CIRCulation model (ADCIRC) and STeady state spectral WAVE (STWAVE) forced by the 3rd generation wind-wave model WAM) for 1050 synthetic tropical storms and 100 historical extratropical storms, with a primary focus on the region from Cape Hatteras to Cape Cod. An unstructured grid with a resolution down to 50 m was used to represent the storm surge in the area flooded during a storm. The surge model, ADCIRC, was validated with data from the National Ocean and Atmospheric Administration-National Ocean Service (NOAA NOS) water observation stations for hurricanes Gloria, Josephine, Irene, and Sandy, and two extratropical storms in 1996 (storms #070 and #073) (see [9] for details on model validation). The wave model, STWAVE, (200 m resolution) was similarly validated with data from offshore wave buoy observations [9]. Given the constraints based on the volume of data generated, the results of model simulations were archived at selected save points, approximately 18,000 in total for the study area and about 1000 in RI. These included the water elevation, mean and peak wave period, zero moment wave heights, mean wave direction, wind speed and direction, and short time series of water level and waves for each storm event. Return period analyses were also available at each save point. The data were made available for Rhode Island via ArcView on line or as GIS layers under the STORMTOOLS® initiative [10,11]. Figure 3 shows the location of the save points for the Kent County study area; Warwick, RI, is shown in the red circle. An analysis was performed in the NACCS study to estimate the water level versus return period for the tropical storms for surge only and surge & tides (96 random tidal phases, linear superposition) [9]. Predictions were provided for the mean, and upper and lower 95% confidence interval values, and are available at all save points.
Flooding maps were generated from the NACCS simulations for selected return periods and are available at the STORMTOOLS web site [11]. The strategy utilized in this application is to apply extremal analysis at a primary water level station to determine the water levels for the varying return period, and then hydrodynamic model simulations to determine the spatial scaling of the water levels for storms/return periods, referenced to this primary gauging station. As an alternative, water levels can be directly determined from the NACCS save point, where return period information is available (e.g., Figure 3). Figure 4 shows the 100-year flooding contour map for the upper 95% confidence interval for the surge & tide case of RI. The water levels (m, referenced to MSL) were spatially interpolated from the save points noted, in part, in Figure 3, to generate this figure. The Warwick, RI, study area is shown in the red circle. The surge amplification in the bay is almost linear with the distance up the bay, referenced to the mouth of the bay [10] and approximately constant in the cross bay direction.
The NACCS simulations included surge and wave coupling, but given the low wave heights in the bay, the model predicted very little wave set up in Warwick, RI [4].

2.2. Wave Modeling

Wave estimates for flood inundated areas were made by applying STWAVE [12,13] to the study area on a 10 m high resolution grid, using NACCS results as boundary and initial conditions. The STWAVE model is a phase average, steady state, spectral wave model and simulates depth-induced wave refraction and shoaling, current induced refraction and shoaling, depth and steepness-induced wave breaking, wind-wave growth, and wave-wave interaction and white-capping, that redistributes and dissipates energy in a growing wave field. The model does not include wave run-up, as this is a time dependent process. To be inclusive, the study domain included most of Narragansett Bay (Figure 5), from the bay mouth to the well north of Warwick, RI. The surge level was defined by NACCS surge results for the 100-year storm, as defined above and shown in Figure 4, interpolated on the computational grid. The wave model was forced by predictions of the NACCS wave model for the mean 100-year return period condition (Figure 6). The 100-year wave spectral parameters provided at the NACCS save points were interpolated at the offshore boundary of the high resolution 10 m grid and were used as boundary conditions to the high resolution wave simulations: the wave period at the boundary location (38 m depth) was set at 20 s and the wave height at 9 m. Swell- and wind-generated gravity waves in shallow water are specified by a Texel, Marsen, and Arsloe (TMA) spectrum [12,13], with a direction of 180 degrees clockwise relative to the North (waves from the south), a spectral peak enhancement factor of eight, and a directional spreading factor of 20. The TMA spectrum is expanded into a directional spectrum using the standard wrapped normal directional spreading function [14]. Parameters of the directional spectrum are selected to simulate a narrow spectrum around the main direction of propagation to be conservative and, at least before propagation, focusing most of the energy in the targeted direction. A sensitivity study, however, shows insignificant differences in waves at the target site using the standard value of 3.3 and 4 for the enhancement factor and directional spreading factor, respectively. In addition to simulating local wind-wave regeneration, the model was forced at the surface, assuming a 100-year wind speed of 35 m/s from the south (180 degrees). The 100-year wind speed value was determined by an extremal analysis at the offshore US Army Corp of Engineers, Wave Information Study (WIS) [15] hindcast point #63079 (WIS station 79). This high wind speed from the south was adopted as a conservative estimate. The results of the simulations are referred to as the NAST method in the following, referring to a combination of NACCS and STWAVE.
Frictional losses from overland flows are addressed by assigning Manning roughness coefficients to each grid and depend on the land cover and vegetation type. Manning roughness was specified based on a 30 m resolution map of ground cover, provided by the RI Geographic Information System (RIGIS) [16]. The land use was translated into Manning roughness based on standard hydraulic literature [17], as implemented by Wamsley et al. [18], resulting in a Manning coefficient map with variations from 0.02 in open water to 0.1 over woody wetlands.

2.3. Digital Elevation Model (DEM)

To represent the topography and bathymetry in the flooded area, the wave analysis used the 2011 Light Detection and Ranging (LIDAR) data Digital Elevation Model (DEM) for RI [16] and the flooding maps to determine the water inundation depths. The data is available at a 1 m horizontal resolution, NAVD88 benchmarked, with a vertical root mean square error (RMSE) of 15 cm. The DEM was benchmarked against 20 Blind Control Points (BCP) that were distributed throughout the state and performed as part of the 2011 LIDAR survey, with an RMSE of 9.4 cm. A detailed comparison was also performed against FEMA elevation certificates, filed with the city, that were available for selected structures using the value of the Lowest Adjacent Grade (LAG) (Figure 7). Their collective RMSE was 32.6 cm. For a comparison, FEMA BFEs are rounded to the nearest 1 ft (32.8 cm). The LIDAR data were interpolated onto a 10 m grid for the wave simulations.

3. Results and Discussion

3.1. Surge and Tide Water Levels

To assess the performance of the NACCS model predictions for the study area, the model predicted water levels versus return period were compared to an analysis performed by the National Ocean and Atmospheric Administration-National Ocean Survey (NOAA NOS), based on the historical data records at Newport (Station ID #8452660) and Providence, RI (Station ID #8454000) [18]. These are the only NOAA NOS primary water level stations in RI and both are inside Narragansett Bay. Figure 8 and Figure 9 show the comparisons for Newport and Providence, RI, respectively. Predictions were provided by NOAA for the mean, and lower and upper 95% confidence intervals, and from NACCS for the mean and upper 95% confidence interval, for the surge only and surge plus tidal cases. Table 1 provides the water levels for NACCS and NOAA for Providence and Newport, RI. The values that FEMA used for these two stations are also provided. The scaling of the value at Providence, relative to Newport, is also provided at the bottom of the table.
The water levels versus return period for the mean value for NACCS surge only and the NOAA mean, are in very good agreement for Newport, RI (Figure 8). The NACCS is higher (typically about 30 cm) than the NOAA values for the surge plus tide case. The structure of the upper 95% confidence limit is dramatically different between the NOAA and NACCS data. NOAA uncertainties are strongly asymmetric with the return period, with the upper confidence deviating more from the mean than the lower confidence interval. This is a result of the limited length of the historical record (approximately 70–85 years) and the presence of several large storm events (1938 and 1954) in the record. The uncertainty in the NACCS, as shown by the upper 95% confidence limit, is approximately independent of the return period. This is attributed to the fact that the analysis is based on synthetic tropical storms, rather than historical events. For the most frequently used 100-year return period water level, the mean value for NOAA is 2.47 m, while the corresponding values from NACCS are 2.42 m (surge only) and 2.76 m (surge plus tide). The upper 95% water level from NOAA for this case is 3.47 m, compared to NACCS values of 3.58 m (surge only) and 3.93 m (surge plus tide). The estimates are thus reasonably comparable.
Similar comparisons for the 100-year event at Providence give 3.48 m for the mean from NOAA, and 4.01 m (surge only) and 4.29 m (surge plus tide) from NACCS (Figure 9). For the upper 95% confidence limit, the NOAA value is 5.56 m, compared to the NACCS values of 5.17 m (surge only) and 5.49 m (surge plus tide). The NACCS values are significantly more conservative than NOAA for the mean. The opposite is true for the upper 95% limit. This difference in values is clearly shown in Figure 9. A review of the historical record at Providence shows that the 1938 storm is the most extreme event and dominates the uncertainty at the upper 95% limit. The 1954 storm is absent from the Providence record, since water level measurements were not collected during this event.
It is noted that the above analysis is based on a return period analysis using only the tropical storms from NACCS and for all data from the historical record. Employing all storms (e.g., tropical and extratropical storms) in the NACCS data, the results of the return period analysis are essentially the same as the tropical storm case only. This is attributed to the fact that tropical storms provide the highest water levels, and hence dominate the upper end of the low frequency tail of the distribution. This is consistent with the NOAA NOS historical observations at the site, with hurricanes responsible for six of the top 10 surge levels [19].
Hashemi et al. [20] developed an artificial neural network (ANN) model using the NACCS models predictions, which also serves to validate NACCS model predictions. Model inputs included the tropical storm strength, forward speed, radius to maximum winds, and storm path. The ANN model predicted the peak water levels at Newport and Providence, for each of these events. Seventy percent (70%) of the storms (approximately 1000 tropical storms) were used to train the algorithm, with 15% utilized for validation, and 15% for testing. The model demonstrated a very good performance for all three phases, with correlation coefficients greater than 0.92. The root mean square error (RMSE) was approximately 30 cm. The ANN model was then applied to predict the peak storm elevation for the largest tropical storms that have historically impacted the area (1938, 1944, 1954, 1991, 2011, and 2012), using the same parameters as for the NACCS storms, but derived from the historical data. Predictions were again performed at Newport and Providence, RI. The model’s performance gave a RMSE of 32 cm for all storms, except 2012 (Sandy). The ANN model under predicted the peak water levels for this storm, by about 50 cm. This reduced performance is attributed to the fact that storms with characteristics like Sandy are extremely rare in the historical record and were not included in the NACCS synthetic tropical storm data base.
Model predictions, in another evaluation [7,10], were compared to a wide array of point observations collected during the 1938, 1954, and 2012 storms, and reported by FEMA in their Kent County FIS [2]. In this case, the slope of the linear regression line from Newport to the head of the bay in Providence, was based on the storm of interest. Spaulding et al. (2017) [7] show a comparison of model predictions to observations for these events. While the data quality is typically quite variable, the analysis does well in capturing the amplification of the storm water levels with distance up the bay. The amplification is clearly strongly storm-dependent and driven primarily by the storm track, relative to the bay, and storm strength. As an example, amplifications for 1938, 1954, and 2012 were 1.3, 1.56, and 1.1, respectively.
In the interest of developing conservative estimates of flooding and addressing uncertainties inherent in the modeling and extreme analysis methods, the NACCS surge & tide, 95% confidence limit case was selected for this study.
Figure 10 shows the flood inundation levels (m) for the Warwick study area for the 100-year storm event. Note that either the spatial scaling method, noted earlier, or the selection of the data from the save points immediately offshore of the study area, can be used to specify the flooding. In this case, linear scaling from the NACCS save point data at Newport and Providence has been used (Table 1, namely 1.4, Providence relative to Newport). This provides a very good approximation to the pattern shown in Figure 4. The water level increases from approximately 4.6 m at the southern end of the study area, to 5.2 m at the northern end.

3.2. Wave Modeling

To validate the input to the wave model, an analysis was performed comparing extreme values for the 100-year wind and significant wave heights, based on the US Army Corp of Engineers, WIS hindcast point #79 and an analysis of the NACCS peak wave heights for save point ID #6859, for the synthetic tropical storms [9]. This is the save point in closest proximity to WIS #79. The WIS analysis is based on a hindcast of winds and waves from 1980 to 2012. In this analysis, a generalized extreme value (GEV) method was applied to both analyses, so the results are internally consistent for the method. The results presented in Spaulding et al. (2017) [7] show that NACCS estimates are consistent with the WIS data for the mean wave height. Based on this comparison, the results from the NACCS study provide reasonable estimates of the 100-year wave height and period. It has been impossible to validate the model predicted wave heights inside Narragansett Bay, since no wave data is available.

3.3. Simulations

Figure 11 shows model predicted wave heights from the mouth of the bay to the lower reach of the Providence River. Results are provided in terms of the controlling wave height, which is the wave height corresponding to the average of 1% of the highest waves. Assuming a Rayleigh distribution, the controlling wave height is estimated from the significant wave height provided as an output from the STWAVE model. It is the wave height used in calculating the BFEs in FEMA’s protocol. The upper panel shows the predictions for the entire study area and the lower panel, the area immediately surrounding Warwick, RI. The study domain is shown in Figure 5.
Figure 11 clearly shows that the wave heights are largest when the fetch, in the direction of the predominant 100-year wind from the south, is largest. Wave heights in protected embayments (e.g., Greenwich Bay) are substantially lower, since they are shielded by the bay geography.
Figure 12 shows the model predicted controlling wave height for Warwick, RI. The figure shows that the waves in the study area are quite low given the location of Warwick well inside Narragansett Bay (Figure 11). There are several locations where the wave heights are approximately 2 m (circled in red/black). Most of the wave heights, however, are a meter or lower. Correspondingly, the wave heights in areas within Greenwich Bay are limited by the east-west orientation of the bay, with the bay mouth to the east of the south east.
Figure 13 shows the combination of the surge level (Figure 11) plus the controlling wave height (Figure 12) for Warwick, RI.
In the interest of better understanding the impact of improvements in the modeling methodologies applied on the final FIRM maps, Figure 14 shows the FEMA (upper panel) and NAST (lower panel) BFE (relative to NAVD88) flooding zones for the study area. Figure 15 provides a detailed comparison of FEMA and NAST predictions of SWEL, waves, and BFEs by transect. A tabulated version of the comparison is provided in Table A1. A comparison of NAST predictions and FEMA results for the lower West Passage of the bay are provided in Schambach et al. (2017) [21].
Restricting attention to the flooded area, the present study (NAST) results show that the region impacted is slightly larger than FEMA’s estimates. This is consistent with the fact that the mean SWEL in the present study is approximately 4.75 m, compared to FEMA’s value of 3.66 m; a difference of 0.91 m. This difference is consistent with the use of the NACCS surge & tides, upper 95% confidence interval value for the 100-year return period in the present study, compared to the value that FEMA used in its 2012 FIS (Table 1).
A close inspection of the FEMA maps (upper panel in Figure 14) shows discrete, step changes in along shore variations in the BFEs at selected locations. Not surprisingly, these occur at the locations of the transects shown in Figure 1 and are clearly the result of the subjective interpolation of BFE values from adjacent transects. This highlights one of the key NRC criticisms of the FEMA methods [5] (see Section 1). The NAST approach shows a continuous variation in BFEs, in response to the 2D wave transformation processes. It is noted that the spatial scale of these variations (Figure 14, lower panel) is much smaller than the spacing between FEMA transect lines (Figure 1).
To provide additional insight into the differences between the two maps, Figure 15 shows the FEMA and NAST SWEL, BFE (referenced to NAVD88), and wave crest height, at the seaward end of each transect. The data used in developing this figure is provided in Table A1. Focusing on the FEMA results, the SWEL generally increases with distance up the bay, increasing from 3.4 to 4.0 m (change of 0.6 m), from transect one to 33, with a slight drop to 3.6 m at transect 34. This compares to a continuous increase from 4.6 to 5.2 m (change of 0.6 m) over the full range of transects for the NAST results. Although the increase in NAST SWEL in the northern transects shows a smoother increase than FEMA, the general trends in SWEL in NAST and FEMA’s values are very similar. NAST SWEL values are on average 1.1 m higher than FEMA’s values. The higher values predicted by NAST are a direct result of using the NACCS surge & tide values at the 95% confidence interval, compared to FEMA’s return period analysis, based on historical data. The variations in FEMA and NAST wave crest heights show little variation with distance up the bay, but exhibit substantial variations between transects. The pattern in variations is quite similar between the two. The highest wave crest height of 2 m is located at Warwick Neck Point (transects 18–19), with secondary peaks at Oakland Beach (transects 14–15) and Conimicut Point (transects 25–27). These result from long southerly fetch distances. Crest heights are substantially lower inside Greenwich Bay (transects 1–14) and north of Conimicut Point (transects 28–34); both areas are geographically shielded from the predominant southerly wind direction. It is noted that the FEMA SWEL variations are correlated with the wave crest variations reflecting the wave induced setup. The NAST results show a limited impact of wave set up. Statistical comparisons of FEMA and NAST estimates of SWEL, waves, and BFEs, are provided at the bottom of Table A1. The standard deviations and RMS values are 0.28 and 1.12 m for SWEL, 0.34 and 0.34 m for waves, and 0.48 and 1.1 m for BFEs, respectively. The variances and RMS differences are small for waves, while the variances and RMS values for SWEL and BFE (as a result of the underlying SWEL values) are substantially larger. The latter reflects the use of the upper 95% confidence level for the NAST SWEL and FEMA’s use of the mean value for the 100-year return period water level.

4. Conclusions

Flood inundation maps have been developed for Warwick, RI, using state of the art, fully coupled high resolution surge and wave models (ADCIRC, WAM, and STWAVE), applied to the study area as part of the NACCS. The model predictions were extensively validated with data from tropical and extratropical storms during the study. NACCS simulations were performed for 1050 synthetic tropical storms and 100 historical extratropical storms. Peak values were archived at selected save points and return period analyses were performed for each storm data set. Additional validation, of both the water levels and wave heights versus return period, with historical water level data at Newport and Providence, RI, for the former, and hindcasts (1980–2012) of wave conditions at a selected location off the coast (RI Sound) for the latter, from the USACE Wave Information Study (WIS) [14], were performed as part of this study, to gain additional insight into the performance of the model. It was impossible to validate the wave model performance inside Narragansett Bay, since no wave data were available. For the comparisons that were performed, the results are generally in reasonable to good agreement.
Simulations were then performed using STWAVE on a high resolution grid (10 m) of the flood inundated study area. The model was driven by offshore water levels and wave height, period, and direction, assuming a shallow water, wave spectral distribution model from the NACCS study along the open boundaries. The model was also forced by 100-year return period winds. The model included wave dissipation based on a comprehensive mapping of the roughness from both structures and ground cover. STWAVE predicted the 2-D wave transformation processes and clearly showed the importance of the 2-D nature of the wave field.

4.1. Key Findings Include

  • FEMA predicted 100-year water levels (SWEL) range from 3.4 to 3.8 m for the study area. Our study results are larger (4.6 to 5.2 m); a difference of approximately 1.1 m. NAST results are approximately 30% higher than FEMA results. The larger values predicted by our analysis are attributed to the use of the upper 95% confidence limit water levels used in the present study, compared to FEMA’s return period analysis, based on the historical data. It is noted FEMA’s values are larger than the NOAA mean value, but smaller than the upper 95% for both Newport and Providence; all using the same historical data set. Hence, the difference is due to use of a different extremal value analysis method. Since NOAA has primary authority for collecting and analyzing the water level data, it is unclear why FEMA would choose an alternative method that gives predictions at variance with NOAA’s values.
  • The comparison of FEMA and NAST wave predictions show comparable results for wave heights (0.5 to 2 m), with the spatial variations controlled by the maximum fetch distance to the south. On average, across all transects, NAST model predictions are approximately 7% lower than FEMA. This small difference is explained by the fact that FEMA’s method and the current STWAVE-based approach are both dominated by fetch distances along the central axis of the West Passage of Narragansett Bay, from the south.
  • FEMA BFE maps show strong variations between transect lines, reflecting the subjective nature of the interpolation of the 2-D maps from the 1-D transects. NAST results, on the other hand, show the importance of 2-D wave transformation processes and the spatial scales are consistent with the topography and bathymetry of the study area.
  • It has proven impossible to recreate the 2-D structure of the wave field in the flood inundated area that FEMA has developed, given the results for the FIS transects. It seems to be primarily controlled by topography. The FIS transect spacing is too large to determine the spatial structure and the 2-D wave processes are not represented by the WHAFIS method used.
  • The mean BFE value, averaged over all transects, is approximately 22% larger, based on the NAST analysis, compared to the FEMA results. The difference is attributed to the 30% higher SWEL for the NAST data, compared to the FEMA value and slightly lower (7%) wave crest heights.
Our analysis presented herein has implemented all of the NRC recommendations raised in their review of the FEMA coastal mapping program [5]. Thus, it represents an advancement over the methods used in the FEMA FIS study [2]. Additionally, the NAST maps represent a more conservative approach for assessing the risk to coastal structures and are more protective of public safety.

4.2. Open Coast vs. Protected Bay Area

Comparing the results of the prior paper on developing flood insurance maps for an open coastal area [7] to the present paper, in which exactly the same methods are applied to a protected coastal area, we note that the differences in NAST versus FEMA results are dominated by waves for the open coastal area and by the water level for the protected bay. The former is due to problems in specifying the wave heights in the immediate offshore area and the use of the 1-D wave transformation model to predict the wave field over the flood inundated area in the FEMA approach. The assumption about the dune profile during the 100-year event using FEMA’s standard methodology versus an analysis of historical observations (NAST approach), is also a contributing factor. For the protected area, the wave predictions using both the FEMA and NAST methods are consistent, although the small spatial scale variations are an issue. These are principally caused by the fact that the FEMA estimates of wave heights at the end of each WHAFIS transect use different fetch directions, thus ignoring the correlation between extreme winds and direction. The differences in water level are the result of using the upper 95% confidence limit for the water level versus return period in NAST, compared to using the mean value by FEMA.

4.3. Some Lessons Learned

In developing alternate FIRMS for the two study areas using well established, state of the art tools, and comparing the results to recently generated (last five years) FEMA maps, we note the following lessons:
  • Given the uncertainty in the various modeling and return period analysis methods employed to determine the water flood level, it seems prudent to develop a quantitative measure of the uncertainty of the results. The use of the upper 95% confidence limit value addresses both issues, is readily available, and simple to implement and understand.
  • It is clear that the application of WHAFIS and the subsequent interpolation of its 1-D results to generate a 2-D map is problematic. The boundary conditions for the end of the WHAFIS transects and the assumption about the profile for the eroded dune are both issues. As noted in the NRC review and documented here, wave transformation is a 2-D process that cannot be represented in a 1-D model. WHAFIS should be replaced by a suitable 2-D model (e.g., STWAVE).
  • Given the evolution of fully coupled, time dependent flood inundation and wave models on the international scale, it would be appropriate for FEMA to update the list of accepted models to include those providing a more comprehensive treatment of the underlying physics and improved predictive performance.

Acknowledgments

The application of the models to Charlestown, RI, was supported by Housing and Urban Development (HUD), Grant # B-13-DS-44-0001, Hurricane Sandy CDBG Disaster Recovery—CFDA #14.269 and administered through the State of Rhode Island, Executive Office of Commerce, Office of Housing and Community Development (OHCD). Joseph Dwyer, an MS student in the University of RI, Kingston, RI, Environmental Science and Management program, performed an analysis comparing elevation data from the certificates of elevation to the LIDAR-based digital elevation model.

Author Contributions

Malcolm L. Spaulding developed the idea for STORMTOOLS/CERI and its application to generate FIRMs. He led the project effort; Annette Grilli was primarily responsible for wave modeling; Implementation in GIS and providing access to the output was provided by Chris Damon; Grover Fugate advised on the design of the system to meet the needs of coastal planners; Tatsu Isaji performed the water level analysis on NACCS data; and Lauren Schambach performed the transect inter-comparisons.

Conflicts of Interest

The authors declare no conflict of interest. The project sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the result.

Acronyms

ADCIRCADvanced CIRCulation model
ANNArtificial Neural Network
BCPBlind Control Points
BFEBase Flood Elevation
CERICoastal Environmental Risk Index CHAMP-Coastal Hazard Analysis and Modeling Program
CH3DCoastal Hydrodynamic Model 3D
CIConfidence Interval
CRMCRI Coastal Resources Management Council
Deltf3DDelft 3D hydrodynamics model
DEMDigital Elevation Model
FEMAFederal Emergency Management Agency
FIRMFlood Insurance Rate Maps
FISFlood Insurance Study
FUNWAVEBoussinesq Wave Model
FVCOMFinite Volume Coastal Ocean Model
GEVGeneralized Extreme Value
HUDHousing and Urban Development
LAGLowest Adjacent Grade
LIDARLaser Imaging, Detection, and Ranging
LiMWALimit of Moderate Wave Action
MSLMean Sea Level
NACCSUSACE, North Atlantic Comprehensive Coastal Study
NASTNorth Atlantic Coast Comprehensive Study-STWAVE
NAVD88North Atlantic Vertical Datum, 1988.
NOAA NOSNational Ocean and Atmospheric Administration-National Ocean Survey
OHCDOffice of Housing and Community Development
RI GISRhode Island-Geographic Information System
RMSERoot Mean Square Error
SAMPSpecial Area Management Plan
SLRSea Level Rise
STWAVESTeady state spectral WAVE model
STORMTOOLStools in support of storm analysis
SWANDelft 3rd generation wave model
SWASHDelft Hydraulics Unsteady Flow Model
SWELStill Water Elevation
TELEMAC-MASCAREsuite of finite element hydrodynamic program
TMATexel, Marsen, and Arsloe wave spectrum
URIUniversity of Rhode Island
USACEUS Army Corp of Engineers
WAMWavewatch III, Model
WHAFISWave Height Analysis for Flood Insurance Studies
WISUSACE Wave Information Study
XBeachDeltares 2D wave and sediment transport model

Appendix A

Table A1. Comparison NAST (STORMTOOLS) and FEMA SWEL, BFE, and controlling wave heights and relative difference (%) for each FEMA transect shown in Figure 1.
Table A1. Comparison NAST (STORMTOOLS) and FEMA SWEL, BFE, and controlling wave heights and relative difference (%) for each FEMA transect shown in Figure 1.
KENT County 100-Year Storm Water Level
Transect NumberNAST-STORMTOOLFEMADifference [NAST-FEMA]
SWEL (m)BFE (m)WAVE (m)SWEL (m)BFE (m)WAVE (m)SWEL (m)BFE (m)WAVE (m)
14.624.960.353.383.570.181.231.400.17
24.615.821.323.574.661.11.041.160.22
34.615.791.243.514.631.131.101.150.11
44.624.730.113.413.660.241.211.08−0.13
54.664.900.243.693.990.30.970.90−0.06
64.704.700.003.383.410.031.321.29−0.03
74.725.240.523.513.960.461.221.280.06
84.724.980.263.844.480.640.880.50−0.38
94.805.030.243.783.840.061.021.190.18
104.905.830.933.754.540.791.151.290.14
114.805.921.143.544.631.11.271.290.04
124.725.861.153.574.691.131.151.170.02
134.716.291.613.665.211.551.051.080.06
144.776.822.113.965.41.430.811.430.68
154.796.692.023.65.41.81.191.290.22
164.745.510.783.474.360.881.261.15−0.10
174.615.510.903.725.551.830.89−0.04−0.93
184.556.331.794.156.162.010.400.18−0.22
194.496.682.194.186.192.010.310.490.18
204.496.191.703.725.521.80.770.67−0.10
214.555.591.043.815.461.650.740.13−0.60
224.605.641.053.695.31.620.910.34−0.56
234.605.671.073.935.581.650.670.10−0.57
244.675.350.683.574.631.071.100.71−0.39
254.725.791.083.65.151.551.120.63−0.48
264.755.630.883.635.061.431.130.57−0.55
274.826.461.683.445.091.651.371.370.03
284.884.980.093.543.60.061.351.380.03
294.935.050.123.543.750.211.391.30−0.09
304.965.870.913.664.390.731.311.480.18
315.065.230.163.573.690.121.501.540.04
325.115.490.383.724.360.641.401.13−0.26
335.135.900.783.994.110.121.141.790.65
345.185.480.303.573.870.31.611.610.00
Mean4.755.640.913.674.640.981.081.0−0.07
Minimum0.31−0.04−0.93
Maximum1.611.790.68
Standard Deviation0.280.480.34
RMS1.121.110.34

References

  1. Federal Emergency Management Agency (FEMA). Atlantic Ocean and Gulf of Mexico Coastal Guidelines Update; FEMA: Washington, DC, USA, 2007.
  2. Federal Emergency Management Agency. Flood Insurance Study, Kent County, RI, FEMA Flood Insurance Study Number 44003CV000B; FEMA: Washington, DC, USA, 2012.
  3. Federal Emergency Management Agency. Updated Tidal Profiles for the New England Coastline. March 2012. Available online: https://www.fema.gov/media-library/assets/documents/85240 (accessed on 16 January 2017). [Google Scholar]
  4. Federal Emergency Management Agency. WHAFIS Documentation Version 4.0. 2007. Available online: https://www.fema.gov/wave-height-analysis-flood-insurance-studies-version-40 (accessed on 16 January 2017). [Google Scholar]
  5. National Research Council. Mapping the Zone: Improving Flood Map Accuracy; Committee on FEMA Flood Maps; Board on Earth Sciences and Resources/Mapping Science Committee; National Research Council: Washington, DC, USA, 2009. [Google Scholar]
  6. FEMA Definitions. Available online: https://www.fema.gov/sites/default/files/images/flood_zones_limwa.jpg (accessed on 16 January 2017).
  7. Spaulding, M.L.; Grilli, A.; Damon, C.; Oakley, B.; Fugate, G.; Isaji, T.; Schambach, L. Application of state of art modeling techniques to predict flooding and waves for an exposed coastal area. J. Mar. Sci. Eng. 2017, 5, 10. [Google Scholar] [CrossRef]
  8. FEMA Approved Models. Available online: https://www.fema.gov/coastal-numerical-models-meeting-minimum-requirement-national-flood-insurance-program (accessed on 17 January 2017).
  9. Cialone, M.A.; Massey, T.C.; Anderson, M.E.; Grzegorzewski, A.S.; Jensen, R.E.; Cialone, A.; Mark, D.J.; Pevey, K.C.; Gunkel, B.L.; McAlpin, T.O. North Atlantic Coast Comprehensive Study (NACCS) Coastal Storm Model Simulations: Waves and Water Levels; Report: ERDC/CHL TR-15-44; Coastal and Hydraulics Laboratory U.S. Army Engineer Research and Development Center: Vicksburg, MS, USA, 2015. [Google Scholar]
  10. Spaulding, M.L.; Isaji, T.; Damon, C.; Fugate, G. Application of STORMTOOLS’s simplified flood inundation model, with and without sea level rise, to RI coastal waters. In Proceedings of the ASCE Solutions to Coastal Disasters Conference, Boston, MA, USA, 9–11 September 2015.
  11. STORMTOOLS Website. Available online: http://www.beachsamp.org/resources/stormtools/ (accessed on 16 January 2017).
  12. Massey, T.C.; Anderson, E.M.; McKee-Smith, J.; Gomez, J.; Rusty, J. STWAVE: Steady State Spectral Wave Model. User’s Manual for STWAVE, Version 6.0; US Army Corp of Engineers, Environmental Research and Development Center: Vicksburg, MS, USA, 2011. [Google Scholar]
  13. Smith, J.M.; Sherlock, A.R.; Resio, D.T. STWAVE: Steady-State Wave Model User’s Manual for STWAVE, Version 3.0; ERDC/CHL SR-01–01; U.S. Army Engineering Research and Development Center: Vicksburg, MS, USA, 2001. [Google Scholar]
  14. Borgman, L.E.; Panicker, N.N. Design Study for a Suggested Wave Gage Array off Point Mugu, California; Technical Report Hel 1-14; Hydraulic Engineering Laboratory University of California at Berkeley: Berkeley, CA, USA, 1970. [Google Scholar]
  15. USACE Wave Information Study. Available online: http://wis.usace.army.mil/hindcasts.shtml?dmn=atlWIS (accessed on 17 January 2017).
  16. RI-GIS Digital Elevation Model. Available online: http://www.rigis.org/data/topo/2011 (accessed on 16 January 2017).
  17. Arcement, G.J.; Schneider, V.R. Guide for Selecting Manning’s Roughness Coefficients for Natural Channels and Flood Plains; U.S. Geological SurveyWater-Supply Paper 2339; U.S. Geological Survey: Denver, CO, USA, 1989.
  18. Wamsley, T.V.; Cialone, M.A.; Smith, J.M.; Ebersole, B.A.; Grzegorzewski, A.S. Influence of landscape restoration and degradation on storm surge and waves in southern Louisiana. Nat. Hazards 2009, 51, 207–224. [Google Scholar] [CrossRef]
  19. Zervas, C. Extreme Water Levels of the United States, 1893–2010; NOAA Technical Report NOS CO-OPS 067; NOAA: Silver Spring, MD, USA, 2013.
  20. Hashemi, R.; Spaulding, M.L.; Shaw, A.; Farhadi, H.; Lewis, M. An efficient artificial intelligence model for prediction of tropical storm surge. J. Nat. Hazards 2016. [Google Scholar] [CrossRef]
  21. Schambach, L.; Grilli, A.R.; Spaulding, M.L. Predicting the 100-year storm inundation along the Narragansett Bay shoreline. Nat. Hazard 2017. in preparation. [Google Scholar]
Figure 1. FEMA (2012) [2] Kent County, RI, study area. Also, shown are the locations of WHAFIS transects, numbered from 1 to 34.
Figure 1. FEMA (2012) [2] Kent County, RI, study area. Also, shown are the locations of WHAFIS transects, numbered from 1 to 34.
Jmse 05 00014 g001
Figure 2. FEMA definition schematic for flooding and structures located on grade and elevated territories. The still water elevation (SWEL) for 1% annual chance (100-year) is shown by the dashed red line and the Base Flood Elevation (BFE) by the blue dashed line. FEMA zones (VE, AE seaward, and landward of the Limit of Moderate Wave Action (LiMWA)), based on wave heights, are also noted at the top of the figure [6].
Figure 2. FEMA definition schematic for flooding and structures located on grade and elevated territories. The still water elevation (SWEL) for 1% annual chance (100-year) is shown by the dashed red line and the Base Flood Elevation (BFE) by the blue dashed line. FEMA zones (VE, AE seaward, and landward of the Limit of Moderate Wave Action (LiMWA)), based on wave heights, are also noted at the top of the figure [6].
Jmse 05 00014 g002
Figure 3. Location of the NACCS save points along the coastline, mid-Narragansett Bay [9]. The location of the Warwick, RI, study area is shown in the red circle. The boundaries of Kent County are shown by the dashed white lines.
Figure 3. Location of the NACCS save points along the coastline, mid-Narragansett Bay [9]. The location of the Warwick, RI, study area is shown in the red circle. The boundaries of Kent County are shown by the dashed white lines.
Jmse 05 00014 g003
Figure 4. Water level contour map for 100-year, upper 95%, surge plus tide case, based on NACCS data. The Warwick, RI, study area is shown in the red circle. Water levels are in meters relative to the Mean Sea Level [7].
Figure 4. Water level contour map for 100-year, upper 95%, surge plus tide case, based on NACCS data. The Warwick, RI, study area is shown in the red circle. Water levels are in meters relative to the Mean Sea Level [7].
Jmse 05 00014 g004
Figure 5. Study domain (highlighted area) for the STWAVE application to Narragansett Bay. Study area shown in red circle.
Figure 5. Study domain (highlighted area) for the STWAVE application to Narragansett Bay. Study area shown in red circle.
Jmse 05 00014 g005
Figure 6. Contour map of the 100-year return period significant wave height (m, MSL referenced) for the mean surge plus tide case for RI coastal waters based on NACCS data (200 m STWAVE grid) [7]. Warwick study area is shown in the red circle.
Figure 6. Contour map of the 100-year return period significant wave height (m, MSL referenced) for the mean surge plus tide case for RI coastal waters based on NACCS data (200 m STWAVE grid) [7]. Warwick study area is shown in the red circle.
Jmse 05 00014 g006
Figure 7. Location of FEMA elevation certificates for Warwick, RI study area provided by the City of Warwick, RI.
Figure 7. Location of FEMA elevation certificates for Warwick, RI study area provided by the City of Warwick, RI.
Jmse 05 00014 g007
Figure 8. Surge elevation (meters, relative to NAVD88) vs. return period (years) for Newport, RI (Station ID #8452660). Estimates are provided from NOAA for the mean, and lower and upper 95% confidence intervals, and for NACCS for the mean and upper 95% confidence interval, for the surge only and surge plus tidal cases [7].
Figure 8. Surge elevation (meters, relative to NAVD88) vs. return period (years) for Newport, RI (Station ID #8452660). Estimates are provided from NOAA for the mean, and lower and upper 95% confidence intervals, and for NACCS for the mean and upper 95% confidence interval, for the surge only and surge plus tidal cases [7].
Jmse 05 00014 g008
Figure 9. Surge elevation (meters, relative to NAVD88) vs. return period (years) for Providence, RI (Station ID #8454000). Estimates are provided from NOAA for the mean, and lower and upper 95% confidence intervals, and for NACCS for the mean and upper 95% confidence interval, for the surge only and surge plus tidal cases [19].
Figure 9. Surge elevation (meters, relative to NAVD88) vs. return period (years) for Providence, RI (Station ID #8454000). Estimates are provided from NOAA for the mean, and lower and upper 95% confidence intervals, and for NACCS for the mean and upper 95% confidence interval, for the surge only and surge plus tidal cases [19].
Jmse 05 00014 g009
Figure 10. Surge elevation (m) relative to NAVD88 in the inundation zone for Warwick, RI for 100-year event.
Figure 10. Surge elevation (m) relative to NAVD88 in the inundation zone for Warwick, RI for 100-year event.
Jmse 05 00014 g010
Figure 11. STWAVE predicted controlling wave height (m) for 100-year storm for entire study domain (upper panel) and for area in the immediate vicinity of Warwick, RI (lower panel). Locations of Greenwich Bay, Oakland Beach and Warwick Neck Point (red circle), and Conimicut Point (black circle) are noted.
Figure 11. STWAVE predicted controlling wave height (m) for 100-year storm for entire study domain (upper panel) and for area in the immediate vicinity of Warwick, RI (lower panel). Locations of Greenwich Bay, Oakland Beach and Warwick Neck Point (red circle), and Conimicut Point (black circle) are noted.
Jmse 05 00014 g011
Figure 12. Controlling wave height (m) for Warwick, RI, 100-year event. Areas with substantially larger wave heights are shown; red circle for Oakland Beach and black circle for Conimicut Point.
Figure 12. Controlling wave height (m) for Warwick, RI, 100-year event. Areas with substantially larger wave heights are shown; red circle for Oakland Beach and black circle for Conimicut Point.
Jmse 05 00014 g012
Figure 13. Total water depth (m) (inundation plus wave height relative to ground level) for Warwick, RI, 100-year event.
Figure 13. Total water depth (m) (inundation plus wave height relative to ground level) for Warwick, RI, 100-year event.
Jmse 05 00014 g013
Figure 14. FEMA (upper panel) and NAST (lower panel) predicted BFEs relative to NAVD88. Note that FEMA mapping includes part of the near shore coastal waters in their BFE zones, while the NAST method does not.
Figure 14. FEMA (upper panel) and NAST (lower panel) predicted BFEs relative to NAVD88. Note that FEMA mapping includes part of the near shore coastal waters in their BFE zones, while the NAST method does not.
Jmse 05 00014 g014
Figure 15. FEMA (black) and NAST (red) SWEL (plain dot), BFE (triangle) (referenced to NAVD88), and controlling wave crest elevations (circle) by transect number (see Figure 5 for locations). All are in meters.
Figure 15. FEMA (black) and NAST (red) SWEL (plain dot), BFE (triangle) (referenced to NAVD88), and controlling wave crest elevations (circle) by transect number (see Figure 5 for locations). All are in meters.
Jmse 05 00014 g015
Table 1. Water levels (m) for the 100-year return period for Providence and Newport, RI, from the NACCS, NOAA, and FEMA [10,18].
Table 1. Water levels (m) for the 100-year return period for Providence and Newport, RI, from the NACCS, NOAA, and FEMA [10,18].
StationWater Levels (m)
NACCSNACCSNOAAFEMA
Surge + TideSurge
MeanUpper 95%MeanUpper 95%MeanUpper 95%FIS
Providence4.295.494.015.173.485.564.2
Newport2.763.932.423.582.473.473.21
Scaling Relative to Newport
Providence1.551.41.661.441.411.61.31
Newport1111111

Share and Cite

MDPI and ACS Style

Spaulding, M.L.; Grilli, A.; Damon, C.; Fugate, G.; Isaji, T.; Schambach, L. Application of State of the Art Modeling Techniques to Predict Flooding and Waves for a Coastal Area within a Protected Bay. J. Mar. Sci. Eng. 2017, 5, 14. https://doi.org/10.3390/jmse5010014

AMA Style

Spaulding ML, Grilli A, Damon C, Fugate G, Isaji T, Schambach L. Application of State of the Art Modeling Techniques to Predict Flooding and Waves for a Coastal Area within a Protected Bay. Journal of Marine Science and Engineering. 2017; 5(1):14. https://doi.org/10.3390/jmse5010014

Chicago/Turabian Style

Spaulding, Malcolm L., Annette Grilli, Chris Damon, Grover Fugate, Tatsu Isaji, and Lauren Schambach. 2017. "Application of State of the Art Modeling Techniques to Predict Flooding and Waves for a Coastal Area within a Protected Bay" Journal of Marine Science and Engineering 5, no. 1: 14. https://doi.org/10.3390/jmse5010014

APA Style

Spaulding, M. L., Grilli, A., Damon, C., Fugate, G., Isaji, T., & Schambach, L. (2017). Application of State of the Art Modeling Techniques to Predict Flooding and Waves for a Coastal Area within a Protected Bay. Journal of Marine Science and Engineering, 5(1), 14. https://doi.org/10.3390/jmse5010014

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop