1. Introduction
Accurate and robust rainfall measurements are highly important for urban water management, flood risk mitigation, urban planning, transportation management, agriculture and more.
Flash floods can be caused by excessive rainfall and are characterized by rapid onset. The presence of large impervious areas, buildings and blocking of storm-water flow impose high risks of flash floods in urban areas. Flash floods are frequently responsible for loss of lives and damage to infrastructure. Climate change and increasing urbanization implies more frequent and severe flash floods in the future. Over the last decade, flash-flood forecast lead-time has expanded up to six hours due to improved rainfall forecasts; yet, unknown future precipitation remains the largest source of uncertainty of flash-flood forecasts [
1]. Javier et al. [
2] showed that high-resolution rainfall rate fields can provide important elements of site-specific flash-flood forecasting systems in small urban watersheds. They also commented that errors in rainfall fields produce the largest sources of uncertainty in quantitative flash-flood forecasting.
The absorption and scattering of electromagnetic waves from hydrometeors along the path of propagation cause attenuation to transmitted signals. This fact was utilized for the first time by Messer et al. [
3] in 2006 for environmental monitoring from commercial microwave link (CML) signal level measurements. There, the authors suggested the use of existing cellular networks as they provide built-in monitoring facilities and are widely distributed and, hence, can serve as a high-resolution atmospheric observation network, without the required cost of constructing new designated environmental sensing networks.
The rapid growth of urbanization and population, in addition to advancements in communication technology, have promoted the concept of smart cities, which support city operations intelligently with minimal human interaction [
4]. As smart cities become larger and the number of everyday objects connecting to existing networks is rising, the number of wireless communication links is rising, as well, to support the increase in demand.
Smart cities provide local municipalities with broadband networks using optical fibers and/or millimeter wave links to support high-bandwidth communication. These networks provide infrastructure for wireless WIFI network solutions, communication of public and education buildings, connection to security cameras and more. Signals transmitted in the E-band frequency range (60 GHz to 90 GHz) provide high bandwidth but do not penetrate easily through buildings and can be attenuated by foliage and rain to a large extent [
5].
Due to these limitation, E-band links are usually used for short-range communication. These short links are also common in the wireless backhaul of emerging 5G cellular networks. Such networks can have dense coverage over urban areas, which brings up great opportunities to obtain high-resolution rainfall maps.
Similar to the use of CMLs from cellular backhaul networks to serve as meteorological sensors, smart-city wireless links can be exploited, as well, to provide opportunistic meteorological sensing abilities from dense urban areas without the additional cost of placing dedicated rain measuring devices.
The network hardware reports the CMLs’ transmitted signal level (TSL) of the transmitter and the received signal level (RSL) of the receiver, which are usually logged by network operators for network management purposes (or can be accessed directly using the simple network management protocol (SNMP)). The attenuation of the transmitted signal is, therefore, obtained by:
where
, in dB, is referred to as the total attenuation and
t indicates time.
As mentioned before, the transmitted signal is scattered by hydrometeors along the path of propagation, which constitutes the rain-induced attenuation of the signal. The total attenuation of a signal is modeled as a summation of free-space attenuation (“path loss”), gaseous attenuation, rain induced attenuation and more.
To summarize, total attenuation
can be described by the simplified model of (
2):
where
is rainfall-induced attenuation,
is attenuation due to the wet antenna effect [
6] and
is attenuation caused by sources other than rain, such as free-space attenuation and gas attenuation, and is referred to as the baseline attenuation. Usually,
slowly changes with time compared to rain-induced factors.
is an additive measurement noise assumed independent among all links.
Attenuation due to rain is usually modeled using the power law [
7]:
where
are coefficients depending on the link-specific frequency, antenna polarization and rain drop size distribution [
8];
L, in km, is the link path length; and
, in mm/h, is the rain rate (i.e., the rain intensity).
A comprehensive study on rainfall and water vapor sensing with microwave links operating at E-band frequencies was conducted in [
9]. The authors showed that E-band links are more sensitive to rain than links operating in the K-band (and, specifically, the 15–40 GHz range) and can observe light rainfall. However, those links were shown to be affected more by errors related to the rainfall drop size distribution.
It was shown in [
10] that the power-law equation from (
3) is less accurate for short links. The authors also showed how recurrent neural networks (RNNs) can be used to estimate the rainfall rate from measurements recorded by a cellular network management system. The RNN-based method outperformed the traditional power-law-based method in terms of the root mean square error (RMSE).
In our previous work [
11], we presented an empirical model for short links, which can be seen as a modification to the power law. We showed that rainfall estimation from short links suffers from overestimation using the power law, and the proposed model is able to eliminate the overestimation.
In this paper, we compare two methods of rainfall estimation from the attenuation measurements of short E-band links: the first is a two-parameter empirical model from [
11], which corrects the attenuation measurements of short links before applying the power law. The model parameters are calibrated using attenuation measurements from a long link in the vicinity of the network. The second is a variation of a data-driven method based on a gated recurrent unit (GRU) from [
12], which showed improved performance in terms of RMSE compared to the traditional algorithm based on the power law.
The main contributions of this paper are:
- 1.
We present a comparison between an empirical short links model and an RNN-based data-driven approach for rainfall estimation from RSL measurements. We show that although the RNN-based approach performs better in terms of RMSE, the simple short links model yields similar results for moderate and strong rain intensities (higher than 5 mm/h), despite being much simpler.
- 2.
We create high-resolution 2D maps of 24 h-accumulated rainfall, which are constructed from the estimates of either method. The constructed maps are compared against rainfall maps provided by the IMS weather radar, and both show good agreement with the ground truth.
The rest of this paper is organized as follows:
Section 2 describes the data used from the city of Rehovot, Israel. In
Section 3, the details of the two rainfall estimation methods are provided. In
Section 4, the experimental results from the two methods are shown, and a comparison of the constructed rainfall maps to the weather radar maps is provided.
Section 5 concludes this paper.
2. Data
RSL measurements from the smart-city network of Rehovot are recorded regularly by the network operator company SMBIT LTD (SMBIT. Ltd, Mazkeret Batya, Israel.
https://www.smbit.co.il). The network consists of 66 links, where each link contains 2 sub-links for the two opposite directions. All links operate in the E-Band frequency range, namely, in the range of 70 GHz to 84 GHz, while the majority of links operate at 74.375 GHz. The RSL values are sampled every 30 s with a quantization level of 1 dB. The TSL is not recorded and is assumed to be constant.
Figure 1 depicts the link map of Rehovot. Colored lines represent links that were used to construct rainfall maps and are marked with an ID number. Dashed black lines represent links that had been excluded in this work due to high unrelated fluctuations of the signals levels that were detected during dry periods, as well as very short links that did not attenuate above a minimal degree. The municipality building is located in the center of the map. The majority of the antennas are located at street level, connecting traffic lights cameras. Others are placed on building rooftops, connecting schools and others to the main municipality building.
A rain gauge (the measurements are provided by The Robert H Smith Faculty of Agriculture, Food and Environment (Rehovot), The Hebrew University of Jerusalem.
http://www.meteo-tech.co.il/faculty/faculty_periodical.asp?client=1 (accessed on 11 April 2023)) in the northern part of the city is used for validation and marked as ‘RG’ in
Figure 1. The rain-gauge measurements are collected by the faculty of agriculture, food and environment (Rehovot), the Hebrew University of Jerusalem. It measures the accumulated rainfall for 10 min intervals, with a resolution of 0.1 mm.
2.1. Data Pre-Processing
Examples of RSL time-series raw data from links 29, 2, 4, 5 and 16 are shown in the top five panels of
Figure 2, according to this order. The bottom panel presents the rainfall intensity measured by the rain gauge as a function of time. The RSL signals from links 29, 2 and 4 show an agreement with the rain-gauge measurements. The RSL is attenuated during rainfall, and it is approximately constant when it stops raining. Note that the RSL of link 4 rises slowly after the rainfall stopped. Links 5 and 16 show a higher noise level. These large fluctuations can stem from multi-path propagation, where the signal can be reflected from buildings, cars, vegetation or other reflective surfaces along the path of propagation [
13]. Moreover, the aging of the electronics of the transmitter and the receiver can increase noise levels. Estimating rainfall from them is a much harder task and will result in large errors.
We noticed some changes in the RSL properties of some links at different years, such as different baseline levels and even changes in the range of attenuation values due to rain. Some links showed more than a 10 dB difference between the median of the winter RSL of 2021 and the winter RSL of 2020. Furthermore, links that were too short showed only small attenuation values of the RSL during rainfall, and others were not attenuated at all. We excluded those links, as well as links showing large fluctuations during dry periods. The excluded links are shown as dashed lines in
Figure 1.
2.2. Dataset Split
The full dataset, consisting of RSL measurements from a fixed number of links is split into three datasets. The first one is used for training the network (TRAIN). The second one is used for validation and hyper-parameter tuning (VALIDATION). The last one is used for testing only (TEST).
The TRAIN set consists of data from different rain events that occurred during 1 October 2019 to 31 March 2020.
The VALIDATION set consists of data from different rain events that occurred during 1 November 2020 to 31 December 2020.
The TEST set consists of data from different rain events that occurred during 1 January 2021 to 28 February 2021.
The datasets’ durations are summarized in
Table 1.
2.3. Data Imbalance
Since, most of the time, the weather in Israel is not rainy, the datasets are imbalanced toward dry periods. The wet and dry samples ratio are summarized in
Table 2 for the different datasets.
The wet samples are also imbalanced toward light rainfall, as can be seen in
Figure 3a–c.
5. Conclusions
In this paper, we conducted a comparison between two methods of rain estimation from RSL measurements of an existing smart-city network operating at E-band frequencies. The first method is based on a calibration process, where every link is assigned two parameters that are estimated from attenuation measurements of a reference link in the network. The second method is based on an RNN model, where we trained a set of links to detect wet and dry periods and estimate the rainfall intensity based on rain-gauge measurements, which serve as the ground truth.
When looking at the rain estimation results for a given link, the RNN-based network provided better results in terms of RMSE compared to the short links model, but the improvement itself was less than 20% in average performance, and the largest difference was achieved at lower rainfall intensities. This indicates that a simple (and almost) linear model can be used for medium to strong rainfalls.
A comparison of the estimated rain maps with the IMS radar maps when accumulated over 24 h resulted in low spatial correlation for both methods. However, the estimated maps show an agreement with the average intensity of the radar, and the two methods are able to distinguish between light and strong rainfalls, as well as produce maps with higher spatial resolution (with respect to the radar). Differences between the estimated maps and the radar maps can arise from changes in the rainfall intensity at different altitudes. The links measure the rainfall at street level, whereas the radar measures the rain at much higher altitude. In addition, the quantization of the RSL and errors in the baseline determination and the wet/dry classification also contribute to the differences between the maps.
Filtering out problematic links is a crucial step before constructing the rainfall maps. Noisy links with large fluctuations in the RSL can yield overestimation. The overestimation is prominent when the rainfall is accumulated for long periods, and this is directly related to the performance of the wet/dry classifier. The change in the RSL properties of some links between different years resulted in poor performance of both the short links model and the RNN. Therefore, a more frequent estimation of the model parameters should be performed.