Evaluating the Performance of LoRa Networks: A Study on Disaster Monitoring Scenarios

Abstract

The development of technologies using the Internet of Things (IoT) concept evolves daily. These numerous technologies, such as LoRa (Long Range) transceivers, find applications in various domains, including monitoring natural disasters and those caused by human error. Security vulnerabilities arise concurrently with the advancement of these new technologies. Cyberattacks seeking to disrupt device availability, such as denial-of-service (DoS) attacks, can effectively exploit vulnerabilities in LoRa devices, hindering disaster monitoring efforts. Therefore, our goal is to assess the network parameters that impact the development of a disaster monitoring environment using LoRaWAN. Specifically, we aim to identify the parameters that could result in network availability issues, whether caused by malicious actors or configuration errors. Our results indicate that certain LoRa network parameters (collision checks, packet size, and the number of nodes) can significantly affect network performance, potentially rendering this technology unsuitable for building robust disaster monitoring systems.

1. Introduction

The development of new devices utilizing the Internet of Things (IoT) has facilitated the application of this paradigm in numerous sectors, including business, medicine, security, agriculture, transportation, and entertainment. The fundamental concept underlying this idea is utilizing diverse objects or devices, such as radiofrequency identifiers, sensors, actuators, cameras, and smartphones. Interconnected through various networking techniques, these devices can collaborate to achieve many shared objectives [1].

An important application of IoT pertains to disaster management, encompassing both natural occurrences and those resulting from human error. Examples of this application include monitoring volcanic activities [2], implementing earthquake early warning systems [3], and deploying online monitoring systems for forest fire detection [4]. While detection devices and early warning systems cannot prevent natural disasters, they play a crucial role in preparing the population and initiating contingency measures to mitigate the extent of the disaster’s consequences [5].

LoRa (Long Range) is a radio frequency technology for transceivers that employs spread spectrum modulation techniques [6]. This technology utilizes more bandwidth for transmission than the original message requires while maintaining the signal strength, thereby increasing the difficulty of interception or obstruction [7]. LoRa technology was developed by Semtech specifically for IoT devices and applications that necessitate the collection of thousands of sensor readings from various regions, including those with limited access [6]. Consequently, it finds application in a wide range of fields, including IoT [8,9], smart cities [10,11], and disaster management [12,13,14,15,16,17].

The LoRaWAN protocol defines the architecture and communication rules for LoRa technology. LoRaWAN, short for Low Power Wide Area Network (LPWAN) [18], utilizes transceivers with extensive operational range, low power consumption, cost-effectiveness, and high scalability [19]. In light of the contemporary significance of disaster detection, these devices must incorporate robust security mechanisms to ensure uninterrupted service and secure communication. In other words, understanding the robustness of LoRa devices when developing disaster monitoring solutions is a relevant research problem.

1.1. Literature Review

In order to understand the current state of research in LoRa applications, particularly the disaster monitoring scenarios, we provide a discussion regarding the following topics: (i) solutions and architectures that use LoRa technology for disaster monitoring [12,13,14,15,16,17], (ii) vulnerabilities in LoRaWAN devices and networks, which can be exploited to cause denial-of-service attacks [19,20], and (iii) performance studies in LoRa networks.

Ref. [12] proposed a LoRa mesh network system for forest fire detection and location alerts through the Google Maps application. This system employs multiple nodes comprising Arduino Uno microcontrollers, LoRa modules, DHT11 humidity and temperature sensors, and MQ2 gas sensors. The authors concluded that, within the forest, it was feasible to transmit data using a LoRa device up to a maximum distance of 500 m, with an acceptable Received Signal Strength Indication (RSSI) level threshold for data transmission within the forest set at -136 dBm. Data may be partially or entirely lost if the RSSI level falls below -136 dBm. To mitigate data collisions in a 4-node mesh network, they recommended a transmission interval exceeding 20,200 milliseconds. This research shares similarities with ours, as it also seeks to identify optimal LoRaWAN network configurations for monitoring potential disaster scenarios.

In their study [13], the authors proposed a flood monitoring system utilizing LoRa technology, which they tested in a real-world scenario. Information is collected by a device equipped with sensors and a microcontroller connected to a LoRa wireless module for data transmission. Subsequently, the data are processed and stored within a web-based infrastructure, where an alarm function is implemented during a flood. Our research also focuses on the investigation of flood monitoring systems.

The research presented in [14] addresses the challenge of selecting alternative communication methods when energy providers and the Internet experience disruptions. The authors employ a LoRaWAN architecture to establish the essential parameters, enabling residents of Coquimbo, Chile, to communicate their status to emergency services and public authorities following an earthquake. In alignment with the authors’ efforts, our study also delves into diverse system configurations to assess their attributes and constraints, enhancing our comprehension of their scalability and adaptability to other contexts.

In [20], the authors identified three vulnerabilities in the LoRaWAN protocol specification that could be exploited to launch denial-of-service attacks. They conducted simulations and analysis using the CPNTools tool, confirming the exploitability of these vulnerabilities in DoS attacks. The first vulnerability pertains to the absence of cryptographic or digital signature protection for ’beacons’ transmitted over ’gateways’ to device class B receive windows. This renders ’beacons’ susceptible to manipulation through eavesdropping and replay attacks. Exploiting this vulnerability can lead to the desynchronization of the receive windows of Class B devices, resulting in a denial-of-service for their downlink traffic. The analysis conducted by the authors further affirms that this vulnerability persists in the LoRaWAN v1.1 specification. Vulnerabilities two and three are also linked to eavesdropping and replay attacks.

In the study presented in [19], the authors investigate the resilience of LoRaWAN concerning energy attacks, which seek to deplete the energy of devices connected to the network. The author assumes that the device’s maximum energy consumption occurs during transmissions, with slightly lower consumption during receptions and very low consumption when the device is idle. Consequently, it was deduced that prolonging the energy consumption duration during transmissions and receptions would compromise the device’s overall energy efficiency. The study analyzed three methods for increasing energy consumption. The first method involves implementing a denial-of-service (DoS) attack. To maintain connectivity and manage device mobility, a device may periodically issue a link verification request or request confirmation of its data packet. In such cases, if no response is received, the device may assume that channel conditions have become more challenging, prompting it to switch to a higher transmit power or Spreading Factor (SF). Both options result in increased power consumption for the device. Specifically, depending on the device’s SF (Spreading Factor), the demonstrated attack increased the total power consumption during a single communication event by a range of 36% to 576%. Notably, this attack does not require the attacker to possess keys or sensitive data, and it can be executed against any class of LoRa devices.

Ref. [15] uses Unmanned Aerial Vehicles (UAVs) as intermediaries to transfer messages between ground-based Long Range (LoRa) nodes and remote base stations. These UAVs create an ad hoc WiFi network, serving as relays for communication. To enhance efficiency, a distributed topology control algorithm is introduced for the UAVs, which adapts to the movement of ground-based LoRa nodes. Simulation results demonstrate the feasibility of this approach, showcasing improvements in packet reception rate and average delay quality of service (QoS) metrics within a UAV-enabled LoRa network.

Ref. [16] discusses the need for communication solutions in the aftermath of disasters when the infrastructure is damaged. According to the authors, the main issue with existing systems is their limited communication range. The authors use LoRa to establish communication among rescuers and external parties, employing the epidemic forwarding protocol. The solution was implemented on commercially available PyCom LoPy4 devices equipped with LoRa and programmed using MicroPython. Initial results indicate that the proposed solution is viable for use by rescuers, and further improvements are planned to optimize its performance in real post-disaster rescue operations.

Ref. [17] proposes a relay station network using low-power wide-area network (LPWAN) technology to monitor environmental conditions in 3G-lacking, landslide-prone areas in Thailand. This network includes relay stations that transmit data from monitored areas to a destination station, which then sends it to a server via the 3G network. The study conducts experiments using the LoRa protocol for multi-hop communication with simple LoRa-based relay stations, focusing on packet loss and coverage area. The results indicate that adding a relay station can extend the communication range to 800 m, albeit with a reduced packet receiving rate (PRR), suggesting the need for a higher-layer protocol to check received packets.

Regarding the performance of LoRa networks, [21] provides a comprehensive overview of this technology and its simulation tools. It highlights the increasing adoption of LoRaWAN in various IoT applications due to its low-power consumption and long-range communication capabilities. The authors conducted a literature survey, identifying variations in parameter configurations across studies, which hinders the reproducibility of results. The authors state that many simulation tools currently focus on European frequency scenarios (868 MHz) and often overlook variations needed for other regions. The authors also note the lack of detailed energy models that account for the entire system’s consumption, including microcontrollers and sensors, which limits the accuracy of network performance assessments. The study underscores the importance of defining and documenting simulation parameters to enhance the reproducibility of results.

Ref. [22] focuses on the performance of LoRaWAN technology, specifically examining Class B and Class C devices, which have been less studied than Class A. The authors conducted a simulation-based performance analysis using the ns-3 simulator to identify critical network parameters affecting these device classes. The study found that optimizing network parameters such as increasing the number of parallel receive paths at gateways and prioritizing reception over transmission significantly improves the performance of LoRaWAN Class B and Class C devices, particularly in uplink packet delivery ratios and confirmed packet success ratios, while also indicating that certain configurations yield diminishing returns beyond specific thresholds.

The study conducted in [23] provides a comparative analysis of two Internet of Things (IoT) protocols: 6LoWPAN and LoRaWAN, focusing on their performance in medium to large-scale scenarios with varying data rates. The study utilizes the NS-3 simulator to evaluate key performance metrics such as throughput, latency, and energy consumption. The main findings indicate that while LoRaWAN exhibits lower latency than 6LoWPAN, the latter demonstrates better throughput and lower average energy consumption across the tested scenarios. Specifically, as the number of nodes increases, packet throughput decreases for both protocols due to network congestion, but 6LoWPAN maintains a slight advantage in throughput. The authors conclude that 6LoWPAN can perform comparably to LoRaWAN and is suitable for deployment in medium to large-scale applications, providing a valuable baseline for future testbed implementations involving numerous nodes.

Ref. [24] focuses on optimizing the placement of LoRaWAN gateways in smart agriculture settings. It employs various clustering algorithms to enhance the network’s performance, particularly regarding energy efficiency. The authors used the LoRaWANSim Simulator to assess the following performance metrics: Uplink Data Rate (ULDR) and the total and mean energy consumption. They found that the FCM and K-Means clustering techniques significantly enhance energy efficiency and reduce power consumption across different scenarios.

Ref. [25] aims to model the performance of a multi-hop LoRaWAN linear sensor network for energy-efficient pipeline monitoring systems by employing an analytical approach that evaluates network performance metrics, identifies limiting factors such as bit rate and duty cycle, and compares single-hop and multi-hop scenarios, ultimately demonstrating the trade-offs and considerations necessary for effective deployment of such networks. The authors emphasize the importance of considering application requirements and network throughput when designing and evaluating multi-hop LoRaWAN networks.

The primary distinction between our research and related studies lies in our evaluation of the varying scenarios’ effects on degrading the performance of a LoRa network, specifically within the context of disaster monitoring. Prior research does not delve into this particular issue. Furthermore, aspects such as selecting a real-world disaster monitoring environment (Belo Horizonte, Brazil) and choosing a simulator for conducting experiments (LoRaSim) are additional facets that have not been addressed in prior works.

1.2. Contributions and Paper Organization

As discussed in the literature review, several studies have explored the performance of LoRaWAN networks across various scenarios. However, a gap in the literature regarding performance degradation in disaster monitoring contexts remains. This paper aims to address this gap by evaluating the impact of different network parameters on the performance of LoRaWAN-based disaster monitoring scenarios. Specifically, we investigate which parameters could lead to network exhaustion, akin to denial-of-service attacks. For this, we constructed an experimental disaster management scenario focusing on critical flooding points in Belo Horizonte, Brazil. Subsequently, we utilized LoRaSim, a discrete-event simulator based on SimPy, to simulate diverse experimental scenarios centered on LoRa networks.

The remainder of this paper is structured as follows. Section 2 offers a concise overview of LoRa technology, introduces the LoRaSim tool, details our simulated disaster management scenario, which is based on the city of Belo Horizonte, Brazil, and present the experimental methodology. Section 3 presents the experimental results. Section 4 discusses the impact of the results and summarizes our findings. Finally, in Section 5 we provide concluding remarks and suggest potential avenues for future research.

2. Methods

This section describes the simulator, the network parameters and the simulated disaster management scenario. Figure 1 outlines the proposed method.

The first step involves defining the disaster monitoring scenario, which will be presented in Section 2.2. Next, we define the network for disaster monitoring and the simulator. Further details on this step are provided in Section 2.1 and Section 2.1.1. Once the network and simulator are defined, we establish the simulation parameters based on the specified environment. Section 2.2 details this process. With the parameters in place, we can construct experimental scenarios to evaluate network performance, conduct the simulations, and analyze the results. These aspects are discussed in Section 3 and Section 4.

2.1. Simulating LoRa Networks

LoRa (Long Range) is a radio frequency technology for transceivers developed by Semtech, employing spread spectrum modulation techniques [6]. Consequently, it utilizes more bandwidth for transmission than the original message requires while maintaining signal strength, thus enhancing resistance to interception or disruption [7]. A key advantage of LoRa lies in its long-range capabilities. A single gateway or base station can cover entire cities or hundreds of square kilometers, with the range determined by deployment environments and their associated obstructions [26].

LoRaWAN is an LPWAN (Low-Power, Wide Area Network) protocol designed for operation on LoRa networks. LoRaWAN networks are characterized by a star topology and consist of various components, including devices or nodes (often referred to as end-points or end-devices), base stations (also known as gateways), network servers, and application servers [27].

Data transmitted by a node (sensor or reader) are typically received by multiple gateways, meaning nodes are not tied to specific gateways. Each gateway forwards the packet received from the node to the cloud-based network server via a backhaul (such as cellular, Ethernet, satellite, or Wi-Fi). The network server holds responsibility for managing the information routed through gateways, filtering redundant packets, conducting security checks, and optimizing data rates. Subsequently, the web server dispatches the packets to the application server, which executes specific actions based on the received information [26,28].

Another significant feature of devices utilizing LoRa technology is their low power consumption, as highlighted in a study by Bouguera et al. [29]. This attribute results from the asynchronous behavior exhibited by the nodes, which engage in communication solely when data is prepared for transmission. In essence, they operate in an event-driven or scheduled manner.

2.1.1. LoRaSim

LoRaSim is a custom-built discrete event simulator developed by Thiemo Voigt and Martin Bor at Lancaster University in the United Kingdom. It is implemented in Python and relies on the SimPy, Matplotlib, and Numpy libraries. This tool enables the simulation of a network consisting of N LoRa devices and M base stations within a two-dimensional space, offering options for grid layout or random distribution [30].

According to [30], each LoRa device possesses distinct communication characteristics, delineated by transmission parameters including Transmission Power (TP), Carrier Frequency (CF), Spreading Factor (SF), Bandwidth (BW), and Coding Rate (CR). The transmission behavior of each node is described by the average transmission rate of packets $\lambda$ and the payload of packet B. The behavior of the nth node during a simulation is therefore described by the tuple $S{N}_n=\{TP;CF;SF;BW;CR;\lambda ;B\}$. Furthermore, it is worth noting that each LoRa base station is equipped to receive multiple signals, each characterized by varying combinations of SF and BW for a given CF.

The primary metrics employed to gauge network scalability and performance are the Data Extraction Rate (DER) and Network Energy Consumption (NEC). The DER metric quantifies the ratio of packets received by base stations to those transmitted by network nodes within a specified time frame. Meanwhile, the NEC metric measures the energy consumption (in Joules) the network requires to extract a packet or message successfully. Throughout this study, we will specifically utilize the DER metric.

The DER value ranges from 0 to 1 and is contingent upon factors such as the quantity, placement, and conduct of nodes and base stations. A DER value approaching 1 signifies higher efficiency in the LoRa network implementation. Equation (1) is employed to calculate the DER value, where PR denotes the count of packets received by at least one base station during the simulation run, and PE represents the cumulative sum of packets transmitted by all nodes over the simulation time interval.

$$DER={\displaystyle \frac{PR}{PE}}$$

(1)

2.1.2. LoRaSim—Input and Output

The simulator expects the following parameters as input:

• Nodes (Nodes)—number of nodes or devices in the LoRa network;
• Avgsend (Average sending interval)—average packet sending time interval in milliseconds;
• Experiment—an integer that determines with which radio settings the simulation is performed:

  0.

  Use the settings with the slowest data rate (SF12, BW125, CR4/8).

  1.

  Similar to experiment 0, but uses a random choice of 3 transmission frequencies.

  2.

  Use the settings with the fastest data rate (SF6, BW500, CR4/5).

  3.

  Optimizes configuration per node based on distance from gateway.

  4.

  Uses the settings defined in LoRaWAN (SF12, BW125, CR4/5).

  5.

  Similar to experiment 3, but also optimizes transmit power (TP).

  It is important to note that the work focuses on LoRaWAN networks; thus, all experiments were performed using the parameter Experiment with the value 4, that is, the settings defined for LoRaWAN.
• Simtime—total simulation run time in milliseconds.
• Basestations—number of base stations or gateways used to simulate. This value can assume the integers 1, 2, 3, 4, 6, 8 or 24.
• Collision—the parameter can assume values of 0 or 1. A value of 1 is chosen to activate full collision checking, while 0 is used to employ a simplified check (the default option). With simplified checking, two messages collide if they arrive simultaneously at the same frequency and spreading factor. On the other hand, full collision checking takes into account the capture effect [30], where one of the two colliding messages may still successfully transmit based on the relative timing and the difference in received power. Therefore, full collision checks, such as this one, provide a more precise assessment of packet collisions, which is crucial in scenarios where network reliability is critical, such as disaster monitoring. However, these methods might be computationally intensive due to the detailed analysis required to detect collisions accurately [31].
• Directionality—can assume values 0 or 1. The value 1 is selected to activate the directional antenna for the nodes.
• Networks (LoRa Networks)—number of LoRa networks.
• Basedist (Distance between base stations)—Distance X between two base stations.

The main output parameters found during the execution of experiments using the simulator are as follows:

• nrCollisions—number of packet collisions during the experiment.
• Energy (in J)—NEC in Joule.
• Sent packets—number of packets sent by network nodes.
• Received Packets—number of packets received by base stations.
• Lost Packets—number of lost packets, i.e., packets that any base station did not receive.
• DER—proportion of packets received by packets sent.
• Packets at BS X—number of packets received by base station X (X varies from 1 to 24, according to the number of base stations provided as input to the simulator).
• Send to BS [X]—number of packets sent to base station X (X varies from 1 to 24, according to the number of base stations provided as input to the simulator).
• DER BS [X]—represents the ratio of packets sent to base station X and those received by base station X (X varies from 1 to 24, depending on the number of base stations specified as input to the simulator).

In addition to the outputs described above, the simulator also provides values for minimum airtime, Lpl (path lost), symbol time, received signal strength indication (RSSI), maximum possible distance between nodes and base stations, xy coordinates of base stations, xy coordinates of nodes and information about collisions.

2.2. Description of the Simulated Disaster Management Scenario

Belo Horizonte, Brazil, annually grapples with floods resulting from heavy rainfall and substantial precipitation. These torrential rains lead to landslides, flooding, destruction, property loss, and, tragically, even loss of life. The city has identified nine critical flooding areas dispersed across seven distinct regions [32,33].

The Córrego do Leitão watershed in the Center-South region of Belo Horizonte encompasses twenty-seven sub-basins and is particularly susceptible to the abovementioned issues. This susceptibility arises from its steep terrain and numerous narrow stretches of streams. We used the Córrego do Leitão watershed as our simulated monitoring environment. Figure 2 visually represents the Córrego do Leitão watershed.

To align the simulation environment with real-world conditions, several experiments were conducted to determine the following aspects: (i) the required number of nodes (LoRa devices), (ii) the number of base stations necessary to support the environment’s nodes and the optimal spacing between them, (iii) the average packet sending interval, and (iv) the number of LoRa networks essential for monitoring the environment.

After running several preliminary experiments and considering the characteristics of the chosen disaster scenario, we defined the following values for the simulator parameters:

• Number of nodes (LoRa devices)—48 (N);
• Number of base stations needed to support environment nodes—24 (BS);
• Distance between two base stations—200 m;
• Average packet sending interval—50,000 ms (APSI);
• Number of LoRa networks needed to monitor the environment—1.

Simulations were conducted using LoRaSim to determine the optimal number of base stations and the average packet transmission interval for monitoring an environment with 27 nodes. The average packet transmission interval was varied, starting at 1000 ms, then 5000 ms, and increasing in increments of 5000 ms up to 60,000 ms. The same experiment was performed for each possible number of base stations supported by the simulator (1, 2, 3, 4, 6, 8, and 24). Network performance was evaluated using the DER metric. The results indicate that the highest DER values were obtained when the number of base stations was set to 24, and the average packet transmission interval was 50,000 ms. We chose a distance of 200 m because preliminary experiments showed that shorter distances increased packet collisions. Regarding the number of nodes, the initial plan was to use one node for each of the 27 sub-basins. However, this number had to be reduced to 24 to ensure that the number of nodes was a multiple of the number of base stations. To provide a minimum level of redundancy, we decided to double the number of nodes from 24 to 48, resulting in approximately 2 nodes per sub-basin.

To investigate the performance of the LoRa network, we adjusted key parameters that could influence its behavior. Specifically, we increased the number of nodes (N), reduced the average transmission interval (APSI), and decreased the number of base stations (BS). Additionally, we conducted experiments by modifying multiple parameters and directly altering the simulator code.

We also conducted the experiments using both types of collision checking: standard collision checking (SCC) and full collision checking (FCC). As previously discussed, full collision checking considers the capture effect, which allows one of the two colliding messages to pass through based on relative timing and differences in received power. In contrast, standard checking does not consider this effect.

Table 1. Description of proposed attacks on the availability of LoRa networks.

Scenario	Goal	Variables	# of Experiments
1	Analyze the impact of increasing the number of nodes on the network availability	number of nodes (N)	4
2	Analyze the impact of decreasing the average packet sending interval on the network availability	average packet sending interval (APSI)	4
3	Analyze the impact of increasing both the number of nodes and decreasing the average packet sending interval on the network availability	number of nodes (N) and average packet sending interval (APSI)	1
4	Analyze the impact of decreasing the number of base stations and increasing the number of nodes on the network availability	number of base stations (BS) and number of nodes (N)	1
5	Analyze the impact of increasing packet length (size) and decreasing the average packet sending interval on the network availability	packet size (PS) and average packet sending interval (APSI)	1

outlines the proposed scenarios’ objectives to impact network availability. Furthermore, it details the corresponding variables that will be adjusted in each scenario and specifies the number of experiments conducted for each scenario. Subsequent sections of this article provide comprehensive descriptions of each scenario and its respective experiments.

Disaster monitoring and alert applications may require a high DER, particularly in the moments leading up to a disaster, to ensure the timely initiation of necessary contingency measures. Therefore, we define optimal performance for disaster monitoring applications as a DER exceeding 0.95, corresponding to a packet loss rate of no more than 5%.

3. Experimental Results

Here, we will present the results of each scenario described in Table 1. In broad terms, these experiments aim to assess how various parameters of a LoRa network, including the number of nodes, average packet sending interval, number of base stations, and packet size, impact the network’s availability within the context of disaster management.

3.1. Scenario 1—Increase the Number of Nodes

The first scenario involves augmenting the number of nodes within the LoRa network while leaving the remaining parameters unchanged. The objective of conducting these simulations is to ascertain whether the influx of devices into the simulated monitoring network of the Córrego do Leitão watershed leads to an overload. The rationale behind this is that a higher number of nodes results in increased packets transmitted within a specific time interval.

In the first scenario, we conducted three experiments. The first one involved a slight increase in the number of nodes, the second experiment involved a more substantial increase in the number of nodes, and the third experiment aimed to amplify further the node count to degrade network performance using the full collision check deliberately. Details of the experiments within scenario 1 can be found in

Table 2. Scenario 1—experiments and network parameters.

ID	Min. Number of Nodes	Max. Number of Nodes	Increasing Node Rate	Metric
1	24	240	24	DER
2	24	12,000	1200	DER
3	12,000	48,000	2400	DER

.

In our first experiment of Scenario 1, we incrementally increased the number of nodes by one per base station. Given that our baseline comprises 24 base stations, the total number of nodes was progressively raised to a maximum of 24 nodes per base station, resulting in a total of 240 nodes. We conducted the same experiment using standard and full collision checking. To account for the stochastic nature of collisions and node placements in LoRaSim, we performed 100 simulations for each node count configuration and calculated the average DER.

In the second experiment of Scenario 1, we progressively increased the number of nodes at a rate of 1200, equivalent to 50 nodes per base station (24 in total), until we reached 12,000 nodes. This experiment aimed to assess the network’s performance as the number of connected devices escalated, comparing the standard collision check with the full collision check. Due to the substantial increase in execution time caused by the larger node count, we conducted only 10 simulations for each configuration. We calculated the DER average as a function of the node increase.

To intentionally degrade network performance, even with full collision checking enabled, we conducted a third experiment involving increasing the number of nodes. This time, we augmented the number of nodes per base station by 100, resulting in an incremental node rate of 2400.

Our results for the first experiment of Scenario 1 are depicted in Figure 3. In this figure, the blue curve illustrates the DER variation concerning the increase in the number of nodes for the full collision check. In contrast, the orange curve portrays the DER variation as the number of nodes increases for the standard collision check.

We observe that the initial increase did not affect the network performance concerning the DER metric. The DER rate remains above 0.95 for both types of collision checking, even when operating with 10 nodes per base station, totaling 240 nodes in the LoRa network. Thus, the network continues to meet the requirements we set for disaster monitoring scenarios.

Even though the increase in the number of nodes leads to a rise in the number of packets transmitted and, consequently, an increase in collisions, our configured network maintains an average packet sending interval of about 1 min and utilizes 24 base stations for receiving and forwarding packets to the network servers. This setup, therefore, represents a robust configuration capable of accommodating this volume of nodes.

Figure 4 illustrates the results of the second experiment within Scenario 1. This figure portrays the variation in the DER metric of the LoRa network, as impacted by the substantial increase in the number of nodes for both the standard collision check and the full collision check. We can observe that with 1200 nodes, the DER for the standard collision check (0.719737) falls below 0.95 and continues to decrease to 0.01412 when the network reaches 12,000 nodes. In contrast, the DER for the full collision check remains above 0.95 and only drops below this threshold when the network reaches 8400 nodes, representing a remarkably high number of connected devices.

As mentioned in Section 4, our primary goal in the third experiment of the first scenario was to attempt to overload the network by significantly increasing the number of nodes for the full collision check. In this case, there was no need to experiment with the standard collision check, as the previous experiment had already yielded a DER value close to zero using this parameter. Figure 5 illustrates the results of Experiment 3 of the first scenario.

From the results we have obtained, we can observe the robustness of the LoRa network implementation when exposed to a substantial surge in the number of nodes while utilizing full collision detection. Remarkably, even with a substantial node count (48,000), the LoRa network continues to function at a capacity exceeding 50%.

3.2. Scenario 2—Decreasing the Interval Between Sent Packets

In the second scenario, we aim to reduce the average packet sending interval while keeping the other input parameters constant. This approach allows us to assess the effects of diminishing this parameter on the LoRa network. We anticipate an increased load on the monitoring network because a smaller average packet sending interval results in more packets sent per node.

Table 3. Scenario 2—experiments and network parameters.

ID	Maximum Packet Sending Interval	Minimum Packet Sending Interval	Decreasing Rate	Metric
1	50,000 ms	1000 ms	5000 ms	DER
2	1000 ms	100 ms	100 ms	DER
3	100 ms	10 ms	10 ms	DER

describes the experiments in scenario 2.

In Experiment 1, we reduced the average packet sending interval by 5000 ms, commencing from two initial configurations: 50,000 ms (maximum) and 1000 ms (minimum). We conducted 100 repetitions of each experiment to account for node placement and collision variability. For each packet sending interval, we calculated the average DER.

The second experiment aimed to explore the consequences of further reducing the average packet sending interval. We initiated at 1000 ms and incrementally decreased by 100 ms until reaching 100 ms. We executed the simulation ten times for each average packet sending interval and computed the average DER for each interval.

To further explore the extent of network degradation, even with full collision checking enabled, we conducted a third experiment in which we gradually reduced the average packet sending interval by 10 ms, starting at 100 ms and continuing until it reached 10 ms.

Figure 6 presents the outcomes of the initial experiment within Scenario 2. The blue curve illustrates the DER variation in response to the average packet sending interval reduction for the full collision check. In contrast, the orange curve represents the same for the standard collision check.

The results depicted in Figure 6 reveal that even with a reduction in the average sending time to 5000 ms, the LoRa network maintains a DER greater than 0.95 for both the standard and full collision checks. This can be attributed to the use of only two nodes due to the high number of base stations (24). Consequently, despite the increased number of packets sent over time, the limited number of nodes transmitting packets allows the base stations to process them efficiently. Notably, a decline in the DER value is observed for the standard collision check when the average packet sending time is reduced to 1 s.

Figure 7 provides an overview of the results from the second experiment within Scenario 2. This experiment was designed to evaluate the performance of the LoRa network when subjected to a significant reduction in the average packet sending time, both for the full collision check and the standard collision check.

In this experiment, we reduced the average sending interval to values smaller than 1 s. Notably, the DER for the full collision check remained consistently above 0.95, even when reduced to 1/10th of a second. However, for the standard collision check, the DER dropped significantly, reaching values close to zero when the interval was reduced to 1/10th of a second.

This disparity in DER between collision checks can be attributed to a key factor: when the average packet sending interval is significantly reduced, a larger number of packets may be sent simultaneously, arriving at the same time, on the same frequency, and with the same propagation factor. In the case of full collision checking, some of these packets can still pass through and be delivered to a base station. However, with standard collision checking, these packets are not delivered, resulting in a reduced rate of packets delivered per packet sent.

The results of our third experiment in Scenario 2 are presented in Figure 8. Remarkably, even when we reduced the average packet sending interval to 1/100th of a second, the DER for full collision checking did not experience any decrease. This can be explained by the collision behavior described in the previous experiment.

3.3. Scenario 3—Increasing the Number of Nodes and Decreasing the Interval Between Sent Packets

In Scenario 3, we conducted a single experiment where we doubled the number of nodes, increasing it from 48 to 1536, while simultaneously reducing the average packet sending interval from 50,000 ms to 1000 ms. This experiment aimed to assess the network’s performance in terms of the DER metric when subjected to a high influx of packets. Both changes in these parameters were expected to result in an overload of network traffic.

We executed this experiment for both standard collision checking and full collision checking. For each configuration of average sending interval and number of nodes, we ran the simulations 10 times. We calculated the average DER concerning the increase in the number of devices on the network and the reduction of the average interval for sending packets.

The results of the experiment we conducted for Scenario 3 using full collision checking are illustrated in Figure 9, while the results for the standard collision check can be seen in Figure 10. Each colored curve in the figures represents the DER variation as a function of the average packet sending interval reduction for a specific number of nodes.

By analyzing Figure 9 and Figure 10, we can observe that, for the first time during our experiment executions, the DER decreased for both types of collision verification. This reduction occurred due to the changes in the two parameters we introduced in the experiment, leading to a significant increase in the number of packets. This increase exceeded the capacity of the 24 base stations specified in our setup.

3.4. Scenario 4—Decreasing the Number of Base Stations and Increasing the Number of Nodes

The objective of Scenario 4 was to simulate a scenario where numerous devices send packets to a limited number of base stations. To achieve this, we conducted 100 simulations for each configuration, systematically doubling the number of nodes from 48 to 3072 while using various numbers of base stations (24, 8, 6, 4, 3, 2, 1) for both collision verification methods, standard and full collision checks.

In Figure 11, we present the DER results for the full collision check, illustrating the impact of increasing the number of nodes while decreasing the number of base stations. The results for the standard collision check are shown in Figure 12. Each curve with a distinct color represents the DER value in relation to the increase in the number of nodes for a specific number of base stations.

From the results presented in Figure 11 and Figure 12, we can observe how the number of nodes significantly impacts the DER when fewer base stations are connected to the network. This result holds true for both the full collision check and the standard collision check. Specifically, when the network comprises 768 nodes, and the standard collision check is applied, the DER falls below 0.95, regardless of the number of base stations. In the case of full collision verification, with 3072 nodes, we can observe that the DER drops below 0.95 for all numbers of base stations except 24.

In conclusion, increasing the number of nodes while decreasing the number of base stations leads to a substantial decline in network performance. This can be attributed to more nodes transmitting packets within the same time interval, and a limited number of base stations cannot efficiently handle the reception of packets from numerous nodes. Consequently, the increased collision rate further explains the notable reduction in network performance, especially when relying solely on standard collision checking.

3.5. Scenario 5—Increasing the Packet Length and Decreasing the Interval Between Sent Packets

The idea behind Scenario 5 is to simulate an attack involving a fast transmission of large packets. Therefore, for each average sending time setting, ranging from 50,000 ms to 1000 ms (decreasing in 10,000 ms increments), we increased the packet size at a rate of 51 bytes, starting from 10 bytes and concluding at 255 bytes. We conducted 100 experiments for each configuration and calculated the average results for the DER metric.

The outcomes of the experiment for Scenario 5, utilizing the full collision check, are visually depicted in Figure 13, while those for the standard collision check are presented in Figure 14. Each colored curve illustrates the DER value in relation to the reduction in the average packet sending interval for different packet size values.

From this experiment, we can conclude that when the packet size is increased, and the average packet sending interval is decreased, the network experiences a performance decline when standard collision checking is employed. For instance, when the average sending time is 1000 ms, and the packet size is 255 bytes, the resulting DER is approximately 0.11. This is notably low for applications requiring sensitivity, such as disaster monitoring. However, in scenarios where full collision checking is enabled, even with the maximum packet payload and a low average sending time (1000 ms), we achieve a DER higher than 0.95.

4. Discussion

Based on the experiments conducted, the performance degradation of LoRa networks in disaster monitoring scenarios can be categorized into two situations. The degradation is more pronounced when using standard collision verification than when using full collision verification.

Table 4. Summary of our findings. N = number of nodes, APSI = average packet sending interval, BS = number of base stations, PS = packet size, SCC = standard collision check and FCC = full collision check.

Scenario	Variables	Parameter Thresholds Affectingthe Network Performance (DER)
1	number of nodes (N)	SCC and N ≥ 1200; FCC and N ≥ 19,200
2	average packet sending interval (APSI)	SCC and APSI ≤ 1000 ms
3	number of nodes (N) and average packet sending interval (APSI)	FCC, APSI ≤ 1000 ms and N ≥ 768 SCC, APSI ≤ 40,000 ms and N ≥ 768
4	number of base stations (BS) and number of nodes (N)	FCC, BS = 2 and N = 1500 SCC, BS = 2, N = 500
5	packet size (PS) and average packet sending interval (APSI)	SCC, APSI ≤ 10,000 ms and PS ≥ 204 bytes SCC, APSI = 1000 ms and PS ≥ 10 bytes

presents a summary of our findings and identifies the parameter thresholds that may impact network performance. It is possible to see that most scenarios involves the use of standard collision check. However, in some cases, even when the full collision check is used the scenario might have network issues. It is important to remember that collision-checking methods might be computationally intensive due to the detailed analysis required to detect collisions accurately. This should be considered when deploying LoRa networks for disaster management scenarios.

An example of this behavior is observed in Experiment 2 of Scenario 2, where the average packet transmission interval is reduced to 100 ms. The network performance remains high for full collision verification, even in the worst-case scenario, maintaining a DER value of at least 0.95. In contrast, the DER value approaches zero for standard collision verification in the worst case. In other words, we would have a packet loss around 100%. This behavior is also observed in other scenarios, such as Scenario 5, where the packet payload size increases while the average transmission interval decreases.

Another key finding of this study is that simultaneous variations in two parameters lead to a significant deterioration in network quality. For instance, in Scenarios 3, 4, and 5, complete and standard collision verification resulted in a DER value below 0.50 in the worst cases. Two studies investigating the impact of network performance on disaster monitoring solutions support our findings. The authors of [15] evaluate the performance of UAV-enabled LoRa networks for disaster management applications. The authors conclude that an increase in the number of ground nodes and a higher number of UAVs can lead to a 20% increase in packet loss, directly affecting incident response. Similarly, Ref. [34] examines the impact of Public Safety Networks (PSNs) using LTE device-to-device (D2D) communication. The authors find that the intensity of traffic per node (in our case, the average packet transmission interval) becomes the dominant factor, surpassing parameters such as queuing capacity, as the system can rapidly become unstable.

In conclusion, configuring a LoRaWAN network for disaster monitoring must ensure robustness so that, even if one network parameter is affected, the overall system can maintain good performance, thereby mitigating the consequences of the disaster. We recommend the use of network traffic monitoring services such as Zabbix and Nagios for anomaly detection and the development of contingency and recovery plans in case the network becomes overloaded.

5. Conclusions and Future Work

The main objective of our work was to assess the impact of DoS attacks on LoRa devices. To achieve this, we first needed to determine the configurations for simulating the designated disaster environment (Córrego do Leitão watershed—Belo Horizonte, Brazil). Once we established the baseline for monitoring this area, we conducted experiments to analyze the performance of the LoRa network under various conditions, aiming to degrade its availability and simulate the effects of a denial-of-service attack.

These experiments significantly impacted network performance, particularly when employing standard collision checking. However, with full collision checking, the network’s performance remained optimal in many scenarios. Furthermore, based on the conducted experiments, we concluded that when multiple parameters are manipulated to attack the network, its performance experiences a substantial decline.

We conclude that in the context of disaster monitoring, the configuration of a LoRaWAN network must be robust. This ensures that even if one network parameter is affected, the others can maintain optimal performance, enabling continuous monitoring and mitigating the consequences of disasters. We recommend using network traffic monitoring services to detect anomalies and develop contingency and recovery plans in the event of a DoS attack.

For future work, we propose applying the network parameters established in this study to another LoRa simulator, such as LoRaWanSim [35], which does not use random node distribution. This allows us to test the proposed scenarios for network overload and observe the results. Additionally, we suggest focusing on experiments that simultaneously assess energy consumption (NEC) alongside data extraction rate (DER), which was impossible due to simulator limitations. Another potential research direction is evaluating the impact of machine learning-based optimization on the performance of LoRa networks in disaster monitoring scenarios. Finally, we aim to implement a LoRa network using real devices in order to verify the impact of reducing the average packet sending interval, decreasing the number of base stations, and increasing packet size.