5.1. Security Verification
This section performs security verification for the four security threats defined in
Section 2.3: Dos/DDoS, MITM, spoofing, and duplicate update. In addition, we categorize existing works into three groups, as shown in
Table 3, to explore effective methodologies that fulfill security requirements by verifying the security of these existing studies. Finally, a comparative analysis was performed to assess and analyze these methodologies.
Table 4 shows a comparative analysis of
MQTree and related studies in terms of security threats. First, Chandra et al. [
15] proposed an OTA firmware update system using the Rest API. This system involves periodic communication between gateway nodes and a server to check for the latest version of the firmware, which is stored in internal storage when necessary. Despite its ability to ensure user identity through TLS and prevent spoofing or man-in-the-middle attacks through encryption, TLS introduces vulnerabilities related to redundant updates, which leads to packet retransmission. Stoev et al. [
16] implemented secure communication between ESP8266 Wi-Fi modules using 256 bit AES encryption through the MQTT protocol. They introduced asymmetry in data transmission using a pseudo-random number generator. Although AES ensures data confidentiality, it addresses the issues associated with network traffic or prevents overload situations. The encryption provided by AES encryption makes it difficult for attackers to understand or manipulate the contents, even if the data is intercepted. However, this implementation lacks the ability to verify or authenticate the identity of a user or device. While it allows for simple and effective secure communication, it remains vulnerable to replay and duplicate update attacks, even with a minimized key validity period. In addition, although the MQTT protocol is designed for lightweight messaging, it lacks native features for the automatic detection and handling of duplicate message transmissions. Consequently, there is a potential risk of mistaking duplicate update messages for original messages and processing them as such. For example, if an attacker captures and resends a firmware update message, the system may mistakenly perceive a previously applied update as new and re-apply it. This scenario can lead to unnecessary resource consumption, increased system instability, and potential malfunctions. Therefore, Sahlmann et al. [
9] used the MYNO update protocol (MUP) to transmit plaintext messages without relying on TLS and used an edge architecture to mitigate certain risks. However, it cannot entirely prevent MITM, spoofing, redundant update, and DoS/DDoS attacks in the event of a security breach at the edge nodes. Da Silva et al. [
24] applied TLS to MQTT to secure the OTA system, but similar to Chandra et al. [
15], it has limitations against duplicate update attacks.
In MQTree, the use of TLS is crucial for effectively defending against spoofing and MITM attacks. Identity verification through encryption and certificates plays an important role in preventing these types of attacks. In addition, MQTree incorporates a Merkle tree-based integrity verification mechanism, which offers an effective defense against duplicate update attacks. This can be explained through two main scenarios: simple retransmission attacks and assuming TLS compromise. When an attacker attempts to confuse the system by retransmitting a previously captured packet, MQTree calculates the hash value of every firmware file within the vehicle and stores it in the Merkle tree, thus generating a root hash. By comparing this root hash during the update process, the system prevents the misidentification and re-application of updates that have already been applied. This process effectively blocks attempts at duplicate updates, thereby protecting the system’s integrity. Even in cases where TLS is compromised, MQTree’s security mechanism continues to play a vital role. Even if encryption communication is threatened due to TLS damage, Merkle tree-based integrity verification ensures the safety of update packets. Even if the firmware’s hash and the root hash are altered, the root hash included in the update packet is ultimately compared with the root hash generated within the vehicle. Consequently, if damaged firmware is stored in SecureStorage, the system can roll back or delete it to eliminate potential security threats. This process is crucial for maintaining the system’s security, even in the event of TLS damage. In both scenarios, the logs generated during the verification process can be utilized by intrusion detection systems (IDSs), providing important information about the security status. However, mechanisms such as TLS and Merkle trees do not provide sufficient defense against DoS attacks on MQTree. Given that these attacks target to deplete system resources, additional security measures, such as firewalls and DDoS mitigation techniques, are required.
5.2. Performance Validation
In order to evaluate the efficiency and applicability of MQTree and explore ways to improve performance reflecting the latest technology trends, the performance validation was divided into two sections, as follows:
Comparison with centralized network protocols: in order to provide a benchmark for
MQTree, we selected MQTT with TLS and HTTPS and MQTT without TLS for comparison. This choice was based on existing research findings, such as those from various studies [
6,
25], indicating that MQTT with TLS outperforms HTTPS. The objective was to demonstrate that this study’s results obtained in the experimental environment align with the observed trend. The comparison between MQTT with TLS and HTTPS analyzed the differences in performance among widely used communication protocols to clarify the advantages and disadvantages of
MQTree in relation to these standard protocols. Essentially, this comparison established that the results of this study conform to a general performance pattern.
Comparison with blockchain protocol: In order to address contemporary technology trends, we evaluated the performance of MQTT systems integrated with the blockchain protocol. This exploration explores novel approaches that can improve the performance of the MQTT protocol. It is important to understand how the application of blockchain affects MQTT systems and its real-world implications. Consequently, a comparative analysis was conducted to evaluate MQTT-A and traditional MQTT (specifically MQTT with TLS) alongside blockchain sharding technologies while considering the impact of broker choice (e.g., HiveMQ and Mosquitto) on the performance of MQTT systems. This comprehensive comparison highlights the roles played by blockchain technology and broker choice in shaping the performance of MQTT systems.
5.2.1. Comparison with Centralized Network Protocols
Figure 7 illustrates the performance measurements derived from 10 independent transactions with average latency calculations. The experimental setup incorporated Spring Boot version 2.7.7 for the server framework, with the server itself powered by an AMD Ryzen Threadripper PRO 5965WX CPU and 256 GB of RAM. On the client side, a system equipped with a 12th Gen Intel Core i7-1260P CPU and 16GB of RAM was used, running Python. The network configuration was established through a router supporting 100 Mbps bandwidth. For MQTT communication, Mosquitto version 2.0.11 served as the broker, and the Eclipse Paho MQTT client version 1.2.5 was utilized as the client library. The JSON payload adhered to a lightweight and efficient standard data format. Conveying the necessary information in OTA communication makes a notable contribution to saving network bandwidth and improving data transmission and processing speed. QoS was set to a default value of 1 in our experiments. The obtained results reveal an average latency of 0.467 s with HTTPS, 0.116 s with MQTT with TLS, and 0.0092 s with MQTT without TLS. Notably, MQTT with TLS demonstrated approximately four times greater speed than HTTPS.
Paris et al. [
25] noted latency variations in the MQTT protocol when implementing QoS-1, with and without TLS. For example, at a QoS-1 level of two messages per minute, latency was 0.111898 microseconds without TLS, whereas with TLS, it increased to 39.87122 microseconds, an increase of approximately 356 times. Conversely, when transmitting 10,000 messages per minute, latency increased from 248.3012 microseconds to 36,132.43 microseconds with TLS, an increase of approximately 145 times. Similarly, Tang et al. [
6] found that MQTT with TLS outperformed HTTPS by approximately 1.33 times, based on tests spanning QoS levels 0 to 2. In a single round-trip, MQTT without TLS and HTTP exhibited comparable durations, but MQTT with TLS required approximately 1.71 times longer. However, in a 10-round-trip test, MQTT with TLS was 1.33 times faster than HTTPS. Across various message payload lengths, MQTT demonstrated an average speed advantage of 1.25 times faster. This implies superior performance compared to an OTA system using the traditional Rest API proposed by Chandra et al. [
15].
Given these comparisons, MQTT with TLS demonstrates a performance advantage over HTTPS, which makes it a robust alternative for satisfying security requirements without compromising performance. Although it may slightly trail behind in certain performance aspects compared to ciphers, such as AES, this consideration is pivotal from scalability and practicality perspectives. Notably, MQTT with TLS offers the advantage of easy integration without requiring significant modifications to the existing network architecture. The performance difference between MQTT and HTTPS can be attributed to MQTT’s lightweight and efficient message-handling protocol. MQTT requires less overhead for data transfer, and its performance advantage is particularly evident when using lightweight data formats. Collectively, these factors explain the reason why MQTT with TLS outperformed HTTPS in this experiment.
5.2.2. Comparison with the Blockchain Protocol
Akshatha et al. [
23] reported that the average latency of an MQTT system with blockchain sharding at the QoS-1 level ranges from approximately 0.04 ms to 0.22 ms. In contrast, MQTT with TLS communication exhibits an average latency of approximately 0.52 ms. This indicates that MQTT with TLS maintains a lower average latency of approximately 0.3 ms to 0.5 ms compared with the proposed method. In addition, in the study by Akshatha et al. [
23], using HiveMQ at QoS-0, MQTT-A with TLS incurred a delay of 192 ms, whereas the proposed system demonstrated a reduced delay of 126 ms. This indicates that the sharding-based system outperforms MQTT-A with TLS by approximately 1.35 times. In their study using HiveMQCloud, Buccafurri et al. [
26] observed that the transmission rate of MQTT with TLS and the proposed MQTT-A with TLS are closely aligned when considering the number of useful information bits received by the subscriber in a unit of time. Furthermore, it outperformed traditional MQTT with TLS by more than 3–5 times at QoS-0 and QoS-2.
However, a crucial distinction lies in the fact that the resource usage and performance of the MQTT protocol significantly hinge on the broker selected. In this study, we employed the Mosquitto broker, whereas other studies opted for HiveMQ. According to Bender et al. [
27], the library can be efficiently implemented on resource-constrained IoT devices with fewer CPU cycles and less system memory usage, even when handling a substantial influx of incoming messages. This underscores that the resource usage and performance of the MQTT are notably dependent on the choice of broker, as shown in
Figure 8. Resource usage tests revealed that the Mosquitto broker consistently exhibited the lowest CPU and memory requirements. In alignment with the experimental environment in this study, Mosquitto demonstrated the lowest average latency compared with other libraries, with the average latency at QoS-1 approximately five times higher than that of HiveMQ. These results highlight the substantial influence of broker choice on the overall performance of an MQTT system. In conclusion, it is essential to emphasize that even when using a centralized network instead of a blockchain, performance improvement is achievable through the optimization of MQTT’s broker selection.
An MQTT system that integrates blockchain technology using the optimization strategy proposed by Gao et al. may outperform the MQTree proposed in this study, depending on the broker choice. However, in practical scenarios where organizations seek to enhance security while seamlessly integrating with existing infrastructure, MQTree proves to be a viable solution. It fulfills these requirements effectively, delivering commendable performance and security results while maintaining compatibility with existing systems. In addition, the lightweight nature of the MQTT protocol optimizes the use of network resources, which is particularly valuable in resource-constrained scenarios, such as IoT environments. Therefore, MQTree emerges as a realistic and practical solution for simultaneously enhancing security and performance while enabling flexible integration with existing systems.