In this section, we present the results obtained from the evaluation of the MPTAC and MSTAC algorithms in the proposed case study.
The results are presented separately for each algorithm in the following subsections, facilitating a more in-depth analysis of the performance of MPTAC and MSTAC.
6.2. MSTAC—Sequential Compression
A grid search was conducted for each instance of MSTAC to identify the optimal parameters for the sequential approach. The best-performing parameters are summarized in
Table 4, which also presents the corresponding compression metrics. The table presents the variables analyzed, along with their respective values for the window limit and compression factor
m. It includes key performance indicators such as CR, CF, RMSE, NCC, and
to evaluate the compression performance.
For instance, the speed variable achieves the highest compression rate of 96.83%, with a compression factor of 31.60, while maintaining a strong correlation (NCC = 0.8552) and relatively low RMSE (13.60). Conversely, variables such as RPM shows higher RMSE values, suggesting greater reconstruction error. These results indicate that the performance of MSTAC varies across different variables, depending on their specific characteristics and the optimal window size determined through the grid search.
Figure 7 shows the original values (blue line), the saved values (red dots), and the decompressed values (red line) for each variable within the interval between samples 4100 and 4600.
Figure 8 displays the probability density functions (PDFs) for each variable, comparing the distributions of the original and decompressed data. Similar to MPTAC, these results, particularly
Figure 8, validate the effectiveness of the MSTAC algorithm by demonstrating its ability to maintain data integrity while achieving significant compression performance.
In addition, to compare the algorithms,
Table 5 consolidates the results from
Table 3 and
Table 4. Generally, the variables show similar hyperparameters and compression metrics across both approaches. However, there is a notable exception for the speed variable: with
= 27 in the MSTAC approach, the compression factor is 31.60, which is more than four times higher than that of the MPTAC, without a significant negative impact on the compression error metrics.
Despite this, the MPTAC algorithm demonstrated significantly faster performance, processing a sample 4.17 times faster than the MSTAC. Consequently, for the dataset considered, MPTAC proves to be a more suitable choice, effectively balancing compression efficiency with processing speed.
Finally, we compare the performance of MPTAC against MSTAC using the same defined parameters (
= 4 and
). These values were selected by averaging the parameter values from the previous section, excluding the parameters of the TAC instance in MSTAC applied to the speed variable. This exclusion was necessary due to the significant discrepancy in parameter values for speed compared to the other variables, as shown in
Table 6. This comparison highlights the performance differences between the two algorithms under consistent settings.
In
Figure 9, we present a visual representation to facilitate the interpretation of the results.
Figure 9 contains two subfigures: (a) it compares the overall compression metrics by variable and algorithm, and (b) it presents the comparison of the absolute root mean square error (RMSE) between the variables and algorithms, highlighting differences in accuracy between MPTAC and MSTAC for the same parameterization conditions.
6.3. Discussion of Results
In this subsection, we discuss the effectiveness of the MPTAC and MSTAC algorithms based on the obtained results and address the proposed research questions.
Evaluating the effectiveness of the MPTAC algorithm in terms of CR (Question 1), MPTAC and MSTAC, it is observed that MPTAC, with the parameters = 5 and , presented a CR of 86.25%, while MSTAC achieved a varied compression ratio, with the best rate of 96.83% for the speed variable when = 27 and . Despite the higher compression rate of MSTAC for the speed variable, MPTAC showed a competitive compression rate in other variables, such as battery voltage and engine load. These results suggest that MPTAC is effective in maintaining a high compression ratio while preserving data integrity.
When comparing the MPTAC algorithm to MSTAC in terms of precision, as measured by RMSE (Question 2), it is evident that the performance varies with different parameter settings—
Table 5, the MPTAC and MSTAC algorithms show distinct performance characteristics in terms of RMSE. For battery voltage, MPTAC with parameters (0.2, 5) achieves an RMSE of 0.56, compared to MSTAC with parameters (0.1, 4), which yields an RMSE of 0.54, indicating slightly better precision for MSTAC. For engine load, MPTAC with parameters (0.2, 5) has an RMSE of 5.33, while MSTAC with parameters (0.1, 5) results in an RMSE of 5.43, showing MPTAC’s advantage in this case. In the RPM category, MPTAC (0.2, 5) has an RMSE of 295.73, whereas MSTAC (0.1, 5) yields 294.68, demonstrating a marginally better precision for MSTAC. For speed, MPTAC with parameters (0.2, 5) has an RMSE of 11.89, compared to MSTAC with parameters (0.2, 27), which has an RMSE of 13.60, indicating better performance by MPTAC. Finally, for throttle, MPTAC (0.2, 5) shows an RMSE of 5.52, whereas MSTAC (0.1, 3) has an RMSE of 5.00, highlighting MSTAC’s superior precision. These observations suggest that the choice between MPTAC and MSTAC may depend on the specific parameters used and the variable being considered, as each algorithm offers distinct advantages in different contexts.
When evaluated with the same parameterization (
),
Table 6, the MPTAC and MSTAC algorithms exhibit similar performance in terms of RMSE, with minor variations across specific variables. MSTAC demonstrated a slight advantage in precision for battery voltage (0.54 vs. 0.55), RPM (289.20 vs. 291.00), and speed (11.93 vs. 11.72), while MPTAC performed better for engine load (5.31 vs. 5.49) and throttle (5.23 vs. 5.56). These differences are subtle, indicating that both algorithms are comparably effective, with the choice between them depending on the relative importance of precision for specific variables within the application’s context.
Regarding the processing time comparison between MPTAC and MSTAC (Question 3), processing time is an important factor in selecting a compression algorithm, especially in applications where efficiency and speed are essential. In the case of the MPTAC and MSTAC algorithms, a significant difference in processing time was observed. MPTAC demonstrated noticeably faster performance, with an average processing time per sample of 0.1997 ms, which is approximately 4.17 times faster than MSTAC, which has an average time of 0.8326 ms per sample. The superior efficiency of MPTAC is especially advantageous in scenarios that require low-latency real-time processing, where the speed of data compression is as crucial as its efficiency. The advantage in processing time makes MPTAC a preferred choice for applications that require high speed, such as embedded systems in vehicles or IoT devices, where fast processing can directly impact overall performance and user experience. On the other hand, although MSTAC presents higher processing time, it can still be considered in situations where compression efficiency is prioritized over speed.
Complementary to these findings, the evaluation of the MPTAC and MSTAC algorithms revealed different trade-offs between compression and accuracy, aspects that are fundamental in applications with bandwidth and processing constraints. In situations where compression rate is a priority, such as remote sensor networks, a more aggressive compression is advantageous to reduce the volume of transmitted data. However, this choice can introduce small losses in accuracy, which in some contexts, such as environmental monitoring, is acceptable to preserve bandwidth and storage efficiency. On the other hand, in applications that require high accuracy, such as real-time anomaly detection in automotive systems, a more moderate compression rate is preferred, which ensures a more accurate reconstruction of the data and, therefore, greater reliability in the analysis.
In addition, the performance of the MPTAC and MSTAC algorithms strongly depends on the adequate calibration of parameters such as the size of the analysis window and the thresholds for compression and anomaly detection. During the study, a careful adjustment of these parameters was performed through empirical tests on the automotive dataset, allowing a balance to be reached between compression efficiency and reconstruction accuracy. This calibration process is also adaptable to other domains, such as health monitoring and smart grids, where specific data characteristics require adjustments to optimize algorithm performance. Thus, the choice between algorithms and their parameters depends both on the specific application requirements and on the conditions and limitations of the available data.
Finally, the main advantages and disadvantages of the MPTAC and MSTAC algorithms are summarized in
Table 7. This comparison highlights the key strengths and limitations of each approach, focusing on important factors such as processing time, compression ratio, and data integrity. By examining these aspects, the table provides a clear overview of the trade-offs between the two algorithms.
In summary, the choice between MPTAC and MSTAC depends on the specific application priorities. If processing speed is the primary concern, MPTAC is more advantageous. However, if the compression ratio is more critical, MSTAC may be preferable, especially for variables where it demonstrates a superior compression ratio. These findings extend beyond the automotive context; they can be applied to various IoT applications, such as wearable devices and smart cities, where efficiency in data transmission and storage is essential.
6.4. Limitations of MPTAC and MSTAC
While the MPTAC and MSTAC algorithms offer frameworks for multivariate time series compression, they also exhibit certain limitations.
Scalability concerns—As the number of variables increases, both MPTAC and MSTAC maintain a computational complexity of O(n) relative to the number of variables. However, the constant multiplicative factor for MSTAC is higher due to its sequential processing approach, which results in longer processing times. While MPTAC’s parallel handling of multidimensional data can lead to increased memory usage and processing time, its recursive and online processing nature allows it to maintain efficient performance even with large sample sizes. Significant performance constraints for MPTAC would likely only arise in extreme scenarios involving a high number of variables combined with very limited computational power. In contrast, the sequential processing of MSTAC may lead to greater inefficiencies in real-time applications as the number of variables grows.
Performance in highly dynamic environments—Both algorithms face challenges in highly dynamic environments where concept drift occurs. Although the anomaly window mechanism in MPTAC and MSTAC provides some level of adaptability, this mechanism may not be sufficient in rapidly changing data distributions. In these situations, reinitialization of the algorithms may be required to maintain accuracy. This could disrupt continuous monitoring and anomaly detection in scenarios where real-time response is critical, such as in industrial IoT or smart grid systems.
Parameter sensitivity—The effectiveness of both MPTAC and MSTAC rely heavily on careful tuning of hyperparameters, such as the comparison threshold and window size. Incorrect selection of these parameters can significantly impact performance, potentially resulting in a higher rate of false positives or missed anomalies. Scenarios that most challenge parameter tuning are those with highly erratic (highly dynamic) time series or, conversely, extremely smooth series, where inappropriate parameter choices can reduce detection accuracy. In diverse applications, these parameters may need continuous adjustment, which could limit the generalizability of the algorithms across varying use cases without prior fine-tuning.
Handling of strong correlations—In MSTAC, the approach of processing variables independently makes it less susceptible to misclassifying one variable due to “double weighting” from strongly correlated variables, as each variable is evaluated on its own. However, this independence assumption may lead to missed anomalies that arise from interdependencies among variables, limiting its effectiveness in capturing the full complexity of data in multivariate systems. MPTAC, by processing variables in parallel as a collective sample, can better handle interdependent variables but may be more sensitive to strongly correlated variables. In such cases, these correlations might carry greater influence when classifying a sample as an anomaly.
Handling outliers—Both MPTAC and MSTAC rely on outliers as indicators of context shifts, using them to identify and save relevant samples. The control of hyperparameters, such as m and the , helps ensure that only significant outliers are considered, avoiding the influence of less relevant anomalies. The eccentricity-based anomaly-detection mechanism in both algorithms further supports this by flagging outliers before compression, allowing for tailored handling. Given this, handling outliers is not a weakness of the algorithms; rather, it is an integral feature that enhances their accuracy in capturing meaningful data shifts and improves the quality of compressed data in dynamic environments.