Bayesian Model Selection for Addressing Cold-Start Problems in Partitioned Time Series Prediction

Abstract

How to effectively predict outcomes when initial time series data are limited remains unclear. This study investigated the efficiency of Bayesian model selection to address the lack of initial data for time series analysis, particularly in cold-start scenarios—a common challenge in predictive modeling. We utilized a comprehensive approach that juxtaposed observational data against various candidate models through strategic partitioning. This method contrasted traditional reliance on distance measures like the L₂ norm. Instead, it applied statistical tests to validate model efficacy. Notably, the introduction of an interactive visualization tool featuring a slide bar for setting significance levels marked a significant advancement over conventional p-value displays. Our results affirm that when observational data align with a candidate model, effective predictions are possible, albeit with necessary considerations of stationarity and potential structural breaks. These findings underscore the potential of Bayesian methods in predictive analytics, especially when initial data are scarce or incomplete. This research not only enhances our understanding of model selection dynamics but also sets the stage for future investigations into more refined predictive frameworks.

1. Introduction

How to effectively predict outcomes in numeric time series data when faced with a cold-start problem where little or no initial data are available remains unclear. The cold-start problem poses a significant challenge to automated data modeling for computer-based information systems, especially when user and item data are insufficient to make accurate predictions [1,2]. This issue is particularly notorious in recommendation systems that utilize information filtering techniques to tailor item displays to user preferences, typically referencing user profiles based on specific features [3]. However, the term “cold-start” becomes less straightforward when it is applied to numeric time series data [4]. In these cases, if the data-generating process (DGP) is accurately known, predictions can be reliably made, sometimes even without any observed data [5]. This capability underscores the importance of understanding and identifying the general DGP, as it allows for effective prediction even at the onset of data collection. Consequently, a substantial body of research has been dedicated to understanding the general DGP rather than directly addressing the cold-start issue in time series. This research is of paramount importance because it addresses the foundational challenge of making informed predictions in the absence of substantial initial data, a common scenario in many practical applications, including economics and environmental science.

The objective of this study was to address the issue of cold start in time series analysis, particularly in the context of specific conditions that might challenge conventional data modeling approaches. These specific conditions include the following:

• Insufficient length of observed time series: It is typically recommended that a sample size of at least 50 should be used for data analysis, though this is not a strict rule [6]. The necessary sample size can vary depending on data characteristics, domain, and analysis methods [7,8]. A cold-start problem exists if available data are insufficient and below the suggested minimum sample size. This scenario can greatly affect the reliability and accuracy of analyses performed.
• Incomplete cyclical or seasonal variation: To achieve accurate predictions, it is necessary to observe a complete cycle for time series exhibiting seasonal or cyclical variations. If available data only cover a partial cycle, as illustrated in Figure 1, it becomes challenging to identify repeating patterns and make accurate forecasts. This limitation can significantly hinder the effectiveness of predictive analytics in such cases.
• Post-structural break and insufficient data: The occurrence of structural breaks in a time series with significant shifts in data distribution further complicates the process of forecasting [9]. As illustrated in Figure 2, inadequate data collection following a structural break can hinder the ability to forecast future trends accurately. This limitation is critical as it affects the prediction of whether the observed anomaly will lead to further structural changes.
• Lack of data on specific features or items: The introduction of new features or items into a dataset can present forecasting challenges when these additions lack sufficient data to support them. To understand and predict the impact of these new elements, it is necessary to have adequate historical data. To make reliable predictions, it is necessary to have a robust dataset that captures the characteristics of new elements fully.

Figure 1. Cases with less than one full seasonal cycle. The letter “A” indicates an incomplete seasonal cycle within the known area, making it difficult to identify repeating patterns and accurately predict future values.

Figure 1. Cases with less than one full seasonal cycle. The letter “A” indicates an incomplete seasonal cycle within the known area, making it difficult to identify repeating patterns and accurately predict future values.

Figure 2. Insufficient data observation following a structural break. The arrows labeled “A” and “B” mark significant structural breaks in the time series, indicating sharp shifts in the distribution of the data. The area following the second structural break “B” shows insufficient data for accurate forecasting, making it difficult to predict future trends or detect further structural changes.

Figure 2. Insufficient data observation following a structural break. The arrows labeled “A” and “B” mark significant structural breaks in the time series, indicating sharp shifts in the distribution of the data. The area following the second structural break “B” shows insufficient data for accurate forecasting, making it difficult to predict future trends or detect further structural changes.

The partitioning of time series data has historically employed methods based on the autocorrelation function (ACF) of related time series as originally proposed by Chow and Lin [10] and Denton [11]. When reference time series are unavailable, interpolated methods have been utilized to segment time series data. This approach is particularly advantageous when the scope or temporal constraint of the research necessitates disaggregation or aggregation of time series data. Moreover, methods that employ the ACF have evolved into Box and Jenkins’ techniques and their derivatives [12]. As these techniques have been refined and adapted to the complexities of modern data analysis, they have proven to be crucial in addressing the nuanced effects of seasonal and trend variations in predictive accuracy. The impact of seasonal and trend variations in time series data on various predictions is significant. In light of this, the Box and Jenkins method has been extensively utilized. With the advancement of artificial neural networks (ANNs) in the 2000s, a combined approach with Box and Jenkins’ techniques has emerged [13]. However, these methods often fail to capture seasonal effects in nonstationary time series data effectively. To minimize variations, Puma-Villanueva et al. [14,15] have segmented time series data, while Sarkar et al. [16] and Leverger et al. [17] have proposed effective partitioning methods to enhance classification accuracy.

The cold-start problem has been a persistent issue in recommender systems, particularly within collaborative filtering technologies. Despite the development of numerous solutions, no perfect solution has been identified. However, the creation of an item–item matrix to determine correlations among items has emerged as a prevalent alternative. This matrix is then used to infer user preferences based on the most recent data. Xie et al. [18] have found that the cold-start problem in time series arises from issues such as missing data and high dimensions. They have attempted long-term forecasting by considering metadata, high-dimensional structures, and seasonality. Building on these insights, we will examine how these approaches are applied in practical scenarios, with a particular focus on challenges of data scarcity at the outset of analysis. Consider the scenario where we are provided with time series data as illustrated in Figure 3, which depicts an example of observed time series data. If the DGP for this series is known, one can directly utilize the DGP. Alternatively, if the DGP is unknown, it is possible to estimate the DGP from available data, enabling the kind of predictions shown in Figure 4, where the application of DGP on observed data is displayed. However, difficulties may arise in instances where a cold-start problem is present and where it is impossible to make predictions with limited data initially available.

Bayesian model selection is particularly well suited to address the cold-start problem in time series analysis because it allows for the integration of prior knowledge and statistical evidence, even when data are sparse or incomplete. This method facilitates the generation of informed decisions on the most likely models that could have generated the observed data. Before we explore the specific scenario involving Bayesian model selection, it is essential to address the following research questions that guide our investigation:

• How can we accurately predict time series outcomes when only limited data are initially available?
• Among various candidate models proposed, which is most likely to have generated the observed time series data?
• How effectively can Bayesian model selection overcome challenges presented by the cold-start problem in time series prediction?

These questions serve to frame our examination of model selection and the application of Bayesian principles to resolve uncertainties in the cold-start scenario. We propose that the time series could originate from one of several candidate models. This setup requires knowledge of these models and introduces a Bayesian model selection scenario, where we must decide which of the competing models is the most probable given the data. This decision process is demonstrated in Figure 3, which displays probabilities of the observed time series data belonging to different candidate models.

This study employed Bayesian model selection in cases where insufficient data prevented a comprehensive analysis of the observed time series. The aim of this study was to derive inferences under these constraints and present results in a visually comprehensible format, facilitating the interpretation of complex statistical decisions. The following contributions of this paper address critical gaps in time series analysis and provide practical tools for managing data scarcity and uncertainty in predictive modeling:

• This paper demonstrates how Bayesian model selection can enhance forecasting accuracy when traditional methods are inadequate due to a lack of data. A robust framework is presented to generate reliable forecasts from limited observations, ensuring that predictions remain viable even when datasets are sparse.
• The proposed innovative approach incorporates prior knowledge into model selection, which is of value when historical data are unavailable or do not reflect current circumstances. Integrating prior knowledge makes model predictions more reliable in challenging situations.
• The proposed visualization techniques can simplify complex Bayesian results, increasing stakeholder understanding and confidence in model predictions. This approach enables a deeper comprehension of statistical outcomes, improving the decision-making quality.

The remainder of this paper is structured as follows: Section 2 reviews related studies in the field. Section 3 provides requisite background information for this study. Section 4 offers a comprehensive account of conducted experiments. Section 5 discusses visualization techniques. Finally, Section 6 presents the conclusion.

2. Related Work

Related work on cold-start problems in recommendation systems and anomaly detection in time series data is categorized into classification and regression problems. This section reviews previous studies, highlighting differences between the present research and previous studies. Moreover, several studies have addressed classification problems in the context of cold-start and anomaly detection.

For example, Xu et al. [19] presented a variational embedding learning framework (VELF) to address the cold-start problem in predicting click-through rate. The VELF addressed data sparsity by learning probabilistic embeddings and applying trainable, regularized priors using information regarding users and advertisements. Experiments revealed that the VELF outperformed traditional methods and enhanced generalization and robustness in cold-start scenarios. In addition, Al Rossais et al. [20] proposed an approach to improve cold-start recommendations by generating item-based stereotypes from metadata without considering user-item ratings. Their experiments on MovieLens/IMDb (MovieLens, Minneapolis, MN, USA) and Amazon datasets (Amazon Inc., Seattle, WA, USA) found that these stereotypes enhanced both recommendation quality and computational performance and outperformed traditional singular value decomposition-based approaches. Pirasteh et al. [21] designed an enhanced hybrid collaborative filtering method to improve personalized recommendations by combining similarity measures. This method combines user and item similarities based on ratings and genres, addressing cold-start problems and outperforming conventional collaborative filtering techniques.

Further, Rohani et al. [22] introduced an enhanced content-based algorithm using social networking to address the cold-start problem in recommender systems by incorporating user preferences with those of friends and faculty. The efficacy of the enhanced content-based algorithm using social networking was evaluated on the MyExpert academic social network (University of Malaya, Kuala Lumpur, Malaysia), demonstrating significantly enhanced recommendation accuracy compared with random, collaborative, and content-based algorithms. Ni et al. [23] predicted student performance on learner-sourced questions by integrating signed graph neural networks with large language model embeddings. Their method modeled student responses using a signed bipartite graph and employed a contrastive learning framework to enhance noise resilience, significantly outperforming existing baselines in predictive accuracy and robustness. In addition, Tey et al. [24] designed a social network-based recommender system to address the cold-start problem by leveraging indirect relationships between users and their friends’ friends. The system integrated user preferences and social media interactions. It significantly improved recommendation accuracy using data from Yelp (Yelp Inc., San Francisco, CA, USA).

Kuznetsov and Kordík [25] addressed the cold-start problem in recommendation systems with ontologies and knowledge graphs to enhance text-based methods. Their approach used ontologies to generate a knowledge graph capturing implicit and explicit characteristics of item–text attributes, enriching item profiles with semantically similar keywords. Their experimental evaluations demonstrated the efficacy of this method compared with state-of-the-art text feature extraction techniques. Recently, Li et al. [26] proposed a novel reinforcement learning approach for time series anomaly detection incorporating human feedback. This approach applied an ensemble of unsupervised anomaly scoring and devised reward strategies to guide the learning process, significantly outperforming five state-of-the-art models on the F1 score and the precision–recall metric of the area under the curve.

Moreover, numerous studies have examined regression problems regarding cold-start forecasting and anomaly detection. For example, Fatemi et al. [27] presented the cold causal demand forecasting framework, combining causal inference with deep learning models to enhance multivariate time series forecasting (TSF) in the context of the cold-start problem. Their study applied several critical techniques, including graph neural networks for representation learning based on causal influence, long short-term memory networks for capturing historical data, and similarity-based approaches using Gaussian mixture models and the extended Frobenius norm to leverage data from similar data centers. Xie et al. [28] designed a unified framework for long-range and cold-start forecasting of seasonal profiles in a time series. The framework combined high-dimensional regression and matrix factorization to address forecasting challenges posed by limited historical data, showing robust performance across multiple datasets.

Additionally, Ryu et al. [29] addressed the problem of cold start in web-service quality of service predictions using location-based matrix factorization with preference propagation, combining invocation similarity and neighborhood similarity, and applying location data for users and services. The method outperformed existing methods for cold-start and warm-start scenarios in their experiments. Xie et al. [18] introduced a unified framework to address long-range forecasts, missing data, and cold-start problems in time series data. The framework applied repeated patterns over fixed periods and employed metadata using low-rank decompositions, yielding accurate predictions and imputing missing values. Chen et al. [30] presented FrAug, a novel frequency domain data augmentation technique for improving TSF. The FrAug technique includes frequency masking and frequency mixing methods. It significantly enhanced forecasting accuracy and mitigated performance degradation under distribution shifts, making it effective for cold-start forecasting.

Key aspects of the proposed approach compared with referenced studies in related work are outlined below to clarify differences between them (see

Table 1. Comparative overview of research techniques and outcomes.

Authors	Key Techniques	Evaluation	Difference from This Research
Fatemi et al. [19]	Integrates causal inference with deep learning, GNNs, LSTM, GMM, Eros	Outperformed traditional methods on cold-start scenarios	Focuses on deep learning and causal inference, not Bayesian model selection
Xu et al. [20]	Probabilistic embeddings, variational inference, regularized priors	Significant improvements in cold-start scenarios	Uses variational inference rather than Bayesian methods for a time series
AlRossais et al. [21]	Item-based stereotypes, metadata independent of user item ratings	Superior to traditional SVD-based approaches	Targets recommendation systems, not TSF
Pirasteh et al. [22]	Combines multiple similarity measures, integrates user and item similarities	Outperformed conventional CF techniques in cold-start conditions	Focuses on collaborative filtering, not Bayesian methods or time series data
Rohani et al. [23]	User preferences, hierarchical preference tree structure	Significantly improved recommendation accuracy	Employs social networking for recommendations, not applicable to TSF
Ni et al. [24]	SGNNs, LLM embeddings, contrastive learning framework	Outperformed existing baselines	Focuses on student performance prediction, different domains, and methods
Tey et al. [25]	Indirect relations, user preferences, social media interactions	Significant improvements in recommendation accuracy	Applies social network data, unrelated to TSF
Kuznetsov and Kordík [26]	Ontologies, knowledge graphs, semantic layer in text-based methods	Effective compared with state-of-the-art text feature extraction techniques	Uses knowledge graphs and ontologies, unlike Bayesian inference
Li et al. [27]	Reinforced active learning, human-in-the-loop, anomaly scoring	Outperformed five state-of-the-art models	Focuses on anomaly detection, not general TSF
Xie et al. [28]	High-dimensional regression, matrix factorization, leveraging metadata	Robust performance across multiple datasets	Employs regression and matrix factorization, not Bayesian model selection
Ryu et al. [29]	Invocation similarity, neighborhood similarity, location data	Better performance in cold- and warm-start scenarios	Uses matrix factorization for web services, different applications and methods
Xie et al. [18]	Repeated patterns, low-rank decompositions, metadata weightings	Accurate predictions and imputes missing values	Focuses on missing data and long-range forecasts, different approaches
Chen et al. [31]	Frequency domain data augmentation, frequency masking, frequency mixing	Enhanced forecasting accuracy, mitigated performance degradation	Uses data augmentation techniques, not Bayesian methods
Ebrahimi et al. [32]	Application-based, checkpoint-based, invocation time prediction-based, cache-based	Discussed various methods and evaluated their effectiveness	Focuses on serverless computing, not TSF

Notes: GNNs, graph neural networks; LSTM, long short-term memory; GMM, Gaussian mixture models; Eros, extended Frobenius norm; SGNNs, signed graph neural networks; LLM, large language model; CF, collaborative filtering; SVD, singular value decomposition; TSF, time series forecasting.

):

• This study is focused on TSF, whereas most of the related studies have concentrated on recommendation systems or anomaly detection. The proposed approach addresses challenges of insufficient data by partitioning time series data and applying Bayesian inference to select the most probable model. R version 4.3.3 (R Foundation for Statistical Computing, Vienna, Austria) and RStudio version 2023.12.1.402 (Posit Software, PBC, Boston, MA, USA) were used as computational tools to ensure accurate and reproducible results. This approach contrasts with those of other studies, which often rely on augmenting the data with additional information or using hybrid models.
• The proposed approach is tailored to numeric time series data, whereas related studies often involve categorical data in recommendation systems or mixed data types in anomaly detection. In addition, the proposed approach differs from previous studies in that Bayesian model selection is employed, which is distinct from typical methods used in related work, such as collaborative filtering, matrix factorization, and deep learning techniques.

3. Bayesian Time Series Analysis

Bayesian time series analysis provides a robust framework that integrates prior knowledge with observed data, thereby facilitating more reliable model selection and inference, particularly in situations where the precise DGP is uncertain. This method offers flexibility by incorporating uncertainty and past information, making it a valuable tool in a diverse range of time series applications.

3.1. Bayesian Model Selection

A general DGP is assumed as follows:

$$z_t=f\left(z_{t-1}\right)+{\epsilon }_t,\mathrm{where}{\epsilon }_t\sim N({\hat{\mu }}_{\epsilon },{{\hat{\sigma }}_{\epsilon }}^2).$$

(1)

In most situations, when data are provided, the exact DGP is unknown. The DGP (ẑ_t) can only be estimated using sample data derived from some DGP. The probability p[X|ẑ_t] that X occurs can be calculated from the estimated DGP (ẑ_t). However, in a cold-start problem, estimating this DGP or estimating p[X|z_t] is infeasible.

Instead, if M DGPs could be tentatively considered ground truths, these might be represented by already-known equations or datasets. These M DGPs are called candidate models, and the mth candidate model is labeled Model_m. Under this setup, the following equation for calculating p[Model_m|X] (known as the posterior probability) indicates the probability that given data X are generated from Model_m:

p[Model_m|X] = p[Model_m, X]/p[X] =
(p[X|Model_m] ⋅ p[Model_m])/Σ (from i = 1 to M) (p[X|Model_i] ⋅ p[Model_i]),

(2)

where p[Model_m] (with m = 1, 2, …, M) represents the prior probability of each model, assigned uniformly assuming that no prior knowledge favors one model over the others.

In this study, Bayesian model selection is of paramount importance, as it integrates prior information (e.g., the uniform prior p[Model_m]) with observed data to calculate the posterior probability p[Model_m|X]. This approach allows for the selection of the most appropriate model among multiple candidates based on both prior beliefs and the likelihood of the observed data fitting each model. The posterior probability is calculated using Bayes’ theorem, whereby the initial model probabilities are updated based on how well each model explains the observed data.

The prior probability, p[Model_m], represents the initial assumption about the likelihood of each model before any data are considered. In the absence of prior knowledge that favors one model over another, a uniform prior is employed, giving each model equal weight initially. This is particularly advantageous in cold-start problems, where there is limited prior information available, as it allows for a more objective evaluation of the models.

The probability p[Model_m|X] indicates the likelihood of the observed data X occurring under each candidate model. The likelihood is calculated using statistical tests and distance metrics, such as the L₂ norm, Pearson distance, or Wasserstein distance, depending on the characteristics of the dataset. These distance measures assist in quantifying the fit between the observed data and the predictions made by each model.

To further elucidate the selection of the most appropriate model, our criterion is primarily based on the analysis of results from various distance measurements under varying levels of statistical significance. This methodology entails a comparison of the capacity of each candidate model to reproduce the observed data, employing specific statistical tests and distance metrics, such as the L₂ norm and others tailored to the characteristics of the dataset.

The contribution of each distance measurement is distinct, and significance thresholds are established to ascertain the probability that a given model represents the generative process responsible for the observed data. This approach ensures that the selected model not only exhibits a high degree of fit with the data but also adheres to predetermined significance levels, reflecting a robust integration of prior knowledge and statistical evidence.

In practice, Bayesian model selection is performed by calculating the posterior probability for each candidate model. The posterior probability reflects the probability that the observed data X were generated by a particular model, taking into account both the prior probability of the model and the likelihood of the data given the model. The model with the highest posterior probability is then selected as the optimal representation of the DGP. This process ensures thorough evaluation of models based on their ability to explain the observed data while integrating prior knowledge through the Bayesian framework.

3.2. Partitioned Time Series

As noted in Section 1, making predictions by predicting them is only feasible when the DGP can be easily estimated. However, candidate models often have missing data or structural changes in characteristics of influencing features, among other complexities, making it challenging to estimate the DGP. Although it would be ideal if the DGP could be assumed to remain constant, structural changes in a time series can occur at any time, requiring preparation for these changes.

When these changes occur, the “belief” regarding the model to which Y belongs must be updated. Re-estimating the DGP each time is highly inefficient. Therefore, the following approach is considered to estimate the marginal likelihood:

• Observed data are denoted by X = {x₁, x₂, …, x_u};
• Given M candidate models, each is denoted by Model_m = Y_m = {y_m,1, y_m,2, …, y_m,nm}, where m = 1, 2, …, M, and nm is the length of Model_m.

Figure 5 illustrates the partitioning of data from each model. If the length of each partitioned data segment is u, equal to the length of X, then the maximum number of partitioned data vectors p_m obtained from Model_m is n_m − u (q_m ≤ n_m − u). Partitioned data vectors are p_m = {p_m,1, p_m,2, …, p_{m,q_m}}. If Model_m is a model for missing data, vectors shorter than u that occur immediately before or after the missing data are disregarded. If y_m,1, …, y_m,u are not missing data, then p_m,1 = {y_m,1, …, y_m,u}. The relationship between each partitioned data vector p_m,j (j = 1, …, q_m) and X, denoted as r(p_m,j, X), is then calculated.

Advantages of this approach include the following:

• When new data arrive, all existing data are not needed, reducing the burden even when data continue to stream in real time;
• This approach is advantageous for addressing structural breaks in model data. It decreases the need for complex analyses considering nonstationarity, reducing the burden of addressing long-term time series;
• This approach simplifies handling missing data. Although this study employs a method that entirely excludes missing data, imputation is possible when necessary. Splitting data into shorter segments can offset irregularities, facilitating imputation;
• Distributed computing is feasible. The process of determining the relationship between each partitioned data point p_m and the observed data point X, denoted as r(p_m,j, X), can be managed in parallel.

3.3. Analysis of Similarity between Observed and Partitioned Data

Section 3.2 discusses how to partition the data for each model. Partitioned data are compared with observed data. Sections below explore methods to make this comparison.

3.3.1. Measuring the Distance between Observed and Partitioned Data

Figure 6 presents how to determine differences between observed and partitioned data. However, using a random distance measure to consider the relationship between the two time series must be avoided. For example, if observed data X = {0.1, 0.4, 0.7, 1.0} and partitioned data from Model_m and Model_o are p_{m,q_i} = {0.0, 0.4, 0.8, 1.2} and p_{o,q_j} = {1.3, 1.6, 1.9, 2.1}, respectively, increments of X and p_{o,q_j} are consistent at 0.3, whereas those of p_{m,q_i} are at 0.4. This result could imply that X and p_{o,q_j} are generated from the same DGP.

However, when relationships were calculated using the L₂ norm, values obtained with p_{m,q_i} were lower, suggesting that they were more similar in distance to the observed data X, although they originated from different generative processes. This outcome illustrates a limitation of using straightforward distance measures such as the L₂ norm to determine data origins. Thus, “similar values” and “values generated from the same process” must be identified. Using the L₂ norm might imply the former, potentially missing a time series generated using different processes.

In contrast, if a particular data-generating function is known and used to generate data, various forms of a time series can be obtained from a single DGP, as depicted in Figure 7. In a statistical approach, current data can be considered a sample that accurately reflects characteristics of a specific population. Therefore, a direct comparison of distances might be inappropriate for verifying a time series generated under the same DGP. The L₂ norm values can vary significantly depending on how the sampling was conducted.

3.3.2. Statistical Testing between Observed and Partitioned Data

Upon obtaining several partitioned data vectors q_m from a model, their distribution can be assessed, and a joint distribution of these partitions can be considered (Figure 8). Each partitioned data vector only reflects characteristics of that segment, not necessarily the entire model. When partition sizes are small, this limitation can become significant. Similarly, initially observed data X may not fully reflect overall characteristics of the model, which is a fundamental reason for the failure to estimate the DGP directly from X alone.

When comparing distributions of observed data f(X) with those of q_m partitioned data vectors g(p_m,j), some might be similar while others might be different. If many g(p_m,j) followed a specific distribution, with f(X) also following such a distribution, observed data X probably originated from this model. This scenario can be represented using the marginal likelihood p[Y|Model_i] for determining the probability that these data came from the ith candidate model. This approach does not assume structural changes. The high marginal likelihood indicates that many sections of the distribution are elevated.

Furthermore, using distribution testing (Figure 9) does not allow the discernment of trend differences. If distance measures are employed for comparison, observing how distances evolve over time can reveal trend differences. However, when employing distribution comparison tests, trend tests must be separately conducted. Nonetheless, if time series data are generated from a specific DGP, statistical independence tests can be used for analyzing data distribution before estimating the DGP. This method also accounts for distances between distributions. However, data samples are from a joint distribution. Hence, statistical testing is appropriate.

3.4. Analytical Procedures

Observed data X and M candidate models are assumed to be known beforehand with a cold-start problem. The following steps outline the proposed approach:

• Data for each candidate model are partitioned. This study followed the method described in Section 3.2 to partition data from each candidate model (designated as p_m). If M candidate models exist, M partitioned datasets p_m are also generated. If the length of observed data is u, each partitioned dataset is also set to length u. If data are standardized, standardization is applied separately to each p_m;
• Users must determine whether to use a distance measure or a statistical test to calculate r(p_m,j, X). If distance is chosen, r(p_m,j, X) is determined by d(p_m,j, X), measuring direct discrepancies between datasets. If a statistical test is selected, r(p_m,j, X) corresponds to a p-value that evaluates the independence between p_m,j and X, indicating the likelihood of a statistical relationship;
• When a statistical test is employed to determine r(p_m,j, X), trend differences between p_m,j and X are not assessable. Therefore, test results for the trend must also be considered. Trend tests for p_m,j and X are conducted to obtain p-values. Closer trends of p_m,j and X indicate a smaller difference between p-values from trend tests of p_m,j and X, which results in a value that approaches zero;
• When calculating marginal likelihood, because q_m instances of p_m,j exist, q_m instances of r(p_m,j, X) are also attained. By setting a significance level for r(p_m,j, X), the probability p[X|Model_m] can be calculated, indicating the likelihood of X belonging to Model_m. Probability calculations regarding significance levels are detailed in Equations (3) and (4):

  p[X∣Model_m] = p[r(p_m,j, X) ≤ SignIf._indep.],

  (3)

  p[X∣Model_m] = p[(p-value_indep.(p_m,j, X) ≤ SignIf._indep.)/(p-value_trend.(p_m,j) ≤ SignIf._trend)];

  (4)
• Posterior probability is calculated using the method described in Equation (2). The prior probability is uniformly assigned across all M candidate models. Comparing posterior probabilities across all M candidate models could identify the model that most likely generated observed data X, enhancing our understanding of the reliability of the model.

The process of evaluating which candidate model best explains observed data is streamlined by structuring the analysis in this way, considering statistical relationships and trends between datasets. This approach guarantees robust decision-making by applying a comprehensive statistical evidence base.

4. Results

4.1. Statistical Testing or Distance

In this experiment, methods employed to calculate r(p_m,j, X) include statistical tests, specifically the Kolmogorov–Smirnov test and runs test for two samples. The Cox–Stuart trend test was applied for trend testing. Although these tests employ historical data regarding distribution and trends, the Bayesian model utilized in this study also incorporates prior knowledge, thereby enhancing the analytical process and providing a more robust foundation for the findings.

Table 2. Distance measures explored in this experiment.

Name	r(pm,j, X) Formula	Family
Squared Euclidean	∑[X − pm,j]2	Squared L2 family (χ2 family)
Pearson 1	∑[(X − pm,j)2/pm,j]	Squared L2 family (χ2 family)
Neyman	∑[(X − pm,j)2/X]	Squared L2 family (χ2 family)
Squared chi	∑[(X − pm,j)2/(X + pm,j)]	Squared L2 family (χ2 family)
Prob. symmetric	2 × ∑[(X − pm,j)2/(X + pm,j)]	Squared L2 family (χ2 family)
Divergence	2 × ∑[(X − pm,j)2/(X + pm,j)2]	Squared L2 family (χ2 family)
Clark	√(∑[|X − pm,j|/(X + pm,j)2])	Squared L2 family (χ2 family)
L2 norm	(|X − pm,j|)1/2	Lp Minkowski family
Manhattan	∑|X − pm,j|	Lp Minkowski family
Chebyshev	max|X − pm,j|	Lp Minkowski family
Sorensen	(∑|X − pm,j|)/(∑(X − pm,j))	L1 family
Gower	1/d × ∑|X − pm,j|	L1 family
Kulczynski’s D	(∑|X − pm,j|)/(∑[max(X, pm,j)])	L1 family
Canberra	(∑|X − pm,j|)/(∑[min(X, pm,j)])	L1 family
Lorentzian	∑[log(1 + |X − pm,j|)]	L1 family
Intersection	∑[min(X, pm,j)]	Intersection family
Nonintersection	1 − ∑[min(X, pm,j)]	Intersection family
Wage hedges	(∑|X − pm,j|)/max(X, pm,j)	Intersection family
Czeanowski	(∑|X − pm,j|)/(∑|X + pm,j|)	Intersection family
Motyka	(∑[min|X − pm,j|])/(∑|X + pm,j|)	Intersection family
Inner product	∑(X × pm,j)	Inner product family
Harmonic mean	2 × ∑[(X × pm,j)/(X + pm,j)]	Inner product family
Cosine	(∑[X × pm,j])/√(∑X2) × √(∑(pm,j)2)	Inner product family
Kumar–Hassebrook	(∑[X × pm,j])/(∑X2 + ∑(pm,j)2 − ∑(X × pm,j))	Inner product family
Dice	(∑[X − pm,j]2)/(∑X2 + ∑(pm,j)2)	Etc.
Wasserstein	infγ∈(P_r,P_g)Ex,y∼γ[‖X − pm,j‖]	Etc.

¹ Pearson distance, as delineated in this text, differs from the Pearson correlation coefficient, which is more common.

lists the distances considered for calculating r(p_m,j, X). The prior information employed in the Bayesian framework is derived from assumptions based on past observations and pertinent domain knowledge, which facilitates the model’s inference process. This prior knowledge was selected based on the distinctive characteristics of the synthetic data presented in

Table 3. Parameters of synthetic candidate models.

Category	Model Label	Mean μ	Standard Deviation σ	Autoregressive Coefficient α
Stationary Time Series	A	0.0	0.2	0.5
	B	0.0	0.4	0.5
	C	0.0	0.6	0.5
	D	0.0	0.8	0.5
	E	0.0	1.0	0.5
Unstable Variance	A	0.0	1.2	0.5
	B	0.0	1.4	0.5
	C	0.0	1.6	0.5
	D	0.0	1.8	0.5
	E	0.0	2.0	0.5
Trend Changes	A	0.5	1.0	1.0
	B	1.0	1.0	1.0
	C	2.0	1.0	1.0
	D	−0.5	1.0	1.0
	E	−1.0	1.0	1.0
	F	−2.0	1.0	1.0
Presence of a Unit Root	A	0.0	1.0	0.2
	B	0.0	1.0	0.4
	C	0.0	1.0	0.6
	D	0.0	1.0	0.8
	E	0.0	1.0	1.0

, including stability, variance, and trends. Its suitability in the context of the time series behaviors examined in the study was then empirically validated.

The reason for extensively examining distances, more so than statistical tests, was that the methods for measuring distances between two time series varied significantly more than those for the statistical testing of their distributions. However, the primary motivation was to verify the caution advised in Section 3.3.1 empirically against randomly using distance measures when comparing two time series. The incorporation of prior knowledge into the Bayesian model facilitated a more structured approach to distance measure selection, thereby reducing the probability of random or inappropriate choices. This contributed to the development of a more robust model, as evidenced by the results.

The synthetic models presented in Table 3 provide a basis for evaluating the distance measures in question. Table 3 outlines four categories of synthetic data generated to represent diverse time series behaviors, including stationary time series, unstable variance, trend changes, and the presence of a unit root. The aforementioned synthetic models permit an evaluation of the efficacy of distance functions and statistical tests in identifying the true model under a variety of conditions. Prior knowledge was incorporated into the Bayesian model in a category-specific manner, which helped to improve model accuracy by providing additional context about the expected behavior of the data.

By establishing a connection between the synthetic models presented in Table 3 and the distance functions outlined in Table 2, we ensure a comprehensive examination of the performance of each method across diverse types of time series data. This methodology allows us to elucidate the strengths and limitations of distance measures and statistical tests in various contexts, which are further elaborated in Section 4.2. Furthermore, the incorporation of prior information through the Bayesian framework not only enhanced the interpretability of the results but also demonstrated the potential advantages of integrating prior knowledge into time series analysis.

4.2. Synthetic Model

For objective experimentation, the following synthetic autoregressive model of length 1000 was considered:

y_m,t = α × y_t−1 + t,

(5)

where y_m,1 = 0, t∼N(μ, σ²), and t = 1, …, 1000 (where m was the index of the candidate model, and α, μ, and σ denoted parameters that could be adjusted to control the stationarity of the synthetic model).

• Stationary time series: The model is considered stable when σ² is less than 1. Although diversifying a stable model is challenging, variations can be introduced in the range, where 0 < σ² < 1, μ = 0, and 0 < α <1;
• Unstable variance: The σ value should be set to ≥1 because a larger σ² yields more dynamic movement in the data;
• Trend changes: The μ value can be applied to determine a stochastic trend. No trend exists when μ = 0. If μ > 0, an upward trend exists. If μ < 0, a downward trend exists. A larger absolute value of μ indicates a steeper trend slope;
• Presence of a unit root: The α term is the coefficient. For a nonstationary model, α should be set to ≥1. If α = 1, a unit root is present.

As presented in Table 3, four types of synthetic data with set parameters were created to analyze the effects of varying these parameters. Observed data X were matched with the type for each model.

The significance level for r(p_m,j, X) was determined using quantiles. However, users can make interactive selections. Settings were adjusted to 0.1, 0.3, 0.5, 0.7, and 0.9 to observe changes. If the method for calculating r(p_m,j, X) operates correctly, whether using distance or statistical testing, a lower significance level is expected to increase the likelihood of selecting the correct model.

Table 4. Posterior probabilities in experiments involving stationary time series.

Distance Measure	SignIf.	0.1	0.3	0.5	0.7	0.9	Distance Measure	SignIf.	0.1	0.3	0.5	0.7	0.9
Squared Euclidean	A	0.025	0.058	0.096	0.135	0.176	Lorentzian	A	0.021	0.056	0.090	0.131	0.175
	B	0.022	0.059	0.103	0.142	0.180		B	0.022	0.059	0.103	0.144	0.180
	C	0.018	0.067	0.103	0.141	0.179		C	0.019	0.067	0.108	0.147	0.183
	D	0.015	0.056	0.102	0.142	0.183		D	0.019	0.058	0.101	0.141	0.182
	E	0.021	0.059	0.095	0.141	0.182		E	0.020	0.059	0.097	0.137	0.180
Pearson	A	0.019	0.060	0.095	0.140	0.180	Intersection	A	0.025	0.069	0.107	0.142	0.181
	B	0.020	0.062	0.106	0.145	0.182		B	0.021	0.058	0.100	0.140	0.176
	C	0.018	0.050	0.088	0.131	0.180		C	0.017	0.055	0.091	0.136	0.182
	D	0.021	0.066	0.108	0.141	0.180		D	0.019	0.058	0.098	0.143	0.182
	E	0.022	0.062	0.103	0.143	0.179		E	0.018	0.061	0.104	0.140	0.180
Neyman	A	0.019	0.061	0.099	0.138	0.180	Nonintersection	A	0.019	0.058	0.093	0.131	0.175
	B	0.022	0.060	0.096	0.138	0.178		B	0.024	0.060	0.100	0.142	0.179
	C	0.020	0.062	0.104	0.143	0.180		C	0.018	0.064	0.109	0.145	0.183
	D	0.022	0.058	0.101	0.143	0.179		D	0.018	0.057	0.102	0.142	0.181
	E	0.018	0.059	0.099	0.139	0.184		E	0.020	0.060	0.096	0.139	0.182
Squared chi	A	0.020	0.057	0.097	0.136	0.180	Wave hedges	A	0.021	0.063	0.100	0.136	0.179
	B	0.018	0.059	0.101	0.141	0.178		B	0.019	0.059	0.099	0.141	0.181
	C	0.020	0.061	0.101	0.143	0.182		C	0.017	0.050	0.090	0.135	0.180
	D	0.019	0.060	0.099	0.140	0.179		D	0.021	0.063	0.107	0.145	0.180
	E	0.023	0.063	0.102	0.141	0.182		E	0.022	0.065	0.103	0.143	0.180
Prob. symmetric	A	0.020	0.057	0.097	0.136	0.180	Czekanowski	A	0.022	0.057	0.100	0.144	0.181
	B	0.018	0.059	0.101	0.141	0.178		B	0.018	0.061	0.103	0.143	0.182
	C	0.020	0.061	0.101	0.143	0.182		C	0.018	0.056	0.101	0.139	0.180
	D	0.019	0.060	0.099	0.140	0.179		D	0.024	0.065	0.099	0.136	0.180
	E	0.023	0.063	0.102	0.141	0.182		E	0.018	0.061	0.097	0.138	0.177
Divergence	A	0.018	0.057	0.101	0.140	0.180	Motyka	A	0.022	0.057	0.100	0.144	0.181
	B	0.020	0.061	0.100	0.140	0.181		B	0.018	0.061	0.103	0.143	0.182
	C	0.019	0.062	0.100	0.143	0.180		C	0.018	0.056	0.101	0.139	0.180
	D	0.022	0.063	0.101	0.138	0.182		D	0.024	0.065	0.099	0.136	0.180
	E	0.020	0.056	0.098	0.139	0.177		E	0.018	0.061	0.097	0.138	0.177
Clark	A	0.018	0.057	0.101	0.140	0.180	Inner product	A	0.023	0.065	0.103	0.141	0.176
	B	0.020	0.061	0.100	0.140	0.181		B	0.021	0.060	0.099	0.141	0.177
	C	0.019	0.062	0.100	0.143	0.180		C	0.022	0.060	0.095	0.133	0.182
	D	0.022	0.063	0.101	0.138	0.182		D	0.017	0.056	0.098	0.143	0.185
	E	0.020	0.056	0.098	0.139	0.177		E	0.018	0.060	0.105	0.142	0.179
L2 norm	A	0.025	0.058	0.096	0.135	0.176	Harmonic mean	A	0.020	0.064	0.103	0.143	0.180
	B	0.022	0.059	0.103	0.142	0.180		B	0.022	0.059	0.099	0.141	0.182
	C	0.018	0.067	0.103	0.141	0.179		C	0.018	0.057	0.099	0.140	0.180
	D	0.015	0.056	0.102	0.142	0.183		D	0.021	0.060	0.101	0.140	0.181
	E	0.021	0.059	0.095	0.141	0.182		E	0.018	0.059	0.097	0.136	0.177
Manhattan	A	0.022	0.057	0.092	0.131	0.176	Cosine	A	0.024	0.065	0.103	0.141	0.176
	B	0.022	0.056	0.104	0.143	0.180		B	0.020	0.060	0.099	0.141	0.178
	C	0.018	0.067	0.109	0.143	0.181		C	0.022	0.059	0.095	0.133	0.182
	D	0.017	0.059	0.100	0.143	0.181		D	0.017	0.056	0.098	0.143	0.185
	E	0.021	0.060	0.095	0.140	0.181		E	0.017	0.060	0.105	0.142	0.180
Chebyshev	A	0.028	0.071	0.114	0.153	0.188	Kumar–Hassebrook	A	0.024	0.065	0.103	0.141	0.176
	B	0.016	0.059	0.098	0.137	0.180		B	0.020	0.060	0.099	0.141	0.178
	C	0.016	0.054	0.093	0.134	0.176		C	0.022	0.059	0.095	0.133	0.182
	D	0.020	0.057	0.100	0.140	0.180		D	0.017	0.056	0.098	0.143	0.185
	E	0.022	0.059	0.096	0.136	0.177		E	0.017	0.060	0.105	0.142	0.180
Sorensen	A	0.022	0.057	0.100	0.144	0.181	Dice	A	0.024	0.059	0.097	0.135	0.176
	B	0.018	0.061	0.103	0.143	0.182		B	0.022	0.059	0.101	0.140	0.180
	C	0.018	0.056	0.101	0.139	0.180		C	0.018	0.067	0.105	0.141	0.178
	D	0.024	0.065	0.099	0.136	0.180		D	0.015	0.057	0.102	0.144	0.183
	E	0.018	0.061	0.097	0.138	0.177		E	0.020	0.058	0.095	0.140	0.183
Gower	A	0.022	0.057	0.092	0.131	0.176	Wasserstein	A	0.048	0.104	0.148	0.175	0.194
	B	0.022	0.056	0.104	0.143	0.180		B	0.023	0.073	0.109	0.146	0.182
	C	0.018	0.067	0.109	0.143	0.181		C	0.016	0.049	0.086	0.127	0.159
	D	0.017	0.059	0.100	0.143	0.181		D	0.012	0.037	0.069	0.114	0.184
	E	0.021	0.060	0.095	0.140	0.181		E	0.002	0.038	0.087	0.138	0.180
Kulczynski‘s D	A	0.012	0.044	0.109	0.162	0.184	Kolmogorov–Smirnov	A	0.038	0.078	0.128	0.162	0.181
	B	0.051	0.121	0.146	0.173	0.195		B	0.029	0.078	0.107	0.141	0.155
	C	0.028	0.069	0.103	0.135	0.191		C	0.004	0.034	0.070	0.108	0.156
	D	0.002	0.041	0.094	0.141	0.182		D	0.008	0.034	0.055	0.095	0.171
	E	0.006	0.024	0.048	0.089	0.149		E	0.016	0.053	0.109	0.148	0.178
Canberra	A	0.022	0.059	0.099	0.141	0.180	Runs	A	0.012	0.052	0.083	0.125	0.161
	B	0.018	0.059	0.099	0.139	0.178		B	0.009	0.048	0.084	0.138	0.172
	C	0.018	0.061	0.100	0.141	0.180		C	0.012	0.053	0.088	0.131	0.171
	D	0.022	0.059	0.099	0.139	0.180		D	0.010	0.052	0.092	0.134	0.171
	E	0.021	0.061	0.103	0.140	0.182		E	0.010	0.053	0.086	0.128	0.167

,

Table 5. Posterior probabilities in experiments with unstable variance.

Distance Measure	SignIf.	0.1	0.3	0.5	0.7	0.9	Distance Measure	SignIf.	0.1	0.3	0.5	0.7	0.9
Squared Euclidean	A	0.017	0.059	0.099	0.140	0.179	Lorentzian	A	0.019	0.056	0.095	0.138	0.177
	B	0.020	0.062	0.102	0.140	0.181		B	0.021	0.064	0.102	0.140	0.181
	C	0.022	0.062	0.100	0.139	0.181		C	0.021	0.061	0.106	0.145	0.184
	D	0.021	0.060	0.098	0.141	0.178		D	0.020	0.059	0.098	0.137	0.178
	E	0.020	0.057	0.101	0.139	0.181		E	0.019	0.061	0.099	0.141	0.180
Pearson	A	0.019	0.058	0.101	0.139	0.179	Intersection	A	0.022	0.061	0.104	0.142	0.180
	B	0.017	0.052	0.097	0.144	0.182		B	0.020	0.061	0.101	0.139	0.180
	C	0.021	0.062	0.097	0.136	0.182		C	0.017	0.058	0.097	0.137	0.180
	D	0.024	0.065	0.104	0.140	0.179		D	0.022	0.063	0.100	0.140	0.180
	E	0.019	0.063	0.101	0.141	0.178		E	0.020	0.058	0.098	0.142	0.180
Neyman	A	0.022	0.062	0.100	0.142	0.182	Nonintersection	A	0.020	0.058	0.096	0.139	0.178
	B	0.020	0.060	0.099	0.142	0.180		B	0.020	0.061	0.099	0.139	0.180
	C	0.021	0.057	0.099	0.137	0.180		C	0.020	0.063	0.103	0.142	0.183
	D	0.017	0.064	0.102	0.141	0.177		D	0.020	0.060	0.100	0.137	0.178
	E	0.020	0.057	0.099	0.139	0.181		E	0.020	0.058	0.102	0.142	0.180
Squared chi	A	0.019	0.060	0.100	0.141	0.181	Wave hedges	A	0.021	0.058	0.099	0.141	0.180
	B	0.020	0.061	0.102	0.141	0.180		B	0.020	0.060	0.095	0.141	0.181
	C	0.013	0.055	0.099	0.137	0.179		C	0.019	0.059	0.101	0.137	0.181
	D	0.027	0.066	0.103	0.139	0.179		D	0.020	0.063	0.104	0.142	0.178
	E	0.020	0.059	0.097	0.143	0.181		E	0.020	0.061	0.100	0.139	0.180
Prob. symmetric	A	0.019	0.060	0.100	0.141	0.181	Czekanowski	A	0.020	0.063	0.101	0.140	0.181
	B	0.020	0.061	0.102	0.141	0.180		B	0.022	0.064	0.108	0.144	0.180
	C	0.013	0.055	0.099	0.137	0.179		C	0.019	0.058	0.095	0.134	0.175
	D	0.027	0.066	0.103	0.139	0.179		D	0.019	0.059	0.098	0.141	0.181
	E	0.020	0.059	0.097	0.143	0.181		E	0.021	0.057	0.097	0.140	0.182
Divergence	A	0.019	0.059	0.101	0.141	0.181	Motyka	A	0.020	0.063	0.101	0.140	0.181
	B	0.021	0.057	0.097	0.137	0.178		B	0.022	0.064	0.108	0.144	0.180
	C	0.023	0.065	0.105	0.145	0.183		C	0.019	0.058	0.095	0.134	0.175
	D	0.019	0.057	0.099	0.135	0.176		D	0.019	0.059	0.098	0.141	0.181
	E	0.019	0.061	0.098	0.142	0.182		E	0.021	0.057	0.097	0.140	0.182
Clark	A	0.019	0.059	0.101	0.141	0.181	Inner product	A	0.021	0.059	0.100	0.143	0.183
	B	0.021	0.057	0.097	0.137	0.178		B	0.020	0.060	0.098	0.137	0.180
	C	0.023	0.065	0.105	0.145	0.183		C	0.018	0.061	0.100	0.139	0.179
	D	0.019	0.057	0.099	0.135	0.176		D	0.022	0.059	0.102	0.137	0.179
	E	0.019	0.061	0.098	0.142	0.182		E	0.020	0.061	0.100	0.145	0.179
L2 norm	A	0.017	0.059	0.099	0.140	0.179	Harmonic mean	A	0.019	0.059	0.100	0.140	0.181
	B	0.020	0.062	0.102	0.140	0.181		B	0.020	0.059	0.098	0.139	0.180
	C	0.022	0.062	0.100	0.139	0.181		C	0.021	0.063	0.101	0.145	0.187
	D	0.021	0.060	0.098	0.141	0.178		D	0.021	0.061	0.097	0.134	0.173
	E	0.020	0.057	0.101	0.139	0.181		E	0.019	0.056	0.103	0.141	0.180
Manhattan	A	0.019	0.058	0.097	0.136	0.177	Cosine	A	0.020	0.059	0.100	0.143	0.183
	B	0.019	0.062	0.100	0.140	0.182		B	0.020	0.060	0.098	0.137	0.180
	C	0.020	0.063	0.103	0.143	0.183		C	0.018	0.061	0.100	0.139	0.179
	D	0.023	0.059	0.099	0.139	0.178		D	0.021	0.059	0.102	0.137	0.179
	E	0.019	0.058	0.100	0.141	0.181		E	0.020	0.061	0.100	0.145	0.179
Chebyshev	A	0.026	0.071	0.108	0.149	0.186	Kumar–Hassebrook	A	0.020	0.059	0.100	0.143	0.183
	B	0.020	0.059	0.102	0.140	0.178		B	0.020	0.060	0.098	0.137	0.180
	C	0.017	0.053	0.091	0.132	0.176		C	0.018	0.061	0.100	0.139	0.179
	D	0.020	0.058	0.099	0.141	0.179		D	0.021	0.059	0.102	0.137	0.179
	E	0.018	0.059	0.099	0.138	0.181		E	0.020	0.061	0.100	0.145	0.179
Sorensen	A	0.020	0.063	0.101	0.140	0.181	Dice	A	0.017	0.057	0.100	0.141	0.180
	B	0.022	0.064	0.108	0.144	0.180		B	0.020	0.063	0.102	0.140	0.180
	C	0.019	0.058	0.095	0.134	0.175		C	0.021	0.061	0.100	0.139	0.182
	D	0.019	0.059	0.098	0.141	0.181		D	0.021	0.063	0.098	0.141	0.179
	E	0.021	0.057	0.097	0.140	0.182		E	0.021	0.055	0.100	0.139	0.180
Gower	A	0.019	0.058	0.097	0.136	0.177	Wasserstein	A	0.024	0.060	0.095	0.138	0.197
	B	0.019	0.062	0.100	0.140	0.182		B	0.024	0.071	0.103	0.136	0.176
	C	0.020	0.063	0.103	0.143	0.183		C	0.009	0.040	0.090	0.139	0.180
	D	0.023	0.059	0.099	0.139	0.178		D	0.031	0.072	0.109	0.146	0.175
	E	0.019	0.058	0.100	0.141	0.181		E	0.012	0.057	0.103	0.140	0.171
Kulczynski‘s D	A	0.018	0.056	0.109	0.154	0.193	Kolmogorov–Smirnov	A	0.026	0.067	0.119	0.160	0.185
	B	0.027	0.065	0.105	0.156	0.188		B	0.016	0.049	0.095	0.133	0.182
	C	0.016	0.069	0.115	0.141	0.187		C	0.008	0.044	0.085	0.134	0.175
	D	0.015	0.052	0.090	0.135	0.188		D	0.038	0.076	0.096	0.135	0.172
	E	0.024	0.057	0.081	0.113	0.144		E	0.010	0.063	0.101	0.136	0.186
Canberra	A	0.018	0.059	0.098	0.141	0.181	Runs	A	0.019	0.051	0.091	0.137	0.177
	B	0.022	0.063	0.100	0.139	0.178		B	0.022	0.060	0.101	0.139	0.178
	C	0.014	0.053	0.100	0.138	0.179		C	0.021	0.058	0.104	0.139	0.184
	D	0.025	0.067	0.104	0.140	0.180		D	0.014	0.049	0.089	0.128	0.176
	E	0.020	0.058	0.098	0.142	0.182		E	0.023	0.061	0.103	0.144	0.183

,

Table 6. Posterior probabilities in experiments on trend changes.

Distance Measure	SignIf.	0.1	0.3	0.5	0.7	0.9	Distance Measure	SignIf.	0.1	0.3	0.5	0.7	0.9
Squared Euclidean	A	0.017	0.040	0.167	0.167	0.167	Lorentzian	A	0.048	0.083	0.167	0.167	0.167
	B	0.040	0.108	0.167	0.167	0.167		B	0.039	0.105	0.167	0.167	0.167
	C	0.043	0.152	0.167	0.167	0.167		C	0.013	0.112	0.167	0.167	0.167
	D	0.000	0.000	0.000	0.092	0.122		D	0.000	0.000	0.000	0.099	0.127
	E	0.000	0.000	0.000	0.062	0.132		E	0.000	0.000	0.000	0.061	0.129
	F	0.000	0.000	0.000	0.046	0.146		F	0.000	0.000	0.000	0.041	0.145
Pearson	A	0.021	0.070	0.106	0.132	0.153	Intersection	A	0.000	0.000	0.000	0.083	0.117
	B	0.018	0.060	0.101	0.125	0.147		B	0.000	0.000	0.000	0.062	0.129
	C	0.020	0.060	0.104	0.130	0.152		C	0.000	0.000	0.000	0.054	0.154
	D	0.012	0.036	0.063	0.102	0.151		D	0.037	0.073	0.167	0.167	0.167
	E	0.015	0.038	0.064	0.106	0.149		E	0.039	0.102	0.167	0.167	0.167
	F	0.014	0.036	0.062	0.104	0.148		F	0.024	0.124	0.167	0.167	0.167
Neyman	A	0.030	0.053	0.072	0.085	0.119	Nonintersection	A	0.050	0.083	0.167	0.167	0.167
	B	0.003	0.033	0.067	0.099	0.147		B	0.037	0.104	0.167	0.167	0.167
	C	0.000	0.004	0.035	0.109	0.164		C	0.013	0.112	0.167	0.167	0.167
	D	0.051	0.083	0.101	0.117	0.139		D	0.000	0.000	0.000	0.093	0.129
	E	0.016	0.067	0.104	0.135	0.164		E	0.000	0.000	0.000	0.065	0.128
	F	0.000	0.060	0.121	0.155	0.167		F	0.000	0.000	0.000	0.042	0.143
Squared chi	A	0.006	0.014	0.052	0.105	0.156	Wave hedges	A	0.017	0.046	0.075	0.110	0.149
	B	0.004	0.014	0.045	0.107	0.157		B	0.015	0.037	0.064	0.103	0.145
	C	0.006	0.015	0.051	0.110	0.156		C	0.015	0.036	0.062	0.106	0.149
	D	0.022	0.071	0.109	0.121	0.143		D	0.017	0.060	0.099	0.127	0.152
	E	0.029	0.089	0.120	0.127	0.142		E	0.018	0.063	0.101	0.127	0.152
	F	0.033	0.098	0.124	0.130	0.145		F	0.018	0.060	0.100	0.127	0.152
Prob. symmetric	A	0.006	0.014	0.052	0.105	0.156	Czekanowski	A	0.066	0.096	0.167	0.167	0.167
	B	0.004	0.014	0.045	0.107	0.157		B	0.027	0.103	0.167	0.167	0.167
	C	0.006	0.015	0.051	0.110	0.156		C	0.007	0.101	0.167	0.167	0.167
	D	0.022	0.071	0.109	0.121	0.143		D	0.000	0.000	0.000	0.074	0.106
	E	0.029	0.089	0.120	0.127	0.142		E	0.000	0.000	0.000	0.065	0.138
	F	0.033	0.098	0.124	0.130	0.145		F	0.000	0.000	0.000	0.062	0.156
Divergence	A	0.039	0.077	0.107	0.133	0.156	Motyka	A	0.066	0.096	0.167	0.167	0.167
	B	0.028	0.077	0.111	0.136	0.157		B	0.027	0.103	0.167	0.167	0.167
	C	0.014	0.068	0.107	0.134	0.155		C	0.007	0.101	0.167	0.167	0.167
	D	0.011	0.035	0.069	0.108	0.146		D	0.000	0.000	0.000	0.074	0.106
	E	0.006	0.029	0.057	0.097	0.143		E	0.000	0.000	0.000	0.065	0.138
	F	0.001	0.014	0.050	0.092	0.143		F	0.000	0.000	0.000	0.062	0.156
Clark	A	0.039	0.077	0.107	0.133	0.156	Inner product	A	0.000	0.000	0.000	0.103	0.124
	B	0.028	0.077	0.111	0.136	0.157		B	0.000	0.000	0.000	0.064	0.127
	C	0.014	0.068	0.107	0.134	0.155		C	0.000	0.000	0.000	0.033	0.148
	D	0.011	0.035	0.069	0.108	0.146		D	0.036	0.063	0.167	0.167	0.167
	E	0.006	0.029	0.057	0.097	0.143		E	0.035	0.103	0.167	0.167	0.167
	F	0.001	0.014	0.050	0.092	0.143		F	0.029	0.134	0.167	0.167	0.167
L2 norm	A	0.017	0.040	0.167	0.167	0.167	Harmonic mean	A	0.011	0.062	0.115	0.153	0.161
	B	0.040	0.108	0.167	0.167	0.167		B	0.010	0.060	0.122	0.153	0.163
	C	0.043	0.152	0.167	0.167	0.167		C	0.010	0.057	0.117	0.151	0.161
	D	0.000	0.000	0.000	0.092	0.122		D	0.023	0.046	0.057	0.096	0.144
	E	0.000	0.000	0.000	0.062	0.132		E	0.024	0.039	0.047	0.078	0.137
	F	0.000	0.000	0.000	0.046	0.146		F	0.021	0.036	0.043	0.069	0.134
Manhattan	A	0.045	0.078	0.167	0.167	0.167	Cosine	A	0.000	0.000	0.000	0.112	0.134
	B	0.041	0.105	0.167	0.167	0.167		B	0.000	0.000	0.000	0.063	0.124
	C	0.014	0.117	0.167	0.167	0.167		C	0.000	0.000	0.000	0.025	0.141
	D	0.000	0.000	0.000	0.104	0.130		D	0.026	0.056	0.167	0.167	0.167
	E	0.000	0.000	0.000	0.060	0.128		E	0.036	0.104	0.167	0.167	0.167
	F	0.000	0.000	0.000	0.036	0.142		F	0.037	0.139	0.167	0.167	0.167
Chebyshev	A	0.016	0.040	0.166	0.167	0.167	Hassebrook	A	0.000	0.000	0.000	0.111	0.134
	B	0.031	0.102	0.167	0.167	0.167		B	0.000	0.000	0.000	0.063	0.124
	C	0.053	0.158	0.167	0.167	0.167		C	0.000	0.000	0.000	0.026	0.141
	D	0.000	0.000	0.001	0.079	0.113		D	0.027	0.057	0.167	0.167	0.167
	E	0.000	0.000	0.000	0.068	0.136		E	0.036	0.104	0.167	0.167	0.167
	F	0.000	0.000	0.000	0.053	0.151		F	0.037	0.139	0.167	0.167	0.167
Sorensen	A	0.066	0.096	0.167	0.167	0.167	Dice	A	0.032	0.055	0.167	0.167	0.167
	B	0.027	0.103	0.167	0.167	0.167		B	0.042	0.103	0.167	0.167	0.167
	C	0.007	0.101	0.167	0.167	0.167		C	0.025	0.141	0.167	0.167	0.167
	D	0.000	0.000	0.000	0.074	0.106		D	0.000	0.000	0.000	0.110	0.140
	E	0.000	0.000	0.000	0.065	0.138		E	0.000	0.000	0.000	0.063	0.130
	F	0.000	0.000	0.000	0.062	0.156		F	0.000	0.000	0.000	0.027	0.130
Gower	A	0.045	0.078	0.167	0.167	0.167	Wasserstein	A	0.015	0.031	0.045	0.067	0.127
	B	0.041	0.105	0.167	0.167	0.167		B	0.019	0.056	0.086	0.125	0.162
	C	0.014	0.117	0.167	0.167	0.167		C	0.012	0.055	0.115	0.155	0.167
	D	0.000	0.000	0.000	0.104	0.130		D	0.021	0.038	0.056	0.072	0.111
	E	0.000	0.000	0.000	0.060	0.128		E	0.020	0.056	0.085	0.128	0.167
	F	0.000	0.000	0.000	0.036	0.142		F	0.014	0.065	0.112	0.153	0.167
Kulczynski‘s D	A	0.000	0.000	0.000	0.110	0.144	Kolmogorov–Smirnov	A	0.017	0.041	0.050	0.070	0.120
	B	0.000	0.000	0.000	0.057	0.160		B	0.019	0.051	0.074	0.119	0.162
	C	0.000	0.000	0.000	0.033	0.096		C	0.007	0.060	0.118	0.157	0.167
	D	0.064	0.167	0.167	0.167	0.167		D	0.018	0.038	0.048	0.070	0.115
	E	0.026	0.093	0.167	0.167	0.167		E	0.020	0.050	0.070	0.124	0.166
	F	0.009	0.040	0.167	0.167	0.167		F	0.010	0.060	0.105	0.155	0.167
Canberra	A	0.010	0.022	0.058	0.107	0.154	Runs	A	0.001	0.006	0.022	0.049	0.117
	B	0.007	0.021	0.053	0.109	0.154		B	0.003	0.027	0.074	0.128	0.163
	C	0.009	0.025	0.057	0.110	0.154		C	0.023	0.102	0.152	0.166	0.167
	D	0.021	0.064	0.102	0.122	0.147		D	0.002	0.009	0.028	0.053	0.108
	E	0.024	0.079	0.113	0.125	0.144		E	0.004	0.026	0.073	0.122	0.163
	F	0.029	0.088	0.117	0.128	0.147		F	0.022	0.099	0.150	0.166	0.167

and

Table 7. Posterior probabilities in experiments determining the presence of a unit root.

Distance Measure	SignIf.	0.1	0.3	0.5	0.7	0.9	Distance Measure	SignIf.	0.1	0.3	0.5	0.7	0.9
Squared Euclidean	A	0.026	0.062	0.098	0.136	0.175	Lorentzian	A	0.023	0.061	0.097	0.137	0.177
	B	0.028	0.063	0.096	0.132	0.172		B	0.025	0.064	0.100	0.136	0.175
	C	0.021	0.060	0.100	0.137	0.179		C	0.021	0.065	0.109	0.145	0.180
	D	0.016	0.057	0.099	0.140	0.181		D	0.022	0.066	0.107	0.142	0.181
	E	0.010	0.057	0.107	0.156	0.193		E	0.010	0.045	0.087	0.139	0.187
Pearson	A	0.022	0.069	0.109	0.144	0.177	Intersection	A	0.019	0.060	0.100	0.142	0.183
	B	0.017	0.051	0.088	0.137	0.182		B	0.024	0.061	0.101	0.137	0.178
	C	0.021	0.059	0.095	0.133	0.181		C	0.019	0.055	0.096	0.137	0.177
	D	0.021	0.059	0.099	0.137	0.177		D	0.022	0.063	0.100	0.139	0.176
	E	0.019	0.062	0.108	0.149	0.182		E	0.016	0.061	0.103	0.145	0.186
Neyman	A	0.019	0.059	0.101	0.143	0.183	Nonintersection	A	0.017	0.058	0.100	0.140	0.181
	B	0.021	0.060	0.100	0.145	0.181		B	0.022	0.063	0.099	0.139	0.176
	C	0.022	0.063	0.101	0.142	0.180		C	0.023	0.063	0.104	0.145	0.181
	D	0.022	0.060	0.101	0.142	0.180		D	0.024	0.061	0.100	0.137	0.178
	E	0.016	0.059	0.096	0.129	0.176		E	0.014	0.055	0.097	0.139	0.184
Squared chi	A	0.020	0.059	0.101	0.142	0.182	Wave hedges	A	0.024	0.063	0.105	0.141	0.183
	B	0.021	0.066	0.105	0.141	0.180		B	0.018	0.056	0.095	0.137	0.181
	C	0.018	0.055	0.103	0.144	0.181		C	0.021	0.062	0.104	0.148	0.182
	D	0.017	0.057	0.098	0.138	0.180		D	0.019	0.057	0.095	0.137	0.180
	E	0.024	0.063	0.093	0.135	0.178		E	0.018	0.061	0.102	0.137	0.174
Prob. symmetric	A	0.020	0.059	0.101	0.142	0.182	Czekanowski	A	0.030	0.078	0.100	0.124	0.174
	B	0.021	0.066	0.105	0.141	0.180		B	0.022	0.074	0.105	0.138	0.178
	C	0.018	0.055	0.103	0.144	0.181		C	0.021	0.058	0.096	0.130	0.178
	D	0.017	0.057	0.098	0.138	0.180		D	0.015	0.051	0.100	0.146	0.182
	E	0.024	0.063	0.093	0.135	0.178		E	0.012	0.038	0.099	0.161	0.187
Divergence	A	0.024	0.063	0.102	0.146	0.180	Motyka	A	0.030	0.078	0.100	0.124	0.174
	B	0.018	0.053	0.093	0.137	0.181		B	0.022	0.074	0.105	0.138	0.178
	C	0.022	0.066	0.109	0.145	0.180		C	0.021	0.058	0.096	0.130	0.178
	D	0.019	0.060	0.099	0.141	0.182		D	0.015	0.051	0.100	0.146	0.182
	E	0.018	0.059	0.096	0.132	0.177		E	0.012	0.038	0.099	0.161	0.187
Clark	A	0.024	0.063	0.102	0.146	0.180	Inner product	A	0.025	0.063	0.099	0.140	0.174
	B	0.018	0.053	0.093	0.137	0.181		B	0.026	0.065	0.103	0.135	0.172
	C	0.022	0.066	0.109	0.145	0.180		C	0.022	0.061	0.099	0.137	0.178
	D	0.019	0.060	0.099	0.141	0.182		D	0.017	0.058	0.101	0.143	0.185
	E	0.018	0.059	0.096	0.132	0.177		E	0.010	0.052	0.099	0.145	0.190
L2 norm	A	0.026	0.062	0.098	0.136	0.175	Harmonic mean	A	0.018	0.058	0.098	0.141	0.180
	B	0.028	0.063	0.096	0.132	0.172		B	0.020	0.059	0.095	0.135	0.179
	C	0.021	0.060	0.100	0.137	0.179		C	0.019	0.056	0.098	0.145	0.182
	D	0.016	0.057	0.099	0.140	0.181		D	0.020	0.062	0.102	0.143	0.183
	E	0.010	0.057	0.107	0.156	0.193		E	0.022	0.065	0.107	0.137	0.176
Manhattan	A	0.024	0.059	0.097	0.136	0.176	Cosine	A	0.025	0.064	0.099	0.140	0.174
	B	0.025	0.065	0.098	0.133	0.173		B	0.026	0.065	0.103	0.135	0.172
	C	0.021	0.065	0.107	0.141	0.180		C	0.022	0.061	0.099	0.136	0.179
	D	0.021	0.066	0.105	0.143	0.182		D	0.017	0.058	0.101	0.143	0.184
	E	0.009	0.044	0.094	0.148	0.189		E	0.010	0.052	0.099	0.146	0.190
Chebyshev	A	0.017	0.058	0.102	0.146	0.186	Kumar–Hassebrook	A	0.025	0.064	0.099	0.140	0.174
	B	0.020	0.060	0.097	0.138	0.181		B	0.026	0.065	0.103	0.135	0.172
	C	0.017	0.054	0.094	0.132	0.176		C	0.022	0.061	0.099	0.136	0.179
	D	0.016	0.050	0.089	0.129	0.170		D	0.017	0.058	0.101	0.143	0.184
	E	0.029	0.079	0.119	0.156	0.187		E	0.010	0.052	0.099	0.146	0.190
Sorensen	A	0.030	0.078	0.100	0.124	0.174	Dice	A	0.026	0.060	0.101	0.136	0.175
	B	0.022	0.074	0.105	0.138	0.178		B	0.028	0.065	0.097	0.135	0.174
	C	0.021	0.058	0.096	0.130	0.178		C	0.021	0.064	0.101	0.139	0.178
	D	0.015	0.051	0.100	0.146	0.182		D	0.016	0.057	0.099	0.142	0.183
	E	0.012	0.038	0.099	0.161	0.187		E	0.010	0.054	0.101	0.148	0.190
Gower	A	0.024	0.059	0.097	0.136	0.176	Wasserstein	A	0.062	0.136	0.178	0.193	0.200
	B	0.025	0.065	0.098	0.133	0.173		B	0.016	0.065	0.120	0.172	0.200
	C	0.021	0.065	0.107	0.141	0.180		C	0.010	0.055	0.116	0.184	0.200
	D	0.021	0.066	0.105	0.143	0.182		D	0.010	0.037	0.071	0.116	0.196
	E	0.009	0.044	0.094	0.148	0.189		E	0.003	0.008	0.015	0.034	0.104
Kulczynski‘s D	A	0.018	0.074	0.145	0.200	0.200	Kolmogorov–Smirnov	A	0.050	0.114	0.159	0.199	0.200
	B	0.007	0.059	0.111	0.169	0.200		B	0.015	0.063	0.102	0.156	0.198
	C	0.023	0.083	0.136	0.193	0.200		C	0.018	0.071	0.133	0.186	0.200
	D	0.052	0.084	0.108	0.139	0.200		D	0.014	0.043	0.081	0.123	0.196
	E	0.000	0.000	0.000	0.000	0.100		E	0.003	0.009	0.019	0.035	0.105
Canberra	A	0.019	0.058	0.098	0.141	0.180	Runs	A	0.026	0.067	0.105	0.147	0.181
	B	0.021	0.064	0.105	0.141	0.180		B	0.016	0.056	0.088	0.137	0.186
	C	0.018	0.060	0.101	0.143	0.180		C	0.022	0.065	0.103	0.155	0.191
	D	0.018	0.057	0.099	0.139	0.180		D	0.019	0.060	0.098	0.150	0.185
	E	0.024	0.061	0.097	0.137	0.179		E	0.012	0.038	0.064	0.101	0.142

present a comprehensive overview of the performance of various distance measures and statistical tests across a range of experimental conditions, including stationary time series, dynamic variance, trend changes, and the presence of a unit root. The tables provide detailed insights into the relative merits and limitations of each method in identifying the true models under these diverse scenarios. Table 4 is devoted to an examination of the effectiveness of the methods in the context of a stationary time series. Table 5, Table 6 and Table 7 are dedicated to an evaluation of the methods under more intricate circumstances, such as unstable variance, trend alterations, and unit roots.

To evaluate the effectiveness of distance metrics and statistical tests employed to calculate r(p_m,j, X), it is essential that observed data and parameters exhibit a high posterior probability for the same model. A high posterior probability for the same model ensures that the model aligns well with the data, thereby providing a robust framework for identifying the true model even under challenging conditions. In an ideal scenario, even when a high significance level is set, the true model should be consistently identified. In experiments concerning candidate models for stationary time series, as presented in Table 4, setting the significance level at the 10% quantile revealed that 13 out of 29 metrics, including the L₂ norm, Manhattan, and Chebyshev distances, along with Kolmogorov–Smirnov and runs tests, successfully identified the true model. The results presented herein demonstrate the efficacy of these metrics in stable time series data, particularly under lower significance levels, where model identification is more precise.

However, as the significance level increased, the majority of methods were unable to identify the true model. The challenge of identifying the optimal model at progressively higher levels of statistical significance underscores the inherent trade-offs involved in selecting an appropriate level of significance, particularly in the context of more complex datasets. From the perspective of a practitioner, the absence of a definitive criterion for setting the appropriate significance level presents a significant challenge. If the significance level is set too low, issues of reproducibility may arise. Conversely, setting it too high could lead to results that are difficult to interpret as meaningful, potentially undermining the novelty of findings. Notwithstanding the aforementioned challenges, the outcomes illustrated in Table 4 indicate that distance metrics such as the Chebyshev distance, the Wasserstein distance, and the Kolmogorov–Smirnov test exhibited resilience across varying levels of statistical significance, consistently identifying the authentic model in stationary time series scenarios. This resilience suggests that these metrics are robust and reliable in stable conditions.

In the examination of candidate models exhibiting unstable variance, specifically where the variance parameter σ₂ > 1, the majority of methods failed to identify the true model even when the significance level was reduced to 0.1. As illustrated in Table 5, the findings reveal that distance-based measures, such as the Chebyshev and Wasserstein distances, demonstrated superior performance compared to traditional statistical tests in scenarios characterized by high variance. These findings highlight the significant advantage of distance-based techniques in addressing fluctuations in variance, rendering them highly effective for data exhibiting dynamic variance. Despite these challenges, certain methodologies, notably the Chebyshev distance and the Kolmogorov–Smirnov test, demonstrated robust performance under these conditions.

In scenarios where both observed data and candidate models exhibit trends, it is of paramount importance to identify models that not only match the trend direction but also its slope. The complexity of trend detection represents a significant challenge in identifying the most appropriate model, as it is not a straightforward process. For example, candidate models A, B, and C were generated with increasing trends, characterized by parameters µ of 0.5, 1.0, and 2.0, respectively. In contrast, models D, E, and F were generated with decreasing trends, showing parameters µ of −0.5, −1.0, and −2.0, respectively. As illustrated in Table 6, while numerous distance measures, including the L₂ norm, Manhattan, and Chebyshev distances, were effective in identifying the trend direction, Pearson distance was the sole method that correctly identified both the trend direction and slope. This underscores Pearson distance’s distinctive capability in detecting subtle variations in trend patterns. These findings are presented in Table 6, emphasizing the necessity of additional processes, such as detrending, when working with data that exhibit trends.

In experiments designed to assess the presence of a unit root, Pearson distance, Wasserstein distance, the Kolmogorov–Smirnov test, and the runs test demonstrated relatively effective performances in correctly selecting the true model. As demonstrated by the findings in Table 7, the Wasserstein and Pearson distances exhibit superior performance in identifying unit roots, a task that frequently presents a challenge for other methods. A comprehensive analysis revealed that, regardless of parameter configurations, the Kolmogorov–Smirnov test consistently identified the true model when calculating r(p_m,j, X).

Moreover, this study demonstrates that the Wasserstein distance is particularly useful when considering the presence of a unit root. Moreover, the findings indicate that the Wasserstein distance, with its capacity to process nonstationary data, is a valuable tool for practitioners. Additionally, in the event that the data exhibit dynamic variance, it is recommended to consider both Chebyshev and Wasserstein distances, along with Pearson distance, when accounting for trends. The comprehensive results demonstrate that the application of multiple tailored approaches, based on the specific characteristics of the data, can markedly enhance model accuracy and provide reliable insights. Given that each distance measure possesses distinctive strengths under different conditions, the utilization of a combination of these methods allows for a more nuanced and comprehensive analysis of time series data.

4.3. Practical Applications in the Energy Sector

The European Network of Transmission System Operators for Electricity (ENTSO-E) plays a pivotal role in the implementation of energy policies across the European Union [32]. It contributes to the achievement of Europe’s energy and climate goals by aggregating and disseminating critical energy data, including monthly power consumption, production, and annual net generating capacities, through its online platform. In the context of this research, we conducted an analysis of hourly net generating capacity data from 1 January 2006 to 31 December 2015. The subset of data pertaining to Austria (AT), specifically from t = 3000 to t = 3999, was treated as observed data in this study. The dataset is systematically cataloged in

Table 8. Detailed ISO country codes with corresponding national profiles.

Country Code	Country Name	Country Code	Country Name
AT	Austria	IE	Ireland
BA	Bosnia Herzegovina	IS	Iceland
BE	Belgium	IT	Italy
BG	Bulgaria	LT	Lithuania
CH	Switzerland	LU	Luxembourg
CS	Serbia and Montenegro	LV	Latvia
CY	Cyprus	ME	Montenegro
CZ	Czech Republic	MK	North Macedonia
DE	Germany	NI	Northern Ireland
DK	Denmark	NL	Netherlands
DKW	Denmark West	NO	Norway
EE	Estonia	PL	Poland
ES	Spain	PT	Portugal
FR	France	RO	Romania
GB	Great Britain	RS	Serbia
GR	Greece	SE	Sweden
HR	Croatia	SI	Slovenia
HU	Hungary	SK	Slovakia

, which provides a detailed view of power generation dynamics across 35 European nations.

As demonstrated in Figure 9, the power data exhibited strong seasonal variations along with pronounced trend changes. In the short term, these points of trend shifts might also indicate structural changes. However, the presence of a unit root cannot be ascertained through visual means alone. Unlike synthetic data, the actual dataset often contains numerous instances where data collection has not occurred, resulting in significant gaps or missing values. As previously discussed in Section 3.2, our approach to handling missing data involved the exclusion of segments containing gaps during the dataset partitioning process. This method simplifies the handling of missing values while simultaneously avoiding an increase in uncertainty that typically accompanies imputation techniques. However, one disadvantage of this approach is the difficulty in obtaining reliable estimates of r(p_m,j, X) for segments where missing values are concentrated.

The analysis, as documented in

Table 9. Analysis of posterior probabilities in ENTSO-E experiments, ranging from squared Euclidean to Canberra distances.

Country Code	Squared Euclidean	Pearson	Neyman	Squared Chi	Prob. Symmetric	Divergence	Clark
AT	0.00268	0.00253	0.00201	0.00262	0.00262	0.00227	0.00227
BA	0.00280	0.00287	0.00243	0.00260	0.00260	0.00257	0.00257
BE	0.00315	0.00332	0.00232	0.00285	0.00285	0.00263	0.00263
BG	0.00329	0.00300	0.00244	0.00281	0.00281	0.00281	0.00281
CH	0.00242	0.00227	0.00212	0.00266	0.00266	0.00220	0.00220
CS	0.00171	0.00291	0.00367	0.00250	0.00250	0.00218	0.00218
CY	0.00171	0.00299	0.00351	0.00224	0.00224	0.00199	0.00199
CZ	0.00336	0.00264	0.00177	0.00296	0.00296	0.00277	0.00277
DE	0.00227	0.00276	0.00281	0.00247	0.00247	0.00303	0.00303
DK	0.00282	0.00218	0.00309	0.00280	0.00280	0.00283	0.00283
DKW	0.00273	0.00235	0.00213	0.00267	0.00267	0.00248	0.00248
EE	0.00423	0.00180	0.00299	0.00296	0.00296	0.00311	0.00311
ES	0.00384	0.00266	0.00173	0.00306	0.00306	0.00287	0.00287
FR	0.00244	0.00239	0.00185	0.00260	0.00260	0.00226	0.00226
GB	0.00393	0.00300	0.00213	0.00310	0.00310	0.00258	0.00258
GR	0.00139	0.00325	0.00329	0.00223	0.00223	0.00226	0.00226
HR	0.00372	0.00299	0.00226	0.00299	0.00299	0.00261	0.00261
HU	0.00298	0.00293	0.00393	0.00244	0.00244	0.00415	0.00415
IE	0.00281	0.00299	0.00354	0.00250	0.00250	0.00395	0.00395
IS	0.00405	0.00229	0.00161	0.00318	0.00318	0.00269	0.00269
IT	0.00328	0.00269	0.00185	0.00295	0.00295	0.00280	0.00280
LT	0.00243	0.00257	0.00247	0.00260	0.00260	0.00216	0.00216
LU	0.00219	0.00321	0.00391	0.00233	0.00233	0.00256	0.00256
LV	0.00250	0.00222	0.00181	0.00269	0.00269	0.00220	0.00220
ME	0.00263	0.00239	0.00160	0.00276	0.00276	0.00235	0.00235
MK	0.00194	0.00232	0.00231	0.00260	0.00260	0.00204	0.00204
NI	0.00255	0.00287	0.00394	0.00238	0.00238	0.00366	0.00366
NL	0.00245	0.00290	0.00319	0.00247	0.00247	0.00253	0.00253
NO	0.00252	0.00227	0.00222	0.00271	0.00271	0.00216	0.00216
PL	0.00261	0.00240	0.00247	0.00264	0.00264	0.00225	0.00225
PT	0.00218	0.00264	0.00320	0.00256	0.00256	0.00309	0.00309
RO	0.00211	0.00295	0.00352	0.00247	0.00247	0.00337	0.00337
RS	0.00125	0.00340	0.00377	0.00207	0.00207	0.00248	0.00248
SE	0.00269	0.00275	0.00423	0.00248	0.00248	0.00346	0.00346
SI	0.00239	0.00239	0.00243	0.00252	0.00252	0.00226	0.00226
SK	0.00244	0.00206	0.00173	0.00263	0.00263	0.00204	0.00204
Country code	L2 norm	Manhattan	Chebyshev	Sorensen	Gower	Kulczynski‘s D	Canberra
AT	0.00268	0.00299	0.00251	0.00259	0.00299	0.00288	0.00257
BA	0.00280	0.00319	0.00270	0.00282	0.00319	0.00000	0.00252
BE	0.00315	0.00345	0.00279	0.00288	0.00345	0.00197	0.00272
BG	0.00329	0.00351	0.00249	0.00257	0.00351	0.00460	0.00274
CH	0.00242	0.00259	0.00212	0.00243	0.00259	0.00053	0.00265
CS	0.00171	0.00185	0.00097	0.00160	0.00185	0.00529	0.00251
CY	0.00171	0.00185	0.00114	0.00146	0.00185	0.00618	0.00232
CZ	0.00336	0.00337	0.00364	0.00332	0.00337	0.00175	0.00291
DE	0.00227	0.00221	0.00221	0.00256	0.00221	0.00243	0.00255
DK	0.00282	0.00236	0.00401	0.00324	0.00236	0.00123	0.00279
DKW	0.00273	0.00267	0.00342	0.00269	0.00267	0.00153	0.00270
EE	0.00423	0.00359	0.00598	0.00476	0.00359	0.00190	0.00295
ES	0.00384	0.00375	0.00413	0.00394	0.00375	0.00074	0.00297
FR	0.00244	0.00277	0.00245	0.00231	0.00277	0.00003	0.00260
GB	0.00393	0.00397	0.00364	0.00378	0.00397	0.00094	0.00295
GR	0.00139	0.00145	0.00067	0.00191	0.00145	0.00125	0.00230
HR	0.00372	0.00373	0.00315	0.00340	0.00373	0.00224	0.00289
HU	0.00298	0.00236	0.00436	0.00246	0.00236	0.00402	0.00248
IE	0.00281	0.00223	0.00393	0.00261	0.00223	0.00467	0.00251
IS	0.00405	0.00397	0.00382	0.00399	0.00397	0.00023	0.00309
IT	0.00328	0.00364	0.00309	0.00328	0.00364	0.00000	0.00285
LT	0.00243	0.00265	0.00189	0.00215	0.00265	0.00251	0.00258
LU	0.00219	0.00232	0.00085	0.00127	0.00232	0.00631	0.00236
LV	0.00250	0.00267	0.00291	0.00259	0.00267	0.00067	0.00271
ME	0.00263	0.00285	0.00273	0.00281	0.00285	0.00090	0.00274
MK	0.00194	0.00202	0.00171	0.00233	0.00202	0.00229	0.00265
NI	0.00255	0.00187	0.00393	0.00303	0.00187	0.00602	0.00247
NL	0.00245	0.00266	0.00143	0.00170	0.00266	0.00529	0.00247
NO	0.00252	0.00258	0.00181	0.00241	0.00258	0.00056	0.00272
PL	0.00261	0.00268	0.00217	0.00233	0.00268	0.00347	0.00265
PT	0.00218	0.00190	0.00192	0.00258	0.00190	0.00489	0.00262
RO	0.00211	0.00176	0.00293	0.00243	0.00176	0.00546	0.00251
RS	0.00125	0.00150	0.00048	0.00166	0.00150	0.00431	0.00220
SE	0.00269	0.00193	0.00415	0.00358	0.00193	0.00528	0.00249
SI	0.00239	0.00264	0.00176	0.00221	0.00264	0.00355	0.00252
SK	0.00244	0.00273	0.00232	0.00236	0.00273	0.00034	0.00259

and

Table 10. Analysis of posterior probabilities in ENTSO-E experiments from Lorentzian distance to runs test.

Country Code	Lorentzian	Intersection	Nonintersection	Wave Hedges	Czekanowski	Motyka	Inner Product
AT	0.00318	0.00258	0.00311	0.00231	0.00259	0.00259	0.00245
BA	0.00345	0.00307	0.00329	0.00239	0.00282	0.00282	0.00290
BE	0.00367	0.00392	0.00346	0.00298	0.00288	0.00288	0.00309
BG	0.00361	0.00391	0.00356	0.00296	0.00257	0.00257	0.00309
CH	0.00267	0.00197	0.00264	0.00231	0.00243	0.00243	0.00218
CS	0.00194	0.00160	0.00186	0.00298	0.00160	0.00160	0.00105
CY	0.00190	0.00093	0.00177	0.00264	0.00146	0.00146	0.00066
CZ	0.00337	0.00444	0.00336	0.00280	0.00332	0.00332	0.00399
DE	0.00219	0.00248	0.00221	0.00244	0.00256	0.00256	0.00280
DK	0.00212	0.00166	0.00233	0.00244	0.00324	0.00324	0.00254
DKW	0.00263	0.00221	0.00275	0.00275	0.00269	0.00269	0.00271
EE	0.00317	0.00372	0.00356	0.00225	0.00476	0.00476	0.00400
ES	0.00368	0.00511	0.00375	0.00263	0.00394	0.00394	0.00435
FR	0.00295	0.00245	0.00285	0.00239	0.00231	0.00231	0.00227
GB	0.00405	0.00476	0.00394	0.00305	0.00378	0.00378	0.00331
GR	0.00148	0.00094	0.00134	0.00296	0.00191	0.00191	0.00071
HR	0.00379	0.00431	0.00374	0.00290	0.00340	0.00340	0.00316
HU	0.00196	0.00293	0.00229	0.00292	0.00246	0.00246	0.00407
IE	0.00193	0.00347	0.00217	0.00302	0.00261	0.00261	0.00395
IS	0.00391	0.00551	0.00394	0.00212	0.00399	0.00399	0.00442
IT	0.00382	0.00414	0.00368	0.00242	0.00328	0.00328	0.00327
LT	0.00275	0.00197	0.00265	0.00264	0.00215	0.00215	0.00213
LU	0.00236	0.00202	0.00237	0.00303	0.00127	0.00127	0.00115
LV	0.00273	0.00249	0.00271	0.00259	0.00259	0.00259	0.00261
ME	0.00299	0.00285	0.00301	0.00242	0.00281	0.00281	0.00290
MK	0.00205	0.00186	0.00193	0.00242	0.00233	0.00233	0.00217
NI	0.00152	0.00198	0.00179	0.00300	0.00303	0.00303	0.00326
NL	0.00276	0.00272	0.00265	0.00279	0.00170	0.00170	0.00221
NO	0.00263	0.00184	0.00262	0.00251	0.00241	0.00241	0.00197
PL	0.00270	0.00208	0.00276	0.00256	0.00233	0.00233	0.00219
PT	0.00177	0.00187	0.00187	0.00255	0.00258	0.00258	0.00274
RO	0.00162	0.00248	0.00168	0.00304	0.00243	0.00243	0.00317
RS	0.00163	0.00112	0.00132	0.00278	0.00166	0.00166	0.00062
SE	0.00144	0.00162	0.00181	0.00284	0.00358	0.00358	0.00299
SI	0.00279	0.00232	0.00263	0.00256	0.00221	0.00221	0.00263
SK	0.00295	0.00232	0.00282	0.00229	0.00236	0.00236	0.00245
Country code	Harmonic mean	Cosine	Kumar–Hassebrook	Dice	Wasserstein	Kolmogorov–Smirnov	Runs
AT	0.00288	0.00247	0.00247	0.00268	0.01404	0.01413	0.01195
BA	0.00293	0.00287	0.00287	0.00278	0.01260	0.01056	0.00441
BE	0.00302	0.00306	0.00306	0.00315	0.00417	0.00221	0.00233
BG	0.00283	0.00305	0.00305	0.00329	0.00102	0.00018	0.00000
CH	0.00281	0.00219	0.00219	0.00242	0.00282	0.00364	0.00707
CS	0.00242	0.00103	0.00103	0.00171	0.00005	0.00012	0.00144
CY	0.00244	0.00064	0.00064	0.00172	0.00211	0.00286	0.00438
CZ	0.00298	0.00398	0.00398	0.00334	0.00043	0.00021	0.00056
DE	0.00269	0.00277	0.00277	0.00225	0.00002	0.00029	0.00107
DK	0.00254	0.00254	0.00254	0.00281	0.00000	0.00000	0.00003
DKW	0.00250	0.00273	0.00273	0.00274	0.00000	0.00000	0.00055
EE	0.00227	0.00401	0.00401	0.00423	0.00000	0.00000	0.00000
ES	0.00280	0.00435	0.00435	0.00382	0.00000	0.00000	0.00004
FR	0.00290	0.00227	0.00227	0.00245	0.00814	0.00727	0.00507
GB	0.00291	0.00331	0.00331	0.00392	0.00000	0.00000	0.00000
GR	0.00242	0.00073	0.00073	0.00142	0.00040	0.00080	0.00000
HR	0.00290	0.00314	0.00315	0.00371	0.00003	0.00001	0.00000
HU	0.00203	0.00406	0.00406	0.00299	0.00003	0.00002	0.00000
IE	0.00220	0.00394	0.00393	0.00281	0.00000	0.00000	0.00000
IS	0.00306	0.00443	0.00443	0.00403	0.00000	0.00000	0.00000
IT	0.00305	0.00327	0.00327	0.00326	0.00518	0.00388	0.00318
LT	0.00260	0.00214	0.00214	0.00244	0.00608	0.00628	0.00696
LU	0.00239	0.00108	0.00108	0.00219	0.00007	0.00013	0.00175
LV	0.00269	0.00264	0.00264	0.00251	0.00145	0.00109	0.00233
ME	0.00308	0.00291	0.00291	0.00262	0.00422	0.00449	0.00353
MK	0.00271	0.00220	0.00220	0.00195	0.00163	0.00195	0.00323
NI	0.00202	0.00325	0.00325	0.00256	0.00000	0.00000	0.00002
NL	0.00265	0.00216	0.00216	0.00245	0.00440	0.00426	0.00575
NO	0.00271	0.00198	0.00198	0.00252	0.00219	0.00208	0.00005
PL	0.00259	0.00221	0.00221	0.00261	0.00052	0.00049	0.00033
PT	0.00256	0.00273	0.00273	0.00217	0.00000	0.00003	0.00000
RO	0.00232	0.00316	0.00316	0.00211	0.00000	0.00008	0.00039
RS	0.00236	0.00062	0.00062	0.00127	0.00365	0.00528	0.00314
SE	0.00188	0.00294	0.00294	0.00268	0.00000	0.00000	0.00003
SI	0.00260	0.00263	0.00263	0.00240	0.00854	0.00693	0.00620
SK	0.00286	0.00249	0.00249	0.00245	0.01066	0.01055	0.01026

, employed the Wasserstein distance, the Kolmogorov–Smirnov test, and the runs test to assess r(p_m,j, X). These methodologies successfully identified the true model, demonstrating robustness in both synthetic and real data scenarios. This is further elaborated in Section 4.2. These methods were demonstrated to be particularly effective for stationary time series within synthetic models. The consistent performance observed in the ENTSO-E dataset was likely attributable to the inherent periodic nature of power data, which remained stable over extended periods. Furthermore, distinctive features of each candidate model in real data scenarios enhanced the clarity of comparative results, as evidenced by detailed evaluations presented in Table 9 and Table 10.

4.4. Discussion

The objective of this study was to assess the efficacy of the L₂ norm, a commonly utilized metric in time series analysis, in differentiating between models exhibiting variation solely in their parameter values. Table 9 and Table 10 present a comprehensive comparison of alternative distance metrics in both synthetic and real-world energy data scenarios, emphasizing their reliability. However, the results indicate that while certain methods, such as the Wasserstein distance and the Kolmogorov–Smirnov test, demonstrated effectiveness in specific contexts, the reliability of the L₂ norm as a metric for model comparison was called into question.

The experiment yielded stationary time series data across a range of autoregressive (AR), moving average (MA), and ARMA models with varying parameters but comparable structures. The utilization of stationary time series data facilitated a direct evaluation of the efficacy of the L₂ norm in differentiating between models exhibiting variation solely in their parameter values. In each instance, models were generated with specific parameter sets, and the L₂ norm of the discrepancies between the observed and split datasets was calculated. It was postulated that the L₂ norm would consistently identify the model that generated the observed data, particularly when m = 3.

In this experiment, the following types of data were generated: A total of 30 models were generated by setting five parameters for each of the six types. The only varying factor was variance.

• Type A: AR(1), y_m,t = φ₀ + φ_m,1y_t−1 + ε_t;
• Type B: AR(2), y_m,t = φ₀ + φ_m,1y_t−1 + φ_m,2y_t−2 + ε_t;
• Type C: MA(1), y_m,t = θ₀ + θ_m,1y_t−1 + ε_t;
• Type D: MA(2), y_m,t = θ₀ + θ_m,1y_t−1 + θ₀ + θ_m,2y_t−2 + ε_t;
• Type E: ARMA(1), y_m,t = φ₀ + φ_m,1y_t−1 + θ_m,1y_t−1 + ε_t;
• Type F: ARMA(2), y_m,t = φ₀ + φ_m,1y_t−1 + φ₀ + φ_m,2y_t−2 + θ_m,1y_t−1 + θ_m,2y_t−2 + ε_t
  where m = 1, …, 5, ε_t ~ N(0, 1²), t = 1, …, 3600;
• φ_1,1 = 0.1, φ_2,1 = 0.3, φ_3,1 = 0.5, φ_4,1 = 0.7, φ_5,1 = 0.9;
• φ_1,2 = −0.1, φ_2,2 = −0.3, φ_3,2 = −0.5, φ_4,2 = −0.7, φ_5,2 = −0.9;
• θ_1,1 = 0.1, θ_2,1 = 0.3, θ_3,1 = 0.5, θ_4,1 = 0.7, θ_5,1 = 0.9;
• θ_1,2 = −0.1, θ_2,2 = −0.3, θ_3,2 = −0.5, θ_4,2 = −0.7, θ_5,2 = −0.9.

For each type, data of length 100 were generated from the model with m = 3, and t = 1, …, 100 was considered as the observed data.

y_{o,t_o} = {y_{m,t_o}}, t_o = 1, …, 100.

(6)

Then, split data of length 100 were generated from the data corresponding to t = 101, …, 3600. In this way, 3400 split datasets were generated for each model. The L₂ norm of the difference between y_{o,t_o} and each split dataset was then computed. Figure 10 exhibits the process of generating split datasets. The red vertical line separates the observed data y_{o,t_o} on the left from the portion of the data used to create the split datasets on the right.

After discussing the challenges of using the L₂ norm as a distance measure, we provide further empirical results in

Table 11. Summary of L₂ norm results across different model types and parameter settings.

Type 1	m = 1	m = 2	m = 3	m = 4	m = 5	Type 2	m = 1	m = 2	m = 3	m = 4	m = 5
Min.	0.000	0.000	0.000	0.000	0.000	Min.	0.000	0.000	0.000	0.000	0.000
1st Qu.	0.491	0.500	0.524	0.731	1.324	1st Qu.	0.509	0.522	0.554	0.801	1.492
Median	1.040	1.058	1.109	1.548	2.803	Median	1.077	1.106	1.174	1.696	3.155
Mean	1.230	1.251	1.312	1.831	3.314	Mean	1.277	1.311	1.390	2.005	3.732
3rd Qu.	1.772	1.803	1.891	2.640	4.779	3rd Qu.	1.839	1.888	2.003	2.890	5.378
Max.	8.571	8.597	9.387	13.480	20.753	Max.	9.176	9.133	9.944	14.549	24.916
Std. Dev.	0.929	0.946	0.991	1.384	2.502	Std. Dev.	0.968	0.993	1.053	1.513	2.820
Type 3	m = 1	m = 2	m = 3	m = 4	m = 5	Type 4	m = 1	m = 2	m = 3	m = 4	m = 5
Min.	0.000	0.000	0.000	0.000	0.000	Min.	0.000	0.000	0.000	0.000	0.000
1st Qu.	0.481	0.489	0.505	0.657	0.826	1st Qu.	0.507	0.522	0.553	0.746	0.979
Median	1.017	1.034	1.070	1.391	1.749	Median	1.074	1.106	1.170	1.579	2.071
Mean	1.204	1.224	1.266	1.645	2.067	Mean	1.269	1.308	1.384	1.867	2.447
3rd Qu.	1.735	1.764	1.825	2.371	2.981	3rd Qu.	1.830	1.885	1.994	2.692	3.529
Max.	9.360	8.398	8.887	11.317	14.086	Max.	9.510	8.909	9.651	13.414	17.055
Std. Dev.	0.910	0.925	0.957	1.241	1.559	Std. Dev.	0.959	0.988	1.046	1.410	1.846
Type 5	m = 1	m = 2	m = 3	m = 4	m = 5	Type 6	m = 1	m = 2	m = 3	m = 4	m = 5
Min.	0.000	0.000	0.000	0.000	0.000	Min.	0.000	0.000	0.000	0.000	0.000
1st Qu.	0.591	0.621	0.694	0.856	1.445	1st Qu.	0.650	0.693	0.785	0.988	1.663
Median	1.251	1.313	1.470	1.812	3.060	Median	1.376	1.465	1.661	2.092	3.518
Mean	1.482	1.555	1.740	2.145	3.619	Mean	1.627	1.734	1.965	2.472	4.161
3rd Qu.	2.134	2.240	2.507	3.092	5.219	3rd Qu.	2.344	2.498	2.832	3.564	5.995
Max.	10.234	10.696	12.267	15.043	23.159	Max.	10.674	12.484	14.564	17.811	27.340
Std. Dev.	1.123	1.177	1.316	1.622	2.732	Std. Dev.	1.230	1.310	1.486	1.866	3.144

. This table compares the L₂ norms obtained for different models under different parameter settings and clearly shows how increasing the value of m affects the magnitude of the L₂ norm. The key observation here is that even when m = 3, for which the original data were generated, the L₂ norm did not consistently show the lowest values as initially expected. This finding suggests that the L₂ norm may not be an appropriate metric for model comparison in this context, even when the experiment is repeated 100 times.

In addition, Figure 11 provides a graphical representation of the L₂ norm distributions for each model type. The histograms show the frequency of L₂ norms across 3400 generated datasets, illustrating how the norms vary across different parameter settings. The sharp peaks indicate that for certain models, there is a higher concentration of L₂ norms near the lower range, while for others, the distribution is more spread out. This variability in the distribution highlights the inconsistency of using the L₂ norm for model selection and reinforces the conclusion that alternative methods may be necessary, especially when trends or other nonstationary factors are present.

While these results highlight the limitations of the L₂ norm, alternative distance metrics such as those presented in Table 9 and Table 10, such as the Wasserstein distance, the Kol-Mogorov-Smirnov test, and various statistical measures, showed more consistent performance in identifying the true model. These alternative methods provided better robustness, especially in real-world datasets with inherent complexities such as missing data and nonstationarity. This underscores the importance of using multiple metrics for model comparison rather than relying solely on the L₂ norm.

The histograms in Figure 11 show how the L₂ norms are distributed for each model, providing further insight into the suitability of this metric for different types of time series. The visual representation complements the numerical data in Table 11 and further supports the argument that while the L₂ norm can provide some information, its overall effectiveness as a model selection criterion remains questionable.

To address the limitations of the preceding experiment, which may have involved an insufficient number of parameter combinations, an expanded experiment was conducted. The objective was to further investigate the inconsistencies observed in the L₂ norm’s performance by considering a wider range of parameter combinations. The goal was to more thoroughly evaluate the L₂ norm’s reliability as a model comparison tool. The following model configuration was considered:

y_t = φ₀ + φ₁y_t−1 + φ₂y_t−2 + θ₁ε_t−1 + θ₂ε_t−2 + ε_t,

(7)

where t = 1, …, 3600 and θ₀ = 100. The remaining parameters, φ₁, φ₂, θ₁, θ₂, and the standard deviation σ of ε_t ~ N(0, σ²), are subject to the following variations:

• p₁ = φ₁ = {0, 0.1, 0.5, 0.9};
• p₂ = φ₂ = {0, −0.1, −0.5, −0.9};
• q₁ = θ₁ = {0, 0.1, 0.5, 0.9};
• q₂ = θ₂ = {0, −0.1, −0.5, −0.9};
• σ = {0.1, 0.5, 1, 2, 3}.

The aforementioned combinations result in 1280 distinct models, which are designated as Model_{p_1,p_2,q_1,q_2,σ}. A total of 3500 data subsets of length 100 were generated for each model. Moreover, a dataset of length 100 was generated from Model_{0.5,0,0.5,0,1} and employed as the observed data. The model equation is as follows:

y_{o,t_o} = φ₀ + φ₁y_t−1 + θ₁ε_t−1 + ε_t = 100 + 0.5y_t−1 + 0.5ε_t−1 + ε_t, t_o = 1, …, 100, ε_t ~ N(0, 1).

(8)

Following this, the L₂ norm of the differences between y_{o,t_o} and each generated subset of data was computed. Since y_{o,t_o} was derived from Model_{0.5,0,0.5,0,1}, it was expected that the L₂ norm would be lowest when comparing against this same model. However, the experimental results shown in

Table 12. L₂ norm results for different model configurations.

Ordered by Min.
p1	p2	q1	q2	σ	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.	Rank of Min.
0.9	−0.1	0	0	0.5	1.4661	1.9177	2.0306	2.0297	2.1354	2.5083	1
0.9	−0.1	0.1	−0.1	0.5	1.5046	1.9368	2.0315	2.0332	2.1366	2.4637	2
0.9	−0.1	0.9	−0.1	0.5	1.5049	2.1460	2.3043	2.3051	2.4645	2.9583	3
…	…	…	…	…	…	…	…	…	…	…	…
0.5	0	0.1	−0.9	0.5	1.7347	1.9058	1.9500	1.9523	1.9960	2.1738	182
0.5	0	0.5	0	1	1.7349	2.2410	2.3845	2.3865	2.5276	3.1189	183
0.1	0	0.5	−0.1	1	1.7352	2.0675	2.1639	2.1653	2.2585	2.8228	184
…	…	…	…	…	…	…	…	…	…	…	…
0.9	−0.9	0.9	−0.9	3	8.0308	12.0483	14.1398	14.3271	16.1338	23.0627	1278
0.5	−0.9	0.5	−0.9	3	8.5648	11.2834	13.0415	13.1584	14.5845	20.1327	1279
0.1	−0.9	0.1	−0.9	3	8.7253	11.5559	12.9434	13.1067	14.6889	18.5814	1280
Ordered by 1st Qu.
p1	p2	q1	q2	σ	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.	Rank of 1st Qu.
0.9	−0.1	0.5	−0.1	0.1	1.7032	1.8142	1.8473	1.8468	1.8780	2.0185	1
0.9	−0.1	0.9	−0.5	0.1	1.7183	1.8167	1.8504	1.8508	1.8822	2.0063	2
0.9	−0.1	0.5	0	0.1	1.7151	1.8173	1.8468	1.8476	1.8781	1.9892	3
…	…	…	…	…	…	…	…	…	…	…	…
0.9	0	0.9	−0.5	0.5	1.6788	2.2398	2.4139	2.4315	2.6208	3.2917	566
0.5	0	0.5	0	1	1.7349	2.2410	2.3845	2.3865	2.5276	3.1189	567
0.5	−0.1	0	−0.9	1	1.8888	2.2413	2.3207	2.3233	2.4049	2.8344	568
…	…	…	…	…	…	…	…	…	…	…	…
0.9	−0.9	0.9	−0.5	3	7.8604	11.9901	13.7685	14.1326	15.5950	23.6238	1278
0.9	−0.9	0.9	−0.9	3	8.0308	12.0483	14.1398	14.3271	16.1338	23.0627	1279
0.5	−0.9	0.9	−0.9	3	6.8878	12.5605	14.0433	14.3611	16.3422	24.0162	1280
Ordered by Median
p1	p2	q1	q2	σ	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.	Rank of Median
0	−0.1	0.1	0	0.1	1.8016	1.8280	1.8343	1.8346	1.8413	1.8694	1
0.1	0	0	0	0.1	1.7911	1.8270	1.8343	1.8346	1.8418	1.8807	2
0	−0.1	0.1	−0.1	0.1	1.8004	1.8280	1.8344	1.8347	1.8416	1.8748	3
…	…	…	…	…	…	…	…	…	…	…	…
0	−0.5	0.9	−0.1	1	1.9180	2.2898	2.3798	2.3851	2.4772	2.8140	583
0.5	0	0.5	0	1	1.7349	2.2410	2.3845	2.3865	2.5276	3.1189	584
0.5	−0.1	0.5	−0.9	1	1.9244	2.2957	2.3894	2.3911	2.4874	2.9116	585
…	…	…	…	…	…	…	…	…	…	…	…
0.9	−0.9	0.9	−0.5	3	7.8604	11.9901	13.7685	14.1326	15.5950	23.6238	1278
0.5	−0.9	0.9	−0.9	3	6.8878	12.5605	14.0433	14.3611	16.3422	24.0162	1279
0.9	−0.9	0.9	−0.9	3	8.0308	12.0483	14.1398	14.3271	16.1338	23.0627	1280
Ordered by Mean
p1	p2	q1	q2	σ	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.	Rank of Mean
0.1	0	0	−0.1	0.1	1.8002	1.8276	1.8345	1.8345	1.8415	1.8769	1
0	−0.1	0.1	0	0.1	1.8016	1.8280	1.8343	1.8346	1.8413	1.8694	2
0	−0.1	0	0	0.1	1.8034	1.8284	1.8346	1.8346	1.8406	1.8659	3
…	…	…	…	…	…	…	…	…	…	…	…
0	−0.5	0.9	−0.1	1	1.9180	2.2898	2.3798	2.3851	2.4772	2.8140	581
0.5	0	0.5	0	1	1.7349	2.2410	2.3845	2.3865	2.5276	3.1189	582
0.5	−0.1	0.5	−0.9	1	1.9244	2.2957	2.3894	2.3911	2.4874	2.9116	583
…	…	…	…	…	…	…	…	…	…	…	…
0.9	−0.9	0.9	−0.5	3	7.8604	11.9901	13.7685	14.1326	15.5950	23.6238	1278
0.9	−0.9	0.9	−0.9	3	8.0308	12.0483	14.1398	14.3271	16.1338	23.0627	1279
0.5	−0.9	0.9	−0.9	3	6.8878	12.5605	14.0433	14.3611	16.3422	24.0162	1280
Ordered by 3rd Qu.
p1	p2	q1	q2	σ	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.	Rank of 3rd Qu.
0	−0.1	0	−0.1	0.1	1.8054	1.8288	1.8346	1.8346	1.8404	1.8645	1
0.1	−0.1	0	−0.5	0.1	1.8059	1.8297	1.8355	1.8356	1.8415	1.8657	2
0	−0.1	0	0	0.1	1.8034	1.8284	1.8346	1.8346	1.8406	1.8659	3
…	…	…	…	…	…	…	…	…	…	…	…
0	−0.5	0.1	−0.9	1	2.0827	2.4569	2.5680	2.5718	2.6840	3.1144	593
0.5	0	0.5	0	1	1.7349	2.2410	2.3845	2.3865	2.5276	3.1189	594
0.1	−0.5	0.5	−0.5	1	1.9588	2.3820	2.4820	2.4880	2.5820	3.1269	595
…	…	…	…	…	…	…	…	…	…	…	…
0.9	−0.9	0.9	−0.5	3	7.8604	11.9901	13.7685	14.1326	15.5950	23.6238	1278
0.5	−0.9	0	−0.9	3	6.8470	10.6696	11.9222	12.6854	14.4462	23.8760	1279
0.5	−0.9	0.9	−0.9	3	6.8878	12.5605	14.0433	14.3611	16.3422	24.0162	1280
Ordered by Max.
p1	p2	q1	q2	σ	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.	Rank of Max.
0.9	−0.1	0	0	0.5	1.4661	1.9177	2.0306	2.0297	2.1354	2.5083	1
0.9	−0.1	0.1	−0.1	0.5	1.5046	1.9368	2.0315	2.0332	2.1366	2.4637	2
0.9	−0.1	0.9	−0.1	0.5	1.5049	2.1460	2.3043	2.3051	2.4645	2.9583	3
…	…	…	…	…	…	…	…	…	…	…	…
0.5	0	0.1	−0.9	0.5	1.7347	1.9058	1.9500	1.9523	1.9960	2.1738	182
0.5	0	0.5	0	1	1.7349	2.2410	2.3845	2.3865	2.5276	3.1189	183
0.1	0	0.5	−0.1	1	1.7352	2.0675	2.1639	2.1653	2.2585	2.8228	184
…	…	…	…	…	…	…	…	…	…	…	…
0.9	−0.9	0.9	−0.9	3	8.0308	12.0483	14.1398	14.3271	16.1338	23.0627	1278
0.5	−0.9	0.5	−0.9	3	8.5648	11.2834	13.0415	13.1584	14.5845	20.1327	1279
0.1	−0.9	0.1	−0.9	3	8.7253	11.5559	12.9434	13.1067	14.6889	18.5814	1280

reveal that the L₂ norm did not consistently identify the correct model.

Despite comparing stationary time series that only differ in parameter values, the L₂ norm-based testing showed that the rank of the model generating the observed data, Model_{0.5,0,0.5,0,1}, appeared in the late 500s out of the 1280 models. This suggests that using the L₂ norm alone to compute the Bayes factor could result in a stronger belief in 500 other models rather than the original model that generated the observed data.

As illustrated in

Table 13. Ranking of Model_{0.5,0,0.5,0,1} across different summary statistics.

Rank of Min.	Rank of 1st Qu.	Rank of Median	Rank of Mean	Rank of 3rd Qu.	Rank of Max.
183	567	584	582	594	183

, the issue of ranking is further emphasized when different summary statistics are considered. While the model responsible for generating the observed data exhibits a high ranking in terms of minimum and maximum values, it performs poorly in other key statistics, such as the first quartile, median, and mean. This variability across summary statistics indicates that the L₂ norm may not be a sufficient criterion for robust model comparison, particularly when dealing with more complex or real-world data scenarios.

In scenarios that are more complex, such as those in which nonstationary elements like trends are introduced, the limitations of traditional distance metrics like the L₂ norm become even more pronounced. This underscores the necessity for sophisticated methodologies, such as graph neural networks (GNNs) and transfer learning. GNNs are particularly well suited to the analysis of graph-structured data, rendering them an excellent choice for complex time series such as energy networks, where interdependencies exist across multiple time points or nodes. However, the application of GNNs or transfer learning in this study presented a number of challenges, including the necessity for extensive labeled datasets and the complexity of model tuning.

In conclusion, this study demonstrates the shortcomings of relying exclusively on conventional distance metrics, such as the L₂ norm, for model comparison in time series analysis. By integrating advanced techniques, such as GNNs and transfer learning, with more reliable statistical measures, future research can overcome these limitations. This hybrid approach has the potential to enhance the precision and computational efficiency of time series models, particularly in domains where accurate predictions are crucial for decision-making, including energy management and beyond.

5. Visualization

While the process of selecting a candidate model by computing the posterior probability from observed data is crucial, as depicted in Figure 4, it is equally important to determine “at which point” the observed data resemble the candidate model. However, this comparison can be challenging to express numerically. It may not be easily understood by the user if presented in such a format. Figure 12 and Figure 13 provide a detailed visualization framework to help illustrate these concepts. Figure 12 shows the components necessary for dataset selection and model analysis, while Figure 13 focuses on the visualization of marginal likelihood. The simplest and most effective way to convey this information to users is through these visualizations, which enhance the user’s ability to understand complex statistical relationships clearly and intuitively.

In particular, when employing distance to compute p[X|Model_m], as illustrated in Equation (3), it is essential to initially ascertain whether r(p_m,j, X) ≤ SignIf._indep. with the candidate model. This can be accomplished by graphically representing the candidate model as a line graph and then highlighting sections that meet this criterion in a specific color, as depicted in Figure 13 “A”. If the method to compute p[X|Model_m] involves statistical tests rather than distance, as indicated in Equation (4), it is also necessary to determine whether p-value_trend(p_m,j) ≤ SignIf._trend. These sections can be independently highlighted, as shown in Figure 13 “B”. Consequently, by visualizing both [indep.(p_m,j, X) ≤ SignIf._indep.] and p-value_trend(p_m,j) ≤ SignIf._trend, as shown in Figure 13 “A∩B”), it becomes easier for users to identify sections of the data that are considered similar to the observed data.

The determination of an appropriate threshold for r(p_m,j, X) to judge similarity is a highly challenging endeavor due to variability observed across different data states and domains. In particular, when employing statistical tests, the conventional approach has been to reject the null hypothesis when p-values are below traditional thresholds of 0.1, 0.05, or 0.01. Nevertheless, these standards are currently the subject of controversy regarding reproducibility due to an increasing number of cases of misuse involving the manipulation of crucial parameters such as standardization, repetition of experiments, and randomization methods. In the medical field, which relies heavily on statistical analysis, there has been a recent consensus to lower the threshold for rejecting the null hypothesis to 0.001 to address these controversies. However, it is still challenging to assert that this is a comprehensive solution.

Regardless of the absolute value of r(p_m,j, X), an analytical method can be considered reproducible if results from analyzing a newly generated independent dataset in the same manner are consistent with those from previous similar analyses. It is therefore desirable for the analysis to yield reproducible results. To facilitate user adaptation to their specific data states or domains within a visualization framework, it is essential to allow users to adjust the significance level directly.

In the course of conducting experiments for this paper, the value of r(p_m,j, X) was calculated for a number of partitioned datasets. Results were then examined by adjusting the significance level based on their quantiles. In particular, in Section 4.2, when experimenting with a synthetic model, results for five different significance levels were summarized in a large table. However, rather than displaying such an extensive table, a slide bar was used to interactively adjust the significance level, as shown in Figure 14. This method is an efficient and clear method for demonstrating changes in results.

Once a candidate model with a high likelihood of encompassing the observed data is selected, the subsequent step is to proceed with a prediction. Nevertheless, selecting a candidate model does not necessarily resolve the inherent cold-start problem in predictions using observed data. Nevertheless, it is possible to directly compare the observed data with a partitioned data vector deemed similar. By displaying the observed data and the partitioned data vector on a line graph as illustrated in Figure 15 and simultaneously reviewing the graph, distance, or p-values if a statistical test was performed, users can conduct a personal assessment of the similarity of the data in question and the extent to which this similarity can be confirmed.

6. Conclusions

This study effectively addressed key research questions posed at the outset, providing valuable insights into time series analysis in cold-start scenarios with limited initial data. Through the application of Bayesian model selection, we were able to not only predict responses with higher accuracy than those achieved with traditional distance measures like the L₂ norm but also highlight the inadequacies of traditional distance measures, such as the L₂ norm, in model comparison.

• Our findings demonstrate that Bayesian model selection can significantly enhance predictive accuracy when faced with sparse data. By partitioning models and analyzing each vector with statistical tests, we bypassed the traditional reliance on distance measures. This approach was proven to be particularly beneficial in scenarios where the conventional methods would likely fail due to insufficient data.
• Another significant contribution of this study is the development of a new visualization technique that employs a slide bar for interactively setting significance levels. This method stands in stark contrast to traditional star-marked displays. It offers a dynamic tool for researchers to adjust and interpret significance with greater clarity, thus reducing potential misunderstandings about p-value implications.
• The operational aspect of employing Bayesian model selection in real-world scenarios was also explored. We found that once the observational data aligned with a candidate model, effective predictions could be made using the model. However, this is contingent upon assumptions of stationarity and the absence of structural breaks, which our research identified as areas requiring further investigation to fully harness the potential of using Bayesian methods for solving cold-start problems.

By systematically addressing these research questions, our study not only advances the field of time series analysis but also equips practitioners with more robust tools for handling the complexities of cold-start scenarios. The insights gained underscore the importance of innovative statistical approaches and the need for the continual refinement of analytical tools to adapt to evolving data challenges.

Despite advances presented in this study, certain limitations must be acknowledged, along with potential directions for future research that could further refine and expand our findings. The effectiveness of Bayesian model selection, as demonstrated in this study, heavily relies on the quality and granularity of the observational data. In scenarios where data are excessively sparse or noisy, the reliability of model alignment and subsequent predictions could be compromised. Our approach assumes stationarity and the absence of structural breaks within the dataset. These assumptions may not hold true in all practical applications, potentially affecting the robustness of predictions. Implementation of the proposed visualization tool and statistical tests requires a certain level of statistical and computational expertise, which may not be readily available in all research or applied settings.

Future studies could explore the application of Bayesian model selection across more diverse datasets, including those with higher levels of noise and nonstationarity. This could help us understand the limits and scalability of our approach. Developing algorithms that can automatically detect and adjust for structural breaks and nonstationarity within the data could significantly enhance the applicability and accuracy of Bayesian model selection in real-world scenarios. There is a need to develop more intuitive and accessible tools that can bring the power of advanced statistical methods to a wider audience. Simplifying the implementation of our visualization technique could facilitate its adoption in nonspecialist contexts, enhancing its practical utility. By addressing these limitations and exploring these avenues for future research, the field can move towards more generalized and robust methods for dealing with cold-start problems and time series analysis under challenging conditions.