Enhancing Portfolio Allocation: A Random Matrix Theory Perspective

Abstract

This paper explores the application of Random Matrix Theory (RMT) as a methodological enhancement for portfolio selection within financial markets. Traditional approaches to portfolio optimization often rely on historical estimates of correlation matrices, which are particularly susceptible to instabilities. To address this challenge, we combine a data preprocessing technique based on the Hilbert transformation of returns with RMT to refine the accuracy and robustness of correlation matrix estimation. By comparing empirical correlations with those generated through RMT, we reveal non-random properties and uncover underlying relationships within financial data. We then utilize this methodology to construct the correlation network dependence structure used in portfolio optimization. The empirical analysis presented in this paper validates the effectiveness of RMT in enhancing portfolio diversification and risk management strategies. This research contributes by offering investors and portfolio managers with methodological insights to construct portfolios that are more stable, robust, and diversified. At the same time, it advances our comprehension of the intricate statistical principles underlying multivariate financial data.

1. Introduction

Markowitz’s seminal paper [1] remains a cornerstone in practical portfolio management, despite facing challenges in out-of-sample analyses due to estimation errors in expected returns and the covariance matrix, as noted in [2,3,4,5,6]. To address these limitations, researchers have investigated the use of robust estimators aimed at providing more accurate and reliable measures of moments and co-moments. Robust estimators, designed to handle outliers and errors, enhance the robustness of portfolio optimization, leading to improved stability and performance, see [7,8,9].

In recent years, there has been a notable shift towards a novel approach known as the market graph for understanding the relationship between asset returns, departing from traditional covariance-based methods. This methodology represents assets as nodes and their relationships as edges in a graph, capturing the broader structure of the financial market. Pioneering work by Mantegna [10] led to the construction of asset graphs using stock price correlations, revealing hierarchical organization within stock markets. Subsequent research conducted by [11] investigates stock network topology by examining return relationships, with the objective of revealing significant patterns inherent in the correlation matrix. Additionally, ref. [12] introduced constraints based on asset connections in the correlation graph to encourage the inclusion of less correlated assets for diversification, employing assortativity from complex network analysis to guide the optimization process. Recent research explores innovative portfolio optimization techniques using clustering information from asset networks [13,14,15]. These approaches replace the classical Markowitz model’s correlation matrix with a correlation-based clustering matrix. In [15], a network-based approach is utilized to model the structure of the financial market and the interdependencies among asset returns. This is achieved through the construction of a market graph, which captures asset connections using various edge weights. An objective function is introduced, considering both individual security volatility and network connections, to enhance the understanding of market dynamics. In [13], the authors proposed a unified mixed-integer linear programming (MILP) framework integrating clustering and portfolio optimization. Empirical results demonstrate the promising performance of the network-based portfolio selection approach, outperforming classical approaches relying solely on pairwise correlation between assets’ returns. Building upon the research outlined in [14,15], we introduce a novel framework that integrates Random Matrix Theory (RMT) into portfolio allocation procedures. Random Matrix Theory (RMT) is a mathematical approach introduced in [16] which is used to analyze correlations in the finance area and especially to improve portfolio management [17,18,19,20]. Specifically, we employ a combination of the Hilbert transformation and RMT to estimate the correlation matrix. This methodological approach draws inspiration from a dimensionality reduction technique introduced in [21] for data analysis, which was subsequently extended to economic time series and quantitative finance [22,23,24]. By employing this technique, we effectively separate noise from significant correlations, thereby enhancing the accuracy and robustness of the portfolio allocation process. On the one hand, the Hilbert transformation technique is a data preprocessing method which we use to add complexity the time series of returns, thus improving the effectiveness in detecting lead/lag correlations in time-series data, as shown in [23,25,26]. On the other hand, RMT offers a robust denoising technique on the basis of the Marčenko–Pastur theorem, which describes the eigenvalue distribution of the covariance matrices [27,28,29]. As a result, we obtain a denoised version of the covariance matrix that captures the true underlying correlation structure more accurately and facilitates the computation of clustering coefficients to assess the interconnectedness of assets.

The proposed method is then tested on four equity portfolios, employing a buy and hold rolling window strategy with various lengths for the in-sample and the out-of-sample periods. The out-of-sample performances are then compared across different methods of estimating the dependence structure between assets, including sample estimates, shrinkage estimates, clustering coefficients, and RMT. Notably, employing the RMT method consistently results in a superior out-of-sample performance compared to traditional methods across all portfolios and rolling window strategies. Furthermore, the analysis of transaction costs, measured by turnover, reveals that the RMT approach consistently led to a lower portfolio turnover compared to other methods. This reduction in turnover translates to lower transaction costs and underscores the practical applicability of the RMT approach in portfolio management.

The paper is organized as follows: Section 2 introduces the basic notions of the Hilbert transformation and RMT-based techniques. Section 3 offers a brief overview of existing portfolio selection models. Section 4 details the in-sample and out-of-sample protocols and analyzes the empirical outcomes. Finally, Section 5 discusses this paper’s key findings.

2. Random Matrix Theory

In the context of financial mathematics, it has been established that extensive empirical correlation matrices exhibit significant noise to the point where, with the exception of their principal eigenvalues and associated eigenvectors, they can be essentially treated as stochastic or random entities. Therefore, it is customary to denoise an empirical correlation matrix before its utilization. In this section, we will elucidate a denoising technique rooted in the principles of RMT where the fundamental result is the Marčenko–Pastur theorem, which describes the eigenvalue distribution of large random covariance matrices.

In the following, we denote by $\mathcal{M}\left({\mathbb{C}}^{n\times T}\right)$ the space of complex-valued matrices with dimensions $n\times T$. Let $X\in \mathcal{M}\left({\mathbb{C}}^{n\times T}\right)$ be an observation matrix, where the columns are the single observations for different n time series. Let $\Sigma \in \mathcal{M}\left({\mathbb{C}}^{n\times n}\right)$ be the empirical covariance matrix associated with X, i.e., $\Sigma =\frac{1}{T}X{X}^{*}$ ($X^{*}$ is the adjoint matrix of X). If $\Sigma$ is Hermitian (that is, if the eigenvalues are real), we can define the empirical spectral distribution function as $F=n^{-1}{\sum }_{j=1}^n{\mathbf{1}}_{{\lambda }_j\le \lambda }$, where ${\lambda }_j$, for $j=1,\cdots ,n$, are the eigenvalues of $\Sigma$. The distribution of the eigenvalues of a large random covariance matrix is actually universal, meaning that it follows a distribution independent of the underlying observation matrix as stated in the following Marčenko–Pastur theorem [27,28,29].

Theorem 1 

(Marčenko–Pastur, 1967 [27]). Suppose $X_i,i=1,\cdots ,n$ are independent and identically distributed random variables with mean 0 and variance ${\sigma }^2$. Suppose that $\frac{T}{n}\to q\in (0,+\infty )$ as $n\to +\infty ,T\to +\infty$. Then, as $n\to +\infty$, the empirical spectral distribution $\widehat{F}$ of the sample covariance $\widehat{\Sigma }$ converges almost surely in distribution to a nonrandom distribution, known as the Marčenko–Pastur law and denoted by $F_q$, whose probability distribution is:

$${\rho }_q\left(\lambda \right) d\lambda =\max \left(1-\frac{1}{q},0\right){\delta }_0+\frac{\sqrt{({\lambda }_{+}-\lambda )(\lambda -{\lambda }_{-})}}{2\pi q\lambda {\sigma }^2} {\mathbf{1}}_{{\lambda }_{-}\le \lambda \le {\lambda }_{+}} d\lambda$$

(1)

where the first term represents a point mass with weight $1-\frac{1}{q}$ at the origin if $q>1$ and ${\lambda }_{\pm }={\sigma }^2{(1\pm \sqrt{q})}^2$ coincide with the lower and the upper edges of the eigenvalue spectrum.

Let us note that, under proper assumptions, the Marčenko–Pastur theorem remains valid for observations drawn from more general distributions, like fat-tailed distributions [18,30].

Utilizing the boundaries of the Marčenko–Pastur distribution within the eigenvalue spectrum facilitates distinguishing between information and noise, enabling the filtration of the sample correlation matrix. Consequently, it is expected that this filtered correlation matrix will provide more stable correlations compared to the standard sample correlation matrix.

2.1. Filtering Covariance by RMT

One can compare the theoretical density of the eigenvalues generated the by the Marčenko–Pastur theorem (in the following indicated as benchmark or null model) with the corresponding empirical one. Thus, we can identify the number of empirical eigenvalues, possessing some known and easily interpretable characteristics, that significantly deviate from the null model. There are many methods for filtering the correlation matrix based on RMT as proposed previously. Essentially, the approaches primarily involve suitable modification of the eigenvalues within the spectrum of the sample correlation matrix while simultaneously preserving its trace, as explained below.

The upper boundary ${\lambda }_{+}$ of the Marčenko–Pastur density serves as a threshold for distinguishing the noisy component of $\Sigma$. Eigenvalues of $\Sigma$ within the interval $[{\lambda }_{-},{\lambda }_{+}]$ adhere to the random correlation matrix hypothesis and represent noise-associated eigenvalues, as well as those below ${\lambda }_{-}$. The rationale behind excluding the smallest deviating eigenvalues from these filters lies in the fact that, unlike large eigenvalues—which are separated from the Marčenko–Pastur (MP) upper bound—the same rationale does not always hold true for the smallest deviating eigenvalues. Typically, small eigenvalues can be situated beyond the lower edge of the spectrum, a pattern consistent with the finite dimension of the observed matrix $(n\times T)$. Furthermore, as emphasized in [17,31] there exists clear evidence of the non-randomness and temporal stability of the eigenvectors corresponding to eigenvalues larger than ${\lambda }_{+}$. Such characteristics have not been consistently confirmed for the eigenvectors associated with eigenvalues smaller than ${\lambda }_{-}$. Therefore, eigenvalues higher than ${\lambda }_{+}$ can be regarded as “meaningful”. This leads to the following denoising method for the estimated covariance matrix $\widehat{\Sigma }$: all the eigenvalues of $\widehat{\Sigma }$ lower than or equal to ${\lambda }_{+}$ are replaced by a constant value either equal to the average value of the “noisy” eigenvalues (as in [17]) or equal to zero (as in [19]) (both methods have the advantage of preserving the trace of $\widehat{\Sigma }$ while eliminating the non-meaningful eigenvalues). All the eigenvalues of $\widehat{\Sigma }$ strictly greater than ${\lambda }_{+}$ are left unchanged.

In this paper, the eigenvalues of $\widehat{\Sigma }$ that fall in the range predicted by the Marčenko–Pastur distribution are set to zero. Then, to build the filtered covariance matrix ${\Sigma }^F$, the following steps are undertaken: suppose there exist k relevant eigenvalues (i.e., higher than ${\lambda }^{+}$). Let us denote them by ${\left\{{\lambda }_j\right\}}_{j=n-k,\dots ,n}$, and also consider the diagonal matrix of filtered eigenvalues ${\Lambda }^F=\mathrm{diag}(0,\dots ,0,{\lambda }_{n-k},\dots ,{\lambda }_n)$. At this point, we use ${\Lambda }^F$ back in the eigendecomposition with the original eigenvectors, here denoted by V, thus obtaining:

$${\Sigma }^F=V{\Lambda }^F{V}^{-1}.$$

(2)

Finally, in order to preserve the trace of the original matrix and prevent system distortion, we adjust the main diagonal elements of the filtered matrix ${\Sigma }^F$ by replacing them with the sample variances of the portfolio components ${\widehat{\sigma }}_{ii}^2,i=1,\cdots ,n$.

Remark 1. 

When applying this method to portfolio selection, we will work with the correlation matrix. However, the Marčenko–Pastur law remains unchanged with the only difference that ${\sigma }^2=1$.

2.2. Data Preprocessing: Hilbert Transformation

Data preprocessing is a crucial step in the data analysis process, involving techniques designed to transform raw data into a clean dataset suitable for further analysis. One powerful method used particularly in signal processing is the Hilbert transform. This mathematical transformation is mainly applied in climatology, signal processing, finance, and economics [23,25,26] and it helps to derive the analytic signal from a real-valued signal, which is crucial for applications like amplitude modulation and phase shifting. This technique is especially well suited for detecting correlations with lead/lags, making it highly effective for time series exhibiting shifts in their co-movements.

Let us consider the panel of asset returns (i.e., portfolio) over time as $\mathit{x}\left(t\right)$ and let us indicate with $\tilde{\mathit{x}}\left(t\right)$ the complex series derived by Hilbert transforming the original time series as $\tilde{\mathit{x}}\left(t\right)=\mathit{x}\left(t\right)+i\mathcal{H}\left[\mathit{x}\left(t\right)\right]$ so that the real part of each component $\mathcal{R}e\left(\tilde{\mathit{x}}\left(t\right)\right)=\mathit{x}\left(t\right)$ coincides with the original time series. The Hilbert transformation is a linear transformation defined in [32] as the following integral:

$$\begin{array}{c}\hfill \mathcal{H}\left[\mathit{x}\left(t\right)\right]=\frac{1}{\pi }\mathrm{P}\underset{-\infty }{\overset{\infty }{\int }}\frac{\mathit{x}\left(\tau \right)}{\tau -t}\mathrm{d}\tau ,\end{array}$$

where P indicates the Cauchy principal value. The sequence $\tilde{\mathit{x}}\left(t\right)$ with added complexity can then be written as:

$$\tilde{\mathit{x}}\left(t\right)=\frac{i}{\pi }\oint \frac{\mathit{x}\left(\tau \right)}{\tau -t}d\tau .$$

(3)

As an example of the effect of the Hilbert transformation on the correlation matrix, let us consider two periodic signals defined as:

$$\begin{array}{cc}\hfill x& =sin\left(\frac{\pi t}{4}\right)\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill y& =cos\left(\frac{\pi t}{4}\right).\hfill \end{array}$$

(5)

The correlation matrix between the two time series is:

$$C=\left(\begin{array}{cc}1& -5.86·{10}^{-4}\\ -5.86·{10}^{-4}& 1\end{array}\right)$$

(6)

from which it is clear how negligible the correlation between the two time series is. Meanwhile, after Hilbert transformations of the time series, the two sequences with added complexity have the following complex correlation matrix:

$$\tilde{C}=\left(\begin{array}{cc}1+i0& -5.86·{10}^{-4}+i0.99\\ -5.86·{10}^{-4}+i0.99& 1-i0\end{array}\right)=\left(\begin{array}{cc}1& 0.99e^{i1.57\pi }\\ 0.99e^{-i1.57\pi }& 1\end{array}\right).$$

(7)

We note from (7) that the magnitude of the correlation is pretty high, as expected if one considers the lags to be just a lack of synchronicity and not a lack of co-movements. The behavior of the two correlation matrices is also evident if one plots the two time series in a real plane and in a Gauss plane, see Figure 1.

If we use the data sequence with added complexity, then the Marčenko–Pastur law has to be modified by replacing p with $2p$, since the imaginary part of a data sequence with added complexity is not independent of its real part, being related by the Hilbert transformation [23].

Utilizing the Hilbert transformation, the covariance (cross-correlation) matrix is represented in a space where the eigenvectors’ components are spread across the complex plane. This facilitates identifying lead–lag relationships between components through angular disparities. The components of the complex covariance matrix can be denoted as follows:

$${\sigma }_{kj}=\mathcal{R}e\left({\sigma }_{kj}\right)+i \mathcal{I}m\left({\sigma }_{kj}\right)=\left|{\sigma }_{kj}\right|e^{i{\varphi }_{kj}},k,j=1,\cdots ,n,$$

where the absolute value of each element of the complex covariance matrix gives the strength of the correlation; meanwhile, ${\varphi }_{kj},k,j=1,\cdots ,n,$ indicates the correlation in phase space. The leading or lagging behavior of each component is determined by ${\varphi }_{kj}$, measuring to what extent the time series k leads the time series j.

We can break down the covariance (correlation) matrix as:

$$\begin{array}{cc}\hfill \Sigma =& V\Lambda V^{-1}=V({\Lambda }_{noise}+{\Lambda }^F)V^{-1}\hfill \\ \hfill =& {\Sigma }_{noise}+{\Sigma }^F\hfill \end{array}$$

where ${\Lambda }_{noise}$ is the diagonal matrix of insignificant eigenvalues and ${\Sigma }^F$ and ${\Sigma }_{noise}$ denote the principal and noisy components of the correlation matrix, respectively.

In the context of portfolio allocation, the Hilbert transformation technique proves valuable for identifying lead/lag correlations in the performance of different assets over time. This allows investors to uncover temporal patterns and optimize portfolio strategies by understanding the timing of movements in asset values. Essentially, it helps to discern how changes in one asset’s value relate to shifts in another asset, leading to more informed and dynamic portfolio decision making.

3. Portfolio Selection Models

In this section, we explore the nuances of asset allocation problems. We begin by outlining the classical portfolio model, inspired by Markowitz’s foundational work, which employs the variance/covariance matrix to analyze the interdependencies among asset returns, typically estimated through a sample approach. Following this, we describe the approach rooted in network theory (see [15]), offering a novel perspective on portfolio optimization. As also observed in recent studies [22,23] across various fields of the economic literature, this methodological approach presents a modern and sophisticated way to address challenges in traditional portfolio selection processes. It aims to unveil an optimal solution by using the interconnected relationships among different assets. Finally, we explore the application of RMT in the portfolio selection model, focusing particularly on the minimum variance portfolio.

3.1. Traditional Global Minimum Variance Portfolio

The classical global minimum variance (GMV) approach is designed to optimize portfolio allocation by determining the fractions $w_i$ of a given capital to be invested in each asset i from a predetermined basket of assets. The objective is to minimize the portfolio risk, identified through its variance. In this context, n represents the number of available assets, $X_i,i=1,\cdots ,n$ symbolizes the random variable of daily returns of the i-th asset and $\Sigma$ is the sample covariance matrix. The GMV strategy is formulated as follows:

$$\left\{\begin{array}{c}\underset{\mathit{w}}{min}{\mathit{w}}^{\top }\Sigma \mathit{w}\hfill \\ {\mathit{w}}^{\top }\mathbf{e}=1\hfill \\ 0\le w_i\le 1,\hfill & i=1,\cdots ,n,\hfill \end{array}\right.$$

(8)

where $\mathbf{e}$ is a vector of ones of length n and $\mathit{w}={\left[w_i\right]}_{i=1,\dots ,n}$ is the vector of the fractions invested in each asset. The first equation is the budget constraint and requires that the whole capital should be invested. Conditions $0\le w_i\le 1$, $i=1,\cdots ,n$ preclude the possibility of short selling (as is well known, a closed-form solution of the GMV problem exists if short selling is allowed). It is evident that the dependence structure between assets, in [1], is assessed through the Pearson correlation coefficient between each pair of asset.

The process of estimating covariance matrices from samples is widely recognized for its susceptibility to noise and its high estimation error. This introduces uncertainty and instability when attempting to estimate the true covariance structure, which frequently results in a suboptimal performance of portfolios when applied out of sample. Portfolios based on unreliable covariance structures may exhibit increased volatility, high risk, and compromised returns. Therefore, ensuring precise and stable covariance estimates is crucial for enhancing the robustness and effectiveness of portfolio optimization models in real-world applications.

3.2. Asset Allocations through Network-Based Clustering Coefficients

To address the challenges of estimating the dependence structure and improve the reliability of covariance matrix estimation, network theory has recently been applied in portfolio allocation models. Here, we employ network theory to enhance the reliability of covariance matrix estimation in portfolio allocation. This approach involves representing financial assets through an undirected graph $G=(V,E)$, where V is the set of nodes representing assets and E is the set of edges denoting dependencies or connections between assets. The weights of the edges are represented by the Pearson correlation matrix ${\rho }_{ij}$, $i,j=1,\cdots ,n$.

At this point, problem (8) is transformed into the following:

$$\left\{\begin{array}{c}\underset{\mathit{w}}{min}{\mathit{w}}^{\top }H\mathit{w}\hfill \\ {\mathbf{e}}^{\top }\mathit{w}=1\hfill \\ 0\le w_i\le 1,\hfill & i=1,\cdots ,n.\hfill \end{array}\right.$$

(9)

where H is a matrix obtained as described below. First, we create a binary undirected adjacency matrix A, associated with the filtered (complex) correlation matrix obtained through the techniques explained in Section 2. The binarization process (binarization is often used to simplify the analysis of networks by reducing them to binary representations; after applying the binarization process to the entire adjacency matrix, it is transformed into a binary adjacency matrix where each entry is either 0 or 1, representing the absence or presence of an edge, respectively, based on the chosen threshold) is assessed with the following threshold criteria: for every $i,j=1,\cdots ,n$

$$A_{ij}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{\rho }_{ij}^F\ge {\rho }_{ij},\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(10)

where ${\rho }_{ij}^F$ are the elements of the filtered correlation matrix.

Second, we calculate for each node i the clustering coefficient $c_i$ as proposed in [33] for binary and weighted graphs.

Third, we construct the interconnectedness matrix C, whose elements are:

$$C_{ij}=\left\{\begin{array}{cc}{c}_i{c}_j\hfill & \mathrm{if}i\ne j\hfill \\ 1\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(11)

Finally, we build $H={\Delta }^{\top }C\Delta$ to be the dependency matrix in (9), where $\Delta =\mathrm{diag}\left(s_i\right)$ is the diagonal matrix with entries $s_i=\frac{{\sigma }_i}{\sqrt{{\sum }_{i=1}^n{\sigma }_i^2}}$ as in [15]. This gives the new objective function to be minimized in order to obtain the optimum allocation.

4. Empirical Protocol and Performance Analysis

In this section, we delineate the empirical protocol employed in this paper and subsequently apply it to assess the effectiveness of the proposed approach through empirical applications. To test the robustness and mitigate data mining bias, we analyze four energy portfolios, as will be explained in Section 4.3.

In portfolio allocation models, in-sample and out-of-sample analyses are crucial components for assessing the performance and robustness of the strategies. The analysis of portfolio strategies centers around key criteria, including portfolio diversification, transaction costs, and risk–return performance measures. This examination is conducted using a rolling window methodology characterized by an in-sample period of length l and an out-of-sample period of length m.

This methodology begins by computing optimal weights during the initial in-sample window spanning from time $t=1$ to $t=l$. These optimal weights remain unchanged during the subsequent out-of-sample period, extending from $t=l+1$ to $t=l+m$. Then, the returns and performance metrics for this out-of-sample period are calculated. This process iterates by advancing both the in-sample and out-of-sample periods by m steps. With each iteration, the weights are recalculated and the portfolio performance is evaluated until the end of the dataset.

4.1. Diversification and Transaction Costs: In-Sample Analysis

We assess the performance of the obtained portfolios by analyzing diversification and transaction costs, both of which play a crucial role in effective portfolio management. Diversification represents a fundamental risk management tool that seeks to optimize returns while minimizing exposure to undue risks associated with concentrated holdings. This is essential to mitigate risk and enhance the stability of an investment portfolio. By holding a diversified set of assets with potentially uncorrelated returns, investors aim to achieve a balance between risk and return. In this paper, as a measure of diversification, we use the modified Herfindahl index, defined as:

$$\mathrm{HI}=\frac{{\left({\mathit{w}}^{★}\right)}^{\top }{\mathit{w}}^{★}-\frac{1}{n}}{1-\frac{1}{n}},$$

where ${\mathit{w}}^{★}=\left[w_i^{★}\right],i=1,\dots ,n$ represents the vector of optimal weights. The index ranges from 0 to 1, where a value 0 is reached in the case of the EW portfolio (deemed as the most diversified one) and a value of 1 indicates a portfolio concentrated on only one asset.

The second index used in this analysis is related to transaction costs which have a direct impact on portfolio performances: higher transaction costs lead to lower returns. Understanding and minimizing transaction costs is an integral component of effective portfolio management, ensuring that investment decisions are cost-efficient and align with the investor’s goals. By addressing these aspects, investors can enhance the overall competitiveness and sustainability of their investment strategies. In this paper, as a proxy for transaction costs, we use the portfolio turnover, denoted as $\varphi$ and computed as:

$$\varphi =\sum _{i=1}^n\left|w_i^{★} -{ }^{-}{w}_i^{★}\right|,$$

where $w_i^{★}$ and ${ }^{-}{w}_i^{★}$ are the optimal portfolio weights of the ith asset, respectively, before and after rebalancing (based on the optimization strategy). A higher turnover often comes with increased transaction costs. Monitoring turnover is crucial for understanding the level of trading activity and its impact on the portfolio’s performance. In particular, a lower turnover percentage indicates that the portfolio has experienced relatively minimal changes in asset weights. This might be indicative of a more stable managed portfolio.

4.2. Performance Measurements: Out-of-Sample Analysis

Analyzing risk-adjusted performance measures for different allocation methods is essential as it evaluates the portfolio’s success in generating returns relative to the level of risk assumed. By examining metrics like the Sharpe ratio and omega ratio, investors gain valuable insights into how effectively a portfolio balances risk and return. This analysis assists investors to select the best approach that delivers favorable returns while aligning with their risk tolerance.

Here, we briefly recap the definitions of the risk-adjusted performance measures used in the empirical part of this paper to compare the different models under analysis. The Sharpe ratio ($\mathrm{SR}$) is defined as:

$$\mathrm{SR}=\frac{\mathbb{E}\left(r_p-r_f\right)}{\sqrt{\mathbb{V}\mathrm{ar}(r_p-r_f)}},$$

where $r_p$ indicates the out-of-sample portfolio returns (obtained using the rolling window methodology) and $r_f$ denotes the risk-free rate (in our empirical analysis, we adopt a constant risk-free rate, set to zero, following the approach used in [9]). This ratio indicates the mean excess return per unit of overall risk. The portfolio with the highest $\mathrm{SR}$ is typically considered the best, especially in cases of positive portfolio excess returns concerning the risk-free rate.

The omega ratio $\left(\mathrm{OR}\right)$, introduced in [34], is defined as:

$$\mathrm{OR}=\frac{\mathbb{E}\left[{\left(r_p-ϵ\right)}^{+}\right]}{\mathbb{E}\left[{\left(ϵ-r_p\right)}^{+}\right]},$$

where $ϵ$ is a specified threshold. Returns above the threshold are considered gains by investors, while those below are considered as losses. An $\mathrm{OR}$ greater than one suggests that the portfolio provides more expected gains than expected losses. The choice of threshold $ϵ$ can vary, and in our empirical analysis, it is set to 0.

4.3. Data Description and Empirical Results

In this empirical analysis, we examine four equity indexes, sourcing our data from Bloomberg. The selected indexes are as follows: (i) the Nasdaq 100 index; (ii) the energy sector of the S&P500 index, which focuses only on energy companies; (iii) the Dow Jones Industrial Average index; and (iv) the benchmark stock market index of the Bolsa de Madrid. We construct each portfolio by including only components listed in the specified index from 2 January 2014 to 19 January 2024. Detailed information regarding the Bloomberg tickers of the selected assets in each portfolio can be found in Appendix A. The dataset for each portfolio is composed of 2622 daily observations spanning from 2 January 2014 to 19 January 2024.

As detailed in Section 3, we address optimization problems (8) and (9), both of which account for the dependence structure in constructing the optimal portfolio. In problem (8), this structure is captured by the covariance matrix $\Sigma$, which is estimated using both the sample and the shrinkage approaches. Conversely, in (9), the dependence structure is represented by the matrix H, leveraging the clustering coefficient method as outlined in [15] and along with the approach proposed in Section 2 (utilizing the Hilbert transformation and RMT).

To evaluate the performances of the approaches considered in this paper, we employ four distinct buy-and-hold rolling window methodologies: the first three are 6, 12 or 24 months in-sample and 1 month out-of-sample; the last one is 24 months in-sample and 2 months out-of-sample.

After obtaining the out-of-sample returns for each approach under analysis, we then proceed to compute their respective out-of-sample performances. Figure 2 illustrates the out-of-sample performances of the four portfolios under analysis, specifically for the rolling window methodology of 6 months in-sample and 1 month out-of-sample. (Please note that, due to space limitations, we have omitted the out-of-sample performances for the other rolling window methodologies. Nevertheless, this information is readily available upon request to the corresponding author.)

Observing Figure 2, it becomes clear that across all portfolios under analysis, and in the case of the rolling window methodology with 6 months in-sample and 1 month out-of-sample, the CC-RMT strategy exhibits a superior out-of-sample performance compared to the “Sample”, “Shrink”, and “CC” strategies. This finding holds true for the other rolling window methodologies as well. However, relying solely on the out-of-sample performance as a singular index may not provide a comprehensive evaluation of the various investment strategies under analysis. This limitation arises because the out-of-sample performance prioritizes returns overlooking the aspect of evaluating risk. Therefore, a more comprehensive assessment requires the consideration of both return and risk metrics to understand the inherent risk–return trade-off in each strategy. For this, we compute two risk-adjusted performance measures: the Sharpe ratio and the omega ratio.

Table 1. Out-of-sample statistics for the various portfolios and the four strategies under analysis are reported for each rolling window methodology. The mean, standard deviation, and Sharpe ratio are provided on an annual basis. The best strategy for each computed statistic is emphasized in bold. The cells corresponding to negative Sharpe ratios are left blank.

RW 6 Months In-Sample, 1 Month Out-of-Sample
	NDX						S5ENRS
	$\mathit{\mu }$	$\mathit{\sigma }$	Skewness	Kurtosis	$\mathrm{SR}$	$\mathrm{OR}$	$\mathit{\mu }$	$\mathit{\sigma }$	Skewness	Kurtosis	$\mathrm{SR}$	$\mathrm{OR}$
Sample	0.150	0.158	−0.388	12.404	0.953	1.193	−0.020	0.278	−2.044	54.406	–	0.985
Shrinkage	0.156	0.158	−0.421	13.869	0.989	1.203	−0.023	0.278	−2.027	53.787	–	0.983
CC	0.170	0.168	−0.432	15.126	1.014	1.212	−0.028	0.286	−2.182	58.677	–	0.980
CC-RMT	0.173	0.169	−0.549	19.508	1.027	1.217	−0.006	0.278	−1.004	26.446	–	0.996
	DJI						IBEX
Sample	0.069	0.135	−0.097	14.955	0.511	1.103	0.011	0.171	−2.202	30.704	0.065	1.012
Shrinkage	0.061	0.135	−0.109	15.777	0.452	1.090	0.014	0.170	−2.233	30.384	0.085	1.016
CC	0.036	0.140	−0.010	15.197	0.260	1.051	0.015	0.178	−2.176	30.847	0.082	1.016
CC-RMT	0.075	0.146	−0.435	25.338	0.514	1.108	0.021	0.174	−1.697	24.944	0.118	1.022
RW 12 Months In-Sample, 1 Month Out-of-Sample
	NDX						S5ENRS
	$\mathit{\mu }$	$\mathit{\sigma }$	Skewness	Kurtosis	$\mathrm{SR}$	$\mathrm{OR}$	$\mathit{\mu }$	$\mathit{\sigma }$	Skewness	Kurtosis	$\mathrm{SR}$	$\mathrm{OR}$
Sample	0.138	0.162	−0.462	15.083	0.852	1.173	0.003	0.273	−1.605	40.578	0.009	1.002
Shrinkage	0.137	0.162	−0.443	15.292	0.846	1.173	0.000	0.274	−1.623	40.714	0.001	1.000
CC	0.138	0.174	−0.435	18.384	0.798	1.165	0.000	0.277	−1.562	38.124	0.001	1.000
CC-RMT	0.149	0.175	−0.550	21.857	0.854	1.179	0.031	0.284	−0.853	24.782	0.110	1.021
	DJI						IBEX
Sample	0.056	0.140	−0.348	17.637	0.403	1.081	0.009	0.168	−1.926	27.381	0.056	1.010
Shrinkage	0.055	0.139	−0.249	16.768	0.393	1.079	0.010	0.167	−1.966	26.984	0.059	1.011
CM	0.036	0.142	−0.234	14.925	0.256	1.050	0.011	0.176	−1.703	24.097	0.060	1.011
CM RMT	0.052	0.150	-0.467	26.307	0.349	1.073	0.017	0.179	−1.499	24.244	0.097	1.018
RW 24 Months In-Sample, 1 Month Out-of-Sample
	NDX						S5ENRS
	$\mathit{\mu }$	$\mathit{\sigma }$	Skewness	Kurtosis	$\mathrm{SR}$	$\mathrm{OR}$	$\mathit{\mu }$	$\mathit{\sigma }$	Skewness	Kurtosis	$\mathrm{SR}$	$\mathrm{OR}$
Sample	0.144	0.168	−0.515	18.621	0.857	1.178	0.038	0.274	−0.903	24.810	0.139	1.028
Shrinkage	0.143	0.168	−0.544	19.674	0.851	1.178	0.037	0.275	−0.926	25.209	0.134	1.027
CC	0.152	0.181	−0.428	22.929	0.840	1.179	0.032	0.284	−1.059	26.698	0.111	1.022
CC-RMT	0.156	0.179	−0.557	21.381	0.868	1.184	0.071	0.294	−0.814	24.310	0.241	1.048
	DJI						IBEX
Sample	0.061	0.142	−0.465	20.465	0.429	1.089	−0.013	0.167	−1.924	27.775	–	0.986
Shrinkage	0.056	0.142	−0.463	20.226	0.392	1.081	−0.014	0.166	−1.982	27.653	–	0.985
CC	0.031	0.145	−0.487	17.511	0.216	1.043	0.006	0.175	−1.676	23.275	0.032	1.006
CC-RMT	0.071	0.153	−0.457	28.351	0.467	1.101	0.037	0.182	−1.552	25.414	0.203	1.039
	NDX						S5ENRS
	$\mathit{\mu }$	$\mathit{\sigma }$	Skewness	Kurtosis	$\mathrm{SR}$	$\mathrm{OR}$	$\mathit{\mu }$	$\mathit{\sigma }$	Skewness	Kurtosis	$\mathrm{SR}$	$\mathrm{OR}$
Sample	0.148	0.170	−0.425	18.565	0.871	1.183	0.055	0.278	−0.844	23.960	0.198	1.040
Shrinkage	0.148	0.171	−0.448	19.505	0.862	1.181	0.054	0.279	−0.868	24.361	0.194	1.039
CM	0.159	0.183	−0.327	22.713	0.867	1.187	0.048	0.289	−0.994	25.798	0.166	1.034
CM RMT	0.153	0.175	−0.465	21.145	0.873	1.177	0.080	0.299	−0.780	23.468	0.267	1.053
	DJI						IBEX
Sample	0.057	0.144	−0.453	20.131	0.392	1.082	−0.016	0.168	−1.882	27.501	–	0.982
Shrinkage	0.051	0.144	−0.436	19.930	0.356	1.074	−0.015	0.169	−1.876	26.742	–	0.983
CM	0.026	0.147	−0.470	17.269	0.176	1.035	0.005	0.177	−1.624	22.751	0.026	1.005
CM RMT	0.076	0.155	−0.394	27.803	0.489	1.107	0.038	0.182	−1.548	25.313	0.211	1.041

displays key out-of-sample statistics, including the first four moments, the $\mathrm{SR}$, and the $\mathrm{OR}$, for the portfolios under analysis across the four rolling window methodologies considered. While the first four moments may not clearly indicate which strategy performs better due to potential inconsistencies in mean, skewness, standard deviation, and kurtosis, a different picture emerges when examining risk-adjusted performance measures like the $\mathrm{SR}$ and $\mathrm{OR}$. In particular, the CC-RMT strategy consistently exhibits a superior risk-adjusted performance across all portfolios and rolling windows considered.

In contrast to more traditional methods, the CC-RMT strategy seems to effectively capture the complex interdependencies and structural patterns within the financial markets, providing a more accurate representation of the market dynamics. Moreover, the two network approaches (CC and CC-RMT) yield differing outcomes. Although both employ the clustering coefficient to measure the dependence structure, they differentiate for the threshold criteria used in the binarization process of the correlation adjacency matrix. Unlike our approach, the authors in [15] do not employ the Hilbert transformation of time series or the RMT filtering method to identify significant correlations exclusively. Instead, they consider different thresholds on correlation levels to generate multiple binary adjacencies matrices and compute the corresponding clustering coefficient of nodes. Finally, they compute the average clustering coefficient for each node to derive the interconnectedness matrix as outlined in Equation (11).

Considering transaction costs is crucial in practical portfolio management, as excessive turnover can erode returns. As mentioned earlier, this study evaluates transaction costs by using turnover as a proxy. Figure 3 illustrates the turnover values in each rebalancing period for the four portfolios under examination, employing a rolling window of 6 months in-sample and 1 month out-of-sample. The figure clearly indicates that CC-RMT consistently yields a lower portfolio turnover. To provide a comprehensive overview of portfolio turnover,

Table 2. Average portfolio turnover and average modified Herfindahl index for the four portfolios under analysis across different rolling window methodologies. The strategies with a lower average turnover and a lower average modified Herfindahl index are highlighted in bold.

RW 6 Months In-Sample and 1 Month Out-of-Sample
	Average Turnover				Average Modified Herfindahl
	NDX	S5ENRS	DJI	IBEX	NDX	S5ENRS	DJI	IBEX
Sample	0.453	0.268	0.407	0.384	0.128	0.358	0.157	0.226
Shrinkage	0.413	0.255	0.347	0.344	0.138	0.372	0.156	0.207
CC	0.473	0.282	0.369	0.391	0.221	0.519	0.256	0.320
CC-RMT	0.446	0.197	0.289	0.161	0.102	0.115	0.067	0.042
RW 12 Months In-Sample and 1 Month Out-of-Sample
	Average Turnover				Average Modified Herfindahl
	NDX	S5ENRS	DJI	IBEX	NDX	S5ENRS	DJI	IBEX
Sample	0.266	0.140	0.226	0.220	0.120	0.366	0.148	0.192
Shrinkage	0.247	0.138	0.197	0.197	0.131	0.374	0.151	0.180
CC	0.285	0.160	0.219	0.215	0.221	0.532	0.242	0.277
CC-RMT	0.287	0.105	0.220	0.112	0.102	0.117	0.070	0.035
RW 24 Months In-Sample and 1 Month Out-of-Sample
	Average Turnover				Average Modified Herfindahl
	NDX	S5ENRS	DJI	IBEX	NDX	S5ENRS	DJI	IBEX
Sample	0.135	0.084	0.120	0.119	0.115	0.364	0.139	0.174
Shrinkage	0.130	0.084	0.109	0.111	0.123	0.370	0.142	0.164
CC	0.168	0.095	0.120	0.122	0.220	0.536	0.222	0.261
CC-RMT	0.190	0.065	0.128	0.026	0.102	0.114	0.070	0.012
RW 24 Months In-Sample and 2 Months Out-of-Sample
	Average Turnover				Average Modified Herfindahl
	NDX	S5ENRS	DJI	IBEX	NDX	S5ENRS	DJI	IBEX
Sample	0.207	0.146	0.186	0.194	0.115	0.359	0.140	0.175
Shrinkage	0.202	0.147	0.170	0.186	0.124	0.366	0.143	0.166
CC	0.258	0.157	0.189	0.191	0.222	0.533	0.222	0.262
CC-RMT	0.266	0.105	0.183	0.045	0.102	0.109	0.073	0.012

presents the average turnover for the four portfolios across all considered rolling window. These results consistently highlight that CC-RMT tends to produce portfolios with a lower turnover, leading to reduced transaction costs.

We have additionally computed the modified Herfindahl index as a proxy for assessing portfolio diversification. Due to space limitations, the average modified Herfindahl indexes for the analyzed portfolios and rolling window methodologies are presented on the right-hand side of Table 2. Notably, the findings reveal that the CC-RMT approach consistently produces a lower modified Herfindahl index, indicating a more diversified portfolio. This observation highlights the contribution of the CC-RMT model to enhancing diversification within the portfolio, a crucial aspect of effective risk management.

5. Discussions

This study introduces a novel method for estimating the dependence structure, a critical aspect in portfolio selection. In particular, we combine the Hilbert transformation technique and Random Matrix Theory. The Hilbert transformation is used as a data preprocessing technique which identifies lead/lag correlations in asset performance, enabling investors to optimize strategies based on timing as explained in Section 2.2. Random Matrix Theory is used as a filtering technique to differentiate between noise and meaningful information in the covariance matrix, as explained in Section 2.1. The proposed approach aims to obtain a denoised version of the dependence structure, capturing the genuine connections between assets more precisely. By integrating these two techniques, this study enables a more accurate and resilient estimation of the portfolio allocation process.

The empirical analysis demonstrates that the resulting CC-RMT model not only reduces transaction costs, as evidenced by a lower turnover, and enhances portfolio diversification by decreasing the modified Herfindahl index, but also excels in risk-adjusted performance measures. Across different portfolios and rolling window, the CC-RMT strategy consistently exhibits a superior risk-adjusted performance compared to other approaches, including the sample, shrinkage, and clustering coefficient strategies (as discussed in [15]). The CC-RMT model provides investors with a clearer understanding of market dynamics, offering a well-informed framework for making investment decisions. This paper only deals with a one-objective function as we consider the global minimum variance portfolio. A possible extension is to apply network theory to the estimation of other portfolio moments and co-moments. This would lead to the investigation of multi-objective portfolio optimization techniques as in [35,36].

This research significantly contributes to portfolio management by providing insights into the underlying structures and dynamics of investments. It assists investors in making smarter and more effective investment decisions by providing valuable insights and guiding them towards strategies that are better adjusted for risk.