Bridging the Gap Between Theory and Practice: Fitness Landscape Analysis of Real-World Problems with Nearest-Better Network

Abstract

For a long time, there has been a gap between theoretical optimization research and real-world applications. A key challenge is that many real-world problems are black-box problems, making it difficult to identify their characteristics and, consequently, select the most effective algorithms to solve them. Fortunately, the Nearest-Better Network has emerged as an effective tool for analyzing the characteristics of problems, regardless of dimensionality. In this paper, we conduct an in-depth experimental analysis of real-world functions from the CEC 2022 and CEC 2011 competitions using the NBN. Our experiments reveal that real-world problems often exhibit characteristics such as unclear global structure, multiple attraction basins, vast neutral regions around the global optimum, and high levels of ill conditioning.

1. Introduction

For a long time, there has been a gap between theoretical research and research oriented towards the real world [1]. Sometimes, a winning algorithm may perform poorly in real-world applications. Each year, numerous optimization algorithms are proposed, and these algorithms can outperform others on certain problems. This can perhaps be explained by the No Free Lunch (NFL) theorem [2]. This theorem essentially states that there is no single metaheuristic that universally outperforms all others across all types of problems. It seems that we need to design different algorithms for different optimization problems. As Marti [3] points out, this theorem emphasizes the significance of understanding problem-specific characteristics and selecting or designing appropriate metaheuristics based on these characteristics. There are numerous optimization problems worldwide, and it is nearly impossible to design an algorithm for each one. What we truly need to focus on is analyzing the characteristics and difficulties of the problems.

Yet, most real-world problems are high-dimensional black-box problems. The characteristics of these problems can be very complex and difficult to analyze. We lack an effective method to analyze the characteristics of problems and the behavior of algorithms to assist us in choosing or designing efficient algorithms. Fortunately, our recent work, the Nearest-Better Network (NBN) [4,5], has been proven to be an effective tool for analyzing the characteristics of problems of any dimensionality. Diao et al. [5] have verified that the characteristics of the fitness landscape can be captured by the NBN visualization, including asymmetry, ill conditioning, neutrality, ruggedness, size of the BoAs, and the number of BoAs.

The main contribution of this paper is to bridge the gap between theory and practice. Due to the black-box nature of real-world problems, it is difficult to uncover their inherent characteristics. Our recent work, NBN, has proven to be an effective tool for analyzing the characteristics of problems across various dimensionalities. For the first time, we visualize all the functions from the CEC 2022 competitions and CEC 2011 real-world problems and conduct a thorough analysis of several problems with distinct characteristics. Several interesting findings are made in this study:

• In some real-world problems, the global structural features are not clearly defined. Algorithms that learn direction based on population distribution, such as CMA-ES, may easily exceed the problem’s boundaries.
• Some real-world problems contain a large number of attraction basins. In such cases, the diversity maintenance mechanism may negatively impact the algorithm’s performance.
• In certain problems, a vast neutral region exists around the global optimum, making it easy for the algorithm to become trapped in this region.
• Some problems are highly ill-conditioned, causing even the best algorithms to fail to solve them.

2. Related Work

2.1. Fitness Landscape

A fitness landscape [6] is a mapping from solutions in the search space to fitness values with a neighborhood relationship. The solution space $\mathit{X}$ is a set of potential solutions to the problem. The fitness of a solution indicates how good the solution is (the larger the value, the better the solution). The neighborhood relationship can be defined as the distance or accessibility between solutions.

There are many problem characteristics that can affect the algorithm’s performance [7,8]. Some of the characteristics are listed below:

• Modality [9]: Multimodal problems have more than one global or local optimum. Basin of attraction (BoA) is an important concept for multimodal problems, such as $\mathit{B}\left({\mathbf{x}}^{*}\right)=\left\{\mathbf{x}\in \mathit{X}\mid {\mathbf{x}}^{*}=\mathrm{local}\text{-}\mathrm{search}\left(\mathbf{x}\right)\right\}$, where the BoA $\mathit{B}\left({\mathbf{x}}^{*}\right)$ of a local optimum ${\mathbf{x}}^{*}$ is the set of solutions $\mathit{B}\left({\mathbf{x}}^{*}\right)$ that approaches ${\mathbf{x}}^{*}$ by utilizing a local search strategy among the decision variable space $\mathit{X}$ [10].
• Ruggedness [11]: Ruggedness is usually manifested as steep ascents and descents in the fitness landscape with the existence of many local optima.
• Neutrality [12]: Neutrality is usually manifested as a flat area of the fitness landscape. In this flat area, the optimization algorithm may have difficulty finding a better solution.
• Ill condition [13]: An ill-conditioned problem indicates that it is extremely sensitive to slight changes. During the optimization process, small perturbations may lead to significant changes in the solution, making it difficult for the algorithm to converge to the global optima or resulting in a very slow convergence speed.

Generally speaking, there are two ways to bridge the gap between real-world problems and benchmarks: (1) Design problems similar to real-world problems. This type of method mainly focuses on benchmark design. (2) Analyze the characteristics of the given real-world problem. This type of method relies on fitness landscape analysis methods.

2.2. Benchmark Design

There have been many attempts to design problems similar to real-world problems. In the early stage, designers would combine the known characteristics existing in real-world problems, such as multimodality, non-convexity, noise, and constraints, to construct benchmark functions [14]. As people’s understanding of real-world problems deepens, these benchmarks are continuously updated, from CEC 2013 [15] to CEC 2022 [16].

Due to the complex and diverse characteristics of real-world problems, many real-world problems were quite different from the existing benchmarks. Then, Li [17] proposed a customizable benchmark framework based on space segmentation, freepeak. Under this framework, people can design subspaces with different characteristics and then combine them to form a complex problem to simulate any real-world problems. However, the design of the benchmark is limited by people’s understanding of real-world problems. If we do not know the characteristics of a real-world problem, we cannot design a benchmark function similar to this problem. Some benchmarks use functions that are very similar to real-world problems [18] or that even directly include several real-world problems [19].

Whether it is designing benchmarks similar to real-world problems or directly including several real-world problems, the characteristics of real-world problems are quite complex and diverse and it is almost impossible for benchmarks to cover all the characteristics of real-world problems. Therefore, directly analyzing the characteristics of real-world problems using the fitness landscape analysis method is a better choice. The fitness landscape analysis method can be divided into numerical methods and visualization methods according to their ability to visualize data.

2.2.1. Numerical Methods

These methods propose a series of metrics to describe specific characteristics of problems or algorithms. Lip uses the correlation length to evaluate ruggedness: [20]. Davidor employs epistasis variance to assess epistasis [21]. Reidys and Stadler utilized a neutral walk to evaluate neutrality [12]. Lunacek proposed the dispersion metric to evaluate global topology or the presence of funnels [22]. Morgan proposed the length scale to analyze gradient information [23]. Bosman attempted to visualize basins of attraction along with the associated stationary points via gradient-based stochastic sampling [24]. Some work has been carried out to analyze multi-objective landscape features [25] and constrained landscape features [26]. However, real-world problems are quite complex. Different regions in real-world problems may have different characteristics. It is very difficult to mine all the information solely through numerical methods.

2.2.2. Visualization Methods

Theoretically, a good visualization method can help us observe the fitness landscape comprehensively, as well as the search behavior of an algorithm, thereby understanding the problem structure and the algorithm’s working mechanism, as well as helping to design efficient algorithms.

Visual FLAs for continuous problems are very few, with only Local Optima Networks [27], Search Trajectory Networks [28]. and Nearest-Better Networks available. Local Optima Networks (LONs) visualize local optima of the fitness landscape in the form of a graph where nodes are local optima and edges represent possible transitions between optima with a given search operator. The Search Trajectory Network (STN) is defined as a graph whose nodes are locations in a search trajectory, which are representative solutions in different subspaces, and the edges represent the connections between solutions in a search. The Nearest-Better Network is a network where the nodes are solutions and the edges are the nearest-better relationship between solutions.

Previous experiments [5] have proven that the NBN can display many characteristics of the landscape in its visualization, while much characteristic information is lost in the LON and STN visualization. We attempt to use the NBN to visualize the fitness landscape of real-world problems and try to uncover the unknown characteristics of these problems.

3. Nearest-Better Network and Experimental Setup

In the experiment, the Nearest-Better Network is employed to analyze the characteristics of the selected problems. Additionally, sampling by algorithms is utilized to collect data. In this section, we will introduce the definition of the Nearest-Better Network, the selected problems and algorithms, and the sampling method.

3.1. Nearest-Better Network

The Nearest-Better Network [4] (NBN) is a directed graph with the sampled solutions as the vertices and the nearest-better relationships for each solution as the edges. The nearest-better relationship is defined as $\mathbf{b}\left(\mathbf{x}\right)=arg{min}_{\mathbf{y}\in \left\{\mathbf{y}\mid \mathbf{y}\in {\mathit{X}}_N,f\left(\mathbf{y}\right)>f\left(\mathbf{x}\right)\right\}}\parallel \mathbf{y}-\mathbf{x}\parallel$, where $\mathbf{b}\left(\mathbf{x}\right)$ is the nearest-better solution for the solution $\mathbf{x}$. The distance to the nearest-better solution for the solution $\mathbf{x}$, $\parallel \mathbf{x},\mathbf{b}\left(\mathbf{x}\right)\parallel$, is also known as its Nearest-Better Distance (NBD). In optimization algorithms, the search process is guided by fitness, and there is a higher probability of exploring the neighborhood of a given solution. Specifically, an algorithm is likelier to find the nearest-better solution from a given solution $\mathbf{x}$. By retaining only the nearest-better relationships, the NBN simplifies the original fitness landscape to make it easier to analyze.

Figure 1a shows the original fitness landscape of CEC 2022 $f_9$ [16], which will be analyzed in detail in the following experiments. Figure 1b shows the structure of NBN visualization as proposed in reference [4]. The NBN visualization is a tree-like structure where each node is a solution that connects its nearest-better solution. The distance of the edge is the NBD. NBN visualization only preserves the nearest-better relationship and the NBD, so that it can display the NBN structure of problems of any dimensionality. From the original fitness landscape, it can be seen that $f_9$ is a smooth multimodal problem, and the NBN visualization also shows a similar structure. In the experiments [5], it has been verified that the NBN visualization can retain characteristics such as asymmetry, ill conditioning, neutrality, ruggedness, and size of the basins of attraction (BoAs), and the number of BoAs can be captured by the NBN visualization.

3.2. Problems for Analysis

For the artificially designed benchmark, we select the CEC 2022 benchmark [16]. This benchmark is relatively new and it also contains rich problem characteristics, including multimodal, ill conditioning, ruggedness, neutrality, separateness, etc. At the same time, there are many works [29,30] that have conducted experiments and discussions on this benchmark.

For real-world problems, we choose the CEC 2011 problem [31]. It was released in 2010 and has a citation count of 701. A great many algorithms have been tested using these functions. However, as of now, the characteristics of the functions are still unknown.

Table 1. CEC 2011 competition on real-world optimization problems, where D is the dimensionality of the problems.

	Name	Variable Encoding	D
$h_1$	Parameter Estimation for Frequency-Modulated Sound Waves	cont.	6
$h_2$	Lennard-Jones Potential Problem	cont.	30
$h_3$	Bifunctional Catalyst Blend Optimal Control Problem	cont.	1
$h_4$	Optimal Control of a Non-Linear Stirred Tank Reactor	cont.	1
$h_5$	Tersoff Potential for Model Si (B)	cont.	30
$h_6$	Tersoff Potential for Model Si (C)	cont.	30
$h_7$	Spread Spectrum Radar Polly Phase Code Design	cont.	20
$h_8$	Transmission Network Expansion Planning Problem	comb.	7
$h_9$	Large-Scale Transmission Pricing Problem	cont.	126
$h_{10}$	Circular Antenna Array Design Problem	cont.	12
$h_{11}$	ELD Problems: DED Instance 1	cont.	120
$h_{12}$	Messenger: Spacecraft Trajectory Optimization Problem	cont.	26
$h_{13}$	Cassini 2: Spacecraft Trajectory Optimization Problem	cont.	22

presents the basic information about the CEC 2011 problem. In this paper, we mainly focus on continuous problems. Therefore, we select the functions with continuous coding for analysis. Since one-dimensional and two-dimensional continuous problems are relatively simple, the fitness landscape and the search behavior of the algorithm can be directly visualized. Therefore, we select functions $h_1$, $h_2$, $h_5$, $h_6$, $h_7$, $h_9$, $h_{10}$, $h_{11}$, $h_{12}$, and $h_{13}$ for visualization and analysis.

3.3. Selected Algorithms for Sampling

In this paper, we use sampling by algorithm. In high-dimensional problems, data from uniform sampling are extremely sparse relative to the entire solution space and it is difficult to capture the problem characteristics. However, many real-world problems are precisely high-dimensional problems. Although algorithm-based sampling is non-uniform sampling and some characteristics may not be reflected in the algorithm’s search data, this indicates that these characteristics have no impact on the algorithm’s search process. What we focus on are precisely the problem characteristics that have an impact on the algorithm’s search trajectory. Moreover, the NBN method used in this paper has no requirement for the uniformity of sampled data. It can analyze and visualize data from any source.

In this paper, the chosen sampling algorithms can be divided into two categories: global optimization algorithms and multimodal optimization algorithms.

For global optimization algorithms, four champion algorithms of CEC 2022 are selected:

• EA4 [32]: This algorithm uses a cooperative model which contains four algorithms, including CMEAS [33].
• NL-LBC [32]: This algorithm uses non-linear population size reduction success-history adaptive differential evolution with linear bias change.
• NL-MID [34]: This algorithm uses non-linear population size reduction success-history adaptive differential evolution with midpoint.
• S-DP [35]: This algorithm uses a differential evolution with a dynamic perturbation mechanism for population diversity management.

For multimodal optimization algorithms, four multimodal optimization algorithms with different diversity maintenance mechanisms are selected:

• ANDE [36]: This algorithm uses an adaptive multi-population mechanism. The number of populations can be adaptively adjusted during the optimization process, and nearest neighbors are used in the elimination mechanism.
• DHNDE [37]: This algorithm uses a dynamic hybrid niching method to maintain diversity.
• HillVall [38]: This algorithm uses clustering to divide the solution space based on randomly initialized solutions. Then, a valley detection mechanism is employed to detect whether the divided population covers a peak. Subsequently, the divided population is used as the initial population for evolution. When all the divided populations have evolved and converged, HillVall combines the best solutions of each evolved divided population and some new randomly initialized solutions for re-clustering and evolution.
• RS-CMSA [39]: This algorithm uses taboo points to repel the subpopulation to prevent convergence to the same basin.

3.4. Sampling Method

The NBN is constructed using all the data from 30 independent runs of each of the eight chosen algorithms mentioned above. However, there is still a problem with this sampling method in that the amount of data is too large. Two schemes to filter the data are used: uniform selection and optimal selection.

Let the set of all of the algorithm search data be $\mathit{S}=\{{\mathbf{s}}_1,{\mathbf{s}}_2,{\mathbf{s}}_3,\dots ,{\mathbf{s}}_m\}$, where each sample ${\mathbf{s}}_i$ has a corresponding fitness value $f\left({\mathbf{s}}_i\right)$. First, sort the samples in $\mathit{S}$ according to their fitness values in descending order. For any $i,j\in \{1,2,\dots ,m\}$, when $f\left({\mathbf{s}}_i\right)\ge f\left({\mathbf{s}}_j\right)$, ${\mathbf{s}}_i$ comes before ${\mathbf{s}}_j$ in the sorted result. Denote the sorted set as ${\mathit{S}}^{\prime }=\{{\mathbf{s}}_1^{\prime },{\mathbf{s}}_2^{\prime },{\mathbf{s}}_3^{\prime },\dots ,{\mathbf{s}}_m^{\prime }\}$, where $f\left({\mathbf{s}}_1^{\prime }\right)\ge f\left({\mathbf{s}}_2^{\prime }\right)\ge \cdots \ge f\left({\mathbf{s}}_m^{\prime }\right)$.

Then, use one of the following two schemes to filter the data:

• Uniform selection: Evenly select N solutions. Let $n_k=\lfloor {\displaystyle \frac{km}{N}}\rfloor$ for $k\in \{1,2,\dots ,N\}$. The set of selected solutions is ${\mathit{S}}_{\mathrm{even}}=\{{\mathbf{s}}_{n_1}^{\prime },{\mathbf{s}}_{n_2}^{\prime },{\mathbf{s}}_{n_3}^{\prime },\dots ,{\mathbf{s}}_{n_N}^{\prime }\}$.
• Optimal selection: Select the N solutions with the best fitness values:
  ${\mathit{S}}_{\mathrm{best}}=\{{\mathbf{s}}_1^{\prime },{\mathbf{s}}_2^{\prime },{\mathbf{s}}_3^{\prime },\dots ,{\mathbf{s}}_N^{\prime }\}$.

Since the z-coordinate of the NBN visualization is the fitness value, the NBN constructed from data based on optimal selection pays more attention to the local structure of the top of the NBN, while the NBN constructed from uniformly selected data focuses on the global structure of the NBN. In this paper, the number of filtered solutions, N, is set to 1 × 10⁶.

To show the local structure of the problem, this paper converts the original problem into one with a smaller boundary around the global optimum $\mathbf{o}=[o_1,o_2,\dots o_D]$. The boundary of this problem is defined as $[o_i-L_i*r,o_i+R_i*r]$, for $k\in \{1,2,\dots ,D\}$, where $[L_i,R_i]$ are the original lower and upper bounds of the i-th dimension of the problem, and r is the reduction ratio of the problem’s search range. D is the dimensionality of the problem.

4. Experimental Analysis

Figure 2 and Figure 3 show the original fitness landscape of the two-dimensional CEC 2022 functions. Figure 4, Figure 5, and Figure 6 are the NBNs of all the functions constructed from uniformly selected data. Figure 7, Figure 8 and Figure 9 are the NBNs based on optimal selection.

Table 2. Results of the algorithms on different functions, where D is the dimensionality of the problem, R is each algorithm’s rank, and “mean” is the mean value of the algorithm result among 30 runs.

CEC 2022 D = 2
	$f_1$		$f_2$		$f_3$		$f_4$		$f_5$		$f_6$		$f_7$		$f_8$		$f_9$		$f_{10}$		$f_{11}$		$f_{12}$
Name	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean
EA4	1	−300	1	−400	1	−600	1	−800	1	−900	1	−1800	1	−2000	1	−2201.72	1	−2300.00	4	−2433.33	1	−2600	1	−2700
NL-LBC	1	−300	1	−400	1	−600	1	−800	1	−900	1	−1800	1	−2000	1	−2201.72	2	−2306.67	5	−2460.04	1	−2600	1	−2700
NL-MID	1	−300	1	−400	1	−600	1	−800	1	−900	1	−1800	1	−2000	1	−2201.72	1	−2300.00	3	−2423.34	1	−2600	1	−2700
S-DP	1	−300	1	−400	1	−600	1	−800	1	−900	1	−1800	1	−2000	1	−2201.72	1	−2300.00	1	−2400.00	1	−2600	1	−2700
ANDE	1	−300	1	−400	1	−600	1	−800	1	−900	1	−1800	1	−2000	1	−2201.72	1	−2300.00	2	−2422.93	1	−2600	1	−2700
DHNDE	1	−300	1	−400	1	−600	1	−800	1	−900	1	−1800	1	−2000	1	−2201.72	1	−2300.00	1	−2400.00	1	−2600	1	−2700
HillVall	1	−300	1	−400	1	−600	1	−800	1	−900	1	−1800	1	−2000	1	−2201.72	1	−2300.00	1	−2400.00	1	−2600	1	−2700
RS-CMSA	1	−300	1	−400	1	−600	1	−800	1	−900	1	−1800	1	−2000	1	−2201.72	1	−2300.00	1	−2400.00	1	−2600	1	−2700
CEC 2022 $D=10$
	$f_1$		$f_2$		$f_3$		$f_4$		$f_5$		$f_6$		$f_7$		$f_8$		$f_9$		$f_{10}$		$f_{11}$		$f_{12}$
Name	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean
EA4	1	−300	3	−400.40	1	−600.00	3	−802.89	1	−900.00	1	−1800.00	1	−2000.00	2	−2200.03	3	−2529.28	7	−2500.07	1	−2600.00	7	−2864.60
NL-LBC	1	−300	2	−400.13	1	−600.00	1	−800.63	1	−900.00	3	−1800.16	1	−2000.00	1	−2200.00	3	−2529.28	8	−2500.1	1	−2600.00	8	−2864.92
NL-MID	1	−300	1	−400.00	1	−600.00	7	−804.15	3	−900.09	2	−1800.07	1	−2000.00	4	−2200.11	2	−2502.55	2	−2403.91	1	−2600.00	6	−2863.57
S-DP	1	−300	1	−400.00	1	−600.00	6	−804.02	1	−900.00	4	−1800.30	1	−2000.00	3	−2200.10	3	−2529.28	1	−2400	1	−2600.00	3	−2861.31
ANDE	1	−300	4	−400.40	3	−600.21	8	−815.42	2	−900.01	6	−1806.66	3	−2005.18	6	−2207.98	3	−2529.28	4	−2486.99	2	−2625.07	4	−2861.43
DHNDE	1	−300	5	−402.16	1	−600.00	4	−803.75	1	−900.00	5	−1801.74	4	−2006.90	7	−2208.89	3	−2529.28	6	−2500.06	1	−2600.00	2	−2859.99
HillVall	1	−300	1	−400.00	2	−600.00	5	−803.98	1	−900.00	8	−1818.20	5	−2009.48	8	−2212.74	1	−2379.19	5	−2493.94	1	−2600.00	1	−2841.43
RS-CMSA	1	−300	1	−400.00	1	−600.00	2	−801.53	1	−900.00	7	−1808.06	2	−2001.71	5	−2202.72	4	−2529.28	3	−2468.98	1	−2600.00	5	−2863.43
CEC 2011
	$h_1$		$h_2$		$h_5$		$h_6$		$h_7$		$h_9$		$h_{10}$		$h_{11}$		$h_{12}$		$h_{13}$
Name	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean	R	mean
EA4	1	0.000	2	31.70	3	35.696	3	28.984	2	−0.583	8	−2.164 × 106	5	22.746	8	−1.027 × 107	2	−11.367	4	−15.602
NL-LBC	5	−0.365	5	30.72	5	34.299	5	27.914	6	−0.876	3	−5.969 × 102	6	21.519	2	−4.973 × 104	5	−14.179	7	−19.386
NL-MID	7	−0.624	1	32.31	1	36.721	1	29.166	4	−0.730	1	−2.085 × 102	1	32.200	1	−4.952 × 104	1	−11.173	1	−12.834
S-DP	3	0.000	6	30.39	2	36.483	2	29.166	8	−0.955	2	−2.482 × 102	2	31.665	3	−5.143 × 104	3	−12.833	2	−13.145
ANDE	8	−3.353	7	26.78	7	34.020	7	24.813	7	−0.946	5	−9.017 × 103	8	14.744	6	−6.131 × 104	8	−15.864	8	−20.153
DHNDE	2	0.000	3	31.14	6	34.024	6	25.869	5	−0.737	4	−2.517 × 103	4	27.372	5	−5.261 × 104	7	−15.145	3	−15.173
HillVall	6	−0.553	4	30.78	8	33.844	8	21.143	1	−0.523	7	−1.583 × 106	7	21.437	4	−5.248 × 104	6	−15.045	5	−18.542
RS-CMSA	4	−0.359	8	0.00	4	35.687	4	28.153	3	−0.616	6	−2.810 × 105	3	27.500	7	−6.858 × 104	4	−13.396	6	−18.598

shows the comparison results of the selected algorithms on these functions. From the NBN visualization, it can be observed that the problem characteristics of real-world problems are very diverse. For example, there is a neutral place around the global optima in $h_6$. $h_2$ has a large neutral place near the global optimum. $h_7$ has many basins of attraction. $h_{12}$ and $h_{13}$ are highly ill-conditioned. In the following experiment, we will conduct an in-depth analysis and discussion on the functions with unique characteristics.

4.1. Comparison of High-Dimension and Low-Dimension Problems

For a long time, it has been widely believed that there is a certain degree of similarity between high-dimensional and low-dimensional problems. Based on this principle, when artificially designing benchmarks or algorithms, the fitness landscape of two-dimensional problems is observed to infer the characteristics of high-dimensional problems and the behavior of algorithms. However, there are also significant differences between them. The difference can lead to incorrect inferences about the behavior of algorithms on high-dimensional problems if we only rely on observations of the two-dimensional problems.

From the NBN visualization, we found that there is indeed some similarity between high-dimensional problems and low-dimensional problems. For example, CEC 2022 $f_2$ is ill-conditioned in two-dimensional space, as shown in Figure 2, and this characteristic is retained in the high-dimensional function, as shown in Figure 7.

However, the differences between the two-dimensional and ten-dimensional functions of $f_9$ and $f_{12}$ are relatively large. As shown in Table 2, the performances of algorithms on these two problems with low and high dimensions are quite different. In the two-dimensional problem, all algorithms can find the global optima. In the ten-dimensional problem, only HillVall finds the global optima, and the performance of the other four champion algorithms is not good. If we only look at the fitness landscape of two-dimensional $f_9$ and $f_{12}$, it is difficult to find the reason. Especially for the two-dimensional $f_9$ problem, as shown in Figure 2, it seems to be a simple multimodal problem: the fitness landscape is relatively smooth, without any very difficult characteristics. The number of modalities is not very large and the size of the BoA of the global optima is relatively large. However, from Figure 10, we found that in high-dimensional $f_9$ and $f_{12}$, the BoA of the global optimal solution becomes very small. The size of the BoA, $|{\mathit{B}}_{\mathbf{o}}|$, of $f_9$ is only $7.25\times {10}^{-4}$ and $|{\mathit{B}}_{\mathbf{o}}|$ of $f_{12}$ is only $1.58\times {10}^{-4}$. This makes the problems very difficult to solve for the four champion algorithms, which are global optimization algorithms.

4.2. Global Structure

In the experiments, we found that EA4 based on CMEAS sometimes exceeds the boundary of the problems, $h_9$ and $f_{11}$. Table 2 also shows that the results of EA4 on $h_9$ and $h_{11}$ are poor. CMEAS calculates the evolution direction of the next-generation population based on the distribution of the current population. If the algorithm exceeds the boundary, it means that the algorithm cannot learn the direction of the global optima based on the current population. It also indicates that global structures of $h_9$ and $f_{11}$ are not clear.

Figures and Figure 6 demonstrate that $h_9$ and $h_{11}$ indeed possess a global structure. Figure 11 shows the population distribution from which EA4 cannot learn the direction of the global optimum. Owing to the loss of some information on the original high-dimensional problems, it is hard to obtain effective information from the NBN visualization. Therefore, some experiments need to be carried out for a further analysis of the global structure of $h_9$ and $h_{11}$.

First, we reduce the search range of the problem to observe the search success rate of EA4 to verify whether the global structure of the problem becomes clearer as the search range of the problem shrinks. As shown in Figure 12, the search success rate of EA4 does increase as the search range of the problem shrinks. However, even when the initial search range is quite small, $r=0.1$, the success rate of EA4 does not exceed 0.5. This indicates that the global structure of $h_9$ and $h_{11}$ is not particularly clear even in the local areas.

However, we also need to take into account the influence of dimensionality. As indicated in Table 1, the dimensionality of $h_9$ and $h_{11}$ is relatively high. $h_9$ is 126-dimensional and $h_{11}$ is 120-dimensional. Is it possible that the global structure of high-dimensional problems is not as clear as that of the low-dimensional problems? In response to this doubt, we generated 10-dimensional, 120-dimensional, and 126-dimensional CEC 2022 problems, respectively, to verify the search success rate of EA4. As shown in

Table 3. The success rate of EA4, where success rates lower than 100% are in bold.

CEC 2022

D

$f_1$

$f_2$

$f_3$

$f_4$

$f_5$

$f_6$

$f_7$

$f_8$

$f_9$

$f_{10}$

$f_{11}$

$f_{12}$

10

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

120

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

126

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

30/30

CEC 2011

D

$h_1$

$h_2$

$h_5$

$h_6$

$h_7$

$h_9$

$h_{10}$

$h_{11}$

$h_{12}$

$h_{13}$

-

30/30

30/30

30/30

30/30

30/30

4/30

30/30

2/30

30/30

30/30

, the success rate of EA4 in these functions is uniformly 100%. This indicates that the unclear global structure is a unique characteristic of $h_9$ and $h_{11}$, which leads to a decline in the performance of EA4, which is based on global structure learning.

4.3. Modality

Multimodality is an important characteristic of the problems. For different types of multimodal characteristics, the behavior of algorithms is quite different. Later, we will analyze algorithm behavior in combination with different types of multimodal characteristics. We noted that when observing the characteristics of different algorithms, the data of the NBN in the figures are different. Due to different data sources, the NBN structures of the same problem are different.

4.3.1. The Size of the BoAs of Global Optima Is Very Small

In Section 4.1, we saw that the BoA of the global optima of CEC 2022 $f_9$ and $f_{12}$ are very small. However, it is still unknown why HillVall performs best among all the other algorithms in these two problems. What mechanism inside HillVall plays a role? Since the problem characteristics of CEC 2022 $f_9$ and $f_{12}$ are similar, only $f_9$ is discussed in this subsection.

Figure 13 shows that all algorithms except HillVall failed to find the global optima. In particular, ANDE, an adaptive multi-population algorithm, only maintains one population at the end of evolution. This indicates that its multi-population mechanism fails in the case where the BoA is very small. We can infer the reason why HillVall finds the global optima with this very small BoA from Figure 14. As shown in the clustering result, HillVall first divides the solution space into many subspaces, with a total of 42 subspaces. In the evolution that converges to the global optimum, although the clustering result is not very accurate, many solutions in the initial population belong to the BoA of the global optima, making it more likely for the algorithm trajectory based on this initial population to converge to the global optimum.

4.3.2. The Problem Has Many BoAs

In real-world optimization problems, there can be a situation where the number of BoAs is extremely large, such as CEC 2011 $h_7$, which has 349 BoAs as shown in Figure 15. It is a difficult problem. In this problem, the behaviors of the algorithms with different mechanisms are very inconsistent with our expectations.

• The multi-population mechanism does not perform well
  Ideally, the multi-population mechanism can achieve the state where one population covers one peak in multimodal problems. However, the performance is not satisfactory on this problem. From Figure 15, it can be observed that in ANDE, which is an adaptive multi-population algorithm, the number of populations is significantly smaller than the number of BoAs. There are even several subpopulations conducting searches in the same BoA. For example, the green subpopulation and orange subpopulation evolved in the same BoA. This indicates that the multi-population mechanism is unable to divide the BoAs accurately in this problem.
• The mechanism for maintaining diversity has a negative effect.
  DHNDE and RS-CMSA possess mechanisms for maintaining diversity. From the algorithm trajectories in Figure 15, it can be observed that the diversity of these two algorithms is maintained well with solutions in each BoA. However, there are almost no solutions around the global optimum. This indicates that these two algorithms cannot converge to the global optimal solution.
  Similarly, among global optimization algorithms, S-DP also has a diversity maintenance mechanism, and it even cannot converge in this problem. The color represents the iteration, and the individuals at the last iteration (in red) are scattered at different BoAs. The results in Table 2 also verify its behavior. As one of the four champion algorithms of CEC 2022, its performance on this problem is the worst among all algorithms. This indicates that in the case of a very large number of BoAs, the mechanism for maintaining diversity even plays a negative effect on the algorithm performance.
• The space segmentation mechanism can reduce the difficulty of the problem.
  HillVall is the algorithm with the best performance on this problem. Why can it outperform other algorithms? Is it similar to the case where the basin of attraction is very small? In fact, in the case where the BoA is very small and the case where the problem has many BoAs, HillVall’s behaviors are quite different.
  As shown in Figure 16, from the figure depicting the distribution of all the best solutions of each evolved population of HillVall, due to the unreasonable positions of some initialized subpopulations, the algorithm converges to slopes many times. This indicates that the clustering mechanism is not very effective.
  In the successful evolution shown in Figure 17, its initialized population and its converged position are not in the same BoA. Although the position of the initialized population is not accurate, evolving based on a population in a small area can still converge to a global optimum. This indicates that although the clustering mechanism is not effective, the space segmentation works. By dividing the solution space into smaller regions, it can reduce the multimodal difficulty of the problem.

4.3.3. Multimodal Optimization Algorithms Do Not Perform Better than Global Optimization Algorithms in Finding Multiple Global Optima

Compared to global optimization algorithms, multimodal optimization algorithms are more focused on finding more global optima. Generally, we would think that in terms of the number of global optimal solutions found, multimodal optimization algorithms can outperform global optimization algorithms. However, this is not the case in real-world problems.

CEC 2011 $h_1$ is a relatively simple multimodal problem with only eight global optima. The objective of the global optima of CEC 2011 $h_1$ is 0, as stated in [31], and many algorithms can find the global optimum as shown in Table 2. This indicates that the search operator can solve this problem well. Thus, we can exclude the influence of the search operator’s solving ability and observe the algorithm’s ability to find multiple global optima with this problem.

Figure 18 shows the distribution of algorithms near the global optimal solution on CEC 2011 $h_1$. The black rectangular boxes are the global optima. It can be observed that multimodal optimization algorithms do not perform better than global optimization algorithms in finding multiple global optima. For example, HillVall and DHNDE only find two global optima, while EA4 without a diversity mechanism can find three global optima. This indicates that population-based algorithms, with multiple individuals evolving simultaneously, can naturally maintain diversity. And the algorithm that finds the most optima is a global optimization algorithm, NL-MID, which finds five global optima. This algorithm does not have a very complex diversity maintenance mechanism. It uses the midpoint mechanism, that is, it makes use of the geometric center of the population to maintain diversity.

4.4. Neutrality

The default dimensionality of $h_2$ is $D=30$, and the objective value of its global optima is −37.967600. As shown in Table 2, no algorithm finds the global optimal solution in any run. Since the NBN is generated from algorithm trajectory data if the algorithm does not find the global optima, we cannot observe the structure near the global optima without data.

Based on the principle that high-dimensional problems and low-dimensional problems have similarities, we try to visualize $h_2$ in low dimensionality. Fortunately, the dimensionality of $h_2$ can be adjusted. Although it cannot be set as a two-dimensional problem to directly observe the original fitness landscape of the problem, we can set a relatively low dimension to reduce the difficulty of the problem so that the algorithm can find the global optima and facilitate the observation of the structure near the global optima. Therefore, we designed $h_2$ with $D=6$. For this problem, there are some algorithms that can find the global optimal solution.

As shown in Figure 19, although the problem is a uni-modal problem, there is a neutral region near the global optimal solution, which is also the main difficulty in solving this problem.

Table 4. Frequency of becoming stuck in neutral regions of algorithms on CEC 2011 $h_2(D=6)$.

EA4	NL-LBC	NL-MID	S-DP	ANDE	DHNDE	HillVall	RS-CMSA
30/30	30/30	30/30	28/30	30/30	30/30	30/30	11/30

shows the number of times the algorithm becomes stuck in a neutral place. It can be seen that these algorithms easily become stuck in the neutral place on this problem.

Very interestingly, although none of these algorithms find the global optima in $h_2(D=30)$, the NBNs of $h_2(D=6)$ and $h_2(D=30)$ are very similar. As shown in Figure 5 and Figure 8, $h_2(D=30)$ is also a uni-modal problem, and there is a neutral region near the global optimal solution. This indicates that although the algorithm did not find the global optima, the neighborhood structure of the global optima is found by the algorithms.

4.5. Ill Conditioning

CEC 2011 $h_{12}$ is a highly ill-conditioned problem and the experiments also show that $h_{12}$ is a very difficult problem. Figure 20a shows that the objective value of the best-found solution decreases as the number of independent runs increases. This indicates that in more than 2000 independent runs, the best solution is found in only one run, and this solution may not necessarily be the global optima. This indicates that even the best algorithms fail to solve this complex problem, $h_{12}$.

From Figure 6 and Figure 9, we find that $h_{12}$ is a uni-modal but highly ill-conditioned problem, with some solutions with long convergence trajectories. One characteristic of being ill-conditioned is that the algorithm has a slow convergence speed. That is, there are long convergence trajectories in the NBN. In the NBN, for any solution $\mathbf{x}\in \mathit{S}$, there is a trajectory that converges to the global optimum $\mathbf{o}$, $\tilde{P}(\mathbf{x},\mathbf{o})=[{\mathbf{p}}_1,{\mathbf{p}}_2,\dots ,{\mathbf{p}}_k]$, where ${\mathbf{p}}_1=\mathbf{x}$, ${\mathbf{p}}_k=\mathbf{o}$, and k represents the number of nodes along the path. It can be seen that the longer this trajectory is, the more difficult it is for the current solution to converge to the global optima. Based on this, we use the indicator of the longest convergence trajectory $\parallel \tilde{P}{\parallel }_{\max }={max}_{\mathbf{x}\in \mathit{S}}\parallel \tilde{P}(\mathbf{x},\mathbf{o})\parallel$ as the indicator for ill conditioning. As shown in

Table 5. $\parallel \tilde{P}{\parallel }_{\max }$ of different functions, where the largest values are in bold.

	$f_1(D=10)$	$f_2(D=10)$	$f_3(D=10)$	$f_4(D=10)$	$f_5(D=10)$	$f_6(D=10)$
even	1.83 × 10−4	4.60 × 10−4	1.04 × 10−4	6.52 × 10−5	5.67 × 10−5	2.71 × 10−4
best	4.2 × 10−5	1.25 × 10−4	3.5 × 10−5	3.6 × 10−5	3.8 × 10−5	2.23 × 10−4
	$f_7(D=10)$	$f_8(D=10)$	$f_9(D=10)$	$f_{10}(D=10)$	$f_{11}(D=10)$	$f_{12}(D=10)$
even	1.00 × 10−4	2.28 × 10−4	3.26 × 10−4	1.50 × 10−4	3.66 × 10−4	3.13 × 10−4
best	8.1 × 10−5	1.09 × 10−4	9.6 × 10−5	6.8 × 10−5	4.8 × 10−5	1.20 × 10−4
	$h_1$	$h_2$	$h_5$	$h_6$	$h_7$	$h_9$
even	1.32 × 10−4	9.83125 × 10−5	6.72 × 10−4	6.30 × 10−4	3.58 × 10−4	6.50 × 10−4
best	1.80 × 10−4	1.04 × 10−4	1.23 × 10−4	1.23 × 10−4	2.22 × 10−4	1.85 × 10−4
	$h_{10}$	$h_{11}$	$h_{12}$	$h_{13}$
even	2.92 × 10−4	6.23 × 10−4	1.27 × 10−3	8.04 × 10−4
best	1.29 × 10−4	3.19 × 10−4	1.76 × 10−3	7.05 × 10−4

, compared to other problems, the value of $\parallel \tilde{P}{\parallel }_{\max }$ of $h_{12}$ is the highest, which indicates that CEC 2011 $h_{12}$ is the most ill-conditioned problem.

But one may wonder whether there are many very small BoAs in the problem that lead to the ill-conditioning characteristic of $h_{12}$? To answer this question, we visualize the local NBN of $h_{12}$ as shown in Figure 21. It can be observed that from $r=0.9$ to $r=0.05$ (r is the reduction ratio of the search range), $h_{12}$ is a uni-modal problem. As the problem search range increases, the ill-conditioned characteristic of the problem becomes more obvious. Figure 20b also verifies the uni-modal characteristic of $h_{12}$. In previous work [5], the NBD is used as an indicator to identify local optimal solutions. If its NBD exceeds the threshold, the current solution is considered the local optima. Then, $max\mathrm{NBD}=\underset{\mathbf{x}\in \mathit{S}-\left\{\mathbf{o}\right\}}{max}\mathrm{NBD}\left(\mathbf{x}\right)$, which is the maximum NBD in the NBN except for the global optimal solution $\mathbf{o}$, can work as an indicator for the multimodal characteristic. The smaller this value is, the less multimodal the problem is. Figure 20b shows the relationship between the reduction ratio r and max NBD. As the problem search range becomes smaller, the max NBD also becomes smaller. This indicates that there is only one BoA at any microscopic scale of the problem. The difficulty of the problem lies only in being ill-conditioned.

5. Discussion

From the experimental results, the NBN is quite useful. It can uncover many problem characteristics, such as multimodality, neutrality, and ill conditioning. It can help us observe the problem characteristics and the behavior of algorithms and it is also a very good tool to assist people in designing algorithms. However, it has some drawbacks. In NBN visualization, some information about high-dimensional problems is inevitably lost. For example, for CEC 2011 $h_9$ and $h_{10}$, EA4 based on global structure learning exceeds the boundary multiple times, but the NBN cannot observe the structure of the distribution of the population. This is due to the loss of some spatial structure information of high-dimensional problems in NBN visualization. Moreover, the NBN is data-dependent. If the data cannot provide effective information, the NBN cannot observe the relevant characteristics either. For instance, if an algorithm fails to find the global optima, the NBN cannot observe the local structure of the global optima either.

From the experiments, we can find that there is a significant gap between real-world problems and benchmarks. Although the CEC 2022 benchmark contains many characteristics, such as multimodality, neutrality, and ill-conditioning, these characteristics do not have much impact on algorithm behaviors. The NBN structures generated based on algorithm data for $f_1(D=10)$, $f_4(D=10)$, $f_5(D=10)$, $f_6(D=10)$, and $f_{11}(D=10)$ are quite similar. The characteristics of these problems have little impact on algorithm behavior, resulting in these characteristics being ignored in algorithm trajectories. The only one that significantly impacts algorithm behavior is the small size of the BoA of global optima, which is observed in $f_9(D=10)$ and $f_{10}(D=10)$. The characteristics of real-world problems are extremely rich and unique. $h_7$ has 349 basins of attraction. $h_2$ has a large neutral space around the global optimum. $h_{12}$ is a highly ill-conditioned problem. Although CEC 2022 considered the difficulty of ill conditioning and designed an ill-condition function, $f_2$, the degree of the ill conditioning of $h_{12}$ is much higher than that of $f_2$, as shown in the result of Table 5. This indicates that our understanding of real-world problems is insufficient, which also leads to a large gap between the benchmarks and real-world problems. This can also explain to some extent why the champion algorithms in competitions sometimes perform poorly on real-world problems.

At the same time, the performance of an algorithm is closely related to problem characteristics. For a long time, we have believed that the ability to maintain diversity is important for algorithms and, many algorithms emphasize their ability to maintain diversity [40]. However, in some cases, the diversity maintenance mechanism may reduce the performance of an algorithm. For example, $h_7$ has a very large number of BoAs (a total of 349). For this problem, algorithms with a diversity maintenance mechanism all have very poor performance and may even become stuck in stagnation, such as S-DP. The characteristics of real-world problems are extremely diverse. If the algorithms blindly use their mechanisms, they cannot adapt to these diverse real-world problems. Future algorithms should have the ability to learn problem characteristics and adaptively adjust their mechanisms.

In this paper, we visualize the real-world CEC 2011 and CEC 2022 benchmarks. While using standardized benchmarks like CEC functions helps facilitate comparisons, they may not fully capture the diversity of real-world optimization challenges. We strongly encourage researchers to apply the NBN to analyze the real-world problems they face, as these may contain many unknown and challenging characteristics. We look forward to researchers sharing their findings with us. A link to the code used in this paper is provided at the end.

The time complexity of the NBN computation consists of two parts: the neighborhood relationship calculation and the nearest-better relationship calculation: (1) The problems analyzed in this paper are continuous, high-dimensional problems. To compute the neighborhood relationships, we use kd-trees [41], which have a time complexity of $O(NlnN)$. (2) The nearest-better relationship calculation algorithm, proposed in [5], has a time complexity of $O(wNDlnN)$, where N is the number of sampled solutions, D is the problem’s dimensionality, and w is a parameter related to the number of peaks and their shapes. Although the time complexity is log-linear, it remains computationally expensive. For instance, computing a single NBN instance with 1000,000,000 samples on a DELL V3670-i7(8700)-GTX1050Ti-48G (manufacturer: Dell Technologies, Wuhan, China) system takes approximately 12 h. Further optimization of the algorithm is still necessary. A parallel implementation is expected to enhance its efficiency, and this is also one of our future projects.

6. Conclusions

A more in-depth perspective on observing real-world problems with the NBN is provided in this paper to enhance people’s understanding of real-world problems and help people design more efficient algorithms. In this paper, we visualize all the functions in CEC 2022 and CEC 2011 and select some of these functions with special characteristics for in-depth analysis with the NBN. The code of this paper can be found on OFEC. There are many interesting findings, such as the following:

• The global structures of some real-world problems are clear, and on these problems, algorithms based on population distribution learning such as EA4 are prone to exceed the boundary.
• In real-world problems, there exist some problems that contain a very large number of BoAs. CEC 2011 $f_7$ has a total of 349 BoAs. In this problem, the diversity maintenance mechanism has a negative impact on the algorithm performance.
• For some real-world problems, such as CEC 2011 $h_2$, there is a large neutral area near the global optimal solution, which makes the algorithm easily stuck in the neutral place.
• There are some highly ill-conditioned problems that are difficult to solve, such as CEC 2011 $f_{12}$. This problem is uni-modal and highly ill-conditioned and it is characterized by long convergence trajectories. The experimental results show that none of the current best algorithms can solve this problem efficiently.

In our future work, we will make good use of the structure of the NBN to design an intelligent algorithm with the ability to learn problem characteristics and adaptively adjust the mechanisms to adapt to diverse real-world problems.