1. Introduction
Metaheuristics are high-level problem-independent algorithmic frameworks that provide a set of guidelines or strategies to develop heuristic optimization algorithms [
1]. These frameworks are designed to solve complex optimization problems that are often too challenging for traditional optimization techniques due to their large size, non-linearity, and the presence of multiple local optima [
2]. The primary objective of metaheuristics is to efficiently explore the search space and find near-optimal solutions within a reasonable amount of computational time [
3].
Metaheuristics can be broadly classified into two categories [
4]: single-solution-based and population-based approaches. Single-solution-based metaheuristics, such as simulated annealing and tabu search, iteratively improve a single solution by exploring its neighborhood [
5]. In contrast, population-based metaheuristics, such as genetic algorithms, particle swarm optimization, and ant colony optimization, maintain and evolve a population of solutions over successive iterations [
6].
One of the key strengths of metaheuristics is their flexibility and adaptability [
7]. They are not confined to a specific problem domain and can be tailored to tackle a wide range of optimization problems, including those in engineering, economics, logistics, and artificial intelligence [
8]. Additionally, metaheuristics can be hybridized with other optimization techniques to enhance their performance further, combining the strengths of different methods to create more robust and efficient algorithms [
9].
The success of metaheuristics in finding high-quality solutions to complex problems has made them a popular choice in both academic research and practical applications [
10]. Their ability to balance exploration and exploitation of the search space, avoid getting trapped in local optima, and adapt to various problem landscapes makes them a powerful tool for solving optimization problems that are otherwise intractable by conventional methods [
6].
In the field of optimization, the principle of symmetry is a cornerstone that underpins effective algorithm design [
11]. Symmetry, characterized by balance and harmony, ensures that different components of an optimization process work together seamlessly. Specifically, the symmetry between global and local search processes is crucial. Global search provides a wide-ranging exploration of the solution space, identifying regions that hold potential for optimal solutions [
12]. In contrast, a local search hones in on these regions, conducting detailed and thorough refinement to fine-tune the solutions [
13]. This balanced approach allows optimization algorithms to effectively combine the broad perspective of global search with the precision of local search, leading to improved convergence rates and solution quality [
14]. The harmonious integration of these dual processes exemplifies the power of symmetry in achieving robust and adaptable optimization strategies capable of addressing a wide array of complex, high-dimensional problems [
15]. This fundamental principle of symmetry not only enhances the efficacy of optimization algorithms but also highlights their versatility and applicability across various domains [
9,
16].
In the landscape of optimization algorithms, the introduction of novel approaches is essential to address the ever-evolving complexity of real-world problems [
17]. The OX optimizer is one such novel optimization algorithm that promises to offer a fresh perspective and enhanced performance in solving complex optimization tasks [
18]. The OX optimizer is designed to leverage the strengths of existing metaheuristic frameworks while introducing unique mechanisms to improve search efficiency and solution quality [
19,
20].
The OX optimizer’s application in enhancing the performance of support vector machines (SVMs) for attack detection exemplifies its potential. SVMs are a powerful class of supervised learning models used for classification and regression tasks. However, their performance heavily depends on the careful tuning of hyperparameters, which is an optimization problem in itself. The OX optimizer can effectively search for optimal hyperparameters, thereby enhancing the SVM’s ability to detect attacks in cybersecurity contexts.
Attack detection in cybersecurity is a critical task that requires high accuracy and real-time performance [
21]. Traditional methods often fall short in handling the dynamic and sophisticated nature of modern cyber threats [
21]. By applying the OX optimizer to optimize SVM parameters, it is possible to improve detection rates and reduce false positives, thereby creating more reliable and efficient attack detection systems.
This research aims to explore the theoretical foundations, algorithmic structure, and practical applications of the OX optimizer. We will investigate its performance through extensive computational experiments and compare it with other state-of-the-art optimization algorithms. Furthermore, the application of the OX optimizer in tuning SVM parameters for attack detection will be thoroughly analyzed, demonstrating its practical relevance and potential impact in enhancing cybersecurity measures.
In this paper, we introduce a novel optimization algorithm called the OX optimizer, inspired by oxen animals, designed to address the challenges posed by complex, high-dimensional optimization problems. The OX optimizer leverages a unique approach that combines the strengths of evolutionary algorithms with advanced local search techniques, ensuring both global exploration and fine-tuned exploitation of the search space; the design of the OX optimizer embodies a fundamental symmetry between global and local search processes. This symmetry ensures a balanced and effective exploration of the solution space, highlighting the algorithm’s innovative contribution to the field of optimization. The global search component, driven by evolutionary algorithms, performs a broad exploration of the search space, identifying promising regions that may contain the global optimum. Concurrently, the local search component, employing advanced local search techniques, meticulously refines the solutions within these regions to ensure high precision and optimality. By maintaining a dynamic interplay between these two processes, the OX optimizer achieves a harmonious balance that maximizes both exploration and exploitation, thereby enhancing its overall performance and robustness. This balanced approach allows the OX optimizer to adaptively navigate complex optimization landscapes, making it particularly effective for a wide range of challenging optimization problems.
The main research contributions of this paper are as follows:
A new optimization algorithm named the OX optimizer is designed, inspired by the strength of oxen, to enhance the optimization process.
The OX optimizer’s mathematical models are based on a fundamental symmetry between global and local search processes. The algorithm combines the strengths of evolutionary algorithms with advanced local search techniques, ensuring both broad exploration of the search space and fine-tuned exploitation.
The OX optimizer is implemented and evaluated using unimodal, multimodal, hybrid, and composition functions from the CEC2017 competition benchmark. Experimental results verify that the OX optimizer outperforms existing state-of-the-art algorithms in terms of convergence speed and solution quality.
The effectiveness of the OX optimizer is further demonstrated by its application in optimizing support vector machines (SVMs), emphasizing parameter selection and feature optimization. Specifically, it has been tested on the NSL-KDD dataset to evaluate its efficacy in an intrusion detection system. The results show that the OX optimizer significantly enhances SVM performance, particularly with the RBF kernel, by integrating traits such as strength and collaboration, facilitating effective and thorough exploration of the parameter space.
The rest of this paper is organized as follows: in the Literature Review, we delve into existing research that forms the foundation of our work. We then introduce the OX optimizer, starting with its inspiration drawn from the social behavior of oxen and how this influences its design. The Mathematical Model and the OX Algorithm are discussed in detail. Next, we explore the concepts of exploration and exploitation within the OX optimizer, providing a theoretical conclusion. The Experimental Results present an overview of the CEC2022 and CEC2017 benchmarks, followed by detailed experiments and results, including convergence curve analysis, search history curve analysis, box-plot analysis, heat map, and OX histogram analysis. We also include a case study on optimizing support vector machines (SVM) using the OX optimizer to detect attacks, covering dataset description, data pre-processing, conversion to LibSVM format, optimization using SVM and the OX optimizer, implementation, and evaluation. Finally, we discuss the classification of attacks using the optimized SVM, experiment results, feature optimization, and parameter and feature optimization using the OX optimizer, culminating in our conclusion.
2. Literature Review
Metaheuristics are advanced optimization techniques designed to solve complex optimization problems that are otherwise difficult to tackle using traditional optimization methods [
22]. These algorithms have become indispensable in various fields due to their ability to find near-optimal solutions within a reasonable timeframe, especially for problems characterized by large search spaces, non-linearity, and multimodality [
22]. This literature review delves into the development, classification, and recent advances in metaheuristic algorithms.
The development of metaheuristics dates back to the 1950s and 1960s, with early methods such as genetic algorithms (GA) and simulated annealing (SA) laying the groundwork. John Holland’s work on genetic algorithms in the 1970s was pivotal, introducing concepts of natural selection and genetic evolution to optimization problems [
23]. His seminal book,
Adaptation in Natural and Artificial Systems, laid the foundation for GAs and demonstrated their potential to solve complex problems through mechanisms inspired by biological evolution. Simulated annealing, introduced by Kirkpatrick, Gelatt, and Vecchi in 1983, is another cornerstone in the history of metaheuristics [
24]. Inspired by the annealing process in metallurgy, SA employs a probabilistic technique to escape local optima by allowing transitions to worse solutions with a probability that decreases over time, mimicking the cooling process of metals.
The 1990s saw a surge in the development of new metaheuristic algorithms, including particle swarm optimization (PSO) by Kennedy and Eberhart in 1995 and ant colony optimization (ACO) by Dorigo and colleagues in the early 1990s [
25]. PSO was inspired by the social behavior of birds flocking or fish schooling, where individuals (particles) adjust their positions based on personal experience and the collective experience of the swarm [
26]. ACO, on the other hand, was based on the foraging behavior of ants, where artificial ants build solutions to optimization problems by exploiting a pheromone-based communication system [
27]. In addition, metaheuristics can be broadly classified into two categories: single-solution-based algorithms and population-based algorithms. Single-solution-based algorithms focus on iteratively improving a single solution. These algorithms are typically characterized by their ability to exploit local search areas effectively. Examples include simulated annealing (SA) and tabu search (TS). Simulated annealing utilizes a probabilistic technique to escape local optima by allowing worse solutions temporarily. The probability of accepting worse solutions decreases over time, allowing the algorithm to converge to a near-optimal solution. The cooling schedule, which controls the decrease in temperature, is a crucial factor in the performance of SA. Tabu search employs a memory structure called the tabu list to avoid revisiting recently explored solutions, thus enhancing its search capability. TS systematically explores the solution space by moving from one potential solution to another while keeping track of a list of previously visited solutions to avoid cycles and encourage exploration [
28].
Population-based algorithms work with a population of solutions, facilitating exploration and exploitation of the search space. These algorithms are generally more robust and capable of avoiding local optima due to their collective search strategy. Examples include genetic algorithms (GA), particle swarm optimization (PSO), and ant colony optimization (ACO). Genetic algorithms utilize crossover, mutation, and selection operators to evolve a population of solutions [
29]. Inspired by natural evolution, GA employs a fitness function to select the best individuals for reproduction, promoting the survival of the fittest concept. The crossover operator combines parts of two parent solutions to create offspring, while the mutation operator introduces random changes to maintain genetic diversity. Particle swarm optimization models social behavior, with particles adjusting their positions based on personal and group experience [
25]. Each particle in the swarm is influenced by its own best-known position and the best-known positions of its neighbors, leading to convergence towards optimal solutions through cooperation and information sharing. Ant colony optimization mimics the pheromone-laying behavior of ants to find optimal paths. Artificial ants construct solutions by moving through a problem space and depositing pheromones on promising paths. The collective behavior of ants, guided by pheromone trails, helps in discovering high-quality solutions [
30].
Recent years have witnessed the emergence of hybrid metaheuristics, combining the strengths of different algorithms to overcome their individual limitations. For example, memetic algorithms integrate local search techniques with global search capabilities of metaheuristics, enhancing solution quality [
31]. These hybrid approaches leverage the complementary strengths of different algorithms, resulting in more efficient and effective optimization strategies. Another significant trend is the development of metaheuristics inspired by new natural phenomena. Algorithms such as the firefly algorithm (FA), cuckoo search (CS), and bat algorithm (BA) have been introduced, each inspired by unique natural behaviors [
32]. The firefly algorithm, proposed by Yang in 2008, is inspired by the flashing behavior of fireflies, where the attractiveness of a firefly is proportional to its brightness, guiding other fireflies towards brighter and potentially better solutions. Cuckoo search, developed by Yang and Deb in 2009, is inspired by the brood parasitism of cuckoo species, using Lévy flights to explore the search space efficiently [
33]. The bat algorithm, introduced by Yang in 2010, is based on the echolocation behavior of bats, where bats adjust their positions based on the distance to their prey [
34].
Moreover, the advent of parallel and distributed computing has significantly enhanced the performance of metaheuristic algorithms. Parallel implementations of GA, PSO, and ACO, among others, have been explored to exploit modern computational resources, leading to faster convergence and the ability to handle larger problem instances. Techniques such as master–slave models, island models, and multi-threading have been employed to distribute the computational load and improve scalability [
35]. Furthermore, metaheuristics represent a powerful class of optimization techniques that have evolved significantly over the past few decades [
36]. Their ability to provide high-quality solutions to complex optimization problems has made them a valuable tool in various scientific and engineering domains. Ongoing research and development in hybrid metaheuristics and parallel computing promise to further enhance their performance and applicability, opening new avenues for tackling ever more challenging optimization problems [
37,
38]. A list of state-of-the-art metaheuristics is shown in
Table 1.
3. OX Optimizer
3.1. Inspiration
In nature, oxen are recognized for their great strength, allowing them to carry heavy loads over long distances [
47] (see
Figure 1). This characteristic can be translated into an algorithmic feature where the optimizer robustly handles complex, high-dimensional optimization problems, demonstrating a strong ability to navigate through challenging search spaces. Unlike other animals that might show bursts of speed, oxen are known for their steady and gradual progress [
48]. This can inspire an optimization algorithm that makes consistent, incremental improvements in the solution, avoiding drastic changes that might lead to suboptimal results. This steady approach could be particularly effective in avoiding premature convergence and ensuring thorough exploration of the search space.
Oxen often work in pairs or teams, showcasing a collaborative effort [
48]. This aspect can be incorporated into the algorithm by allowing individual agents (representing oxen) to collaborate or share information, potentially improving the convergence speed and quality of the solutions. This collaboration could mimic the way oxen coordinate their efforts in tasks like plowing. Oxen are adaptable to various environmental conditions, whether it be plowing fields or transporting goods [
49]. This adaptability can be mirrored in the algorithm’s ability to tackle a wide range of optimization problems, from continuous to discrete and from simple to complex, making it a versatile tool for different application domains.
The endurance of oxen, capable of working for long periods without tiring, could inspire an optimization algorithm that performs consistently over extended iterations. This would be particularly useful in problems where the search space is vast and complex, requiring sustained effort to locate the optimal solution.
3.2. Social Behavior of Oxen and Their Influence on the OX Optimizer
The social behavior of oxen is characterized by their strong sense of community, teamwork, and hierarchical structure. Oxen work collaboratively, especially when performing tasks like plowing or hauling, demonstrating collective intelligence and shared efforts [
50]. This cooperative nature is mirrored in the OX optimizer, where agents work together, share information, and follow a leading entity akin to a lead ox guiding the herd. The hierarchical and collaborative dynamics of oxen ensure focused and effective problem-solving, which is essential for the optimization process. Additionally, oxen exhibit resilience and adaptability, traits that are crucial for navigating diverse and complex environments, further informing the design of the OX optimizer.
3.3. Mathematical Model Equations and Description
This section discusses the mathematical model and then lists the equations of the OX optimizer. The mathematical model for the OX optimizer begins with the initialization phase. Let represent a population of n OX agents in the search space, where each is a potential solution vector. Each is initialized randomly within the bounds of the problem’s search space. The next phase involves defining a strength function that evaluates the load-carrying capacity of each OX agent, analogous to the fitness function in other optimizers. This function assesses how well an agent solves the optimization problem. Additionally, a persistence mechanism is implemented to ensure that agents do not fluctuate wildly between solutions, representing the steady nature of oxen. This is mathematically modeled as a damping factor that reduces the step size over iterations.
In the steady and gradual progress phase, a gradual progress function is introduced to modify the position of each OX agent based on the current iteration . This ensures steady movement in the search space, avoiding abrupt changes. The function gradually refines the position of by considering both the current position and a weighted influence from the best solution found so far. Collaborative work among the agents is represented by a collaboration term , where each agent adjusts its position by considering the positions of other agents in the population. This mimics the collaborative nature of oxen and can be modeled as an averaging or weighted sum of the positions of neighboring agents, adjusted by a collaboration coefficient.
The adaptability phase incorporates a mechanism where the algorithm adjusts its parameters based on the type of problem or the characteristics of the search space. This involves dynamically adjusting the strength and collaboration coefficients based on feedback from the environment or the problem’s nature. Endurance and longevity in the OX algorithm are represented by a long-term memory mechanism, where the algorithm stores and occasionally revisits promising solutions found in earlier iterations. This mechanism prevents stagnation in local optima and encourages exploration of the search space over an extended period.
Finally, the positions of the OX agents are updated at each iteration using a combination of strength, gradual progress, collaboration, and endurance mechanisms. The algorithm terminates when a stopping criterion is met, such as reaching a maximum number of iterations or achieving a satisfactory solution quality.
The OX animal optimizer, through this mathematical model, aims to emulate the robust and consistent attributes of oxen, providing a novel approach to optimization problems. This model can be further refined and tested on various optimization problems to assess its effectiveness and efficiency. The OX animal optimizer (OX), inspired by the behaviors of oxen, is designed to effectively balance the crucial aspects of exploration and exploitation in the optimization process. This balance is achieved through various mechanisms that are reflective of the natural attributes of oxen, such as their strength, endurance, gradual progress, and collaborative tendencies.
3.4. Mathematical Model Equations
The positions of the population, referred to as
and
, are initialized randomly within the given bounds, as shown in Equations (
1) and (
2):
where
N is the population size, dim is the dimensionality of the search space, and
and
are the upper and lower bounds of the search space.
3.4.1. Fitness Evaluation
The fitness of each initial population member
is evaluated using the objective function
, as shown in Equation (
3):
The population is then sorted based on fitness, and the best position is identified, as shown in Equations (
4) and (
5):
where
are the sorted population members based on their fitness,
are the indices of the sorted population members,
is the position of the best member, and
is the fitness of the best member.
3.4.2. Main Optimization Loop
For each iteration , the algorithm performs the following steps:
Offspring Generation Using Order Crossover
Two parents are selected using tournament selection, and offspring are generated using order crossover, as shown in Equations (
6) and (
7):
where
and
are the selected parents, and
is the generated offspring.
Boundary Checking
The offspring positions are checked to ensure they remain within the search space bounds, as shown in Equation (
9):
where
is the offspring position after boundary checking.
Fitness Evaluation of Offspring
The fitness of each offspring is evaluated, as shown in Equation (
10):
where
is the fitness of the offspring
.
Updating Population and Best Position
The population is updated by combining and sorting the current population and the offspring based on fitness, as shown in Equations (
11) and (
12):
where
is the combined population of sorted and current offspring, and
is the combined fitness of the sorted and current offspring.
The best position is updated if a fitter individual is found, as shown in Equation (14):
where
is updated to the position of the best individual in the sorted population if its fitness
is better.
Best Solution
The convergence curve is updated to track the best fitness value found so far, as shown in Equation (
15):
where
is the fitness of the best member at iteration
t.
3.5. OX Pseudo-Code and Description
The OX optimizer is an evolutionary optimization algorithm inspired by oxen to simulate their behavior; it combines order crossover and Levy flights to enhance search efficiency, as shown in Algorithm 1. It begins by initializing two populations, OX1 and OX, within specified bounds. The algorithm then evaluates the fitness of the initial OX1 population, sorting them to identify elite individuals. During each iteration, offspring are generated through tournament selection and order crossover of parent solutions, followed by applying Levy flights for mutation-like effects. These offspring undergo boundary checking to ensure they remain within feasible limits. The fitness of the offspring is evaluated and combined with the existing population, from which the best solutions are selected, and the elite position is updated if a fitter solution is found. The convergence curve is updated to reflect the best fitness value obtained in each iteration. This process iterates until the maximum number of iterations is reached, ultimately returning the best fitness value and the corresponding position.
3.6. Exploration and Exploitation in OX Optimizer
Exploration refers to the algorithm’s ability to investigate a wide range of the search space to avoid local optima and discover diverse solutions. In the OX optimizer, exploration is primarily driven by the Levy flights applied to offspring solutions. The Levy flight mechanism introduces large, random jumps in the search space, enabling the algorithm to explore new, unvisited regions effectively. This stochastic component is essential for maintaining diversity in the population and ensuring that the algorithm does not prematurely converge on suboptimal solutions.
Exploitation focuses on refining existing solutions to improve their quality and converge towards the global optimum. In the OX optimizer, exploitation is facilitated by the order crossover process and the selection mechanisms. Order crossover recombines parts of two parent solutions to produce offspring that inherit desirable traits, effectively exploiting the search space around the current best solutions. Additionally, tournament selection ensures that better-performing individuals are more likely to be chosen as parents, further enhancing the exploitation process. The fitness evaluation and sorting of the population, along with the updating of the elite individual, continuously drive the algorithm towards better solutions.
Algorithm 1 A pseudo-code summarizing the optimization process of the OX optimizer. |
Require: Population size N, Maximum iterations , Lower bounds , Upper bounds , Dimensionality dim, Objective function Ensure: Best fitness value , Best position
- 1:
Initialize and positions using ( 1) and ( 2) - 2:
Initialize best position and fitness - 3:
Evaluate fitness of initial population using ( 3) - 4:
Sort based on fitness and identify best using ( 4) and ( 5) - 5:
for to do - 6:
for to N do - 7:
Select two parents using tournament selection ( 6) - 8:
Generate offspring using Order Crossover ( 7) - 9:
Apply Levy flight to offspring ( 8) - 10:
Ensure offspring within bounds ( 9) - 11:
Evaluate fitness of offspring using ( 10) - 12:
end for - 13:
Combine and sort current population and offspring ( 11) and ( 12) - 14:
Update best position if a fitter individual is found (14) - 15:
Update best solution found ( 15) - 16:
end for
|
3.7. Theoretical Discussion about the OX Optimizer
Robustness through strength and endurance: The OX optimizer’s approach to optimization is grounded in robustness, much like the enduring nature of oxen. The strength of each agent in the optimizer, analogous to the fitness measure in evolutionary algorithms, is a critical factor in evaluating and advancing towards optimal solutions. The endurance aspect, mirroring the persistent display of oxen, ensures that the optimizer can sustain its search over long iterations. This endurance is particularly beneficial in complex optimization landscapes where a thorough and persistent search is key to locating the optimal solution.
Adaptability to varied problem spaces: The OX optimizer exhibits a high degree of adaptability, akin to how oxen dynamically change in response to environmental conditions. This adaptability allows the optimizer to adjust its search strategy dynamically based on the specific requirements and characteristics of the problem at hand. Whether the problem is continuous or discrete, simple or complex, the OX optimizer can tailor its approach to suit the specific nature of the problem, thereby enhancing its effectiveness.
Balanced exploration and exploitation: Central to the OX optimizer’s effectiveness is its ability to balance exploration and exploitation, a key challenge in any optimization algorithm. Exploration is driven by Levy flights, introducing large, random jumps that enable the search of new regions in the search space, thus avoiding premature convergence on local optima. Simultaneously, exploitation is facilitated through order crossover and tournament selection, ensuring that promising areas are thoroughly refined. These mechanisms ensure that once a potentially optimal region is identified, the optimizer focuses its efforts on converging towards the best solution.
Collaborative information sharing: The OX optimizer incorporates collaborative mechanisms into its algorithmic structure, akin to the collective behavior observed in natural systems. Agents within the optimizer share information about their positions and experiences, leading to collective intelligence. This collaborative aspect allows the optimizer to benefit from a diverse range of insights and strategies, contributing to a more effective and comprehensive exploration of the search space.
5. Statistical Results and Discussion
5.1. Results over CEC2022
The performance of the OX optimizer compared to other state-of-the-art algorithms over the CEC2022 benchmark suite (F1–F12) demonstrates its superior performance; as shown in
Table 5, the OX optimizer consistently achieves competitive results across different functions. For instance, in F1, the OX achieves a mean score of 3.03
, significantly outperforming CMAES and WOA, which have mean scores of 2.13
and 1.58
, respectively. This trend is observed across several functions, where the OX optimizer ranks among the top performers, often achieving first or second place. In F2, the OX optimizer achieves the best mean score of 4.05
, while in F3, it ranks second with a mean score of 6.08
, closely behind the leading algorithm. The robustness of the OX optimizer is further evident in functions like F6 and F7, where it maintains a leading position with minimal standard deviation, indicating consistent performance. Compared to traditional algorithms such as PSO, AOA, and SCA, the OX optimizer demonstrates lower mean scores and standard errors, highlighting its efficiency and reliability.
5.2. Results over CEC2017
The results of the OX optimizer in comparison to other state-of-the-art optimizers over the CEC2017 benchmark functions (F1–F14) highlight its competitive performance and robustness. As shown in
Table 6, the OX optimizer consistently achieves lower mean values for most functions, indicating superior optimization capability. For example, in F1, the OX optimizer achieves a mean value of 1.50
, outperforming CMAES (6.02
), WOA (2.45
), and other algorithms. Similarly, for F2, the OX optimizer’s mean value of 5.56
is significantly better than the means of CMAES (1.93
) and BOA (1.14
). The standard deviations and standard errors for the OX optimizer are also lower or comparable, suggesting stability and reliability in its optimization process. Furthermore, the OX optimizer shows strong performance in multimodal functions such as F3 and F4, maintaining lower means and ranks compared to competitors like PSO, AOA, and SCA.
Furthermore, OX demonstrates notable performance compared to other state-of-the-art optimizers over the CEC2017 benchmark functions (F15–F30), as evidenced by the results in
Table 7. The OX optimizer frequently achieves lower mean values, indicating superior optimization capabilities. For example, in F15, the OX optimizer has a mean of 2.53
, outperforming other algorithms such as CMAES (4.17
) and WOA (5.86
). Similarly, in F16, the OX optimizer’s mean of 1.91
is competitive with other top-performing algorithms. In multimodal and hybrid functions like F18 and F19, the OX optimizer shows its robustness with significantly lower means, such as 1.31
for F18, compared to CMAES (2.90
) and BOA (4.13
). These results highlight the OX optimizer’s efficiency in handling complex and diverse optimization problems, making it a strong contender against other well-known algorithms such as PSO, AOA, and SCA. The standard deviations and standard errors further affirm the stability and reliability of the OX optimizer’s performance across different test functions.
5.3. OX Convergence Diagram
As can be seen in
Figure 4, the convergence curves for the OX optimizer over CEC2022 (F1 to F6) show an initial improvement followed by stabilization. For F1, the optimizer starts with a high initial value and quickly decreases, then flattens, indicating efficient convergence to a good solution early on. Similar behavior is observed for F2, F3, and F4, where there is a steep drop in the best value obtained early in the iterations, followed by a plateau. For F5, the optimizer demonstrates a rapid decline, followed by a period of stability, indicating that most of the optimization work is done early. F6 exhibits a more gradual decline throughout the iterations, suggesting that this function requires continuous improvements over a longer period before stabilization. Moreover, the OX optimizer shows efficiency in finding good solutions quickly for F1 to F6, with minor refinements occurring later in the process.
Furthermore, as can be seen in
Figure 5 for functions F7 to F12, the OX convergence also displays a pattern of rapid initial improvement but with slight variations in the rate and timing of stabilization. For F7, the optimizer achieves fast convergence to a near-optimal solution. F8 follows a similar pattern, with good convergence performance and the best value obtained, then a long period of stability. The convergence curve for F9 shows a steep decline in the best value within the initial iterations, followed by leveling off, demonstrating efficient early convergence and minimal improvements thereafter. For F10, the optimizer again shows a good convergence performance in the best value obtained, followed by a plateau, indicating that it quickly finds a good solution and maintains stability. Moreover, the OX optimizer demonstrates consistent and efficient performance across F7 to F12, converging to good solutions and maintaining stability throughout the iterations.
The convergence curves for the OX optimizer over the CEC2017 benchmarks (F1–F15) provide a detailed view of its performance. For F1, the curve shows a significant decrease in the best value obtained, with gradual improvements thereafter, indicating fast initial convergence followed by slower refinements. Similarly, F2 exhibits a steep drop early on, followed by a steady state, suggesting the optimizer quickly approaches a near-optimal solution. F3’s convergence curve is quite steep at the beginning and then levels off, showing efficient optimization in the early stages. F4 and F5 both show rapid initial convergence, with minor improvements thereafter, demonstrating the optimizer’s quick adaptability to these functions. F6 follows a similar pattern but with a more pronounced plateau phase, indicating some difficulty in further refining the solution.
5.4. OX Search History Diagram
As it can be seen in
Figure 6, the search history plots for the OX optimizer over selected functions of CEC2017 benchmark functions reveal distinct patterns in its exploration and exploitation behavior. For functions F1–F6, the search history is concentrated around the origin with a high density of points in a circular pattern, indicating thorough exploration within a broad region and gradual convergence towards the optimal solutions. This demonstrates the optimizer’s ability to extensively search the solution space before focusing on more promising areas.
Furthermore, in
Figure 7, for functions F7 and F12, the optimizer’s search appears more linear but also concentrated around the optimal points, suggesting that it quickly narrows down on promising regions but potentially at the cost of broader exploration, indicating balanced exploration across the dimensions with a focus on a central area.
5.5. Box-Plot Analysis
Analyzing the box-plot for the OX optimizer across various functions provides valuable insights into the optimizer’s performance and variability. As can be seen in
Figure 8 and
Figure 9, for F1, the fitness scores show a wide range with significant spread, indicating considerable variability in the optimizer’s performance. F2 follows a similar pattern, though the spread is slightly narrower, indicating a slight improvement in consistency. However, there is still a high median score suggesting room for better optimization. For F3, the spread narrows further, indicating improved consistency, although the median score remains relatively high, showing that there is still variability in achieving lower fitness scores. F4 shows a more pronounced narrowing of the spread, indicating a further improvement in consistency, with the median score slightly lower, reflecting better optimization performance.
F5 displays a wider spread once again, with some outliers, suggesting that the optimizer encounters more difficulty with this function. The median score is high, emphasizing the challenge posed by this function. F6 shows a substantial spread with a higher range of scores, indicating significant variability and difficulty in optimization. In F7, the spread narrows with fewer outliers, reflecting improved consistency, but the median score remains relatively high. F8 shows a similar pattern with a widespread and high median score. F9 exhibits a substantial spread and a high median score, with some outliers indicating variability and difficulty in optimization. Finally, F12 shows a significant spread with a high median score, reflecting variability in performance and the challenge of achieving lower fitness scores. Moreover, the OX optimizer shows variability in performance across different functions, with certain functions posing more significant challenges and resulting in higher median fitness scores and broader spreads. This analysis highlights areas where the optimizer could potentially be improved to achieve more consistent and lower fitness scores across different benchmark functions.
5.6. OX Heat Map Analysis
The sensitivity analysis heatmaps for the functions illustrate how the performance of the optimizer is influenced by varying the number of search agents and the maximum iterations. As can be seen in
Figure 10 and
Figure 11, the heatmap of OX over F10 indicates that the optimizer performs best with around 30 to 40 search agents and 300 iterations, achieving the lowest fitness scores. As the number of search agents increases beyond 40, the performance slightly decreases. F9’s heatmap reveals a consistent performance improvement with increasing iterations, particularly noticeable with 30 and 40 search agents, where the fitness scores are more favorable in the mid-range iterations.
For F8, the optimizer exhibits optimal performance with 30 search agents and 300 iterations, where the fitness scores are minimized. This trend is slightly altered for F7, where the best performance is seen with 20 search agents and around 300 iterations, highlighting a different optimal configuration. F6 shows significant variability, indicating that higher iterations and search agents generally lead to better performance, but there are notable fluctuations. For F5, the optimal configuration is around 40 search agents and 300 iterations, achieving the lowest fitness scores. F4’s heatmap suggests a broader optimal range, with good performance across various combinations, but slightly favoring 30 search agents and 300 iterations. F3 displays a consistent improvement with increasing iterations, particularly with 30 and 40 search agents. F2 and F1 also show similar trends, where mid-range iterations with 30 to 40 search agents generally yield better fitness scores. The heatmaps collectively demonstrate the optimizer’s sensitivity to these parameters, highlighting the importance of tuning for optimal performance.
5.7. OX Histogram Analysis
As can be seen in
Figure 12 and
Figure 13, the OX histograms of the final fitness values for the OX optimizer across CEC2022 benchmark functions provide a comprehensive view of the distribution of final fitness values achieved during optimization runs. For F1, the histogram shows a broad distribution with a significant concentration of values around 2000 to 3000, indicating a tendency to converge around this range. However, there are also a notable number of runs that resulted in much higher fitness values, showing some variability in performance.
For F2, the histogram exhibits a bimodal distribution with peaks around 400 and 450, suggesting that the optimizer often converges to these two different ranges. This bimodal pattern could indicate the presence of multiple local minima or varying algorithm behavior. Similarly, the histogram for F3 shows a predominant concentration around 600 to 605, highlighting a more consistent performance with lesser variability. The histograms for F4, F5, F6, F7, F8, F9, and F12 also show distinct patterns, with F6 showing a wide range of final fitness values, indicating high variability and potential challenges in optimization, while others like F8 and F9 exhibit more concentrated distributions, suggesting more stable convergence behavior. Overall, these histograms reveal the optimizer’s performance consistency and the variability in achieving optimal solutions across different benchmark functions.
6. Case Study: Optimization of SVM Using an OX Optimizer to Detect Attacks
The process of optimizing SVM with the OX optimizer involves four key steps, mirroring the original approach but with modifications to leverage the OX optimizer’s strengths.
6.1. Dataset Description
The NSL-KDD dataset is an improved version of the original KDD Cup 1999 dataset, which is widely used for evaluating intrusion detection systems. The original KDD Cup 1999 dataset has been criticized for various issues, including redundant records that can lead to biased performance evaluations. To address these issues, the NSL-KDD dataset was introduced.
The NSL-KDD dataset contains 41 features for each connection record, along with a label that indicates whether the record is normal or represents a specific type of attack. These features fall into three main categories: basic features, derived from packet headers without inspecting the payload, including attributes like duration, protocol type, service, and flag. Content features are derived from the payload of the packet and include attributes such as the number of failed login attempts. Traffic features are computed using a two-second time window and include attributes like the number of connections to the same host in the past two seconds.
The dataset includes a variety of attack types grouped into four main categories. Denial of service (DoS) attacks and floods a network to disrupt service, with examples including SYN flood attacks. Remote to local (R2L) attacks occur when an attacker sends packets to a machine over the network without having an account on that machine, such as in password guessing attacks. User to root (U2R) attacks involve an attacker starting with access to a normal user account and then gaining root access to the system, like in buffer overflow attacks. Probe attacks involve scanning the network to gather information or find known vulnerabilities, such as port scanning.
The NSL-KDD dataset offers several improvements over the KDD Cup 1999 dataset. It has fewer redundant records, which helps provide a more balanced training and testing dataset. The dataset offers a more balanced distribution of attack and normal records, preventing classifiers from being biased towards more frequent records. Additionally, it contains records of varying difficulty levels, making it more challenging for classifiers and ensuring they generalize better.
The NSL-KDD dataset is divided into two main parts: KDDTrain+ and KDDTest+. KDDTrain+ is used for training models and contains 125,973 records, while KDDTest+ is used for testing models and contains 22,544 records. Each record in the NSL-KDD dataset includes a combination of numeric and categorical features, which can be preprocessed and transformed into a suitable format for machine learning models. The goal of using this dataset is to develop and evaluate intrusion detection systems that can effectively distinguish between normal and malicious network activities.
6.2. Data Pre-Processing
The NSL-KDD training dataset, comprising approximately 4,900,000 single connection vectors, each with 42 features indicating either an attack or normal status, is the starting point. In this step, labels are mapped to numeric values suitable for the SVM algorithm. Normal connections are assigned a target class of ‘zero’, and deviations (i.e., attacks) are assigned a target class of ‘one’. This step also involves filtering and modifying data, such as converting textual items to numeric values and ensuring each connection vector has 41 attributes.
6.3. Conversion to LibSVM Format
The pre-processed datasets are then converted to LibSVM format, with categorical features translated into numeric values. This process involves defining two target classes: ‘zero’ for normal instances and ‘one’ for attacks or intrusions. The converted data is then saved in LibSVM format, consisting of a label indicating the target class, an index for each feature, and the corresponding value of that feature. This step is followed by linear scaling of the datasets to enhance SVM classification performance.
6.4. Optimization Using SVM and an OX Optimizer
Here, the primary modification is the replacement of PSO with the OX optimizer. The NSL-KDD dataset, scaled in the range [0, 1] and formatted in LibSVM, is used for this purpose. The SVM algorithm, with its various kernel functions like linear, RBF, and polynomial, requires optimization of parameters like cost (C) and gamma (g), particularly for the RBF kernel. The OX optimizer, with its attributes of strength, persistence, gradual progress, and collaborative work, is employed to optimize these parameters and features. The endurance and adaptability mechanisms of the OX optimizer make it particularly suitable for navigating through the complex parameter space of the SVM, ensuring that the algorithm consistently moves towards the most promising solutions without premature convergence. The adaptability of the OX optimizer allows it to fine-tune its approach based on the feedback from the ongoing optimization process, dynamically adjusting its parameters to enhance the SVM’s performance.
6.5. Implementation and Evaluation
In this final step, the optimized SVM model, with parameters and features refined by the OX optimizer, is implemented for classification tasks. The effectiveness of this optimized model is evaluated based on its accuracy and ability to classify new data correctly. The endurance aspect of the OX optimizer ensures that the optimization process is thorough, potentially leading to a more robust and reliable SVM model.
6.6. Classification Using SVM Optimized by an OX Optimizer
The classification process using SVM involves training the system with a portion of the data to identify several support vectors that represent these training data. These vectors form the foundation of the SVM model. In this revised approach, the OX optimizer is employed to optimize the parameters
C and
g (as previously denoted in Equations (1) and (2)) and to select the feature subset. This optimization aims to refine and improve the SVM model’s performance. The SVM then classifies unknown datasets using the following input and output data formats:
where
and
Here, represents the training data records, n is the number of samples, m is the input vector, and Y belongs to the category class ‘0’ or ‘1’.
In problems involving linear classification, a hyperplane divides the two categories. The formula for the hyperplane is given as follows:
The categorization can be described by the following conditions:
The classification task involves training and testing data, each consisting of data instances. Each instance in the training set contains one “target value” (class labels: normal or attack) and several “attributes” (features). The goal of the SVM is to produce a model which predicts the target value of data instances in the testing set, given only the attributes. To achieve this, different kernel functions are used. In this experiment, the RBF kernel function is employed.
The formula for the RBF kernel optimization function is given as follows:
The process of finding vectors from the training data is formulated as follows:
Robustness through strength and endurance: The OX optimizer’s approach to optimization is grounded in the robust nature of oxen. The strength of each agent in the optimizer, analogous to the fitness measure in evolutionary algorithms, is a critical factor in evaluating and advancing towards optimal solutions. The endurance aspect, mirroring the ox’s ability to work for extended periods, ensures that the optimizer can sustain its search over long iterations. This endurance is particularly beneficial in complex optimization landscapes where a thorough and persistent search is key to locating the optimal solution.
Adaptability to varied problem spaces: The OX optimizer exhibits a high degree of adaptability, akin to how oxen adapt to different environmental conditions. This adaptability allows the optimizer to adjust its search strategy dynamically based on the specific requirements and characteristics of the problem at hand. Whether the problem is continuous or discrete, simple or complex, the OX optimizer can tailor its approach to suit the specific nature of the problem, thereby enhancing its effectiveness.
Balanced exploration and exploitation: Central to the OX optimizer’s effectiveness is its ability to balance exploration and exploitation, a key challenge in any optimization algorithm. The optimizer’s steady and gradual progress, inspired by the ox’s methodical approach, ensures that new regions of the search space are explored to avoid premature convergence on local optima. Simultaneously, its mechanisms for exploiting promising areas ensure that once a potentially optimal region is identified, the optimizer focuses its efforts on refining and converging towards the best solution.
Collaborative information sharing: The OX optimizer incorporates the collaborative nature of oxen into its algorithmic structure. Agents within the optimizer share information about their positions and experiences, leading to collective intelligence. This collaborative aspect allows the optimizer to benefit from a diverse range of insights and strategies, contributing to a more effective and comprehensive exploration of the search space.
Systematic and incremental improvement: The algorithm’s approach to making incremental improvements in the solution mimics the ox’s steady nature. This systematic progression prevents drastic, suboptimal changes in the solution trajectory and supports a more stable convergence towards the optimal solution.
6.7. Experimental Results
In this experiment, the OX optimizer optimizes the parameters of the RBF SVM and also reduces the features of the training set, effectively removing noisy features. The training set contains 25,149 records, and the testing set includes 11,850 records. The experiment compares different kernel functions of SVM with feature selection in terms of accuracy, as measured by a confusion matrix.
The kernel functions examined include linear, Gaussian, RBF, and polynomial. The results as shown in
Table 8, indicate that the RBF kernel function, with features optimized by the OX optimizer, provides the highest accuracy. This improvement is attributed to the OX optimizer’s strength in handling complex optimization landscapes, its gradual and steady progress in refining solutions, and its collaborative approach to feature selection, all of which contribute to an enhanced SVM model.
6.8. Parameter and Feature Optimization Using an OX Optimizer
The OX optimizer, drawing inspiration from the characteristics of oxen, adopts a unique approach to optimization tasks. In parameter optimization, the goal is to find the optimal set of parameters for an SVM model, typically the parameters C and g. The process begins with the initialization of a population of ‘OX agents’, each representing a potential solution with a position corresponding to a set of parameter values. The fitness of each OX agent is evaluated based on the SVM’s performance, with the aim of minimizing error rates or maximizing accuracy. The strength of each OX agent, analogous to the ox’s load-carrying capacity, is associated with its fitness value. This strength, along with the OX agent’s persistence, emulates the steady and consistent nature of oxen, ensuring gradual changes in parameter values. The steps for OX optimizer in Feature Optimizatio are shown in Algorithm 2.
Collaboration is a key aspect of the OX optimizer, where OX agents share information and adjust their parameters based on collective knowledge. This collaborative behavior leads to a more effective search strategy. The optimizer’s endurance allows for sustained search efforts over time, while its adaptability helps adjust the strategy based on feedback from the environment. The position of each OX agent, representing the SVM parameters, is updated iteratively, and the algorithm continues until a satisfactory solution is found or a predefined number of iterations is reached.
In feature optimization, the focus shifts to selecting the optimal subset of features from a dataset to enhance the SVM’s performance. Each OX agent’s position in this scenario represents a binary-encoded feature set. The initial random selection of features introduces diversity, which is crucial for an effective optimization search. The fitness evaluation here is based on the SVM’s performance with the selected feature set. The OX optimizer employs its strength and collaboration mechanisms to refine the feature selection process. This iterative approach, coupled with the OX optimizer’s endurance and adaptability to feedback, ensures a thorough exploration of the feature space. The process converges upon identifying an optimal or near-optimal feature set or upon meeting a stopping criterion.
Algorithm 2 Feature optimization using an OX optimizer. |
- 1:
Step 1: Initialization - 2:
Define l as a binary string of size 40. - 3:
Step 2: OX Agent Definition - 4:
Define OX agent as a set {position, fitness, bestPosition, bestFitness}. - 5:
Step 3: Initialize Population - 6:
Create an array of OX agents of size max_size, and initialize variables bestGlobalPosition and bestGlobalFitness. - 7:
for each OX agent i from 1 to max_size do - 8:
position = random_string(l) - 9:
writeRandomFeatures(position, Str) ▹ Generate feature.txt from binary string l, where Str is the scaled training dataset - 10:
Fitness = SVMF(feature.txt, Stest, C, g) ▹Stest is the scaled test dataset, and C and g are parameters obtained from parameter optimization - 11:
Update OX agent i with {position, Fitness, position, Fitness} - 12:
if OX agent[i].Fitness < bestGlobalFitness then - 13:
Update bestGlobalFitness and bestGlobalPosition - 14:
end if - 15:
end for - 16:
Step 4: Optimization Loop - 17:
while
do - 18:
for each OX agent j in the population do - 19:
Select two parents using tournament selection - 20:
Generate NewPosition using Order Crossover - 21:
Apply Levy flight to NewPosition - 22:
Ensure NewPosition within bounds - 23:
writeRandomFeatures(NewPosition, Str) - 24:
NewFitness = SVMF(feature.txt, Stest, C, g) - 25:
if NewFitness < OX agent[j].bestFitness then - 26:
Update OX agent[j].bestPosition = NewPosition - 27:
Update OX agent[j].bestFitness = NewFitness - 28:
end if - 29:
if NewFitness < bestGlobalFitness then - 30:
Update bestGlobalPosition = NewPosition - 31:
Update bestGlobalFitness = NewFitness - 32:
end if - 33:
end for - 34:
end while - 35:
return BestFitness and BestPosition
|