MTLBORKS-CNN: An Innovative Approach for Automated Convolutional Neural Network Design for Image Classification

Abstract

Convolutional neural networks (CNNs) have excelled in artificial intelligence, particularly in image-related tasks such as classification and object recognition. However, manually designing CNN architectures demands significant domain expertise and involves time-consuming trial-and-error processes, along with substantial computational resources. To overcome this challenge, an automated network design method known as Modified Teaching-Learning-Based Optimization with Refined Knowledge Sharing (MTLBORKS-CNN) is introduced. It autonomously searches for optimal CNN architectures, achieving high classification performance on specific datasets without human intervention. MTLBORKS-CNN incorporates four key features. It employs an effective encoding scheme for various network hyperparameters, facilitating the search for innovative and valid network architectures. During the modified teacher phase, it leverages a social learning concept to calculate unique exemplars that effectively guide learners while preserving diversity. In the modified learner phase, self-learning and adaptive peer learning are incorporated to enhance knowledge acquisition of learners during CNN architecture optimization. Finally, MTLBORKS-CNN employs a dual-criterion selection scheme, considering both fitness and diversity, to determine the survival of learners in subsequent generations. MTLBORKS-CNN is rigorously evaluated across nine image datasets and compared with state-of-the-art methods. The results consistently demonstrate MTLBORKS-CNN’s superiority in terms of classification accuracy and network complexity, suggesting its potential for infrastructural development of smart devices.

1. Introduction

1.1. Problem Background

Convolutional neural networks (CNNs) have garnered significant attention in recent years within the field of artificial intelligence due to their exceptional performance in addressing diverse real-world tasks such as object recognition [1], image classification [2], defect identification [3], denoising [4], etc. Typically, CNNs consist of convolutional layers, pooling layers, and fully-connected layers. This network architecture exhibits remarkable versatility when dealing with a wide range of data types, including images, audio, and video. One fundamental challenge in these applications is the development of efficient signal preprocessing and feature extraction systems to generate appropriate data structures for specific tasks such as classification. CNNs, however, have alleviated the need for preprocessing in many domains by automatically extracting valuable features. In several cases, the convolutional layers of CNNs can perform this automatic preprocessing task as effectively as human experts could achieve, through the meticulous design process of network architecture.

Nevertheless, crafting meaningful CNN architectures remains a meticulous and labor-intensive process, often requiring the expertise of specialists [5]. This network design process involves identifying the most suitable CNN components, both in terms of architecture and hyperparameters. The performance of CNNs is primarily influenced by two key factors: architectures and trainable parameters (i.e., weights and biases) [6]. While gradient descent algorithms have been effective for optimizing trainable parameters, there are no explicit functions available for directly determining the optimal CNN architecture needed to achieve competitive results on specific datasets [6]. Notable CNN architectures, including AlexNet [7], ResNet [8], VGGNet [9], MobileNet [10], and GoogleNet [11], have demonstrated remarkable accuracy enhancements in computer vision tasks due to their distinctive network architectures characterized by layer count, interlayer connections, and basic unit designs. However, the manual design of these network architectures remains challenging due to the large number of parameters [12]. Notably, these CNN architectures are hand-crafted and cannot autonomously learn optimal configurations, necessitating designers with extensive expert knowledge in CNN architecture design [13]. Furthermore, the optimal design of CNN architecture is problem-specific, and determined by the varying data distributions. These manually crafted architectures lack flexibility, often necessitating time-consuming trial-and-error approaches. Additionally, these manually designed networks may have limited adaptability to diverse datasets, potentially hindering network generalization [6].

To address these challenges and limitations of manually crafted network architectures, there is an increasing demand for automated CNN architecture design methods that reduce reliance on domain-specific human expertise. The goal of these automated methods is to efficiently search for the optimal CNN architectures that can surpass the performance of manually crafted counterparts. The development of automated CNN architecture design approaches, capable of accommodating diverse datasets, represents a critical step in enhancing CNN efficiency and effectiveness. Such automated strategies yield network architectures tailored to specific tasks and dataset requirements, ultimately enhancing CNN generalization capabilities.

1.2. Recent Progress in Network Architecture Design Techniques

Designing optimal CNN architectures for specific datasets has traditionally demanded substantial manual effort. Recent advancements in network architecture design have introduced three main approaches to automate this process: reinforcement learning-based [14,15,16], gradient-based [17,18], and metaheuristic search algorithm (MSA)-based [19,20] methods.

Reinforcement learning-based approaches, such as efficient architecture search (EAS) [15] and BlockQNN [16], have exhibited impressive performance in discovering competitive architectures. EAS employs network transformation techniques to evolve existing model architectures, while BlockQNN employs a reinforcement technique based on Q-Learning, with an epsilon-greedy strategy to balance exploitation and exploration. However, these approaches demand significant computational resources. For instance, EAS and BlockQNN require 5 and 32 graphics processing units (GPUs), respectively, for tasks such as solving CIFAR-10 and ImageNet datasets. Gradient-based methods, such as differentiable architecture search (DARTS) [17], offer higher efficiency compared to reinforcement learning-based strategies, but may yield inconsistent results. Nonetheless, they still require substantial human knowledge and computational resources during the network construction phase.

In contrast, MSA-based approaches present a promising solution by integrating nature-inspired search operators, facilitating the discovery of optimal network architectures without the need for specialized domain expertise. These methods, including particle swarm optimization (PSO), grey wolf optimization (GWO), teaching-learning-based optimization (TLBO), and differential evolution (DE), exhibit robust global search capabilities and find extensive application across various domains [21,22,23,24]. Due to their appealing features, MSA-based techniques have emerged as popular alternatives to conventional design methods, offering researchers a versatile tool to effectively address a wide array of deep learning challenges.

1.3. General Challenges of MSA-Based Network Design Techniques

MSA-based approaches hold great promise for robustly searching for optimal CNN architecture designs tailored to given datasets. However, despite their potential, several fundamental challenges persist. One major challenge is that optimal CNN architectures for different datasets are generally unknown in advance, and can encompass a wide range of architectures and parameters, including the number and type of layers, number of filters, kernel size, pool size, pool stride, and number of neurons. Addressing this challenge requires the adoption of an appropriate encoding strategy that can represent individual solutions as potential CNN architectures with varying network lengths, while maintaining manageable search complexity.

Additionally, integrating effective network constraints into MSA-based approaches during the optimization of CNN architectures is crucial. These constraints prevent the construction of invalid networks, while preserving flexibility in the discovery of novel network architectures. Another challenge associated with population-based MSAs is the significant computational time and resources required to evaluate the effectiveness of each candidate solution representing potential CNN architectures for a given dataset. Therefore, it is imperative to implement a fitness evaluation process with enhanced computational efficiency to make MSAs more practical for optimizing CNN architectures.

While numerous state-of-the-art MSAs inspired by various natural phenomena have emerged due to the no free lunch theorem, their applicability to complex optimization tasks remains relatively unexplored. Although established MSAs such as PSO, GA, and DE have been used to address CNN architecture optimization, there is a critical need to advance the field by exploring the potential of other MSAs in tackling these intricate real-world challenges.

1.4. Drawbacks of Original TLBO in CNN Architecture Optimization

TLBO has recently emerged as a promising approach for optimizing the search for optimal CNN architecture designs based on given datasets [25]. However, this automatic network design method primarily relies on search operators from the original TLBO version, which has certain limitations. One notable concern is that in the original TLBO, all learners are guided by the same exemplars (population mean and teacher solution) during the teacher phase, overlooking potentially valuable information contributed by other population members. While the original TLBO shows promising convergence speed, it is highly susceptible to premature convergence, especially when both the teacher and population mean become trapped in suboptimal regions early in the optimization process.

Moreover, the learner phase of the original TLBO does not emulate a realistic classroom learning scenario, as it confines each learner’s interaction to a randomly selected peer for acquiring new knowledge. Therefore, considering alternative strategies such as self-learning and adaptive interaction with multiple peers during the learner phase could enhance TLBO’s learning capability. Lastly, TLBO uses a greedy selection scheme based solely on the fitness criterion to determine the survival of learners in the subsequent generation. While this scheme is straightforward to implement, it may neglect potentially superior learners that exhibit temporarily poor fitness values, but could substantially improve the overall population quality in long terms. These limitations can hinder the effectiveness of the original TLBO in solving complex tasks, including CNN architecture optimization, due to the imbalance between exploration and exploitation strengths.

1.5. Research Significances and Contributions

This paper introduces a new variant, i.e., modified TLBO with refined knowledge sharing (MTLBORKS), designed to autonomously discover optimal CNN architectures tailored to specific datasets without human intervention. This process involves searching for the optimal combination of network hyperparameters across three vital CNN building blocks: the convolutional block, pooling block, and fully-connected block. Determining the best combinations of these hyperparameters, encompassing layer numbers and types, filter numbers, kernel sizes, pool size and stride, and neuron numbers, contributes to the optimal design of CNN architecture. MTLBORKS incorporates several enhancements throughout its teacher phase, learner phase, and selection scheme, collectively promoting a more balanced exploration and exploitation search.

The key highlights of this study include:

• Introduction of MTLBORKS-CNN, an automatic network design method demonstrating outstanding accuracy in image classification tasks. This approach leverages on the robust global search capabilities of MTLBORKS to autonomously identify the optimal CNN architectures tailored to specific datasets without human intervention.
• The proposed MTLBORKS-CNN method incorporates a comprehensive solution encoding strategy that facilitates the search for optimal network hyperparameters. This encoding strategy enables the construction of robust and innovative CNN architectures with varying network lengths for diverse datasets, while avoiding the generation of infeasible models. To ensure practicality, a fitness evaluation process with reduced computational intensity is implemented.
• In the modified teacher phase of MTLBORKS-CNN, the unique population mean and social exemplar are calculated for each learner by harnessing the valuable information from better-performing learners through a social learning concept. This individualized approach enhances the potential of learners for discovering novel CNN architectures, while preserving population diversity.
• The modified learner phase of MTLBORKS-CNN integrates two innovative strategies: self-learning and adaptive peer learning, aiming to enhance the knowledge of learners effectively. The self-learning mechanism promotes exploration, empowering each learner to independently search for new CNN architectures through personal efforts. Conversely, the adaptive peer learning encourages exploitation by facilitating knowledge sharing among learners through adaptive interactions with multiple peers based on their fitness values during the search for optimal CNN architectures.
• MTLBORKS-CNN integrates a dual-criterion selection scheme that comprehensively evaluates learners for their survival in the next generation. This scheme considers the fitness and diversity values of learners, reducing the risk of premature convergence. It ensures that learners with the relatively lower fitness but promising diversity values are not prematurely excluded, allowing them to contribute to the long-term enhancement of the overall population’s quality.
• This study conducts thorough simulation studies on nine image datasets derived from MNIST variations to meticulously evaluate the efficacy and feasibility of MTLBORKS-CNN in automatically discovering optimal CNN architectures. The results highlight the considerable merit of MTLBORKS-CNN, as the produced optimal CNN architectures consistently outperform state-of-the-art methods across most of the datasets.

This paper is structured as follows: Section 2 provides an overview of prior research efforts. Section 3 explains the search mechanisms used by MTLBORKS-CNN to autonomously discover optimal CNN architectures. Section 4 presents detailed performance assessments conducted on nine distinct image datasets derived from MNIST variations. Lastly, Section 5 concludes the paper and outlines avenues for future research.

2. Related Works

2.1. Conventional TLBO

TLBO was originally devised to tackle mechanical design problems, drawing inspiration from the pedagogical processes observed in real classrooms [26]. Let N represent the population size, and D is the number of decision variables to be optimized for a given problem. In the initialization stage of TLBO, a group of N learners, each with D dimensions, is randomly generated. Each learner is represented by a position vector $X_n=\left[X_{n,1},\dots ,X_{n,d},\dots ,X_{n,D}\right]$, indicating a potential solution for the given problem. Here, n denotes the learner index, and d indicates the dimension index. The knowledge level of each learner is assessed based on its fitness value, denoted as $F(X_n)$.

The optimization process of TLBO consists of two phases, each incorporating unique learning mechanisms to enhance the learners’ knowledge. In the teacher phase, each n-th learner acquires new knowledge by considering the differences between the most knowledgeable teacher within the population, represented as $X^{Teacher}$, and the population mean, denoted as $\overline{X}$, which reflects the collective knowledge level of the classroom. The value of $\overline{X}$ is determined by averaging the position vectors of all individual learners in the population, as depicted below:

$$\overline{X}=\frac{1}{N}{\displaystyle \sum }_{n=1}^N{X}_n$$

(1)

For each n-th learner, a new position denoted as $X_n^{New}$ is computed using Equation (2). This equation involves a random number, $r_1\in \left[0,1\right]$, generated from a uniform distribution, and a teacher factor, $F^T\in \left\{0,1\right\}$, which dictates the extent to which mainstream knowledge influences the learning process, i.e.,

$$X_n^{New}=X_n+r_1\left(X^{Teacher}-F^T\overline{X}\right)$$

(2)

In the learner phase, the knowledge of the n-th learner $X_n$ can be improved through interaction with a randomly selected peer learner, denoted as $X_m$, where $m\ne n$ and $m=1,\dots ,N$. In the context of minimization problems, smaller values of $F(X_n)$ and $F(X_m)$ indicates better levels of knowledge among the learners. For each n-th learner, a new position $X_n^{New}$ is computed using Equation (3), which is based on a random number, r₂ ∈ [0,1], generated using the uniform distribution. Equation (3) demonstrates that the n-th learner has the option to either move away from the m-th learner when the latter has inferior fitness to encourage exploration, or move closer to the m-th learner when the later has superior fitness to facilitate exploitation.

$$X_n^{New}=\{\begin{array}{c}{X}_n+r_2\left(X_n-X_m\right), \mathrm{if} F(X_n)<F(X_m) \\ X_n+r_2\left(X_m-X_n\right), \mathrm{otherwise} \end{array}$$

(3)

Upon completion of either the teacher or learner phase, the fitness value corresponding to the new position of the n-th learner, denoted as $F\left(X_n^{New}\right)$, is compared to the current value of $F(X_n)$. If $F(X_n^{\mathit{New}})<F(X_n)$, the new values of $X_n^{\mathit{New}}$ and $F\left(X_n^{New}\right)$ replace the current values of $X_n$ and $F(X_n)$, respectively. Otherwise, $X_n^{New}$ and $F\left(X_n^{New}\right)$ are discarded. The iterative learning processes of each learner within both the teacher and learner phases of TLBO continue until the predefined termination criteria are met, at which point $X^{Teacher}$ is returned as the optimal solution.

2.2. Existing MSA-Based Approaches for CNN Optimization

Over the past few decades, various neuroevolutionary approaches have emerged with the goal of optimizing both the hyperparameters and architectures of ANNs. Their aim is to reduce the computational time required for training ANNs compared to conventional backpropagation methods. NeuroEvolution of Augmenting Topologies (NEAT) is recognized as a population-based variant of Topology and Weight Evolving Artificial Neural Networks (TWEANNs) [27]. NEAT mimics natural evolution by evolving from a basic structure to a more complex one, while adjusting connection weights. It introduced a mechanism called speciation to maintain diversity in neural networks while evolving toward more complex structures. Another TWEANN, Evolutionary Acquisition of Neural Topologies (EANT) [28] consists of two layers: (i) an outer layer for structural exploration through mutation, and (ii) an inner layer for structural exploitation by optimizing parameters within the current structure using an evolution strategy to enhance network performance. Hypercube-Based NeuroEvolution of Augmenting Topologies (HyperNEAT) [29] was introduced to address limitations in NEAT’s direct encoding strategy for representing complex network structures. HyperNEAT uses an indirect solution representation scheme known as Compositional Pattern Producing Networks (CPPN) to generate complex networks with multimillions of connections. However, HyperNEAT encountered challenges in image classification [30]. Although most TWEANN approaches show promise in optimizing ANN architectures and hyperparameters, they still suffer with certain limitations, including high computational demands and inefficiencies in encoding complex structures.

Metaheuristic search algorithms (MSAs) are widely utilized for optimizing the hyperparameters and architectures of CNNs. One prominent MSA is genetic algorithm (GA), extensively applied to various optimization and deep learning problems. In 2016, GA was employed to discover an optimized CNN structure for face recognition tasks, where a support vector machine replaced the final CNN layer [31]. Another application involved using GA to optimize a multichannel CNN for stock market prediction [32]. This optimization targeted the hyperparameters of the CNN’s feature extraction module, including pooling size, kernel size, and kernel numbers, which were encoded into a chromosome structure and optimized through GA. In 2021, a recent GA variant, called the two-level GA (2LGA) emerged [33]. This variant focuses on optimizing CNN hyperparameters and architectures for pattern recognition tasks. The 2LGA method involves partial training and evaluation of multiple individuals with fewer epochs in the first level, followed by full training of the best-performing individual from the first level with more epochs in the second level. This approach resulted in significantly improved CNN models for solving datasets such as six MNIST variants and CIFAR-10.

Particle swarm optimization (PSO) has gained prominence as a swarm intelligence algorithm for optimizing CNNs in recent years. In 2018, an Improved PSO (IPPSO) [34] was introduced to enhance CNN solution diversity. IPPSO utilized a variable-depth encoding scheme inspired by the internet protocol address to optimize both CNN architecture and network parameters for image classification. While the CNN model optimized by IPPSO exhibited competitive performance on three MNIST-variants datasets, its encoding scheme introduced prolonged computational time due to frequent parameter value conversions between decimal and binary representations. Another novel PSO variant, called OLPSO [35], was proposed to fine-tune the dropout rate and minibatch size for ensemble classifiers comprising two pretrained CNNs: VGGNet-16 and VGGNet-19 [9]. Additionally, OLPSO integrated a learning rate with an exponential decay characteristic to optimize model training. The model optimized by OLPSO exhibited superior performance over Inception-v3 and Xception in diagnosing plant diseases. In 2021, Jayanthi et al. [36] devised a PSO-CNN model for detecting and classifying diabetic retinopathy from color fundus images. PSO-CNN was employed for noise reduction and feature extraction to isolate relevant features, which were subsequently input into a decision tree for classification. PSO-CNN outperformed several pretrained models, including AlexNet [7], M-AlexNet [7], VGGNet-S, VGGNet-16 [9], VGGNet-19 [9], GoogleNet [11], and ResNet [8], achieving the specificity, sensitivity, and accuracy values of 98.63%, 94.80%, and 97.75%, respectively.

Grey wolf optimization (GWO) stands out as a robust swarm intelligence algorithm that has gained popularity for optimizing CNNs in recent years. In a study by [37], a CNN-based transmission line protection scheme was proposed to detect and classify faults arising from power swings, both with symmetrical and asymmetrical characteristics. GWO was employed to finely tune the optimal combination of CNN parameters, including the depth and filter size of convolutional layers, the maximum size of the pooling layer, and the dropout rate. In 2020, a new GWO variant [38] was introduced, incorporating a nonlinear exploration scheme, a chaotic leadership dispatching strategy, local exploitation with a rectified spiral, and leader enhancement based on probability distribution. This variant aimed to optimize the CNN-LSTM model for enhanced performance in addressing diverse time series prediction and classification problems. In 2022, a GWO-based CNN (GWOCNN) [39] was proposed for the detection of skin cancer. GWOCNN incorporated GWO to determine the optimal combination of CNN hyperparameters for achieving competitive performance. A direct encoding strategy was employed to encode the CNN hyperparameters, which include kernel numbers and sizes of convolutional layers, kernel size of the pooling layer, dropout rate, and number of dropout layers. Each of these parameters was mapped onto distinct dimensional components of each GWO agent. GWOCNN exhibited a faster convergence rate compared to GA-based and PSO-based CNN optimization methods during the optimization process, highlighting GWO’s excellent global search ability and high convergence rate.

3. Proposed MTLBORKS-CNN

This study introduces MTLBORKS-CNN as an automatic technique for designing optimized CNN architectures for image classification tasks. The proposed approach eliminates the need for domain-specific expertise from humans. Figure 1 provides an overview of the MTLBORKS-CNN framework, and specific modifications within MTLBORKS-CNN will be detailed in subsequent subsections.

3.1. Overview of CNN Architecture

Figure 2 depicts a typical sequential CNN architecture, comprising a feature extraction module with two convolutional layers and two pooling layers, along with a trainable classification module consisting of three fully-connected layers. Each CNN layer is defined by specific network hyperparameters, vital for network construction and training, as detailed in the following subsections.

3.1.1. Convolutional Layer

CNNs utilize two types of convolution processes: SAME convolution and VALID convolution. SAME convolution produces feature maps with the same dimensions as the input data by using zero padding. In contrast, VALID convolution generates smaller feature maps without any padding. Convolutional blocks employ filters with predetermined height and width to generate feature maps from input data.

In the convolution process, a filter horizontally slides from left to right with a specified stride width, then vertically moves downward with a stride height, repeating the left-to-right process to form a complete feature map. Feature map elements are computed by summing the product of filter elements and corresponding input elements covered by the filter. The key network hyperparameters of convolutional layer considered for CNN architecture optimization in this study include: number of convolutional layers ($N^{Conv}$), number of filters in each l-th convolutional layer ($N_l^{Fil}$) and kernel size (i.e., $row\times column$) of filter in each l-th convolutional layer ($S_l^{Ker}$).

3.1.2. Pooling Layer

Pooling is critical in CNNs for achieving local translation invariance and results in down-sampled feature maps that exhibit increased robustness to variations in feature locations within input data. There are two common pooling types: average pooling and maximum pooling. Average pooling computes the mean values of elements within a kernel to create down-sampled feature maps. Meanwhile, maximum pooling identifies the largest values among the captured elements.

Pooling involves applying a kernel with predefined type, height, and width to input data. Down-sampled feature maps are generated by sliding the kernel from the top-left to the bottom-right, guided by predetermined stride height and width. The network hyperparameters of pooling layer considered for CNN architecture optimization in this study include: selection probability of pooling type connected to each l-th convolutional layer, ($P_l^{Pool}$), kernel size (i.e., $row\times column$) of pooling layer connected to each l-th convolutional layer ($S_l^{Pool}$), and stride size (i.e., $row\times column$) of pooling layer connected to each l-th convolutional layer, $S_l^{Str}$.

3.1.3. Fully-Connected Layer

The feature extraction module, comprising convolutional and pooling layers, extracts relevant features from raw data. Once extracted, these features can be fed into a classifier, often a fully-connected layer used for classification. To introduce the output feature maps to the fully-connected layer, the feature maps must be flattened and reshaped into a vector.

The CNN’s classification module can consist of one or multiple fully-connected layers, comprising neurons that process input to generate an output. These layers receive inputs from neurons in the preceding layers via connections with assigned weights. Generally, the fully-connected layers are used alongside backpropagation to learn network weights. CNN training aims to minimize errors between predicted and actual dataset outputs via the reduction of cross-entropy loss. The network hyperparameters of fully-connected layers considered for CNN architecture optimization in this study include: number of fully-connected layer ($N^{FC}$) and number of neurons in the q-th fully-connected layer ($N_q^{Neu}$).

3.2. Solution Encoding Scheme of MTLBORKS-CNN

Once the network hyperparameters crucial for optimizing the CNN architecture are identified, an efficient solution scheme is designed for MTLBORKS-CNN, as depicted in Figure 1. This scheme allows each learner to encode vital network hyperparameters necessary for generating optimal CNN architectures. These encoded hyperparameters encompass network length, layer types, filter numbers, kernel sizes, pool sizes, pool strides, and neuron numbers. The solution encoding approach in MTLBORKS-CNN is meticulously designed to prevent the generation of invalid network architectures, while maintaining the flexibility required to discover effective CNN architectures for diverse image classification tasks.

Figure 3 illustrates D-dimensional position vector $X_n$ of the n-th MTLBORKS-CNN learner. Each d-th dimension ($X_{n,d}$) in this vector contains essential information for constructing a unique CNN architecture, including layer types and their corresponding hyperparameters. The information encoded in $X_n$ is categorized into three main sections: convolutional, pooling, and fully-connected.

Table 1. Summary of network hyperparameters to be optimized in each section of CNN.

Sections	Network Hyperparameters to Be Optimized	Data Type	Encoded Dimension Index	Index Number	Lower Limit	Upper Limit
Convolutional Layer	$\mathrm{Number}\mathrm{of}\mathrm{convolutional}\mathrm{layers},N^{Conv}$	Integer Value	$d=1$	-	$N_{min}^{Conv}$	$N_{max}^{Conv}$
	Number of filters in each l-th convolutional layer, $N_l^{Fil}$	Integer Value	$d=2l$	$l\in \left\{1,N_{max}^{Conv}\right\}$	$N_{min}^{Fil}$	$N_{max}^{Fil}$
	$\mathrm{Kernel}\mathrm{size}(\mathrm{i}.\mathrm{e}.,row\times column$) of filter in each l-th convolutional layer, $S_l^{Ker}$	Integer Value	$d=2l+1$	$l\in \left\{1,N_{max}^{Conv}\right\}$	$S_{min}^{Ker}$	$S_{max}^{Ker}$
Pooling Layer	Selection probability of pooling type connected to each l-th convolutional layer, $P_l^{Pool}$	Real-Value	$d=2N_{max}^{Conv}+3l-1$	$l\in \left\{1,N_{max}^{Conv}\right\}$	0	1
	$\mathrm{Kernel}\mathrm{size}(\mathrm{i}.\mathrm{e}.,row\times column$) of pooling layer connected to each l-th convolutional layer, $S_l^{Pool}$	Integer Value	$d=2N_{max}^{Conv}+3l$	$l\in \left\{1,N_{max}^{Conv}\right\}$	$S_{min}^{Pool}$	$S_{max}^{Pool}$
	$\mathrm{Stride}\mathrm{size}(\mathrm{i}.\mathrm{e}.,row\times column$) of pooling layer connected to each l-th convolutional layer, $S_l^{Str}$	Integer Value	$d=2N_{max}^{Conv}+3l+1$	$l\in \left\{1,N_{max}^{Conv}\right\}$	$S_{min}^{Str}$	$S_{max}^{Str}$
Fully-Connected Layer	Number of fully-connected layers, $N^{FC}$	Integer Value	$d=5N_{max}^{Conv}+2$	-	$N_{min}^{FC}$	$N_{max}^{FC}$
	Number of neurons in the q-th fully-connected layer, $N_q^{Neu}$	Integer Value	$d=\left(5N_{max}^{Conv}+2\right)+q$	$q\in \left\{1,N_{max}^{FC}\right\}$	$N_{min}^{Neu}$	$N_{max}^{Neu}$

summarizes the attributes of network hyperparameters to optimize in each section, such as data type, encoded dimension index, index number (if applicable), lower limit, and upper limit. Notably, the pooling layer hyperparameters, $P_l^{Pool}\in \left[0,1\right]$, encoded in dimensions $d=2N_{max}^{Conv}+3l-1$, where $l=1,\dots ,N_{max}^{Conv}$, indicate the types of pooling layers connected to each l-th convolutional layer. Specifically, (a) no pooling layer for $0\le P_l^{Pool}<1/3$, (b) maximum pooling for $1/3\le P_l^{Pool}<2/3$, and (c) average pooling for $P_l^{Pool}\ge 2/3$. With predefined $N_{max}^{Conv}$ and $N_{max}^{FC}$ values, the total dimensional size of $X_n$ for each n-th learner is $D=5N_{max}^{Conv}+N_{max}^{FC}+2$.

Referring to the solution encoding scheme depicted in Figure 3 and detailed in Table 1, Algorithm 1 has been devised to decode the network hyperparameters contained within each learner and transform it into a valid CNN architecture. It is worth noting that despite all position vectors $X_n$ share the same D-dimensional size for $n=1,\dots ,N$, each n-th MTLBORKS learner can generate CNNs of varying network lengths based on values $N^{Conv}\in \left\{N_{min}^{Conv},N_{max}^{Conv}\right\}$, $P_l^{Pool}\in \left[0,1\right]$, and $N^{FC}\in \left\{N_{min}^{FC},N_{max}^{FC}\right\}$ encoded in different dimensional indices, as indicated in Table 1. For instance, when $N^{Conv}<N_{max}^{Conv}$, only the first $N^{Conv}$ pieces of information (i.e., $N_l^{Fil}$, $S_l^{Ker}$, $P_l^{Pool}$, $S_l^{Pool}$ and $S_l^{Str}$) for $l=1,\dots ,N^{Conv}$ are utilized for constructing the convolutional and pooling sections of CNN. Information pertaining to the remaining convolutional and pooling sections of the CNN, specifically for $l=N^{Conv}+1,\dots .,N_{max}^{Conv}$, are disregarded during network construction. It should be noted that information pertaining to $S_l^{Pool}$ and $S_l^{Str}$ are also omitted when the corresponding value of $P_l^{Pool}$ falls within the range of $\left[0,1/3\right]$. This indicates that no pooling layer is linked to the l-th convolutional layer under this condition. Similarly, when $N^{FC}<N_{max}^{FC}$, only the first $N^{FC}$ pieces of information (i.e., $N_q^{Neu}$), for $q=1,\dots ,N^{FC}$, are used to construct the fully-connected layer of CNN. Any remaining information related to the fully-connected layer, for $q=N^{FC}+1,\dots .,N_{max}^{FC}$, is omitted.

Algorithm 1: Decoding Learner to CNN Architecture
Input:$X_n$, $N_{max}^{Conv}$
01:	Initialize an empty CNN architecture;
02:	Extract $N^{Conv}$ and $N^{FC}$ encoded in dimensions $d=1$ and $d=5N_{max}^{Conv}+2$ of learner $X_n$, respectively;
03:	Initialize the indices of convolutional layer and fully-connected layer as $l\leftarrow 1$ and $q\leftarrow 1$, respectively;
04:	while $l\le N^{Conv}$ do /*only the first  $N^{Conv}$ pieces of information are used to construct the convolutional and pooling sections*/
05:	Extract $N_l^{Fil}$ and $S_l^{Ker}$ encoded in dimensions $d=2l$ and $d=2l+1$ of learner $X_n$, respectively;
06:	Append the l-th convolutional layer with $N^{Conv}$ and $N^{FC}$ to the CNN architecture;
07:	Extract $P_l^{Pool}$, $S_l^{Pool}$ and $S_l^{Str}$ encoded in dimensions $d=2N_{max}^{Conv}+3l-1$, $d=2N_{max}^{Conv}+3l$ and $d=2N_{max}^{Conv}+3l+1$
	of learner $X_n$, respectively;
08:	if $0\le P_l^{Pool}<1/3$ then
09:	No pooling layer is appended to the l-th convolutional layer of CNN architecture;
10:	elseif $1/3\le P_l^{Pool}<2/3$ then
11:	Append a maximum pooling layer with $S_l^{Pool}$ and $S_l^{Str}$ to the l-th convolutional layer of CNN architecture;
12:	else /*when $P_l^{Pool}\ge 2/3$*/
13:	Append an average pooling layer with $S_l^{Pool}$ and $S_l^{Str}$ to the l-th convolutional layer of CNN architecture;
14:	end if
15:	$l\leftarrow l+1$;
16:	end while
17:	while $q\le N^{FC}$ do /*only the first  $N^{FC}$ pieces of information are used to construct the fully-connected section
18:	Extract $N_q^{Neu}$ encoded in dimension $d=\left(5N_{max}^{Conv}+2\right)+q$ of learner $X_n$, respectively;
19:	Append the q-th fully-connected layer with $N_q^{Neu}$ to the CNN architecture;
20:	$q\leftarrow q+1$;
21:	end while
Output: A valid CNN architecture corresponds to learner $X_n$

Table 2. Maximum and minimum values of CNN network hyperparameters.

Sections	Network Hyperparameters	Values
Convolutional	Minimum number of convolutional layers $N_{min}^{Conv}$	1
	Maximum number of convolutional layers $N_{max}^{Conv}$	3
	$\mathrm{Minimum} \mathrm{filter} \mathrm{numbers} N_{min}^{Fil}$	3
	$\mathrm{Maximum} \mathrm{filter} \mathrm{numbers} N_{max}^{Fil}$	256
	$\mathrm{Minimum} \mathrm{kernel} \mathrm{size} S_{min}^{Ker}$	$3\times 3$
	$\mathrm{Maximum} \mathrm{kernel} \mathrm{size} S_{max}^{Ker}$	$9\times 9$
Pooling	$\mathrm{Minimum} \mathrm{pooling} \mathrm{size} S_{min}^{Pool}$	$1\times 1$
	$\mathrm{Maximum} \mathrm{pooling} \mathrm{size} S_{max}^{Pool}$	$3\times 3$
	$\mathrm{Minimum} \mathrm{stride} S_{min}^{Str}$	$1\times 1$
	$\mathrm{Maximum} \mathrm{stride} S_{max}^{Str}$	$2\times 2$
Fully connected	Minimum number of fully-connected layers $N_{min}^{FC}$	1
	Maximum number of fully-connected layers $N_{max}^{FC}$	2
	$\mathrm{Minimum} \mathrm{neuron} \mathrm{numbers} N_{min}^{Neu}$	1
	$\mathrm{Maximum} \mathrm{neuron} \mathrm{numbers} N_{max}^{Neu}$	300

provides an overview of the feasible search ranges for network hyperparameters that are subject to CNN architecture optimization, and the total dimension of for each learner is determined as $D=5N_{max}^{Conv}+N_{max}^{FC}+2=19$. As depicted in Figure 4, Algorithm 1 demonstrates the capability to construct different valid CNN architectures based on the unique combination of network hyperparameters encoded within each MTLBORKS-CNN learner. This holds true, provided that these network hyperparameters encoded in each MTLBORKS-CNN learner fall within their predefined boundary limits.

3.3. Population Initialization of MTLBORKS-CNN

Algorithm 2 outlines the initialization process of the MTLBORKS-CNN population. During this stage, diverse candidates of CNN architectures are created by randomly generating position vectors, $X_n$ for N learners, where $n=1,\dots ,N$. The dimension size of each $X_n$ is $D=5N_{max}^{Conv}+N_{max}^{FC}+2$, considering the predefined values of $N_{max}^{Conv}$ and $N_{max}^{FC}$.

For each potential CNN architecture represented by the n-th learner, network hyperparameters for convolutional, pooling, and fully-connected sections are randomly generated within their feasible search ranges in $X_{n,d}$, where $d=1,\dots ,D$. For example, the convolutional section’s hyperparameters, such as $N^{Conv}\in \left\{N_{min}^{Conv},N_{max}^{Conv}\right\}$, $N_l^{Fil}\in \left\{N_{min}^{Fil},N_{max}^{Fil}\right\}$, and $S_l^{Ker}\in \left\{S_{min}^{Ker},S_{max}^{Ker}\right\}$, are initialized in dimensions $d=1$, $d=2l$, and $d=2l+1$, where $l=1,\dots ,N_{max}^{Conv}$. The remaining network hyperparameters are initialized based on the solution encoding scheme and attributes of network hyperparameters described in Figure 3 and Table 1, respectively.

Algorithm 2: Population Initialization of MTLBORKS-CNN
Input:$N, N_{min}^{Conv}$$, N_{max}^{Conv}$$, N_{min}^{Fil}$$, N_{max}^{Fil}$$, S_{min}^{Ker}$, $S_{max}^{Ker}$$, S_{min}^{Pool}$, $S_{max}^{Pool}$$, S_{min}^{Str}$$, S_{max}^{Str}$, $N_{min}^{FC}$, $N_{max}^{FC}$, $N_{min}^{Neu}$, $N_{max}^{Neu}$
01:	Calculate the total dimension size of $D=5N_{max}^{Conv}+N_{max}^{FC}+2$;
02:	Initialize teacher solution, i.e., $X^{Teacher}\leftarrow \varnothing$$\mathrm{and} F\left(X^{Teacher}\right)\leftarrow \infty$;
03:	for n = 1 to N do
04:	$\mathrm{Initialize} X_n\leftarrow \varnothing$;
05:	for d = 1 to D do /*Initialize each dimension of learner*/
06:	Identify the types of network hyperparameter encoded in $X_{n,d}$;
07:	Randomly initialize the network hyperparameter in $X_{n,d}$ based on its corresponding attributes described in Table 1;
08:	end for
09:	$\mathrm{Evaluate} \mathrm{fitness} \mathrm{of} X_n$$\mathrm{as} F\left(X_n\right)$ based on Algorithm 3;
10:	$\mathbf{if} F\left(X_n\right)<F\left(X^{Teacher}\right)$ then
11:	$X^{Teacher}\leftarrow X_n$$, F(X^{Teacher})\leftarrow F\left(X_n\right)$;
12:	end if
13:	end for
Output:  $P=\left[X_1,\dots ,X_n,\dots .,X_N\right]$$, X^{Teacher}$$, F(X^{Teacher})$

Once $X_n$ is generated for each n-th learner, its fitness value in classification error, $F\left(X_n\right)$, is determined through a fitness evaluation process detailed in the next section. This initialization is performed for all N learners, forming an initial population $P=\left[X_1,\dots ,X_n,\dots ,X_N\right]$. The learner with the lowest classification error becomes the teacher solution, represented by its position vector $X^{\mathit{Teacher}}$ and fitness $F\left(X^{\mathit{Teacher}}\right)$.

3.4. Fitness Evaluation of MTLBORKS-CNN

Each MTLBORKS-CNN learner possesses a position vector representing a potential CNN architecture for solving a given problem. The learner’s fitness is evaluated based on the corresponding classification error of this CNN. Learners with lower error rates in classifying datasets are considered to have superior fitness, and vice versa. In this study, the automatic network architecture design problem is formulated as a minimization problem. The primary goal of MTLBORKS-CNN is to identify the optimal CNN model that solves the assigned tasks with the fewest classification errors. Algorithm 3 outlines the two major stages of fitness evaluation for each MTLBORKS-CNN learner, involving (i) constructing and training a potential CNN architecture with the training set, and (ii) evaluating the trained CNN architecture using the validation set.

Algorithm 3: Fitness Evaluation of MTLBORKS-CNN
Inputs: $X_n$, ${ℛ}^{train}$, ${ℛ}^{valid}$, $S^{batch}$, ${\epsilon }^{train}$, $R^L$, $C^{num}$
01:	Compile a full-fledged CNN architecture based on the network information extracted from $X_n$ using Algorithm 1 and a fully-connected layer with $C^{num}$ output neurons added as the last layer of the CNN;
02:	Compute ${\tau }^{train}$ and ${\tau }^{valid}$ using Equations (4) and (6), respectively;
03:	Initialize the weights of compiled CNN model as $\mathit{\varpi }=\left\{{\varpi }_1,{\varpi }_2,\dots \right\}$ using He Normal initializer;
04:	for  $\epsilon =1$ to ${\epsilon }^{train}$ do /*First stage of fitness evaluation as explained in Section 3.4.1*/
05:	$\mathbf{for} i=1$ $\mathrm{to} {\tau }^{train}$ do
06:	Calculate the cross-entropy function of CNN model based on $\mathit{\varpi }$ and the i-th batch data ${ℛ}_i^{train}$$\mathrm{as} f_{CE}\left(\mathit{\varpi },{ℛ}_i^{train} \right)$;
07:	$\mathrm{Update} \mathrm{the} \mathrm{weights} {\mathit{\varpi }}^{\mathit{n}\mathit{e}\mathit{w}}=\left\{{\varpi }_1^{new},{\varpi }_2^{new},\dots \right\}$ based on Equation (5);
08:	end for
09:	end for
10:	for  $j=1$$\mathrm{to} {\tau }^{valid}$ do /*Second stage of fitness evaluation as explained in Section 3.4.2*/
11:	Use the trained CNN model to classify ${ℛ}_j^{valid}$ dataset;
12:	Calculate the classification errors of trained CNN model based on the j-th batch data ${ℛ}_j^{valid}$$\mathrm{as} Err\_Batc{h}_j$;
13:	end for
14:	Calculate the mean classification error of the CNN constructed based on $X_n$, using Equation (7), to obtain its fitness value $F\left(X_n\right)$;
Output: $F\left(X_n\right)$

3.4.1. Stage 1: Construction and Training of Potential CNN Architecture

When evaluating the fitness of each n-th learner, a CNN architecture is constructed using Algorithm 1 based on network hyperparameters decoded from the corresponding position vector, $X$. These hyperparameters include $N^{Conv}$, $N_l^{Fil}$, $S_l^{ker}$, $P_l^{Pool}$, $S_l^{Pool}$, $S_l^{Str}$, $N^{FC}$, and $N_q^{Neu}$ for $l = 1,\dots ,N^{Conv}$, and $q = 1,\dots ,N^{FC}$. This compiled CNN architecture is also augemented with a fully-connected layer having the same number of output neurons as the required classification classes, denoted as $C^{num}$.

All convolutional and fully-connected layers’ weights are initialized using the He Normal weights initializer [40]. These trainable CNN parameters are denoted as $\mathit{\varpi }=\left\{{\varpi }_1,{\varpi }_2,\dots \right\}$. Let ${ℛ}^{train}$ be the training dataset with size $\left|{ℛ}^{train}\right|$, employed to train each potential CNN architecture from MTLBORKS-CNN learners. Each CNN architecture undergoes training in ${\tau }^{train}$ steps, with a predefined batch size $S^{batch}$, where

$${\tau }^{train}=\frac{\left|{ℛ}^{train}\right|}{S^{batch}}$$

(4)

The compiled CNN architecture is trained using the Adam optimizer [41] for a predetermined number of epochs (${\epsilon }^{train}$), performed over ${\tau }^{train}$ data batches extracted from the training dataset (${ℛ}^{train}$). At eachi-th training step ($i=1,\dots ,{\tau }^{train}$), the cross-entropy loss function of the CNN model is evaluated as $f_{CE}\left(\mathit{\varpi },{ℛ}_i^{train} \right)$ using the current weight parameters ($\mathit{\varpi }=\left\{{\varpi }_1,{\varpi }_2,\dots \right\}$) and the i-th batch data (${ℛ}_i^{train}$). The learning rate is represented as $R^L$, and the gradient of the cross-entropy loss is denoted as $\nabla f_{CE}\left(\mathit{\varpi },{ℛ}_i^{train}\right)$. The updated weight parameters of the CNN model, i.e., ${\mathit{\varpi }}^{\mathit{n}\mathit{e}\mathit{w}}=\left\{{\varpi }_1^{new},{\varpi }_2^{new},\dots \right\}$, are calculated as follows:

$${\mathit{\varpi }}^{\mathit{n}\mathit{e}\mathit{w}}=\mathit{\varpi }-R^L\nabla f_{CE}\left(\mathit{\varpi },{ℛ}_i^{train}\right)$$

(5)

3.4.2. Stage 2: Evaluation of the Trained CNN Architecture

Next, the trained CNN model is evaluated using a validation dataset, ${ℛ}^{valid}$, with $\left|{ℛ}^{valid}\right|$ samples. The resulting classification error is assigned as the fitness value for the corresponding MTLBORKS-CNN learner. This evaluation is performed over multiple steps, ${\tau }^{valid}$, where

$${\tau }^{valid}=\frac{\left|{ℛ}^{valid}\right|}{S^{batch}}$$

(6)

In each j-th evaluation step, different data batches, ${ℛ}_j^{valid}$, are used to evaluate the trained CNN models, resulting in distinct classification errors, indicated as $Err\_Batc{h}_j$ for $j=1,\dots ,{\tau }^{valid}$. The mean classification error of the trained CNN model, computed across all ${\tau }^{valid}$ batches of data within ${ℛ}^{valid}$, determines the fitness value for each n-th learner, represented as $F\left(X_n\right)$, i.e.,

$$F\left(X_n\right)=\frac{1}{{\tau }^{valid}}{\displaystyle \sum }_{j=1}^{{\tau }^{valid}}Err\_Batc{h}_j$$

(7)

3.4.3. Design Consideration of Epoch Numbers during Fitness Evaluation Process

CNNs are often characterized by deep architectures, which makes full training with dataset ${ℛ}^{train}$ for minimal classification error computationally expensive and time-consuming. This is primarily due to the requirement for a large number of training epochs ${\epsilon }^{train}$, typically exceeding 100, for full training of CNNs. However, when employing population-based MTLBORKS-CNN for identifying the optimal CNN architectures tailored to a specific dataset, each learner undergoes full training in each generation. This approach becomes impractical because it requires evaluating numerous candidate CNN architectures represented by all learners, resulting in a significant number of fitness evaluations. In the context of optimizing CNN architectures, each fitness evaluation involves the full training of a potential CNN architecture represented by an MTLBORKS-CNN learner. These excessive computational demands pose challenges to the feasibility of MTLBORKS-CNN in handling a large number of alternatives for achieving substantial improvements in the optimization process of CNN architectures.

To tackle this issue, a fitness approximation method is implemented. It involves training the potential CNN architecture represented by each MTLBORKS-CNN learner with a reduced training epoch (e.g., ${\epsilon }^{train}=1$) during the fitness evaluation process. While this may result in less precise evaluations, it effectively mitigates the computational load. In the selection process for the next generation of the population, it becomes more vital to ensure a fair comparison among learners to establish their relative superiority, rather than achieving precise fitness evaluations for each individual.

Moreover, a candidate CNN architecture is more likely to demonstrate a promising final classification error if it displays superior performance in the initial training epochs. Upon completing the MTLBORKS-CNN search process, the optimal CNN architecture formulated based on the network hyperparameters derived from the teacher can undergo full training with a larger epoch size to obtain its final classification error. To address potential network overfitting, the dropout and batch normalization techniques are incorporated between different layers of candidate CNN architectures [13].

3.5. Modified Teacher Phase of MTLBORKS-CNN

In the teacher phase of original TLBO, all learners are guided by the same exemplars, specifically the teacher solution and population mean, as defined by Equation (1). However, this conventional approach disregards valuable directional information contributed by other nonfittest population members. While both teacher solution and population mean can expedite learners’ convergence towards promising solution regions in the early stages of optimization, their influence diminishes as population diversity declines over time. This limitation becomes especially evident when addressing real-world optimization problems characterized by complex fitness landscapes, such as CNN architecture optimization in this study. These complex optimization problems often feature numerous local optima, which can misguide TLBO towards suboptimal regions. This undesirable phenomenon impedes the effectiveness of TLBO in searching for satisfactory CNN architectures due to premature convergence. To address this inherent limitation of teacher phase in the original TLBO, it is vital to incorporate a robust diversity maintenance mechanism. In this study, the modified teacher phase in MTLBORKS-CNN integrates a social learning concept. This mechanism considers the valuable direction information provided by other nonfittest learners, enabling more diversified and tailored guidance to each learner during the teacher phase, ultimately resulting in more effective CNN architecture searching.

3.5.1. Construction of Unique Mean Positions

In the modified teacher phase of MTLBORKS-CNN, a social learning concept is employed to calculate unique mean positions for all learners. This process begins by arranging all learners in descending order based on their fitness values, denoted as $F\left(X_n\right)$ for $n=1 \mathrm{to} N$. It is assumed that any learner outperforming the n-th learner falls within the population indices $m\in \left\{n+1,\dots ,N\right\}$. For each n-th learner, a unique mean position ${\overline{X}}_n$ is defined to represent the mean CNN architecture information for $n=1 \mathrm{to} N-1$. Particularly, each d-th dimension of unique mean position (i.e., ${\overline{X}}_{n,d}$) is calculated as follows:

$${\overline{X}}_{n,d}=\frac{1}{N-n+1}{\displaystyle \sum }_{m=n}^N{X}_{m,d}$$

(8)

The rounding operator, i.e., $Round\left(·\right)$, transforms all dimensional components of ${\overline{X}}_n$ into integer values. This conversion excludes the network hyperparameters $P_l^{Pool}$ stored in $d=2N_{max}^{Conv}+3l-1$ for $l=1,\dots ,N_{max}^{Conv}$, which signify the selection probabilities of pooling layers connected to the l-th convolutional layer. The mean CNN architecture, represented by ${\overline{X}}_n$, varies across learners due to its computation relying on distinct groups of learners with superior fitness. However, the best-performing learner, with n = N, lacks other better-performing peers to mimic. Therefore, it does not receive a unique mean position using Equation (8). Figure 5 illustrates the calculation of the unique mean position, ${\overline{X}}_1$, for the worst-performing learner using Equation (8), considering directional information from other superior learners ($X_n$ for $n=2,\dots ,5$) under $N_{max}^{Conv}=3$ and $N_{max}^{FC}=2$. Algorithm 4 outlines the pseudocode used to compute the mean CNN architecture, represented by ${\overline{X}}_n$, for every n-th learner ($n=1,\dots N-1$).

Algorithm 4: Computation of Unique Mean Positions Based on Social Learning Concept
Inputs: $N$, $D$, $P=\left[X_1,\dots ,X_n,\dots .,X_N\right]$
01:	Sort all learners in descending order based on their fitness values (i.e., $F\left(X_n\right)$ for $n=1,\dots ,N$);
02:	for  $n=1$ to $N-1$ do
03:	Initialize ${\overline{X}}_n \leftarrow \varnothing$;
04:	for $d=1$ to D do
05:	Calculate ${\overline{X}}_{n,d}$ using Equation (8);
06:	if $d\ne 2N_{max}^{Conv}+3l-1$ with $l=1,\dots ,N_{max}^{Conv}$ do
07:	${\overline{X}}_{n,d} \leftarrow Round\left({\overline{X}}_{n,d} \right)$;
08:	end if
09:	end for
10:	end for
Output:  ${\overline{X}}_n$ for $n=1,\dots ,N-1$

3.5.2. Construction of Unique Social Exemplar

To enhance knowledge exchange within a classroom, learners often benefit not only from their teachers, but also from peers who excel in various subjects. A similar approach is employed in the modified teacher phase of MTLBORKS-CNN, where each n-th learner is assigned a unique social exemplar, $X_n^{SE}$. This exemplar is designed to provide more effective guidance, drawing from valuable insights offered by other population members. Specifically, any randomly selected learner who outperforms the n-th learner (with a population index $k_{rand}\in \left\{n+1,\dots ,N\right\}$) contributes to the d-th dimension of the unique social exemplar allocated to each n-th learner, denoted as $X_{n,d}^{SE}$. This contribution is achieved via the corresponding dimension of the $k_{rand}$-th learner, as shown below:

$$X_{n,d}^{SE}=X_{k_{rand},d}^{SE} , \mathrm{where} n=1,\dots .,N-1 \mathrm{and} k_{rand}\in \left\{n+1,\dots ,N\right\}$$

(9)

Note that the unique social exemplar, $X_n^{SE}$, differs for each n-th learner, as it is computed based on a distinct group of learners who demonstrate superior fitness compared to the n-th learner. Similar to Equation (8), Equation (9) does not apply to the top-performing learner with a population index of $n=N$, as it surpasses all other learners. Figure 6 illustrates the process of constructing a unique social exemplar for the worst-performing learner, $X_1^{SE}$, by considering the position vectors of four other learners with superior fitness (i.e., $X_n$ for $n=2,\dots ,5$). Algorithm 5 presents the pseudocode detailing the creation of the unique social exemplar, $X_n^{SE}$, for every n-th learner, where $n=1,\dots .,N-1$.

Algorithm 5: Computation of Unique Social Exemplars Based on Social Learning Concept
Inputs: $N$, $D$, $P=\left[X_1,\dots ,X_n,\dots .,X_N\right]$
01:	for  $n=1$ to $N-1$ do
02:	Initialize $X_n^{SE}\leftarrow \varnothing$;
03:	for  $d=1$ to D do
04:	Randomly select a fitter learner with the population index of $k_{rand}\in \left\{n+1,\dots ,N\right\}$;
05:	Assign $X_{n,d}^{SE} \leftarrow X_{k_{rand},d}^{SE}$ based on Equation (9);
06:	end for
07:	end for
Output:  $X_n^{SE}$ for $n=1,\dots ,N$

3.5.3. Construction of New CNN Architecture

In the modified teacher phase of MTLBORKS-CNN, a new CNN architecture is determined for each n-th offspring learner using the position vector $X_n^{off}$. This vector is derived by incorporating both the unique mean position ${\overline{X}}_n$ and the unique social exemplar, $X_n^{SE}$, computed for each learner with $n=1,\dots .,N-1$, i.e.,

$$X_n^{off}=X_n+r_3\left(X^{Teacher}-F^T{\overline{X}}_n \right)+r_4\left(X_n^{SE}-X_n\right)$$

(10)

where $r_3,r_4\in \left[0,1\right]$ and $F^T\in \left\{1,2\right\}$ are randomly generated from the uniform distribution. As shown in Equation (10), the position vector for the n-th offspring learner, crucial for generating the new CNN architecture, is determined by the differences between the CNN architectures represented by ($X^{Teacher}-F^T{\overline{X}}_n$) and $\left(X_n^{SE}-X_n\right)$.

After obtaining $X_n^{off}$ from Equation (10), boundary checking is performed to ensure all network hyperparameters are within their search boundaries, as specified in Table 1. A rounding operator, $Round\left(·\right)$, is applied to convert the network hyperparameters into integer values. This conversion excludes the network hyperparameters stored in $d=2N_{max}^{Conv}+3l-1$ for $l=1,\dots ,N_{max}^{Conv}$, which signify the selection probabilities of pooling layers connected to the l-th convolutional layer. Algorithm 3 measures the fitness of $X_n^{off}$ as $F\left(X_n^{off}\right)$. The teacher solution can be replaced by the n-th offspring learner with superior fitness. All generated offspring solutions, $X_n^{off}$ for $n=1,\dots ,N-1$, along with the best-performing N-th learner, are stored in an offspring population set, denoted as $P^{off}=\left[X_1^{off},\dots ,X_n^{off},\dots .,X_N^{off}\right]$. These solutions will be used in the subsequent modified learner phase alongside the original population, $P=\left[X_1,\dots ,X_n,\dots .,X_N\right]$. The search mechanisms for the modified teacher phase of MTLBORKS-CNN are described in Algorithm 6.

Algorithm 6: Modified Teacher Phase of MTLBORKS-CNN
Inputs: $N$, $D$, $P=\left[X_1,\dots ,X_n,\dots .,X_N\right]$, $X^{Teacher}$, $F\left(X^{Teacher}\right)$, ${ℛ}^{train}$, ${ℛ}^{valid}$, $S^{batch}$, ${\epsilon }^{train}$, $R^L$, $C^{num}$
01:	Calculate the unique mean position ${\overline{X}}_n$ of each learner for $n=1,\dots ,N$ using Algorithm 4;
02:	Calculate the unique social exemplar $X_n^{SE}$ of each learner for $n=1,\dots ,N$ using Algorithm 5;
03:	Initialize the offspring population $P^{off}\leftarrow \varnothing$;
04:	for  $n=1$ to $N-1$ do
05:	if $n\ne N$ then
06:	Randomly generate $r_3,r_4\in \left[0,1\right]$ and $F^T\in \left\{1,2\right\}$;
07:	Calculate $X_n^{off}$ using Equation (10) and perform boundary checking;
08:	for  $d=1$ to D do
09:	if $d\ne 2N_{max}^{Conv}+3l-1$ with $l=1,\dots ,N_{max}^{Conv}$ do
10:	$X_{n,d}^{off}\leftarrow Round\left(X_{n,d}^{off}\right)$;
11:	end if
12:	end for
13:	Perform fitness evaluation on $X_n^{off}$ to obtain $F\left(X_n^{off}\right)$ using Algorithm 3;
14:	else /*i.e.,$n=N$*/
15:	$X_N^{off}\leftarrow X_N$;
16:	end if
17:	If  $F\left(X_n^{off}\right)<F\left(X^{Teacher}\right)$ then
18:	$X^{Teacher}\leftarrow X_n^{off}$, $F\left(X^{Teacher}\right)\leftarrow F\left(X_n^{off}\right)$; /*Update the teacher*/
19:	end if
20:	$P^{off}\leftarrow P^{off}{{\displaystyle \cup }}^{ }{X}_n^{off}$; /*Store the new learner into the offspring population*/
21:	end for
Output:  $P^{off}=\left[X_1^{off},\dots ,X_n^{off},\dots .,X_N^{off}\right]$, $X^{Teacher}$, $F\left(X^{Teacher}\right)$

3.6. Modified Learner Phase of MTLBORKS-CNN

In the learner phase of original TLBO, each learner performs searching within the solution space by interacting with a randomly selected peer learner from the population. This interaction, as described in Equation (3), involves attracting all dimensional components of the learner toward the peer learner if the peer has superior fitness, or repelling the learner away from the peer if the peer has inferior fitness. However, as the number of iterations increases, the likelihood of triggering the exploration-focused repelling mechanism diminishes. This is primarily because most learners converge toward specific solution regions, resulting in reduced population diversity. When dealing with complex optimization problems, such as CNN architecture optimization in the current study, the decreasing exploration strength of the original TLBO in later optimization stages can hinder its ability to discover new CNN architectures, as it becomes highly prone to becoming trapped in local optima. Additionally, the learning mechanisms of the original TLBO, as defined in Equation (3), fail to accurately emulate real classroom learning dynamics, as they overlook individual efforts and adaptive interactions among peer learners for more effective knowledge improvement. This inaccurate emulation of real classroom learning dynamics limits the balance between exploration and exploitation searches in the original TLBO, thereby constraining its effectiveness in searching for promising CNN architectures during the learner phase. To address these limitations, the modified learner phase of MTLBORKS-CNN introduces self-learning and adaptive peer learning schemes to achieve a more accurate emulation of real classroom learning dynamics. The incorporation of both self-learning and adaptive peer learning into the modified learner phase aims to strike a delicate balance between exploration and exploitation searches in MTLBORKS-CNN, ultimately enhancing its efficiency in optimizing CNN architectures during the modified learner phase.

3.6.1. Self-Learning Scheme

The self-learning scheme introduced in the modified learner phase of MTLBORKS-CNN aims to simulate learners’ preference for improving their knowledge in specific subjects through individual efforts using a probabilistic mutation operator. This scheme provides learners with an additional momentum through random perturbations, helping them break away from local optima.

After completing modified teacher phase, each n-th learner is assigned a probability of $P^{SL}=1/D$ to engage in the self-learning scheme during the modified learner phase. To facilitate the n-th learner’s self-learning process, a dimension index $d_{rand}\in \left[1,D\right]$ is randomly generated. This index is used to apply a random perturbation to $X_{n,d_{rand}}^{off}$, which denotes the $d_{rand}$-th dimension of the offspring solution generated by the n-th learner:

$$X_{n,d_{rand}}^{off}=X_{n,d_{rand}}^{off}+r_5\left(X_{d_{rand}}^{max}-X_{d_{rand}}^{min} \right)$$

(11)

where $r_5\in \left[-1,1\right]$ is a random number obtained from a uniform distribution; $X_{d_{rand}}^{max}$ and $X_{d_{rand}}^{min}$ represent the upper and lower limits of the $d_{rand}$-th dimension of the arrays containing network hyperparameters, as defined in Table 1.

Subsequently, a rounding operator $Round\left(·\right)$ is applied to $X_{n,d_{rand}}^{off}$ to convert network hyperparameters into integer values. However, this rounding operation excludes values that pertain to the selection probability of the pooling layer connected to the l-th convolutional layer. These specific values are situated within the dimension indices of $d=2N_{max}^{Conv}+3l-1$ for $l=1,\dots ,N_{max}^{Conv}$. The fitness value of $X_n^{off}$ is then evaluated using Algorithm 3 to obtain $F\left(X_n^{off}\right)$. If $X_n^{off}$ can construct a CNN architecture with lower classification error than $X^{Teacher}$, the latter solution will be replaced by $X_n^{off}$. Algorithm 7 outlines the pseudocode for the self-learning scheme.

Algorithm 7: Self-Learning Scheme of MTLBORKS-CNN
Inputs: $D$, $X_n^{off}$, $X^{Teacher}$, $F\left(X^{Teacher}\right)$, $X^{max}$, $X^{min}$, ${ℛ}^{train}$, ${ℛ}^{valid}$, $S^{batch}$, ${\epsilon }^{train}$, $R^L$, $C^{num}$
01:	Generate a random dimension index, $d_{rand}\in \left[1,D\right]$;
02:	Retrieve $d_{rand}$-th dimension of position vectors $X_n^{off}$, $X^{max}$ and $X$;
03:	for  $d=1$ to D do
04:	if  $d=d_{rand}$ then
05:	Update $X_{n,d_{rand}}^{off}$ using Equation (11);
06:	if $d\ne 2N_{max}^{Conv}+3l-1$ with $l=1,\dots ,N_{max}^{Conv}$ do
07:	$X_{n,d_{rand}}^{off}\leftarrow Round(X_{n,d_{rand}}^{off})$;
08:	end if
09:	end if
10:	end for
11:	Perform fitness evaluation on $X_n^{off}$ to update $F(X_n^{off})$ using Algorithm 3;
12:	if  $F(X_n^{off})<F(X^{Teacher})$ then
13:	$X^{Teacher}\leftarrow X_n^{off}$, $F(X^{Teacher})\leftarrow F(X_n^{off})$;
14:	end if
Output: Updated $X_n^{off}, F\left(X_n^{off}\right)$, $X^{Teacher}$ and $F\left(X^{Teacher}\right)$

3.6.2. Adaptive Peer Learning Scheme

In the modified learner phase of MTLBORKS-CNN, offspring solutions of learners who do not opt for the self-learning scheme are updated using the proposed adaptive peer learning approach. To initiate this phase, the fitness values of all offspring learners stored in $P^{off}=\left[X_1^{off},\dots ,X_n^{off},\dots .,X_N^{off}\right]$ are utilized to arrange them in descending order, from worst to best. The ranking $R_n$ for each n-th offspring learner is calculated as follows:

$$R_n=N-n$$

(12)

Based on Equation (12), an offspring learner with a lower value of $F(X_n^{off})$ receives a higher value of $R_n$ and vice versa. With these assigned ranking values, the probability of interaction with peers for each n-th offspring learner, denoted as $P_n^{PL}$, is calculated as:

$$P_n^{PL}=\frac{R_n}{N}$$

(13)

Let $X_{n,d}^{off}$ represent the d-th dimension of each n-th offspring learner. A random number $r_6\in \left[0,1\right]$ is generated using a uniform distribution and compared with the corresponding $P_n^{PL}$ when updating $X_{n,d}^{off}$. If $r_6$ is smaller than $P_n^{PL}$, three peer offspring learners, denoted as $X_p^{off}$, $X_q^{off}$, and $X_s^{off}$, are randomly selected from ${\mathbf{P}}^{\mathbf{o}\mathbf{f}\mathbf{f}}$. The network hyperparameters stored in their respective d-th dimension are used to update $X_{n,d}^{off}$, where $p\ne s\ne u\ne n$. Otherwise, the original value stored in $X_{n,d}^{off}$ is retained. Let ${\chi }_n\in \left[0.5, 1\right]$ be a peer learning factor generated randomly from a uniform distribution for each n-th offspring learner. The adaptive peer learning scheme updates $X_{n,d}^{off}$ for each n-th offspring learner as follows:

$$X_{n,d}^{off}=\{\begin{array}{c}{X}_{p,d}^{off}+{\chi }_n\left(X_{s,d}^{off}-X_{u,d}^{off} \right), \mathrm{if} r_6<P_n^{PL}\\ X_{n,d}^{off}, \mathrm{otherwise}\end{array}$$

(14)

After obtaining $X_n^{off}$ from Equation (14), boundary checking in performed to ensure all network hyperparameters are within their search boundaries, as specified in Table 1. A rounding operator $Round\left(·\right)$ is applied to all network hyperparameters encoded in $X_n^{off}$, except for those stored in the dimension indices where $d=2N_{max}^{Conv}+3l-1$ for $l=1,\dots ,N_{max}^{Conv}$. Algorithm 3 is utilized to evaluate the fitness value of the n-th offspring learner as $F(X_n^{off})$ based on its updated $X_n^{off}$. If the CNN architecture represented by $X_n^{off}$ results in a lower classification error than that of $X^{Teacher}$, $X_n^{off}$ replaces $X^{Teacher}$. Equations (12)–(14) illustrate that the n-th offspring learner with a larger (indicating worse fitness) is more inclined to learn from multiple peers and update most of its dimensional components in $X_n^{off}$, as indicated by its larger $P_n^{PL}$ value, and vice versa.

This adaptive peer learning scheme effectively regulates the explorative and exploitative behaviors of MTLBORKS-CNN. It enables learners to adjust their CNN architectures through interactions with peers, or retain their original architectures based on their fitness levels. A detailed description of the adaptive peer learning scheme is presented in Algorithm 8. Algorithm 9 outlines the comprehensive search mechanisms for the modified learner phase of MTLBORKS-CNN, combining both self-learning and adaptive peer learning schemes detailed in Algorithms 7 and 8, respectively.

Algorithm 8: Adaptive Peer Learning of MTLBORKS-CNN
Inputs: $D$, $P^{off}=[X_1^{off},\dots ,X_n^{off},\dots .,X_N^{off}]$, $X^{Teacher}$, $F\left(X^{Teacher}\right)$, ${ℛ}^{train}$, ${ℛ}^{valid}$, $S^{batch}$, ${\epsilon }^{train}$, $R^L$, $C^{num}$
01:	Calculate $R_n$ and $P_n^{PL}$ using Equations (12) and (13), respectively.
02:	for  $d=1$ to D do
03:	Randomly select $X_p^{off}$, $X_q^{off}$, and $X_s^{off}$ from $P^{off}$, where $p\ne s\ne u\ne n$;
04:	Update $X_{n,d}^{off}$ using Equation (14) and perform boundary checking;
05:	if  $d\ne 2N_{max}^{Conv}+3l-1$ with $l=1,\dots ,N_{max}^{Conv}$ do
06:	$X_{n,d}^{off}\leftarrow Round(X_{n,d}^{off})$;
07:	end if
08:	Perform fitness evaluation on the updated $X_n^{off}$ to obtain $F(X_n^{off})$ using Algorithm 3;
09:	if $F(X_n^{off})<F(X^{Teacher})$ then
10:	$X^{Teacher}\leftarrow X_n^{off}$, $F(X^{Teacher})\leftarrow F(X_n^{off})$;
11:	end if
Output: Updated $X_n^{off}$, $F(X_n^{off})$, $X^{Teacher}$, and $F(X^{Teacher})$

Algorithm 9: Modified Learner Phase of MTLBORKS-CNN
Inputs: $D$, $P^{off}=[X_1^{off},\dots ,X_n^{off},\dots .,X_N^{off}]$, $X^{Teacher}$, ${ℛ}^{train}$, ${ℛ}^{valid}$, $S^{batch}$, ${\epsilon }^{train}$, $R^L$, $C^{num}$
01:	Sort the offspring learners stored in $P^{off}$ in descending order (i.e., from worst to best);
02:	for  $n=1$ to N do
03:	if  $r_6<P_n^{PL}$ do
04:	Perform self-learning scheme to update $X_n^{off}$ and $X^{Teacher}$ using Algorithm 7;
05:	else
06:	Perform adaptive peer learning scheme to update $X_n^{off}$ and $X^{Teacher}$ using Algorithm 8;
07:	end
08:	end for
Output: Updated $P^{off}=[X_1^{off},\dots ,X_n^{off},\dots .,X_N^{off}]$, $X^{Teacher}$, and $F(X^{Teacher})$

3.7. Dual-Criterion Selection Scheme of MTLBORKS-CNN

The selection scheme plays a crucial role in forming the next generation population in MSAs, using predefined criteria during the optimization process. Traditional selection schemes, such as greedy selection and tournament selection, determine the survival of individual solutions solely based on their fitness values. For example, in the original TLBO, the greedy selection scheme compares the fitness values of current learners with those of new learners generated through teacher and learner phases. Despite their straightforward implementation, these fitness-based selection schemes have drawbacks when dealing with complex optimization problems, such as CNN architecture optimization in this study. In the context of CNN architecture optimization, the excessive reliance on fitness-based selection schemes may hinder the ability of original TLBO to discover novel CNN architectures that might initially show slightly lower performance, but have the potential for long-term usefulness, such as maintaining population diversity. To overcome this limitation, MTLBORKS-CNN introduces a dual-criterion selection approach. This approach constructs the next generation population by considering both the fitness and diversity of learners. It allows learners with slightly lower fitness but greater diversity to move forward to the next iteration, thereby preserving population diversity.

Upon the completion of the modified learner phase in MTLBORKS-CNN, a merged population is formed, denoted as $P^{MG}$. This merged population combines the current population, represented as $P=\left[X_1,\dots ,X_n,\dots ,X_N\right]$, with the updated offspring population, denoted as $P^{off}=\left[X_1^{off},\dots ,X_n^{off},\dots .,X_N^{off}\right]$. The population size of $P^{MG}$ is 2N, and it is represented as follow:

$$P^{MG}=P{{\displaystyle \cup }}^{ }{P}^{off}=\left[X_1^{MG},\dots ,X_n^{MG},\dots ,X_{2N}^{MG}\right]$$

(15)

where $X_n^{MG}$ refers to the n-th solution member stored in $P^{MG}$, which may originate from either the current learners in $P$ or the offspring learners in $P^{off}$. These solution members within $P^{MG}$ are sorted in ascending order based on the classification error $F\left(X_n^{MG}\right)$ associated with the CNN architecture represented by the corresponding $X_n^{MG}$. Let $Di{s}_n^{MG}$ represents the Euclidean distance between the n-th solution member of $P^{MG}$ (i.e., $X_{\mathrm{n}}^{MG}$) and the best solution member (i.e., $X_1^{MG}$). The value of $Di{s}_n^{MG}$ is determined using a Euclidean distance operator $Eud(·,·)$, where

$$Di{s}_n^{MG}=EuD\left(X_n^{MG},X_1^{MG}\right)$$

(16)

The dual-criterion selection scheme proposed in MTLBORKS-CNN utilizes the calculated values of $F\left(X_n^{MG}\right)$ and $Di{s}_n^{MG}$ for each n-th solution within $P^{MG}$, where $n=1,\dots ,2N$. To construct the next generation population, denoted as $P^{Next}$, with a population size of N, K solution members with the best $F\left(X_n^{MG}\right)$ values are directly selected from $P^{MG}$ in each iteration of MTLBORKS-CNN. Here, $K\in \left\{1,N\right\}$ represents a randomly generated integer. For the remaining $\left(2N-K\right)$ solution members in $P^{MG}$, with population indices $n=K+1,\dots ,2N$, a weighted fitness value is calculated for each n-th solution member, denoted as $WF\left(X_n^{MG}\right)$. This calculation takes into account both the classification error (i.e., $F(X_n^{MG})$) and the diversity (i.e., $F\left(X_n^{MG}\right)$), as follows:

$$WF\left(X_n^{MG}\right)=\alpha \left(\frac{F\left(X_n^{MG}\right)-F^{min}}{F^{max}-F^{min}}\right)+(1-\alpha )(\frac{Di{s}^{max}-Di{s}_n^{MG}}{Di{s}^{max}-Di{s}^{min}})$$

(17)

where $\alpha$ is a weighted factor randomly generated from a normal distribution $N\left(0.9,0.05\right)$. Its value is constrained within the range [0.8,1.0] to ensure that diversity does not dominate the selection process for the remaining population members in the subsequent iteration. $F^{max}$ and $F^{min}$ represent the worst and best fitness values observed within $P^{MG}$, respectively, while $Di{s}^{max}$ and $Di{s}^{min}$ correspond to the largest and smallest Euclidean distances measured from the best solution member $X_1^{MG}$, respectively.

After calculating the weighted fitness value $WF\left(X_n^{MG}\right)$ for each n-th solution member, where $n=K+1,\dots ,2N$, a binary tournament strategy randomly selects two solution members, $X_a^{MG}$ and $X_b^{MG}$ from $P^{MG}$, where $a\ne b$. The solution member with the smaller weighted fitness value is then chosen to be one of the remaining solution members of $P^{Next}$, denoted as $X_n^{Next}$, for $n=K+1,\dots ,N$, as follows:

$$X_n^{Next}=\{\begin{array}{c}{X}_a^{MG}, \mathrm{if} WF\left(X_a^{MG}\right)\le WF\left(X_b^{MG}\right) \\ X_b^{MG}, \mathrm{otherwise}\end{array}$$

(18)

The remaining $\left(N-K\right)$ solution members for the next iteration are selected from $P^{Next}$ by repeating the binary tournament strategy described in Equation (18). Algorithm 10 provides the pseudocode for the proposed dual-criterion selection scheme. In contrast to fitness-based selection schemes, such as greedy selection and tournament selection, this dual-criterion selection scheme not only retains the K elite solution members for $P^{Next}$ in the subsequent iteration, but also effectively maintains population diversity by considering both the fitness and diversity levels of solutions when choosing the remaining $\left(N-K\right)$ solution members of $P^{Next}$.

Algorithm 10: Dual-Criterion Selection Scheme of MTLBORKS-CNN
Inputs: $N$, $P=[X_1,\dots ,X_n,\dots ,X_N]$, $P^{off}=\left[X_1^{off},\dots ,X_n^{off},\dots .,X_N^{off}\right]$
01:	Initialize the next generation population $P^{Next}\leftarrow \varnothing$;
02:	Construct the merged population $P^{MG}$ using Equation (15);
03:	Rearrange the solutions of $P^{MG}$ in ascending order by referring to their fitness values;
04:	for  $n=1$ to 2N do
05:	Calculate the $Di{s}_n^{MG}$ for every n-th solution of $P^{MG}$ using Equation (16);
06:	end for
07:	Randomly generate an integer $K\in \left\{1,N\right\}$;
08:	for $n=1$ to K do /*Select the first K learners by only considering their fitness*/
09:	$X_n^{Next}\leftarrow X_n^{MG}$;
10:	$P^{Next}\leftarrow P^{Next}{{\displaystyle \cup }}^{ }{X}_n^{Next}$;
11:	end for
12:	for  $n=K+1$ to 2N do
13:	Randomly generate $\alpha$ from a normal distribution $N\left(0.9,0.05\right)$;
14:	if  $\alpha <0.8$ then
15:	$\alpha \leftarrow 0.8$;
16:	else if $\alpha >1$ then
17:	$\alpha \leftarrow$1;
18:	end if
19:	Calculate the $WF\left(X_n^{MG}\right)$ of each n-th solution member of $P^{MG}$ using Equation (17);
20:	end for
21:	for  $n=K+1$ to N do /*Select the remaining learners by considering their fitness and diversity*/
22:	Randomly select two solution members of $X_a^{MG}$ and $X_b^{MG}$ from $P^{MG}$, where $a\ne b$;
23:	Determine $X_n^{Next}$ using Equation (18);
24:	$P^{Next}\leftarrow P^{Next}{{\displaystyle \cup }}^{ }{X}_n^{Next}$;
25:	end for
Output:  $P^{Next}=\left[X_1^{Next},\dots ,X_n^{Next},\dots .,X_N^{Next}\right]$

3.8. Complete Mechanisms of MTLBORKS-CNN

Algorithm 11 delineates the comprehensive mechanisms of MTLBORKS-CNN, designed for the search of an optimized CNN architecture tailored to a specific dataset. Throughout this algorithm, a counter variable t keeps track of the current iteration, with $T^{max}$ representing the predefined maximum iteration number, which acts as the termination criterion for MTLBORKS-CNN.

The procedure commences by loading the training and validation datasets, denoted as ${ℛ}^{train}$ and ${ℛ}^{valid}$, respectively, from the designated directory. Following this, the MTLBORKS-CNN population is initialized using Algorithm 2. Subsequently, Algorithms 6 and 9, elucidating the modified teacher phase and modified learner phase of MTLBORKS-CNN, respectively, are iteratively employed to generate a population set, $P^{off}$, comprising diverse new CNN architectures. The next-generation population, $P^{Next}$, is then derived from the merged population, $P^{MG}=P{{\displaystyle \cup }}^{ }{P}^{off}$, employing the dual-criterion selection scheme proposed in Algorithm 10. The optimization process for CNN architecture, facilitated by the proposed MTLBORKS-CNN method, concludes when the termination condition $t>T^{max}$ is met.

As explained in the previous subsection, the fitness evaluation process, as delineated in Algorithm 3, employs a reduced epoch number, denoted as ${\epsilon }^{train}$, for training each CNN architecture generated by every MTLBORKS-CNN learner. While this approach effectively reduces computational demands, its efficacy in addressing complex problems may be limited. Therefore, after the termination of MTLBORKS-CNN, the CNN architecture constructed from the teacher solution (i.e., $X^{Teacher}$) undergoes a full training process. This full training process shares the same mechanisms as Algorithm 3, but with a larger number of epochs, denoted as ${\epsilon }^{FT}$. Once the full training is completed, all relevant network details of the fully trained CNN model, including its architecture, classification error, number of trainable parameters, and more, are returned.

Algorithm 11: Proposed MTLBORKS-CNN
Inputs: $N$, $D$, ${ℛ}^{train}$, ${ℛ}^{valid}$, $S^{batch}$, ${\epsilon }^{train}$, ${\epsilon }^{FT}$, $R^L$, $C^{num}$, $N_{min}^{Conv}$, $N_{max}^{Conv}$, $N_{min}^{Fil}$, $N_{max}^{Fil}$, $S_{min}^{Ker}$, $S_{max}^{Ker}$, $S_{min}^{Pool}$, $S_{max}^{Pool}$, $S_{min}^{Str}$, $S_{max}^{Str}$, $N_{min}^{FC}$, $N_{max}^{FC}$, $N_{min}^{Neu}$, $N_{max}^{Neu}$
01:	Load ${ℛ}^{train}$ and ${ℛ}^{valid}$ from the directory;
02:	Initialize the population $P=\left[X_1,\dots ,X_n,\dots ,X_N\right]$ using Algorithm 2;
03:	Initialize the iteration counter as $t\leftarrow 0$;
04:	while $t<T^{max}$ do
05:	Generate $P^{off}$ and update $X^{Teacher}$ and $F\left(X^{Teacher}\right)$ using modified teacher phase (Algorithm 6);
06:	Update $P^{off}$, $X^{Teacher}$ and $F\left(X^{Teacher}\right)$ using modified learner phase (Algorithm 9);
07:	Determine $P^{Next}$ using dual-criterion selection scheme (Algorithm 10);
08:	$P\leftarrow P^{Next} ;$
09:	$t\leftarrow t+1$;
10:	end while
11:	Fully train the CNN architecture constructed from $X^{Teacher}$ with larger ${\epsilon }^{FT}$ (Algorithm 3);
Output: An optimal CNN architecture constructed from $X^{Teacher}$ with all related network information

3.9. Complexity Analysis of MTLBORKS

The time complexity of the proposed MTLBORKS, compared to the original TLBO, is evaluated using Big O analysis. Since both TLBO and MTLBORKS are used to solve the CNN architecture optimization problem in the current study, the time complexity for fitness evaluation is the same for both algorithms. The original TLBO incurs a time complexity of $O\left(ND\right)$ across population initialization, teacher phase, and learner phase, where N is the population size, and D is problem dimensionality. Thus, the time complexity of the original TLBO in each iteration is $O\left(ND\right)$ in the worst-case scenario. Similar to the original TLBO, the time complexity for initializing MTLBORKS population is $O\left(ND\right)$. The computational time complexity of MTLBORKS is affected by three key modifications: the modified teacher phase, the modified learner phase, and the dual-criterion selection scheme.

In the modified teacher phase, learners are rearranged in descending order based on their fitness values, incurring a time complexity of $O\left(N\log N\right)$ per iteration. Additionally, the values of ${\overline{X}}_n$, $X_n^{SE}$, and $X_n^{off}$ are calculated for all N learners using Equations (8), (9), and (10), respectively, resulting in a time complexity of $O\left(ND\right)$ in every iteration. Therefore, the total time complexity of the modified teacher phase of MTLBORKS amounts to $O\left(ND\right)$ per iteration, due to its growth rate being higher than $O\left(N\log N\right)$.

During the modified learner phase, learners are rearranged in descending order based on their fitness values, incurring a time complexity of $O\left(N\log N\right)$ per iteration. Furthermore, the values of $R_n$ and $P_n^{PL}$ are calculated for all N learners in each iteration using Equations (12) and (13), respectively, with a time complexity of $O\left(ND\right)$. When updating $X_n^{off}$ using the self-learning scheme and adaptive peer learning, as described in Equations (11) and (14), respectively, a computational time complexity of $O\left(D\right)$ is incurred for each learner. Thus, the total time complexity of the modified learner phase of MTLBORKS remains $O\left(ND\right)$ per iteration.

For the proposed dual-criterion selection scheme, a time complexity of $O\left(N\right)$ per iteration is incurred when merging $P$ and $P^{off}$ to produce $P^{MG}$ using Equation (15). Then, a time complexity of $O\left(N\log N\right)$ per iteration is incurred to rearrange all learners in $P^{MG}$ in descending order based on their fitness values. Time complexities of $O\left(ND\right)$ and $O\left(N\right)$ are incurred in each iteration to calculate the values of $Di{s}_n^{MG}$ and $WF\left(X_n^{MG}\right)$ for all N learners using Equations (16) and (17), respectively. When constructing $P^{Next}$ for the next iteration of MTLBORKS, a time complexity of $O\left(N\right)$ is incurred. Thus, the total time complexity of the dual-criterion selection scheme of MTLBORKS is $O\left(ND\right)$ per iteration.

In conclusion, based on the time complexity analyses presented above, it is evident that the overall time complexity for each iteration of MTLBORKS, encompassing the modified teacher phase, modified learner phase, and dual-criterion selection scheme, remains $O\left(ND\right)$ under the worst-case scenario.

4. Performance Analysis of Proposed Method

4.1. Selection of Benchmark Datasets

The performance of CNN architectures optimized by the proposed MTLBORKS-CNN for image classification tasks is assessed across various benchmark datasets, and is compared with other established algorithms. Similar to [13], this evaluation covers nine benchmark datasets: MNIST, MNIST-RD, MNIST-RB, MNIST-BI, MNIST-RD+BI, Rectangles, Rectangles-I, Convex, and MNIST-Fashion. Four of these datasets (MNIST-RD, MNIST-RB, MNIST-BI, and MNIST-RD+BI) are more challenging than the original MNIST due to the presence of irrelevant features in their input images.

The Rectangle dataset is used for training machine learning and deep learning models to recognize the dimensions of rectangles, while Rectangle-I focuses on discerning whether an image patch is inside or outside a rectangle. The Convex and Fashion datasets are employed for training classifiers to differentiate between convex and nonconvex input images and various categories of fashion items, respectively. These MNIST variant datasets can be accessed at http://www.iro.umontreal.ca/~lisa/icml2007data/ (accessed on 3 June 2023). For visual reference, Figure 7 provides sample images from each dataset. All selected benchmark datasets feature images with an input size of 28 × 28 × 1, and

Table 3. Summary of all selected benchmark datasets used for performance study.

Dataset	# Training Samples	# Testing Samples	# Classes
MNIST	60,000	10,000	10
MNIST-RD	12,000	50,000	10
MNIST-RB	12,000	50,000	10
MNIST-BI	12,000	50,000	10
MNIST-RD+BI	12,000	50,000	10
Rectangles	1200	50,000	2
Rectangles-I	12,000	50,000	2
Convex	8000	50,000	2
MNIST-Fashion	60,000	10,000	10

offers a comprehensive overview of their characteristics.

4.2. Simulation Settings

Performance evaluations of CNN architectures constructed via MTLBORKS-CNN for the first eight datasets (MNIST, MNIST-RD, MNIST-RB, MNIST-RI, MNIST-RD+BI, Rectangle, Rectangle-I, and Convex) are compared with 18 peer algorithms, which include psoCNN [42], EvoCNN [13], RandNet-2 [43], PCANet-2 [43], CAE-1 [44], CAE-2 [44], SVM+Poly [45], ScatNet-2 [46], SVM+RBF [45], NNet [45], SAA-3 [45], LDANet-2 [43], DBN-3 [45], AlexNet [7], ResNet-50 [8], VGGNet [9], MobileNet [10], and GoogleNet [11]. For the comparison using MNIST-Fashion, another 17 classifiers, including psoCNN [42], EvoCNN [13], SqueezeNet-200 [47], AlexNet [7], ResNet-50 [8], VGGNet [9], MobileNet [10], GoogleNet [11], MLP 256-128-64, MLP 256-128-100, GRU+SVM+Dropout, GRU+SVM, 2C1P2F+Dropout, 2C1P2F, 3C1P2F+Dropout, 3C2F, and HOG+SVM, are selected.

EvoCNN and psoCNN share conceptual similarities with MTLBORKS-CNN, using population-based search mechanisms within a predefined maximum iteration limit. While most compared methods were tailored for specific problems, it is crucial to assess the ability of MTLBORKS-CNN to autonomously identify superior CNN architectures, distinguishing it from handcrafted approaches reliant on human expertise (e.g., AlexNet, ResNet-50, VGGNet, MobileNet, and GoogleNet).

Table 4. Simulation Settings of MTLBORKS-CNN.

Parameters	Values
Dimensional size D	19
Population size N	20
$\mathrm{Maximum} \mathrm{iteration} \mathrm{number} T^{max}$	10
Minimum number of convolutional layers $N_{min}^{Conv}$	1
Maximum number of convolutional layers $N_{max}^{Conv}$	3
$\mathrm{Minimum} \mathrm{filter} \mathrm{numbers} N_{min}^{Fil}$	3
$\mathrm{Maximum} \mathrm{filter} \mathrm{numbers} N_{max}^{Fil}$	256
$\mathrm{Minimum} \mathrm{kernel} \mathrm{size} S_{min}^{Ker}$	$3\times 3$
$\mathrm{Maximum} \mathrm{kernel} \mathrm{size} S_{max}^{Ker}$	$9\times 9$
$\mathrm{Minimum} \mathrm{pooling} \mathrm{size} S_{min}^{Pool}$	$1 \times 1$
$\mathrm{Maximum} \mathrm{pooling} \mathrm{size} S_{max}^{Pool}$	$3\times 3$
$\mathrm{Minimum} \mathrm{stride} S_{min}^{Str}$	$1\times 1$
$\mathrm{Maximum} \mathrm{stride} S_{max}^{Str}$	$2\times 2$
Minimum number of fully-connected layers $N_{min}^{FC}$	1
Maximum number of fully-connected layers $N_{max}^{FC}$	2
$\mathrm{Minimum} \mathrm{neuron} \mathrm{numbers} N_{min}^{Neu}$	1
$\mathrm{Maximum} \mathrm{neuron} \mathrm{numbers} N_{max}^{Neu}$	300
Dropout rate	0.5
Epoch numbers during fitness evaluation ${\epsilon }^{train}$	1
Epoch numbers during full training ${\epsilon }^{FT}$	100

summarizes the simulation settings of MTLBORKS-CNN, adhering to conventions in deep learning and MSA research. MTLBORKS-CNN is implemented in Python 3.8.5 on a personal computer with an Nvidia GeForce RTX 3090, and simulations are performed over 30 runs.

4.3. Simulation Results

4.3.1. Classification Accuracy in Handling the First Eight Benchmark Datasets

To assess the effectiveness of MTLBORKS-CNN and its peers, a comprehensive performance analysis is conducted on eight benchmark datasets: MNIST, MNIST-RD, MNIST-RB, MNIST-BI, MNIST-RD+BI, Rectangles, Rectangles-I, and Convex. The generalization performances of these classifiers are evaluated using test data from these datasets, and the results are presented in

Table 5. Comparison of classification accuracies between MTLBORKS-CNN and its peers for solving the first eight image datasets.

Classifier	MNIST	MNIST-RD	MNIST-RB	MNIST-BI	MNIST-RD+BI
ScatNet-2	98.73% (+)	92.52% (+)	87.70% (+)	81.60% (+)	49.52% (+)
CAE-1	98.60% (+)	95.48% (+)	93.19% (+)	87.58% (+)	61.46% (+)
CAE-2	97.52% (+)	90.34% (+)	89.10% (+)	84.50% (+)	54.77% (+)
SAA-3	96.54% (+)	89.70% (+)	88.72% (+)	77.00% (+)	48.07% (+)
LDANet-2	98.95% (+)	92.48% (+)	93.19% (+)	87.58% (+)	61.46% (+)
SVM+Poly	96.31% (+)	84.58% (+)	83.38% (+)	75.99% (+)	43.59% (+)
SVM+RBF	96.97% (+)	88.89% (+)	85.42% (+)	77.49% (+)	44.82% (+)
NNet	95.31% (+)	81.89% (+)	79.96% (+)	72.59% (+)	37.84% (+)
RandNet-2	98.75% (+)	91.53% (+)	86.53% (+)	88.35% (+)	56.31% (+)
DBN-3	96.89% (+)	89.70% (+)	93.27% (+)	83.69% (+)	52.61% (+)
PCANet-2	98.60% (+)	91.48% (+)	93.15% (+)	88.45% (+)	64.14% (+)
AlexNet	96.28% (+)	91.35% (+)	94.38% (+)	93.40% (+)	78.11% (+)
ResNet-50	99.38% (+)	94.40% (+)	94.11% (+)	96.21% (+)	79.98% (+)
VGGNet-16	97.74% (+)	92.12% (+)	95.42% (+)	94.77% (+)	79.01% (+)
MobileNet-v2	99.42% (+)	94.26% (+)	97.21% (+)	96.37% (+)	80.19% (+)
GoogleNet	99.46% (+)	94.42% (+)	97.59% (+)	96.55% (+)	81.81% (+)
psoCNN	99.51% (+)	94.56% (+)	97.61% (+)	96.87% (+)	81.05% (+)
EvoCNN	98.82% (+)	94.78% (+)	97.20% (+)	95.47% (+)	64.97% (+)
MTLBORKS-CNN (Best)	99.65%	96.50%	98.11%	97.15%	82.28%
MTLBORKS-CNN (Mean)	99.59%	95.90%	97.82%	96.67%	81.17%
Classifier	Rectangles	Rectangles-I	Convex	w/t/l	#BCA
ScatNet-2	99.99% (−)	91.98% (+)	93.50% (+)	7/0/1	1
CAE-1	99.86% (+)	83.80% (+)	NA	7/0/0	0
CAE-2	98.46% (+)	78.00% (+)	NA	7/0/0	0
SAA-3	97.59% (+)	75.95% (+)	81.59% (+)	8/0/0	0
LDANet-2	99.86% (+)	83.80% (+)	92.78% (+)	8/0/0	0
SVM+Poly	97.85% (+)	75.95% (+)	80.18% (+)	8/0/0	0
SVM+RBF	97.85% (+)	75.96% (+)	80.87% (+)	8/0/0	0
NNet	92.84% (+)	66.80% (+)	67.75% (+)	8/0/0	0
RandNet-2	99.91% (+)	83.00% (+)	94.55% (+)	8/0/0	0
DBN-3	97.39% (+)	77.50% (+)	81.37% (+)	8/0/0	0
PCANet-2	99.51% (+)	86.61% (+)	95.81% (+)	8/0/0	0
AlexNet	97.88% (+)	93.85% (+)	94.34% (+)	8/0/0	0
ResNet-50	99.51% (+)	95.19% (+)	97.20% (+)	8/0/0	0
VGGNet-16	98.54% (+)	94.11% (+)	95.74% (+)	8/0/0	0
MobileNet-v2	99.61% (+)	95.47% (+)	97.29% (+)	8/0/0	0
GoogleNet	99.72% (+)	95.96% (+)	97.41% (+)	8/0/0	0
psoCNN	99.93% (+)	96.03% (+)	97.74% (+)	8/0/0	0
EvoCNN	99.99% (−)	94.97% (+)	95.18% (+)	7/0/1	1
MTLBORKS-CNN (Best)	99.98%	97.39%	97.95%	NA	7
MTLBORKS-CNN (Mean)	99.97%	96.00%	97.64%	NA	NA

. The best and second-best performances are highlighted using bold and underline formatting, respectively. Table 5 provides insights into how MTLBORKS-CNN compares to its competitors for each dataset, denoted as “(+)” for outperforming, “(−)” for underperforming, and “(=)” for achieving comparable results. In cases where no results were available, “NA” is reported. Additionally, Table 5 summarizes classification accuracies for all classifiers across the eight datasets, using “w/t/l” to indicate whether MTLBORKS-CNN performed better, tied, or worse than competitors on “w”, “t”, and “l” datasets, respectively. “#BCA” is also included to specify the number of datasets in which a classifier achieved the best accuracy.

Table 5 presents the best and mean accuracies achieved by the compared classifiers on each benchmark dataset. MTLBORKS-CNN demonstrates superior performance, emerging as the top-performing algorithm with the highest classification accuracy on seven out of eight image datasets. For the eight benchmark image datasets, MTLBORKS-CNN achieves the following best classification accuracies: 99.65% for MNIST, 96.50% for MNIST-RD, 98.11% for MNIST-RB, 97.15% for MNIST-BI, 82.82% for MNIST-RD+BI, 97.39% for Rectangle-I, and 97.95% for Convex. Figure 8 visually illustrates the distribution of test errors generated by MTLBORKS-CNN while solving each benchmark dataset, providing deeper insights into its performance. Remarkably, the two best-performing classifiers consistently involve at least one developed using MSAs, such as MTLBORKS-CNN, psoCNN, or EvoCNN. In comparison to other handcrafted networks (e.g., AlexNet, MobileNet, GoogleNet, ResNet-50, and VGGNet), the simulation results in Table 5 reveal that MSAs offer better flexibility in searching for optimal CNN architectures tailored to specific datasets with minimal human intervention. The findings in Table 5 strongly.

Support MTLBORKS-CNN’s effectiveness in addressing diverse image classification tasks with high accuracy due to its excellent ability to construct CNN models tailored to specific datasets. MTLBORKS-CNN’s superior classification performance compared to psoCNN and EvoCNN for most benchmark datasets highlights the effectiveness of the modifications introduced in its teacher and learner phases, achieving a better balance between exploratory and exploitative characteristics, which is crucial for constructing robust CNN models. Additionally, the dual-criterion selection scheme in MTLBORKS-CNN proves valuable in preserving information stored in learners with higher diversity and temporarily lower fitness, leading to significant long-term improvements.

The Wilcoxon signed-rank test [48] is employed to assess performance deviations between MTLBORKS-CNN and each competitor across the eight benchmark datasets. It is noteworthy that CAE-1 and CAE-2 are excluded from these analyses due to the absence of accuracy data for the Convex dataset. These pairwise comparison results are presented in

Table 6. Pairwise comparison results obtained from Wilcoxon signed-rank test.

MTLBORKS-CNN vs.	${\mathit{R}}^{+}$	${\mathit{R}}^{-}$	p-Value	h-Value
ScatNet-2	28.0	0.0	$1.42\times {10}^{-2}$	+
SAA-3	36.0	0.0	$9.58\times {10}^{-3}$	+
LDANet-2	36.0	0.0	$9.58\times {10}^{-3}$	+
SVM+Poly	36.0	0.0	$9.58\times {10}^{-3}$	+
SVM+RBF	36.0	0.0	$9.58\times {10}^{-3}$	+
NNet	36.0	0.0	$9.58\times {10}^{-3}$	+
RandNet-2	36.0	0.0	$9.58\times {10}^{-3}$	+
DBN-3	36.0	0.0	$9.58\times {10}^{-3}$	+
PCANet-2	36.0	0.0	$9.58\times {10}^{-3}$	+
AlexNet	36.0	0.0	$8.37\times {10}^{-3}$	+
ResNet-50	36.0	0.0	$9.58\times {10}^{-3}$	+
VGGNet-16	36.0	0.0	$9.58\times {10}^{-3}$	+
MobileNet-v2	36.0	0.0	$9.58\times {10}^{-3}$	+
GoogleNet	36.0	0.0	$7.26\times {10}^{-3}$	+
psoCNN	36.0	0.0	$8.37\times {10}^{-3}$	+
EvoCNN	28.0	0.0	$1.05\times {10}^{-2}$	+

, including $R^{+}$, $R^{-}$, p-, and h-values. Specifically, $R^{+}$ and $R^{-}$ represent the total ranks assigned to MTLBORKS-CNN for outperforming and underperforming each competitor, respectively. The p-value indicates the significance level for detecting performance differences between two classifiers, with $\alpha =0.05$ serving as the minimum threshold for statistical significance. A p-value below $\alpha =0.05$ indicates that one classifier significantly outperforms the other. The h-value is expressed as “+”, “−”, or “=” to signify whether MTLBORKS-CNN is significantly better than, worse than, or comparable to its competitors in solving the eight selected benchmark datasets. Based on the h-values presented in Table 6, it becomes evident that MTLBORKS-CNN consistently and significantly outperforms all of its competitors.

To comprehensively assess performance differences between MTLBORKS-CNN and its competitors across the eight chosen benchmark datasets, a multiple comparison analysis known as the Friedman test [48] is conducted. This test ranked all compared algorithms based on their classification accuracies, with the overall ranking presented in

Table 7. Friedman test results.

Classifier	Ranking	Chi-Square Statistic	p-Value
ScatNet-2	9.6875	108.7892	$0.00\times {10}^0$
SAA-3	13.9375
LDANet-2	9.1250
SVM+Poly	15.4375
SVM+RBF	14.2500
NNet	17.0000
RandNet-2	9.8750
DBN-3	12.8125
PCANet-2	9.3125
AlexNet	10.0625
ResNet-50	5.8125
VGGNet-16	8.0000
MobileNet-v2	4.7850
GoogleNet	3.3750
psoCNN	2.8125
EvoCNN	5.5000
MTLBORKS-CNN	1.1250

. MTLBORKS-CNN achieves the best rank, followed by psoCNN, GoogleNet, MobileNet-v2, EvoCNN, ResNet-50, VGGNet-16, LDANet-2, PCANet-2, ScatNet-2, RandNet-2, AlexNet, DBN-3, SAA-3, SVM+RBF, SVM+Poly, and NNet. Additionally, Table 7 demonstrates significant global differences among all compared classifiers, with the p-value below the threshold of $\alpha =0.05$.

To investigate concrete deviations, three post hoc procedures [48], i.e., Bonferroni–Dunn, Holm, and Hochberg, are applied by using MTLBORKS-CNN as the control algorithm. The results of these post hoc analyses, including z-values, unadjusted p-values, and adjusted p-values (APVs), are presented in

Table 8. Post hoc test results.

MTLBORKS-CNN vs.	z	Unadjusted p	Bonferroni-Dunn p	Holm p	Hochberg p
NNet	$6.29\times {10}^0$	$0.00\times {10}^0$	$0.00\times {10}^0$	$0.00\times {10}^0$	$0.00\times {10}^0$
SVM+Poly	$5.67\times {10}^0$	$0.00\times {10}^0$	$0.00\times {10}^0$	$0.00\times {10}^0$	$0.00\times {10}^0$
SVM+RBF	$5.20\times {10}^0$	$0.00\times {10}^0$	$3.00\times {10}^{-6}$	$3.00\times {10}^{-6}$	$3.00\times {10}^{-6}$
SAA-3	$5.07\times {10}^0$	$0.00\times {10}^0$	$6.00\times {10}^{-6}$	$5.00\times {10}^{-6}$	$5.00\times {10}^{-6}$
DBN-3	$4.63\times {10}^0$	$4.00\times {10}^{-6}$	$5.90\times {10}^{-5}$	$4.40\times {10}^{-5}$	$4.40\times {10}^{-5}$
AlexNet	$3.54\times {10}^0$	$4.00\times {10}^{-4}$	$6.40\times {10}^{-3}$	$4.40\times {10}^{-3}$	$4.40\times {10}^{-3}$
RandNet-2	$3.47\times {10}^0$	$5.29\times {10}^{-4}$	$8.47\times {10}^{-3}$	$5.29\times {10}^{-3}$	$5.29\times {10}^{-3}$
ScatNet-2	$3.39\times {10}^0$	$6.96\times {10}^{-4}$	$1.11\times {10}^{-2}$	$6.26\times {10}^{-3}$	$6.26\times {10}^{-3}$
PCANet-2	$3.24\times {10}^0$	$1.18\times {10}^{-3}$	$1.89\times {10}^{-2}$	$9.47\times {10}^{-3}$	$9.47\times {10}^{-3}$
LDANet-2	$3.17\times {10}^0$	$1.53\times {10}^{-3}$	$2.45\times {10}^{-2}$	$1.07\times {10}^{-2}$	$1.07\times {10}^{-2}$
VGGNet-16	$2.72\times {10}^0$	$6.47\times {10}^{-3}$	$1.04\times {10}^{-1}$	$3.88\times {10}^{-2}$	$3.88\times {10}^{-2}$
ResNet-50	$1.86\times {10}^0$	$6.34\times {10}^{-2}$	$1.01\times {10}^0$	$3.17\times {10}^{-1}$	$3.17\times {10}^{-1}$
EvoCNN	$1.73\times {10}^0$	$8.31\times {10}^{-2}$	$1.33\times {10}^0$	$3.33\times {10}^{-1}$	$3.33\times {10}^{-1}$
MobileNet-v2	$1.49\times {10}^0$	$1.37\times {10}^{-1}$	$2.20\times {10}^0$	$4.12\times {10}^{-1}$	$4.12\times {10}^{-1}$
GoogleNet	$8.91\times {10}^{-1}$	$3.73\times {10}^{-1}$	$5.97\times {10}^0$	$7.46\times {10}^{-1}$	$5.04\times {10}^{-1}$
psoCNN	$6.68\times {10}^{-1}$	$5.04\times {10}^{-1}$	$8.06\times {10}^0$	$7.46\times {10}^{-1}$	$5.04\times {10}^{-1}$

. All post hoc procedures confirm the significant superiority of MTLBORKS-CNN compared to SVM+RBF, SVM+Poly, NNet, DBN-3, AlexNet, RandNet-2,ScatNet-2, PCANet-2, and LDANet-2, as evidenced by APVs consistently falling below $\alpha =0.05$. Furthermore, the Holm and Hochberg procedures confirm the significant improvement of MTLBORKS-CNN over VGGNet-16.

4.3.2. Classification Accuracy in Handling MNIST-Fashion Dataset

Table 9. Performance of MTLBORKS-CNN and its competitors in solving MNIST-Fashion.

Classifier	Classification Accuracy	# Trainable Parameters
HOG+SVM 1	92.60%	NA
2C1P 1	92.50%	100 k
2C1P2F+Dropout 1	91.60%	3.27 M
AlexNet	89.90%	60 M
ResNet-50	91.12%	23 M
VGGNet-16	90.08%	41 M
MobileNet-v2	92.45%	7 M
GoogleNet	92.69%	16 M
SqueezeNet-200 [47]	90.00%	500 k
MLP 256-128-64 1	90.00%	41 k
MLP 256-128-100 1	88.33%	3 M
GRU+SVM+Dropout	89.70%	NA
GRU+SVM 1	88.80%	NA
3C1P2F+Dropout 1	92.60%	7.14 M
3C2F 1	90.70%	NA
EvoCNN [13]	94.53%	6.68 M
psoCNN [42]	92.81%	2.58 M
Human Performance 1	83.50%	NA
MTLBORKS-CNN (Best)	93.27%	1.98 M
MTLBORKS-CNN (Mean)	93.09%	2.31 M

¹ https://github.com/zalandoresearch/fashion-mnist (accessed on 3 June 2023).

presents the accuracies and network complexities (determined by the total number of trainable parameters) for all classifiers used with the MNIST-Fashion dataset. As previously observed in Table 5 for the initial eight benchmark datasets, MSA-based approaches continue to demonstrate superior performance in classifying the MNIST-Fashion dataset. This outcome reaffirms the promising capability of MSAs to autonomously design optimal CNN architectures for diverse classification tasks, minimizing the need for human domain expertise.

For the MNIST-Fashion dataset, MTLBORKS-CNN achieves a commendable classification accuracy of 93.27%, securing the second-best performance. EvoCNN and psoCNN claim the top and third positions, with classification accuracies of 94.53% and 92.81%, respectively. Although EvoCNN outperforms MTLBORKS-CNN by a marginal 1.26% in classification accuracy on the MNIST-Fashion dataset, it is noteworthy that the optimal CNN architecture produced by MTLBORKS-CNN comprises a mere 1.96 million trainable parameters. This represents a significant reduction, specifically 70.36% less than EvoCNN (with 6.68 million trainable parameters) and 61.38% less than psoCNN (with 2.58 million trainable parameters). In comparison to EvoCNN and psoCNN, MTLBORKS-CNN demonstrates a better balance between accuracy and network complexity when crafting optimal CNN architectures for a specific classification task. In an era where automated intelligent systems, such as facial recognition systems, traffic monitoring systems, and vehicle auto-driving systems, are increasingly integrated into daily life, designing resource-efficient deep learning models in terms of computing power and energy consumption is crucial. Given its potential to automatically identify network architectures that offer both high accuracy and low complexity, MTLBORKS-CNN emerges as an attractive solution for developers aiming to deploy various mobile intelligent systems.

Table 9 also reveals relatively lower classification accuracies for other hand-crafted classifiers when dealing with the MNIST-Fashion dataset. These classifiers, including AlexNet, ResNet-50, VGGNet-16, MobileNet-v2, and GoogleNet, feature more complex network architectures, with 60 million, 23 million, 41 million, 7 million, and 16 million trainable parameters, respectively. These manually designed deep learning models often incorporate the excessive trainable parameters, resulting in a significant increase in computational power consumption without a substantial enhancement in classification accuracy. In contrast, MTLBORKS-CNN demonstrates competitive performance when solving the MNIST-Fashion dataset without the need for data augmentation techniques or complex network structures. MTLBORKS-CNN initiates its learners with simpler network architectures, contributing to lower complexity levels and faster convergence rates during the search process. The remarkable classification accuracy exhibited by MTLBORKS-CNN reveals the potential to achieve competitive results through the utilization of relatively straightforward network configurations.

4.3.3. CNN Architectures Optimized by MTLBORKS-CNN for Different Datasets

In this section, the ability of MTLBORKS-CNN to optimize CNN architectures based on provided datasets is investigated. Figure 9 displays the convergence curves generated by MTLBORKS-CNN when optimizing the network hyperparameters crucial for solving each selected dataset. It is evident that MTLBORKS-CNN effectively identifies optimal network hyperparameters for CNN architectures, achieving the highest classification accuracy for each dataset within a relatively small iteration number. These observations emphasize the competitive accuracy and rapid convergence speed of MTLBORKS-CNN in the context of CNN architecture optimization.

Table 10. Optimal CNN architecture constructed by MTLBORKS-CNN in solving nine datasets.

Dataset	Layers	Optimized Network Hyperparameters
MNIST	Convolutional	$N_1^{Fil}=97$$, S_1^{Ker}=9\times 9$
	Maximum Pooling	$S_1^{Poor}=2\times 2$$, S_1^{Str}=1\times 1$
	Convolutional	$N_2^{Fil}=174$$, S_2^{Ker}=9\times 9$
	Convolutional	$N_3^{Fil}=149$$, S_3^{Ker}=8\times 8$
	Average Pooling	$S_3^{Pool}=3\times 3$, $S_3^{Str}=1\times 1$
	Fully-Connected	$N_1^{Neu}=10$
MNIST-RD	Convolutional	$N_1^{Fil}=143$$, S_1^{Ker}=9\times 9$
	Convolutional	$N_2^{Fil}=71$$, S_2^{Ker}=9\times 9$
	Convolutional	$N_3^{Fil}=155$$, S_3^{Ker}=9\times 9$
	Average Pooling	$S_3^{Pool}=3\times 3$$, S_3^{Str}=1\times 1$
	Fully-Connected	$N_1^{Neu}=10$
MNIST-RB	Convolutional	$N_1^{Fil}=115$$, S_1^{Ker}=3\times 3$
	Maximum Pooling	$S_1^{Pool}=3\times 3$$, S_1^{Str}=1\times 1$
	Convolutional	$N_2^{Fil}=132$$, S_2^{Ker}=9\times 9$
	Maximum Pooling	$S_2^{Pool}=3\times 3$$, S_2^{Str}=1\times 1$
	Convolutional	$N_3^{Fil}=37$$, S_3^{Ker}=9\times 9$
	Fully-Connected	$N_1^{Neu}=10$
MNIST-BI	Convolutional	$N_1^{Fil}=247$$, S_1^{Ker}=3\times 3$
	Convolutional	$N_2^{Fil}=159$, $S_2^{Ker}=5\times 5$
	Maximum Pooling	$S_2^{Pool}=2\times 2$$, S_2^{Str}=1\times 1$
	Convolutional	$N_3^{Fil}=107$$, S_3^{Ker}=9\times 9$
	Average Pooling	$S_3^{Pool}=3\times 3$, $S_3^{Str}=1\times 1$
	Fully-Connected	$N_1^{Neu}=10$
MNIST-RD+BI	Convolutional	$N_1^{Fil}=145$$, S_1^{Ker}=7\times 7$
	Convolutional	$N_2^{Fil}=161$, $S_2^{Ker}=7\times 7$
	Average Pooling	$S_2^{Pool}=3\times 3$$, S_2^{Str}=1\times 1$
	Convolutional	$N_3^{Fil}=201$, $S_3^{Ker}=9\times 9$
	Fully-Connected	$N_1^{Neu}=10$
Rectangles	Convolutional	$N_1^{Fil}=155$$, S_1^{Ker}=4\times 4$
	Maximum Pooling	$S_1^{Pool}=1\times 1$$, S_1^{Str}=1\times 1$
	Convolutional	$N_2^{Fil}=114$$, S_2^{Ker}=6\times 6$
	Average Pooling	$S_2^{Pool}=3\times 3$$, S_2^{Str}=2\times 2$
	Convolutional	$N_3^{Fil}=210$$, S_3^{Ker}=9\times 9$
	Average Pooling	$S_3^{Pool}=3\times 3$$, S_3^{Str}=1\times 1$
	Fully-Connected	$N_1^{Neu}=2$
Rectangles-I	Convolutional	$N_1^{Fil}=105$$, S_1^{Ker}=3\times 3$
	Maximum Pooling	$S_1^{Pool}=3\times 3$$, S_1^{Str}=1\times 1$
	Convolutional	$N_2^{Fil}=97$$, S_2^{Ker}=9\times 9$
	Convolutional	$N_3^{Fil}=94$$, S_3^{Ker}=9\times 9$
	Maximum Pooling	$S_3^{Pool}=1\times 1$$, S_3^{Str}=2\times 2$
	Fully-Connected	$N_1^{Neu}=2$
Convex	Convolutional	$N_1^{Fil}=57$, $S_1^{Ker}=9\times 9$$,$
	Maximum Pooling	$S_1^{Pool}=3\times 3$$, S_1^{Str}=1\times 1$
	Convolutional	$N_2^{Fil}=63$$, S_2^{Ker}=9\times 9$
	Maximum Pooling	$S_2^{Pool}=1\times 1$$, S_2^{Str}=1\times 1$
	Convolutional	$N_3^{Fil}=40$$, S_3^{Ker}=9\times 9$
	Fully-Connected	$N_1^{Neu}=2$
MNIST-Fashion	Convolutional	$N_1^{Fil}=166$$, S_1^{Ker}=3\times 3$
	Maximum Pooling	$S_1^{Pool}=2\times 2$$, S_1^{Str}=1\times 1$
	Convolutional	$N_2^{Fil}=181$$, S_2^{Ker}=5\times 5$
	Average Pooling	$S_2^{Poor}=2\times 2$$, S_2^{Str}=1\times 1$
	Fully-Connected	$N_1^{Neu}=10$

and Figure 10 present the optimal network hyperparameters and corresponding CNN architectures identified by MTLBORKS-CNN, achieving the highest classification accuracy across all nine selected datasets. MTLBORKS-CNN consistently designs optimal CNN architectures with a single fully-connected layer for all benchmark datasets. This aligns with prior research [49], suggesting that CNN models with a single fully-connected layer can outperform those with multiple fully-connected layers. Additionally, this investigation reveals that for specific benchmark datasets such as MNIST-RD, MNIST-BI, MNIST-RB+BI, and Rectangles-I, including a pooling layer between consecutive convolutional layers is not imperative for achieving the highest classification accuracy. These findings confirm that MTLBORKS-CNN can autonomously design high-performance network architectures with low complexity for various classification tasks, without the need for prior domain knowledge.

4.3.4. Ablation Experiment

Extensive performance analyses confirm the superiority of MTLBORKS-CNN in classifying nine selected image datasets. This exceptional performance can be attributed to three key innovations: (i) the integration of a social learning concept in the modified teacher phase, (ii) the introduction of a self-learning mechanism and an adaptive peer-learning scheme in the modified learner phase, and (iii) the utilization of a dual-criterion selection scheme to determine the survival of learners in subsequent generations. To gain deeper insights into the impacts of these modifications on solving complex problems such as CNN architecture optimization, an ablation experiment is conducted to assess the individual contributions of each modification.

In this ablation experiment, the original TLBO-CNN served as the baseline method for solving CNN architecture optimization. Three MTLBORKS-CNN variants were introduced: MTLBORKS-CNN-v1, MTLBORKS-CNN-v2, and MTLBORKS-CNN-v3, each designed to isolate the effects of different modifications. MTLBORKS-CNN-v1 incorporated the modified teacher phase while retaining the original learner phase, whereas MTLBORKS-CNN-v2 employed the modified learner phase alongside the original teacher phase. MTLBORKS-CNN-v3 integrated both the modified teacher and learner phases. All three MTLBORKS-CNN variants retained the original greedy-based selection scheme to assess the impact of the proposed dual-criterion selection scheme.

Table 11. Ablation experiment results to investigate the effectiveness of each modification introduced into MTLBORKS-CNN.

Datasets	Metrics	TLBO-CNN	MTLBORKS-CNN-v1	MTLBORKS-CNN-v2	MTLBORKS-CNN-v3	MTLBORKS-CNN
MNIST	Classification Accuracy	98.54%	99.54%	99.55%	99.57%	99.65%
	# Trainable Parameters	12.10 M	9.10 M	6.88 M	6.12 M	3.50 M
MNIST-RD	Classification Accuracy	94.66%	95.90%	96.01%	96.48%	96.50%
	# Trainable Parameters	10.10 M	4.24 M	3.99 M	3.57 M	2.77 M
MNIST-RB	Classification Accuracy	96.91%	97.47%	97.45%	98.07%	98.11%
	# Trainable Parameters	7.23 M	2.72 M	3.11 M	2.84 M	2.68 M
MNIST-BI	Classification Accuracy	95.53%	96.82%	97.02%	97.13%	97.15%
	# Trainable Parameters	5.02 M	3.48 M	4.31 M	3.21 M	3.03 M
MNIST-RD+BI	Classification Accuracy	77.58%	79.53%	79.66%	82.17%	82.28%
	# Trainable Parameters	3.71 M	5.93 M	4.03 M	3.55 M	1.84 M
Rectangles	Classification Accuracy	99.68%	99.88%	99.90%	99.97%	99.98%
	# Trainable Parameters	12.60 M	4.72 M	4.87 M	3.64 M	2.63 M
Rectangles-I	Classification Accuracy	95.71%	96.03%	96.14%	97.30%	97.39%
	# Trainable Parameters	6.63 M	1.17 M	4.62 M	2.84 M	1.60 M
Convex	Classification Accuracy	95.20%	96.41%	96.59%	97.85%	97.95%
	# Trainable Parameters	1.54 M	4.00 M	2.14 M	1.48 M	0.55 M
MNIST-Fashion	Classification Accuracy	91.89%	92.44%	92.43%	92.70%	93.27%
	# Trainable Parameters	4.31 M	3.09 M	3.21 M	2.69 M	1.98 M

presents the performance of each MTLBORKS-CNN variant, including classification accuracy and network complexity (i.e., number of trainable parameters), across the nine selected image datasets. The best results obtained in each dataset are presented in bold font. These results clearly indicate that all MTLBORKS-CNN variants significantly improve classification accuracy and network complexity compared to TLBO-CNN when applied to most image datasets. These results confirm the effectiveness of incorporating the modified teacher and learner phases into the original TLBO to enhance its CNN architecture optimization capabilities. Compared to the original teacher phase in TLBO-CNN, MTLBORKS-CNN introduces valuable enhancements. It achieves improved population diversity by generating unique social exemplars and unique population means for each learner, based on different sets of superior population members. Despite the increasing population diversity, all learners continue to benefit from personalized guidance via their unique exemplars, which accelerates their convergence toward promising solution regions. Additionally, the modified learner phase in MTLBORKS-CNN, designed to realistically simulate classroom learning dynamics, provides greater flexibility compared to the original learner phase in TLBO-CNN. It empowers learners to explore new CNN architectures using diverse learning strategies. Specifically, the self-learning mechanism encourages exploration by enabling each learner to search for novel CNN architectures through random perturbations. Meanwhile, the adaptive peer learning scheme fosters exploitation by facilitating collaboration and knowledge sharing among learners through probabilistic interactions with multiple peers, based on their fitness values. This collaborative approach significantly enhances the efficiency of the search for optimal CNN architectures.

The results from MTLBORKS-CNN-v1 and MTLBORKS-CNN-v2 reveal varying degrees of performance improvement over the original TLBO-CNN when searching for optimal CNN architectures. Specifically, MTLBORKS-CNN-v2 outperforms MTLBORKS-CNN-v1 in terms of classification accuracy and network complexity for four image datasets (MNIST, MNIST-RD, MNIST-RD+BI, and Convex). Conversely, MTLBORKS-CNN-v1 achieves higher classification accuracy and lower network complexity than MTLBORKS-CNN-v2 for two image datasets (MNIST-RB and MNIST-Fashion). Both MTLBORKS-CNN-v1 and MTLBORKS-CNN-v2 exhibit superior network complexity and classification accuracy, respectively, when solving the remaining three image datasets (MNIST-BI, Rectangles, and Rectangles-I), suggesting the unique contributions of the modified teacher and learner phases in automating network design tasks. Furthermore, it is worth noting that MTLBORKS-CNN-v3 outperforms both MTLBORKS-CNN-v1 and MTLBORKS-CNN-v2 when solving most of the datasets in terms of higher classification accuracy and lower network complexity. These results confirm the synergistic effects generated by both the modified teacher and learner phases in achieving high-performing yet low-complexity CNN architectures for the designated classification tasks.

Finally, it is noteworthy that the complete MTLBORKS-CNN surpasses MTLBORKS-CNN-v3 in terms of classification accuracy and network complexity for all nine selected image datasets. While the advantage of MTLBORKS-CNN over MTLBORKS-CNN-v3 in classification accuracy is marginal for most datasets, the performance gains achieved by MTLBORKS-CNN in network complexity are substantial. Unlike TLBO-CNN and the other three MTLBORKS-CNN variants, which rely on a fitness-based selection scheme, the proposed MTLBORKS-CNN employs a dual-criterion selection scheme. This approach determines the survival of learners in the next generation by considering both fitness and diversity criteria. This enhancement maintains population diversity throughout the search process and mitigates the risk of premature convergence, allowing learners with relatively lower fitness but greater diversity to continue into the next iteration. In the context of CNN architecture optimization, the dual-criterion selection scheme has proven effective to assist MTLBORKS-CNN in discovering innovative CNN architectures with reduced network complexity that achieve high accuracy in solving classification tasks.

5. Conclusions

The proposed MTLBORKS offers an automatic and flexible solution for optimizing CNN architecture in image classification. MTLBORKS-CNN achieves this optimization by effectively encoding scheme and design constraints, leading to the discovery of novel network architectures. The modified teacher phase integrates social learning to enhance diversity, while the modified learner phase employs self-learning and adaptive peer learning to better represent real-world learning processes. Additionally, a dual-criterion selection scheme preserves promising learners, ensuring long-term population quality by considering both fitness and diversity criteria. The performance of MTLBORKS-CNN is validated on nine benchmark datasets, showcasing competitive accuracy and simplified architectures ideal for edge devices.

Future directions encompass the integration of advanced network blocks, such as DenseNet and ResNet, into the automatic network design task to create innovative CNN architectures for tackling complex problems. Another avenue involves exploring the formulation of automatic network architecture design as a multiobjective optimization problem, considering conflicting requirements such as inference speed and network complexity. This approach could lead to the development of a multiobjective version of MTLBORKS-CNN, capable of discovering optimized CNN architectures to meet diverse stakeholder needs. Lastly, it is recommended to investigate MTLBORKS-CNN’s effectiveness in handling more complex real-world datasets, such as those related to human action recognition and medical diagnosis. These efforts hold the potential to further enhance the performance and versatility of MTLBORKS-CNN.