Deriving Controllable Local Optimal Solutions through an Environment Parameter Fixed Algorithm

Abstract

This paper addresses the challenge of optimizing objective functions in engineering problems influenced by multiple environmental factors, such as temperature and humidity. Traditional modeling approaches often struggle to capture the complexities of non-ideal situations. In this research, we propose a novel approach called the Environment Parameter Fixed Algorithm (EPFA) for optimizing the objective function of a deep neural network (DNN) trained in a specific environment. By fixing the environmental parameters in the DNN defined objective function, we transform the original optimization problem into a control parameter optimization problem. We integrate EPFA-CLS (Controllable local-Optimal Solution) with Gradient Descent and algorithms such as Adagrad to obtain the optimal solution. To demonstrate the concept, we apply our approach to an optimal course model and validate it using optimal course and Boston house price datasets. The results demonstrate the effectiveness of our approach in handling optimization problems in complex environments, offering promising outcomes for practical engineering applications.

1. Introduction

Optimization can arrive at the most suitable solution from among various solutions by finding the parameters that maximize or minimize the value of the function in an objective function [1]. Constraint optimization is a branch of optimization which deals with problems where the objective function is subject to certain constraints [2]. Another type of optimization problem is based on the specific definition of the objective function, such as quadratic programming (QP), which optimizes a quadratic objective function [3]. These different types of optimization problems find applications in solving engineering problems, including optimizing space vehicle trajectories, designing cost effective civil engineering structures such as frames, foundations, bridges, towers, chimneys, and dams, and addressing various forms of random loading [4].

However, accurately representing the objective function in non-ideal scenarios influenced by diverse environmental factors poses challenges in optimization problems. To address these challenges, we introduce a novel optimization problem called trained Deep Neural Network (DNN) objective optimization. This approach utilizes a trained DNN as the objective function.

Since the DNN reflects environmental constraints as inputs to the model in [5], environment parameters (E) and control parameters (C) have been defined. The term E refers to the diverse set of measured values associated with the environment, such as temperature and humidity. On the other hand, C refer to the factors that enable manipulation and control over the system. Additionally, parameters C and E exhibit an independent relationship. Figure 1 illustrates the relationship between the defined parameters and input parameters. A trained DNN Controllable Local optimal Solution (CLS) is defined as the solution of the trained DNN objective optimization and represents an optimal control parameter when a specific environment parameter is used.

One of the key advantages of defining the objective function as a trained DNN is its flexibility. Not only can functions be modeled-based on data through deep learning-based regression [6,7,8,9,10], but they can also incorporate non-ideal cases influenced by various environments [5]. Data-driven approaches can accurately capture the effects of diverse environments on both input and output data [11]. Moreover, objective function modeling is adaptable and can be applied to control problems, within the definition of regression. The function value of the objective function can be defined by the user. In Figure 2, the current input is represented by a red dot. In such a situation, the user can identify a position (blue dot) where the function value can be optimized and adjust the input parameter accordingly. This allows the user to acquire data aligned with their desired purpose and then model the objective function for control purposes.

To obtain the solution for optimizing the Trained Deep Neural Network (DNN) objective, we propose the Environment Parameter Fixed Algorithm with Controllable Local Optimal Solution (EPFA-CLS).

The EPFA-CLS algorithm reduces the original optimization problem to an optimization problem for control parameters by fixing the environment parameters in the DNN defined objective function [12]. Subsequently, the CLS is obtained through the application of Gradient Descent (or Gradient Ascent) optimization techniques. By incorporating environmental constraints and leveraging deep learning-based regression analysis, our proposed EPFA-CLS algorithm offers a promising approach to obtaining controllable local optimal solutions for complex engineering problems.

EPFA-CLS is a useful approach for determining the optimal course of a ship when evading infrared guided missiles [13]. To evade these missiles, the first step is to change the missile’s locked-on target [14]. Flares are commonly used as a deceptive measure, emitting a higher temperature than the ship and diverting the missile’s attention [15]. As shown in Figure 3, the ship fires the flare to change the locked-on object from the ship to the flare.

After deploying a flare, the ship needs to navigate an optimal course, considering its direction, velocity, and specific conditions. The course must be reachable based on the ship’s top speed, such as the FFG-2 large class ship with a maximum speed of 15.4 m/s [16]. Additionally, the miss distance, which represents the distance between the ship and the missile when the missile hits the flare, should be increased [17]. This miss distance is influenced by wind conditions, and an optimization problem needs to be formulated to determine the ship’s course. The objective function takes inputs such as ship direction, velocity, wind direction, and wind velocity, with the output being the miss distance (MD). Figure 3 illustrates miss distance.

In summary, EPFA-CLS provides advantages for modeling the problem, including non-ideal factors such as wind information, and enables precise control over the ship’s direction and velocity to navigate the optimal path.

A similar approach is presented in several prior works [18,19,20,21]. The work by [18] addresses the problem of finding the global optimum by employing a convex approach to transform the ICNN problem. However, our focus is on solving the problem of CLS derivation, which is distinct from their work. On the other hand, [19] does not adequately reflect the current point as it only considers a uniform initial condition and lacks the incorporation of control concepts. A structured prediction is achieved by leveraging the concept of an energy function commonly used for training neural networks. Furthermore, both [20] and [21] introduce the notion of a control variable and an uncontrollable variable. However, they establish a dependent relationship between these variables, where the past control variable becomes the present uncontrollable variable due to the inclusion of time conditions. In contrast, our method employs a different optimization approach by defining a new loss function as the objective function.

Our contributions in this paper can be summarized from three aspects:

First, we raise another problem of constraint optimization, named trained DNN objective optimization, and we define terms that fit the context of the problem.

Second, we propose the EPFA-CLS method, which enables the representation of a trained DNN under a specific environment parameter, effectively capturing the desired constraints. By fixing the environment parameter, we transform the original optimization problem into a problem of optimizing control parameters in the DNN defined objective function. Subsequently, well-established optimization algorithms such as Gradient Descent are employed to solve this problem. Once the environment parameter is fixed, the CLS can be obtained by capturing the current state of the trained DNN under the specific environment.

Third, the EPFA-CLS algorithm verifies feasibility with the optimal course dataset and with the Boston housing dataset.

This paper is structured as follows. Section 2 provides background on the optimization; the problem covered in this paper is explained in detail and the terms and concepts are organized. In Section 3, the proposed method is explained in detail with a flow chart of the algorithm. Section 4 shows whether the EPFA-CLS is valid by experimenting with the datasets. We verify that the CLS can be derived with the EPFA-CLS by using the Boston Housing Dataset and the optimal course dataset. Section 5 summarizes the results.

2. Background

This section consists of two parts. The section on regression analysis presents a brief description of regression and discusses how to measure accuracy. The section titled Optimization introduces the concept and the optimizers of deep learning algorithms.

2.1. Deep Learning-Based Regression

Deep learning regression is widely used to solve tasks where the goal is to predict continuous values through training. The training data are sample values from myriad data that explain social phenomena. Therefore, by following the principle of regression analysis, sample data can be trained and used as a continuous nonlinear mapping function [22]. To obtain a predicted value, you can output a value by providing input for the mapping function. The EPFA-CLS uses a DNN because it needs to construct a regression model when the data are one-dimensional. The DNN is one of the networks widely used in regression models in which one-dimensional data are the input [23]. The DNN has been used in many studies, such as for speech development.

DNN-based regression analysis also comes in several types, including simple regression and multiple regression [24,25]. This paper uses multiple regression analysis in which the number of dependent variables is 1, so the output neuron is 1 [26]. The intermediate layer between input and output layers is called the hidden layer. The structure of such a network is determined by the number of hidden layers and neurons, and the number is determined by the complexity of the problem, the amount of data, and the performance of the hardware [27].

The accuracy of deep learning regression is measured with $R^2$. The sum of the squares of the deviations from the mean is a measure of the overall variance of the dependent variable. The sum of squared deviations for the regression line is a measure of the extent to which the regression fails to account for the dependent variable (a measure of noise). Therefore, $R^2$ is a measure of the degree to which the overall variation of the dependent variable is explained by regression. A high value for $R^2$ indicates that the regression model explains the variation in the dependent variable quite well, and is clearly important if the model is to be used for predictive purposes [28].

In Equation (1), total sum of squares ($SST$) is total change, residual sum of squares ($SSR$) means unexplainable change, and explained sum of squares ($SSE$) is explainable change. In Equation (1), when $SSR$ increases, the coefficient of determination $R^2$ increases, so an increase in $SSR$ means a decrease in $SSE$. Therefore, the estimated regression model predicts dependent variable y quite well. The coefficient of determination $R^2$ is the rate of change in dependent variable y that can be explained by independent variable x in the regression model. The range of $R^2$ is $0\le R^2\le 1$. If it is 0, there is no correlation between the independent variable and the dependent variable; the closer to 1, the greater the correlation [28].

$$\begin{array}{cc}\hfill R^2& =\frac{SSR}{SST}=1-\frac{SSE}{SST}\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill SST& =\sum _{i=0}^n{\left(y_i-\overline{y}\right)}^2\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill SSE& =\sum _{i=0}^n{\left({\hat{y}}_i-\overline{y}\right)}^2\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill SSR& =\sum _{i=0}^n{\left(y_i-{\hat{y}}_i\right)}^2\hfill \end{array}$$

(4)

However, even if the value of $R^2$ is large, it may not be an important model. $R^2$ will increase as the proportion of $SSR$ increases. When it is clear which x can explain dependent variable y (as in an experiment), $SSR$ will have a large value. However, regression through machine learning is greatly affected by $R^2$ because the predicted value is important. In a DNN, prediction is the main goal. Therefore, it can be said that the larger the $R^2$ value, which is the criterion for accuracy in regression analysis, the higher the explanatory power of the model. After training, $R^2$ is considered important when trying to obtain training and validation accuracy [28].

2.2. Optimization

In the this section, the concept of optimization and the optimizer of the training algorithm are explained.

$$\begin{array}{c}\hfill \hat{x}=\underset{x}{argmax}f\left(x\right)\end{array}$$

(5)

In Equation (5) [1], optimization is the set of variables x for which objective function f can reach an optimal value. Optimization problems arise in various areas of mathematics, including applied sciences, engineering, economics, medicine, and statistics [1]. These problems involve finding the set of variables, denoted as x, for which an objective function f can achieve an optimal value. Depending on the scope of the search area, optimization can be classified into global optimization and local optimization [2,29]. Global optimization aims to find the best solution over the entire search area, while local optimization focuses on finding the best solution within a specific region. When constraints are imposed on the allowable values of variables, such as independent variables, it becomes a constraint optimization problem [2].

Table 1. Organized by type of optimization problem and explanations of the problems it raises.

Problem	Explanation
Linear Programming [34]	A problem in which the objective function and all constraint functions are linear
Non-Linear Programming [35]	A problem in which either the objective function or any constraint function is nonlinear
Quadratic Programming [3]	A problem in which the objective function is quadratic
Trained DNN objective optimization (proposed)	Constraint optimization problem in a specific environment when the objective function is a trained DNN

illustrates various problems that fall under the category of optimization problems. These problems require formulating objective functions and employing algorithms to obtain the optimal solutions. For instance, in linear programming problems, the production cost optimization can be achieved by formulating objective functions using appropriate formulas [30]. In the financial field, quadratic programming offers solutions by modeling the objective function as a quadratic function [31]. Consequently, problems are defined based on the structure of the objective function and the shape of the constraint function. However, the modeling of complex engineering systems is computationally expensive [32,33].

Thus, this paper focuses on constraint optimization where a data-driven objective function, reflecting multiple environments, is applied within a specific environment. The data-based modeling process is a form of meta-modeling that involves acquiring data and has the advantage of simplifying complex models [33].

We raise a new problem, the trained DNN objective optimization, from the existing problems. Trained DNN objective optimization is constraint optimization in which a data-driven objective function reflecting multiple environments is applied in a specific environment. As a type of metamodelling, [36], deep learning-based regression techniques offer the advantage of modeling situations, even in non-ideal circumstances where traditional modeling approaches face difficulties. For example, in the domain of fish weighing in turbid water, deep learning regression, such as the one proposed by [5], can accurately estimate weights under non-ideal conditions.

Trained deep neural network (DNN) objective optimization is a similar approach to modeling with metamodel-based simulation optimization (MBSO) [33,36] and metamodeling, but it differs in terms of its focus on considering a specific environment.

Modeling these non-ideal situations requires redefining the optimization variables. The outcome of deep learning regression, denoted as $\hat{y}$, is influenced by various inputs, including the environmental conditions during data acquisition.

We can solve the problem by defining an environment parameter ($\mathbf{E}$) and a control parameter ($\mathbf{C}$). The control parameters are defined as optimization variables, while the environment parameter represents the specific environmental conditions and is not subject to optimization. The consideration of $\mathbf{C}$ and $\mathbf{E}$ is necessary due to the data-driven modeling approach, and they are independent of each other within the input parameter $\mathbf{X}$.

$$\begin{array}{ll}\hfill \mathbf{X}& =[x_1,x_2,\cdots ,x_{\phi }]=[\mathbf{E},\mathbf{C}]\hfill \end{array}$$

(6)

$$\begin{array}{ll}\hfill \mathbf{C}& =[c_1,c_2,\cdots ,c_m]\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill \mathbf{E}& =[e_1,e_2,\cdots ,e_n]\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill \phi & =n+m\hfill \end{array}$$

(9)

The term “controllable” originates from the concepts of environment parameters and control parameters. It refers to a constraint optimization problem with specific environment parameters. It can be understood in terms of control, where environment parameters cannot be controlled, but control parameters can be. Consequently, the optimization problem we need to solve can be derived by introducing the concepts of environment parameters and control parameters into Equation (5), leading to Equation (10).

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \underset{\mathbf{C}}{argmax}f\left([\mathbf{C},{\mathbf{E}}_{\mathit{initial}}]\right)\end{array}\end{array}$$

(10)

The proposed EPFA-CLS utilizes the existing gradient-based local optimization algorithm. So, the concept is explained as follows.

(1)

Gradient Descent [37]

This is one of the methods to find the local minimum of a function. It is a method of gradually finding parameters to minimize the value of the function by using the gradient specification to reach the minimum of the function. In Equation (11), the convergence speed of Gradient Descent controls the learning rate, $\alpha$. To reach the local maximum, change the sign of Equation (11). This is called Gradient Ascent.

$$\begin{array}{cc}\hfill w_{new}& =w_{old}-\alpha \frac{\partial y}{\partial w}\hfill \end{array}$$

(11)

(2)

Momentum [38]

Momentum is an optimization algorithm that gives inertia to the moving process during Gradient Descent. Giving inertia is a method of additionally moving by a certain value while remembering the slope value in the past. $\gamma$ is a parameter corresponding to the momentum, and momentum is accumulated in the past value and becomes faster, resulting in faster convergence and reduced vibration. Similarly, by changing the sign of Equation (12), the local maximum can be reached.

$$\begin{array}{cc}\hfill v_{new}& =\gamma v_{old}-\tau {▽}_{w}C\left(w\right)\hfill \\ \hfill w_{new}& =w_{old}-v_{new}\hfill \end{array}$$

(12)

(3)

Adagrad [39]

When the learning rate is too small, the learning time is too long, and when the learning rate is too large, Adagrad solves the problem of divergence through learning rate decay. The squares of the slopes in the term h are cumulatively added, and h term is used to adjust the learning rate in the formula to be updated. The downside of Adagrad is that the gradient accumulates in the denominator, making it too large. Similarly, by changing the sign of Equation (13), the local maximum can be reached.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill v_{new}& =v_{old}+{\left({▽}_{w}C\left(w_t\right)\right)}^2\hfill \\ \hfill w_{new}& =w_{old}-\frac{\eta }{\sqrt{v_{new}+ϵ}}{▽}_{w}C\left(w_t\right)\hfill \end{array}\end{array}$$

(13)

3. The Proposed EPFA-CLS

In this section, we introduce the EPFA-CLS, a proposed algorithm for deriving the CLS. Figure 4 presents a flowchart illustrating the algorithm of EPFA-CLS. The algorithm proceeds as follows: build a dataset to model the desired engineering phenomenon. The dataset comprises various environments as independent variables. Model the objective function using DNN-based nonlinear regression analysis [36]. Define independent variables as environment parameters and control parameters to derive optimal control parameters in a specific environment. The CLS is obtained by keeping the environment parameters fixed and updating the control parameters iteratively until the gradient approaches zero. This update process is performed using the Gradient Descent method. The code implementation of our proposed method can be found at https://github.com/otaejang/EPFA-CLS, accessed on 10 June 2023.

3.1. Preparing the Dataset

The initial step involves deep learning regression, which requires the preparation of a dataset for training purposes. Regression aims to capture the relationship between the input and output variables. When preparing the dataset, it is crucial to examine causality and identify the input parameters that significantly influence the output. Sufficient data should be collected to ensure an accurate representation of the underlying relationship. This dataset will serve as the foundation for training the regression model.

Table 2. Dataset reflecting environment parameters.

${\mathit{e}}_1$	${\mathit{e}}_2$	⋯	${\mathit{e}}_{\mathit{n}}$	${\mathit{c}}_1$	${\mathit{c}}_2$	⋯	${\mathit{c}}_{\mathit{m}}$	y
$v_{11}^e$	$v_{12}^e$	⋯	$v_{1n}^e$	$v_{1,n+1}^c$	$v_{1,n+2}^c$	⋯	$v_{1,n+m}^c$	$y_1$
$v_{21}^e$	$v_{22}^e$	⋯	$v_{2n}^e$	$v_{2,n+1}^c$	$v_{2,n+2}^c$	⋯	$v_{2,n+m}^c$	$y_2$
⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯
$v_{k1}^e$	$v_{k2}^e$	⋯	$v_{kn}^e$	$v_{k,n+1}^c$	$v_{k,n+2}^c$	⋯	$v_{k,n+m}^c$	$y_k$

is a generalization of the dataset constructed. Because the EPFA-CLS’s deep learning regression utilizes multiple regression analysis, the output variable $\left(y\right)$ is defined as 1, and the number of input variables ($\phi$) is irrelevant. Collect the dataset according to the definitions of $\mathbf{E}$ and $\mathbf{C}$ in input parameter $\mathbf{X}$. In Table 2, $x_1$ to $x_n$ are defined as $\mathbf{E}$, and $x_{n+1}$ to $x_{n+m}$ are defined as $\mathbf{C}$. Equation (9) is the total number of input variables, $\phi$ is the sum of the number of $\mathbf{E}$ parameters (n), and the number of $\mathbf{C}$ parameters (m). Additionally, since only the parameter is changed, the values in v are the same. Since deep learning regression needs to learn the corresponding dataset, v values are also important.

3.2. DNN-Based Regression

Figure 5 shows the structure of the DNN used for regression [26]. After setting $\mathbf{E}$ and $\mathbf{C}$, a dataset was collected for DNN training.

Data input parameter $\mathbf{X}$ is used for the input node, and y is ground truth, which is used for the cost function with the last output node for supervised learning. An input node is required to match the number of datasets ($\phi$) and an output node is set to 1. The DNN is composed of two hidden layers to prevent overfitting because the amount of data is not large. In addition, a rectified linear unit (ReLU) is an activation function, and Stochastic Gradient Descent (SGD) is an optimizer. After training is in progress, $R^2$ is used to determine the accuracy of the DNN. As mentioned in the Background section, after training, $R^2$ is used to determine prediction accuracy. If $R^2$ is close to 1, it means the DNN’s prediction has high accuracy and can be used as a mapping function.

Equations (14)–(17) are functions representing the relationship between input and output in a trained DNN and can be thought of as a black box [26]. After training is in progress, each weight (w) is learned. S stands for bias, and the values of the nodes of layer a are defined as $\mathbf{A}$, while the values of the nodes of layer b are defined as $\mathbf{B}$. Each w in Equations (14)–(16) is a different weight. The predicted value can be obtained through the following operation. Output $\hat{y}$ is the ReLU function for nonlinear operation and the weighted sum of the learned weights. The above process is denoted as f. Going forward, in this paper, f is a function of output y from inputs $\mathbf{E}$ and $\mathbf{C}$.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \left[\begin{array}{cccc}{w}_{11}& w_{12}& \cdots & w_{1,n+m}\\ w_{21}& w_{22}& \cdots & w_{2,n+m}\\ ⋮& ⋮& \ddots & ⋮\\ w_{\sigma 1}& w_{\sigma 2}& \cdots & w_{\sigma ,n+m}\end{array}\right]& \left[\begin{array}{c}{e}_1\\ ⋮\\ e_n\\ c_1\\ ⋮\\ c_m\end{array}\right]+\left[\begin{array}{c}{s}_1\\ ⋮\\ s_{\sigma }\end{array}\right]=\left[\begin{array}{c}{a}_1\\ ⋮\\ a_{\sigma }\end{array}\right]\hfill \\ \hfill {\mathbf{W}}_{\sigma ,n+m}\mathbf{X}& +\mathbf{s}=\mathbf{A}\hfill \end{array}\end{array}$$

(14)

$$\begin{array}{c}\hfill ReLU\left(\left[\begin{array}{cccc}{w}_{11}& w_{12}& \cdots & w_{1,\sigma }\\ w_{21}& w_{22}& \cdots & w_{2,\sigma }\\ ⋮& ⋮& \ddots & ⋮\\ w_{\rho 1}& w_{\rho 2}& \cdots & w_{\rho \sigma }\end{array}\right]\left[\begin{array}{c}{a}_1\\ ⋮\\ a_{\rho }\end{array}\right]+\left[\begin{array}{c}{s}_1\\ ⋮\\ s_{\rho }\end{array}\right]\right)=\left[\begin{array}{c}{b}_1\\ ⋮\\ b_{\rho }\end{array}\right]\end{array}$$

$$\begin{array}{cc}\hfill {\mathbf{W}}_{\rho \sigma }\mathbf{A}& +\mathbf{s}=\mathbf{B}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill ReLU\left(\left[\begin{array}{cccc}{w}_1& w_2& \cdots & w_{\rho }\end{array}\right]\left[\begin{array}{c}{b}_1\\ ⋮\\ b_{\rho }\end{array}\right]+\left[\begin{array}{c}s\end{array}\right]\right)& =\hat{y}\hfill \end{array}$$

$$\begin{array}{cc}\hfill {\mathbf{W}}_{\rho }\mathbf{B}+\mathbf{s}& =\hat{y}\hfill \end{array}$$

(16)

$$\begin{array}{c}\hfill \hat{y}=f\left(\mathbf{X}\right)=f\left([\mathbf{E},\mathbf{C}]\right)\end{array}$$

(17)

3.3. Environment Parameter Fixed

This section shows the limitations of optimization without fixation, as in Equation (5), and explains how fixation works in the algorithm. We fix the environment parameter and show what it means with the following formula:

$$\begin{array}{c}\hfill \underset{\mathbf{E},\mathbf{C}}{argmax}f\left([\mathbf{E},\mathbf{C}]\right)\end{array}$$

(18)

Equation (18) shows the equation when optimizing without fixing. However, the result of this equation is not the CLS. As mentioned in [17], multiple environments can be represented by incorporating environment parameters as inputs, and we are dealing with optimizations of specific environment parameters. Although optimal $\mathbf{E}$ plays a role in increasing the output of the regression model, it cannot be moved in practice.

$$\begin{array}{cc}\hfill {\mathbf{E}}_{initial}& =[{\epsilon }_1,{\epsilon }_2,\cdots ,{\epsilon }_n]\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill y& =f\left([{\mathbf{E}}_{initial},\mathbf{C}]\right)\hfill \end{array}$$

(20)

The following problems can be solved through parameter fixing, which is the core idea of the EPFA-CLS. The EPFA-CLS algorithm reduces the original optimization problem to an optimization problem for control parameters by fixing the environment parameters in the DNN-defined objective function [12]. Equation (20) demonstrates a method of deriving the CLS through fixation. One f is generated by training the dataset for all $\mathbf{E}$ and $\mathbf{C}$. Fixing ${\mathbf{E}}_{initial}$ to a specific value in the algorithm is to substitute a value with a constant $\left(\epsilon \right)$. Therefore, after training the regression model, input the value as $\epsilon$.

3.4. Optimization

This section introduces the optimization process, explaining the meaning of local optimization in the EPFA-CLS and how it is reflected in the algorithm. The CLS can be derived through the optimization process. First, you need to select one of the optimizers introduced in the Background section.

The EPFA-CLS utilizes local optimization algorithms such as Gradient Descent and Adagrad by using the first derivative. Then, the current situation is reflected by inputting the initial value of $\gamma$ into $\mathbf{C}$. Therefore, the initial values of $\mathbf{E}$ and $\mathbf{C}$ are reflected in the EPFA-CLS. Another thing to consider after setting the optimizer to Gradient Descent is to use Gradient Descent if the parameter to be derived in the problem situation is the minimum value, and to use Gradient Ascent if the parameter is the maximum value. The expressions in (21) input the initial value of the control parameter. After setting the basics, you can update the input parameters through Gradient Ascent or Gradient Descent.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\mathbf{X}}_{initial}& =[{\mathbf{E}}_{initial},{\mathbf{C}}_{initial}]\hfill \\ \hfill {\mathbf{C}}_{initial}& =[{\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_m]\hfill \\ \hfill {\mathbf{E}}_{initial}& =[{\epsilon }_1,{\epsilon }_2,\cdots ,{\epsilon }_n]\hfill \end{array}\end{array}$$

(21)

The expressions in Equation (22) have the same effect as those in (23). The formula that combines the EPFA-CLS with the optimizer is shown. After finding the slope from the initial value, it is multiplied by $ϵ$ to update the control parameter. As the iteration progresses, the output of the corresponding control parameter gradually reaches the local minimum/maximum value. This optimizer uses the first derivative and introduces a method of finding the first derivative of the trained DNN.

$$\begin{array}{c}\hfill {\mathbf{X}}_{new}={\mathbf{X}}_{old}+ϵ\frac{\partial \hat{y}}{\partial \mathbf{X}}\end{array}$$

(22)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill [{\mathbf{C}}_{\mathit{new}},{\mathbf{E}}_{\mathit{new}}]& =[{\mathbf{C}}_{\mathit{old}},{\mathbf{E}}_{\mathit{old}}]+ϵ[\frac{\partial \hat{y}}{\partial \mathbf{C}},\frac{\partial \hat{y}}{\partial \mathbf{E}}]\hfill \\ \hfill {\mathbf{E}}_{\mathit{new}}& ={\mathbf{E}}_{\mathit{initial}}\hfill \\ \hfill {\mathbf{C}}_{\mathit{final}}& ={\mathbf{C}}_{\mathit{new}}\hfill \end{array}\end{array}$$

(23)

The chain rule is used to find the rate of change of another parameter for a specific parameter [40]. We need to find the rate of change of the output with respect to input x of the DNN. This uses partial differentiation, and by using the gradient with each hidden layer in the DNN, $\frac{\partial output}{\partial x}$ can be obtained with the chain rule. As the epoch increases, the $\mathbf{C}$ and $\mathbf{E}$ values are updated, but since the $\mathbf{E}$ value is always fixed as the initial value, ${\mathbf{C}}_{final}$ can be derived.

$$\begin{array}{c}\hfill \left[\frac{\partial \hat{y}}{\partial \mathbf{X}}\right]=\left[\frac{\partial \hat{y}}{\partial c_1},\cdots ,\frac{\partial \hat{y}}{\partial c_m},\frac{\partial \hat{y}}{\partial e_1},\cdots ,\frac{\partial \hat{y}}{\partial e_n}\right]\end{array}$$

(24)

$$\begin{array}{c}\hfill \frac{\partial \hat{y}}{\partial c_m}=\sum _{k=1}^{\sigma }\sum _{i=1}^{\rho }\frac{\partial A_k}{\partial c_m}\frac{\partial B_i}{\partial A_k}\frac{\partial \hat{y}}{\partial B_i}\end{array}$$

(25)

$$\begin{array}{c}\hfill \frac{\partial \hat{y}}{\partial e_n}=\sum _{k=1}^{\sigma }\sum _{i=1}^{\rho }\frac{\partial A_k}{\partial e_n}\frac{\partial B_i}{\partial A_k}\frac{\partial \hat{y}}{\partial B_i}\end{array}$$

(26)

3.5. Controllable Local Optimal Solution

After fixing $\mathbf{E}$, the proposed algorithm to derive the optimal $\mathbf{C}$ is introduced. In this section, the formula for the final result is explained. As $\mathbf{C}$ is updated through optimization, each $\gamma$ corresponding to $\mathbf{C}$ also reaches a final value.

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathbf{CLS}={\mathbf{C}}_{final}=[{\gamma }_1^{final},{\gamma }_2^{final},\cdots ,{\gamma }_m^{final}]\end{array}\end{array}$$

(27)

In optimization, the CLS can be derived through the updated $\gamma$ value. The equations in (27) are a representative formula for a parameter where the output is a maximum. For a problem optimized through Gradient Descent, min is used instead of max.

4. Experiments

In this section, the datasets and working environment for the experiment are described, and the three experiments (C, D and E) we conducted are discussed. First, the experiment to verify the feasibility of the EPFA-CLS described in Experiment C is shown. Second, the reason for the fixed environment parameter as described in Experiment D is shown as an experiment comparing data with non-fixed optimization. Finally, an experiment using the Boston Housing Dataset shows that the EPFA-CLS can derive the CLS from other datasets.

The Boston Housing Dataset is one with 13 independent variables and one dependent variable, which satisfies the multiple regression condition. In addition, since parameters $\mathbf{C}$ and $\mathbf{E}$ can be clearly divided, they were adopted as experimental data.

While performing these three experiments, comparisons of several optimizers were conducted at the same time. During DNN training, the experimental environment used a GTX 1080 Ti 11GB 4way GPU. In addition, when optimizing the trained DNN, CPU operations used a 16-core AMD Ryzen Threadripper 1950X.

Experiments were conducted using the optimal course dataset in experiments C, D, and E. In Experiment E, the Boston Housing Dataset was used in addition to the optimal course dataset. Introducing the experimental conditions, the initial values of $\mathbf{E}$ and $\mathbf{C}$ are assumed as follows:

• Scenario 1: SD = 150 deg, SV = 10 m/s, WD = 315 deg, WV = 10 m/s
• Scenario 2: SD = 150 deg, SV = 10 m/s, WD = 90 deg, WV = 5 m/s

In addition, the optimizer with the first derivative was used for local optimization. In this experiment, four optimizers were used: Gradient Ascent, Adagrad, Momentum, and RmsProp [37].

4.1. The Prepared Dataset

The optimal course model dataset was constructed for this study by using a simulator based on an infrared synthetic image generation study [41]. First, it is necessary to accurately model the influence of wind when deploying a deception device such as a flare. The wind parameters indicate the strength of the wind in meters per second and its direction in degrees, as shown in Figure 6. For wind direction, in particular, blowing along the X-axis is 0 degrees.

When modeling the wind vector in this way, the velocity of the flare is expressed as a relative velocity by subtracting the wind vector from the original velocity vector of the flare, as shown in Equation (28) [42]:

$$\begin{array}{c}\hfill {\overrightarrow{V}}_r\left(t\right)=\overrightarrow{V}\left(t\right)-\overrightarrow{W}\end{array}$$

(28)

Based on this, the influence of the wind is reflected in the infrared composite image generation simulator. Then, since the infrared guided missile follows the locked-on object in the simulator, a process for linking the tracker to the simulator is required [43]. After creating every frame in the simulator, the binary centroid tracker was immediately driven so that the missile followed the locked-on target, and the tracked target came to the center of the image. For this purpose, three items have been added: (1) missile dynamics modeling and reflection, (2) aim-point update through tracking, and (3) measuring the miss distance in 3D space through the tracking result [41].

Figure 7 shows a flow chart for generating the integrated composite image and processing the tracker model produced through this process. Synthesis tracking is performed every frame based on user-set parameters; the tracking process is shown on a 2D plot, and miss distance is provided. So far, wind modeling and tracker linkage have been described in the simulator. Based on this, the miss distance can be derived according to the input variables for ship direction, ship speed, and wind direction. The direction of the ship is a polar plot, and the speed of the ship is displayed in different colors. The closer the graph coordinates are to the center, the smaller the miss distance and the lower the survival probability of the ship; conversely, the farther from the center, the greater the miss distance, and the higher the survival probability.

Table 3. This is an example of the data acquired through the simulator, and it consists of a total of 2407 training data points.

SD [deg]	SV [m/s]	WD [deg]	WV [m/s]	MD [m]
0	0	0	0	64.89487486
0	0	0	3	19.02400837
0	0	0	5	24.66201077
15	0	0	5	14.66237099
30	0	0	5	14.75703005
45	0	0	5	15.70170131
60	0	0	5	30.37929284
75	0	0	5	28.28568179
90	0	0	5	30.14618032
105	0	0	5	29.61211902
120	0	0	5	27.89175802
135	0	0	5	23.66285094
150	0	0	5	21.35341014
165	0	0	5	24.42776638
180	0	0	5	14.68399028
195	0	0	5	14.37340086
210	0	0	5	14.52094585
225	0	0	5	15.05996721
240	0	0	5	18.23890991

shows the results created by using a CSV file for deep learning. In particular, in this result, the miss distance according to the direction of the ship was measured. The number of input parameters $\left(\phi \right)$ was 4, and ground truth y was 1. Each parameter has a range: WD and SD range from 0 deg to 360 deg, while WV and SV range from 0 m/s to 15 m/s.

After building the dataset, we need to define $\mathbf{E}$ and $\mathbf{C}$. In the optimal course dataset, $\mathbf{E}$ is defined as WD and WV, and $\mathbf{C}$ is defined as SD and SV; $x_1$ to $x_2$ are defined as $\mathbf{E}$, and $x_3$ to $x_4$ are defined as $\mathbf{C}$.

4.2. Training

In the previous section, 2407 training data points were prepared with CSV files through the simulator, with 85% used for training and 15% for testing. We used the model architecture in Figure 5. Parameters during training with the dataset included $n=2$, $m=2$, $\sigma =512$, $\rho =512$, with the ReLU as an activation function, and SGD as an optimizer. The training was conducted across 1000 epochs.

Figure 8 shows a training set graph after training the DNN with the optimal course dataset. The result for $R^2$ was 0.978, and $R^2$ from the test set was 0.849. A value closer to 1 means the input has a stronger relationship with the output. This means the DNN has high prediction accuracy and can be used as a mapping function.

4.3. The Robustness of the Trained DNN

Local optimization derives the CLS using the EPFA-CLS under the experimental conditions described above. In this experiment, we compared the predicted result (MD) of the trained DNN with the output (MD) of the real simulator in the derived CLS.

Table 4. Derived CLS results in Scenario 1: among the four results, the mode value is input to the simulator. The predicted MD and the MD of the simulator are compared with time and output values. Ground Truth (GT).

Type	Parameters	EPFA-CLS				Simulator (GT)
		Adagrad	GA	Momentum	RMSProp
CLS	SD [deg]	182.67	182.57	182.67	182.68	182.67
	SV [m/s]	15	15	15	15	15
Output	MD [m]	268	267.98	268	267.9	271.3
Time [s]		8.69	9.47	10.106	10.42	19 [min]

and

Table 5. Derived CLS results in Scenario 2: the description of the table is the same as in Table 4.

Type	Parameters	EPFA-CLS				Simulator (GT)
		Adagrad	GA	Momentum	RMSProp
CLS	SD [deg]	169.7	169.9	174.9	169.7	169.7
	SV [m/s]	15	15	15	15	15
Output	MD [m]	379.9	379.9	380.5	379.98	381.4
Time [Sec]		9.58	8.81	9.96	11.5	19 [min]

show the results for scenarios 1 and 2, respectively. Local optimization was performed with the initial value ($E_{initial}$), and the result derived. Four local optimizers were used; the input to the simulator was the mode of the derived CLS. Using the results of the simulator as ground truth, we proved the robustness of the trained DNN. Table 4 shows the CLS derived using the local optimizer. Among the four optimizers, the most frequently derived CLS was from 182.67 deg for SD and 15 m/s for SV. So, each value was the simulator’s input. Naturally, the environment parameter was the same value as in scenario 1. The predicted MD through the trained DNN was 268 m. In addition, MD (the output of the simulator) was 271.3 m. We can see that the value of $R^2$ is predicting the correct value, as demonstrated by 0.97. Table 5 shows the most frequently obtained results from the CLS were SD at 169.7 deg and SV at 15 m/s. So, each value was the simulator’s input. The predicted MD through the trained DNN was 379.9 m, and the MD through simulator was 381.4 m. The trained DNN predicted exact values.

The results from the simulator were more accurate than the values predicted by the trained DNN. However, these results take quite a long time, as shown in Table 4 and Table 5. Additionally, it is not possible to find SD and SV in large MD values in one simulation. This is because the simulator outputs only the MD. In this experiment, we measured the accuracy and the reason for requiring the trained DNN objective optimization.

The following is a comparison between local optimizers in Table 4 and Table 5. In Table 4, the CLS is the same, and the time is also similar. Table 5 gives a similar CLS and time, but since the concept of the velocity of the gradient is reflected in Momentum, we can see that the result is a little bit more.

4.4. Fixed Environment Parameter

In this experiment, the CLS is induced by using the EPFA-CLS, and the effect of parameter fixation on the CLS is understood through the experiment with the conditions described above. This experiment shows the results of the CLS depending on whether or not it is fixed in the algorithm, which is the third step of the EPFA-CLS. Experimental conditions were all the same except for being fixed or not. Not fixing the CLS is all about finding the optimal environment and both control parameters. In some cases, this can also be a good result, but we are solving the constraint optimization problem in a specific environment.

Table 6. With scenario 1, results of the EPFA-CLS and optimization without fixing.

	Type	Parameters	Adagrad	GA	Momentum	RmsProp
EPFA-CLS	CLS	SD [deg]	182.67	182.57	182.67	182.68
		SV [m/s]	15	15	15	15
		WD [deg]	315	315	315	315
		WV [m/s]	10	10	10	10
	Output	MD [m]	268.01	268.2	268.01	267.9
Unfixed	CLS	SD [deg]	168,17	168.15	194.69	168.13
		SV [m/s]	15	15	15	15
		WD [deg]	282.8	282.78	215.02	282.77
		WV [m/s]	15	15	15	15
	Output	MD [m]	461	461.23	589.5	461.23

and

Table 7. With scenario 2, results of EPFA-CLS and results of optimization without fixing.

	Type	Parameters	Adagrad	GA	Momentum	RmsProp
EPFA-CLS	CLS	SD [deg]	169.7	169.9	174.9	169.7
		SV [m/s]	15	15	15	15
		WD [deg]	90	90	90	90
		WV [m/s]	5	5	5	5
	Output	MD [m]	379.9	379.9	380.5	379.98
Unfixed	CLS	SD [deg]	167.76	194.7	194.69	194.78
		SV [m/s]	15	15	15	15
		WD [deg]	90.56	215.02	215.02	215.01
		WV [m/s]	2.9	15	15	15
	Output	MD [m]	398.02	461.23	589.5	589.5

show the results for scenarios 1 and 2, respectively. Experiment D compared the results of the EPFA-CLS and an unfixed Equation (5) with the optimal course dataset. In the EPFA-CLS, $\mathbf{E}$ was fixed for ${\epsilon }_1$ and ${\epsilon }_2$, and optimal $\mathbf{C}$ was derived using the local optimizer. In Equation (5), the optimal solution for all parameters of $\mathbf{E}$ and $\mathbf{C}$ is derived without being fixed.

First, Table 6 shows the results for scenario 1. From using the EPFA-CLS, the CLS is 182 deg for SD and 15 m/s for SV. Since the environment parameter was fixed here, WD and WV have the same values as the environment parameter in scenario 1. In the unfixed result excluding Momentum, SD is 168.17 deg, SV is 15 m/s, WD is 282.8 deg, and WV is 15 m/s. From Momentum, SD is 194.17 deg, SV is 15 m/s, WD is 215.8 deg, and WV is 15 m/s. Comparing miss distance in the results for scenario 1, the EPFA-CLS was 268.01 m, but when unfixed was 461 m and 589 m. The unfixed miss distances came out higher. However, what is important to consider here is the value of the environment parameter, rather than the miss distance.

In Table 6, the current environment parameter values are ${\epsilon }_1$ = 315 deg and ${\epsilon }_2$ = 10 m/s. The control parameter can be changed to the CLS to increase the miss distance, but the environment parameter cannot be changed for the ship. Therefore, even if the unfixed miss distance is higher than the EPFA-CLS, it is not a CLS because the environment parameter cannot be modified. In addition, since the miss distance is simultaneously affected by the environment parameter and the control parameter, it is necessary to derive the optimal control parameter from the current environment parameter value. Here, it was derived using the local optimizer. Therefore, it is not the CLS because environment parameters cannot be changed based on results derived from unfixed experiments. Therefore, we derived the CLS by fixing the environment parameter.

Table 7 shows the results for the second scenario. In these results, the same conclusion is reached as in scenario 1. However, the unique feature here is that the result of Momentum optimizer derived from unfixed values in scenario 1 and the result of Momentum optimizer derived from unfixed values in scenario 2 are the same. If you unfix and optimize based on this, you eventually find the same point. In addition, finding the same point without fixing it means finding the local maximum in the five-dimensional graph (four inputs, one output) of the DNN’s f as trained with the optimal course dataset. It can be experimentally confirmed that this is the same characteristic obtained by using differentiation in the optimization algorithm, which can derive the optimal solution without high-dimensional plots.

In the end, it can be confirmed that the EPFA-CLS, which was unfixed and optimized in Experiment D, is the same as the result derived from Equation (5) for optimization. The EPFA-CLS is an algorithm that optimizes by dividing solutions in Equation (5). It optimizes by fixing a specific environment parameter.

Experiment D also confirmed the results for several local optimizers. Compared with other optimizers, Momentum can confirm that the effect of speed is reflected in the EPFA-CLS. Therefore, if you want to maximize the output value, it is better to use the Momentum optimizer. In general, similar results can be confirmed, but in Experiment D, there is a large difference in values. Based on this point, a selection of the EPFA-CLS’s optimizer is the user’s (similar to a hyperparameter).

4.5. Additional Validation of the CLS

In this experiment, we show whether the EPFA-CLS can derive the CLS through two datasets. The Boston Housing Dataset consists of 13 input parameters [44] such as crime rate, number of rooms, and the local tax rate, and the output parameter is the house price in the suburbs of Boston in the mid-1970s. The other dataset is the optimal course dataset, which was prepared for this paper. The CLS is the local maximum reachable from the current location. Therefore, this experiment aims to validate the robustness of the DNN trained on the dataset, ensuring its ability to produce the CLS regardless of its position. With $n=2$, $m=2$, $\sigma =1024$, $\rho =1024$, the ReLU activation function, and the SGD optimizer, the training procedure before the experiment used 90% of the Boston Housing Dataset while testing used 10%. With the training set, $R^2$ showed accuracy of 0.971, and in testing, $R^2$ showed accuracy of 0.849. In Experiment E, the optimizer used Gradient Ascent to show how robustly the CLS was derived from any location by using all the EPFA-CLS conditions as is, changing only the dataset.

(1) Boston Housing Dataset

This Boston house price dataset includes the following independent variables: crime rate per capita (CRIM), the proportion of residential area exceeding 25,000 square bits (ZN), the proportion of land occupied by non-retail commercial areas (INDUS), a dummy variable for the Charles River (CHAS), concentrated monoxide per 10 ppm nitrogen (NOX), the average number of rooms per household (RM), the proportion of owned homes built before 1940 (AGE), an index of accessibility to five Boston career centers (DIS), an index of accessibility to radial roads (RAD), the property tax rate per $\$10,000$ in assessed value (TAX), the student/teacher ratio by town (PTRATIO), $1000{(Bk-0.63)}^2$ where $B_k$ is the percentage of municipality (B), and a substratum of the population ratio (LSTAT).

This experiment focuses on examining how robustly a DNN trained on the Boston dataset can produce the CLS regardless of its position. The experimental procedure is as follows: firstly, the Boston dataset consists of 506 data points with a total of 13 independent variables. In this scenario, four random independent variables are defined as control parameters, and the entire dataset of 506 instances is utilized as initial values. Using EPFA-CLS, the optimization is performed from the initial values to obtain the CLS, resulting in a total of 4 ∗ 506 data points. The key points of interest are the distribution of control parameters at the initial values and after obtaining the CLS, as well as the distribution of housing prices at the initial control parameters and after obtaining the CLS.

We introduced the control parameters: nitric oxide concentration, average number of rooms per dwelling, weighted distances to five Boston employment centers, and full-value property tax rate per $\$10,000$. The results are intriguing. Upon examining the distributions in the left histograms and the distribution of housing prices on the right in Figure 9, it can be observed that the majority of values have moved towards optimal values compared to the black region representing the initial values. For instance, it is evident that a higher average number of rooms per dwelling leads to a higher housing price, given the same environmental conditions. Additionally, when analyzing the distribution of CLS, it can be seen that the values have reached higher levels compared to the initial values, resulting in an increase in housing prices. However, from a global optimization point of view, some outliers were also observed. This phenomenon occurred due to the optimization based on gradient descent. Since gradient descent seeks local optima, it considers closer points as more optimal. In conclusion, it has been demonstrated that the DNN can produce Controllable Local Optimal Solutions (CLS) from any position, emphasizing its ability to find CLS regardless of the starting point.

(2) Optimal course dataset

Table 8. Table illustrates the results of CLS derivation for 12 random initial values using EPFA-CLS.

Initial Value (Random)						Final Value (CLS)
No.	SD [deg]	SV [m/s]	WD [deg]	WV [m/s]	${\mathit{MD}}_{\mathit{initial}}$[m]	${\mathit{CLS}}_{\mathit{SD}}$[deg]	${\mathit{CLS}}_{\mathit{SV}}$[m/s]	${\mathit{MD}}_{\mathit{maximum}}$[m]
1	101	8	15	2	79.3849	180.9525	15	387.5419
2	117	11	322	6	57.7895	188.0334	15	279.0536
3	142	14	242	5	308.4076	174.3665	15	381.5254
4	171	4	237	1	187.3034	149.0326	12.1728	416.7096
5	209	12	356	1	250.1607	172.6729	15	404.1164
6	220	4	281	7	109.1765	195.6159	15	262.6963
7	232	7	231	7	203.9565	196.0379	15	319.9787
8	238	6	161	3	120.9584	148.9174	15	424.4204
9	252	9	148	3	141.9169	148.0669	15	411.2176
10	254	7	223	15	332.7779	196.8761	15	586.8834
11	263	1	232	7	107.3001	195.9918	15	319.0679
12	311	7	256	6	70.8144	189.7698	15	335.1269

and Figure 10 show the results when deriving the CLS from 12 random initial values that the trained DNN used in Section 3 and Section 4. It illustrates the scenario where instead of the one mentioned at the beginning of Section 4, the initial values are defined at twelve randomly chosen points, and the resulting CLS is derived.

We will describe the subgraphs from a to d in Figure 10. Starting from the initial red dot, it can be observed that EPFA-CLS reaches local optima. Graphs A, B, C, and D represent the graphs and CLS for scenarios 1, 2, 6, and 8, respectively, as shown in Table 8. Particularly in Graph B, there are three local maximum points, but optimization is achieved by considering the gradient direction. Additionally, in Graph D, the initial values are updated through optimization, leading to the derivation of CLS. This demonstrates that the trained neural network can be optimized using the Gradient Descent method, allowing it to reach local optima.

If you check the output variable for miss distance, you can see that MD of the CLS increases. Compared to the initial MD especially, when the initial value for SD was 101 deg, we can see that MD is much higher, even if it moves only 20 deg. In Figure 10, each heading displays its initial value. The size of the graph indicates the shape of the graph in which the initial value of the environment parameter is reflected. By conducting random tests for various scenarios, EPFA-CLS demonstrates its ability to be derived from initial values.

5. Summary and Conclusions

In this paper, in the classification of optimization problems according to existing objective functions such as LP and NLP, we present the constraint optimization problem in a DNN where the objective function is trained. In addition, we proposed the EPFA-CLS to solve this problem. Because multiple environments are reflected, multiple environment parameters are entered as input to the objective function. Therefore, we defined the input parameters as independent $\mathbf{E}$ and $\mathbf{C}$, and derived the CLS through local optimization at a specific $\mathbf{E}$. In conclusion, optimization was used two times in total. The first was training a deep neural regression network, defining the objective function as a loss function and finding the optimal weight. The second was to train the DNN defined as an objective function with the CLS derived according to the purpose desired by the user. The approach through fixing environment parameters transforms constrained optimization into unconstrained optimization. The algorithm was verified with the optimal course dataset and a Boston house price dataset. Through these experiments, an optimization approach was demonstrated by modeling non-ideal situations to solve various engineering problems.