Workshop Facility Layout Optimization Based on Deep Reinforcement Learning

Abstract

With the rapid development of intelligent manufacturing, the application of virtual reality technology to the optimization of workshop facility layout has become one of the development trends in the manufacturing industry. Virtual reality technology has put forward engineering requirements for real-time solutions to the Workshop Facility Layout Optimization Problem (WFLOP). However, few scholars have researched such solutions. Deep reinforcement learning (DRL) is effective in solving combinatorial optimization problems in real time. The WFLOP is also a combinatorial optimization problem, making it possible for DRL to solve the WFLOP in real time. Therefore, this paper proposes the application of DRL to solve the dual-objective WFLOP. First, this paper constructs a dual-objective WFLOP mathematical model and proposes a novel dual-objective DRL framework. Then, the DRL framework decomposes the WFLOP dual-objective problem into multiple sub-problems and then models each sub-problem. In order to reduce computational workload, a neighborhood parameter transfer strategy is adopted. A chain rule is constructed for the appealed sub-problem, and an improved pointer network is used to solve the bi-objective WFLOP of the sub-problem. Finally, the effectiveness of this method is verified by using the facility layout of a chip production workshop as a case study.

1. Introduction

The Facility Layout Optimization Problem (WFLOP) is one of the important elements in the field of workshop optimization design. The International Association for Manufacturing Technology shows that excellent workshop facility layout can reduce the total production cost by 10% to 30% [1]. As the manufacturing industry continues to develop and expand, the layout of workshop facilities has shown some problems, such as wasteful logistics and handling, unreasonable location, and low production efficiency. Because early workshop design considered mainly the realization of production functions and did not carry out a refined design of workshop facility layout, there are a number of unreasonable workshop facility layout solutions, which are most common in developing countries. In this situation, optimizing the layout of workshop facilities has become an important research topic in the field of workshop optimization design, first studied by Flavio D. P. (1996) [2]. Subsequently, a large number of scholars have successively carried out research on traditional facility layout methods such as the shape of facility layout, mathematical models of facility layout, and solution algorithms, and have produced rich research results [3,4]. With the continuous advancement of Industry 4.0 and the breakthroughs in emerging technologies such as sensors, digital twins, and virtual reality, scholars have used emerging technologies for facility layout optimization. Andreas K (2019) [5] studied technologies such as infrared scanners, photogrammetry, and LiDAR to build virtual reality models of workshop facilities. Mona S (2023) [6] proposed realizing a visual facility layout plan in urban construction through a 3D BIM platform. Seungnam Y (2022) [7] studied the DES-VR platform for nuclear facilities to implement thermal battery equipment design methods and verified the practicality of the resulting process equipment layout and floor plan. Virtual reality technology has been widely used in workshop facility layout optimization, as shown in Figure 1.

The application of virtual reality technology to the optimization of workshop facility layout is a development trend in the manufacturing industry, and many companies have already applied it. The engineering application of virtual reality technology puts forward real-time solution requirements for the solution method of facility layout. There are many algorithms for solving WFLOP, and scholars such as Zhang Z Q (2019) [11], Goli A (2021) [12], Kalyanmoy Deb (2020) [13], and Julia R (2022) [14] have successively applied algorithms such as MOEA, NSGA, and PSO. The above intelligent algorithms have been improved, updated, and mixed, and the results are better at solving the WFLOP. However, although these algorithms have better results in solving the WFLOP, they take longer to find solutions and cannot achieve real-time solution results. Oriol Vinyals et al. (2017) [15] of Google Brain first proposed machine learning to solve combinatorial optimization problems and used single-objective TSP to verify the effectiveness of the method, and the trained model achieved real-time results. Kaiwen Li et al. (2020) [16] proposed reinforcement learning to solve the dual-objective combinatorial optimization problem and used TSP to verify the efficiency and accuracy of the algorithm. Since then, results in machine learning for solving combinatorial optimization problems have continued to emerge.

According to the research results of many scholars, there are few studies on machine learning solution methods for the WFLOP. However, the latest facility layout methods put forward real-time solution requirements for solution speed. In order to meet the needs of engineering applications, it is urgent and necessary to explore methods for real-time solutions to facility layout problems.

In summary, research on the WFLOP is extremely important to workshop production and manufacturing. The integration of virtual reality technology into the layout of workshop facilities is a current research hotspot. At the same time, traditional intelligent algorithms cannot meet the requirements of virtual reality for real-time solutions to the WFLOP. However, many scholars have used machine learning to achieve real-time solutions to combinatorial optimization problems such as job shop scheduling, TSP, and machine slot allocation, which has provided a method for DRL to solve the WFLOP in real time. Therefore, this paper examines DRL as a solution to the dual-objective workshop facility layout optimization problem in real time. It also establishes a mathematical model for the optimization problem of workshop facility layout.

When using DRL to solve multi-dimensional targets, the facility layout problem is first converted into a pointer network problem, and then a decomposition strategy and a neighborhood parameter transfer strategy need to be used. The decomposition strategy decomposes the problem into a set of sub-problems and solves them in a collaborative manner. The neighborhood parameter transfer strategy builds a neural network for sub-problems, and the network parameters are transferred from the previous sub-problem to the next sub-problem in order. Finally, a chip production workshop with numerous automated equipment is used as an application case to verify the effectiveness of the above method. The main contributions of this article include the following:

(1)

The DRL is proposed to solve the facility layout optimization problem.

(2)

The engineering application requirements of virtual reality facility layout for a real-time solution of dual-objective workshop facility layout are realized.

(3)

A dual-objective mathematical model for facility layout optimization is constructed.

(4)

High-quality cases can provide a reference for the layout of other similar types of workshop facilities.

The remainder of this paper is organized as follows: Section 1 presents the literature review. Section 2 presents mathematical model building. Section 3 proposes the design of the DRL solution process. Section 4 presents a case application. Finally, Section 5 presents conclusions.

2. Mathematical Model

Before constructing the mathematical model, it is necessary to divide the functional areas of the workshop and establish a direct coordinate system, as shown in Figure 2.

Some variables of the mathematical model are marked in Figure 2, and their meanings are as follows:

The facility number is represented by i. The length and width of the facility are represented by (l_i, w_i). The coordinate system of the facility is represented by (x_i, y_i). (L_x, W_y) represent the length and width of the workshop. A₁ represents the main channel width limit. A₂ represents this channel width limit. (L, W) indicate the workshop length and width restrictions. A 0–1 variable is represented by f_ij. The quantity of product q is represented by n_q. D represents the carrying distance. S represents the workshop area. The horizontal arrangement of equipment is represented by r and e. J represents the minimum spacing between devices.

The objective function of minimizing the carrying distance is shown in Formula (1).

$$\min D={\displaystyle \sum _q{\displaystyle \sum _i{\displaystyle \sum _j{n}_q\left(\left|x_i-x_j\right|+\left|y_i-y_j\right|\right)}}}{f}_{ij}$$

(1)

The objective function with the smallest area is shown in Equation (2). The facility can change direction by 90°, with l and w replacing each other.

$$\left\{\begin{array}{l}\min S=L_x*W_y\\ L_x=\max \left\{x_i+l_i/2+A_2\right\}\\ W_y=\max \left\{y_i+w_i/2+A_1\right\}\end{array}\right.$$

(2)

The constraints are shown in Equations (3)–(7).

$$x_i^r-x_j^r\ge J+\frac{l_i+l_j}{2}\begin{array}{cc}& \forall i,j,r\end{array} and i\ne j$$

(3)

$$y_i^r-y_j^e\ge A_2+\frac{w_i+w_j}{2}\begin{array}{cc}& \forall i,j,r\end{array},e and r\ne e$$

(4)

$$\frac{l_i}{2}+A_1\le x_i^{}\le L-\frac{l_i}{2}-A_1$$

(5)

$$\frac{w_i}{2}+A_1\le y_i^{}\le W-\frac{w_i}{2}-A_1$$

(6)

$$y_i^r=y_j^r\begin{array}{cc}& \forall r\end{array}$$

(7)

The facility numbers are represented by i and j. Different horizontal rows of the facility layout are represented by r and e. The length and width of the facility are represented by (l, w). (L, W) represents the length and width of the workshop. The coordinates are represented by (x, y). Facility non-interference constraints include mainly non-interference between facilities and non-interference between equipment and surrounding facilities. Equation (3) means that the spacing between devices in the same row does not interfere. Equation (4) indicates that the spacing between different rows of equipment does not interfere. Equation (5) indicates that there must be secondary passages on the left and right sides of the workshop. Equation (6) indicates that this passage up and down the workshop is necessary. Equation (7) indicates that the centers of equipment in the same row are on the same center line.

3. DRL Solves WFLOP

3.1. Decomposition Strategy

The decomposition strategy is widely used in solving multi-objective optimization problems and, as it is very effective, this strategy is also used for the bi-objective optimization problem of facility layout in this article. The decomposition strategy entails decomposing the facility layout optimization problem into multiple sub-problems and then solving each sub-problem. When all the sub-problems have obtained solutions, then a set of ideal solutions is obtained, and the optimal solution is included in this set of solutions. The dual objectives of facility layout are processed using the well-known weighting method, where the weight vector is ${\mathrm{\delta }}^j=\left[{\mathrm{\delta }}^1,{\mathrm{\delta }}^2,\cdots ,{\mathrm{\delta }}^N\right]$, and ${\mathrm{\delta }}_i^j$ represents the multi-objective weight coefficient in the j-th sub-problem. For example, if ${\mathrm{\delta }}_i^j$ is $\left({\mathrm{\delta }}_1^j,{\mathrm{\delta }}_2^j\right)$, then the bi-objective weighting of the j-th sub-problem is $\left({\mathrm{\delta }}_1^j*\mathrm{distance} \mathrm{targe},{\mathrm{\delta }}_2^j*\mathrm{area} \mathrm{arget}\right)$. ${\mathrm{\delta }}^j$ is a uniformly distributed sub-problem. The coordinate system composed of the area and distance of the dual-objective FLOP as shown in Figure 3 can be composed of N sub-problems of {P₁, P₂, P₃, …, P_N}.

3.2. Parameter Transfer Strategy

DRL solves the facility layout problem by establishing sub-problems as neural networks, but there are many sub-problems, and training for each of them results in a vast amount of calculation and difficulty in implementation. Therefore, parameter transfer design is required to make it possible to use deep reinforcement learning to solve the WFLOP. As shown in Figure 3, the weight vectors of uniformly distributed sub-problems are adjacent, and adjacent sub-problems have close optimal solutions. Therefore, the neural network parameters of adjacent sub-problems can be used as the parameter starting point for the next sub-problem, and then the corresponding parameters of the neural network for the neighbor problem are iterated from generation to generation. For example, $\left[m_{{\mathrm{\delta }}^j},n_{{\mathrm{\delta }}^j}\right]$ represents the neural network parameters of the j-th sub-problem. $\left[m_{{\mathrm{\delta }}^j}^{*},n_{{\mathrm{\delta }}^j}^{*}\right]$ indicates that the network parameters of the optimal solution to the sub-problem have been obtained through training. $\left[m_{{\mathrm{\delta }}^{j+1}}^{},n_{{\mathrm{\delta }}^{j+1}}^{}\right]$ uses $\left[m_{{\mathrm{\delta }}^j}^{*},n_{{\mathrm{\delta }}^j}^{*}\right]$ as the initial parameter of the j + 1th sub-problem for training to obtain the optimal parameter $\left[m_{{\mathrm{\delta }}^{j+1}}^{*},n_{{\mathrm{\delta }}^{j+1}}^{*}\right]$, and so on, until J = N ends the training. The algorithm pseudocode is shown below.

Step 1: Randomly generate $\left[m_{{\mathrm{\delta }}^1}^{*},n_{{\mathrm{\delta }}^1}^{*}\right]$, calculate the optimal solution F_1, and set $\left[m_{}^{*},n_{}^{*}\right]$ to max.

Step 2: If N ≠ 1

    for j = 1:N,

       use $\left[m_{{\mathrm{\delta }}^j}^{*},n_{{\mathrm{\delta }}^j}^{*}\right]$ as the initial parameter input of $\left[m_{{\mathrm{\delta }}^{j+1}}^{},n_{{\mathrm{\delta }}^{j+1}}^{}\right]$ for training, and calculate F_j+1.

       If F_j+1 < $\left[m_{}^{*},n_{}^{*}\right]$

         $\left[m_{}^{*},n_{}^{*}\right]$ = $\left[m_{{\mathrm{\delta }}^{j+1}}^{*},n_{{\mathrm{\delta }}^{j+1}}^{*}\right]$

   else $\left[m_{}^{*},n_{}^{*}\right]$ = $\left[m_{{\mathrm{\delta }}^1}^{*},n_{{\mathrm{\delta }}^1}^{*}\right]$

   return $\left[m_{}^{*},n_{}^{*}\right]$.

Step 3: Calculate the optimal solution through $\left[m_{}^{*},n_{}^{*}\right]$.

The above-mentioned decomposition strategy is used, and adjacent parameters are transferred, which greatly reduces the amount of calculation, and simplifies and modularizes the neural network. The above method realizes the problem decomposition of the facility layout problem and allows the sub-problems to be cyclically trained. However, the solution method of the sub-problems has not yet been demonstrated. Next, the modeling and analysis of the problem will be carried out.

3.3. DRL Solves WFLOP Subproblem

DRL solves combinatorial optimization problems starting from the pointer network in the literature [15]. The facility layout optimization problem is also a combinatorial optimization problem, so this paper uses the pointer network in [15,16] to solve the WFLOP sub-problem.

In the WFLOP shown in Figure 2, the model input set X is {E_i, i = 1, 2, 3, …, n}, where n is the number of devices, which is 6 in Figure 2. E_i is composed of a set of vectors, representing the attributes of the objective function, such as the coordinate system (x_i, y_i) in the dual-objective facility layout problem to calculate the distance objective function, and the length and width (l_i, w_i) to calculate the area objective function. The input structure is shown in Figure 4.

The output of the model is the permutation and combination of facilities; for example, the output Y is {ρ₁, ρ₂, ρ₃, …, n}. To map input X to output Y, use the probabilistic chain rule:

$$P(Y/X)={\Pi }_{t=1}^{n}P({\rho }_{t+1}\left|{\rho }_1,\dots ,{\rho }_{t+1},X_t\right.)$$

(8)

First, any facility ρ₁ is placed at the entrance of the workshop, and the next facility among the remaining X_t facilities is selected and placed in the second position at the entrance of the workshop. X_t will update the optional facilities as the facilities are selected until all equipment is selected, at which time all corresponding equipment has been placed in the workshop. Fan [17] and Yuan [18] map the sequence of facilities to a scalar, which is the workshop logistics handling volume or workshop area. It can be seen from Section 3.1 of this article that this article weights the logistics handling volume and the workshop area. The reward of a WFLOP solution is a weighted negative number. When the solution is optimal, the weighted value is the smallest, the optimal logistics handling volume and area are obtained, and the reward is the largest. The pointer network models Formula (8). For details, see documents [15,16]. The basic structure is the sequence-to-sequence model, a powerful model recently proposed in the field of machine translation that maps one sequence to another. A general sequence-to-sequence model consists of two RNN networks, called encoder and decoder. The encoder RNN encodes the input sequence into a code vector containing the input knowledge. Based on the encoding vector, the decoder RNN is used to decode the knowledge vector into the desired sequence. Therefore, the properties of sequence-to-sequence models that map an input sequence to an output sequence are suitable for solving facility layout problems.

4. Case Application

The influence on the semiconductor manufacturing business is especially noticeable due to the fast growth of electronic gadgets. For example, by 2030, China’s semiconductor manufacturing business will have grown by 31.9 per cent, with a tremendous market potential [19,20]. The semiconductor production process is complicated, and the system is massive. Few businesses have mastered the whole manufacturing process, while most companies can only complete a portion of the production process [21]. As a fundamental step in the semiconductor manufacturing process, chip (computer chips or microchips, etc., are referred to as chip) packaging accounts for a significant portion of semiconductor output. As a result, careful consideration should be given to the chip packaging production process [22,23]. Few scholars have used machine learning to solve the problem of facility layout optimization in chip workshops. With the global rise of digital manufacturing, immersive facility layout optimization is also widely used in chip manufacturing workshops. This application puts forward real-time solution requirements for facility layout optimization. Therefore, this case uses the production of a chip workshop in Chengdu as an application case.

4.1. Chip Packaging Workshop Analysis

(1)

Chip packaging equipment

Because there are few chip equipment manufacturers in China, a chip packaging workshop (CPW) depends mostly on equipment imported from European and American nations for production. HJW equipment focuses primarily on maintaining normal functioning, and there are few locations where the equipment may be enhanced and improved. The HJW CPW consists mainly of the disc, chip loading, bonding, plastic sealing, electroplating equipment, printer and rib, and other items, some of which are shown in Figure 5.

(2)

Packaging Process

It is only necessary to complete one chip packing process, and the kind of manufacturing equipment used seldom varies. In addition, the equipment is highly automated. The key process phases of the workshop are represented by the nine steps shown in Figure 6.

• Dicing is the process of separating each chip with its own set of electrical characteristics. The dicing process is classified into two types: semi-automatic cutting and full-automatic cutting. The HJW CPW uses full-automatic cutting.
• Chip loading is the process of assembling the chip on the frame. Resin bonding, eutectic welding, and lead-tin alloy welding are all common chip loading processes employed by the HJW CPW.
• Bonding involves connecting the bonding wires between the electrodes in contact with the chip and the frame pins. This process places exceptionally high demands on the wires used for bonding, since the wires significantly impact the chip’s durability and stability.
• Plastic wrapping protects the item from environmental impacts and allows it to perform steadily and consistently for an extended time. Plastic packaging is classified into two types: air-tight packaging and non-airtight packaging. Because the air-tight packing effect is a better option, the HJW CPW uses air-tight packaging.
• The primary goal of electroplating is to cover the silicon wafer in the chip with a thick, homogeneous coating of gold free of holes, gaps, and other flaws to improve the chip’s conductivity and solderability.
• Printing entails performing laser engraving on the front or back of the chip, with the content of the engraving including product information, which is mostly utilized for chip identification and tracking. Ink printing and laser marking are two basic types of printing. The HJW CPW makes use of laser marking.
• Rib cutting consists mostly of two processes: rib cutting and bending. Rib cutting is a technology that divides a whole piece into multiple pieces. Bending is the removal of undesirable material from a work-in-process and pressing the work-in-process into certain predetermined forms.
• The test’s purpose is to complete the operation by collaborating between the sorting machine and the testing machine, checking the function and electrical characteristics of the finished chips, and the sorting machine marking, sorting, and receiving the chips.
• The packing procedure necessitates that the packed items be easily picked up.

(3)

Workshop Layout Analysis

The HJW CPW is rectangular in shape, with a length of around 140 metres, a width of 60 metres, and a total area of more than 8000 square metres. Entry and exit points are located on the left and right sides of the CPW, and the workshop is divided into many tunnels. The main corridor is 6 metres in width, whereas this passage measures 3 metres in width. Scribing, loading, and bonding areas are the most common features of the CPW. There is also a warehouse with several shelves and raw material areas, as well as an electro-plating area, a plastic sealing area, bonding and testing areas, and rib cutting and testing locations. Figure 7 depicts the original general layout.

Table 1. Size of each area.

Number	Area Name	Length (m)	Width (m)
1	Dicing area	7.64	9.78
2	Raw material area	26.00	11.30
3	Loading area	20.10	17.90
4	Return cleaning area	18.73	9.45
5	Shelf area after loading	40.00	6.27
6	Gold wire bonding area	28.00	6.90
7	Aluminum wire bonding area	28.00	3.49
8	Plastic area	28.00	15.30
9	Plastic cold storage	20.00	15.30
10	Electroplating area	28.00	11.30
11	Printing area	10.77	1.60
12	Shelf area after printing	20.00	8.50
13	Rib cutting area	36.00	11.30
14	Testing area	34.74	4.00
15	Packaging area	35.14	5.60
16	Finished product Warehouse area	39.94	16.40
17	Bathroom	12.77	16.40

illustrates the size of each area in Figure 3.

Following the layout of the CPW in Figure 7 and Table 1, it may be split into 17 major areas. For study efficiency, we separated the two loading areas into loading zone 1 and loading zone 2 and then divided these two zones further into 18 zones. Figure 8 depicts a simplified representation of the CPW.

In the manner shown in Figure 8, a Cartesian coordinate system is formed, the starting point (numbered 0) and ending point (numbered 19) are input, and the coordinate system of each zone is generated as depicted in

Table 2. Coordinate values of each area.

Number	Coordinate Value	Number	Coordinate Value
1	[0, 20.65] or [0, 38.24]	11	[57.21, 45.48]
2	[9.07, 45.48]	12	[82.20, 41.45]
3	[27.86, 45.48]	13	[82.43, 45.48]
4	[21.78, 29.69]	14	[109.58, 45.48]
5	[21.78, 14.30]	15	[108.63, 33.56]
6	[32.01, 29.69]	16	[108.63, 26.52]
7	[57.21, 13.09]	17	[91.98, 11.68]
8	[57.21, 6.43]	18	[120.95, 11.68] [130, 20.65] or [130, 38.24]
9	[57.21, 29.69]	19
10	[82.40, 29.69]

.

In Figure 8, we can see that A₁ is 6 m and A₂ is 3 m. According to Formula (2), the area before optimization can be calculated as 7460.0354 m².

The material flow of each O-D pair may be computed using the information in Table 2 and the output from the workshop, as shown in

Table 3. Coordinate values of each area.

Number	O-D	Distance (m)	Output (pcs)	Logistics (m·pcs)
1	0-2	28.79	4200	120,918
2	2-1	18.79	4200	78,918
3	1-3	16.21	2100	34,041
4	1-4	33.67	2100	70,707
5	3-5	19.29	3000	57,870
6	4-5	18.48	3000	55,440
7	5-6	24.33	3500	85,155
8	5-7	30.18	1500	45,270
9	5-8	34.29	1000	34,290
10	6-7	35.85	2000	71,700
11	6-8	35.45	1500	53,175
12	7-9	16.60	5000	83,000
13	8-9	23.26	4000	93,040
14	10-9	25.19	2000	50,380
15	9-11	15.79	11,000	173,690
16	11-12	25.31	12,500	316,375
17	12-13	4.04	10,000	40,400
18	12-14	27.67	2500	69,175
19	13-14	27.15	10,000	271,500
20	14-15	11.96	9000	107,640
21	15-16	7.04	9000	63,360
22	16-17	22.30	11,000	245,300
23	17-19	39.06	11,000	429,660
total				2,651,004

.

As shown in Table 3, the shortest distance for single-chip production and handling in this workshop is around 540.7 metres, and the total logistics volume of the CPW’s current layout is 2,651,004 m·pcs.

4.2. Solution Result Analysis

This case study uses Python for programming, and the operating environment uses the Windows 10 (64-bit) operating system, configured as Intel i7-9750H 2.60 Ghz 12 cores and 24 G memory. Model training uses most of the parameters found in the literature [15,16]. For this study, we used the neural network of GRU RNN to solve the WFLOP problem. Specifically, the parameter settings of the network model are shown in

Table 4. Model parameter settings [15,16].

Neural Network	Parameter
Actor network (Pointer network)	Encoder: 1D-Conv(Dinput, 128, kernel size = 1, stride = 1)
	Decoder: GRU(hidden size = 128, number of layer = 1)
Critic network	1D-Conv(Dinput, 128, kernel size = 1, stride = 1) 1D-Conv(128, 20, kernel size = 1, stride = 1) 1D-Conv(20, 20, kernel size = 1, stride = 1) 1D-Conv(20, 1, kernel size = 1, stride = 1)

. D_input represents the dimension of input, i.e., D_input = 4 for bi-objective WFLOP. We employed a one-layer GRU RNN with a hidden size of 128 in the decoder. For the critic network, the hidden size was also set to 128. We trained both the actor and critic networks using the Adam optimizer with a learning rate η of 0.0001 and batch size of 200. The Xavier initialization method was used to initialize the weights for the first subproblem [24]. Weights for the following subproblems were generated by the introduced neighbourhood-based parameter transfer strategy [16].

The trained model can achieve real-time solution results, and the final decoding of the calculation results is shown in Figure 9.

CPW device coordinate values are presented in

Table 5. Coordinate values for each area after optimization.

Number	Coordinate Value	Number	Coordinate Value
1	[0, 20.65] or [0, 38.24]	11	[57.21, 42.48]
2	[9.07, 42.48]	12	[73.70, 42.48]
3	[27.86, 42.48]	13	[84.50, 42.48]
4	[21.84, 29.69]	14	[104.58, 42.48]
5	[8.15, 15.13]	15	[108.63, 33.56]
6	[29.44, 14.30]	16	[108.63, 26.52]
7	[57.21, 13.09]	17	[103.98, 11.68]
8	[57.21, 6.43]	18	[79.01, 11.68] [130, 20.65] or [130, 38.24]
9	[57.21, 29.69]	19
10	[82.40, 29.69]

because the optimum area location has been modified.

The optimized area index is 6993.596 m².

Table 6. Coordinate values for each area.

Number	O-D	Distance (m)	Output (pcs)	Logistics (m·pcs)
1	0-2	28.79	4200	120,918
2	2-1	18.79	4200	78,918
3	1-3	20.31	3585	72,811.35
4	1-4	30.36	615	18,671.4
5	3-5	17.16	5121	87,876.36
6	4-5	21.31	879	18,731.49
7	5-6	10.16	3500	35,560
8	5-7	27.80	1500	41,700
9	5-8	28.86	1000	28,860
10	6-7	35.85	2000	71,700
11	6-8	35.45	1500	53,175
12	7-9	16.60	5000	83,000
13	8-9	23.26	4000	93,040
14	10-9	25.19	2000	50,380
15	9-11	15.79	11,000	173,690
16	11-12	16.49	12,500	206,125
17	12-13	10.80	10,000	108,000
18	12-14	35.88	2500	89,700
19	13-14	25.08	10,000	250,800
20	14-15	11.96	9000	107,640
21	15-16	7.04	9000	63,360
22	16-17	14.85	11,000	163,350
23	17-19	23.78	11,000	261,580
total				2,279,587

depicts the calculated CPW data after optimization.

According to the above study results, the shortest distance for CPW single-chip production and handling has decreased from 540.7 m to 501.55 m, which is 39.15 m shorter than the previous distance. The area has been reduced from 7460.0354 m² to 6993.596 m², which is 466.4394 m² less than before optimization. Furthermore, its logistics volume has decreased from 2,651,004 m·pcs in the previous year to 2,279,587 m·pcs in the current year, representing a loss of 371,417 m·pcs. Because of the data analysis findings, it is not difficult to conclude that the new layout has achieved the aim of chip packaging workshop optimization.

4.3. Optimization Algorithm Comparison

NSGA-II is widely used in solving the WFLOP, and the solution effect is good. In order to verify the superiority of DRL in solving the WFLOP, this study uses the classic NSGA-II as the comparison object. The population size of the NSGA-II algorithm is 100, the maximum number of iterations is 3000 generations, and each is run 10 times. The comparison of optimal solution sets is shown in Figure 10.

From Figure 10, we can find that both NSGA-II and DRL have better solution results. The DRL optimal solution set is slightly better than the NSGA-II algorithm, and both algorithms obtain optimal solutions (227.9 × 10⁴, 669.3 × 10¹). The calculation time of NSGA-II is between 15 and 60 min, so it cannot meet the real-time solution requirements of virtual reality technology. After training, the DRL solution to the WFLOP can achieve a real-time solution effect and can meet the engineering needs of facility layout for virtual reality technology. The DRL method of solving the WFLOP provides theoretical support for the facility layout of immersive virtual reality technology.

5. Discussion and Conclusions

This paper proposes the application of DRL to solve a dual-objective WFLOP. It first constructs a dual-objective WFLOP mathematical model and proposes a novel dual-objective DRL framework. The DRL framework decomposes the WFLOP dual-objective problem into multiple sub-problems and then models each sub-problem. In order to save computational workload, a neighborhood parameter transfer strategy is adopted. A chain rule is constructed for the appealed sub-problem, and an improved pointer network is used to solve the bi-objective WFLOP of the sub-problem. Finally, the effectiveness of this method is verified by using the facility layout of a chip production workshop as a case study.

The research presented in this article has achieved some results, but there are some limitations. Further research is required on the following points:

(1)

This article only studies DRL (Deep Reinforcement Learning) to solve the dual-objective WFLOP (Workshop Facility Layout Optimization Problem). The DRL framework of the WFLOP for high-dimensional objectives needs further research.

(2)

The WFLOP platform for virtual reality technology has not yet been built. In the future, Unity 3D, Demo 3D, etc. will be used to build the WFLOP platform.

(3)

The generalization ability of DRL to solve the WFLOP needs further study to improve the compatibility of this method.