**About the Editor**

#### **Yaping Ren**

Yaping Ren, Ph.D., is Associate Professor in the Department of Industrial Engineering of the College of Intelligent Science and Engineering in Jinan University. In September 2019, he received his Ph.D. degree from the School of Mechanical Science and Engineering, Huazhong University of Science and Technology. From September 2017 to August 2019, he conducted joint doctoral training at Purdue University. He is mainly engaged in intelligent optimization and scheduling research for manufacturing/transportation logistics. So far, he has presided over two national research projects, five provincial and ministerial research projects, and one project funded by the Fundamental Research Funds for the Central Universities. Thirty-six SCI/EI high-level academic papers of his have been published and accepted, among which twenty-two SCI papers and four EI papers have been published and accepted with him as the first author/corresponding author, including *IEEE Transactions on Systems, Man, and Cybernetics: Systems; IEEE Transactions on Industrial Informatics; IEEE Transactions on Intelligent Transportation Systems; Robotics and Computer-Integrated Manufacturing; Journal of Cleaner Production; Expert Systems with Applications*; and other top journals in Zone 1 of the Chinese Academy of Science Journal Partition, as well as several authoritative Chinese journals such as *Journal of Mechanical Engineering* and *Computer Integrated Manufacturing Systems*. More than thirty national invention patents have been applied, twelve of which have been authorized. He has won three provincial and ministerial awards and is the youth editorial board member of journals such as *Journal of Applied Basic and Engineering Science and Mechanical and Electrical Engineering Technology*.

## *Review* **A Review of Combinatorial Optimization Problems in Reverse Logistics and Remanufacturing for End-of-Life Products**

**Yaping Ren 1,2,3, Xinyu Lu 4, Hongfei Guo 1,2,3,\*, Zhaokang Xie 4, Haoyang Zhang <sup>4</sup> and Chaoyong Zhang <sup>5</sup>**


**Abstract:** During the end-of-life (EOL) product recovery process, there are a series of combinatorial optimization problems (COPs) that should be efficiently solved. These COPs generally result from reverse logistics (RL) and remanufacturing, such as facility location and vehicle routing in RL, and scheduling, planning, and line balancing in remanufacturing. Each of the COPs in RL and remanufacturing has been reviewed; however, no review comprehensively discusses and summarizes the COPs in both. To fill the gap, a comprehensive review of the COPs in both RL and remanufacturing is given in this paper, in which typical COPs arising at the end of the product life cycle are discussed and analyzed for the first time. To better summarize these COPs, 160 papers published since 1992 are selected and categorized into three modules: facility location and vehicle routing in RL, scheduling in remanufacturing, and disassembly in remanufacturing. Finally, the existing research gaps are identified and some possible directions are described.

**Keywords:** reverse logistics; remanufacturing; EOL product; combinational optimization

**MSC:** 90-10

#### **1. Introduction**

Reverse logistics (RL) and remanufacturing are the two main processes for end-of-life (EOL) products' recovery, aiming to maximize resource utilization by means of collecting, disassembling, refurbishing, and reassembling to grant the EOL products the same quality and functionality as new products. In order to improve the efficiency of product recovery, it is vital to solve the combinatorial optimization problems (COPs) involved effectively; therefore, scholars have conducted much research on them.

The concept of RL was put forward by Stock [1] in 1992, whose essence was to transfer EOL products from the consumer to the producer for processing. The COPs in RL include facility location and the vehicle-routing problem (VRP). The facility location is to build an appropriate network structure to determine the location of various facilities, such as collection centers, remanufacturing centers, distribution centers, etc. The VRP is to formulate a specific transportation plan to transport EOL products to the above facilities, including the driving path, number of vehicles, types of vehicles, etc. There are some reviews about the facility location and VRP in RL [2–5].

However, only the transportation process of RL cannot truly realize the reuse of resources; remanufacturing is the key to achieve sustainable development [6]. Remanufacturing is a process to recover EOL products to the same state as new products through inspection, disassembly, cleaning, maintenance, replacement, reassembly, etc. [7]. The COPs involved include disassembly sequence planning (DSP), disassembly-line-balancing

**Citation:** Ren, Y.; Lu, X.; Guo, H.; Xie, Z.; Zhang, H.; Zhang, C. A Review of Combinatorial Optimization Problems in Reverse Logistics and Remanufacturing for End-of-Life Products. *Mathematics* **2023**, *11*, 298. https://doi.org/10.3390/ math11020298

Academic Editor: João Paulo Costa

Received: 25 November 2022 Revised: 28 December 2022 Accepted: 3 January 2023 Published: 6 January 2023

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

problem (DLBP), disassembly scheduling, production scheduling, reassembly, etc. There are also some reviews about each of the COPs in remanufacturing [8–11].

Relevant reviews have been provided for each COP involved in RL and remanufacturing as shown in Table 1. However, it lacks a literature to review all COPs from the perspective of the recovery process for EOL products recovery, which is important to draw a framework of COPs for product recovery. In this paper, the COPs involved in RL and remanufacturing are divided into three categories, namely, facility location and VRP in RL, scheduling in remanufacturing, and disassembly in remanufacturing. These COPs are analyzed from two perspectives: a mathematical model and intelligent optimization methods, to fill the blanks of current research.

The rest of this article is organized as follows: Sections 2–4 summarize the mathematical model of facility location and VRP in RL, scheduling in remanufacturing, and disassembly in remanufacturing, respectively. Section 5 analyzes and discusses the literature from the perspective of optimization methodology and problem uncertainty. Section 6 summarizes the full text and proposes the future research directions.

**Table 1.** Previous reviews about RL and remanufacturing.


#### **2. Facility Location and VRP in RL**

Before remanufacturing EOL products, collecting them from users is the first step, which is essentially a process of RL. To perform this efficiently, it is necessary to properly plan the location of various facilities and products' flow routes. Specifically, companies need to choose how to collect EOL products from users and transport them to collection centers, where to inspect EOL products, where to remanufacture EOL products to make them available for resale, and how to sell remanufactured products to potential users [12]. That is to solve the problem of facility location and the VRP in RL.

#### *2.1. Facility Location in RL*

Generally, the facility location determines the location of the collection center, remanufacturing center, distributing center, and so on to minimize the logistics costs. Scholars have designed different network structures based on these facilities, which can be summarized as three types. This section will analyze the three kinds of network structures from the perspective of mathematical models, including: general network structure, closed-loop network structure, and hybrid network structure.

#### 2.1.1. General Network Structure

The general network structure consists of four parts: consumers, collection centers, remanufacturing centers, and secondary markets, as shown in Figure 1. The collection center collects EOL products from consumers, inspects and disassembles the products, and then, according to the value of the components, chooses to discard or remanufacture; finally, the remanufactured finished products flow back to the market for sale. It is worth noting that in this network structure, remanufactured products are usually different from new products and mainly flow to the second-hand market.

**Figure 1.** General network structure.

It can be seen from Table 2 that most researchers choose the single-objective mixedinteger linear programming (MILP) model when establishing the mathematical model of the general network structure. Two papers established the mixed-integer nonlinear programming model (MILNP) [13,14], and they were single-objective optimization to maximize the total profit or minimize the total cost [14–20]. However, while minimizing the total cost, Roghanian and Pazhoheshfar [21] considered the uncertainty of capacity, demand, and product quantity in RL parameters, so they proposed a probabilistic mixed-integer linear programming model (P-MILP) and converted it into an equivalent deterministic model when solving. Tari and Alumur [22] considered the fairness between different companies and the problem of providing a stable product flow for each company while minimizing the total cost, thus establishing multi-objective mixed-integer linear programming (M-MILP).

In addition to minimizing the total cost and maximizing the total profit, the main purpose of establishing the mathematical model is to find out the location, capacity, and quantity of the core facilities, such as the collection center and the remanufacturing center that need to be opened in the RL network. Sasikumar et al. [13] provided decisions related to the number and location of facilities to be opened and the allocation of corresponding product flows through the establishment of MILP. Roghanian and Pazhoheshfar [21] proposed a multi-product, multi-stage RL network problem. It was not only necessary to determine the subset of disassembly centers and machining centers to be opened, but also necessary to determine the transportation strategy to meet the needs of manufacturing centers and recycling centers, with the minimum fixed opening cost and total transportation cost. Alshamsi and Diabat [16] also introduced important transportation considerations by providing options for using internal fleets and outsourcing options. Liao [14] introduced a modular remanufacturing process and emission reduction; two papers [14,16] also considered the carbon footprint while determining the location of factories (inspection/remanufacturing), the transportation of cores/remanufactured products between factories, and the route of vehicles between factories.


**Table 2.** Mathematical model of general network structure.

#### 2.1.2. Closed-Loop Network Structure

To protect the environment, many countries expand producer responsibility through legislation. Driven by economic benefits, many manufacturers began to integrate RL [23], thus evolving a closed-loop network structure (CLNS), as shown in Figure 2. The CLNS integrates forward logistics and reverse logistics, producing huge economic and environmental benefits, and is the most widely studied network structure by scholars. In the CLNS, manufacturers obtain raw materials from suppliers, and the products manufactured are sold to customers through distributors. The products used by customers are collected by the collection center and selectively sent to the remanufacturing center. After remanufacturing, they return to forward logistics. In the CLNS, the terminals of forward logistics and reverse logistics are the same customer group.

**Figure 2.** Closed-loop network structure.

Tables 3 and 4 are, respectively, the previous mathematical model research on CLNS and the corresponding abbreviations of terms. In paper [24], the environmental problems are integrated into an integer CLNS model, and a genetic algorithm based on the spanning tree structure is proposed to solve the NP-hard problem. Pishvaee et al. [25] also established a multi-objective fuzzy mathematical model to design the environmental protection supply chain. They used life cycle assessment to quantify the environmental impact of the network. Zohal and Soleimani [26] also regard the model as a green CLNS according to the CO2 emissions of the gold industry. As shown in Table 3, most scholars are studying how to use metaheuristics and heuristic algorithms to solve the model. Devika et al. [27] proposed six

different mixed metaheuristics to solve the sustainable CLNS problem they developed. In paper [28], a CLNS decision model under uncertainty was proposed, and the imperialist competitive algorithm, particle swarm optimization, and genetic algorithm were used to solve the large-scale NP-hard model developed by them. In another study, Fard et al. [29] considered the three-level decision model to express their forward/reverse supply chain network design problem, and adopted a variety of metaheuristic algorithms, including tabu search, variable neighborhood search, particle swarm optimization, water wave optimization, and Keshtel algorithm. The results show that the metaheuristic algorithm is an effective method to solve the model in practice.

**Table 3.** Mathematical model of CLNS.


**Table 4.** Abbreviations comparison table.


#### 2.1.3. Hybrid Network Structure

Some scholars have put forward the concept of hybrid facilities, that is, the merger of a manufacturing center and remanufacturing center, and the merger of a distribution center and collection center, as shown in Figure 3, thus representing integrated forward and reverse logistics [34]. Due to the existence of mixed facilities, RL can use the nodes of forward logistics to optimize its design, thereby effectively reducing or eliminating the cost of building new RL networks. Therefore, in recent years, it has also become a research hotspot of scholars.

**Figure 3.** Hybrid network structure.

In this growing research field, the number of literature sources is growing rapidly. Fleischmann et al.'s pioneering work in hybrid network structure modeling studied the impact of product recycling on the design of a logistics network [35]. The study argued that the impact of product recycling depended heavily on the environment. In some cases, it may be feasible to integrate this activity into the existing logistics structure, while in other cases, it may be necessary to redesign the logistics network in an overall manner. Since this study, Salma et al. [36] incorporated capacity constraints, demand uncertainty, and returns into the multi-product planning based on this model. Later, they integrated strategic and tactical decisions by considering two inter-related time scales: at the strategic level, they gave a discretization of the time range, which must meet the needs and reporting values; at the tactical level, more detailed planning was allowed to achieve this goal [37]. Cardoso et al. [38] analyzed the integration of RL activities under demand uncertainty, took the maximum expected net present value as the objective function, and made decision variables for facility size and location, process installation, forward and reverse logistics, and inventory level. Later, the author expanded this work to solve the uncertainty and characterize the elastic closed-loop network structure [39].

Subsequently, environmental and social sustainability issues began to be considered. Paksoy et al. [40] analyzed the supply plan and considered the emission cost (total cost minimization) and the profit maximization of the recycled products in the economic objective function. Mota et al. [41] considered the sustainability of the economy, environment, and society. They proposed a mathematical model with the minimization of the total cost of the network structure as the economic objective function; the minimization of the life cycle assessment index ReCipe as the environmental objective function; and the location of the network structure activities in underdeveloped areas as the social objective function. Gao et al. [42], based on the existing forward logistics network, proposed a double-objective stochastic integer programming model aiming at economic and environmental benefits, which aimed to support production, remanufacturing, and waste activities by addressing the uncertainty factors of new product demand and the return volume of old products in the customer area.

#### *2.2. VRP in RL*

After locating critical facilities, the next question is how to reversely transport EOL products, that is, the vehicle-routing problem in RL. The VRP in RL refers to how to arrange vehicle types, quantities, capacities, routes, etc., to collect waste products from customers, to achieve the goal of the minimum cost or the shortest route. Collection is the starting point of EOL product recovery, so scholars have carried out much research to improve the efficiency of this link.

The VRP in RL can be described as: a certain number of customers must use a certain number of fleets with limited capacity, which are usually assumed to be the same. The vehicles are stationed in a central parking lot and return after collecting waste products from customers according to the route requirements. Cao et al. [43] used RL to reuse the Internet of Things through identifying the resources required for road infrastructure, and modeled RL to transfer the Internet of Things from a tailings dam to a processing plant, and then to a road construction site. For minimizing the total cost, Richnák and Gubová [44] established a heterogeneous-fleet electric-vehicle path-recovery time window model considering vehicle load constraints. The type of vehicle limited the weight of recyclable waste and the time limit allowed by the customer. Chen et al. [45] proposed a nonlinear programming model including the number of second-hand products and reprocessed products. For both types of requirements, analysis and insight were provided in the form of a complete strategy consisting of different scenarios that allow optimal decisions to be made under a variety of conditions. Through sensitivity analysis, numerical examples supplemented the understanding of the model. Santana et al. [46] considered the risks in the e-waste recycling process and modeled the reverse logistics process of electrical and electronic equipment as a MILP with biological objectives under uncertainty. The cooperative alliance strategy was employed by Mishra et al. [47] to actualize the constrained capabilities of VRP in RL. The cooperative mechanism made it possible to prevent inefficient resource distribution, cut back on circular logistics, and minimize long-distance travel. In order to minimize recovery tasks, Chen et al. [48] used an improved ant colony algorithm to handle vehicle design and route optimization problems. Foroutan et al. [49] established a mixed-integer nonlinear programming model for multi-mode green vehicle routing and scheduling with the objective of minimizing operating costs and environmental costs and considering return, lead time, and delay costs. A mathematical model for the recovery of EOL cars was developed by Chaabane et al. [50] by combining the traditional VRP with the receiving and delivery problem as well as the restrictions of various vehicle kinds and time periods.

The multi-vehicle routing optimization problem with time limitations has also been the subject of interest for scholars' in-depth research. In order to meet the needs of minimizing transportation and procurement costs, the research in the literature [51] included choosing suppliers and setting up homogeneous fleets to buy various products from chosen suppliers. It also defined new branching rules, introduced new inequality families, and established the competitiveness of the new branching price-reduction method. To optimize the multidepot production material-allocation system and research the routing problem of delivery vehicles, Xu et al. [52] took into account a variety of factors, including multiple warehouses, multiple vehicle types, multiple commodities, mismatches between customer supply and demand, and arbitrary segmentation of delivery and delivery demands. Fan et al. [53] designed a genetic variable neighborhood algorithm for multi-vehicle routing optimization problem with fuzzy set requirements by improving the adaptive search strategy. A multimode electric vehicle routing optimization model was developed by Guo et al. [54] taking into account the differentiating service costs under the consideration of vehicle diversity, charging strategy, person vehicle matching, and service time difference.

With the improvement in logistics network and the development of e-commerce, in order to improve customer satisfaction, logistics service providers need to handle a large number of delivery orders and return orders at the same time. In this case, the

joint optimization of simultaneous pickup and delivery VRP (VRPSPD) can significantly improve the utilization rate of vehicles [55].

Studies have shown that solving the problem of collecting refurbished goods in forward and reverse logistics can improve the utilization rate of refurbished products, shorten the return time, and improve customer satisfaction [55]. Dethloff [56] studied VRPSPD in order to avoid redundant handling work, considering that customers have both picking and delivery needs. They developed an insertion-based heuristics method, which can be used to construct initial feasible solutions, which can be improved by subsequent application of the local search process. A specific case of VRPSPD that allows for the decomposition of the picking and delivery needs was developed by Masson et al. [57]. This method can be used in real-world transportation systems with many of pickup trucks but few deliveries. Nagy et al. [58] proposed a VRP with separable delivery and picking, and studied the cost reduction caused by demand segmentation of simultaneous delivery and pickup. In order to optimize vehicle scheduling to satisfy freight requests, Ghilas et al. [59] created a VRPSPD with scheduling lines and took synchronization and time window limitations into account in the solution algorithm. Gschwind et al. [60] evaluated the performance of branch cut and price algorithms in VRPSPD to solve the shortest path issue under time windows and resource limitation. Goeke [61] investigated the VRPSPD of electric cars, in which the route design of electric vehicles attempted to optimize the pick-up and delivery services in metropolitan areas with a lower environmental impact. Wolfinger [62] proposed a mixed-integer programming model to develop a single warehouse VRPPD with split load, and tested the algorithm performance through extensive computing experiments through large-scale neighborhood search. Haddad et al. [63] developed a multi-warehouse VRPPD to design a sustainable picking and distribution route between multiple warehouses. The efficiency of transportation can be increased at the same time, within the limitations of vehicle capacity and time window.

#### **3. Scheduling in Remanufacturing**

The remanufacturing system is generally composed of three subsystems: disassembly, reprocessing and reassembly, as shown in Figure 4. Previous research usually took the scheduling of reprocessing and reassembly as the production scheduling in remanufacturing. Disassembly scheduling can be defined as the problem of determining which products or subassemblies, how many, and when to disassemble EOL products to satisfy the demand of their parts or components. This paper reviews scheduling in remanufacturing into disassembly scheduling, production scheduling, and integrated scheduling

**Figure 4.** Composition of a typical remanufacturing system.

#### *3.1. Disassembly Scheduling in Remanufacturing*

In the last few years, there has been a growing interest in the disassembly-scheduling problem. Since Gupta and Taleb [64] described the basic form of the disassembly-scheduling problem, a number of research articles have been released. We review literature sources on approaches employed to model capacitated or incapacitated problems.

The majority of the early research took into account the scheduling issue of incapable disassembly. Firstly, Gupta and Taleb [64] proposed a reverse material requirement planning algorithm (RMRP) without an explicit objective function. Later, Taleb et al. [65] proposed a different RMRP algorithm to reduce the number of products that needed to be disassembled by taking into account the commonality of parts in this fundamental instance. In order to reduce the total cost of setup, disassembly operations, and inventory holding, Kim et al. [66] applied a heuristics algorithm and a linear programming (LP) relaxation technique to the basic problem put forth by Taleb et al. [65]. In order to reduce the total cost of product purchase, installation, disassembly operation and inventory holding, Lee et al. [67] adopted a two-stage heuristic algorithm to solve the problem based on the research of Gupta and Taleb [64]. Lee and Xirouchakis [68] proposed and solved three integer programming models for three problem cases, i.e., a single product type without parts commonality and single and multiple product types with parts commonality, they used CPLEX to solve the problem. Inderfurth and Langella [69] adopted a heuristic algorithm to reduce the estimated cost of disassembly operation, purchase and disposal, taking into account such factors as multiple product types, commonality of components, two-stage structure and uncertainty of production rate. Later, Kongar and Gupta [70] further improved their earlier work by incorporating uncertainty into the consideration of the problem and expressing the uncertainty of the problem through fuzzy goal programming [71]. Barba-Gutierrez and Adenso-Diaz [72] extended their earlier work by integrating uncertainty requirements. Barba-Gutierrez, Adenso-Diaz et al. [73], proposed an algorithm: F-RMRP (RMRP based on fuzzy logic) to solve the problem. Kim and Lee [74] considered a multi-period version problem and proposed a heuristic algorithm using priority rules to solve it. Recently, Kim et al. [75] proposed a two-stage heuristic algorithm based on their earlier model of Kim and Lee [74]. Most recent studies considered the capability of the disassembly-scheduling problem. Lee et al. [76] proposed an integer programming model whose objective function was to minimize the sum of disassembly operation, product purchase, and inventory carrying costs. Later, Kim et al. [77] proposed an optimization algorithm with the minimum number of disassembled products as the objective function. Later, Kim et al. [78], based on the research of Kim et al. [77], considered the more complex actual situation, added the minimum installation, disassembly operation and inventory carrying cost into the objective function, and proposed a Lagrange heuristic algorithm to solve the problem. In addition, Prakash et al. [79] proposed a disassembly-scheduling problem model for parts commonality and proposed a constrained simulated annealing algorithm to solve it. Liu and Zhang [80] built a non-convex mixed-integer model based on the research of Prakash et al. [79]. Aiming at the optimal collection price, appropriate disassembly time and quantity of recycled products, Liu and Zhang proposed a particle swarm optimization algorithm based on dynamic programming to solve the problem. Ullerich and Buscher [81] established an integer linear programming model considering complete disassembly scheduling by considering the capacity constraints of each time period. Later, Ji et al. [82] added the consideration of start-up and setup costs on the basis of Ullerich and Buscher [81], and proposed a Lagrange relaxation heuristic algorithm to solve the problem. Godichaud et al. [83] built an MILP model considering the penalty cost of sales loss and disassembly capacity overload, and proposed a genetic algorithm to solve it. Hrouga et al. [84] adopted a hybrid genetic algorithm and fix-and-optimization heuristic algorithm to solve the disassembly batch scale problem of multi-type products with sales loss and capacity constraints. Based on the original study by Hrouga et al. [85], the problems of disassembly batch size under sales loss, multiple product types, and two-tier and capacity constraints were considered by Hrouga et al. [84], and the objective

function was to minimize the sum of installation, inventory, sales losses and overload costs. An efficient optimization method based on a genetic algorithm and fix-and-optimization heuristic was proposed. Kim and Xirouchakis [86] considered the problem of multi-cycle, multi-product type, two-level product structure and random demand, aiming to minimize the sum of expectation setting, inventory holding and penalty costs of unmet requests, and proposed a Lagrange relaxation heuristic algorithm for solving the problem. Inderfurth and Langella [69] proposed a disassembly-scheduling problem of random output considering the impact of yield uncertainty on the stochastic scale model and binomial model, and proposed a two-root three-leaf mathematical model to describe the problem. Liu and Zhang [80] constructed a mixed-integer nonlinear programming considering the capable disassembly-scheduling problem of stochastic yield and demand, and proposed an algorithm based on external approximation to solve it. Tian and Zhang [87] considered the problems of capable disassembly scheduling and pricing, and established a non-convex mixed-integer model with the objective of determining the appropriate collection price of recycled products and the appropriate disassembly time and quantity, which was solved by the algorithm combining particle swarm optimization and dynamic programming. Zhou, He et al. [88] considered the capability disassembly-scheduling problem with the uncertainty of demand and disassembly operation time, constructed a new stochastic programming model, and proposed a hybrid genetic algorithm to solve it. Yuan, Yang et al. [89] proposed a capable fuzzy disassembly-scheduling model with cycle time and environmental cost as parameters, proposed a mixed-integer mathematical programming model with the goal of minimizing cycle time and environmental cost, and proposed a metaheuristic algorithm based on the fruit fly optimization algorithm to solve the problem. Slama, Ben-Ammar et al. [90] constructed a new mixed-integer programming model with the goal of maximizing disassembly process gain by considering external procurement, defects and late-order items, setup time, and capable dynamic batch problems. Slama et al. [91] considered the random multi-period disassembly batch problem and proposed a special optimization method according to specific different scenarios. (i) A two-stage mixed-integer linear programming model was proposed to solve all possible scenarios of small cases. (ii) The sample average approximation method based on Monte Carlo simulation was proposed for all possible scenarios of medium-scale examples. (iii) For all possible scenarios of large-scale instances, an optimization algorithm based on Monte Carlo simulation and genetic algorithm was proposed.

#### *3.2. Production Scheduling in Remanufacturing*

Process planning and production scheduling for remanufacturing are more challenging than traditional manufacturing because there are many uncertain factors in the remanufacturing system, for example, there are uncertainties in the processing route and time of processing different kinds of materials in the remanufacturing process. These uncertainties will lead to the failure of the usual process planning and scheduling methods. Therefore, many researchers have conducted a great amount of meaningful exploration on the uncertainty of remanufacturing production.

Wen H et al. [92] took the minimum remanufacturing time as the objective function, established a production-scheduling comprehensive optimization model with double random variable constraints, and proposed a hybrid algorithm of stochastic simulation technology, a neural network, and a genetic algorithm to solve the problem. He P [93] proposed a quality evaluation standard for the remanufacturing production-scheduling problem under the two uncertain conditions of randomness and fuzziness of job scheduling. Then, based on this, he built a remanufacturing production-scheduling model under uncertain conditions, and adopted the hybrid algorithm of the BP neural network and genetic algorithm to solve the problem. Peng S et al. [94] took the high value-added cylinder block of the engine as the research object, took the minimum manufacturing span as the objective function, considered the uncertainty of processing time and path, and proposed a new rule-based dynamic window algorithm to solve the problem. Zhang [95] considered the remanufacturing production scheduling problem under random and fuzzy conditions, built a quality evaluation method for remanufacturing recovered resources, and based on this, established a remanufacturing production scheduling model under uncertain conditions, and proposed a hybrid algorithm combining a double-fuzzy algorithm, BP neural network, and genetic algorithm to solve the problem. Shi J et al. [96] proposed a new double-fuzzy remanufacturing scheduling model, which considered many double uncertainties in remanufacturing and used double fuzzy variables to describe these uncertainties. Extended discrete particle swarm optimization algorithm was used to solve the problem.

#### *3.3. Integrated Scheduling in Remanufacturing*

Kim et al. [97] encouraged the integration of all remanufacturing operations (disassembly, remanufacturing/repair, and reassembly) into remanufacturing scheduling decisions. The past research on the scheduling problem of integrated remanufacturing system can be divided into flow shop type and job shop type. The flow-shop-type reprocessing shop is oriented to small-batch, multiple varieties of personalized remanufacturing products; the job-shop-type reprocessing shop is oriented to small-batch, multiple varieties of a single type of remanufacturing product type.

For the remanufacturing system with a flow-shop-type reprocessing shop, Stanfield, King et al. [98], aiming to minimize in-process operations and maximizing system utilization, proposed a stochastic scheduling heuristic algorithm to solve the problem. In addition, Kim et al. [99], aiming to minimize the total flow time of a remanufacturing system with parallel flow-shop reprocessing lines, proposed three heuristic algorithms for solving the problem, namely, the heuristic algorithm based on priority rules, the heuristic algorithm based on Nawaz–Enscore–Ham and the iterative greedy algorithm. Later, Kim et al. [75], based on the research by Kim et al. [99], proposed an algorithm based on priority rules to minimize the total delay. Qu et al. [100] proposed a new FPA algorithm based on the hormone regulation mechanism to solve the waiting-free flow-shop scheduling problem, and introduced hormone-regulatory factors to enhance the global search capability of the algorithm. Wang et al. [101] studied the scheduling problem of a remanufacturing system with a parallel disassembly workstation, parallel flow-shop-reprocessing line, and parallel reassembly workstation, and adopted the improved multi-objective invasive weed optimization algorithm to solve it. Wang, Tian et al. [102] considered the scheduling problem with parallel disassembly workstations, multiple parallel flow-shop-reprocessing lines, and parallel reassembly workstations. Aiming to minimize the total energy consumption, they proposed an improved genetic algorithm to solve the problem.

For the remanufacturing system with a shop–shop reprocessing shop, Guide Jr [103] proposed a drum-slow Okinawa scheduling method for a military warehouse. Subsequently, Daniel and Guide Jr [104] reported the performance of various scheduling rules and order release strategies based on the drum-cached Okinawa scheduling method. In addition, Souza et al. [105] proposed a two-stage solution, aiming to meet the profit maximization of customer service level, built a queuing network model, and solved using priority rules. See Guide Jr et al. [106] and Li et al. [107] for other models and solving algorithms. Kang and Hong [108] studied the disassembly and reassembly optimization problem; established an integer programming model; and solved the problem with the minimum disassembly cost, inventory cost, and manufacturing cost of new parts as targets. Lin D et al. [109] took the optimal factory selection and optimal remanufacturing job scheduling as objectives under resource constraints, and used linear physical programming and the multi-level-coded genetic algorithm to solve the problem. Giglio et al. [110] considered an integrated-batch-size and energy-saving job-shop scheduling problem, constructed a mixed-integer programming model, and proposed a relaxation-fixed heuristic algorithm to solve it. Yu and Lee [111] considered the scheduling problem of a remanufacturing system with parallel disassembly workstations, shop-like reprocessing workstations, and parallel reassembly workstations, constructed an integer programming model, and proposed two solving algorithms: a decomposition algorithm and integration algorithm. Li et al. [112] proposed a hybrid metaheuristic algorithm embedded with a colored, timed Petri net scheduling strategy to solve the problem of the optimal recovery route and recovery operation sequence for searching the worn core on the workstation, aiming to minimize the total production cost. Fu, Zhou et al. [113] adopted a multi-objective discrete Drosophila optimization algorithm to solve the stochastic multi-objective disassembly reprocessing–reassembly integrated scheduling problem in order to minimize the expected production cycle and total delay. Zhang, Zheng et al. [114] considered the integrated process planning and scheduling problem of a remanufacturing system containing a parallel disassembly workstation, a flexible job-shop-type reprocessing shop, and a parallel reassembly workstation, and proposed an improved artificial swarm algorithm to solve it.

#### **4. Disassembly in Remanufacturing**

Disassembly refers to obtaining valuable components from EOL products with various resource constraints. Disassembly in remanufacturing mainly includes two COPs: disassembly sequence planning (DSP) and disassembly line balancing problem(DLBP). DSP means obtaining the required components in an optimal sequence. DLBP means assigning disassembly operations to different workstations on a disassembly line to achieve one or multi-objectives.

#### *4.1. Disassembly Sequence Planning*

The most common objective of DSP is to improve the disassembly efficiency and reduce the cost. Disassembly sequence planning can be divided into three steps as shown in Figure 5: First, choosing a suitable disassembly mode which could be complete/partial disassembly or sequential/parallel disassembly. Second, building a model about the disassembly relationship of parts or components of EOL products. Third, choosing an optimization method to solve the DSP problem. There has been a great amount of research on DSP, and various methods have been developed. The most commonly used methods are heuristic algorithms. Additionally, the most prominent advantage of the heuristic algorithms is that they can obtain high-quality solutions in an acceptable time for largescale problems [115]. For example, Tseng et al. [116] developed a new block-based genetic algorithm (GA) with the penalty function matrix in the crossover and mutation mechanism for disassembly sequence planning. Similarly, ElSayed et al. [117] and Li et al. [118] used a GA to address DSP on different occasions. It is obvious that a GA is easy to use and effective. However, more and more studies have investigated other heuristic algorithms. Zhong et al. [119] solve the DSP including fasteners using Dijkstra's algorithm and particle swarm optimization (PSO), which is another metaheuristic algorithm. Guo et al. [120] proposed a lexicographic multi-objective scatter search method to overcome the complexity explosion caused by a large-scale DSP considering a multi-objective resource-constrained operation. Liu et al. [121] built the disassembly model by using the modified feasiblesolution-generation method, and a robotic DSP was solved by using an enhanced discrete bee colony algorithm. Additionally, Tao et al. [122] proposed an improved Tabu search heuristic algorithm with an exponentially decreasing diversity management strategy; a partial and parallel DSP problem was solved to show the proposed algorithm was feasible and efficient. Guo et al. [123] and Ren et al. [124–127] investigated various types of DSP problems by using different heuristic algorithms. In summary, a heuristic algorithm is one of the most effective methods to solve DSP.

**Figure 5.** The three steps in DSP.

There are few studies which have investigated the exact mathematical methods. The exact mathematical methods can quickly obtain high-quality solutions for small-scale problems without traversing the entire solution space [115]. However, exact mathematical methods are more restrictive than other techniques. If the problem is large-scale, that is, with many variables and constraints, it is difficult to obtain a solution in an acceptable time. However, still some valuable papers are proposed because exact mathematical methods still have the advantage of quickly producing solutions when solving small-scale problems. For example, Zhu et al. [128] introduced a disassembly information model with dynamic capabilities to handle state-dependent information and presented a linearprogramming-based optimization model to obtain the optimal disassembly sequence. Costa et al. [129] developed a recursive branch-and-bound algorithm to obtain the optimal disassembly sequence. They also proposed a best-first search algorithm to accelerate the optimization process. Some literature sources [130–132] also investigated the exact mathematical methods, but the solutions were not good enough when encountering largescale DSP.

#### *4.2. Disassembly-Line-Balancing Problem*

The DLBP was first proposed by Güngör and Gupta [133] and more and more studies have investigated this problem. As Figure 6 shows, the main difference between DSP and the DLBP is the disassembly operation assigned to the workstation, which makes the DLBP more complex than DSP. Hence, there are some unique variables in the DLBP, such as objectives which could minimize the number of workstations or idle time [134].

**Figure 6.** An example of the DLBP.

Like DSP, many studies have also applied a metaheuristic algorithm to solve the DLBP. For example, Yin et al. [135] considered a partial DLBP with multi-robot workstations; they established an exact mixed-integer programming model and proposed a multi-objective hybrid driving algorithm to effectively address the above problem. Li et al. [136] introduced the profit-oriented U-shaped partial DLBP for the first time. They established an integer linear programming model for solving the small-scale problem. Additionally, a novel discrete cuckoo search algorithm was implemented and improved to solve the considered DLBP. Wang et al. [137] proposed a discrete multi-objective artificial bee colony algorithm to address the partial DLBP considering both economic benefits and environmental impacts. Ren et al. [138–141] also investigated the problem of disassembly line balancing and applied different heuristic methods to meet different objectives.

The exact mathematical methods can produce higher-quality solutions when solving the small-scale DLBP, so there are some papers which have investigated them. Yılmaz et al. [142] focused on a multi-objective DLBP considering the tactical-level strategies and operational-level scenarios. They also developed the improved augmented --constrained method to obtain the Pareto solutions for small-scale problems. Edis et al. [143] studied a DLBP with balancing issues, the hazardousness of parts, and other factors. They developed a generic MILP model for the above problem and their proposed method proved effective after a series of tests. There are literature sources [144–146] which have also studied the exact mathematical methods to solve the DLBP, but still could avoid the problem of not being able to produce better solutions when encountering large-scale problems.

In conclusion, the DLBP has been well-researched and developed over decades. The exact mathematical methods and heuristics algorithm have shown their effectiveness at different scales of problems.

#### *4.3. Integrated Disassembly and Reassembly*

Reassembly is the operation following disassembly. Reassembly refers to the assembly of reused parts, remanufactured parts, and original manufacturing parts through specific processes to produce reusable products. However, because it is not the same as normal assembly, reassembly will encounter different problems. For example, uncertainty in reassembly can lead to a huge increase in the complexity of the problem. The quantity or quality of the remanufactured parts is uncertain, which leads to a decline in production efficiency, to unstable product quality, and to many other problems [147].

A few literature sources investigated the optimization of only reassembly but without considering disassembly. For example, Su et al. [148] proposed an optimal selectiveassembly method for remanufacturing based on an ant colony algorithm in order to gain optimal reassembly combinations and reduce the influence of the uncertainty factors on the quality. Additionally, Liu et al. [149] proposed a simulated annealing genetic algorithm to solve the assembly deviation degree on-line optimization model.

As mentioned above, optimization that only considers reassembly is incomplete. Therefore, it is necessary to consider the complexities of reassembly when solving any disassembly problem, that is, the integrated problem of disassembly and reassembly. For example, Behdad and Thuston [150] proposed graph-based integer linear programming combined with multi-attribute utility analysis to obtain the optimal sequence of disassembly operations which considers the costs and uncertainties of both disassembly and reassembly. Su et al. [151] developed a multi-objective optimization method based on the TS-NSGAII hybrid algorithm which provided a new direction for the optimization of the remanufacturing system. Li et al. [152] established an optimization decision-making model for the reassembly process, and proposed a decision-making method based on the improved T-S FNN which focused on minimizing the remanufacturing time and costs. Oh and Behad [153] proposed a network flow graph and integer linear programming to produce solutions of the type and number of parts that should be reassembled and procured. Some literature sources focused on the whole remanufacturing system; Lahmar et al. [154] studied the remanufacturing production planning problem to minimize economic costs and

carbon emissions. They developed a non-dominated sorting genetic algorithm (NSGA-II) based approach to overcome the complexity of the whole remanufacturing system. Polotski et al. [155] investigated a hybrid manufacturing–remanufacturing system considering the optimization of manufacturing, remanufacturing, and maintenance policies. The stochastic dynamic programming approach was used to address this problem. In summary, it is a trend to study the optimization of the whole remanufacturing system in order to fully optimize all aspects of the system.

#### **5. Analysis and Discussion**

This section will discuss and analyze the above-mentioned literature sources from two aspects: optimization methodology and problem uncertainty.

#### *5.1. Optimization Methodology*

The problems described in Section 2 to Section 4 are essentially COPs. For COPs, the most important step is how to solve them after establishing the relevant mathematical model, and the solution methods are mainly divided into exact and heuristic or metaheuristic algorithms. The exact algorithm includes the branch and bound method, the dynamic programming method, the cutting-plane method, etc. However, due to the complexity of practical problems and the NP-hard attribute of most COPs, it is difficult for the exact algorithm to obtain a satisfactory solution for large-scale problems in a reasonable time. Therefore, most of the current research focuses on the development of heuristic or metaheuristic algorithms. This section will analyze the solution methods of the COPs mentioned in the above sections.

Tables 5–7 summarize exact algorithms, heuristic or metaheuristic algorithm, and some other algorithms, respectively. From the results of horizontal comparison, the number and types of heuristic or metaheuristic algorithm are far greater than those of the exact algorithm. This shows that in the field of RL and remanufacturing, using heuristic or metaheuristic algorithms to solve problems is the mainstream method of current research. Moreover, some classical algorithms have many applications in different problems, such as genetic algorithms [156], particle swarm optimization [157], ant colony algorithms [158], simulated annealing [159], and Tabu search [160].



**Table 6.** Heuristic or metaheuristic algorithms.



#### **Table 6.** *Cont.*

#### **Table 7.** Other algorithms.


Deep reinforcement learning (DRL) solving COPs in the process of EOL product recovery has attracted extensive attention in recent years. Bengio et al. [161] proposed three types of paradigms for the application of machine learning to COPs. Lei et al. [162] proposed an end-to-end DRL framework to solve the TSP and the CVRP. Yang et al. [163] proposed the framework of robotic disassembly sequence planning using DRL to solve the robotic disassembly sequence planning problem. In general, compared with heuristic or metaheuristic algorithms, DRL is seldom used to solve COPs in the process of EOL product recovery at present. However, DRL has been proved to be superior to metaheuristic algorithms in solving certain problems. Therefore, future research can focus more on DRL.

#### *5.2. Problem Uncertainty*

Due to the complexity of the reality and the instability of the state of EOL products, the recovery process will face a series of uncertainties. At present, most of the studies only have considered the deterministic conditions, but the uncertain studies are more practical.

In the stage of facility location and the VRP in RL, it is easy to face uncertainty of quantity, facility capacity, demand, etc. Therefore, Roghanian and Pazhoheshfar [21] considered the uncertainty degree of the demands, capacities, and quantity of EOL products, and proposed a P-MILP model to decide which subsets of processing and disassembly centers will be opened. Lee et al. [34] established a two-stage stochastic programming model which assumed the demand of forward items and the supply of returned products on customers to be stochastic parameters with known distribution. To solve the S3PRLP selection problem with unknown attributes and decision-maker weights, Arunodaya et al. [47] developed a hybrid methodology that combined CRITIC and EDAS methods with Fermatean fuzzy sets (FFSs). In addition, there is also other research on uncertainty in the collection stage of EOL products [36,38,39].

In the area of scheduling in remanufacturing, it often faces the uncertainty of the quality of old products, the demand for new products, and the supply of raw materials. Many scholars have studied the uncertainty in this field [95–97,113]. Kongar et al. [70] established a multi-criteria optimization model of a disassembly-to-order (DTO) system under uncertainty and adopted the fuzzy goal programming technique to solve the problem. Because real-world data on the demand for used components is frequently ambiguous, vague, or imprecise, Barba-Gutiérrez et al. [72] used a fuzzy logic approach to develop the reverse MRP algorithm, incorporating subjectivity and imprecision into the model formulation and solution process. Other similar studies such as Liu and Zhang [80] studied the capacitated disassembly-scheduling problem under stochastic yield and demand. Wen et al. [92] optimized the integration of remanufacturing production planning and a scheduling system under uncertainty. He [93] developed a useful optimization method for the production scheduling in remanufacturing under uncertain conditions. Peng et al. [94] studied a Petri-net-based scheduling scheme and energy model for the remanufacturing of a cylinder block under uncertainty.

In the field of disassembly in remanufacturing, there are many failure features, such as wear, fracture, deformation, and corrosion, which may influence the disassembly time and cost. Behdad et al. [132] used immersive computing technology as a tool to explore an alternative disassembly sequence scheme in an intuitive manner, taking into account uncertain conditions such as time, cost, and the probability of causing damage. Liu et al. [144] researched the DLBP with partial uncertain knowledge, that is, the task-processing time mean and covariance matrix. A new distributionally robust formulation with a joint chance constraint was proposed. Behdad and Thurston [150] considered the costs and uncertainties associated with disassembly and reassembly. To find the best set of tradeoffs, graph-based integer linear programming was combined with multi-attribute utility analysis.

#### **6. Conclusions**

The COPs in RL and remanufacturing have important academic value and practical value. In this paper, we divided the COPs into three categories, including facility location and the VRP in RL, scheduling in remanufacturing, and disassembly in remanufacturing; each of them contained several subcategories. At present, the research on these COPs mainly focuses on mathematical models and optimization methodology, which are also the two perspectives of this paper to review the current literature.

At the mathematical model level, through the analysis of this paper, the following suggestions are proposed for future research. Firstly, concerning the facility location and VRP in RL, researchers can pay more attention to the hybrid network structure, as described in Section 2, which can reduce the cost of logistics facilities and improve logistics efficiency. In addition to the uncertainty of the quantity and quality of EOL products, the demand for remanufactured products is also uncertain. Therefore, how to study the hybrid network

structure under uncertain conditions may be the direction that researchers should work hard towards. Secondly, in the area of scheduling in remanufacturing, scholars can consider more practical factors, such as multi-objective, limited buffer, uncertainty of task arrival time, etc. Moreover, the combination of the disassembly process and the scheduling process is also a direction that can be considered. Thirdly, for disassembly in remanufacturing, researchers can conduct more study on two-sided or U-shaped disassembly lines. Due to the danger of disassembly operations, human–robotic cooperation is also a field of concern. Similarly, due to the poor quality of EOL products, how to carry out disassembly sequence planning in the case of disassembly failure will be a reality that has to be considered.

At the optimization methodology level, DRL can solve the COPs end-to-end, thus avoiding the complex design of traditional optimization algorithms and its characteristics of low efficiency and high complexity. Therefore, researchers can consider developing relevant DRL algorithms.

**Author Contributions:** Conceptualization, Y.R.; project administration, H.G.; writing—original draft preparation X.L., Z.X. and H.Z.; writing—review and editing, X.L.; supervision, C.Z. All authors have read and agreed to the published version of the manuscript.

**Funding:** The research is financially supported by the National Natural Science Foundation of China (No. 52205526, No. 52205538), the Basic and Applied Basic Research Project of Guangzhou Basic Research Program (202201010284), the National Foreign Expert Project of Ministry of Science and Technology of China (G2021199026L), the Guangdong Province Graduate Education Innovation Project (82620516), the National Key Research and Development Program of China (2021YFB3301701), the Guangzhou Leading Innovation Team Program (201909010006), the Guangdong Province "Quality Engineering" Construction Project (210308), the Research Project of Characteristic Innovation of University Teachers (2021DZXX01), and Zhuhai Science and Technology Planning Project in the Field of Social Development (2220004000302).

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
