Optimal Control Policy of Unreliable Production Systems Generating Greenhouse Gas Emission

Abstract

The current paper addresses the production planning problem of unreliable manufacturing systems generating greenhouse gas (GHG) emissions, producing a single product type in a dynamic and stochastic context. This work aims to develop a control policy that minimizes the sum of backlog, inventory, and emission costs. To achieve this goal, the stochastic optimal control theory is used to develop the optimality conditions solved by numerical techniques to establish the control policy structure. Sensitivity analyses are provided to depict and validate the obtained structure of the production policy characterized by multiple thresholds, which regulate the production rate with the emission and inventory levels. Furthermore, the performance of the developed optimal control policy is compared with the most pertinent ones identified in the literature. The developed optimal control policy outperformed those in the literature by significantly reducing the total cost incurred by these policies. Finally, the developed control policy is implemented to equip the manager of the considered manufacturing system with a practical and robust decision-support tool.

1. Introduction

Since sustainable development was defined by Brundtland [1], many countries have implemented measures to actively promote individuals and businesses to minimize their environmental footprints and behave ethically [2]. Over time, governmental and non-governmental entities have prioritized substantial investments to mitigate the challenges presented by global climate change. Over the last few decades, decision-makers have faced numerous challenges linked to environmental constraints. These challenges revolve around the simultaneous pursuit of production objectives and the imperative of reducing greenhouse gas emissions. In the literature, papers addressing the ecological dimension, in conjunction with control policies, predominantly examine two distinct approaches: regulatory and voluntary. The government and authorities set a specific limit for GHG emissions generated by various industries as part of regulatory programs to control their detrimental effects. In recent years, there has been a growing interest in voluntary initiatives, such as managing environmental impacts, to enhance the effectiveness and scope of existing regulations. It is noteworthy that voluntary initiatives are appealing because they can accomplish environmental objectives more innovatively and with greater speed and cost-effectiveness compared to regulatory methods [3]. Kang et al. [4] raised an increasing concern regarding manufacturing companies’ tendency to prioritize their economic endeavors while neglecting the repercussions of these endeavors on the environment and society. In this context, Setchi and Maropoulos [5] presented important state-of-the-art theoretical, methodological, and applicative aspects of sustainable design and manufacturing. They discussed ways to achieve a balance between economic and environmental sustainability.

The current work carries significant implications for unreliable manufacturing systems, presenting the potential for applying both a regulatory and voluntary approach to reduce GHG emissions. Hence, the primary research questions are as follows: (i) what constitutes the most cost-effective control policy for managing the unreliable manufacturing system under consideration? (ii) In this context, is it necessary to apply voluntary and regulatory approaches? (iii) How many optimal stock products (threshold levels) are needed? (iv) When should they be changed? To answer these research questions, the objective of this paper is to develop an optimal control policy tailored to unreliable manufacturing systems of businesses committed to mitigating greenhouse gas emissions, thereby minimizing the total incurred cost.

In this paper, environmental and economic issues are jointly integrated into the optimization model for planning the control of unreliable manufacturing systems that generate greenhouse gas emissions during production. Thus, to minimize their environmental impact, it becomes economically necessary for these systems to determine the right time to adjust the production rate based on GHG emissions generated and the inventory level. Consequently, this research focuses on developing an optimal control policy aimed at minimizing the sum of costs associated with backlogs, inventory, and penalty emissions, particularly in settings where machines are subject to dynamic, stochastic failures and repairs.

The subsequent sections are structured as follows. A comprehensive review of pertinent literature is summarized in Section 2. The manufacturing system and the problem statement are presented in Section 3. Section 4 outlines the optimality conditions and presents the numerical techniques used. Section 5 summarizes the structure of the optimal production control policy and conducts a sensitivity analysis. The performance of the developed optimal policy is compared with that of those adapted from existing literature, presented in Section 6, while Section 7 engages in a comprehensive discussion of the findings. Section 8 serves as the paper’s conclusion.

2. Literature Review

Greenhouse gas emissions generated during production are one of the critical drivers of global warming. Many countries have moved toward reducing these emissions by adopting mechanisms that focus on emission taxes and cap-and-trade policies to comply with existing legislation and to reduce their environmental footprints [6]. The same observation was made by Entezaminia et al. [7] when they stated that under the environmental legislation mandated by governments for each industry, the emissions generated by the manufacturing systems stand as a pivotal performance metric for assessing sustainable manufacturing practices. Fundamentally, to preserve the environment and its natural resources, it is essential to control harmful emissions to stem the damage of pollution from industrial activities. Consequently, industries with high emissions, such as pulp and paper, mining operations, automobile manufacturing, steel, and concrete, will improve their operational strategies for production control problems in a stochastic context and by integrating environmental requirements. In the literature, numerous contributions to production planning issues within manufacturing systems can be categorized into two primary classes according to harmful emission control. The first class consists of work on production planning of manufacturing systems without GHG emission control, while the second class consists of work on production planning within manufacturing systems with GHG emission control. The work of the second class is classified into two categories according to the deterministic nature (first category) and the stochastic nature (second category) of the system dynamics. Table 1 highlights the key contributions of this work, as derived from the literature review. The rows of the table present the studied research classes/categories and the columns classify the discussed articles according to five attributes, namely, (i) stochastic and dynamic context, (ii) adapted control policy, (iii) developed optimal control policy, (iv) regulatory environmental approach, and (v) voluntary environmental approach.

The first class includes contributions in the literature relying on the production planning of manufacturing systems without GHG emission control. An in-depth examination of the literature revealed that several authors have considered optimal production planning problems in the manufacturing domain. According to Bouslikhane et al. [8], feedback control policies are the most effective strategies for managing systems in dynamic stochastic environments. An important branch of research has formulated the problem using stochastic optimal control models, as in [9]. In such pioneering research, a stochastic dynamic programming approach based on the infinite-horizon control problem was adopted to develop the optimality conditions related to the production planning problem. The resulting policies were defined by a distinctive structure, known as Hedging Point Policy (HPP), which aims to control the rate of production by considering a stock threshold and the system’s current state. Numerous extensions have since been formulated, considering various aspects of production planning management from diverse perspectives. Works based on the stochastic optimal control approach in a dynamic context have developed different Multiple HPP (MHPP) policies for systems with multiple states. For instance, Yang et al. [10] extended HPP by developing a feedback production and setup control policy. They utilized the surplus/backlog space of stocks to determine production rates as well as the times of setup changes, minimizing the total cost. In systems featuring multiple states, Diop et al. [11] investigated the impact of human errors during maintenance on production planning to enhance the safety of a flexible manufacturing system (FMS) with a failure-prone machine and Markovian demand patterns. A MHPP was also investigated in [12], which formulated a stochastic optimal production control problem for a single-machine multi-product manufacturing system with deteriorating items. The model aims to minimize expected discounted costs of inventory holdings and shortages, with optimal conditions derived through Hamilton–Jacobi–Bellman equations. In the same vein, Aghdam et al. [13] proposed a joint optimization strategy for maintenance and inventory management in production systems, employing a numerical approach to handle uncertain demand and shortages. The policy structures presented in the aforementioned works and their references were obtained by numerically solving the optimality conditions. However, the proposed control policies were obtained without considering the GHG emission generated by manufacturing activities.

The second class comprises contributions based on production planning of manufacturing systems with GHG emission control by adhering to standards set by regulatory agencies and adopting voluntary practices. Numerous strategies and approaches have been developed to incorporate environmental impacts, particularly those related to greenhouse gas emissions, as illustrated by examples presented in [14,15], which provide comprehensive overviews of the subject. Manufacturers’ environmental concerns are GHG emissions that involve economic issues, market-based cap-and-trade systems, and carbon taxes to reduce emissions [16,17].

The first category of this class concerns research work integrating the environmental dimension without reducing inventory levels. The issue of lot-sizing in production arises from considering the stochastic context of the dynamics of the machine in the optimization models used to determine the production control policies. Among the first works, Gong and Zhou [18] introduced policies for optimal production and greenhouse gas emissions’ trading, designed to minimize the total cost in a single-product manufacturing system. A target interval policy incorporating two thresholds was proposed as the optimal allowance trading policy. This approach, which addresses the Economic Order Quantity (EOQ), was explored in [19] for firms subject to carbon tax and cap-and-trade regulations. In [20], a sequentially structured dynamic optimization framework was developed to determine operational choices for managing manufacturing systems under cap-and-trade regulations. This framework focuses on choices regarding the acquisition of carbon credits and the management of excess emissions. Xu et al. [21] addressed multi-product manufacturing systems operating under the same regulations while focusing on the dual challenges of production and pricing. Zhao et al. [22] integrated the effects of carbon emissions into an economic model to balance economic and environmental interests. The model developed in [23] explores the balance between carbon emissions and production costs. It highlights that green production, while resulting in fewer emissions than conventional methods, incurs higher production costs. The proposal allows managers to regulate the balance by managing production strategies. Without considering the unreliability of the machines in their models, the obtained production policies could not be appropriate in a context where the non-production times due to random machine breakdowns are noted.

Table 1. Overview of literature contributions.

Research Classes/Categories	Stochastic and Dynamic Context	Adapted Control Policy	Developed New Optimal Control Policy	Regulatory Environmental Approach	Voluntary Environmental Approach
Class I. production planning of manufacturing system without GHG emission control
Akella and Kumar [9]	✓		✓
Yang et al. [10]	✓		✓
Diop et al. [11]	✓
Ouaret [12]	✓		✓
Aghdam et al. [13]	✓		✓
Class II. production planning of manufacturing system with GHG emission control
Category 1. Models considering reliability machines
Gong and Zhou [18]		✓		✓
He et al. [19]		✓		✓
Zhou et al. [20]		✓		✓
Xu et al. [21]		✓		✓
Zhao et al. [22]		✓		✓
Jauhari et al. [23]				✓
Category 2. Models considering unreliable of machines
Hajej et al. [24]	✓			✓
Turki and Rezg [25]	✓			✓
Turki et al. [26]	✓			✓
Ben-Salem et al. [27]	✓	✓		✓	✓
Afshar-Bakeshloo et al. [28]	✓	✓		✓	✓
Entezaminia et al. [7]	✓	✓		✓	✓
Behnamfar et al. [29]	✓	✓		✓
This paper	✓	✓	✓	✓	✓

Table 1. Overview of literature contributions.

Research Classes/Categories	Stochastic and Dynamic Context	Adapted Control Policy	Developed New Optimal Control Policy	Regulatory Environmental Approach	Voluntary Environmental Approach
Class I. production planning of manufacturing system without GHG emission control
Akella and Kumar [9]	✓		✓
Yang et al. [10]	✓		✓
Diop et al. [11]	✓
Ouaret [12]	✓		✓
Aghdam et al. [13]	✓		✓
Class II. production planning of manufacturing system with GHG emission control
Category 1. Models considering reliability machines
Gong and Zhou [18]		✓		✓
He et al. [19]		✓		✓
Zhou et al. [20]		✓		✓
Xu et al. [21]		✓		✓
Zhao et al. [22]		✓		✓
Jauhari et al. [23]				✓
Category 2. Models considering unreliable of machines
Hajej et al. [24]	✓			✓
Turki and Rezg [25]	✓			✓
Turki et al. [26]	✓			✓
Ben-Salem et al. [27]	✓	✓		✓	✓
Afshar-Bakeshloo et al. [28]	✓	✓		✓	✓
Entezaminia et al. [7]	✓	✓		✓	✓
Behnamfar et al. [29]	✓	✓		✓
This paper	✓	✓	✓	✓	✓

In the second category, some studies have addressed the production control problem in a stochastic and dynamic context. In such a context, Hajej et al. [24] addressed the problem of production and maintenance planning regulated by a carbon tax, considering the effects of system deterioration and subcontracting to support both remanufacturing and manufacturing systems. Their approach aimed to minimize the total costs of maintenance, production, inventory, and emissions over a finite horizon. Turki and Rezg [25] developed an optimal inventory production policy for a system that segregates new and remanufactured products and sorts used products based on quality, aiming to maximize profit while accounting for carbon emissions in the decision-making process. Building on this, Turki et al. [26] focused on optimizing manufacturing and remanufacturing planning under the carbon cap and trade policy. They highlighted how setup costs, return rates, and carbon policies significantly affect production and storage decisions, influencing system performance and emissions. Ben-Salem et al. [27] were the first authors to incorporate GHG emissions during production based on a balance between emission tax, backlog, and inventory costs. They thus proposed an extended HPP, called the Environmental Hedging Point Policy (EHPP), under carbon tax regulation. After surpassing a voluntary emission cap, they reduced the stock threshold to reduce the machine usage and, consequently, the GHG emission level to minimize the expected overall cost. They showed that the resulting control policies have economic advantages over the conventional HPP. Many other studies were based on this pioneering research. For instance, Afshar-Bakeshloo et al. [28] introduced an EHPP under carbon tax regulation. Their work simultaneously controls the production rates of a low-emission facility (LEF) and a high-emission facility (HEF). When total emissions surpass a predefined threshold, the manufacturing system halts HEF operations and switches to LEF production. Entezaminia et al. [7] proposed an MHPP policy that enables manufacturers to determine optimal timings for purchasing or selling allowances and adjust production rates to minimize total costs while reducing carbon emissions. More recently, Behnamfar et al. [29] examined the effect of carbon emission control policies on production planning and inventory management. They compared cap-and-trade and command-and-control policies using a simulation-based optimization approach to determine their impact on costs, resource utilization, and environmental performance.

The literature review has identified numerous production planning models for manufacturing systems that generate emissions during production. However, none have developed an optimal production policy for dynamic systems encountering stochastic failures and repairs, as highlighted in the last line of Table 1. The table outlines the research gaps and contributions of the work. Previous research did not create new optimal control policies but instead adapted existing policies from the literature or relied on common-sense approaches. This study addresses this gap by employing stochastic optimal control theory, grounded in the dynamic programming framework, to develop a new optimal control policy.

The considered system is modeled as an unreliable single machine dedicated to producing finished products. The problem of production planning over an infinite planning horizon is studied, focusing on the dynamics of finished product inventory fluctuations while accounting for the randomness of machine failures and repairs. To meet environmental regulations and consider these factors, decision-makers must adjust their production rates and determine the optimal safety stock levels. This aims to achieve the dual objectives of minimizing the expected total cost and reducing greenhouse gas (GHG) emissions.

The proposed model delivers an exact solution developed through stochastic dynamic programming, employing the maximum principle for the first time in this context. By numerically solving the optimality conditions derived from the maximum principle, the structure of the optimal production policy is determined. This forms the primary contribution of this paper.

3. Methods

First, this section presents the studied manufacturing system. Then, it delineates the formulation of the production planning problem.

The studied manufacturing system is depicted in Figure 1. It is subject to random events (failures, repairs) and is dedicated to the production of one type of product at a rate $u\left(t\right)$. The finished products are stored to establish an inventory ($x\left(t\right)$) to meet customer demand at a constant rate $d$. The raw materials for the machine are available at all times. On the environmental aspect, the machine generates GHG emissions ($e\left(t\right)$) during production. According to the carbon tax regulation, the excess emission units are penalized as an environmental tax if the cumulative quantity of emissions during a given period exceeds the regulatory threshold established by pertinent authorities. The machine pollution is characterized by a constant emission index ${\theta }_0$ (in the unit of emission volume per finished product).

The dynamics of the inventory level of finished products are described by a one-dimensional ordinary differential equation:

$$\dot{x}\left(t\right)=u\left(t\right)-d$$

(1)

where $x\left(0\right)=x_0$, and $x_0$ is the given initial stock level. If $x\left(t\right)\ge 0$, the system maintains an inventory; when there is less, there is a backlog.

Regarding the evaluation of emissions, as in [27], the linear model can be employed to describe the correlation between the machine’s production rate, $u\left(t\right)$, and its emission rate, $\dot{\mathrm{e}}\left(t\right)$. The cumulative emission of the machine at time t is determined by solving the following differential equation:

$$\dot{e}\left(t\right)={\theta }_{0}u\left(t\right)$$

(2)

where $e\left(0\right)=e_0$, $e\left(T^{+}\right)=e\left(T^{-}\right)$, where $e_0$ stands for the initial emission and ${\mathsf{\theta }}_0$ is the index emission. $T^{+}$and $T^{-}$ stand for the last repair and operation times, respectively.

At each time t, the finite-state homogeneous Markov process, $\xi \left(t\right)$, describes the behaviour of the considered system, taking values in $A=\left\{\mathrm{1,2}\right\}$, such that:

$$\xi \left(t\right)=\left\{\begin{array}{l}1\hspace{0.33em}\mathrm{i}\mathrm{f}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{m}\mathrm{a}\mathrm{c}\mathrm{h}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{i}\mathrm{s}\mathrm{o}\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l}\\ 2\hspace{0.33em}\mathrm{i}\mathrm{f}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{m}\mathrm{a}\mathrm{c}\mathrm{h}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{i}\mathrm{s}\mathrm{u}\mathrm{n}\mathrm{d}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{o}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{r}\mathrm{e}\mathrm{p}\mathrm{a}\mathrm{i}\mathrm{r}\mathrm{s}\end{array}\right.$$

The transition probabilities are defined by:

$$P\left[\xi \left(t+\delta t\right)=\beta |\xi \left(t\right)=\alpha \right]=\left\{\begin{array}{c}{q}_{\alpha \beta }\left(.\right)\delta t+0\left(t\right)if\alpha \ne \beta \\ 1+q_{\alpha \beta }\left(.\right)\delta t+0\left(t\right)if\alpha =\beta \end{array}\right\}$$

(3)

where $q_{\alpha \beta }$ denotes the transition rate from mode $\alpha$ to mode $\beta$, with$q_{\alpha \beta }\ge 0\left(\alpha \ne \beta \right)q_{\alpha \alpha }=-\sum _{\alpha \ne \beta }{q}_{\alpha \beta }$, $\alpha ,\beta ϵA$, and $\underset{\mathit{\delta t}\to 0}{\lim }\frac{o\left(t\right)}{\delta t}=0$.

Over an infinite horizon, the machine may randomly switch between the two modes. Hence, the stochastic process considered is defined by a $2\times 2$ transition rate matrix, $Q=\left[q_{\alpha \beta }\right]$, given by the following equation:

$$Q=\left(\begin{array}{cc}-q_{12}& q_{12}\\ q_{21}& -q_{21}\end{array}\right)$$

(4)

The limiting probabilities of the mode $i$, $iϵ\left\{1,2\right\}$, are the steady-state solutions of the forward Kolmogorov equations:

$$\dot{\pi }=\pi \times Q$$

(5)

where ${\pi }_1+{\pi }_2=1.$ To ensure demand fulfilment over an infinite horizon, the following feasibility condition of the manufacturing system must be verified:

$${\pi }_1\times u_{\max }>d$$

(6)

where ${\pi }_1$ represents the operational mode’s limiting probability.

The following equation describes the instantaneous cost, which includes backlog, inventory, and environmental costs, given by:

$$g\left(x,e,\alpha \right)=c^{+}{x}^{+}+{\mathrm{c}}^{-}{x}^{-}+{\mathrm{c}}_e\left(e\right)$$

(7)

where constants $c^{+}$and $c^{-}$ are costs per unit of time to penalize parts’ inventory and backlog, respectively, with $x^{+}$= max (0, x), and $x^{-}$ = max (0, −x).

Referring to the carbon tax regulation, if the cumulative quantity of emissions during a given period surpasses a standard limit, L, set by pertinent authorities, the surplus quantity incurs an environmental cost, represented by:

$$c_e\left(e\right)=c^e\times max\left\{0,\left(e\left(t\right)-L\right)\right\}$$

where $c^e$ is the penalty cost for emissions exceeding L.

This research aims to find the production rate, $u(·)$, that would minimize the expected discounted cost, $J\left(·\right)$, given by the following equation:

$$J\left(x,e,u,\alpha \right)=E\left[{{\displaystyle \int }}_0^{\infty }{e}^{-\rho t}\left[g\left(x,e,\alpha \right)\right]dt|x\left(0\right)=x,e\left(0\right)=e,\xi \left(0\right)=\alpha \right]$$

(8)

where $\rho$ is the discount rate, while $x$, $e$, and $\alpha$ are, respectively, the initial values of the state variables. The admissible decisions set, $\mathsf{\Gamma }\left(\alpha \right)$, which identifies the feasible production rate, $u(·)$, depends on the stochastic process (i.e., $\xi \left(t\right)=\alpha$) and is determined by:

$$\Gamma \left(\alpha \right)=\left\{\begin{array}{c}\left(u\right)\in \mathbb{R}|0\le u(\cdot )\le u_{max}Ind\left\{\alpha =1\right\}\end{array}\right\}$$

(9)

Let us define the value function $\upsilon (·)$ as the minimum of cost over $uϵ\mathsf{\Gamma }\left(\alpha \right)$, i.e.,

$$v\left(x,e,\alpha \right)=\underset{u\in \mathsf{\Gamma }\left(\alpha \right)}{min}J\left(x,e,u,\alpha \right)$$

(10)

the subsequent section, the properties of the value function, $\upsilon (·)$, are outlined by (10), establishing its satisfaction with the optimality conditions described by the Hamilton–Jacobi–Bellman (HJB)-type equations.

4. Optimality Conditions and Structure of Optimal Control Policy

The value function $v\left(·\right)$, given by (10), meets a set of coupled partial differential equations, called HJB equations, derived from the dynamic programming approach. The properties of such $v\left(·\right)$ and the method used for obtaining these equations can be found in [30], and references therein. Such equations describe the optimality conditions to be solved to determine the optimal production control policy for the considered planning problem. In a dynamic and stochastic context and concerning the optimality principle, the HJB equations can be expressed as follows:

$$\rho v\left(x,e,\alpha \right)=\underset{u\in \Gamma \left(\alpha \right)}{min}\left\{J\left(x,e,u,\alpha \right)+\frac{\partial v\left(\cdot \right)}{\partial x}\left(u-d\right)+\frac{\partial v\left(\cdot \right)}{\partial e}\left({\theta }_{0}u\right)+{\displaystyle \sum _{\beta \in A}}{q}_{\alpha \beta }v\left(x,{\phi }_e\left(\beta \right),e,\beta \right)\right\}$$

(11)

where $\alpha ϵA$, and ${\phi }_e\left(\xi \right)$ is given by:

$${\phi }_e\left(\xi \right)=\left\{\begin{array}{l}e\left({\tau }^{-}\right)\mathrm{if}\begin{array}{c}\xi \left({\tau }^{+}\right)=1and\xi \left({\tau }^{-}\right)=2\end{array}\\ e\left({\tau }^{-}\right)\mathrm{otherwise}\end{array}\right.$$

The reset function, ${\phi }_e\left(\xi \right)$, denotes the benefit of repairs and describes the emission discontinuity. The optimal solution, over Γ(α) of the right-hand side of (11), is the production rate, denoted as $u^{\mathrm{*}}\left(.\right).$ When the $\upsilon (·)$, described by (10), is available, the production rate can be determined as in Equation (11). However, deriving an analytical solution for (11) is nearly impossible. Solving the HJB equation (11) numerically poses an insurmountable challenge, which demonstrates the feasibility of addressing such issues by implementing Kushner’s method [31] within the production planning context. Numerical methods are used in Appendix A to develop the numerical version of the HJB equations. The obtained discrete HJB equations can be solved using successive approximation and policy improvement methods, as in [32]. The numerical algorithm used to solve the HJB equations for the production planning problem was implemented in MATLAB Version R2020b and executed on a computer with an Intel Xeon CPU at 3.50 GHz and 16 GB of RAM.

The computational domain, $D$, is defined by:

$$D=G_x^h\times G_e^h$$

where $G_x^h=\left\{x:-5\le x\le 120\right\}$; $G_e^h=\left\{e:0\le e\le 350\right\}$.

Table 2. Numerical example data.

${\mathit{c}}^{+}$	${\mathit{c}}^{-}$	${\mathit{c}}^{\mathit{e}}$	${\mathit{u}}_{\mathit{m}\mathit{a}\mathit{x}}$	$\mathit{d}$	$\mathit{L}$	${\mathit{\theta }}_0$	${\mathit{q}}_{21}^{-1}$	${\mathit{q}}_{12}^{-1}$	$\mathit{\rho }$
5	100	40	3.25	3	250	2	6	105	0.01

presents the parameters required for the numerical example, with values adapted from the literature. These values respect the feasibility condition stated by Equation (6).

The numerical results presented below allow us to characterize the structure of the obtained optimal control policy, $u^{\mathrm{*}}\left(x,e,\alpha \right)$ (see Figure 2), for small discretization steps on the associated state variables according to Theorem A1 in Appendix A. It is an environmental Hedging Point Policy improvement from the pioneering work [27]. This figure shows the manufacturing system’s production rate as a function of the stock and emissions in the operational mode. To better understand the interpretation of the production policy, the plan $(x,e)$ was divided into three zones: (1), (2), and (3), as illustrated in Figure 2b.

Due to the GHG emissions of the production system, the critical production threshold is a function of its emission level. This critical production threshold is defined by three thresholds, $Z_1,Z_2\left(e\right),\mathrm{a}\mathrm{n}\mathrm{d}{Z}_3$, given by Equations (12)–(14):

$$\left\{\begin{array}{l}if\hspace{0.33em}e<\hspace{0.33em}V:\\ u(x,e,1)=\left\{\begin{array}{l}{u1}_{max}\\ d\hspace{1em}if\hspace{0.33em}x=Z_1\\ 0\hspace{1em}if\hspace{0.33em}x>Z_1\end{array}\right.\end{array}\right.$$

(12)

$$\left\{\begin{array}{l}if\hspace{0.33em}V\le e<\hspace{0.33em}L:\\ u(x,e,1)=\left\{\begin{array}{l}{u2}_{max}\\ d\hspace{0.33em}if\hspace{0.33em}x\hspace{0.33em}=Z_2\left(e\right)\\ 0\hspace{0.33em}if\hspace{0.33em}x>Z_2\left(e\right)\end{array}\right.\end{array}\right.$$

(13)

$$\left\{\begin{array}{l}if\hspace{0.33em}e\ge \hspace{0.33em}L\hspace{0.33em}\hspace{0.33em}:\\ u(x,e,1)=\left\{\begin{array}{l}{u3}_{max}\\ d\hspace{0.33em}if\hspace{0.33em}x=Z_3\\ 0\hspace{0.33em}if\hspace{0.33em}x>Z_3\end{array}\right.\end{array}\right.$$

(14)

Depending on the value of the emission ($e)$, the obtained optimal production control policy follows distinct guidelines across three zones:

• The production must be halted ($u\left(t\right)=0$) if the current inventory level exceeds the critical threshold, $Z_i$.
• The production rate must be adjusted to the value of the demand rate when the current inventory level matches the critical $Z_i$.
• The production rate must be adjusted to its maximum value, $u_{max}$, if the current inventory is below the critical threshold, $Z_i$.

The symbol $V$, called the voluntary emission limit, is highlighted in Figure 2b to illustrate the emission level to switch from the critical threshold, $Z_1$, to the lower boundary of the critical threshold, $Z_2\left(e\right)$, and $L$, the emission standard limit to switch from the upper boundary of the critical threshold, $Z_2\left(e\right)$, to the critical threshold, $Z_3$.

In the literature, the closest proposed policies based on [27] are characterized by two constant critical thresholds. According to their work, if emission levels surpass a voluntary threshold and the stock level is deemed sufficient, the critical threshold would drop from $Z_1$ to $Z_2$ ($Z_1>Z_2$). The obtained results indicate that the optimal control policy is defined by a GHG emission-based threshold value specified in three zones: $Z_1,Z_2\left(e\right),\mathrm{a}\mathrm{n}\mathrm{d}{Z}_3.$ The policy recommends that the manager decreases production when approaching the limit, $L$, to anticipate and mitigate excess inventory and emission costs. The proposed model is confirmed in the following section, and the optimal control policy structure is obtained through sensitivity analysis.

5. Sensitivity Analysis

To validate the obtained control policy structure depicted in Figure 2, extensive sensitivity analysis was conducted using varying system and cost parameters. The behavior of the critical production thresholds, $Z_i\left(i=1,2,3\right)$, was analyzed depending on the emission level and the voluntary emission limit, $V,$ by varying the following parameters: shortage cost, inventory cost, emission cost, and emission index. The three points, $V_1,V_2,$ and $V_3$, represent the small, medium, and high values of the parameters observed in the sensitivity analysis. The analyses were performed utilizing the numerical example presented in the previous section as the basis of the study (basic case).

5.1. Effect of Shortage Cost

Figure 3 illustrates the results for three distinct values of the shortage cost, $c^{-}(75,100,\mathrm{a}\mathrm{n}\mathrm{d}125)$. It shows that when $c^{-}$ increased, the control policy recommended increasing the optimal production thresholds, Z. The opposite phenomenon was observed regarding the variation in V. Indeed, when the value of $c^{-}$ increased, a higher value of $Z$ was adopted, a scenario that, therefore, resulted in more emissions. To avoid prematurely reaching the emission limit, $L$, the policy recommended reducing the value of $V$ with the increase in $c^{-}$, namely, $V_3<V_2<V_1$, leading to a decrease in the production rate.

5.2. Effect of Inventory Cost

Figure 4 illustrates the results for three distinct values of inventory cost, $c^{+}(3,5,\mathrm{a}\mathrm{n}\mathrm{d}8)$. It shows that when $c^{+}$ increased, the policy recommended decreasing the optimal production threshold, $Z$. Regarding the variation in $V$, the opposite phenomenon as that of $Z$ was observed. Indeed, when the value of $c^{+}$ increased, a lower value of $Z$ was adopted, a scenario that, therefore, resulted in less emission. This behavior reduces the risk of prematurely reaching the emission limit, $L.$ The policy recommended increasing the value of $V$ with the increase in $c^{+}$, namely, $V_1<V_2<V_3$, leading to an increase in the production rate.

5.3. Effect of Emission Cost

Figure 5 illustrates the results for three distinct values of emission cost, $c^e(20,40,\mathrm{a}\mathrm{n}\mathrm{d}60)$. It shows that when $c^e$ increased, the policy recommended decreasing the optimal production threshold, $Z$. Regarding the variation in $V$, the same phenomenon as that of $Z$ was observed. Indeed, to avoid the excess costs of massive emissions after the emission limit, $L$, the policy recommended reducing the value of $V$ with the increase in $c^e$, namely, $V_3<V_2<V_1$, leading to a decrease in the production rate.

5.4. Effect of Emission Index

Figure 6 illustrates the results for three distinct values of the emission index, ${\theta }_0(1.5,2,\mathrm{a}\mathrm{n}\mathrm{d}2.5)$. In the absence of accurate data on the emissions index, a sensitivity analysis covering this range of possible values for this index could establish the generality of the approach and the robustness of the proposed control policy. It showed that when ${\theta }_0$ increased, the policy recommended decreasing the optimal production threshold, $Z$. Regarding the variation in $V$, the same phenomenon as that of $Z$ was observed. Indeed, to avoid the excess costs of massive emissions after the emission limit, $L$, the policy recommended reducing the value of $V$ with the increase in ${\theta }_0$, leading to more emissions and the early reaching of the emission limit, namely, $V_3<V_2<V_1$, leading to a decrease in the production rate.

From Figure 2 and the sensitivity analysis illustrated in Figure 3, Figure 4, Figure 5 and Figure 6, one can conclude that the optimal control policy structure, governed by the control parameters $Z_1$, $Z_2\left(e\right)$, and $Z_3$, and described by Equations (12)–(14) at the operational mode, held. In addition, the sensitivity analysis results make sense, which validates the proposed resolution approach’s robustness.

In the next section, a comparative study is conducted, highlighting the economic advantages of the proposed optimal control policy over those adapted from the literature.

6. Comparative Study

This section aims to assess the economic advantage of the developed optimal control policy compared to other policies found in the literature. To do this, the performance (total incurred cost) when the system is controlled by the developed optimal production policy (Policy I) is compared to the performance of those adapted from the literature.

• Policy II: This policy is that of the work presented in [27] and is characterized by two constant production thresholds, $Z_1^{BS}$ and $Z_2^{BS}$, and a voluntary emission limit, $V^{BS}$, for changing the production threshold. Thus, this two-threshold policy, given by Equations (15) and (16), can be seen as a simplified version of the developed policy, referred to as Policy I:

  $$\left\{\begin{array}{l}if\hspace{0.33em}e\le \hspace{0.33em}{V}^{BS}\\ u(x,e)=\left\{\begin{array}{l}{u}_{max}\hspace{1em}if\hspace{0.33em}x<{Z_1}^{BS}\\ d\hspace{1em}if\hspace{0.33em}x={Z_1}^{BS}\\ 0\hspace{1em}if\hspace{0.33em}x>{Z_1}^{BS}\end{array}\right.\end{array}\right.$$

  (15)

  $$\left\{\begin{array}{l}if\hspace{0.33em}e>\hspace{0.33em}{V}^{BS}:\\ u(x,e)=\left\{\begin{array}{l}{u}_{max}\hspace{1em}if\hspace{0.33em}x<{Z_2}^{BS}\\ d\hspace{1em}if\hspace{0.33em}x={Z_2}^{BS}\\ 0\hspace{1em}if\hspace{0.33em}x>{Z_2}^{BS}\end{array}\right.\end{array}\right.$$

  (16)
• Policy III: This policy is that of [9], characterized by one constant production threshold, $Z^{AK}$. Thus, this policy, obtained without considering emissions, is given by Equation (17):

  $$u(x)=\left\{\begin{array}{l}{u}_{max}\hspace{1em}if\hspace{0.33em}x<Z^{AK}\\ d\hspace{1em}if\hspace{0.33em}x=Z^{AK}\\ 0\hspace{1em}if\hspace{0.33em}x>Z^{AK}\end{array}\right.$$

  (17)

The comparative analysis of the developed optimal control Policy I versus the policies adapted from existing literature (namely, Policy II and Policy III) was performed while varying the shortage cost, $c^{-}$, the inventory cost, $c^{+},$and the emission cost, $c^e$. The economic performance (total incurred cost given by the value function) of Policy I was compared to that of the other policies (i.e., Policy II and Policy III).

6.1. Comparison of Policies While Varying Backlog Unit Cost

The critical thresholds for Policies I, II, and III, along with their total incurred costs for different values of the backlog unit cost, $c^{-}$, are presented in

Table 3. Policies’ performances for different values of the shortage cost $c^{-}$.

${\mathit{c}}^{-}$	50	100	200	300	400
$Z_1(P$olicy-I)	47	63	81	91	99
$Z_{2max}$ (Policy-I)	47	63	81	91	99
$Z_{2min}$ (Policy-I)	41	57	73	83	91
$Z_3$ (Policy-I)	3	17	35	45	51
$Z_1^{BS}$ (Policy-II)	33	55	69	77	85
$Z_2^{BS}$ (Policy-II)	1	15	33	43	51
$Z^{AK}$ (Policy-III)	55	71	89	99	109
Total incurred cost (Policy-I)	83,946	84,051	84,287	84,488	84,669
Total incurred cost (Policy-II)	86,434	87,367	88,193	88,675	89,024
Total incurred cost (Policy-III)	88,603	88,975	89,369	89,577	89,763

. The lower boundary of the critical threshold, $Z_2\left(e\right)$, and the upper boundary of the critical threshold, $Z_2\left(e\right)$, are represented by $Z_{2min}$ and $Z_{2max}$, respectively.

From Table 3, when the shortage unit cost increased, the corresponding critical production thresholds increased (the system stored more) to reduce the shortage. The total incurred costs of the three policies (Policy I, Policy II, and Policy III) increased due to the increasing storage and emission costs. The more the system stores, the more it produces with the maximum production rates, which generates more emissions. Consequently, the inventory and the emission costs increased, and so did the total incurred cost.

From Table 3 and Figure 7, Policy I had the lowest total incurred cost, compared to that of Policies II and III for a wide range of backlog costs. Policy II stored less than Policy I, so it underwent more backlog costs. At the same time, since Policy III stored more than Policy I, its system produced more with maximum production rates and generated more emissions, which increased the total cost. This explains the superiority of Policy I over Policy II and Policy III in terms of total incurred costs.

From Figure 7, the gap between the total incurred cost of the three policies increased as the backlog unit cost $c^{-}$ increased. In fact, as $c^{-}$ increased, the critical production thresholds increased. Consequently, the storage cost of Policy II increased, and the emission cost of Policy III increased more rapidly compared to that of Policy I.

6.2. Comparison of Policies While Varying Inventory Unit Cost

The critical thresholds for Policies I, II, and III, along with their total incurred costs for different values of the inventory unit cost, $c^{+}$, are presented in

Table 4. Policies’ performances for different values of the inventory costs $c^{+}$.

${\mathit{c}}^{+}$	1	2	3	4	5
$Z_1$ (Policy-I)	81	65	57	51	45
$Z_{2max}$ (Policy-I)	81	65	57	51	45
$Z_{2min}$ (Policy-I)	73	61	53	47	41
$Z_3$ (Policy-I)	35	31	29	27	25
$Z_1^{BS}$ (Policy-II)	69	55	45	45	37
$Z_2^{BS}$ (Policy-II)	33	29	27	25	23
$Z^{AK}(\mathrm{P}$olicy-III)	89	71	61	55	49
Total incurred cost (Policy-I)	84,287	84,519	84,677	84,801	84,924
Total incurred cost (Policy-II)	88,193	88,279	88,344	88,405	88,425
Total incurred cost (Policy-III)	89,369	89,244	89,195	89,189	89,162

. The lower boundary of the critical threshold, $Z_2\left(e\right)$, and the upper boundary of the critical threshold, $Z_2\left(e\right)$, are represented by $Z_{2min}$ and $Z_{2max}$, respectively.

Table 4 shows that when the inventory unit cost, $c^{+}$, increased, the corresponding critical production thresholds decreased (the system stored less) to reduce the inventory cost. The total incurred costs of Policies I and II increased due to the increasing backlog costs. In fact, the less the system stores, the more it undergoes shortage and the less it produces with the maximum production rates, which generates less emissions. In the case of Policies I and II, the increase in shortage cost was more important than the decrease in emission costs. That is why the total incurred cost increased. In Policy III’s case, the increase in the shortage cost was less significant than the decrease in emission costs. That is why the total incurred cost decreased.

From Table 4 and Figure 8, Policy I had the lowest total incurred cost compared to that of Policies II and III for a wide range of the inventory unit cost, $c^{+}$. In fact, Policy II stored less than Policy I, so it incurred more shortage costs. At the same time, since Policy III stored more than Policy I, its system produced more with maximum production rates and generated more emissions. This explains the superiority of Policy I over Policy II and Policy III in terms of total incurred costs.

From Figure 8, the gap between the total incurred costs of the three policies decreased as the inventory unit cost, $c^{+}$, increased. In fact, as $c^{+}$ increased, the critical production thresholds decreased. Consequently, the emission costs of the three policies decreased. Reducing the emission cost is the main advantage of Policy I compared to Policies II and III and of Policy II compared to Policy III. When this advantage was reduced, the gap between the total incurred costs of the three policies was reduced.

6.3. Comparison of Policies While Varying Emission Unit Cost

The critical thresholds for Policies I, II, and III, along with their total incurred costs for different values of the emission unit cost, $c^e$, are presented in

Table 5. Policies’ performances for different values of the emission cost, $c^e$.

${\mathit{c}}^{\mathit{e}}$	5	10	20	40	60
$Z_1$ (Policy-I)	87	85	83	81	79
$Z_{2max}$ (Policy-I)	87	85	83	81	79
$Z_{2min}$ (Policy-I)	85	81	77	73	71
$Z_3$ (Policy-I)	73	63	49	35	25
$Z_1^{BS}$ (Policy-II)	77	73	73	69	69
$Z_2^{BS}$ (Policy-II)	71	61	47	33	23
$Z^{AK}$ (Policy-III)	89	89	89	89	89
Total incurred cost (Policy-I)	7842	18,335	40,083	84,287	128,777
Total incurred cost (Policy-II)	9850	21,078	43,461	88,193	132,856
Total incurred cost (Policy-III)	9983	21,351	44,029	89,369	134,710

. The lower boundary of the critical threshold, $Z_2\left(e\right)$, and the upper boundary of the critical threshold, $Z_2\left(e\right)$, are represented by $Z_{2min}$ and $Z_{2max}$, respectively.

From Table 5, when the emission unit cost, $c^e$, increased, the corresponding critical production thresholds for Policies I and II decreased. The system stored less to reduce emission costs, which increased backlog costs. At the same time, the threshold of Policy III remained unchanged since this policy was not designed to account for emission costs. The total incurred costs of the three policies (Policy I, Policy II, and Policy III) increased due to the rising backlog and emission costs for Policies I and II, as well as the increasing emission costs for Policy III. In fact, the less the system stores, the more it incurs backlog costs and the less it produces at maximum production rates, thereby generating fewer emissions. For Policies I and II, the increase in shortage costs, coupled with the rise in emission costs (as $c^e$ increased), significantly increased the total incurred costs.

From Table 5 and Figure 9a, it is evident that Policy I had the lowest total incurred cost compared to Policies II and III over a wide range of emission unit costs, $c^e$. Policy II stored less than Policy I, leading to higher backlog costs. At the same time, since Policy III stored more than Policy I, its system consequently produced more at maximum production rates and generated more emissions, significantly increasing its emission costs. This explains the superiority of Policy I over Policies II and III in terms of total incurred costs.

From Figure 9b, the disparity in total incurred costs among the three policies expanded as the emission unit cost, $c^e$, increased. In fact, as $c^e$ increased, the critical production thresholds decreased. Consequently, the backlog cost of Policy II increased, and the emission cost of Policy III increased more rapidly compared to that of Policy I. Reducing the emission cost is the main advantage of Policy I compared to Policies II and III and of Policy II compared to Policy III. When this advantage was increased (as $c^e$ increased), the gaps between the total incurred costs of the three policies increased.

Finally, the comparative study confirmed that using the developed Policy I to control unreliable manufacturing systems generating greenhouse gas (GHG) emissions and evolving in a dynamic and stochastic context resulted in improved economic performance compared to existing control policies. This was attributed to its enhanced ability to balance shortages, inventory, and emission costs.

7. Discussion

Successfully implementing the developed control policy in business operations relies on maintaining comprehensive information about the state of the manufacturing system. Managers can effectively utilize the developed policy to control the system by monitoring the stock level and greenhouse gas emissions. The implementation of the obtained production policy is further facilitated by using a logical implementation diagram, which guides decision-making. Considering the operational status of the machine and its anticipation of the following breakdown with a stock level $\left(x\right)$ and an emission level $\left(e\right)$, the production rate can be easily defined in three stages. The first is to build the safety stock to avoid the unavailability of finished products due to breakdowns and repairs. The second stage involves deciding when to reduce stock in accordance with voluntary emission limits, while the final stage determines when to further decrease the safety stock if excess emission costs become prohibitively high. These stages are represented in intervals and delimited in zones, as follows: $e<V$, $V\le e<L$, and $e\ge L$ (Equations (12)–(14)), and zone 1, zone 2, and zone 3, respectively (Figure 2b). Since the second zone of the developed optimal policy is defined by a GHG emission-based threshold value, given by $Z_2\left(e\right)$, it is challenging to implement in practice, and a target value of $Z_2\left(e\right)$ is determined to obtain the best approximation of the theoretical and optimal control policy. The control policy to be implemented is presented in Figure 10, taking the minimum of $Z_2\left(e\right)$ as the critical threshold in zone 2 (i.e., $Z_2\left(e\right)=Z_{2min}$). It approximates the optimal policy well, with a cost difference of 0.36%. The results obtained from the numerical values of the optimal parameters of the basic case (Table 2) to be used in decision-making to facilitate the implementation of the proposed policy are summarized as $\mathrm{V}=46,\mathrm{L}=250,{\mathrm{Z}}_1=27.5,{\mathrm{Z}}_2=24.5,\mathrm{and}{\mathrm{Z}}_3=8.5.$

Figure 10 depicts the logic chart for implementing the proposed control policy. Initially, the decision-maker needs to monitor both the stock and emission levels. If emission levels are lower than the voluntary threshold (46), the production machine should reach the safety stock level (27.5 units) by producing at maximum rate (3.25). This will reduce the risk of backlog and carbon emissions. The decision-maker should adjust the production rate to meet demand (3) if the safety stock is built. Subsequently, if the inventory surpasses the stock threshold, production should be halted. When emission levels are between the voluntary limit (46) and the standard limit emission (250; respectively, is higher than the standard limit), the system should follow the same rules of production with the safety inventory level (24.5 units; respectively, 8.5 units).

With reasonable assumptions, the findings of this paper hold potential benefits for various industrial sectors, including mining, material handling equipment, construction machinery manufacturing plants, rail and aircraft assembly lines, and the automotive industry. Possible extensions of this work could integrate maintenance to the model with progressively deteriorating manufacturing systems and the rising emission index due to degradation phenomena under minimal repairs after failures.

8. Conclusions

This paper discussed the development of an optimal production control policy for a manufacturing system subject to random failures and repairs, with a focus on controlling greenhouse gas (GHG) emissions. The optimal control policy was derived using a stochastic dynamic programming approach, specifically through solving Hamilton–Jacobi–Bellman (HJB)-type equations. This control policy, termed the Environmental Hedging Point Policy (EHPP), relies on both emission and inventory levels to determine optimal production decisions. Numerical solutions of the HJB equations indicated that optimal production rates adjusted dynamically to balance inventory costs and emission penalties. The sensitivity analysis showed the robustness of the EHPP under various parameter changes, validating the model’s adaptability and reliability. Differences in the data arose from variations in key parameters, such as failure rates, repair times, and emission costs, demonstrating the policy’s flexibility in diverse scenarios.

The comparative study highlighted the novelty and superiority of the EHPP over existing policies, which often lack integrated emission control mechanisms. The EHPP ensured production efficiency and environmental compliance, a feature absent in traditional models. Using stochastic dynamic programming, particularly through the solution of HJB equations, provided a systematic approach to handling uncertainties in system failures and repairs. This method ensures that the optimal policy adapts to real-time changes in the manufacturing environment, significantly improving the outcomes compared to static control policies.

The significant reduction in GHG emissions demonstrated the feasibility of integrating environmental considerations into manufacturing control systems without compromising production efficiency. The proposed policy offers practical guidelines for managers, facilitating the implementation of environmentally conscious production strategies. The obtained EHPP optimally balances production efficiency and emission control, providing a robust, adaptable, and environmentally sustainable solution for manufacturing systems. This research advances the field by integrating stochastic dynamic programming with environmental considerations into production control.

Future research should incorporate advanced artificial intelligence (AI) and machine learning (ML) techniques to optimize production control policies. These technologies can enhance predictive maintenance, optimize scheduling, and improve decision-making processes by analyzing extensive real-time data from manufacturing systems. Intelligent scheduling and AI can help unlock energy efficiency from the equipment level to the entire supply chain.