Petroleum Reservoir Control Optimization with the Use of the Auto-Adaptive Decision Trees

Abstract

The global increase in energy demand and the decreasing number of newly discovered hydrocarbon reservoirs caused by the relatively low oil price means that it is crucial to exploit existing reservoirs as efficiently as possible. Optimization of the reservoir control may increase the technical and economic efficiency of the production. In this paper, a novel algorithm that automatically determines the intelligent control maximizing the NPV of a given production process was developed. The idea is to build an auto-adaptive parameterized decision tree that replaces the arbitrarily selected limit values for the selected attributes of the decision tree with parameters. To select the optimal values of the decision tree parameters, an AI-based optimization tool called SMAC (Sequential Model-based Algorithm Configuration) was used. In each iteration, the generated control sequence is introduced into the reservoir simulator to compute the NVP, which is then utilized by the SMAC tool to vary the limit values to generate a better control sequence, which leads to an improved NPV. A new tool connecting the parameterized decision tree with the reservoir simulator and the optimization tool was developed. Its application on a simulation model of a real reservoir for which the CCS-EOR process was considered allowed oil production to be increased by 3.5% during the CO₂-EOR phase, reducing the amount of carbon dioxide injected at that time by 16%. Hence, the created tool allowed revenue to be increased by 49%.

1. Introduction

Due to the reduction of hydrocarbon resources caused by increasing consumption as well as difficulties in discovering new hydrocarbon reservoirs, more and more research is focused on the effective exploitation of existing reservoirs. For this purpose, the location and construction of new production and injection wells are optimized. The number of open perforations, the degree of opening of downhole valves, or the working mode of already existing wells are also optimized. This paper focuses on the optimization of the reservoir control to achieve a higher recovery factor and an additional income without any expenditures, as only the control scheme changes.

Optimization is to find a solution to a specific problem that is the best in terms of a specific quality indicator [1]. The decision space $Z$ includes a set of feasible solutions $Z_d$ constituting the set of all vectors of decision variables $\mathit{z} \in Z$ that satisfy given constraints. The quality indicator $J: Z_d\to R$ assigns a numerical value to a feasible solution $\mathit{z} \in Z_d$, making it possible to compare the quality of solutions. In the case of single-criteria optimization, an ordering relation can be defined. The problem of the single-criteria optimization is to find the optimal solution, i.e., such a feasible solution $\widehat{\mathit{z}} \in Z_d$ that:

$$\forall \mathit{z}\in Z_d J\left(\widehat{\mathit{z}}\right)\le J\left(\mathit{z}\right)$$

(1)

The formulation of the optimization problem presented in Equation (1) also allows the maximization problem to be solved, as it can be reduced to the presented minimization problem [1]:

$$\underset{\mathit{z}}{\max }J\left(\mathit{z}\right)=-\underset{\mathit{z}}{\min }\left[-J\left(\mathit{z}\right)\right]$$

(2)

Depending on the type of solution and the properties of the quality indicator, the optimization problem can be divided into static and dynamic optimization. In the case of static optimization, the quality indicator is a function. Static optimization aims to find the optimal operating point, i.e., the operating point for which the objective function is extreme [1]. The dynamic optimization aims to find the optimal control $\widehat{\mathit{u}}\in U$ belonging to the set of feasible controls $U_d=\left\{\mathit{u}:\left[0,T\right]\to U\right\}$, where $U\subset R^l$ is a set of feasible control values [2]. Such optimal control minimizes the quality indicator defined as a functional (its argument is a function and a result is a number) [1]:

$$J\left(\widehat{\mathit{u}}\right)=\underset{\mathit{u}}{\min }J\left(\mathit{u}\right)$$

(3)

with constraints resulting from the system state equations:

$$\frac{d\mathit{x}}{dt}=f\left(\mathit{x},\mathit{u},t\right)$$

(4)

and state constraints $\mathit{x}\in R^n$, time $t\in \left[0,T\right]$, and control constraints $\mathit{u}\in U\subset R^l$.

To determine the optimal production control, it is necessary to define the optimal control problem, which is generally based on the following dynamics model:

$$\frac{d\mathit{x}\left(t\right)}{dt}=f\left(\mathit{x}\left(t\right),\mathit{u}\left(t\right),t\right)\wedge \mathit{x}\left(0\right)=x_0$$

(5)

where $\mathit{x}\left(t\right)$ is a vector of state variables, $\mathit{u}\left(t\right)$ represents control belonging to the set of feasible control values $U$, and time belongs to the interval $\left[0,T\right]$. The typical optimal control problem is to select control $\mathit{u}\left(t\right)$, which minimizes the following functional [3,4]:

$$J\left(\mathit{u}\left(t\right)\right)=\mathsf{\Phi }\left[\mathit{x}\left(t_k\right),t_k\right]+{\displaystyle \underset{t_0}{\overset{t_k}{\int }}\mathsf{\Psi }\left[\mathit{x}\left(t\right),\mathit{u}\left(t\right),t\right]dt}$$

(6)

treated as the cost of evolution, where $\mathsf{\Phi }$ is a cost function related to the final state of evolution $x\left(t_k\right)$, and the integral represents the cost resulting from the selected control $u$ and the state trajectory generated by this control [4].

The hydrocarbon reservoir is a complicated structure with complex geometry and spatial heterogeneity of geological parameters. It is filled with water, numerous hydrocarbon components, and other substances. Therefore, the description of the dynamics of the oil and gas reservoir is complex. Reservoir simulations are used for mathematical modelling of the processes taking place in the reservoir. To optimize production from the reservoir it is needed to predict the future behavior of the reservoir. Hence, it is necessary to build a simulation model that maps the reservoir geometry, the reservoir fluids, and the phenomena taking place in the reservoir [5]. For the analysis of multiphase flow in oil and gas reservoirs, two main types of mathematical models are currently used: the “black oil” model and the compositional model [6]. The dynamics of the hydrocarbon reservoir are given by the equation [7]:

$$\frac{d\mathit{x}}{dt}=f\left(\mathit{x},\nabla \mathit{x}, {\nabla }^2\mathit{x},\mathit{u}\right)$$

(7)

where the exact form of the state vector $\mathit{x}$ and the function $f$ depend on the mathematical model used to describe the reservoir.

The “black oil” model assumes the presence of three homogeneous phases with a constant chemical composition. The properties of phases depend only on pressure due to the assumed temperature consistency [8]. The model takes into account water and two pseudo hydrocarbon components: oil and gas, which are immiscible with water [9]. The advantage of the “black oil” model is its ease of use and relatively short computation time. Therefore, the “black oil” model is widely used in industrial practice, as it is applicable to most oil and gas fields. It is mainly used to describe the primary production of oil and gas reservoirs. It is also used to model water and gas injection, as long as the injected fluids do not differ significantly from the fluids originally present in the reservoir.

The compositional model also assumes the presence of water and two hydrocarbon phases, but their chemical composition is determined by the mole fractions of individual components [9]. The most common are from 6 to 12 components, including hydrocarbon components (C1, C2, etc.) and N₂, H₂S, CO₂ [9]. The compositional model assumes that the composition of the phases changes according to the state equations depending on the pressure and temperature [9]. Hence, the compositional model realistically models the behavior of the fluids. However, it requires a large computational effort. The compositional model is applicable to gas-condensate reservoirs, problems of mixing of reservoir fluids, and other complex problems that cannot be described with the “black oil” model [6]. An example of an advanced problem requiring the use of the compositional model is the CCS-EOR process, considered in this paper. Due to the injection of CO₂ into the reservoir, it is impossible to use the “black-oil” model. Moreover, to model the mixing of carbon dioxide with oil, it is necessary to take into account the molar fractions of the individual components of the reservoir fluids.

The main goal of optimal reservoir control is to define a production strategy that maximizes or minimizes a specific criterion, subject to the constraints imposed by the physical nature of the problem in the form of differential equations describing the dynamics of the system [7]. For most production processes, the quality indicator used to solve the optimization problem is the economic efficiency of the investment, i.e., the net present value (NPV). For well control optimization, the functional based on the NPV can be expressed as [10,11,12]:

$$J\left(\mathit{u}\right)={\displaystyle \underset{t_0}{\overset{t_k}{\int }}\left[\frac{q_z\left(t\right)c_z-q_k\left(t\right)c_k}{{\left(1+d\right)}^{\tau }}\right]}dt$$

(8)

where $d$ is the annual discount rate [/], and $\tau$ is the duration of the investment in years. The numerator stores the profits and costs associated with the production or injection of fluids. Their exact form depends on the production process under consideration. The flow rates of the fluids included in the profit are always positive, and the flow rates of the fluids generating costs are negative. The well control variable $u$ can be the flow rate of a particular reservoir fluid or the well bottom hole pressure. In practice, the integral form of a functional (8) is approximated by the sum of the following form [10,12]:

$$J\left(\mathit{u}\right)={\displaystyle \sum _{i=1}^M\frac{q_{zi}{c}_z-q_{ki}{c}_k}{{\left(1+d\right)}^{{\tau }_i}}\mathsf{∆}{t}_i}$$

(9)

where $M$ corresponds to the total number of time steps and ${\tau }_i$ is the period from the beginning of the investment to step $i$. In the discretized form of the NPV, each period $\mathsf{\Delta }t$ has a corresponding discrete fluids performance. Thus, the problem of optimal control of the oil and gas reservoir is to determine the control $u$, which maximizes the return on investment in the form of a functional described by Equation (9).

Classical optimization methods used in the optimal control theory are related to the calculus of variations dealing with the search for functional extremes [1]. Optimum control can be determined by the Pontryagin maximum principle describing the necessary conditions of optimality [2]. In line with this principle, the Hamilton function was introduced to solve the problem of optimal control for a dynamical system. The Hamilton function combines the state equations and the objective function. The Pontryagin maximum principle states that it is necessary for any optimal control and the optimal state trajectory to solve the Hamiltonian system and select the control and the corresponding trajectory that maximize the Hamiltonian. After solving a properly defined system of equations, the optimal process control can be determined as a function of time. Another possibility is to use dynamic programming. In this case, the optimal control is determined in an N-stage decision-making process that can be reduced to a series of N one-stage decision-making processes [1].

In most practical problems, the use of analytical methods to find the optimal control is impossible due to the non-linear nature of the problem or computational difficulties. Hence, numerical methods are used to solve such problems. In the case of optimal control, numerical methods can be divided into direct and indirect. In the indirect methods, the boundary conditions of the optimization problem are replaced with the necessary optimality conditions resulting from the maximum principle [2]. The disadvantage of indirect methods is the large amount of analytical work required to prepare for the calculations. On the other hand, direct methods approximate the control problem by the problem obtained by discretization of control and system dynamics equations [2]. These methods do not use the maximum principle and their advantage is versatility and simplicity. The overview of some important prior work presenting numerical methods used to solve the optimal control problem is given in [13].

One of the direct methods used to determine the optimal control of practical cases is the parameterization of the control function [4]. In this method, control is represented by a parameterized time function [14]:

$$\mathit{u}=\mathit{u}\left(t,\mathit{p}\right)$$

(10)

This method aims to select the optimal values of the parameters contained in the $p$ vector, i.e., values that minimize the quality indicator [3]. Another approach is a stepped approximation where the control time is divided by $N_d$ discretization points [2]:

$$0=t_1<t_2<\dots <t_{N_d}=t_k$$

(11)

This method assumes that the continuous control function can be approximated by functions that are constant in the assumed intervals [2]:

$$\mathit{u}\left(t\right)={\mathit{u}}_{\mathit{i}}\mathrm{for}t\in [t_i,t_{i+1}]$$

(12)

Then, the decision variables are values of $u_i$, the optimal values of which form a vector $\widehat{\mathit{u}}=\left[{\widehat{u}}_1,{\widehat{u}}_2,\dots ,{\widehat{u}}_{Nd} \right]$.

Most of the methods used for solving the problem of optimal control are impractical for optimizing the production of hydrocarbon reservoirs. In this case, it is impossible to functionally link the optimized quality indicator and decision variables. Optimizing reservoir production for a given vector of decision variables, a reservoir simulation is run. Quality indicator values are determined based on the simulation results. The use of the reservoir simulator to describe the dynamics of a reservoir limits the use of commonly known optimization methods.

The ECLIPSE reservoir simulator (Schlumberger Limited, Houston, TX, USA) has a built-in tool that allows a given problem to be optimized using gradient methods. However, in most cases, this tool gives unsatisfactory results due to numerical errors. Currently, there is a lot of research on the use of artificial intelligence methods to optimize various processes related to the production of hydrocarbon reservoirs. Intelligent control using artificial intelligence to determine system control [2] is an alternative to traditional optimization methods. Intelligent control methods include algorithms that learn from experience [2]. In the initial learning phase, the output depends on the randomly assigned control. Then, further control actions are generated based on the gained knowledge about the system. The learning process allows the quality of the solution to be improved and as a result, the best decision for a given criterion can be made [2]. In practice, there are many different approaches to intelligent control resulting from the variety of artificial intelligence methods and various possibilities of the learning process. A serious disadvantage of the solutions most commonly used in the literature, such as genetic algorithms or artificial neural networks, is the fact that they require thousands of simulations. Moreover, most of the research is carried out on synthetic, significantly simplified reservoir models. However, a real hydrocarbon reservoir model has a high degree of complexity resulting from the complex geological structure and heterogeneity of the reservoir. For such models, a single simulation takes from a few to several hours. Hence, these methods are no longer applicable due to their duration.

One such production process that needs to be modelled by the complex simulation model is the CCS-EOR process. In the CCS-EOR method, two successive processes are distinguished: CO₂-EOR and CCS [15]. The CO₂-EOR method is one of the enhanced oil recovery methods and it allows the total oil production from the reservoir to be increased by an additional 15–20% [16]. Carbon dioxide, after injection into the reservoir, chemically and physically interacts with rock and oil, allowing oil to be displaced from the pores and production to be increased [17]. The phenomena occurring at that time cause a decrease in oil density and viscosity, an increase in permeability, a reduction in the surface tension at the CO₂–oil and water–oil interface, and the evaporation of some of the oil components [17]. Hence, in the first phase of the CCS-EOR process, oil flowing into production wells due to injected carbon dioxide is produced (CO₂-EOR). When the oil production ceases to be profitable, production wells are closed and carbon dioxide is injected until the total storage capacity is reached for its geological sequestration (CCS). The technology known as Carbon Capture and Storage consists of capturing and long-term underground storage of CO₂ in geological structures. The process involves the separation of CO₂ from the exhaust gases from fossil fuel combustion processes, the transport of the separated carbon dioxide, and its injection into the geological structure for long-term storage [18]. The advantage of the CCS-EOR method is that it makes it possible to increase oil production as well as reduce carbon dioxide emissions. It allows a synergistic effect of environmental protection and increase of the oil recovery factor to be obtained [19]. In the CCS-EOR process, the costs of capturing and storing CO₂ can be compensated through revenues obtained during the EOR process [20]. Hence, the optimal control of such a process can increase revenue from the overall project. However, when storing carbon dioxide in geological structures, an important issue is the isolation of CO₂ to prevent it from escaping into undesirable rock formations. To model such a process, application of a complex simulation model that considers the process of mixing carbon dioxide with oil and allows the CO₂ flow to be monitored is required. Optimal control of such a process is not a trivial problem.

The aim of the work reported here was to develop a novel tool that automatically determines intelligent control of real hydrocarbon reservoirs maximizing the NPV of a given production process. The idea is to build an auto-adaptive parameterized decision tree that replaces the arbitrarily selected limit values for the selected attributes of the decision tree with parameters. To select the optimal values of the decision tree parameters, an AI-based optimization tool was used. A new procedure connecting the parameterized decision tree with the reservoir simulator and the optimization tool has been created. This work consists of five parts. The first part contains theoretical foundations and mathematical formulations of the optimization problem and control theory. The second part describes the dynamics model of a hydrocarbon reservoir, introduces the problem of hydrocarbon reservoir control, and presents the developed tool for automatic determination of the optimal control of the hydrocarbon reservoir. The next part contains an exemplary implementation of the developed methodology on the example of the CCS-EOR process. The fourth part contains the results obtained for the exemplary implementation of the created tool. The last part contains the conclusions resulting from this work.

2. Proposed Intelligent Reservoir Control Approach

2.1. Model of the Analyzed Problem

In this study, it was assumed that the problem of hydrocarbon reservoirs is to determine the control that maximizes the NPV value of a given production process. The control function of a well adopted in this paper is its flow rate:

$$\mathit{u}\left(t\right)=q\left(t\right)$$

(13)

and the limitation is its bottom hole pressure. Due to the practical limitations of production processes conducted in a real reservoir, the control cannot be changed in the continuous-time domain. Thus, control is constant over assumed intervals. Hence, the optimization aims to determine such a control vector that includes controls in the successive periods:

$$\widehat{\mathit{u}}=\left[{\widehat{u}}_1,{\widehat{u}}_2,\dots ,{\widehat{u}}_M \right]$$

(14)

where $M$ is the number of time steps [21].

Control of a given process, in a global aspect, can be considered as a strategy that depends on the control at each time step of an assumed length. Therefore, the problem of production process control can be decomposed into sub-problems. To solve them, it is enough to model the control scheme that determines the decision to be made in a single time step.

Such a problem may be modelled by a decision tree (Figure 1). Tree nodes represent attributes characterizing the analyzed process; for example, the average reservoir pressure or the reservoir performance. The branches connecting the nodes represent the limit values corresponding to individual attributes. The leaves (nodes that have no additional nodes coming off them) determine the reservoir control indicating actions that should be taken in a given time step. The path from the root (the top of the first/upper level) to a given leaf is a conjunction of the conditions that must be met for a given decision to be selected. The knowledge of reservoir dynamics related to the conducted experiments and engineering knowledge are used to formulate the conditional rules for reservoir control.

The problem with this approach is that the limit values for the decision tree attributes are most often selected arbitrarily based on experience. It does not guarantee an optimal decision. The original solution proposed in this paper is to build an auto-adaptive parameterized decision tree that replaces the arbitrarily selected values with parameters determined in the optimization process using artificial intelligence methods. Such a parameterized decision tree can be modified and adjusted to a specific production process. Engineering knowledge is needed to design a tree structure suitable for a particular problem. Then, artificial intelligence is applied to select the best limit values of attributes. The combination of engineering knowledge with artificial intelligence allows the quality of decisions made to be improved and the economic efficiency of the considered processes to be increased.

2.2. Applied Optimization Tool

Having a mathematical model of the analyzed decision problem, it is possible to use machine learning. It is a field of artificial intelligence considering algorithms that improve their operation along with experience gained. The purpose of machine learning is to determine the parameters of an appropriate model. Thanks to the use of machine learning, it is possible to automatically improve the control quality.

Due to the limitations of artificial intelligence methods used in the literature to optimize the reservoirs control, in this paper an innovative optimization tool in the field of machine learning, which is intended for problems with a high computational cost, is used. The SMAC (Sequential Model-based Algorithm Configuration) tool enables the selection of optimal values of the parameters of complex algorithms treated as a “black box” after performing only twice as many iterations as the number of unknowns [22]. Such high-efficiency results from the fact that the tool learns from previous results which parameter values bring more favorable results. First, the tool collects data from the initial realizations, and then iteratively performs the following three steps within the learning process [23]:

• Based on the data collected so far, it builds a search space model,
• It uses the determined model to select parameter values that allow the greatest improvement in solution quality described by a so-called expected improvement criterion. It is high for those parameters that are expected to be good and those that are located in the area that has not been tested so far,
• It calculates the value of the quality indicator using the target algorithm, creating a new instance that is then added to the dataset.

Before running the SMAC tool, it is necessary to specify the parameter domains and their initial values, as they are decision variables. Then, for the next sets of parameter values determined with the procedure presented above, the configured target algorithm is run to solve subsequent instances of the problem. These instances are cases of the considered problem with specific values of the input data. The target algorithm is a procedure solving the considered problem and the parameters of this target algorithm are optimized. With the use of this target algorithm, the quality indicator, which is used to determine subsequent parameter values, is calculated. As a result of the SMAC tool application, such values of the parameters of a given algorithm are determined, which minimize the specified quality indicator [23]. The principle of the SMAC optimization tool is presented in Figure 2.

2.3. Principle of the Proposed Solution Operation

The proposed solution of intelligent reservoir control relies on determining reservoir control (at each time step of the simulation) based on an auto-adaptive parameterized decision tree. Its parameters values are determined using the SMAC tool, which optimizes the quality indicator in the form of NPV of the project. The SMAC tool matches perfectly with the considered problem of reservoir control due to the low computational cost of this tool. The originality of the proposed solution is confirmed by the fact that the SMAC tool has not been widely used in the oil and gas industry so far. Therefore, it was necessary to develop a new procedure enabling the connection of a parameterized decision tree with the ECLIPSE reservoir simulator (Schlumberger Limited, Houston, TX, USA) and with the SMAC optimization tool.

Internal programming in the ECLIPSE reservoir simulator (Schlumberger Limited, Houston, TX, USA) made it possible to directly declare a parameterized decision tree within the simulation. It also made it possible to implement the function calculating the quality indicator value that is used to compare the obtained solutions. The combination of the reservoir simulator and the optimization tool has been implemented in the Python programming language. Hence, the developed tool enables full automation of the optimal reservoir control determination.

In practice, the use of the developed tool consists of running the SMAC tool, which at each iteration calls the program written in the Python programming language. This program downloads successive parameter values from the SMAC tool ${\mathit{p}}_i=\left[p{1}_i, p{2}_i,\dots ,p{N}_i\right]$. Then, it automatically replaces a part of the simulation input file where the parameters are stored and runs the reservoir simulation for the updated simulation input file using the system commands. The simulation is controlled by an auto-adaptive parameterized decision tree developed individually for a particular production process. At each time step $j$, the values of the tree attributes are determined by the simulator ${\mathit{a}}_{\mathit{i},\mathit{j}}=\left[a{1}_{i,j}, a{2}_{i,j},\dots ,a{R}_{i,j}\right]$. Based on the configured decision tree (directly declared within the simulation input file), the reservoir control prevailing in the next time step $u_{i, j+1}=\mathit{u}\left(t, {\mathit{p}}_i\right)$ is determined. This decision-making process is repeated in each of the $M$ simulation time steps. Hence, for given parameters values, the control vector contained in the vector ${\mathit{u}}_i=\left[u_{i,1}, u_{i,2},\dots ,u_{i,M}\right]$ is defined. In addition, as a result of the simulation, the value of the quality indicator is also calculated. The considered quality indicator is the NPV value of the project obtained at the end of the whole process, i.e., after performing M time steps: $J_i\left({\mathit{u}}_i\right)=NP{V}_i\left(M,{\mathit{u}}_i\right)$. The NPV value is directly calculated by the reservoir simulator. Then, it is read from the simulation output file by the program written in the Python programming language. It also passes the NPV value (with the opposite sign) to the SMAC tool. In the SMAC tool, the considered quality indicator is used to determine subsequent parameter values contained in the vector $p_{i+1}$. The selection of the subsequent parameters values is based on a so-called expected improvement criterion, which describes how individual parameters values can improve the quality of the solution. This criterion is high for those parameter values that are expected to improve NPV (based on the search space model), and for those that are located in the areas that have not been tested so far to improve the quality of the search space model and consequently be more precise in the next iterations. In each iteration, such parameter values are selected by the SMAC tool that maximizes this criterion. Hence, the NPV value calculated by the reservoir simulator is used by the SMAC tool to optimize parameter values. After performing a certain number of iterations, the proposed procedure results in process control that maximizes revenue from investment: $\widehat{\mathit{u}}=\left[{\widehat{u}}_1,{\widehat{u}}_2,\dots ,{\widehat{u}}_M\right]\wedge NPV\left(M,\widehat{\mathit{u}}\right)=\max \left[NP{V}_i\left(M, {\mathit{u}}_i\right)\right]$. Both the optimal parameters values and the maximized value of the NPV are returned by the developed tool written in the Python programming language. The diagram of the created tool discussed above is presented in Figure 3.

3. Case Study–CCS-EOR Process

To illustrate the effectiveness of the developed tool, an example of its use was carried out on a simulation model of a real gas-condensate reservoir. The production process considered for the analyzed reservoir is the CCS-EOR process. Its control was optimized with the use of the developed tool.

3.1. Reservoir Characteristics

The analyzed reservoir has been in operation since 1993. Basic parameters characterizing the reservoir are included in

Table 1. Basic parameters of the reservoir.

Reservoir Parameter	Value
Area	158.1 ha
Reservoir rock	Zechstein Main Dolomite (Upper Permian, Ca2)
Depth	3143–3224 m
Effective thickness	13.6 m
Average porosity	5.8%
Average permeability	3.4 mD
Initial reservoir pressure	670 bar
Reservoir temperature	114 °C
Oil resources remaining in the reservoir	27%
Oil density	794 kg/m3

. The reservoir has a complicated geological structure and the shape of an approximately semicircular elevation. There are three wells located on the reservoir, but production is currently carried out with two of them. The reservoir structure and location of the wells are presented in Figure 4.

The reservoir has a complex structure, as it is characterized by a double porosity system, which consists of a porous rock matrix and a fracture system. The matrix has negligible permeability, but considerable storage capacity, while the fractures have significant permeability and the dominant flow takes place in them [24]. Production from the analyzed reservoir is difficult also because it is a gas-condensate field. Thus, a drop in the formation pressure below the dew point pressure is accompanied by the release of condensate and the formation of a two-phase system in the reservoir, which complicates the production process [25]. The rock matrix is evenly filled with a mixture of gas and water, while condensate has condensed in the fractures.

The simulation model of the reservoir was made with the use of a compositional simulator ECLIPSE 300 (Schlumberger Limited, Houston, TX, USA). It includes a fifteen-component reservoir fluid containing hydrocarbon components from C1 to C10, C11+ (grouped C11 to C20) and C21+, and also N₂, H₂S, CO₂. Water was considered as a separate phase that participates in the flow and affects the flow of hydrocarbons. Within the model, three fluids deposited in the reservoir was taken into account: oil, gas, and water. The reservoir simulation model consists of 8800 blocks defined by the coordinates of their vertices. Due to the presence of the double porosity system, the simulation model is divided into two identical regions. One of them reflects the properties of the rock matrix and the other of the fracture system. The model was created based on data collected from all three wells located on the reservoir and seismic records, and then calibrated.

The reservoir production is carried out using primary methods dependent on the natural energy of the reservoir. The recovery factor of the analyzed reservoir is very high considering production under primary mechanisms only, and it equals 70%. It is possible as the analyzed reservoir is a small volumetric gas-condensate reservoir with a very high initial pressure and relatively good filtration properties (Table 1). In the initial conditions and for most of the production history, hydrocarbons were in the gas phase, which facilitated their extraction. The condensate released later as the pressure dropped. Therefore, so much of the hydrocarbons could be extracted. Although the reservoir is in the final stage of production, almost 30% of the initial oil resources are still there. Hence, in the case of the analyzed reservoir, the implementation of the CCS-EOR method was considered. Three wells located in the reservoir (two in production) are not equipped appropriately for the CCS-EOR method implementation for now, as it is currently in the conceptual phase. These wells are old (they were drilled almost 30 years ago), but their current technical condition is good and it is constantly monitored for the content of H₂S. If needed, wells equipment can be easily adopted for CO₂ injection/production.

3.2. CCS-EOR Process Implementation

For the implementation of the CCS-EOR process on the analyzed reservoir, two active wells located on the reservoir were applied. It is related to the very high cost of drilling new wells, which would significantly reduce the economic efficiency of the process. As the reservoir is characterized by a double porosity system, the simultaneous injection of carbon dioxide and the production of reservoir fluids would cause instantaneous breakthrough of carbon dioxide through the fracture system into the production well [26]. In the case of dual-porosity reservoirs, a more efficient realization of the CO₂-EOR method is the CO₂ huff-n-puff process consisting of the cyclic injection of carbon dioxide directly into the production well [26]. A single cycle (Figure 5) of the CO₂ huff-n-puff process includes three phases: the injection phase, the soaking phase, and the production phase [27]. During the injection stage, CO₂ is injected into the production well. In the soaking phase, the well is closed for a while. The soaking period aims to allow carbon dioxide to interact with the reservoir fluids [28]. In the production stage, oil is produced from the well. Such a cycle is repeated several times until a certain minimum flow rate is reached. The CO₂ huff-n-puff variant of the CO₂-EOR method allows the oil recoverability from the reservoir to be increased, especially in the case of the dual-porosity reservoirs [26]. Therefore, the CO₂ huff-n-puff process implemented in both existing wells was selected for the analyzed reservoir.

3.3. Developed CCS-EOR Process Control Scheme

To determine the scheme of the CO₂ huff-n-puff method suitable for the considered reservoir, the effect of the soaking time on the process effectiveness was firstly analyzed. The impact of the soaking stage on the CO₂ huff-n-puff process effectiveness applied on the dual-porosity reservoir has been studied in [26]. It was discovered that it depends on the factor affecting the conductivity between the matrix and fractures. The lower the factor, the higher increase in the recovery factor can be obtained thanks to the soaking stage. It was also discovered that for large factor values the soaking phase can negatively affect the oil volume that could be produced during the huff-n-puff process. For the analyzed reservoir the coefficient determining the conductivity between the matrix and fractures equals 0.95. For this value, the estimated duration of the soaking stage calculated based on the equation determined in [26] equals 2 months. Based on [26], it was also calculated that comparing to the huff-n-puff variant with the soaking stage skipped, the difference between cumulative oil production equals −45 m³. As in the case of the analyzed reservoir, the soaking phase reduces the effectiveness of the huff-n-puff process and should be skipped. In this case, the soaking phase has the opposite effect to the intended one. Hence, the CO₂ huff-n-puff method with the omitted soaking period was selected as an effective method of carbon dioxide injection into the analyzed reservoir.

During the injection phase, a constant carbon dioxide injection flow rate (90,000 m³/day) was assumed due to practical aspects related to its transmission and collection as well as the amount of CO₂ generated by the emitter selected for the analyzed reservoir. Production is carried out at a constant bottom hole pressure of 50 bar, taking into account the limitations of the surface installation. Theoretically, the carbon dioxide injection flow rate and the bottom hole pressure (during production) can be optimized with the SMAC tool. However, from a practical point of view, it is not necessary, as operating conditions other than those assumed would be impractical. Optimization of the operating conditions could be considered in a large project where there is no installation yet.

For the CO₂-EOR method configured in the way described above, followed by carbon dioxide storage in the reservoir (CCS-EOR), a parameterized decision tree was developed, as shown in Figure 6. The design of the developed decision tree was based on simulation tests and engineering knowledge. This is in line with the proposed concept of the auto-adaptive parameterized decision tree. In the methodology proposed in this paper, engineering knowledge is utilized to design a tree structure suitable for a particular problem, and artificial intelligence is used to select the best limit values of attributes. The auto-adaptive decision tree presented in Figure 6 is a visualization of the proposed methodology. It presents the decision tree proposed by the authors for the considered production process. However, in the proposed methodology, a decision tree can be modified and adjusted to a specific production process. Hence, the selected attributes can be replaced by any other attributes. These attributes can be selected by domain knowledge or the selection process can be automated with the use of, e.g., feature selection algorithm. Hence, the proposed methodology is general and as a result, it applies also to more complex systems.

In the proposed decision scheme of the CCS-EOR process, as only two stages of the huff-n-puff process were taken into account, the well control function takes the following form:

$$\mathit{u}\left(t,\mathit{p}\right)=\{\begin{array}{l}{q}_1\left(t\right)>0\mathrm{for}t:\mathsf{\Upsilon }\left(t\right)<0\\ q_2\left(t\right)<0\mathrm{for}t:\mathsf{\Upsilon }\left(t\right)>0\end{array}$$

(15)

where $\mathsf{{\rm Y}}$ is the control function, which is determined by the proposed decision tree. Thus, this tree describes the decision-making process to be carried out at each time step of the simulation $j$ to determine the control $u_{j+1}=\mathit{u}\left(t,\mathit{p}\right)$ for the next step.

Following the proposed decision tree, in the case of the EOR process, first, the value of the average reservoir pressure is checked. If it reaches the initial pressure level, it is necessary to stop the injection and start the production period due to the possible destruction of the reservoir structure. For lower pressures, the oil flow rate is checked. A value of zero means that CO₂ injection is currently in progress and it is continued until the reservoir pressure limit value (which has been optimized) is reached. When it is exceeded, the production period is automatically started. The oil flow rate greater than zero means that there is a production period. If the oil flow rate exceeds the economic limit, the production continues until the minimum performance for the cycle, which has also been parameterized, is reached. For the performance lower than the assumed level, the injection period is started to rebuild the reservoir pressure. This scheme is repeated until the moment when, although the production period lasts, the oil flow rate does not exceed the profitability threshold imposed on the oil performance. For the analyzed reservoir, the threshold was set at 0.25 m³/day. Due to the inability to continue oil production from the reservoir, the CCS process is automatically started. As part of this phase of the CCS-EOR process, carbon dioxide is injected into the reservoir until the initial reservoir pressure is reached. Then, the injection is stopped and the entire analyzed CCS-EOR process is completed. Thus, the developed decision scheme can determine both the process control and the duration of the investment.

The limits of the average reservoir pressure to which injection should be continued and the oil flow rate to which production should be carried out in each cycle of the CO₂ huff-n-puff process have been parameterized to optimize the CCS-EOR process. The proposed decision tree was directly implemented in the ECLIPSE reservoir simulator (Schlumberger Limited, Houston, TX, USA).

3.4. Intelligent Process Control Determination with the Use of the Created Tool

To use the created tool to determine the optimal control of the CCS-EOR process, it was necessary to define a quality indicator (describing the effectiveness of the CCS-EOR process) and the domains of decision variables (two parameters of the proposed decision tree).

In this work, a quality indicator that enables the evaluation of the obtained results is the NPV of the project. For the CO₂-EOR process carried out using the CO₂ huff-n-puff method followed by carbon dioxide storage in the reservoir (CCS), the NPV takes the following form:

$$J\left(\mathit{u}\right)=NPV\left(M,\mathit{u}\right)=\to \{\begin{array}{ll}{\displaystyle \sum _{i=1}^M\frac{q_{ri}{c}_r-q_{C{O}_{2}i}{k}_{C{O}_2}}{{\left(1+d\right)}^{{\tau }_i}}\mathsf{∆}{t}_i}& \mathrm{for}\mathrm{EOR}\\ {\displaystyle \sum _{i=1}^M\frac{q_{ri}{c}_r-q_{C{O}_{2}i}{k}_{C{O}_2}+q_{C{O}_{2}i}{c}_{C{O}_2}}{{\left(1+d\right)}^{{\tau }_i}}\mathsf{∆}{t}_i}& \mathrm{for}\mathrm{CCS}\end{array}$$

(16)

where $q_r$ is the oil flow rate [m³/day], $q_{C{O}_2}$ is the carbon dioxide flow rate [m³/day], $c_r$ is the oil price, $c_{C{O}_2}$ is the price for carbon dioxide storage (revenue from selling the storage capacity for CO₂), $k_{C{O}_2}$ is the cost of CO₂ injection [USD/m³], $d$ [/] is the annual discount rate and $\tau$ is the investment time [year]. For the CCS-EOR process, it can be assumed that the emitter pays for the disposal of carbon dioxide. In this paper, it is assumed for the considered CCS process. However, in the case of the assumed cyclic injection of carbon dioxide into the reservoir, during the CO₂-EOR process, carbon dioxide cannot constitute a profit as the emitter is not able to continuously utilize the produced gas. This assumption is in line with the goal of the CO₂-EOR process, which is to maximize the oil recovery factor while minimizing the amount of carbon dioxide used [17]. Therefore, the NPV is defined by two different formulas depending on the process being carried out. In addition, in both cases, the income from oil production and the cost of CO₂ injection were also taken into account. Due to the practical lack of water production (negligible water flow rate), the related costs were omitted [29,30]. In this analysis, it was not necessary to take into account the costs related to the implementation of the CCS-EOR method, such as the construction of a pipeline transporting carbon dioxide, as these costs are fixed and do not depend on the control. Moreover, it was assumed that the gas containing carbon dioxide produced in the huff-n-puff process would be injected into the reservoir located in the vicinity of the analyzed reservoir. Therefore, it is also not an additional cost for the analyzed reservoir. The presented form of the NPV function has been implemented directly in the ECLIPSE reservoir simulator (Schlumberger Limited, Houston, TX, USA). Hence, the NPV value is directly calculated by the reservoir simulator and then it is used by the SMAC tool to optimize parameters values. Finally, both the maximized value of the NPV and the optimal process control are determined by the created tool written in the Python programming language.

Due to the practical limitations, the domains of the oil flow rate and the average reservoir pressure were defined by finite sets of numbers from a technologically justified range. The assumed domain of the oil flow rate contains values changing every 1 m³/day, while for the average reservoir pressure every 10 bars. Parameters domains defined in this way allowed for a significant reduction of the search space. As a result, it was possible to reduce the number of iterations performed in the optimization process with the use of the SMAC tool. It translated into a significant reduction in the calculation time, as the duration of a single simulation ranged from a few to several hours depending on the selected process control.

To assess the effectiveness of the proposed solution, the developed tool was used to automatically determine the optimal control of the CCS-EOR process implemented on the analyzed reservoir. The scenario assuming an arbitrary selection of the decision tree parameter values was also considered. This scenario reflects a more traditional way of selecting the process control. In the analyzed case, the limit values were set at 300 bar and 4 m³/day based on tests and analyses performed. These values also served as a starting point for the calculations performed by the SMAC tool. The base variant that assumes the continuation of the current production scheme is also considered. Production is carried out at constant bottom pressure until the economic limit imposed on the oil flow rate is reached. This scenario makes it possible to determine the impact of the implementation of the CCS-EOR process on reservoir production.

4. Results

As a result of intelligent reservoir control determined with the use of the developed tool, the parameters of the CCS-EOR process were set at 210 bar and 9 m³/day. To illustrate the feasibility of using the developed tool to control production processes, a comparative analysis of the considered scenarios was made. The most important reservoir and production parameters, such as the average reservoir pressure and cumulative oil production were compared. To finally compare the analyzed scenarios, the NPV indicator was also compared.

The comparison of the average reservoir pressure is presented in Figure 7. In the base scenario, the reservoir energy is quickly depleted, which results in the reservoir abandonment after less than two years. The implementation of the CO₂-EOR method results in a partial reconstruction of the reservoir pressure at the initial process phase as in the proposed decision scheme it starts with the CO₂ injection. An increase in the reservoir pressure enables it to produce more oil during the production period. As in the scenario optimized with the developed procedure, the pressure limit is reduced from 300 bars (assumed in the arbitrary scenario) to 210 bars, and the first production period starts earlier. Hence, the costs of the initial CO₂ injection can be covered quicker by the revenue from the produced oil. The lower value of the pressure limit and higher value of the oil flow rate limit (9 instead of 4 m³/day) also increase the number of huff-n-puff cycles, and therefore the number of production periods. It allows oil to be produced and carbon dioxide to be injected in different market states, reducing the risk of producing oil with low oil prices and injecting CO₂ when its cost is high. The second production period is shorter due to the quicker decrease of the oil flow rate comparing to the first cycle. In addition, the CCS process in the optimized scenario starts earlier as thanks to the reduction of the length of the huff-n-puff cycles the inability to produce oil above the critical oil flow rate could be detected faster. The whole CCS-EOR process ends a bit later as the additional production period causes the final part of the pressure curve to be shifted in time compared to the arbitrary scenario. It brings a positive economic effect as, during the CO₂-EOR process, the injected carbon dioxide is a cost, while during the CCS phase it is a profit. The application of the CCS process results also in the extension of the reservoir’s life. In the optimized scenario, the CO₂-EOR period lasts 16 years. During this process, 586 million m³ of CO₂ is injected into the reservoir, and 336 million m³ of CO₂ is recovered to the surface. Thus, in the optimized scenario, 250 million m³ of carbon dioxide is stored during the EOR process and 405 million m³ of CO₂ is injected and stored during the CCS process. In the variant based just on engineering knowledge, only 270 million m³ is injected in the CCS period. It is 111 million m³ less than in the optimized option, which is as much as 16 %. The increase in the amount of carbon dioxide injected during the CCS period (obtained thanks to the use of the created tool) translates directly into the additional income that can be obtained when the proposed intelligent process control is applied.

The summary of changes in the cumulative oil production obtained during the analyzed process is presented in Figure 8. If the current production scheme is continued only 2550 m³ of oil can be produced as the critical oil flow rate is quickly reached due to the lack of reservoir energy. However, during the CCS-EOR process designed without the use of optimization 52,000 m³ of additional oil is produced. It confirms the potential of the CCS-EOR method. In this case, the whole oil volume is produced during one production period. Constant parts of the curve relate to the CO₂ injection periods when there is no oil production. Application of the proposed intelligent control of the process makes it possible to perform two production stages and increase the additional oil volume that can be produced by 3.5%. It gives the cumulative oil production equal to 54,000 m³. This value, with the geological resources of 308,000 m³ for 1997, translates into an increase in the recovery factor by 17.5%. The result obtained is so high as it is a small volumetric gas-condensate reservoir with relatively good filtration properties (Table 1). As production under primary mechanisms enabled to achieve a very high recovery factor (70%), the oil remaining in the reservoir is condensate, i.e., relatively light hydrocarbons that are easy to extract. Moreover, such a high result assumes another 16 years of production. The additional oil production in the optimized scenario during this period is obtained at no cost, as only the process control is changed. As the pressure limit value is lowered and the oil flow rate limit value is increased, the duration of a single cycle of the CO₂ huff-n-puff method is shortened. It results in the production of most of the additional oil earlier than in the unoptimized variant. It allows the revenue from the implementation of the CCS-EOR process to be achieved faster.

The NPV comparison for the considered scenarios of further exploitation of the analyzed reservoir is presented in Figure 9. The present values for each year of the analyzed scenarios are summarized in

Table 2. Comparison of present values.

Time [years]	CCS-EOR Optimized Values [USD]	CCS-EOR Arbitrary Values [USD]	Production Only [USD]
0	0	0	0
1	−929,409.08	−929,409.08	1,347,989.00
2	−929,437.93	−929,437.93	193,039.00
3	−792,685.00	−792,685.00	-
4	1,330,283.75	−734,742.00	-
5	2,965,430.13	−522,804.00	-
6	1,702,562.88	2,336,160.13	-
7	935,204.75	2,038,083.35	-
8	806,034.50	1,846,598.03	-
9	−365,077.00	1,338,357.75	-
10	−463,012.00	652,428.50	-
11	−429,167.00	459,756.25	-
12	−94,643.00	17,617.00	-
13	679,379.50	−366,393.50	-
14	491,052.50	−339,610.75	-
15	−260,780.50	−314,786.00	-
16	−279,301.50	−291,776.25	-
17	188,176.00	−219,436.00	-
18	176,719.50	167,172.25	-
19	154,903.00	154,903.50	-
20	155,060.50	155,060.50	-
21	132,246.00	-	-
22	151,797.00	-	-

. The production continuation with the current production scheme enables a continuous increase in NPV, but it is small and translates into the revenue of USD 1,540,000. Since the implementation of the CCS-EOR method begins with the injection of carbon dioxide into the reservoir (no oil production), the NPV value drops below zero in the initial stage of the process. The production period causes compensation for the costs incurred and significant revenues. In the optimized variant, the compensation is obtained earlier, as stages of the huff-n-puff process are shorter. In the first production stage of the optimized scenario, the oil production is smaller than in the production stage of the unoptimized option. However, as this production occurs earlier, the NPV at the end of this stage is greater than in the case of the unoptimized variant. Then, the decrease in the curve is related to the CO₂ injection stage. As it is shorter in the optimized variant, this stage generates a smaller decrease in NPV comparing to the unoptimized scenario. In the optimized variant, there is one more period when NPV increases and then decreases as the number of huff-n-puff cycles is increased. The CCS process, in which the injection of carbon dioxide brings a profit, causes an additional increase in NPV in the last period of the whole CCS-EOR process. As this period lasts longer for the optimized scenario, it generates higher income. In the case of an unoptimized scenario, it is possible to obtain total revenues of USD 3,725,000. The process control determined by the developed tool enables the increase of revenues to USD 5,325,000. Thus, process control according to the proposed intelligent strategy can increase revenue by 49% comparing to the unoptimized variant. It results from the fact that thanks to the application of the created tool, the oil production can be increased by 3.5% during the CO₂-EOR phase while reducing the amount of carbon dioxide injected at that time by 16%. Hence, application of the process control determined with the use of the developed tool can quickly and effectively allow the recovery factor to be increased, and the costs of the process to be reduced, and thus the profit on the project to be increased. In addition, such an intelligent process control application allows an ecological effect to be achieved, thanks to the more efficient use of the geological resources of the reservoir.

5. Conclusions

In this work, a novel tool that enables the determination of intelligent reservoir control for a given production process, which brings the highest possible profit, was created. The developed tool is based on a combination of artificial intelligence, control theory, and computer simulation of hydrocarbon reservoirs. It determines the control in each time step of the simulation based on an auto-adaptive parameterized decision tree implemented in the ECLIPSE reservoir simulator (Schlumberger Limited, Houston, TX, USA). The optimal values of the parameters are selected with the use of the AI-based optimization tool called SMAC. The combination of a reservoir simulator and an optimization tool has been implemented in the Python programming language. The developed tool enables self-improving automatic control of a given production process. The structure of the developed procedure and its high computational efficiency make it possible to apply it to simulation models of real hydrocarbon reservoirs with a high degree of complexity and a long simulation time.

The main advantage of the method proposed in this paper is that with the use of the created tool, the optimal process control can be determined automatically. As in the proposed methodology, the control scheme is presented in the form of the decision tree, and the optimized scheme can be easily used by the operator to control the real production process. The disadvantage is that to determine this optimal control scheme, the reservoir model is needed.

An example of the application of the created tool for the automatic determination of the CCS-EOR process control on a real gas-condensate reservoir confirmed the effectiveness of the developed tool. In the optimized variant of the CCS-EOR process, the CO₂-EOR period lasts 16 years. During this time, 250 million m³ of carbon dioxide can be stored and 54,000 m³ of additional oil can be produced. During 5 years of the CCS process, 405 million m³ of CO₂ is injected and stored. Process control according to the proposed intelligent strategy can increase revenue by 49% compared to the unoptimized variant. It results from the fact that thanks to the application of the created tool oil production can be increased by 3.5% during the CO₂-EOR phase while reducing the amount of carbon dioxide injected at that time by 16%. Application of the proposed intelligent control of the process determined with the use of the developed tool makes it possible to increase the recovery factor and also to reduce the costs of the process, and thus increase the profit on the project without incurring additional financial outlays. The effect of more rational management of resources can be also obtained as the efficiency of hydrocarbon reservoir production can be increased.