Application of the ALRW-DDPG Algorithm in Offshore Oil–Gas–Water Separation Control

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This study introduces an adaptive learning rate weighted Deep Deterministic Policy Gradient (ALRW-DDPG) control algorithm to enhance the stability and efficiency of liquid level control in three-phase separators, authors demonstrate significant improvements over traditional PID and DDPG methods through simulation experiments. The detailed comments are listed below:

1.        The description of the adaptive learning rate weighting strategy is central to the paper. Please provide more detailed mathematical justifications for the chosen formulae in Equation 10 and 11. How does the adaptive learning rate impact the stability and convergence speed in comparison to other adaptive strategies?

2.        While the paper compares the proposed ALRW-DDPG with traditional DDPG and PID, it is suggested to include a discussion on how this method compares with other recent advances in reinforcement learning applied to control systems, such as PPO or SAC.

3.        Please discuss the potential impact of the assumptions behind the three-phase separator model on the generalizability of the ALRW-DDPG algorithm to other types of separators or under different operational conditions?

4.        The simulations are based on the Century FPSO’s separator data. Could the authors provide real-world testing results or a discussion on how the simulation results correlate with actual operational data from other platforms?

5.        The paper mentions using square and random waveforms to simulate slug flow. Could you expand on how these waveforms capture the complexity of real slug flow patterns?

6.        It would be beneficial to include an introduction of slug flow. The following experimental study of gas-water slug flow is recommended to incorporate:

-       https://doi.org/10.3390/en14030578

7.        How scalable is the ALRW-DDPG algorithm to separators of different sizes, or with significantly different fluid properties?

8.        How does the computational efficiency of the ALRW-DDPG algorithm compared to standard DDPG in terms of processing time and resource requirements?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Review

energies-3179507

Application of ALRW-DDPG Algorithm on Offshore Oil-Gas- 2 Water Separation Control

The presented paper is well written and structured. Despite the fact that the introduction is rather small, the authors pointed out the main advantages and disadvantages of using various algorithms and methods to solve the problem of regulating the operation of a three-phase separator. All necessary references are provided. The main part of the paper is also clearly stated, and the results obtained are beyond doubt. There are several comments on the paper content that should be considered as minor revision:

1) In my opinion, the authors should more clearly state the purpose of the study, based on the journal scope (https://www.mdpi.com/journal/energies/about). In its present form the paper is more closely related to computer technology and artificial intelligence usage.

2) Table 1 (Pressure   50~70 ℃) – “Pressure” should be replaced by temperature

3) The proposed algorithm takes into account the disadvantages of the previously proposed methods of solving the problem, however, they all relate more to the design of the device or the condition of its operation (for example, pressure). Meanwhile, the specificity of the formation of the boundary between the aqueous and oily phases is associated with the features of the phase transition (liquid-liquid equilibrium). The composition of oil may vary. Has the sensitivity of the proposed method been tested depending on the component composition of the oil? Or is this factor not fundamental?

4) The parameters affecting the liquid level height were set randomly. Whether the range of parameter variation (limit values) has been set? If so, what were the limit values based on?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

In a separator (metallic cistern) there are 3 layers of gas, oil, water and
the inlet flow is known. We can control the valve opening for
outlet flows. The aim is to get a quasi-constant liquid level height.

The authors use SIMULINK environment and an
deep deterministic policy gradient with adaptive learning rate weights (ALRW-DDPG)
control algorithm.

For square wave disturbance at the inlet as well as for random wave disturbance,
the liquid level height is almost constant, by using this algorithm.

Qestions

1) We want constant liquid level. What is the mathematical indicator that you employ, in order to appreciate that the level is constant?

2) Fig. 10. For the DDPG algorithm, the reward is decreasing until 90 episode. Haw can you explain this?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf