Robust Statistic Estimation of Constrained Optimal Control Problems of Pollution Accumulation (Part I)
Abstract
:1. Introduction
2. Problem Statement
2.1. Control Policies and Stability Assumptions
- is the Borel -algebra spawned by the Borel set B.
- is, as usual, the space of all real-valued continuous functions on a bounded, open and connected subset .
- stands for the space of all real-valued continuous bounded functions f on the bounded, open and connected subset .
- is the space of all real-valued continuous functions f on the bounded, open and connected subset , with continuous derivatives up to order .
- is, as is customary, the Lebesgue space of functions g on such that , with a suitable measure space, and .
- is the family of probability measures on B endowed with the topology of weak convergence.
- (a)
- for each and , , and for each and , is a Borel function on ;
- (b)
- for each and , the function is Borel-measurable in .
2.2. Reward, Cost and Constraint Rates
3. Main Results
3.1. Discounted Control with Constraints
3.2. Lagrange Multipliers Approach
- (a)
- Notice that
- (b)
- By Definitions 4 and 5,
- (c)
- Given that the cost and constraint rates satisfy Assumption A3, we deduceThus,
- (d)
- The function is locally Lipschitz on In fact,
- (e)
- Parts (c) and (d) imply that the function satisfies Assumption A3. Thus, the rate is Lipschitz-continuous and . Furthermore, by virtue of (9) and Proposition 1,implying that .
3.3. Convergence of Value Functions and
- (a)
- If Assumptions A1–A3 hold, then by Proposition 3.4 in [11], the mappings , and are continuous on Π for each and
- (b)
- Let be a sequence of USC estimators of . Then, using Theorem 4.5 in [36], for every measurable function that satisfies the Assumptions A1–A3, the sequence converges to -a.s., for each and .
- (c)
- Let be a sequence in Π. Since Π is a compact set, there exists a subsequence such that , thus, combining parts (a) and (b), and using the following triangular inequality:we deduce that, for every measurable function satisfying Assumption A3, we have that
- (d)
- The optimal discount reward for the -DUP, satisfies Proposition 3. In addition, Proposition 3(ii) ensures the existence of stationary policy .
- (e)
- For each , and , we denoteSince can be seen as an embedding of Π, Proposition 3(ii) ensures that the set is nonempty.
- (f)
- Under the hypotheses of Proposition 3, Lemma 3.15 in [11] ensures that for each fixed and any sequence in converging to some ; if there exists a sequence for each , such that it converges to a policy , then . That is, π satisfies
- (g)
- Lemma 3.16 in [11] ensures that the mapping is differentiable on , for any and ; in fact, for each and
3.4. Estimation Methods for Our Application
4. Numeric Illustration
4.1. The -DUP
4.2. The Dpc
4.3. Numerical Results for the Optimal Accumulation Problem
4.3.1. Numerical Results for the -DUP
4.3.2. Numerical Results for the DPC
5. Concluding Remarks
- 1.
- For each m, there are optimal control policies for the -DUP and -DPC.
- 2.
- For each initial state , and , and is almost certainly .
- 3.
- For the DUP, there is a subsequence of and a policy , such that converges to in the topology of relaxed controls, and, moreover, is optimal for the -DUP. Moreover, if is a critical point of , then is optimal for the -DCP.
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A. Technical Assumptions
- (a)
- The random process (1) belongs to a complete probability space . Here, is a filtration on , such that each is complete relative to ; and is the law of the state process given the parameter and the control .
- (b)
- The drift coefficient in (1) is continuous and locally Lipschitz in the first and third arguments uniformly in u; that is, for each , there exist nonnegative constants and such that, for all , all and ,Moreover, is continuous on U.
- (c)
- The diffusion coefficient satisfies a local Lipschitz condition; that is, for each , there is a positive constant such that, for all ,
- (d)
- The coefficients b and σ satisfy a global linear growth condition of the formwhere is a positive constant.
- (e)
- (Uniform ellipticity). The matrix satisfies that, for some constant ,
- (a)
- .
- (b)
- for all , and x in .
- (a)
- The payoff rate the cost rate and the constraint rate are continuous on and , respectively. Moreover, they are locally Lipschitz on , uniformly on U and Θ; that is, for each , there are positive constants and such that for all
- (b)
- The rates , and are in uniformly on U and Θ; in other words, there exists such that, for all
References
- Kurano, M. Discrete-time Markovian decision processes with an unknown parameter-average return criterion. J. Oper. Res. Soc. Jpn. 1972, 15, 67–76. [Google Scholar]
- Mandl, P. Estimation and control in Markov chains. Adv. Appl. Probab. 1974, 6, 40–60. [Google Scholar] [CrossRef]
- Hernández-Lerma, O.; Marcus, S. Technical note: Adaptive control of discounted Markov Decision chains. J. Optim. Theory Appl. 1985, 46, 227–235. [Google Scholar] [CrossRef]
- Hilgert, N.; Minjárez-Sosa, A. Adaptive control of stochastic systems with unknown disturbance distribution: Discounted criteria. Math. Methods Oper. Res. 2006, 63, 443–460. [Google Scholar] [CrossRef]
- Broadie, M.; Cvitanic, J.; Soner, H.M. Optimal replication of contingent claims under portfolio constraints. Rev. Fin. Stud. 1998, 11, 59–79. [Google Scholar] [CrossRef]
- Cvitanic, J.; Pham, H.; Touzi, N. A closed-form solution for the super-replication problem under transaction costs. Financ. Stochastics 1999, 3, 35–54. [Google Scholar] [CrossRef]
- Cvitanic, J.; Pham, H.; Touzi, N. Superreplication in stochastic volatility models under portfolio constraints. J. Appl. Probab. 1999, 36, 523–545. [Google Scholar] [CrossRef]
- Soner, M.; Touzi, N. Super replication under gamma constraints. SIAM J. Control Optim. 2000, 39, 73–96. [Google Scholar] [CrossRef]
- Mendoza-Pérez, A.; Jasso-Fuentes, H.; Hernández-Lerma, O. The Lagrange approach to ergodic control of diffusions with cost constraints. Optimization 2015, 64, 179–196. [Google Scholar] [CrossRef]
- Prieto-Rumeau, T.; Hernández-Lerma, O. The vanishing discount approach to constrained continuous-time controlled Markov chains. Syst. Control Lett. 2010, 59, 504–509. [Google Scholar] [CrossRef]
- Jasso-Fuentes, H.; Escobedo-Trujillo, B.A.; Mendoza-Pérez, A. The Lagrange and the vanishing discount techniques to controlled diffusion with cost constraints. J. Math. Anal. Appl. 2016, 437, 999–1035. [Google Scholar] [CrossRef]
- Bielecki, T. Adaptive control of continuous-time linear stochastic systems with discounted cost criterion. J. Optim. Theory Appl. 1991, 68, 379–383. [Google Scholar] [CrossRef]
- Vrabie, D.; Pastravanu, O.; Abu-Khalaf, M.; Lewis, F. Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 2009, 45, 477–484. [Google Scholar] [CrossRef]
- Di Masp, G.; Stettner, L. Bayesian ergodic adaptive control of diffusion processes. Stochastics Stochastics Rep. 1997, 60, 155–183. [Google Scholar] [CrossRef]
- Borkar, V.; Ghosh, M. Ergodic Control of Multidimensional Diffusions II: Adaptive Control. Appl. Math. Optim. 1990, 21, 191–220. [Google Scholar] [CrossRef]
- Borkar, V.; Bagchi, A. Parameter estimation in continuous-time stochastic processes. Stochastics 1982, 8, 193–212. [Google Scholar] [CrossRef]
- Huzak, M. Estimating a class of diffusions from discrete observations via approximate maximum likelihood method. Statistics 2018, 52, 239–272. [Google Scholar] [CrossRef]
- Shoji, I. A note on asymptotic properties of the estimator derived from the Euler method for diffusion processes at discrete times. Stat. Probab. Lett. 1997, 36, 153–159. [Google Scholar] [CrossRef]
- Ralchenko, K. Asymptotic normality of discretized maximum likelihood estimator for drift parameter in homogeneous diffusion model. Mod. Stochastics Theory Appl. 2015, 2, 17–28. [Google Scholar] [CrossRef]
- Duncan, T.; Pasik-Duncan, B.; Stettner, L. Almost self-optimizing strategies for the adaptive control of diffusion processes. J. Optim. Theory Appl. 1994, 81, 479–507. [Google Scholar] [CrossRef]
- Durham, G.; Gallant, A. Numerical Techniques for Maximum Likelihood Estimation of Continuous-Time Diffusion Processes. J. Bus. Econ. Stat. 2002, 20, 297–316. [Google Scholar] [CrossRef] [Green Version]
- Athanassoglou, S.; Xepapadeas, A. Pollution control with uncertain stock dynamics: When, and how, to be precautious. J. Environ. Econ. Manag. 2012, 63, 304–320. [Google Scholar] [CrossRef]
- Jiang, K.; You, D.; Li, Z.; Shi, S. A differential game approach to dynamic optimal control strategies for watershed pollution across regional boundaries under eco-compensation criterion. Ecol. Indic. 2019, 105, 229–241. [Google Scholar] [CrossRef]
- Kawaguchi, K. Optimal Control of Pollution Accumulation with Long-Run Average Welfare. Environ. Resour. Econ. 2003, 26, 457–468. [Google Scholar] [CrossRef]
- Kawaguchi, K.; Morimoto, H. Long-run average welfare in a pollution accumulation model. J. Econ. Dyn. Control 2007, 31, 703–720. [Google Scholar] [CrossRef]
- Morimoto, H. Optimal Pollution Control with Long-Run Average Criteria. In Stochastic Control and Mathematical Modeling: Applications in Economics; Encyclopedia of Mathematics and its Applications, Cambridge University Press: Cambridge, UK, 2010; pp. 237–251. [Google Scholar] [CrossRef]
- Jasso-Fuentes, H.; López-Barrientos, J.D. On the use of stochastic differential games against nature to ergodic control problems with unknown parameters. Int. J. Control 2015, 88, 897–909. [Google Scholar] [CrossRef]
- Zhang, G.; Zhang, Z.; Cui, Y.; Yuan, C. Game Model of Enterprises and Government Based on the Tax Preference Policy for Energy Conservation and Emission Reduction. Filomat 2016, 30, 3963–3974. [Google Scholar] [CrossRef]
- Zhang, Z.; Zhang, G.; Su, B. The spatial impacts of air pollution and socio-economic status on public health: Empirical evidence from China. Socio-Econ. Plan. Sci. 2022, 83, 101167. [Google Scholar] [CrossRef]
- Cox, L.A.T., Jr. Confronting Deep Uncertainties in Risk Analysis. Risk Anal. 2012, 32, 1607–1629. [Google Scholar] [CrossRef]
- López-Barrientos, J.D.; Jasso-Fuentes, H.; Escobedo-Trujillo, B.A. Discounted robust control for Markov diffusion processes. Top 2015, 23, 53–76. [Google Scholar] [CrossRef]
- Escobedo-Trujillo, B.A.; López-Barrientos, J.D.; Garrido-Meléndez, J. A Constrained Markovian Diffusion Model for Controlling the Pollution Accumulation. Mathematics 2021, 9, 1466. [Google Scholar] [CrossRef]
- Borkar, V.; Ghosh, M. Controlled diffusions with constraints. J. Math. Anal. Appl. 1990, 152, 88–108. [Google Scholar] [CrossRef]
- Borkar, V. Controlled diffusions with constraints II. J. Math. Anal. Appl. 1993, 176, 310–321. [Google Scholar] [CrossRef]
- Duncan, T.; Pasik-Duncan, B. Adaptive control of continuous time linear stochastic systems. Math. Control. Signals Syst. 1990, 3, 45–60. [Google Scholar] [CrossRef]
- Escobedo-Trujillo, B.; Hernández-Lerma, O.; Alaffita-Hernández, F. Adaptive control of diffusion processes with a discounted criterion. Appl. Math. 2020, 47, 225–253. [Google Scholar] [CrossRef]
- Arapostathis, A.; Borkar, V.; Ghosh, M. Ergodic control of diffusion processes. In Encyclopedia of Mathematics and its Applications; Cambridge University Press: Cambridge, UK, 2012; Volume 143. [Google Scholar]
- Warga, J. Optimal Control of Differential and Functional Equations; Academic Press: New York, NY, USA, 1972. [Google Scholar]
- Fleming, W.; Nisio, M. On the stochastic relaxed control for partially observed diffusions. Nagoya Mathhematical J. 1984, 93, 71–108. [Google Scholar] [CrossRef]
- Jasso-Fuentes, H.; Yin, G. Advanced Criteria for Controlled Markov-Modulated Diffusions in an Infinite Horizon: Overtaking, Bias, and Blackwell Optimality; Science Press: Beijing, China, 2013. [Google Scholar]
- Jasso-Fuentes, H.; Hernández-Lerma, O. Characterizations of overtaking optimality for controlled diffusion processes. Appl. Math. Optim. 2007, 57, 349–369. [Google Scholar] [CrossRef]
- Pedersen, A.R. Consistency and asymptotic normality of an approximate maximum likelihood estimator for discretely observed diffusions process. Bernoulli 1995, 1, 257–279. [Google Scholar] [CrossRef]
- Ghosh, M.K.; Arapostathis, A.; Marcus, S.I. Ergodic control of switching diffusions to flexible manufacturing systems. SIAM J. Control Optim. 1993, 31, 1183–1204. [Google Scholar] [CrossRef] [Green Version]
- Rogers, L.; Williams, D. Diffusions, Markov Processes and Martingales, Vol.1, Foundations; Cambridge University Press: Cambridge, UK, 2000. [Google Scholar]
240 | 0.000509042 | ||||
300 | 0.00040062 | ||||
400 | 0.000285924 | ||||
600 | 0.000155536 | ||||
1200 | |||||
Real | 0 | 0 | 0 |
m | |||||
240 | 0.000718096 | ||||
300 | 0.000629978 | ||||
400 | 0.000423607 | ||||
600 | 0.000437829 | ||||
1200 | 0.00033381 | ||||
Real | 0 | 0 |
m | ||||
---|---|---|---|---|
240 | ||||
300 | ||||
400 | ||||
600 | ||||
1200 | ||||
Real | 0 | 0 |
m | ||||
---|---|---|---|---|
240 | 0.000104368 | |||
300 | 0.000100842 | |||
400 | ||||
600 | 0.000103271 | |||
1200 | 0.000103226 | |||
Real | 0 | 0 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Escobedo-Trujillo, B.A.; López-Barrientos, J.D.; Higuera-Chan, C.G.; Alaffita-Hernández, F.A. Robust Statistic Estimation of Constrained Optimal Control Problems of Pollution Accumulation (Part I). Mathematics 2023, 11, 923. https://doi.org/10.3390/math11040923
Escobedo-Trujillo BA, López-Barrientos JD, Higuera-Chan CG, Alaffita-Hernández FA. Robust Statistic Estimation of Constrained Optimal Control Problems of Pollution Accumulation (Part I). Mathematics. 2023; 11(4):923. https://doi.org/10.3390/math11040923
Chicago/Turabian StyleEscobedo-Trujillo, Beatris Adriana, José Daniel López-Barrientos, Carmen Geraldi Higuera-Chan, and Francisco Alejandro Alaffita-Hernández. 2023. "Robust Statistic Estimation of Constrained Optimal Control Problems of Pollution Accumulation (Part I)" Mathematics 11, no. 4: 923. https://doi.org/10.3390/math11040923
APA StyleEscobedo-Trujillo, B. A., López-Barrientos, J. D., Higuera-Chan, C. G., & Alaffita-Hernández, F. A. (2023). Robust Statistic Estimation of Constrained Optimal Control Problems of Pollution Accumulation (Part I). Mathematics, 11(4), 923. https://doi.org/10.3390/math11040923