Kybernetika 51 no. 4, 629-638, 2015

Note on stability estimation in average Markov control processes

Jaime Martínez Sánchez and Elena ZaitsevaDOI: 10.14736/kyb-2015-4-0629

Abstract:

We study the stability of average optimal control of general discrete-time Markov processes. Under certain ergodicity and Lipschitz conditions the stability index is bounded by a constant times the Prokhorov distance between distributions of random vectors determinating the "original and the perturbated" control processes.

Keywords:

stability index, discrete-time Markov control processes, average criterion, Prokhorov metric

Classification:

90C40, 93E20

paper.pdf

References:

  1. A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus: Discrete-time controlled Markov processes with average cost criterion: A survey. SIAM J. Control and Optimization 31 (1993), 282-344.   DOI:10.1137/0331018
  2. R. M. Dudley: Real Analysis and Probability. Wadsworth and Brooks/Cole, Pacific Grove 1989.   DOI:10.1017/s0013091500018265
  3. R. M. Dudley: The speed of mean Glivenko-Cantelly convergence. Ann. Math. Stat. 40 (1969), 40-50.   DOI:10.1214/aoms/1177697802
  4. E. B. Dynkin and A. A. Yushkevich: Controlled Markov Processes. Springer-Verlag, New York 1979.   DOI:10.1002/zamm.19810610317
  5. E. Gordienko, E. Lemus-Rodríguez and R. Montes-de-Oca: Average cost Markov control processes: stability with respect to the Kantorovich metric. Math. Methods Oper. Res. 70 (2009), 13-33.   DOI:10.1007/s00186-008-0229-6
  6. E. I. Gordienko: Stability estimates for controlled Markov chains with a minorant. J. Soviet. Math. 40 (1988), 481-486.   DOI:10.1007/bf01083641
  7. E. I. Gordienko and A. A. Yushkevich: Stability estimates in the problem of average optimal switching of a Markov Chain. Math. Methods Oper. Res. 57 (2003), 345-365.   CrossRef
  8. O. Hernández-Lerma: Adaptive Markov Control Processes. Springer-Verlag, New York 1989.   DOI:10.1007/978-1-4419-8714-3
  9. R. Montes-de-Oca and F. Salem-Silva: Estimates for perturbations of average Markov decision processes with a minimal state and upper bounded by stochastically ordered Markov chains. Kybernetika 41 (2005), 757-772.   CrossRef
  10. S. T. Rachev and L. Rüschendorf: Mass Transportation Problem, Vol. II: Applications. Springer, New York 1998.   CrossRef
  11. O. Vega-Amaya: The average cost optimality equation: a fixed point approach. Bol. Soc. Math. Mexicana 9 (2003), 185-195.   CrossRef