Kybernetika 57 no. 3, 474-492, 2021

Markov stopping games with an absorbing state and total reward criterion

Rolando Cavazos-Cadena, Luis Rodríguez-Gutiérrez and Dulce María Sánchez-GuillermoDOI: 10.14736/kyb-2021-3-0474

Abstract:

This work is concerned with discrete-time zero-sum games with Markov transitions on a denumerable space. At each decision time player II can stop the system paying a terminal reward to player I, or can let the system to continue its evolution. If the system is not halted, player I selects an action which affects the transitions and receives a running reward from player II. Assuming the existence of an absorbing state which is accessible from any other state, the performance of a pair of decision strategies is measured by the total expected reward criterion. In this context it is shown that the value function of the game is characterized by an equilibrium equation, and the existence of a Nash equilibrium is established.

Keywords:

hitting time, non-expansive operator, monotonicity property, fixed point, equilibrium equation, bounded rewards

Classification:

91A10, 91A15

paper.pdf

References:

  1. E. Altman and A. Shwartz: Constrained Markov Games: Nash Equilibria. In: Annals of Dynamic Games (V. Gaitsgory, J. Filar and K. Mizukami, eds. 6, Birkhauser, Boston 2000, pp. 213-221.   DOI:10.1007/978-1-4612-1336-9\_11
  2. R. Atar and A. Budhiraja: A stochastic differential game for the inhomogeneous $\infty$-Laplace equation. Ann. Probab.2 (2010), 498-531.   DOI:10.1214/09-aop494
  3. N. Bäuerle and U. Rieder: Zero-sum risk-sensitive stochastic games. Stoch. Proc. Appl. 127 (2017), 622-642.   DOI:10.1016/j.spa.2016.06.020
  4. T. Bielecki, D. Hernández-Hernández and S. R. Pliska: Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management. Math. Methods Oper. Res. 50 (1999), 167-188.   DOI:10.1007/s001860050094
  5. R. Cavazos-Cadena and D. Hernández-Hernández: Nash equilibria in a class of Markov stopping games. Kybernetika 48 (2012), 1027-1044.   CrossRef
  6. J. A. Filar and O. J. Vrieze: Competitive Markov Decision Processes. Springer, Berlin 1996.   CrossRef
  7. O. Hernández-Lerma: Adaptive Markov Control Processes. Springer, New York 1989.   CrossRef
  8. O. Hernández-Lerma and J. B. Lasserre: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York 1996.   CrossRef
  9. V. N. Kolokoltsov and O. A. Malafeyev: Understanding Game Theory. World Scientific, Singapore 2010.   CrossRef
  10. V. M. Martínez-Cortés: Bipersonal stochastic transient Markov games with stopping times and total reward criteria. Kybernetika 57 (2021), 1-14.   DOI:10.14736/kyb-2021-1-0001
  11. G. Peskir: On the American option problem. Math. Finance 15 (2005), 169-181.   DOI:0.5840/leibniz20051510
  12. G. Peskir and A. Shiryaev: Optimal Stopping and Free-Boundary Problems. Birkhau\-ser, Boston 2010.   CrossRef
  13. A. B. Piunovskiy: Examples in Markov Decision Processes. Imperial College Press, London 2013.   CrossRef
  14. M. Puterman: Markov Decision Processes. Wiley, New York 1994.   CrossRef
  15. L. S. Shapley: Stochastic games. Proc. Natl. Acad. Sci. USA 39 (1953), 1095-1100.   CrossRef
  16. A. Shiryaev: Optimal Stopping Rules. Springer, New York 1978.   CrossRef
  17. K. Sladký: Ramsey growth model under uncertainty. In: Proc. 27th International Conference Mathematical Methods in Economics (H. Brozová, ed.), Kostelec nad Černými lesy 2009, pp. 296-300.   CrossRef
  18. K. Sladký: Risk-sensitive Ramsey growth model. In: Proc. 28th International Conference on Mathematical Methods in Economics (M. Houda and J. Friebelová, eds.) České Budějovice 2010, pp. 560-565.   CrossRef
  19. K. Sladký: Risk-sensitive average optimality in Markov decision processes. Kybernetika 54 (2018), 1218-1230.   DOI:10.14736/kyb-2018-6-1218
  20. D. J. White: Real applications of Markov decision processes. Interfaces 15 (1985), 73-83.   DOI:10.1287/inte.15.6.73
  21. D. J. White: Further real applications of Markov decision processes. Interfaces 18 (1988), 55-61.   DOI:10.1287/inte.18.5.55
  22. D. J. White: A survey of applications of Markov decision processes. J. Opl. Res. Soc. 44 (1993), 1073-1096.   DOI:10.1016/0042-207X(93)90304-S
  23. L. E. Zachrisson: Markov Games. In: Advances in Game Theory (M. Dresher, L. S. Shapley and A. W. Tucker, eds.), Princeton Univ. Press, Princeton N.J. 1964, pp. 211-253.   CrossRef