Kybernetika 58 no. 2, 163-179, 2022

Deterministic Markov Nash equilibria for potential discrete-time stochastic games

Alejandra Fonseca-MoralesDOI: 10.14736/kyb-2022-2-0163

Abstract:

In this paper, we study the problem of finding deterministic (also known as feedback or closed-loop) Markov Nash equilibria for a class of discrete-time stochastic games. In order to establish our results, we develop a potential game approach based on the dynamic programming technique. The identified potential stochastic games have Borel state and action spaces and possibly unbounded nondifferentiable cost-per-stage functions. In particular, the team (or coordination) stochastic games and the stochastic games with an action independent transition law are covered.

Keywords:

optimal control, dynamic programming, stochastic games, potential approach

Classification:

91A50, 91A25, 93E20, 91A14, 91A10

paper.pdf

References:

  1. D. Dragone, L. Lambertini, G. Leitmann and A. Palestini: Hamiltonian potential functions for differential games. Automatica 62 (2015), 134-138.   DOI:10.1016/j.automatica.2015.09.036
  2. W. H. Fleming and R. W. Rishel: Deterministic and stochastic optimal control. Springer Science and Business Media 1 (2012).   CrossRef
  3. A. Fonseca-Morales and O. Hernández-Lerma: Potential differential games. Dyn. Games Appl. 8 (2018), 254-279.   DOI:10.1007/s13235-017-0218-6
  4. A. Fonseca-Morales and O. Hernández-Lerma: Stochastic differential games: the potential approach. Stochastics 92, (2020), 1125-1138.   DOI:10.1080/17442508.2019.1691208
  5. D. González-Sánchez and O. Hernández-Lerma: Discrete-time Stochastic Control and Dynamic Potential Games: The Euler-equation Approach. Springer, New York 2013.   CrossRef
  6. R. Gopalakrishnan, J.Ŕ. Marden and A. Wierman: Potential games are necessary to ensure pure Nash equilibria in cost sharing games. Math. Oper. Res. 39 (2014), 1252-1296.   DOI:10.1287/moor.2014.0651
  7. O. Hernández-Lerma and J. B. Lasserre: Discrete-time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York 1996.   CrossRef
  8. O. Hernández-Lerma and J. B. Lasserre: Further Topics on Discrete-Time Markov Control Processes. Springer-Verlag, New York 1999.   CrossRef
  9. F. Luque-Vázquez and J. A. Minjárez-Sosa: Empirical approximation in Markov games under unbounded payoff: discounted and average criteria. Kybernetika 53 (2017), 4, 694-716.   DOI:10.14736/kyb-2017-4-0694
  10. V. V. Mazalov, A. N. Rettieva and K. E. Avrachenkov: Linear-quadratic discrete-time dynamic potential games. Autom. Remote Control 78 (2017), 1537-1544.   DOI:10.1134/S0005117917080136
  11. D. Mguni: Stochastic potential games.    CrossRef
  12. J. A. Minjárez-Sosa: Zero-Sum Discrete-Time Markov Games with Unknown Disturbance Distribution: Discounted and Average Criteria. Springer Nature, Cham 2020.   DOI:10.1007/978-3-030-35720-7
  13. D. Monderer and L. S. Shapley: Potential games. Game Econ. Behav. 14 (1996), 124-143.   DOI:10.1006/game.1996.0044
  14. J. A. M. Potters, T. E. S. Raghavan and S. H. Tijs: Pure equilibrium strategies for stochastic games via potential functions. Adv. Dyn. Games Appl. Birkhauser, Boston 2009, pp. 433-444.   DOI:10.1007/978-0-8176-4834-3\_22
  15. A. D. Robles-Aguilar, D. González-Sánchez and J. A. Minjárez-Sosa: Estimation of equilibria in an advertising game with unknown distribution of the response to advertising efforts. In: Modern Trends in Controlled Stochastic Processes, Theory and Applications, V.III, (A. Piunovskiy and Y. Zhang Eds.), Springer Nature, Cham 2021. pp. 148-165.   DOI:10.1007/978-3-030-76928-4\_8
  16. R. W. Rosenthal: A class of games possessing pure-strategy Nash equilibria. Int. J. Game Theory 2 (1973), 65-67.   DOI:10.1007/BF01737559
  17. E. M. Slade: What does an oligopoly maximize? J. Ind. Econ. 42 (1994), 45-61.   DOI:10.1177/216507999404200110
  18. S. V. Macua, S. Zazo and J. Zazo: Learning parametric closed-Loop policies for Markov potential games.    CrossRef
  19. S. Zazo, J. Zazo and M. Sánchez-Fernández: A control theoretic approach to solve a constrained uplink power dynamic game. In: IEEE, 22nd European Signal Processing Conference on (EUSIPCO) 2014, pp. 401-405.   CrossRef
  20. S. Zazo, S. Valcarcel, M. Sánchez-Fernández and J. Zazo: Dynamic potential games with constraints: fundamentals and applications in communications. IEEE Trans. Signal Proc. 64 (2016), 3806-3821.   DOI:10.1109/TSP.2016.2551693