Kybernetika 57 no. 3, 493-512, 2021

Partially observable queueing systems with controlled service rates under a discounted optimality criterion

Yofre H. García, Saul Diaz-Infante and J. Adolfo Minjárez-SosaDOI: 10.14736/kyb-2021-3-0493

Abstract:

We are concerned with a class of $GI/GI/1$ queueing systems with controlled service rates, in which the waiting times are only observed when they take zero value. Applying a suitable filtering process, we show the existence of optimal control policies under a discounted optimality criterion.

Keywords:

queueing models, partially observable systems, discounted criterion, optimal policies

Classification:

90C39, 90B22

paper.pdf

References:

  1. A. Bensoussan, M. Cakanyildirim and S. P. Sethi: Partially observed inventory systems: the case of zero-balance walk. SIAM J. Control Optim. 46 (2007), 176-209.   DOI:10.1137/040620321
  2. N. Bauerle and U. Rieder: Markov Decision Processes with Applications to Finance. Springer, Berlin 2011.   CrossRef
  3. D- P. Bertsekas and S. E. Shreve: Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York 1978.   CrossRef
  4. E. B. Dynkin and A. A. Yushkevich: Controlled Markov Processes. Springer-Verlag, New York 1979.   CrossRef
  5. R. J. Elliott, L. Aggoun and J. B. Moore: Hidden Markov Models: Estimation and Control. Springer-Verlag, New York 1994.   CrossRef
  6. E. Gordienko and O. Hernandez-Lerma: Average cost Markov control processes with weighted norms: value iteration. Appl. Math. 23 (1995), 219--237.   DOI:10.4064/am-23-2-219-237
  7. E. Gordienko and J. A. Minjarez-Sosa: Adaptive control for discrete-time Markov processes with unbounded costs: discounted criterion. Kybernetika 34 (1998), 217--234.   CrossRef
  8. O. Hernandez-Lerma: Adaptive Markov Control Processes. Springer-Verlag, New York 1989.   CrossRef
  9. O. Hernandez-Lerma and M. Munoz-de-Ozak: Discrete-time Markov control processes with discounted unbounded costs: optimality criteria. Kybernetika 28 (1992), 191-221.   CrossRef
  10. M. Y. Kitaev and V. V. Rykov: Controlled Queueing Systems. CRC Press, Boca Raton 1995.   CrossRef
  11. D. V. Lindley: The theory of queues with a single server. Proc. Cambridge Philos Soc. 48 (1952), 277-289.   DOI:10.1017/S0305004100027638
  12. N. Hilgert and J. A. Minjarez-Sosa: Adaptive policies for time-varying stochastic systems under discounted criterion. Math. Methods Oper. Res. 54 (2001), 491-505.   DOI:10.1007/s001860100170
  13. J. A. Minjarez-Sosa: Approximation and estimation in Markov control processes under discounted criterion. Kybernetika 40 (2004), 681-690.   CrossRef
  14. J. A. Minjarez-Sosa: Markov control models with unknown random state-action-dependent discount factors. TOP 23 (2015), 743-772.   DOI:10.1007/s11750-015-0360-5
  15. W. J. Runggaldier and L. Stettner: Approximations of Discrete Time Partially Observed Control Problems. Appl. Math. Monographs CNR 6, Giardini, Pisa 1994.   CrossRef
  16. L. I. Sennott: Stochastic Dynamic Programming and the Control of Queueing Systems. Wiley, New York 1999.   CrossRef
  17. C. Striebel: Optimal Control of Discrete Time Stochastic Systems. Lecture Notes Econ. Math. Syst. 110, Springer-Verlag, Berlin 1975.   CrossRef
  18. A. A. Yushkevich: Reduction of a controlled Markov model with incomplete data to a problem with complete information in the case of Borel state and control spaces. Theory Probab. Appl.21 (1976), 153-158.   DOI:10.1137/1121014