Kybernetika 59 no. 1, 160-178, 2023

An extended version of average Markov decision processes on discrete spaces under fuzzy environment

Hugo Cruz-Suárez, Raúl Montes-de-Oca and R. Israel Ortega-GutiérrezDOI: 10.14736/kyb-2023-1-0160

Abstract:

The article presents an extension of the theory of standard Markov decision processes on discrete spaces and with the average cost as the objective function which permits to take into account a fuzzy average cost of a trapezoidal type. In this context, the fuzzy optimal control problem is considered with respect to two cases: the max-order of the fuzzy numbers and the average ranking order of the trapezoidal fuzzy numbers. Each of these cases extends the standard optimal control problem, and for each of them the optimal solution is related to a suitable standard optimal control problem, and it is obtained that (i) the optimal policy coincides with the optimal policy of this suitable standard control problem, and (ii) the fuzzy optimal value function is of a trapezoidal shape. Two models: a queueing system and a machine replacement problem are provided in order to examplify the theory given.

Keywords:

Markov decision process, average criterion, trapezoidal fuzzy cost, max-order, average ranking

Classification:

90C40, 93C40

paper.pdf

References:

  1. A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Gosh and S. I. Marcus: Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optim. 32 (1993), 2, 282-344.   DOI:10.1137/0331018
  2. K. Carrero-Vera, H. Cruz-Suárez and R. Montes-de-Oca: Discounted Markov decision processes with fuzzy rewards induced by non-fuzzy systems. In: Proc. 10th International Conference on Operations Research and Enterprise Systems ICORES 2021, pp. 49-59.   CrossRef
  3. K. Carrero-Vera, H. Cruz-Suárez and R. Montes-de-Oca: Markov decision proceses on finite spaces with fuzzy total reward. Kybernetika 58 (2022), 2, 180-199.   DOI:10.14736/kyb-2022-2-0180
  4. S. J. Chen and S. M. Chen: Fuzzy risk analysis based on the ranking of generalized trapezoidal fuzzy numbers. Appl. Intell. 26 (2007), 1, 1-11.   DOI:10.1007/s10489-006-0003-5
  5. Y. L. Chung and Z. N. Tsai: A quantized water-filling packet scheduling scheme for downlink transmissions in LTE-advanced systems with carrier aggregation. In: SoftCOM 2010, 18th International Conference on Software, Telecommunications and Computer Networks IEEE (2010), pp. 275-279.   CrossRef
  6. P. Diamond and P. Kloeden: Metric Spaces of Fuzzy Sets: Theory and Applications. World Scientific, Singapore 1994.   CrossRef
  7. A. Ebrahimnejad: A simplified new approach for solving fuzzy transportation problems with generalized trapezoidal fuzzy numbers. Appl. Soft Comput. 19 (2014), 171-176.   DOI:10.1016/j.asoc.2014.01.041
  8. N. Furukawa: Parametric orders on fuzzy numbers and their roles in fuzzy optimization problems. Optimization 40 (1997), 171-192.   DOI:10.1080/02331939708844307
  9. O. Hernández-Lerma and J. B. Lasserre: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York, 1996.   CrossRef
  10. M. Kageyama: Credibilistic Markov decision processes: the average case. J. Comput. Appl. Math. 224 (2009), 1, 140-145.   DOI:10.1016/j.cam.2008.04.035
  11. A. Kaur and A. Kumar: A new approach for solving fuzzy transportation problems using generalized trapezoidal fuzzy numbers. Appl. Soft Comput. 12 (2012), 3, 1201-1213.   DOI:10.1016/j.asoc.2011.10.014
  12. E. Konstantin, E. Avrachenkov and E. Sanchez: Fuzzy Markov chains and decision-making. Fuzzy Optim. Decis. Making 1 (2002), 12, 143-159.   CrossRef
  13. M. Kurano, M. Yasuda, J. Nakagami and Y. Yoshida: A fuzzy treatment of uncertain Markov decision processes: average case. In: Proc. ASSM2000 International Conference on Applied Stochastic System Modeling, Kyoto 2000, pp. 148-157.   CrossRef
  14. M. Kurano, M. Yasuda, J. Nakagami and Y. Yoshida: Markov-type fuzzy decision processes with a discounted reward on a closed interval. Eur. J. Oper. Res. 92 (1996), 3, 649-662.   DOI:10.1016/0377-2217(95)00140-9
  15. M. Kurano, M. Yasuda, J. Nakagami and Y. Yoshida: Markov decision processes with fuzzy rewards. J. Nonlinear Convex Anal. 4 (1996), 1, 105-116.   CrossRef
  16. M. Kurano, M. Yasuda, J. Nakagami and Y. Yoshida: Perceptive evaluation for the optimal discounted reward in Markov decision processes. In: International Conference on Modeling Decisions for Artificial Intelligence, Springer 2005, pp. 283-293.   CrossRef
  17. M. Kurano, M. Yasuda, J. Nakagami and Y. Yoshida: A fuzzy approach to Markov decision processes with uncertain transition probabilities. Fuzzy Sets and Systems 157 (2006), 19, 2674-2682.   DOI:10.1016/j.fss.2004.10.023
  18. M. López-Díaz and D. A. Ralescu: Tools for fuzzy random variables: embeddings and measurabilities. Comput. Statist. Data Anal. 51 (2006), 109-114.   DOI:10.1016/j.csda.2006.04.017
  19. M. L. Puri and D. A. Ralescu: Fuzzy random variable. J. Math. Anal. Appl. 114 (1986), 402-422.   DOI:10.1016/0022-247x(86)90093-4
  20. M. Puterman: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, 1994.   CrossRef
  21. D. Rani and T. R. Gulati: A new approach to solve unbalanced transportation problems in imprecise environment. J. Transp. Secur. 7 (2014), 3, 277-287.   DOI:10.1007/s12198-014-0143-5
  22. D. Rani, T. R. Gulati and A. Kumar: A method for unbalanced transportation problems in fuzzy environment. Sadhana 39 (2014),3, 573-581.   DOI:10.1007/s12046-014-0243-8
  23. S. Rezvani and M. Molani: Representation of trapezoidal fuzzy numbers with shape function. Ann. Fuzzy Math. Inform. 8 (2014), 89-112.   CrossRef
  24. S. Ross: Applied Probability Models with Optimization Applications. Holden Day, 1996.   CrossRef
  25. A. Semmouri, M. Jourhmane and Z. Belhallaj: Discounted Markov decision processes with fuzzy costs. Ann. Oper. Res. 295 (2020), 769-786.   DOI:10.1007/s10479-020-03783-6
  26. L. Sennott: Stochastic Dynamic Programming and Control of Queueing Systems. Systems. Wiley, New York 1999.   CrossRef
  27. A. Syropoulos and T. Grammenos: A Modern Introduction to Fuzzy Mathematics. Wiley, New Jersey 2020.   CrossRef
  28. J. Wang, X. Ma, Z. Xu and J. Zhan: Three-way multi-attribute decision making under hesitant fuzzy environments. Inform. Sci. 552 (2021), 328-351.   DOI:10.1016/j.ins.2020.12.005
  29. L. Zadeh: Fuzzy sets. Inform. Control 8 (1965), 338-353.   DOI:10.1016/S0019-9958(65)90241-X