Kybernetika 61 no. 1, 58-78, 2025

Discounted Markov decision processes with fuzzy costs

Salvador De-Jesús-Hernández, Hugo Cruz-Suárez and Raúl Montes-de-OcaDOI: 10.14736/kyb-2025-1-0058

Abstract:

This article concerns a class of discounted Markov decision processes on Borel spaces where, in contrast with the classical framework, the cost function $\widetilde C$ is a fuzzy function of a trapezoidal type, which is determined from a classical cost function $C$ by applying an affine transformation with fuzzy coefficients. Under certain conditions ensuring that the classical (or standard) model with a cost function $C$ has an optimal stationary policy $f_{o}$ with the optimal cost $V_{o}$, it is shown that such a policy is also optimal for the fuzzy model with a cost function $\widetilde C$, and that the optimal fuzzy value $\tilde{V}_{o}$ is obtained from $V_{o}$ via the same transformation used to go from $C$ to $\widetilde C$. And these results are obtained with respect to two cases: the max-order of the fuzzy numbers and the average ranking order of the trapezoidal fuzzy numbers. Besides, a fuzzy version of the classical linear-quadratic model without restrictions is presented.

Keywords:

discounted Markov decision processes, max-order, average ranking, trapezoidal fuzzy costs

Classification:

90C40, 93C42

References:

  1. J. P. Aubin and H. Frankowska: Set-Valued Analysis. Birkhäuser, Boston 2009.   CrossRef
  2. D. Bertsekas: Dynamic programming and optimal control: Volume I. Athena Sci. (2012).   CrossRef
  3. K. Carrero-Vera, H. Cruz-Suárez and R. Montes-de-Oca: Finite-horizon and infinite-horizon Markov decision processes with trapezoidal fuzzy discounted rewards. Commun. Comput. Inf. Sci., Springer, Cham 1623 (2022), 171-192.   CrossRef
  4. K. Carrero-Vera, H. Cruz-Suárez and R. Montes-de-Oca: Markov decision processes on finite spaces with fuzzy total reward. Kybernetika 58 (2022), 2, 180-199.   DOI:10.14736/kyb-2022-2-0180
  5. H. Cruz-Suárez and R. Montes-de-Oca: An envelope theorem and some applications to discounted Markov decision processes. Math. Oper. Res. 67 (2008), 299-321.   DOI:10.1007/s00186-007-0155-z
  6. H. Cruz-Suárez, R. Montes-de-Oca and R. I. Ortega-Gutiérrez: An extended version of average Markov decision processes on discrete spaces under fuzzy environment. Kybernetika 59 (2023), 1,160-178.   DOI:10.14736/kyb-2023-1-0160
  7. H. Cruz-Suárez, R. Montes-de-Oca and R. Ortega-Gutiérrez: Deterministic discounted Markov decision processes with fuzzy rewards/costs. Fuzzy Inf. Engrg. 15 (2023), 3, 274-290.   DOI:10.26599/FIE.2023.9270020
  8. J. de Andrés-Sánchez: A systematic review of the interactions of fuzzy set theory and option pricing. Expert Syst. Appl. 223 (2023), 119868.   DOI:10.1016/j.eswa.2023.119868
  9. P. Diamond and P. Kloeden: Metric Spaces of Fuzzy Sets: Theory and Applications. World Scientific, Singapore 1994.   CrossRef
  10. J. C. Figueroa-García, G. Hernández and C. Franco: A review on history, trends and perspectives of fuzzy linear programming. Oper. Res. Perspect. 9 (2022), 100247.   DOI:10.1016/j.orp.2022.100247
  11. N. Furukawa: Parametric orders on fuzzy numbers and their roles in fuzzy optimization problems. Optimization 40 (1997), 171-192.   DOI:10.1080/02331939708844307
  12. O. Hernández-Lerma and J. B. Lasserre: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York, 1996.   CrossRef
  13. S. Kambalimath and P. C. Deka: A basic review of fuzzy logic applications in hydrology and water resources. Appl. Water Sci. 10 (2020), 8, 1-14.   DOI:10.1007/s13201-020-01276-2
  14. M. Kurano, M. Yasuda, J. Nakagami and Y. Yoshida: Markov-type fuzzy decision processes with a discounted reward on a closed interval. Eur. J. Oper. Res. 92 (1996), 3, 649-662.   DOI:10.1016/0377-2217(95)00140-9
  15. M. Kurano, M. Yasuda, J. Nakagami and Y. Yoshida: Markov decision processes with fuzzy rewards. J. Nonlinear Convex Anal. 4 (1996), 1, 105-116.   CrossRef
  16. M. Kurano, M. Hosaka, J. Song and Y. Huang: Controlled Markov set-chains with discounting. J. Appl. Prob. 35 (1998), 3, 293-302.   DOI:10.1006/cogp.1998.0683
  17. M. López-Díaz and D. A. Ralescu: Tools for fuzzy random variables: embeddings and measurabilities. Comput. Statist. Data Anal. 51 (2006), 109-114.   DOI:10.1016/j.csda.2006.04.017
  18. M. L. Puri and D. A. Ralescu: Fuzzy random variable. J. Math. Anal. Appl. 114 (1986), 402-422.   CrossRef
  19. D. Rani and T. R. Gulati: A new approach to solve unbalanced transportation problems in imprecise environment. J. Transp. Secur. 7 (2014), 3, 277-287.   DOI:10.1007/s12198-014-0143-5
  20. D. Rani, T. R. Gulati and A. Kumar: A method for unbalanced transportation problems in fuzzy environment. Sadhana 39 (2014), 3, 573-581.   DOI:10.1007/s12046-014-0243-8
  21. S. Rezvani and M. Molani: Representation of trapezoidal fuzzy numbers with shape function. Ann. Fuzzy Math. Inform. 8 (2014), 89-112.   CrossRef
  22. A. Semmouri, M. Jourhmane and Z. Belhallaj: Discounted Markov decision processes with fuzzy costs. Ann. Oper. Res. 295 (2020), 769-786.   DOI:10.1007/s10479-020-03783-6
  23. A. Syropoulos and T. Grammenos: A Modern Introduction to Fuzzy Mathematics. Wiley, New Jersey 2020.   CrossRef
  24. L. Zadeh: Fuzzy sets. Inform. Control 8 (1965), 338-353.   DOI:10.1016/S0019-9958(65)90241-X
  25. W. Zhou, D. Lou and Z. Xu: Review of fuzzy investment research considering modelling environment and element fusion. Int. J. Syst. Sci. 53 (2022), 9, 1958-1982.   DOI:10.1080/00207721.2022.2031340