Kybernetika 47 no. 6, 909-929, 2011

A consumption-investment problem modelled as a discounted Markov decision process

Hugo Cruz-Suárez, Raúl Montes-de-Oca and Gabriel Zacarías


In this paper a problem of consumption and investment is presented as a model of a discounted Markov decision process with discrete-time. In this problem, it is assumed that the wealth is affected by a production function. This assumption gives the investor a chance to increase his wealth before the investment. For the solution of the problem there is established a suitable version of the Euler Equation (EE) which characterizes its optimal policy completely, that is, there are provided conditions which guarantee that a policy is optimal for the problem if and only if it satisfies the EE. The problem is exemplified in two particular cases: for a logarithmic utility and for a Cobb-Douglas utility. In both cases explicit formulas for the optimal policy and the optimal value function are supplied.


discounted Markov decision processes, differentiable value function, differentiable optimal policy, stochastic Euler equation, consumption and investment problems


93E12, 62A10


  1. C. D. Aliprantis and O. Burkinshaw: Principles of Real Analysis. Academic Press, San Diego 1998.   CrossRef
  2. G. M. Angelatos: Uninsured idiosyncratic investment risk and aggregate saving. Rev. Econom. Dynam. 10 (2007), 1-30.   CrossRef
  3. K. J. Arrow: A note on uncertainty and discounting in models of economic growth. J. Risk Unc. 38 (2009), 87-94.   CrossRef
  4. D. P. Bertsekas: Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Belmont 1987.   CrossRef
  5. W. Brock and L. Mirman: Optimal economic growth and uncertainty: the discounted case. J. Econom. Theory 4 (1972), 479-513.   CrossRef
  6. D. Cruz-Suárez, R. Montes-de-Oca and F. Salem-Silva: Conditions for the uniqueness of optimal policies of discounted Markov decision processes. Math. Meth. Oper. Res. 60 (2004), 415-436.   CrossRef
  7. H. Cruz-Suárez and R. Montes-de-Oca: Discounted Markov control processes induced by deterministic systems. Kybernetika 42 (2006), 647-664.   CrossRef
  8. H. Cruz-Suárez and R. Montes-de-Oca: An envelope theorem and some applications to discounted Markov decision processes. Math. Meth. Oper. Res. 67 (2008), 299-321.   CrossRef
  9. E. B. Dynkin and A. A. Yushkevich: Controlled Markov Processes. Springer-Verlag, New York 1980.   CrossRef
  10. L. Epstein and S. Zin: Substitution, risk aversion, and the temporal behaviour of consumption and asset returns I: Theoretical framework. Econometrica 57 (1989), 937-969.   CrossRef
  11. A. De la Fuente: Mathematical Methods and Models for Economists. Cambridge University Press, Cambridge 2000.   CrossRef
  12. R. S. Gurkaynak: Econometric tests of asset price bubbles: taking stock. J. Econom. Surveys 22 (2008), 166-186.   CrossRef
  13. B. Heer and A. Maussner: Dynamic General Equilibrium Modelling: Computational Method and Application. Second edition, Springer-Verlag, Berlin 2005.   CrossRef
  14. O. Hernández-Lerma and J. B. Lasserre: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York 1996.   CrossRef
  15. O. Hernández-Lerma and J. B. Lasserre: Value iteration and rolling plans for Markov control processes with unbounded rewards. J. Math. Anal. Appl. 177 (1993), 38-55.   CrossRef
  16. A. Jaskiewics and A. S. Nowak: Discounted dynamic programming with unbounded returns: application to economic models. J. Math. Anal. Appl. 378 (2011), 450-462.   CrossRef
  17. R. Korn and H. Kraft: A stochastic control approach to portfolio problems with stochastic interest rates. SIAM J. Control Optim. 40 (2001), 1250-1269.   CrossRef
  18. T. Kamihigashi: Stochastic optimal growth with bounded or unbounded utility and bounded or unbounded shocks. J. Math. Econom. 43 (2007), 477-500.   CrossRef
  19. D. Levhari and T. N. Srinivasan: Optimal savings under uncertainty. Rev. Econom. Stud. 36 (1969), 153-163.   CrossRef
  20. L. Mirman and I. Zilcha: On optimal growth under uncertainty. J. Econom. Theory 2 (1975), 329-339.   CrossRef
  21. F. P. Ramsey: A Mathematical theory of saving. Econom. J. 38 (1928), 543-559.   CrossRef
  22. N. Stokey, R. Lucas and E. Prescott: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge 1989.   CrossRef