Kybernetika 47 no. 6, 909-929, 2011

A consumption-investment problem modelled as a discounted Markov decision process

Hugo Cruz-Suárez, Raúl Montes-de-Oca and Gabriel Zacarías

Abstract:

In this paper a problem of consumption and investment is presented as a model of a discounted Markov decision process with discrete-time. In this problem, it is assumed that the wealth is affected by a production function. This assumption gives the investor a chance to increase his wealth before the investment. For the solution of the problem there is established a suitable version of the Euler Equation (EE) which characterizes its optimal policy completely, that is, there are provided conditions which guarantee that a policy is optimal for the problem if and only if it satisfies the EE. The problem is exemplified in two particular cases: for a logarithmic utility and for a Cobb-Douglas utility. In both cases explicit formulas for the optimal policy and the optimal value function are supplied.

Keywords:

discounted Markov decision processes, differentiable value function, differentiable optimal policy, stochastic Euler equation, consumption and investment problems

Classification:

93E12, 62A10

paper.pdf

References:

  1. C. D. Aliprantis and O. Burkinshaw: Principles of Real Analysis. Academic Press, San Diego 1998.   CrossRef
  2. G. M. Angelatos: Uninsured idiosyncratic investment risk and aggregate saving. Rev. Econom. Dynam. 10 (2007), 1-30.   CrossRef
  3. K. J. Arrow: A note on uncertainty and discounting in models of economic growth. J. Risk Unc. 38 (2009), 87-94.   CrossRef
  4. D. P. Bertsekas: Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Belmont 1987.   CrossRef
  5. W. Brock and L. Mirman: Optimal economic growth and uncertainty: the discounted case. J. Econom. Theory 4 (1972), 479-513.   CrossRef
  6. D. Cruz-Suárez, R. Montes-de-Oca and F. Salem-Silva: Conditions for the uniqueness of optimal policies of discounted Markov decision processes. Math. Meth. Oper. Res. 60 (2004), 415-436.   CrossRef
  7. H. Cruz-Suárez and R. Montes-de-Oca: Discounted Markov control processes induced by deterministic systems. Kybernetika 42 (2006), 647-664.   CrossRef
  8. H. Cruz-Suárez and R. Montes-de-Oca: An envelope theorem and some applications to discounted Markov decision processes. Math. Meth. Oper. Res. 67 (2008), 299-321.   CrossRef
  9. E. B. Dynkin and A. A. Yushkevich: Controlled Markov Processes. Springer-Verlag, New York 1980.   CrossRef
  10. L. Epstein and S. Zin: Substitution, risk aversion, and the temporal behaviour of consumption and asset returns I: Theoretical framework. Econometrica 57 (1989), 937-969.   CrossRef
  11. A. De la Fuente: Mathematical Methods and Models for Economists. Cambridge University Press, Cambridge 2000.   CrossRef
  12. R. S. Gurkaynak: Econometric tests of asset price bubbles: taking stock. J. Econom. Surveys 22 (2008), 166-186.   CrossRef
  13. B. Heer and A. Maussner: Dynamic General Equilibrium Modelling: Computational Method and Application. Second edition, Springer-Verlag, Berlin 2005.   CrossRef
  14. O. Hernández-Lerma and J. B. Lasserre: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York 1996.   CrossRef
  15. O. Hernández-Lerma and J. B. Lasserre: Value iteration and rolling plans for Markov control processes with unbounded rewards. J. Math. Anal. Appl. 177 (1993), 38-55.   CrossRef
  16. A. Jaskiewics and A. S. Nowak: Discounted dynamic programming with unbounded returns: application to economic models. J. Math. Anal. Appl. 378 (2011), 450-462.   CrossRef
  17. R. Korn and H. Kraft: A stochastic control approach to portfolio problems with stochastic interest rates. SIAM J. Control Optim. 40 (2001), 1250-1269.   CrossRef
  18. T. Kamihigashi: Stochastic optimal growth with bounded or unbounded utility and bounded or unbounded shocks. J. Math. Econom. 43 (2007), 477-500.   CrossRef
  19. D. Levhari and T. N. Srinivasan: Optimal savings under uncertainty. Rev. Econom. Stud. 36 (1969), 153-163.   CrossRef
  20. L. Mirman and I. Zilcha: On optimal growth under uncertainty. J. Econom. Theory 2 (1975), 329-339.   CrossRef
  21. F. P. Ramsey: A Mathematical theory of saving. Econom. J. 38 (1928), 543-559.   CrossRef
  22. N. Stokey, R. Lucas and E. Prescott: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge 1989.   CrossRef