Kybernetika 32 no. 2, 139-158, 1996

Time-discretization for controlled Markov processes. II. A jump and diffusion application

Nico M. van Dijk and Arie Hordijk

Abstract:

In a first Part I ([24]) a method of time-discretization was investigated in order to approximate continuous-time stochastic control problems over a finite time horizon. This approximation was based on using recursive discrete-time dynamic programming. To this end, three conditions are to be fulfilled: \begin{itemize} \item {\sl Smoothness} of the continuous-time functions \item {\sl Consistency} or convergence of the discrete-time generators \item {\sl Stability} or uniform boundedness of the discrete-time constructions. \end{itemize} In this Part II, these conditions will be verified for two practical applications: \begin{itemize} \item A {\sl controlled infinite server queue}, as example of a controlled Markov jump process \item A {\sl controlled cash-balance model}, as example of a controlled diffusion model. \end{itemize} For both applications it is shown and illustrated that the discrete-time constructions lead to a computational feasible scheme to approximate the optimal cost function as well as to construct an $\varepsilon$-optimal control.

Classification:

93E20, 90C40, 93C57, 49M25, 49K45