Abstract:

In a first Part I ([24]) a method of time-discretization was investigated in order to approximate continuous-time stochastic control problems over a finite time horizon. This approximation was based on using recursive discrete-time dynamic programming. To this end, three conditions are to be fulfilled: \begin{itemize} \item {\sl Smoothness} of the continuous-time functions \item {\sl Consistency} or convergence of the discrete-time generators \item {\sl Stability} or uniform boundedness of the discrete-time constructions. \end{itemize} In this Part II, these conditions will be verified for two practical applications: \begin{itemize} \item A {\sl controlled infinite server queue}, as example of a controlled Markov jump process \item A {\sl controlled cash-balance model}, as example of a controlled diffusion model. \end{itemize} For both applications it is shown and illustrated that the discrete-time constructions lead to a computational feasible scheme to approximate the optimal cost function as well as to construct an $\varepsilon$-optimal control.