Kybernetika 45 no. 5, 716-736, 2009

The Risk-Sensitive Poisson Equation for a Communicating Markov Chain on a Denumerable State Space

Rolando Cavazos-Cadena

Abstract:

This work concerns a discrete-time Markov chain with time-invariant transition mechanism and denumerable state space, which is endowed with a nonnegative cost function with finite support. The performance of the chain is measured by the (long-run) risk-sensitive average cost and, assuming that the state space is communicating, the existence of a solution to the risk-sensitive Poisson equation is established, a result that holds even for transient chains. Also, a sufficient criterion ensuring that the functional part of a solution is uniquely determined up to an additive constant is provided, and an example is given to show that the uniqueness result may fail when that criterion is not satisfied.

Keywords:

possibly transient Markov chains, discounted approach, first return time, uniqueness of solutions to the multiplicative Poisson equation

Classification:

90C40, 93E20, 60J05