Abstract
Hidden Markov models (HMM) are used in different fields to study the dynamics of a process that cannot be directly observed. However, in some cases, the structure of dependencies of a HMM is too simple to describe the dynamics of the hidden process. In particular, in some applications in finance and in ecology, the transition probabilities of the hidden Markov chain can also depend on the current observation. In this work, we are interested in extending the classical HMM to this situation. We refer to the extended model as the observation-driven hidden Markov model (OD-HMM). We present a complete study of the general non parametric OD-HMM with discrete and finite state spaces. We study its identifiability and the consistency of the maximum likelihood estimators. We derive the associated forward-backward equations for the E-step of the EM algorithm. The quality of the procedure is tested on simulated datasets. We illustrate the use of the model on an application focused on the study of annual plant dynamics. This work establishes theoretical and practical foundations for this framework that could be further extended to the parametric context in order to simplify estimation and to hidden semi-Markov models for more realism.