Résumé
Control theory plays a pivotal role in understanding and optimizing the
behavior of complex dynamical systems across various scientific and engineering
disciplines. Two key frameworks that have emerged for modeling and solving
control problems in stochastic systems are piecewise deterministic Markov
processes (PDMPs) and Markov decision processes (MDPs). Each framework has its
unique strengths, and their intersection offers promising opportunities for
tackling a broad class of problems, particularly in the context of impulse
controls and decision-making in complex systems.
The relationship between PDMPs and MDPs is a natural subject of exploration,
as embedding impulse control problems for PDMPs into the MDP framework could
open new avenues for their analysis and resolution. Specifically, this
integration would allow leveraging the computational and theoretical tools
developed for MDPs to address the challenges inherent in PDMPs. On the other
hand, PDMPs can offer a versatile and simple paradigm to model continuous time
problems that are often described as discrete-time MDPs parametrized by complex
transition kernels. This transformation has the potential to bridge the gap
between the two frameworks, enabling solutions to previously intractable
problems and expanding the scope of both fields. This paper presents a
comprehensive review of two research domains, illustrated through a recurring
medical example. The example is revisited and progressively formalized within
the framework of thevarious concepts and objects introduced