Résumé
We introduce a Bayes-adaptive framework for controlling Piecewise-Deterministic Markov Processes (PDMPs) under partial observability and uncertain parameters.PDMPs form a class of continuous-time Markov processes that capture hybrid (discrete–continuous) dynamics, allowing highly flexible modeling with a small number of interpretable parameters.We cast the problem as a hybrid state-space Bayes-Adaptive Partially Observable Markov Decision Process (BAPOMDP), which accounts for model uncertainty offline, without requiring prior interaction with the system. This BAPOMDP can be seen as a higher-dimensional hybrid Partially Observable Markov Decision Process (POMDP).Because computing optimal policies in such hybrid state-space POMDPs is intractable, we rely on simulation-based deep reinforcement learning algorithms to obtain effective solutions.The approach is demonstrated in a medical patient follow-up scenario, where numerical experiments highlight the feasibility of applying this framework in realistic settings with partial observability and uncertain dynamics.