Paper deep dive
POMDPPlanners: Open-Source Package for POMDP Planning
Yaacov Pariente, Vadim Indelman
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 7/20/2026, 1:54:24 PM
Summary
The paper introduces POMDPPlanners, an open-source Python package designed for the empirical evaluation of Partially Observable Markov Decision Process (POMDP) planning algorithms. It addresses gaps in existing toolkits by providing a unified framework that integrates state-of-the-art planning algorithms, benchmark environments with safety-critical variants, automated hyperparameter optimization via Optuna, and persistent caching with failure recovery. The package supports scalable, reproducible research on decision-making under uncertainty, particularly in risk-sensitive settings, and includes features like parallel simulation and MLflow logging for comprehensive experiment management.
Entities (14)
Relation Signals (13)
POMDPPlanners → implements → POMDP Planning Algorithms
confidence 95% · We present POMDPPlanners, an open-source Python package for empirical evaluation of Partially Observable Markov Decision Process (POMDP) planning algorithms.
POMDPPlanners → implementedin → Python
confidence 92% · an open-source Python package
POMDPPlanners → developedby → Yaacov Pariente
confidence 90% · Yaacov Pariente... Faculty of Mathematics Technion
POMDPPlanners → developedby → Vadim Indelman
confidence 90% · Vadim Indelman... Faculty of Aerospace Engineering Technion
POMDPPlanners → uses → Optuna
confidence 90% · automated hyperparameter optimization via Optuna
POMDPPlanners → uses → MLflow
confidence 88% · recorded as MLflow runs, enabling cross-experiment comparison
POMDPPlanners → includes → ConstrainedZero
confidence 85% · The package provides the following online planners: ... ConstrainedZero
POMDPPlanners → includes → BetaZero
confidence 85% · The package provides the following online planners: ... BetaZero
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We present POMDPPlanners, an open-source Python package for empirical evaluation of Partially Observable Markov Decision Process (POMDP) planning algorithms. The package integrates state-of-the-art planning algorithms, a suite of benchmark environments with safety-critical variants, automated hyperparameter optimization via Optuna, persistent caching with failure recovery, and configurable parallel simulation -- reducing the overhead of extensive simulation studies. POMDPPlanners is designed to enable scalable, reproducible research on decision-making under uncertainty, with particular emphasis on risk-sensitive settings where standard toolkits fall short.
Tags
Links
- Source: https://arxiv.org/abs/2602.20810v1
- Canonical: https://arxiv.org/abs/2602.20810v1
Trouble viewing inline? Open PDF directly →
Full Text
12,372 characters extracted from source content.
Expand or collapse full text
POMDPPlanners: Open-Source Package for POMDP Planning Yaacov Parienteyaacovp@campus.technion.ac.il Faculty of Mathematics Technion - Israel Institute of Technology Haifa 3200003, Israel Vadim Indelmanvadim.indelman@technion.ac.il Stephen B. Klein Faculty of Aerospace Engineering Faculty of Data and Decision Sciences Technion - Israel Institute of Technology Haifa 3200003, Israel Abstract We present POMDPPlanners, an open-source Python package for empirical evaluation of Partially Observable Markov Decision Process (POMDP) planning algorithms. The pack- age integrates state-of-the-art planning algorithms, a suite of benchmark environments with safety-critical variants, automated hyperparameter optimization via Optuna, per- sistent caching with failure recovery, and configurable parallel simulation—reducing the overhead of extensive simulation studies. POMDPPlanners is designed to enable scalable, reproducible research on decision-making under uncertainty, with particular emphasis on risk-sensitive settings where standard toolkits fall short. Keywords: POMDP planning, risk-averse planning, open-source software, Python 1 Introduction Partially Observable Markov Decision Processes (POMDPs) model sequential decision- making under uncertainty, with applications spanning robotics, autonomous navigation, and medical decision-making. Conducting robust, reproducible simulation studies, however, remains practically challenging: researchers must manually integrate planners, belief rep- resentations, and environments from disparate sources, implement hyperparameter search routines, and manage interrupted campaigns—all without standardized tooling in Python. Existing frameworks include C++ toolkits (Bargiacchi et al., 2020; Oliehoek et al., 2017) and the Julia-based POMDPs.jl (Egorov et al., 2017), while Python alternatives (Emami et al., 2015; Zheng and Tellex, 2020) implement only basic solvers and do not support contin- uous spaces. We present POMDPPlanners, an open-source Python package that addresses these gaps with a unified, extensible framework for POMDP planning research. Table 1 summarizes the key differences. ©2025 Pariente and Indelman. License: C-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. arXiv:2602.20810v1 [cs.AI] 24 Feb 2026 Pariente and Indelman Table 1: Comparison of POMDP planning frameworks.✓ = supported; – = not sup- ported. All frameworks support discrete spaces; Cont. = continuous spaces; Hyp. opt. = automated hyperparameter optimization; Safety Envs. = safety-critical environments with violation metrics; Modern Planners = includes planners be- yond POMCP appearing in post-2018 POMDP literature (e.g., POMCPOW, PFT- DPW, BetaZero). † POMDPs.jl provides a solver interface; planners are separate community packages. PackageLanguage Cont. Modern Planners Hyp. opt. Safety Envs. Parallel Caching AI-ToolboxC++– MADPC++– POMDPs.jlJulia✓ † –✓– pomdpyPython– pomdppyPython– POMDPPlannersPython✓ Direct Evaluation Optimize-and-Evaluate (Optuna) Task Manager & Persistent Cache EnvironmentBelief Policy Results & MLflow Logging dynamics Figure 1: Architectureof POMDPPlanners. Workflows feed into a shared Task Man- ager with persistent caching.Dashed arrowsindicate data flow between abstractions;out- puts are logged via MLflow. 2 Architecture 2.1 Core Abstractions Environment defines POMDP dynamics via state transition and observation models, re- ward function, terminal condition, and initial distributions; a SpaceInfo dataclass specifies discrete, continuous, or mixed spaces, enabling runtime compatibility checking. Belief maintains a distribution over states and performs Bayesian updates using the environ- ment’s models; supported representations include weighted and unweighted particle filters, Gaussian, and Gaussian mixture beliefs. Policy exposes an action(belief) method re- turning an action together with a PolicyRunData structure of per-step diagnostics (e.g., expanded nodes, visit counts in MCTS-based planners), supporting post-hoc analysis of planner behavior. 2.2 Workflows and Experiment Management POMDPPlanners supports two complementary workflows: direct evaluation, which re- turns aggregated statistics (mean return, Conditional Value-at-Risk (CVaR), Value-at-Risk (VaR), confidence intervals), and optimize-and-evaluate, which runs an Optuna (Akiba et al., 2019) search over each planner’s hyperparameter space before forwarding the best 2 POMDPPlanners configuration to evaluation. Both workflows share a fault-tolerant task manager: each simulation is keyed by a SHA-256 hash of its full specification (environment parameters, policy, belief, seed), so cache hits return instantly and interrupted experiments resume au- tomatically. The task manager is backed by a pluggable execution layer supporting Joblib (multi-core), Dask (distributed, including multi-machine clusters via a scheduler address), and PBS (HPC batch queues via dask-jobqueue); switching backends requires only chang- ing a single configuration object. Per-episode returns, safety metrics, hyperparameters, and PolicyRunData diagnostics are recorded as MLflow runs, enabling cross-experiment com- parison via the MLflow UI or programmatic queries. The framework optionally caches a GIF of each episode’s trajectory alongside numerical results, since aggregate statistics can be misleading—e.g., a zero obstacle-hit rate may simply reflect an agent that never moved. The package exposes flexible interfaces for custom planners, environments, and metrics. 3 Benchmark Environments and Algorithms 3.1 Benchmark Environments The package includes nine benchmark environments: Tiger, LightDark, RockSample, Cart- Pole, MountainCar, Push, LaserTag, SafetyAnt, and PacMan. LightDark and LaserTag each provide discrete and continuous variants. Most environments incorporate configurable dangerous areas with associated penalty rewards; the specific mechanisms vary: LightDark exposes stochastic obstacle hits via obstacle hitprobability and an isobstaclehitterminal flag; LaserTag and RockSample provide configurable dangerousareas with tunable penalty magnitudes; SafetyAnt enforces a hard velocity safety constraint with a critical-violation termination threshold. Risk-enriched environments report dedicated safety metrics (viola- tion rates, total counts) alongside standard return statistics. 3.2 Planning Algorithms The package provides the following online planners: POMCP (Silver and Veness, 2010), POMCP-DPW, POMCPOW (Sunberg and Kochenderfer, 2018), SparsePFT, PFT-DPW (Sunberg and Kochenderfer, 2018), Sparse Sampling (Kearns et al., 2002), DiscreteAction- Sequences (open-loop baseline), BetaZero (Moss et al., 2024a), and the following risk-averse planners for safe planning: ConstrainedZero (Moss et al., 2024b), ICVaR Sparse Sampling, ICVaR POMCPOW, and ICVaR PFT-DPW (Pariente and Indelman, 2026). Each plan- ner exposes a standardized hyperparameter interface compatible with Optuna search. The selection covers state-of-the-art planners commonly used as baselines in recent POMDP literature. 4 Usage Example The listing illustrates direct evaluation of POMCPOW and PFT-DPW on ContinuousLightDarkPOMDP- DiscreteActions. Two planners and a vectorized particle-filter belief are passed to LocalSimulationsAPI, which runs parallelized simulations and returns aggregated statistics. For hyperparameter optimization, run optimizeandevaluate() accepts HyperParameterRunParams with Op- tuna search ranges and forwards the best configuration to evaluation automatically. 3 Pariente and Indelman from POMDPPlanners.environments import ContinuousLightDarkPOMDPDiscreteActions from POMDPPlanners.planners.mcts_planners.pomcpow import POMCPOW from POMDPPlanners.planners.mcts_planners.pft_dpw import PFT_DPW from POMDPPlanners.utils.action_samplers import DiscreteActionSampler from POMDPPlanners.utils.belief_factory import create_environment_belief from POMDPPlanners.simulations.simulation_apis. local_simulations_api import ( LocalSimulationsAPI) from POMDPPlanners.core.simulation import EnvironmentRunParams env = ContinuousLightDarkPOMDPDiscreteActions(discount_factor =0.95) sampler = DiscreteActionSampler(env.get_actions ()) pomcpow = POMCPOW(environment=env , discount_factor =0.95 , depth=10, exploration_constant =10.0, k_o=2.0, k_a=2.0, alpha_o =0.5, alpha_a =0.5, n_simulations =500, action_sampler=sampler , name="POMCPOW") pft_dpw = PFT_DPW(environment=env , discount_factor =0.95 , depth=10, exploration_constant =10.0, n_simulations =500, action_sampler=sampler , name="PFT_DPW") belief = create_environment_belief(env , n_particles =200) api = LocalSimulationsAPI () _, stats = api.run_multiple_environments_and_policies( environment_run_params =[ EnvironmentRunParams( environment=env , belief=belief , policies =[pomcpow , pft_dpw], num_episodes =100, num_steps =30)], alpha =0.1, experiment_name="LightDark_Evaluation") The call returns aggregated statistics including mean return, CVaR, goal rate, and safety metrics, alongside MLflow-logged per-episode data. Conclusions. We have presented POMDPPlanners, an open-source Python framework that unifies state-of-the-art POMDP planners, risk-enriched benchmark environments, and automated experimentation tooling, lowering the barrier to rigorous simulation studies so that researchers can focus on algorithmic contributions rather than infrastructure. Availability. Source code (MIT license), documentation, and Jupyter notebook exam- ples are available at https://github.com/yaacovpariente/POMDPPlanners. The package requires Python 3.10+ and is installed via pip install POMDPPlanners. References Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In The 25th ACM 4 POMDPPlanners SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2623– 2631, 2019. Eugenio Bargiacchi, Diederik M. Roijers, and Ann Now ̃ A©. Ai-toolbox: A c++ library for reinforcement learning and planning (with python bindings). Journal of Machine Learning Research, 21(102):1–12, 2020. URL http://jmlr.org/papers/v21/18-402.html. Maxim Egorov, Zachary N. Sunberg, Edward Balaban, Tim A. Wheeler, Jayesh K. Gupta, and Mykel J. Kochenderfer. Pomdps.jl: A framework for sequential decision making under uncertainty. Journal of Machine Learning Research, 18(26):1–5, 2017. URL http: //jmlr.org/papers/v18/16-300.html. Patrick Emami, Alan J. Hamlet, and Carl Crane. Pomdpy: An extensible framework for implementing pomdps in python, 2015. Michael Kearns, Yishay Mansour, and Andrew Y Ng. A sparse sampling algorithm for near- optimal planning in large markov decision processes. Machine learning, 49(2):193–208, 2002. Robert J. Moss, Anthony Corso, Jef Caers, and Mykel J. Kochenderfer. BetaZero: Belief- State Planning for Long-Horizon POMDPs using Learned Approximations. In Reinforce- ment Learning Conference (RLC), 2024a. Robert J. Moss, Arec Jamgochian, Johannes Fischer, Anthony Corso, and Mykel J. Kochen- derfer. ConstrainedZero: Chance-Constrained POMDP Planning Using Learned Proba- bilistic Failure Surrogates and Adaptive Safety Constraints. In International Joint Con- ference on Artificial Intelligence (IJCAI), 2024b. Frans A. Oliehoek, Matthijs T. J. Spaan, Bas Terwijn, Philipp Robbel, and Jo ̃ A£o V. Messias. The madp toolbox: An open source library for planning and learning in (multi- )agent systems. Journal of Machine Learning Research, 18(89):1–5, 2017. URL http: //jmlr.org/papers/v18/17-156.html. Yaacov Pariente and Vadim Indelman. Online risk-averse planning in POMDPs using iter- ated CVaR value function, 2026. David Silver and Joel Veness. Monte-carlo planning in large pomdps. Advances in neural information processing systems, 23, 2010. Zachary Sunberg and Mykel Kochenderfer. Online algorithms for pomdps with continuous state, action, and observation spaces. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 28, pages 259–263, 2018. Kaiyu Zheng and Stefanie Tellex.pomdp py:A framework to build and solve pomdp problems. In ICAPS 2020 Workshop on Planning and Robotics (PlanRob), 2020. URL https://icaps20subpages.icaps-conference.org/wp-content/uploads/ 2020/10/14-PlanRob_2020_paper_3.pdf.Arxiv link:”https://arxiv.org/pdf/ 2004.10099.pdf”. 5