Enhancing HTN planning with deep reinforcement learning for method selection

Abstract

Automated planning is a central area within Artificial Intelligence (AI), enabling intelligent behavior in domains such as cloud computing, autonomous systems, context-aware activity recognition, and smart environments. Hierarchical Task Network (HTN) planning, which decomposes complex tasks into simpler subtasks using predefined methods, has proven effective in such structured domains. However, its performance is often constrained by static method selection strategies that lack adaptability to varying planning contexts. To address this limitation, this thesis proposes a neuro-symbolic framework that integrates HTN planning with Deep Reinforcement Learning (DRL), combining the strengths of symbolic reasoning and data-driven learning. Among the available DRL algorithms, Deep Q-Learning (DQL) is particularly suitable due to its off-policy nature, batch-efficient learning, and robust generalization across symbolic planning states. These characteristics align well with deterministic and hierarchical planners, enabling offline learning from curated datasets without requiring interactive exploration. The proposed integration introduces a learning-based decision layer that improves adaptability while preserving the reproducibility and determinism of the underlying planner. The effectiveness of this approach is demonstrated through a comprehensive evaluation across planning efficiency, memory consumption, and plan quality. Results highlight the potential of reinforcement learning to enhance classical HTN systems and support intelligent decision-making in complex, structured environments.