Report
State-Constrained Offline Reinforcement Learning
| العنوان: | State-Constrained Offline Reinforcement Learning |
|---|---|
| المؤلفون: | Hepburn, Charles A., Jin, Yue, Montana, Giovanni |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science Statistics |
| مصطلحات موضوعية: | Statistics - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Machine Learning |
| الوصف: | Traditional offline reinforcement learning methods predominantly operate in a batch-constrained setting. This confines the algorithms to a specific state-action distribution present in the dataset, reducing the effects of distributional shift but restricting the algorithm greatly. In this paper, we alleviate this limitation by introducing a novel framework named \emph{state-constrained} offline reinforcement learning. By exclusively focusing on the dataset's state distribution, our framework significantly enhances learning potential and reduces previous limitations. The proposed setting not only broadens the learning horizon but also improves the ability to combine different trajectories from the dataset effectively, a desirable property inherent in offline reinforcement learning. Our research is underpinned by solid theoretical findings that pave the way for subsequent advancements in this domain. Additionally, we introduce StaCQ, a deep learning algorithm that is both performance-driven on the D4RL benchmark datasets and closely aligned with our theoretical propositions. StaCQ establishes a strong baseline for forthcoming explorations in state-constrained offline reinforcement learning. |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2405.14374 |
| رقم الانضمام: | edsarx.2405.14374 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |