The e-commerce industry is quickly transforming towards more automation and technological advancements. With the growing intricacy of warehouse operations, there is a need for control systems that can efficiently handle this complexity. This study considers a Robotic Mobile Fulfillment System (RMFS), a semi-automated warehousing system. This system employs autonomous mobile robots (AMRs) to retrieve inventory racks from the storage area; this way, human activity is eliminated within the storage area itself. The fleet of robots both store and retrieve the inventory racks to either workstations, where human pickers are stationed that pick items from the racks, or replenishment stations, where depleted inventory racks can be restocked with items. An attractive characteristic of the RMFS is that it dynamically changes the positioning of the inventory racks based on the frequency of inventory rack requests and the state of their stock levels. The optimization objective considered in this study for the dynamic positioning problem of the racks within the storage area is to minimize the average cycle time of the mobile robots to perform retrieval and replenishment activities. We propose a deep reinforcement learning approach to train a decision-making agent to learn a policy for the storage assignment and replenishment of inventory racks. The learned policy is compared to the commonly used decision rules in the academic literature on this problem. The experimental results show the potential benefits of training an agent to learn a storage and replenishment policy. Cycle time improvements up to 5.4 % can be achieved over the best-performing decision rules. This research contributes to advancing the understanding of intelligent storage assignment and replenishment strategies for the real-time decision-making process within an RMFS.

中文翻译：

---

针对实时库存货架存储分配和补货问题的深度强化学习

电子商务行业正在迅速向更多的自动化和技术进步转型。随着仓库运营的日益复杂，需要能够有效处理这种复杂性的控制系统。本研究考虑了机器人移动配送系统 （RMFS），这是一种半自动化仓储系统。该系统采用自主移动机器人 （AMR） 从存储区域检索库存货架;这样，存储区域本身就消除了人类活动。机器人车队将库存货架存储和检索到工作站，人工拣选员驻扎在那里从货架上拣选物品，或者补货站，在那里可以用耗尽的库存货架补充物品。RMFS 的一个吸引人的特点是，它根据库存货架请求的频率及其库存水平的状态动态改变库存货架的位置。本研究针对存储区域内货架的动态定位问题考虑的优化目标是最小化移动机器人执行检索和补货活动的平均周期时间。我们提出了一种深度强化学习方法来训练决策代理学习库存货架的存储分配和补货策略。将学习到的策略与学术文献中关于此问题的常用决策规则进行了比较。实验结果表明，训练代理学习存储和补货策略的潜在好处。与性能最佳的决策规则相比，周期时间最多可缩短 5.4%。 这项研究有助于促进对 RMFS 中实时决策过程的智能存储分配和补货策略的理解。