Integrating simulation and reinforcement learning for optimized working capital management in supply chains

Effective working capital management in supply chains requires the joint coordination of physical and financial flows. While existing literature on simulation-integrated reinforcement learning has primarily focused on inventory dynamics, this study extends the scope to include financial dynamics across supply chain tiers. We propose a framework that integrates discrete-event simulation (DES) with deep reinforcement learning (DRL) to optimize both inventory and financial management in a multi-echelon supply chain. A Proximal Policy Optimization (PPO) algorithm is used to train an agent within the simulated environment, enabling it to learn adaptive policies for inventory replenishment, production planning, and cash collection. Comparative results against a Genetic Algorithm (GA)-based benchmark demonstrate that the DRL agent outperforms heuristic policies in terms of convergence stability, cumulative rewards, and responsiveness to stochastic demand. The findings highlight the potential of simulation-integrated DRL frameworks to improve coordination between financial and operational decisions in supply chains. Practically, the framework can be embedded in digital twins to support real-time decision-making in supply chains and offers actionable insights for managers seeking to improve working capital efficiency through adaptive policies.