Attention-Enabled Hierarchical Multi-Agent Deep Reinforcement Learning for Coordinated Voltage Control in Active Distribution Systems

The increasing integration of distributed renewable energy resources (DRES) is transforming active distribution systems (ADS), introducing challenges in maintaining voltage levels and minimising power losses. Effective volt/VAR control (VVC) requires coordinated operation of conventional devices, such as switched capacitors, alongside modern inverter-based resources. Although model-based optimisation methods can achieve this coordination, their practical use is limited by high computational demands and dependence on accurate grid models. Model-free deep reinforcement learning (DRL) offers a promising alternative but often suffers from instability and limited interpretability, particularly in multiagent settings based on DDPG architectures. This paper proposes an attention-enabled hierarchical multiagent twin delayed deep deterministic policy gradient (AE-HMATD3) framework for coordinated VVC. The approach employs a two-layer structure to manage fast- and slow-responding devices at different temporal scales, improving control alignment and system reliability. Learning stability is enhanced through twin critics and delayed policy updates, whereas a Gumbel–SoftMax method enables unified handling of discrete and continuous control actions. An attention mechanism further improves coordination and scalability by capturing spatial dependencies. Validation on IEEE 33- and 118-bus systems demonstrates that the proposed approach reduces power loss by 12%–44% and achieves 84%–99% fewer voltage violations compared to hierarchical multiagent DRL baselines.