Physics-shielded deep reinforcement learning for adaptive emergency frequency control

Brown, Alinane and Bukhsh, Waqquas and Papadopoulos, Panagiotis (2026) Physics-shielded deep reinforcement learning for adaptive emergency frequency control. IEEE Transactions on Power Systems, 41 (6). pp. 4716-4729. ISSN 0885-8950 (https://doi.org/10.1109/TPWRS.2026.3706436)

[thumbnail of Brown-etal-2026-Physics-shielded-deep-reinforcement-learning-for-adaptive-emergency-frequency-control]
Preview
Text. Filename: Brown-etal-2026-Physics-shielded-deep-reinforcement-learning-for-adaptive-emergency-frequency-control.pdf
Accepted Author Manuscript
License: Creative Commons Attribution 4.0 logo

Download (5MB)| Preview

Abstract

The increasing penetration of renewable energy technologies is making power system frequency behaviour more complex. This evolving landscape challenges the reliability of conventional frequency control methods in maintaining system security, particularly the Under-Frequency Load Shedding (UFLS) scheme. Specifically, UFLS relies on predefined fixed thresholds and assumes stable patterns of generation and demand. These assumptions no longer hold in today's systems with variable renewables, changing demand, and bidirectional power flows. Although Adaptive UFLS (AUFLS) schemes have emerged, many rely on physics-agnostic Reinforcement Learning (RL), leading to inefficient training, high computational requirements—making scalability in large networks challenging—and suboptimal convergence. To address this, we propose a Physics-Shielded Deep RL (PS-DRL) method. Based on the Deep Deterministic Policy Gradient (DDPG) algorithm, the framework embeds physics-inspired constraints through a “Physics Shield” and uses real-time detection of system coherence to identify locational frequency characteristics, reducing unnecessary sampling and improving training efficiency. Validation results on the modified IEEE 39-bus and Texas 2000-bus systems demonstrate that PS-DRL converges up to 35% faster and reduces load shedding by 70% compared to existing RL approaches, while also showing good scalability.

ORCID iDs

Brown, Alinane, Bukhsh, Waqquas ORCID logoORCID: https://orcid.org/0000-0002-5765-0747 and Papadopoulos, Panagiotis ORCID logoORCID: https://orcid.org/0000-0001-7343-2590;