Comparison of Deep Reinforcement Learning Algorithms in Enhancing Energy Trading in Microgrids

This paper aims to introduce a solution to Sudan’s inadequate electricity supply; focusing on current unconnected rural areas and the high cost of connecting these areas to Sudan’s national grid. Microgrids were introduced as a viable option to create small scale distributed grids that depend solely on renewable energy to generate sufficient electricity to satisfy their loads. The paper also aims to enhance the usability of Microgrids by introducing a Machine learning technique to their secondary control that uses energy trading to ensure that all loads in islanded Microgrids are secured. The algorithm uses Reinforcement learning as control for the trading procedure. Data was extracted from a Matlab simulation and was then used to enhance the design of the Reinforcement learning environment. A generic environment for microgrids was designed and implemented which be further used in Reinforcement learning smart grids applications. A set of trading rules were implemented so that the Reinforcement Learning agents can use them across three Microgrids. The agent sees the three micro grids as one primary and two acting as trading game players. Two deep Reinforcement learning algorithms were explored as a solution; the first was an on-policy algorithm; Proximal Policy Optimization (PPO), and the second was an off-policy algorithm; Deep Deterministic Policy Gradients (DDPG). The results of applying both algorithms at three villages in Northern Kordufan State, Hamza ELsheikh, Tannah, and Um Bader were then compared. The algorithms achieved grid equilibrium without any grid loss and achieved profit from the trading process, reducing the time of return for the initial cost of the microgrid.