0.95plus0.95minus0.9510.950.95data-driven coordination of ...€¦ · 19/5/2011 · distributed...

Data-driven Coordination ofDistributed Energy Resources

Alejandro D. Domınguez-Garcıa

Coordinated Science LaboratoryDepartment of Electrical and Computer Engineering

University of Illinois at Urbana-Champaign

PSERC Webinar SeriesNovember 5, 2019

Outline

1 Introduction

2 DER Coordination for Active Power Provision

3 LTC Coordination for Voltage Regulation

4 Concluding Remarks

Impacts of Renewable-Based Power Generation Resources

I Deep penetration of renewable-based generation imposes additionalrequirements on ancillary services including:

• Frequency regulation (in bulk power systems)

• Reactive power support (in distribution systems)

I Frequency regulation in bulk power systems is typically achieved bycontrolling large synchronous generators

• Resources in distribution systems are not utilized for this task

I Reactive power support in distribution systems is provided by devicessuch as load tap changers (LTCs) and fixed/switched capacitors

• These devices are not designed to manage high variability in voltagefluctuations induced by renewable-based generation

Domınguez-Garcıa (ECE ILLINOIS) Data-driven Coordination [email protected] 1 / 32

The Solution

I An increasing number of DERs are being integrated into distributionsystems

I DERs could potentially be utilized to provide ancillary services ifproperly coordinated by, e.g., an aggregator

PV systems Electric Vehicles Fuel Cells Residential Storage


Need for Data-Driven Coordination

G

G

bulk power system power distribution system

tie line

aggregator

I DER aggregators needs to develop appropriate coordination schemes soDERs can collectively provide services that meet certain requirements

I Model-based schemes may be infeasible due to the lack of accurate models

I Data-driven schemes that only rely on measurements provide a promisingalternative for developing efficient coordination schemes


Presentation Overview

Objective

To develop data-driven coordination frameworks for assets (DERs, LTCs)in distribution systems

I Part I. Active power provision problem• total active power exchanged between the distribution and bulk systems

needs to equal to some amount requested by the bulk system operator

I Part II. Voltage regulation problem• the voltage magnitude at each bus needs be maintained to stay close

to some reference value


Outline

1 Introduction




Optimal DER Coordination Problem (ODCP)

Synchronous generator

Inverter-interfaced source

Load



Load

Bus

Legend



Load

Legend

tie line

distribution system<latexit sha1_base64="5oXhHblagDTaRG/DPbSkMCuxC24=">AAACBXicbVDJSgNBEO1xjXEb9SLooTEInsIkHvQY8OJFiWAWSELo6dQkTXoWumvEMATBS/yUXDwo4tV/8CD4N3aWgyY+KHi8V0VVPTeSQqPjfFsLi0vLK6uptfT6xubWtr2zW9ZhrDiUeChDVXWZBikCKKFACdVIAfNdCRW3ezHyK3egtAiDW+xF0PBZOxCe4AyN1LQP6wj3qL2kZXYp4cYjmeqeRvD7TTvjZJ0x6DzJTUmmsP80GH5dPRSb9me9FfLYhwC5ZFrX8k6EjYQpFFxCP12PNUSMd1kbaoYGzAfdSMZf9OmxUVrUC5WpAOlY/T2RMF/rnu+aTp9hR896I/E/rxajd95IRBDFCAGfLPJiSTGko0hoSyjgKHuGMK6EuZXyDlOMowkubULIzb48T8r5bO40m78xaVyTCVLkgByRE5IjZ6RALkmRlAgnj2RIXsirNbCerTfrfdK6YE1n9sgfWB8/Hs+dWA==</latexit>

bulk grid

fp5<latexit sha1_base64="thDo6BlNAY6gcbrq0yNuhPekWd8=">AAAB7HicbVBNS8NAEJ34WetX1aOXxSJ4KklV9FjQg8cKpi20sWy2k3bpZhN2N0Ip/Q1ePCji1R/kzX/jts1BWx8MPN6bYWZemAqujet+Oyura+sbm4Wt4vbO7t5+6eCwoZNMMfRZIhLVCqlGwSX6hhuBrVQhjUOBzXB4M/WbT6g0T+SDGaUYxLQvecQZNVbyo+7lY9otld2KOwNZJl5OypCj3i19dXoJy2KUhgmqddtzUxOMqTKcCZwUO5nGlLIh7WPbUklj1MF4duyEnFqlR6JE2ZKGzNTfE2Maaz2KQ9sZUzPQi95U/M9rZya6DsZcpplByeaLokwQk5Dp56THFTIjRpZQpri9lbABVZQZm0/RhuAtvrxMGtWKd16p3l+Ua7d5HAU4hhM4Aw+uoAZ3UAcfGHB4hld4c6Tz4rw7H/PWFSefOYI/cD5/AHynjno=</latexit>

fp3<latexit sha1_base64="wC7T1+CdVSgRk59wH6wVx/8cs7Q=">AAAB7HicbVBNS8NAEJ34WetX1aOXxSJ4Kkkr6LGgB48VTFtoY9lsJ+3SzSbsboRS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dtbWNza3tgs7xd29/YPD0tFxUyeZYuizRCSqHVKNgkv0DTcC26lCGocCW+HoZua3nlBpnsgHM04xiOlA8ogzaqzkR73aY9orld2KOwdZJV5OypCj0St9dfsJy2KUhgmqdcdzUxNMqDKcCZwWu5nGlLIRHWDHUklj1MFkfuyUnFulT6JE2ZKGzNXfExMaaz2OQ9sZUzPUy95M/M/rZCa6DiZcpplByRaLokwQk5DZ56TPFTIjxpZQpri9lbAhVZQZm0/RhuAtv7xKmtWKV6tU7y/L9ds8jgKcwhlcgAdXUIc7aIAPDDg8wyu8OdJ5cd6dj0XrmpPPnMAfOJ8/eZuOeA==</latexit>

fp2<latexit sha1_base64="kLXDoQq/OrRB22SGCQN3LRo4XkQ=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCHjxWsB/QxrLZbtqlm03YnQgl9Dd48aCIV3+QN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38z89hPXRsTqAScJ9yM6VCIUjKKVmmG/9pj0yxW36s5BVomXkwrkaPTLX71BzNKIK2SSGtP13AT9jGoUTPJpqZcanlA2pkPetVTRiBs/mx87JWdWGZAw1rYUkrn6eyKjkTGTKLCdEcWRWfZm4n9eN8Xw2s+ESlLkii0WhakkGJPZ52QgNGcoJ5ZQpoW9lbAR1ZShzadkQ/CWX14lrVrVu6jW7i8r9ds8jiKcwCmcgwdXUIc7aEATGAh4hld4c5Tz4rw7H4vWgpPPHMMfOJ8/eBWOdw==</latexit>

fp4<latexit sha1_base64="CrB9/Fk0anrkXjJlxq0FKMuzZ4U=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lqQY8FPXisYGuhjWWznbRLN5uwuxFK6G/w4kERr/4gb/4bt20O2vpg4PHeDDPzgkRwbVz32ymsrW9sbhW3Szu7e/sH5cOjto5TxbDFYhGrTkA1Ci6xZbgR2EkU0igQ+BCMr2f+wxMqzWN5byYJ+hEdSh5yRo2VWmG//pj0yxW36s5BVomXkwrkaPbLX71BzNIIpWGCat313MT4GVWGM4HTUi/VmFA2pkPsWipphNrP5sdOyZlVBiSMlS1pyFz9PZHRSOtJFNjOiJqRXvZm4n9eNzXhlZ9xmaQGJVssClNBTExmn5MBV8iMmFhCmeL2VsJGVFFmbD4lG4K3/PIqadeq3kW1dlevNG7yOIpwAqdwDh5cQgNuoQktYMDhGV7hzZHOi/PufCxaC04+cwx/4Hz+AHshjnk=</latexit>

0<latexit sha1_base64="rsPGDo38dCUrLsAt/ftnosrChUA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeiF48t2FpoQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCoreNUMWyxWMSqE1CNgktsGW4EdhKFNAoEPgTj25n/8IRK81jem0mCfkSHkoecUWOlptsvV9yqOwdZJV5OKpCj0S9/9QYxSyOUhgmqdddzE+NnVBnOBE5LvVRjQtmYDrFrqaQRaj+bHzolZ1YZkDBWtqQhc/X3REYjrSdRYDsjakZ62ZuJ/3nd1ITXfsZlkhqUbLEoTAUxMZl9TQZcITNiYgllittbCRtRRZmx2ZRsCN7yy6ukXat6F9Va87JSv8njKMIJnMI5eHAFdbiDBrSAAcIzvMKb8+i8OO/Ox6K14OQzx/AHzucPemeMuA==</latexit>

1<latexit sha1_base64="TtIgPQprnJE4HSS++PuM3etxya8=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeiF48t2FpoQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCoreNUMWyxWMSqE1CNgktsGW4EdhKFNAoEPgTj25n/8IRK81jem0mCfkSHkoecUWOlptcvV9yqOwdZJV5OKpCj0S9/9QYxSyOUhgmqdddzE+NnVBnOBE5LvVRjQtmYDrFrqaQRaj+bHzolZ1YZkDBWtqQhc/X3REYjrSdRYDsjakZ62ZuJ/3nd1ITXfsZlkhqUbLEoTAUxMZl9TQZcITNiYgllittbCRtRRZmx2ZRsCN7yy6ukXat6F9Va87JSv8njKMIJnMI5eHAFdbiDBrSAAcIzvMKb8+i8OO/Ox6K14OQzx/AHzucPe+uMuQ==</latexit>

2<latexit sha1_base64="jk/1fpohXujb3eq/tOFNvjxoFrw=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeiF48t2FpoQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCoreNUMWyxWMSqE1CNgktsGW4EdhKFNAoEPgTj25n/8IRK81jem0mCfkSHkoecUWOlZq1frrhVdw6ySrycVCBHo1/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJu1b1Lqq15mWlfpPHUYQTOIVz8OAK6nAHDWgBA4RneIU359F5cd6dj0VrwclnjuEPnM8ffW+Mug==</latexit>

3<latexit sha1_base64="LNUgVcgcWmrkAqMqFwJgrNe8JZM=">AAAB6HicbVDLTgJBEOzFF+IL9ehlIjHxRHbBRI9ELx4hkUcCGzI79MLI7OxmZtaEEL7AiweN8eonefNvHGAPClbSSaWqO91dQSK4Nq777eQ2Nre2d/K7hb39g8Oj4vFJS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfju7nffkKleSwfzCRBP6JDyUPOqLFSo9ovltyyuwBZJ15GSpCh3i9+9QYxSyOUhgmqdddzE+NPqTKcCZwVeqnGhLIxHWLXUkkj1P50ceiMXFhlQMJY2ZKGLNTfE1MaaT2JAtsZUTPSq95c/M/rpia88adcJqlByZaLwlQQE5P512TAFTIjJpZQpri9lbARVZQZm03BhuCtvrxOWpWyVy1XGlel2m0WRx7O4BwuwYNrqME91KEJDBCe4RXenEfnxXl3PpatOSebOYU/cD5/AH7zjLs=</latexit>

4<latexit sha1_base64="2R4aOFSwETOsp54DtxcLBinmYfY=">AAAB5HicbVBNS8NAEJ3Urxq/qlcvi0XwVJIq6LHoxWMF+wFtKJvtpF272YTdjVBCf4EXD4pXf5M3/43bNgdtfTDweG+GmXlhKrg2nvftlDY2t7Z3yrvu3v7B4VHFPW7rJFMMWywRieqGVKPgEluGG4HdVCGNQ4GdcHI39zvPqDRP5KOZphjEdCR5xBk1Vnq4GlSqXs1bgKwTvyBVKNAcVL76w4RlMUrDBNW653upCXKqDGcCZ24/05hSNqEj7FkqaYw6yBeHzsi5VYYkSpQtachC/T2R01jraRzazpiasV715uJ/Xi8z0U2Qc5lmBiVbLooyQUxC5l+TIVfIjJhaQpni9lbCxlRRZmw2rg3BX315nbTrNf+yVq82boswynAKZ3ABPlxDA+6hCS1ggPACb/DuPDmvzseyseQUEyfwB87nDxYRi5I=</latexit>

5<latexit sha1_base64="AhcfTCM0SoNzrW5wWWD2fdOk/P0=">AAAB6HicbVDLTgJBEOzFF+IL9ehlIjHxRHZRo0eiF4+QyCOBDZkdemFkdnYzM2tCCF/gxYPGePWTvPk3DrAHBSvppFLVne6uIBFcG9f9dnJr6xubW/ntws7u3v5B8fCoqeNUMWywWMSqHVCNgktsGG4EthOFNAoEtoLR3cxvPaHSPJYPZpygH9GB5CFn1FipftUrltyyOwdZJV5GSpCh1it+dfsxSyOUhgmqdcdzE+NPqDKcCZwWuqnGhLIRHWDHUkkj1P5kfuiUnFmlT8JY2ZKGzNXfExMaaT2OAtsZUTPUy95M/M/rpCa88SdcJqlByRaLwlQQE5PZ16TPFTIjxpZQpri9lbAhVZQZm03BhuAtv7xKmpWyd1Gu1C9L1dssjjycwCmcgwfXUIV7qEEDGCA8wyu8OY/Oi/PufCxac042cwx/4Hz+AIH7jL0=</latexit>

N = 5

L = 5

n = 4<latexit sha1_base64="ehc+d5cyWaMmZLse/g19iyaZNZQ=">AAAB+3icbVDLSsNAFL3xWesr1qWbYBFclaRWdFMo6MKFSAX7gCaUyXTSDp1MwsxELKW/4saFIm79EXf+jZM0C209MJfDOfdy7xw/ZlQq2/42VlbX1jc2C1vF7Z3dvX3zoNSWUSIwaeGIRaLrI0kY5aSlqGKkGwuCQp+Rjj++Sv3OIxGSRvxBTWLihWjIaUAxUlrqm6W7+rnrFm+zyus1Xftm2a7YGaxl4uSkDDmaffPLHUQ4CQlXmCEpe44dK2+KhKKYkVnRTSSJER6jIelpylFIpDfNbp9ZJ1oZWEEk9OPKytTfE1MUSjkJfd0ZIjWSi14q/uf1EhVcelPK40QRjueLgoRZKrLSIKwBFQQrNtEEYUH1rRYeIYGw0nGlITiLX14m7WrFOatU72vlxnUeRwGO4BhOwYELaMANNKEFGJ7gGV7hzZgZL8a78TFvXTHymUP4A+PzB05bkgg=</latexit>

fp1<latexit sha1_base64="a+QKp/0DS/KgPvX+uxck1C6flCc=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCHjxWMG2hjWWznbRLN5uwuxFK6W/w4kERr/4gb/4bt20O2vpg4PHeDDPzwlRwbVz32ymsrW9sbhW3Szu7e/sH5cOjpk4yxdBniUhUO6QaBZfoG24EtlOFNA4FtsLRzcxvPaHSPJEPZpxiENOB5BFn1FjJj3reY9orV9yqOwdZJV5OKpCj0St/dfsJy2KUhgmqdcdzUxNMqDKcCZyWupnGlLIRHWDHUklj1MFkfuyUnFmlT6JE2ZKGzNXfExMaaz2OQ9sZUzPUy95M/M/rZCa6DiZcpplByRaLokwQk5DZ56TPFTIjxpZQpri9lbAhVZQZm0/JhuAtv7xKmrWqd1Gt3V9W6rd5HEU4gVM4Bw+uoA530AAfGHB4hld4c6Tz4rw7H4vWgpPPHMMfOJ8/do+Odg==</latexit>

y<latexit sha1_base64="fW06iR5PqGZKR0/FY1imoD+Ocgo=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCHjy2YGuhDWWznbRrN5uwuxFK6C/w4kERr/4kb/4bt20O2vpg4PHeDDPzgkRwbVz32ymsrW9sbhW3Szu7e/sH5cOjto5TxbDFYhGrTkA1Ci6xZbgR2EkU0igQ+BCMb2b+wxMqzWN5byYJ+hEdSh5yRo2VmpN+ueJW3TnIKvFyUoEcjX75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14l7VrVu6jWmpeV+m0eRxFO4BTOwYMrqMMdNKAFDBCe4RXenEfnxXl3PhatBSefOYY/cD5/AOmljQM=</latexit>

pg1<latexit sha1_base64="aZMEN4pYL58tdiNfvRayCJrhG04=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCHjxWsB/QxrLZTtqlm03Y3Qgl9Dd48aCIV3+QN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfjm5nffkKleSwfzCRBP6JDyUPOqLFSM+l7j8N+ueJW3TnIKvFyUoEcjX75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NgpObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsczLgCpkRE0soU9zeStiIKsqMzadkQ/CWX14lrVrVu6jW7i8r9ds8jiKcwCmcgwdXUIc7aEATGHB4hld4c6Tz4rw7H4vWgpPPHMMfOJ8/eDuOdw==</latexit>

pg3<latexit sha1_base64="ev9tgPqiT4P4++Nab8hghps/3KU=">AAAB7HicbVBNS8NAEJ34WetX1aOXxSJ4Kkkr6LGgB48VTFtoY9lsJ+3SzSbsboRS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dtbWNza3tgs7xd29/YPD0tFxUyeZYuizRCSqHVKNgkv0DTcC26lCGocCW+HoZua3nlBpnsgHM04xiOlA8ogzaqzkp73a46BXKrsVdw6ySryclCFHo1f66vYTlsUoDRNU647npiaYUGU4EzgtdjONKWUjOsCOpZLGqIPJ/NgpObdKn0SJsiUNmau/JyY01noch7Yzpmaol72Z+J/XyUx0HUy4TDODki0WRZkgJiGzz0mfK2RGjC2hTHF7K2FDqigzNp+iDcFbfnmVNKsVr1ap3l+W67d5HAU4hTO4AA+uoA530AAfGHB4hld4c6Tz4rw7H4vWNSefOYE/cD5/AHtHjnk=</latexit>

pg4<latexit sha1_base64="+rJPQYvx83/OLtqC6y+2/XWmDek=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lqQY8FPXisYGqhjWWznbRLN5uwuxFK6W/w4kERr/4gb/4bt20O2vpg4PHeDDPzwlRwbVz32ymsrW9sbhW3Szu7e/sH5cOjlk4yxdBniUhUO6QaBZfoG24EtlOFNA4FPoSj65n/8IRK80Tem3GKQUwHkkecUWMlP+3VHwe9csWtunOQVeLlpAI5mr3yV7efsCxGaZigWnc8NzXBhCrDmcBpqZtpTCkb0QF2LJU0Rh1M5sdOyZlV+iRKlC1pyFz9PTGhsdbjOLSdMTVDvezNxP+8Tmaiq2DCZZoZlGyxKMoEMQmZfU76XCEzYmwJZYrbWwkbUkWZsfmUbAje8surpFWrehfV2l290rjJ4yjCCZzCOXhwCQ24hSb4wIDDM7zCmyOdF+fd+Vi0Fpx85hj+wPn8AXzNjno=</latexit>

pg2<latexit sha1_base64="KyCWBbhvUTl/NAWJALrGYiR3K7g=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCHjxWMG2hjWWz3bRLN5uwOxFK6W/w4kERr/4gb/4bt20O2vpg4PHeDDPzwlQKg6777RTW1jc2t4rbpZ3dvf2D8uFR0ySZZtxniUx0O6SGS6G4jwIlb6ea0ziUvBWObmZ+64lrIxL1gOOUBzEdKBEJRtFKftqrPQ565Ypbdecgq8TLSQVyNHrlr24/YVnMFTJJjel4borBhGoUTPJpqZsZnlI2ogPesVTRmJtgMj92Ss6s0idRom0pJHP198SExsaM49B2xhSHZtmbif95nQyj62AiVJohV2yxKMokwYTMPid9oTlDObaEMi3srYQNqaYMbT4lG4K3/PIqadaq3kW1dn9Zqd/mcRThBE7hHDy4gjrcQQN8YCDgGV7hzVHOi/PufCxaC04+cwx/4Hz+AHnBjng=</latexit>

pd5

<latexit sha1_base64="M6z2TYLklUKTJhB6eSfGRt/vw+o=">AAAB7HicbVBNS8NAEJ34WetX1aOXxSJ4KklV9FjQg8cKpi20sWw2m3bpZjfsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MOVMG9f9dlZW19Y3Nktb5e2d3b39ysFhS8tMEeoTyaXqhFhTzgT1DTOcdlJFcRJy2g5HN1O//USVZlI8mHFKgwQPBIsZwcZKftq/fIz6lapbc2dAy8QrSBUKNPuVr14kSZZQYQjHWnc9NzVBjpVhhNNJuZdpmmIywgPatVTghOognx07QadWiVAslS1h0Ez9PZHjROtxEtrOBJuhXvSm4n9eNzPxdZAzkWaGCjJfFGccGYmmn6OIKUoMH1uCiWL2VkSGWGFibD5lG4K3+PIyadVr3nmtfn9RbdwWcZTgGE7gDDy4ggbcQRN8IMDgGV7hzRHOi/PufMxbV5xi5gj+wPn8AXnHjng=</latexit>

pd3

<latexit sha1_base64="IRUlathTDAytvDfxlh/NReiQKJE=">AAAB7HicbVBNS8NAEJ34WetX1aOXxSJ4Kkkr6LGgB48VTFtoY9lsNu3S3U3Y3Qgl9Dd48aCIV3+QN/+N2zYHbX0w8Hhvhpl5YcqZNq777aytb2xubZd2yrt7+weHlaPjtk4yRahPEp6obog15UxS3zDDaTdVFIuQ0044vpn5nSeqNEvkg5mkNBB4KFnMCDZW8tNB4zEaVKpuzZ0DrRKvIFUo0BpUvvpRQjJBpSEca93z3NQEOVaGEU6n5X6maYrJGA9pz1KJBdVBPj92is6tEqE4UbakQXP190SOhdYTEdpOgc1IL3sz8T+vl5n4OsiZTDNDJVksijOOTIJmn6OIKUoMn1iCiWL2VkRGWGFibD5lG4K3/PIqaddrXqNWv7+sNm+LOEpwCmdwAR5cQRPuoAU+EGDwDK/w5kjnxXl3Phata04xcwJ/4Hz+AHa7jnY=</latexit>

pd2<latexit sha1_base64="gMdhP9FL/e9C16IxAR0usCA2vME=">AAAB7HicbVBNS8NAEJ2tX7V+VT16WSyCp5JUQY8FPXisYNpCG8tms2mXbjZhdyOU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmBang2jjONyqtrW9sbpW3Kzu7e/sH1cOjtk4yRZlHE5GobkA0E1wyz3AjWDdVjMSBYJ1gfDPzO09MaZ7IBzNJmR+ToeQRp8RYyUsHjcdwUK05dWcOvErcgtSgQGtQ/eqHCc1iJg0VROue66TGz4kynAo2rfQzzVJCx2TIepZKEjPt5/Njp/jMKiGOEmVLGjxXf0/kJNZ6Ege2MyZmpJe9mfif18tMdO3nXKaZYZIuFkWZwCbBs89xyBWjRkwsIVRxeyumI6IINTafig3BXX55lbQbdfei3ri/rDVvizjKcAKncA4uXEET7qAFHlDg8Ayv8IYkekHv6GPRWkLFzDH8Afr8AXU1jnU=</latexit>

pd4<latexit sha1_base64="yqrkG/Rgqhi2VSlTUC12/oCKJhI=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lqQY8FPXisYGqhjWWz2bRLN5tldyOU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhZIzbVz32ymtrW9sbpW3Kzu7e/sH1cOjjk4zRahPUp6qbog15UxQ3zDDaVcqipOQ04dwfD3zH56o0iwV92YiaZDgoWAxI9hYyZeD5mM0qNbcujsHWiVeQWpQoD2ofvWjlGQJFYZwrHXPc6UJcqwMI5xOK/1MU4nJGA9pz1KBE6qDfH7sFJ1ZJUJxqmwJg+bq74kcJ1pPktB2JtiM9LI3E//zepmJr4KcCZkZKshiUZxxZFI0+xxFTFFi+MQSTBSztyIywgoTY/Op2BC85ZdXSadR9y7qjbtmrXVTxFGGEziFc/DgElpwC23wgQCDZ3iFN0c4L86787FoLTnFzDH8gfP5A3hBjnc=</latexit>

pd1<latexit sha1_base64="Y9M8vq5BYqITmzKC5L/04PXNJP8=">AAAB7HicbVBNS8NAEJ2tX7V+VT16WSyCp5JUQY8FPXisYNpCG8tms2mXbjZhdyOU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmBang2jjONyqtrW9sbpW3Kzu7e/sH1cOjtk4yRZlHE5GobkA0E1wyz3AjWDdVjMSBYJ1gfDPzO09MaZ7IBzNJmR+ToeQRp8RYyUsH7mM4qNacujMHXiVuQWpQoDWofvXDhGYxk4YKonXPdVLj50QZTgWbVvqZZimhYzJkPUsliZn28/mxU3xmlRBHibIlDZ6rvydyEms9iQPbGRMz0sveTPzP62UmuvZzLtPMMEkXi6JMYJPg2ec45IpRIyaWEKq4vRXTEVGEGptPxYbgLr+8StqNuntRb9xf1pq3RRxlOIFTOAcXrqAJd9ACDyhweIZXeEMSvaB39LFoLaFi5hj+AH3+AHOvjnQ=</latexit>

Determine the DER active power injection vector, pg, that minimizes totalcost of operation while satisfying:

C1. The power exchanged with the bulk system, y, tracks somepre-specified value, y?

C2. The active power injection from each DER does not exceed itscorresponding capacity limits, i.e., pg ≤ pg ≤ pg

C3. The power flow on each line does not exceed its maximum capacity,i.e., −f ≤ f ≤ f


Input-Output System Model

I y can be written as a function of pg,pd, qd as follows:

y = h(pg,pd, qd)

I h captures the impacts from both the physical laws as well as theeffect of any reactive power control scheme

Assumption 1

H1. The rate of change in y w.r.t. pg is bounded for bounded changes inthe DER active power injections

H2. The total active power provided to the bulk power system will increasewhen more active power is injected in the power distribution system


ODCP Formulation

I The ODCP can be formulated as:

minimizepg∈[pg ,pg ]

c(pg)

subject to

h(pg,pd, qd) = y?,

−f ≤M−1(Cpg − pd) ≤ f

pg DER active power injectionspg,pg DER upper and lower capacity limitspd, qd Load active, reactive power demandsy? Requested power to be exchanged with bulk gridf Line flow limitsM Reduced node-to-edge incidence matrixC Matrix mapping DER indices to buses


Data-Driven DER Coordination Framework

I Data defining the ODCP problem:• Cost function, c(·)• DER capacity limits, pg,pg

• Network topology and DER location, M ,C• Load active and reactive power demand, pd, qd

• Line flow limits, f• Input-output model, h(·, ·, ·) ← Assumed unknown

I Real-time measurements available:• DER active power injections, pg[k], k = 1, 2, . . .• Active power exchanged with the bulk system, y[k], k = 1, 2, . . .

Framework Building Blocks

I An input-output (IO) model estimator that uses available real-timemeasurement data

I A controller that uses uses the identified IO model to solve the ODCP


Two-Timescale Coordination Framework

time

controller solves ODCP

time

iterations in estimation process

estimator updates sensitivities

Estimation Process

pd and qd remain approximately constant between two time instants;therefore, changes in y[k] that occur across time steps in the estimationprocess depend only on changes in pg[k]; thus,

y[k] = h(pg[k],pd, qd), k = 0, 1, . . .


Input-Output Model as a Linear Time-Varying System

I For notational simplicity, define u[k] = pg[k], u = pg, u = pg, and

π = [(pd)>, (qd)>]>; then, the IO model can be written as:

y[k] = h(u[k],π), k = 0, 1, . . . ,

I For k > 1, the above equation can be transformed into the followingequivalent linear time-varying model:

y[k] = y[k − 1] + φ[k]>(u[k]− u[k − 1])

where φ[k]> = [φi[k]] =∂h

∂u

∣∣∣∣u[k]

, with u[k] = aku[k] + (1− ak)u[k]

I φ[k] is referred to as the sensitivity vector at time step k

I The entries of φ[k] are bounded for all k by Assumption 1


Estimator – Estimation Step

I At time step k, the objective of the estimator is to obtain an estimateof φ[k], denoted by φ[k], using available measurements of u and y

I Problem P1:

φ[k] = arg minφ∈Q=[b1,b1]

n

Je(φ) =1

2(y[k − 1]− y[k − 1])2

subject to

y[k − 1] = y[k − 2] + φ>(u[k − 1]− u[k − 2])


Estimator – Control Step

I The objective of the controller during the estimation process is toensure that the output tracks the target [Different from the ODCP]

I Problem P2:

u[k] = arg minu∈U=[u,u]

Jc(u) =1

2(y? − y[k])2

subject to

y[k] = y[k − 1] + φ[k]>(u− u[k − 1])

I Note that φ[k] is used to predict the value of y[k] for a given u


Estimation Process Workflow

· · ·u[k − 1]→ y[k − 1]→ φ︸︷︷︸estimation step

control step︷︸︸︷[k]→ u[k]→ y[k]→ φ[k + 1] · · ·

I At the beginning of iteration k, y[k − 1] is used in Problem P1 toupdate the sensitivity vector estimate, φ[k]

I The updated sensitivity vector estimate, φ[k], is then used inProblem P2 to determine the control, u[k]

I Then, the DERs are instructed to change their active power injectionset-points based on u[k]

I Problems P1 and P2 are not solved to completion for each k

I Instead, we iterate the projected gradient descent algorithm thatwould solve them for one step at each iteration k


Simulation Setup

0

1

2

3

45 6

7 8

9

12

14

10

11

13

15

16

17

1819

20

2122

2324

25

2627

28

29

30

31

32

33

34

35

3637

3839

40

41

42

43

44 4546

4748

4950

51

52

5354

55

56

57

58

59

60

61

62

63

6465

66

6768

6970

71

72

7374

75

76

77 7879

80

81

82

8485

83

8687

88

89

90

91

92

93

94

95

96

97

98

99

101

100

102103

104

105

106107

108

109

110

111

112 113 114

117

115

118

119

116121

122

120

DER

Figure 1: The IEEE 123-bus distribution test feeder.


Tracking Performance During Estimation

0.0 2.0 4.0 6.0 8.0 10.0

time (s)

−600

−400

−200

0

tracking

error(kW)

-3000 kW

-2900 kW

-2800 kW

-2700 kW

-2600 kW

-2500 kW

Figure 2: Tracking error for βk = 0.02 under various tracking targets.

0.0 2.0 4.0 6.0 8.0 10.0

time (s)

−100

−75

−50

−25

0

tracking

error(kW)

0.005

0.01

0.02

0.03

0.04

0.05

Figure 3: Tracking error for y? = −3000 kW and various constant control step sizes.


Estimation Accuracy

I Mean absolute error (MAE) of estimation errors:

MAE =

∑ni=1

∣∣φi[k]− φi[k]∣∣

n,

where n is the number of DERs

0.0 2.0 4.0 6.0 8.0 10.0

time (s)

0.00

0.02

0.04

0.06

MAE(p.u./p.u.)

0.005

0.01

0.02

0.03

0.04

0.05

Figure 4: Estimation error under various control step sizes.


Outline

1 Introduction




Background

I Voltage regulation transformers—also referred to as Load TapChangers (LTCs)—are widely utilized in power distribution systems toregulate voltage magnitudes along a feeder

I Model for a load tap changer on a line connecting buses i and j:

+

−

+

−

i

j

Vi

Vj

rℓ + i xℓtℓ : 1

+

−

Vj′ =Vi

t2ℓ

Vi Primary side voltage magnitudeVj0 Secondary side voltage magnitudeVj Line receiving end voltage magnitudet` Tap ratior` Transformer + line resistancex` Transformer + line reactance

<latexit sha1_base64="o95+ZhqUihFZ3qgD3xJ3PvnIViA=">AAAE6nicjVRLb9NAEHabACW8WjhyGagDSFSRnagtPSBVlAOHHopIH1KcRpv1Jtl2vTa765TI9U/gwgGEuPKLuPFvmHXStGkLYiU7szPffPN0uong2nje77n5UvnGzVsLtyt37t67/2Bx6eGejlNF2S6NRawOukQzwSXbNdwIdpAoRqKuYPvd4y1r3x8ypXksm2aUsHZE+pL3OCUGVZ2lkhN0WZ/LzBB0yVtPTLsCEOiICFGpoqSYZCc0jiIiwywgSpGRNooZOsgzv9bILZoyaZjism8vU7pUEJVnVFhI1Q0oEduHBqnTrmaGfYRC48IzwCvEPbAlaDjhZgDbzS29Au7pxOkUXgP+uBAEyOXudbh121E8ImoEmocMhrEwpM+gKM+kqDnDZkfPcwv/wGgsw/9wOLJo2MZsQDHK+BALAybDv7mAazoBE6Jwa5IElO3tmU1NbU1FpO7FKmIKXhbVIr/GERNJx1Tup3+BEU1oAS6oq+AGbxlmhC2PMpNjj1oTxTif9mFg4sTFPtobBFzCpJ82wJBREyvbdzNNmQ6I7OMMpvwIbyJvkHm1jRXAl7dWX11BmjA2OCC/5m001let4Af5hVG6jYYLPUY0x6U6D3DGPE38nB9JkwgwhDcbwmq9mj8OM77UZ0OtrV4XaraWAMc3XckLd9z4Smdx2Ua1B64K/kRYdiZnp7P4Kwhjmka49FQQrVu+l5h2RpTh1BIGqWYJoce4Ki0UJYmYbmfFp5pDFTUh4GDxkQYK7UWPjERaj6IuIiNiBvqyzSqvs7VS03vVzrhMUsMkHQfqpQJMDPa7h5DjMhsxQoFQxTFX2yKFG4X/DrYJ/uWSrwp79ZrfqNXf15c330zaseA8dp46LxzfWXc2nXfOjrPr0FK/9Ln0tfStLMpfyt/LP8bQ+bmJzyNn5pR//gHbvoGb</latexit>

I The tap ratio, tl, typically takes 33 discrete values ranging from 0.9to 1.1, by an increment of 5/8%, i.e.,

tl ∈ T = {0.9, 0.90625, · · · , 1.09375, 1.1}


Problem Motivation

I Current LTC control schemes are myopic:

• Based on local voltage measurements

• Do not account for future uncertainty effects on current control actions

I These schemes are no longer suitable because of increased variabilityand uncertainty in uncontrolled power injections arising from:

• Residential PV installations

• Electric vehicles

I In addition, the use of controlled DERs for providing frequencyregulation to the bulk grid has an impact on voltage regulation


Volt/VAR Control Architecture

t0 t1 t2

Slow Time-Scale Control

time

Fast Time-Scale Control

I Slow Time-Scale: LTCs are periodically dispatched so as to reducemechanical wear

I Fast Time-Scale Control: Power-electronic-interfaced DERs withreactive power provision capability


Optimal LTC Dispatch Problem



Load



Load

Bus

Legend



Load

Legend

tie line

distribution system<latexit sha1_base64="5oXhHblagDTaRG/DPbSkMCuxC24=">AAACBXicbVDJSgNBEO1xjXEb9SLooTEInsIkHvQY8OJFiWAWSELo6dQkTXoWumvEMATBS/yUXDwo4tV/8CD4N3aWgyY+KHi8V0VVPTeSQqPjfFsLi0vLK6uptfT6xubWtr2zW9ZhrDiUeChDVXWZBikCKKFACdVIAfNdCRW3ezHyK3egtAiDW+xF0PBZOxCe4AyN1LQP6wj3qL2kZXYp4cYjmeqeRvD7TTvjZJ0x6DzJTUmmsP80GH5dPRSb9me9FfLYhwC5ZFrX8k6EjYQpFFxCP12PNUSMd1kbaoYGzAfdSMZf9OmxUVrUC5WpAOlY/T2RMF/rnu+aTp9hR896I/E/rxajd95IRBDFCAGfLPJiSTGko0hoSyjgKHuGMK6EuZXyDlOMowkubULIzb48T8r5bO40m78xaVyTCVLkgByRE5IjZ6RALkmRlAgnj2RIXsirNbCerTfrfdK6YE1n9sgfWB8/Hs+dWA==</latexit>

bulk grid

V0<latexit sha1_base64="UWBxE37KNPRxcRhAec6ZeR5sUDE=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCHjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38z89hPXRsTqEScJ9yM6VCIUjKKVHlp9t1+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmnVqt5FtXZ/Wanf5nEU4QRO4Rw8uII63EEDmsBgCM/wCm+OdF6cd+dj0Vpw8plj+APn8wfYOY2D</latexit>

V1<latexit sha1_base64="YI4Xx+GEkdjmHVvEks46BTlj0vc=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCHjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38z89hPXRsTqEScJ9yM6VCIUjKKVHlp9r1+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmnVqt5FtXZ/Wanf5nEU4QRO4Rw8uII63EEDmsBgCM/wCm+OdF6cd+dj0Vpw8plj+APn8wfZvY2E</latexit>

V2<latexit sha1_base64="iwfnaRBVV/Ir21Cwc6KNHjU4I4w=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCHjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38z89hPXRsTqEScJ9yM6VCIUjKKVHlr9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjbhCJqkxXc9N0M+oRsEkn5Z6qeEJZWM65F1LFY248bP5qVNyZpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0SjYEb/nlVdKqVb2Lau3+slK/zeMowgmcwjl4cAV1uIMGNIHBEJ7hFd4c6bw4787HorXg5DPH8AfO5w/bQY2F</latexit>

V3<latexit sha1_base64="mG/wC2kbs6KdRY4OVTAEbkrDsig=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0laQY8FPXisaGuhDWWznbRLN5uwuxFK6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkRwbVz32ymsrW9sbhW3Szu7e/sH5cOjto5TxbDFYhGrTkA1Ci6xZbgR2EkU0igQ+BiMr2f+4xMqzWP5YCYJ+hEdSh5yRo2V7tv9er9ccavuHGSVeDmpQI5mv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5J2rerVq7W7i0rjJo+jCCdwCufgwSU04Baa0AIGQ3iGV3hzhPPivDsfi9aCk88cwx84nz/cxY2G</latexit>V4<latexit sha1_base64="q0oBHNDEKVfC4SJAf9ZS4vPOUrw=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSJ4Kkkt6LGgB48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O2vrG5tb24Wd4u7e/sFh6ei4peNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPxzcxvP6HSPJaPZpKgH9Gh5CFn1FjpodWv9Utlt+LOQVaJl5My5Gj0S1+9QczSCKVhgmrd9dzE+BlVhjOB02Iv1ZhQNqZD7FoqaYTaz+anTsm5VQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6RRuCt/zyKmlVK95lpXpfK9dv8zgKcApncAEeXEEd7qABTWAwhGd4hTdHOC/Ou/OxaF1z8pkT+APn8wfeSY2H</latexit>V5

<latexit sha1_base64="FJptUISjDg8qzZw3Xo2eztgltPk=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSJ4KklV9FjQg8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvSKQw6Lrfzsrq2vrGZmGruL2zu7dfOjhsmjjVjDdYLGPdDqjhUijeQIGStxPNaRRI3gpGN1O/9cS1EbF6xHHC/YgOlAgFo2ilh2bvslcquxV3BrJMvJyUIUe9V/rq9mOWRlwhk9SYjucm6GdUo2CST4rd1PCEshEd8I6likbc+Nns1Ak5tUqfhLG2pZDM1N8TGY2MGUeB7YwoDs2iNxX/8zophtd+JlSSIldsvihMJcGYTP8mfaE5Qzm2hDIt7K2EDammDG06RRuCt/jyMmlWK955pXp/Ua7d5nEU4BhO4Aw8uIIa3EEdGsBgAM/wCm+OdF6cd+dj3rri5DNH8AfO5w/fzY2I</latexit>

N = 5

L = 5

n = 4<latexit sha1_base64="ehc+d5cyWaMmZLse/g19iyaZNZQ=">AAAB+3icbVDLSsNAFL3xWesr1qWbYBFclaRWdFMo6MKFSAX7gCaUyXTSDp1MwsxELKW/4saFIm79EXf+jZM0C209MJfDOfdy7xw/ZlQq2/42VlbX1jc2C1vF7Z3dvX3zoNSWUSIwaeGIRaLrI0kY5aSlqGKkGwuCQp+Rjj++Sv3OIxGSRvxBTWLihWjIaUAxUlrqm6W7+rnrFm+zyus1Xftm2a7YGaxl4uSkDDmaffPLHUQ4CQlXmCEpe44dK2+KhKKYkVnRTSSJER6jIelpylFIpDfNbp9ZJ1oZWEEk9OPKytTfE1MUSjkJfd0ZIjWSi14q/uf1EhVcelPK40QRjueLgoRZKrLSIKwBFQQrNtEEYUH1rRYeIYGw0nGlITiLX14m7WrFOatU72vlxnUeRwGO4BhOwYELaMANNKEFGJ7gGV7hzZgZL8a78TFvXTHymUP4A+PzB05bkgg=</latexit>

Load tap changer (LTC)

Objective

Find a policy for determining the LTC tap positions based onmeasurements of current

I tap ratios

I bus voltage magnitudes

so as to minimize bus voltages deviations from some reference value aspower injections change as time evolves


Power Distribution System Model

I The relation between square voltage magnitudes and active/reactivepower injections and LTC tap ratios at instant k can be written as

V [k] = g(p[k], q[k], t[k])

V [k] Vector of voltage magnitudes at instant kp[k], q[k] Vector of active, reactive, power injections at instant kt[k] Vector of tap ratios at instant k

I Network topology and transmission line parameters are encapsulatedinto g


Assumptions

T1. Changes in active and reactive power injections at instant k,∆p[k] := p[k + 1]− p[k] and ∆q[k] := q[k + 1]− q[k], are random

T2. Active and reactive power injections k, p[k] and q[k], are notmeasured and their joint probability distribution is unknown

T3. Network topology is known, i.e., M is known

T4. Line parameters are unknown, i.e., r and x are unknown

I Because of Assumption T1 is natural to formulate the problem as aMarkov Decision Process (MDP)

I Because of Assumptions T2 – T4, we will have to resort toreinforcement learning (RL) techniques to solve this MDP


LTC Coordination Problem as an MDPI State space, S: the state, s, is composed of tap ratio and squared

voltage magnitude vector, i.e., s = (t,v), t ∈ T Lt, v ∈ RN ; thus,

S ⊆ T Lt × RN

I Action space, A: the action, a, is the change in LTC tap ratiobetween two consecutive time instants, i.e., a = ∆t,∆t ∈ ∆T Lt

=: A, where

∆T = {0,±0.00625, · · · ,±0.19375,±0.2}

is the set set of feasible tap ratio changes

I Reward function, R: the reward when the system transitions fromstate s = (t,v) into state s′ = (t′,v′) after taking action a = ∆t is

R(s,a, s′) = − 1

N‖v′ − v?‖,

i.e., we are penalizing voltage deviation from some reference value v?


LTC Coordination Problem as an MDP

I State transitions: governed by random changes in active and reactivepower injection vectors, ∆p := p′ − p and ∆q := q′ − q:

s′ = h(s,a,∆p,∆q)

• Network topology and transmission line paramters are encapsulatedinto h

• Probability of transitioning from s to s′ under action a, P(s′ | s,a),could be computed if h and the joint pdf of ∆p and ∆q were know

Objective

Find a policy π : (t,v) 7→ ∆t, t ∈ T Lt, v ∈ RN ,∆t ∈ ∆T Lt

so that

− 1

N

∞∑k=0

γkE[‖v[k + 1]− v?‖

∣∣ t[0] = t0,v[0] = v0

]is maximized (equivalent to minimizing voltage deviations)


Solving the LTC Coordination ProblemI Let R(s,a) denote the expected reward for the pair (s,a)

I The MDP is solved when we find Q∗(s,a), s ∈ S,a ∈ A, andπ∗(s), s ∈ S, satisfying

Q∗(s,a) = R(s,a) + γ∑s′∈SP(s′|s,a) max

a′∈AQ∗(s′,a′)

π∗(s) = arg maxa

Q∗(s,a)

I Two issues in our setting:

I1. We do not know P(· | ·, ·)

I2. Even if we knew P(· | ·, ·), we could not solve for Q∗(s,a) efficientlybecause of the curse of dimensionality in the state and action spaces

I To circumvent Issue I1, we apply a model-free RL algorithm thatutilizes transition samples obtained via a virtual transition generator

I To circumvent Issue I2, we use function approximation and a learningscheme for sequential estimation of the action-value function


LTC Coordination Framework Building Blocks

Power Distribution

System

Environment

Greedy Actor

Acting Agent

action

reward

Action-Value Function

Estimator (Critic)

History

Learning Agent

Virtual Transition

Generator

Exploratory Actor

state

action

action-value

function estimate


Action-Value Function Value EstimatorI Let Q(·, ·) denote an approximation of the optimal action-value

function Q(·, ·)

I Methods for obtaining Q(·, ·) include:

• Parametric functions

• Neural networks

I Here we consider a linear parametrization of Q(·, ·):

Q(s,a) = w>φ(s,a),

where w ∈ Rf is the parameter vector and φ : S ×A → Rf is somebasis function

I Let D = {(s,a, r, s′) : s, s′ ∈ S,a ∈ A} denote a set (batch) oftransition samples obtained via observation or simulation

I We use least-square policy iteration (LSPI) algorithm [Lagoudakis andParr, 2003] to find w that best fits the transition samples in D


Challenges

I The LSPI algorithm requires adequate transition samples that spreadover S ×A

I This is challenging in power systems since the system operationalreliability might be jeopardized when exploring randomly

• We address this by developing by generating samples via a virtualtransition generator that leverages historical system operational data

I The LSPI also suffers from the curse of dimensionality when theaction space is large—the case in the LTC coordination problem

• We address this by using a sequential scheme that breaks the learningproblem into smaller problems and uses the LSPI algorithm on those


IEEE 123-bus Test Feeder

0

1

2

3

4 5 6

7 89

12

1410

11

13

15

16

17

181920

2122

2324

25

2627

28

29

30

31

32

33

34

35

3637

38 39

4041

4243

44 4546

4748

4950

51

5253 54

5556

5758

59

60

61

62

63

6465

66

6768

69 7071

7273

74 75

7677 78 79

80

81

82

84 85

8386

87

88

89

90

91

92

93

94

95

96

97

9899

101100

102103

104105

106107108

109110

111112 113 114

117

115

118

119

116 121

122

120


IEEE 123-bus Test Feeder Results

0 2 4 6 8 10 12 14 16 18 20 22 24time (hour)

-8

-6

-4

-2

0

2

4

tapposition

Batch RL

1

2

3

4

0 2 4 6 8 10 12 14 16 18 20 22 24time (hour)

-8

-6

-4

-2

0

2

4

tapposition

Exhaustive search

1

2

3

4

Figure 5: Tap positions.

0 2 4 6 8 10 12 14 16 18 20 22 24time (hour)

−10

−5

reward(×

10−

3)

batch RL

exhaustive search

conventional scheme

Figure 6: Immediate rewards.

1 2 3 4 5 6 7 8time (day)

−10

−5

0

reward(×

10−

3)

batch RL

exhaustive search

conventional scheme

Figure 7: Rewards over 7 days.


Outline

1 Introduction




Conclusions

I We developed a data-driven coordination framework for coordinatingassets in distribution systems to provide ancillary services:

I The proposed framework:

• It assumes no prior information on the distribution system model,except knowledge of network topology

• It mainly relies on measurements

• It is adaptive and robust to changes in operating conditions

I Refer to the following papers for more details:

1. H. Xu, A. Domınguez-Garcıa, and P. Sauer, “Data-driven Coordinationof Distributed Energy Resources for Active Power Provision,” IEEETransactions on Power System, 2019.

2. H. Xu, A. Domınguez-Garcıa, and P. W. Sauer, “Optimal tap setting ofvoltage regulation transformers using batch reinforcement learning,”IEEE Transactions on Power System, 2019.


Questions?

Alejandro D. Domınguez-Garcıae-mail: [email protected]

url: https://aledan.ece.illinois.edu


References I

[Lagoudakis and Parr, 2003] Lagoudakis, M. G. and Parr, R. (2003).Least-squares policy iteration.Journal of Machine Learning Research, 4:1107–1149.

A Primer on Markov Decision ProcessesI A Markov Decision Process (MDP) is defined as a 5-tuple

(S,A,P,R, γ) where• S is a finite set of states• A is a finite set of actions• P is a Markovian transition model• R : S ×A× S → R is a reward function• γ ∈ [0, 1) is a discount factor

I Let s[k] and a[k] denote random variables (r.v.’s) respectivelydescribing the value the state and action take at time instant k

I Let r[k] denote a r.v. describing the reward received after takingaction a[k] in state s[k] and transitioning to state s[k + 1]; then

r[k] = R(s[k],a[k], s[k + 1]

)I A deterministic policy π is a mapping from S to A, i.e.,a = π(s), s ∈ S,a ∈ A


A Primer on Markov Decision Processes

Objective

Given some initial state, s0, we want to find a deterministic policy, π∗, thatmaximizes the expected value of the cumulative discounted reward, i.e.,

π∗ = arg maxπ

∞∑k=0

γkE[r[k]

∣∣ s[0] = s0

]


A Primer on Markov Decision ProcessesI The action-value function under policy π is defined as

Qπ(s,a) =

∞∑k=0

γkE[r[k]

∣∣ s[k] = s,a[k] = a;π], s ∈ S, a ∈ A

I The optimal action-value function, Q∗(s,a), s ∈ S,a ∈ A is themaximum action-value function over all policies, i.e.,

Q∗(s,a) = maxπ

Qπ(s,a)

I Q∗(s,a), s ∈ S,a ∈ A, satisfies the following Bellman equation:

Q∗(s,a) = E [r | s,a] + γ∑s′∈SP(s′|s,a) max

a′∈AQ∗(s′,a′)

I The optimal policy, π∗(s), s ∈ S, is obtained as follows:

π∗(s) = arg maxa

Q∗(s,a)

I The MDP is solved if we find Q∗(·, ·) and the corresponding π∗(·)Domınguez-Garcıa (ECE ILLINOIS) Data-driven Coordination [email protected] 35 / 32

LSPI Algorithm IterationI Let wi denote the estimate of w at the beginning of iteration i

I For each transition sample (s,a, r, s′) ∈ D, compute

a′ = arg maxα∈A

w>i φ(s′,α)

I Update the estimate of w as follows:

wi+1 = B−1b

where

B =∑

(s,a,r,s′)∈D

φ(s,a)(φ(s,a)− γφ(s′,a′)

)>︸︷︷︸

rank-1 matrix

b =∑

(s,a,r,s′)∈D

φ(s,a)r


Timeline

t0 t1 t2

Slow Time-Scale Control

time

Fast Time-Scale Control

I Policy updated every K∆T units of time, e.g., 2 hours

I Updated policy used for tap setting for K time instants


0.95plus0.95minus0.9510.950.95data-driven coordination of ...€¦ · 19/5/2011 · distributed...

Documents