needs, challenges, architecture and validation jean bélanger, ceo, july 6, 2009 jean bélanger...
TRANSCRIPT
Needs, Challenges, Architecture and Validation
Jean Bélanger, CEO, July 6, 2009
Jean Bélanger
eMEGAsim Digital Real-Time Simulatorfor Power Electronic and Power Grid Systems
CHALLENGES & SOLUTIONS - INDEX
IPST 2009, Kyoto - Japan
Computational powerFast and low latency IOs
Fast and low latency inter-processor communication
Precise and stable network solver - ARTEMiS
Scalability to simulate very large systems
Simulation of IGBT switching Finite Element-based Motor Models Simulation of MANY IGBTs
and Thyristors
TRUSTWOORTHY TECHNOLOGIES AND TYPICAL USERS: Power Industries, Aerospace & Defense, and Universities
HARDWARE ARCHITECTURE
POWER GRID AND SIMULATOR EVOLUTION REAL-TIME SIMULATION BENEFITS
SOFTWARE ARCHITECTURE & ACCURACY
Integration with Simulink and SimPowerSystems
Firing pulse accuracy of Thyristor-based systems
ADVANTAGES OF OPEN TECHNOLOGIES
CONCLUSIONS
Challenges: More Complex Power Grids
IPST 2009, Kyoto - Japan
1960 1970 1980 1990 2000
Conventional AC Power System
April 19, 2023 3
HVDC, SVC & Thyristor-based FACTSHVDC, SVC & Thyristor-based FACTS
More Complex SystemsMore and Faster Power Electronic Systems(MORE NUMERICAL CONTROLLERS)
IGBT-based FACTS and HVDC IGBT-based FACTS and HVDC
Challenges: More Complex Power Grids
IPST 2009, Kyoto - Japan
1960 1970 1980 1990 2000
Conventional AC Power System
Fast and Real-Time simulators will become more importantand must evolve with power system technology
Distribution systems arebecoming more complex thanhigh-voltage transmission systems
INTELLIGENT POWER GRIDSDistributed Generation and ControlsGlobal Power Grid ControlMore Complex Protection SystemsIntegration made by several manufacturersUtilities have less control over the system
High-Power IGBT SystemsHigh-Power IGBT Systems
Drives, Active Loads, FACTSDrives, Active Loads, FACTS
Wind farmsWind farms
Plug-in & Electric Vehicles
Plug-in & Electric Vehicles
Micro GridsMicro Grids
HVDC, SVC & Thyristor-based FACTSHVDC, SVC & Thyristor-based FACTS
IGBT-based FACTS and HVDC IGBT-based FACTS and HVDC More Complex Systems
More and Faster Power Electronic Systems(MORE NUMERICAL CONTROLLERS)
April 19, 2023 4
Evolution of Real-Time Electric Simulator Technology
1960 1970 1980 1990 2000
Conventional AC Power System
Hybrid (Analog/Digital) SimulatorsHybrid (Analog/Digital) Simulators
High-Voltage Transmission, HVDC, SVC & Thyristor-based FACTSHigh-Voltage Transmission, HVDC, SVC & Thyristor-based FACTS
IGBT-based FACTS and HVDC IGBT-based FACTS and HVDC Analog SimulatorsAnalog Simulators
IPST 2009, Kyoto - Japan April 19, 2023 5
Evolution of Real-Time Electric Simulator Technology
1960 1970 1980 1990 2000
Conventional AC Power System
Hybrid (Analog/Digital) SimulatorsHybrid (Analog/Digital) Simulators
High-Voltage Transmission, HVDC, SVC & Thyristor-based FACTSHigh-Voltage Transmission, HVDC, SVC & Thyristor-based FACTS
IGBT-based FACTS and HVDC IGBT-based FACTS and HVDC Analog SimulatorsAnalog Simulators
Custom Digital SimulatorsCustom Digital SimulatorsType 1
Commercial Supercomputers
Commercial SupercomputersType 2
Type 2- Commercial Supercomputers- ARENE by EDF , HP CONVEX (abandoned)- HYPERSIM-2 by Hydro-Quebec, SGI ALTIX (for HQ internal use only)
IPST 2009, Kyoto - Japan April 19, 2023 6
Evolution of Real-Time Electric Simulator Technology
1960 1970 1980 1990 2000
Conventional AC Power System
Hybrid (Analog/Digital) SimulatorsHybrid (Analog/Digital) Simulators
High-Voltage Transmission, HVDC, SVC & Thyristor-based FACTSHigh-Voltage Transmission, HVDC, SVC & Thyristor-based FACTS
IGBT-based FACTS and HVDC IGBT-based FACTS and HVDC
PC-Based DigitalSimulators
PC-Based DigitalSimulators
FPGA SimOCFPGA SimOC
Type 3eMEGAsimeDRIVEsimRT-LAB
High-Power IGBT SystemsHigh-Power IGBT Systems
Drives, Active Loads, FACTSDrives, Active Loads, FACTS
Micro-GridsWind farmsMicro-GridsWind farms
Hybrid VehiclesHybrid Vehicles
Analog SimulatorsAnalog Simulators
Custom Digital SimulatorsCustom Digital SimulatorsType 1
Commercial Supercomputers
Commercial SupercomputersType 2
Type 2- Commercial Supercomputers- ARENE by EDF , HP CONVEX (abandoned)- HYPERSIM-2 by Hydro-Quebec, SGI ALTIX (for HQ internal use only)
IPST 2009, Kyoto - Japan April 19, 2023 7
What is a Real-Time Simulator?
Real-Time ComputerIntegrated with Modeling and Simulation
Software (Simulink, SPS , RTW)
Input/Output system Real-Time Data Logging CPU Monitoring Host Computer for
User Interface and Waveform Display
REAL-TIME PLANT DIGITAL SIMULATOR
REAL or PROTOTYPE (SIMULINK/RTW)
Designed to Meet Hard Real-Time Constraints for Hardware-in-the Loop All model calculations MUST be
completed within the specified time period
Must include Real-Time processor monitoring and overrun detection
Capable of Emulating… Simulated plant; Control systems or both SIMULTANEOUSLY
… with Good Accuracy Better than 50 us time step Better than 200 nanosecond
timing
April 19, 2023 8IPST 2009, Kyoto - Japan
April 19, 20239 9
FAST OR REAL-TIME OPERATING MODES
SpecifyFeasabiltySpecify
Feasabilty
DesignDesign ImplementImplement
TestTest
PrototypingPrototyping
RTRT
RTRT
RTRT
HARD REAL-TIME(LINUX or QNX RTOS) DESIGN & OPTIMISATION
With Real Control & Protection Hardware
With Actual RT Software
RANDOM HIL TESTS Performance, Stability Controller Interaction Stress Analysis
TROUBLESHOOTING & TRAINING With Actual Control &
Protection Hardware
FAST PARALLEL SIMULATION – NON REAL-TIME OPTIMISATION RANDOM TESTS –
Performance & Stress Analysis SOFTWARE-IN-THE-LOOP
Control & Protection Algorithm Test
PREPARATION FOR HIL TEST With Normal PCs
(Windows, LINUX or QNX)
IPST 2009, Kyoto - Japan
Random Tests on Simple and Complex Networks
To find Worst Case Scenarios or Statistical Distribution Line Energization (3 breakers) Fault + Line Reclosing (9 breakers) Overvoltages and Arrester Energization
April 19, 2023IPST 2009, Kyoto - Japan 10
Protection and Controls Operating Times – Min, Max, Average Test for Fault Operation Controls – FACTS, HVDC, SVC, Series Capacitors … Influence of System Transients Harmonic Overvoltages and Inrush Currents Stresses on Components
Interactions between Distributed Control and Protection Functions
WIDEBAND: Simultaneous Simulation of Very Slow (minutes) and Very Fast (nanoseconds) Transients
BLACKOUT SIMULATION AND PREVENTION
April 19, 2023IPST 2009, Kyoto - Japan11
Benefits of HIL Testing and Simulation
Making Design Tradeoffs Early
Design issues can be discovered earlierwithout waiting for testing on a physical prototype of car, aircraft, train, industrial drives …
Reduced Length of Development Cycle
Concurrent engineering Initial calibration done in the lab before test
systems becomes available.
Reduced Testing Cost Testing is done in a repeatable environment.
Test automation
Better and More Tests Tests difficult or too risky to do with physical
prototypes can be performed at minimum cost
More tests made in several extreme operating conditions
Monte Carlo analysis
FEWER DEFECTS AND FASTER
DEVELOPMENT CYCLE
But Developing a High AccuracyPower Electronics Real-Time
Simulator is a Challenge
Myths or Reality?
IPST 2009, Kyoto - Japan
PC processors are too slowPC IOs are to slow
Interprocessor communication between PC systems is too slow
MATLAB, SIMULINK and the SimPowerSystems State-Space Solver cannot simulate large power systems as accurately as the classical Tustin nodal solver
PC Clusters cannot be scaled up to simulate large systems
Simulation of IGBT switching effects is not possible with standard processors Simulation of systems with MANY IGBTs cannot be made using PCs
PC technologies are not used and trusted by large organizations
CHALLENGES & SOLUTIONS
April 19, 2023IPST 2009, Kyoto - Japan 13
Computational powerFast and low latency IOs
INTEL/AMD CPU FPGA ( 250 nanosecond time step)
MULTI-CORE CPU PCI Express switches 20 nanoseconds/signal
Fast and low latency interprocessor communication
HOST-TARGET ARCHITECTURE
HOST Model Preparation Simulator Control User Interface Post-processing
Hardware & Software Simulator Architecture
April 19, 2023IPST 2009, Kyoto - Japan 14
Equipmentunder test
Signalconditioning,
monitoring andbreakout
HOST
REAL-TIMETARGET
Windows
REAL-TIME TARGET COMPUTER Execute the Real-Time Model Real-Time Signal Processing IO, Task and Communication
Scheduler Data Logging, Hard Disk and Ethernet
HILBox PC1HILBox PC1
PC
I E
XP
RE
SS
PC
I E
XP
RE
SS
CPUCPU
SimulinkModel
SimulinkModel
Single-, Dual-, or
Quad-Core
Single-, Dual-, or
Quad-Core
RT-LAB eMEGAsim Simulator Hardware Architecture
April 19, 2023IPST 2009, Kyoto - Japan 1515
CPUCPU
Sh.Mem.Sh.Mem.
SimulinkModel
SimulinkModel
Host/Target Architecture Windows QNX & RT-Linux RTOS SIMULINK/RTW based
Multi-core Processors Shared-Memory Multi-CPU board
HILBox PC1HILBox PC1
PC
I E
XP
RE
SS
PC
I E
XP
RE
SS
CUCU
SimulinkModel
SimulinkModel
Single-, Dual-, or
Quad-Core
Single-, Dual-, or
Quad-Core
RT-LAB eMEGAsim Simulator Hardware Architecture
April 19, 2023IPST 2009, Kyoto - Japan 1616
16 AO16 AO 16 AI16 AI
Carrier w (op511x)Carrier w (op511x)
16 DO16 DO 16 DI16 DI
Carrier (op5210)Carrier (op5210)
FastComFastCom
CPUCPU
Sh.Mem.Sh.Mem.
SimulinkModel
SimulinkModel
Host/Target Architecture Windows QNX & RT-Linux RTOS SIMULINK/RTW based
Multi-core processors Shared-Memory Multi-CPU board
Ultra-fast ReconfigurableFPGA-based I/Os 10 nanosecond time stamp
Very Fast Model Update Xeon & Opteron Target:
10-20 us time step Xilinx FPGA Target:
2.5 µs AD, 1 µs DA, 250 nanosecond update
FP
GA
(o
p51
42)
FP
GA
(o
p51
42)
16 DO16 DO 16 DI16 DI
Carrier (op5210)Carrier (op5210)
16 AO16 AO 16 AI16 AI
Carrier w (op511x)Carrier w (op511x)PCI Express
Xilinx Blockset
Model
Xilinx Blockset
Model
09/04/07
MODELS ON FPGA CHIPS WITH RT-XSG
RT-XSG integrates System Generator subsystems into the RT-LAB Parallel simulation paradigm.
Subsystem #1Simulink
Rate = 10µs DIO
AIO
RTW XSGRT-LAB
Simulink Model
Code Generation
Distributed Real-Time Model
DIO
AIO
Single/dual multi-core CPU PC FPGA card with embedded IO
Host PCSW
link
Ethe
rnet
link
Subsystem #2Simulink
Rate = 50µs
Subsystem #3Xilinx System Generator
Rate = 10ns
RT-XSG is compatible with • the RT-LAB Parallel simulation paradigm• xPC Target software from The MathWorks.
CHALLENGES & SOLUTIONS
April 19, 2023IPST 2009, Kyoto - Japan 18
Computational powerFast and low latency IOs
INTEL/AMD CPU FPGA ( 250 nanosecond time step)
MULTI-CORE CPU PCI Express switches 20 nanosecond/signal
Fast and low latency interprocessor communication
OK
OK
Scalability to simulate very large systems
100 3-phase busses on one PC! Cluster of 8-CPU Supercomputers PCI Express Switches Hundreds of 3-phase busses
HILBox PC2HILBox PC2FastComFastCom
PC
IP
CI
Supercomputer cluster FireWire INFINIBAND switch DOLPHIN SCI /PCIe
(2 to 5 us latency)
HILBox PC1HILBox PC1
PC
I E
XP
RE
SS
PC
I E
XP
RE
SS
CUCU
SimulinkModel
SimulinkModel
Single-, Dual-, or
Quad-Core
Single-, Dual-, or
Quad-Core
RT-LAB eMEGAsim Simulator Hardware Architecture
April 19, 2023 1919
16 AO16 AO 16 AI16 AI
Carrier w (op511x)Carrier w (op511x)
16 DO16 DO 16 DI16 DI
Carrier (op5210)Carrier (op5210)
FastComFastCom
CPUCPU
Sh.Mem.Sh.Mem.
SimulinkModel
SimulinkModel
Host/Target Architecture Windows QNX & RT-Linux RTOS SIMULINK/RTW based
Multi-core processors Shared-Memory Multi-CPU board
Ultra-fast ReconfigurableFPGA-based I/Os 10 nanosecond time stamp
Very Fast Model Update Xeon & Opteron Target:
10-20 us time step Xilinx FPGA Target:
2.5 µs AD, 1 µs DA, 250 nanos update
FP
GA
(o
p51
42)
FP
GA
(o
p51
42)
16 DO16 DO 16 DI16 DI
Carrier (op5210)Carrier (op5210)
16 AO16 AO 16 AI16 AI
Carrier w (op511x)Carrier w (op511x)PCI Express
Xilinx Blockset
Model
Xilinx Blockset
Model
Off-the-Shelf Supercomputer PC Cluster
FPGAIO
CONTROLLER& PROTECTIONCONTROLLER
& PROTECTIONCONTROLLER
& PROTECTION
72 -CPU ClusterCP
UC
PU
CP
UC
PU
Sh
.Mem
.S
h.M
em.
PCI Express Switch
8 CPUper PC
20 nanosecondsper signal
IPST 2009, Kyoto - Japan April 19, 2023 20
eMEGAsim Configuration for MTDC & AC Systems
IPST 2009, Kyoto - Japan April 19, 2023
CP
UC
PU
CP
UC
PU
Sh
.Mem
.S
h.M
em.
PCI Express
FPGA
PCI Express
SIGNAL MONITORING AND SCREW TERMINALS
IO (AD, DA, DIO)
CP
UC
PU
CP
UC
PU
Sh
.Mem
.S
h.M
em.
PCI Express
IO (AD, DA, DIO)
PCI Express
SIGNAL MONITORING AND SCREW TERMINALS
FPGA
CP
UC
PU
CP
UC
PU
Sh
.Mem
.S
h.M
em.
PCI Express
PCI Express
SIGNAL MONITORING AND SCREW TERMINALS
FPGA
CP
UC
PU
CP
UC
PU
Sh
.Mem
.S
h.M
em.
PCI Express
FPGA
PCI Express
SIGNAL MONITORING AND SCREW TERMINALS
8-CPU, AC System(100 3-phase busses)
8-CPUTwo Bipoles
AC & DC filters
8-CPUTwo Bipoles
AC & DC filters
IO (AD, DA, DIO)
IO (AD, DA, DIO)
PCI Express
SIGNAL MONITORING AND SCREW TERMINALS
FPGA
PCI Express
SIGNAL MONITORING AND SCREW TERMINALS
FPGA
IO EXPANSIONCHASSIS
IO (AD, DA, DIO) IO (AD, DA, DIO)
IO EXPANSIONCHASSIS
SIGNAL MONITORING AND SCREW TERMINALS
FPGA
PCI Express
IO (AD, DA, DIO)
SIGNAL MONITORING AND SCREW TERMINALS
FPGA
PCI Express
IO (AD, DA, DIO)
IO EXPANSIONCHASSIS
IO EXPANSIONCHASSIS
PCI Express Switch (12 ports, 20 Gbits/s)
DISTRIBUTED PROCESSING AND IO ARCHITECHTURE
8-CPU, AC System(100 3-ph busses)
22
eMEGAsim 24-CPU 330-Bus Power System
System Diagram
2B6
3M15
3B15
3B14
3B13
3B10
3B9
3B12
3B11
2B9
2B112B10
2M10
2B4
2B52B82B7
2B1
2B2
2M1
2B3
3B8
1B9
3B7
3B5
3B6
1M10
1B51B61B71B8
1B4
1B3
1B2
1B1
1M1
3B4
3B3
3B2
3M4
3M3
3M11
1B10
1B111B12
1B13
1B16
2B12
1B151B14
2B13
2B14
2B15
2B16
3B1
3B16
PLANT
TRANSFORMER
BUS
LOAD
500KVHVAC
5B6
6B95B9
5B115B10
5M10
5B4
5B55B85B7
5B1
5B2
5M1
5B3
6B8
4B9
6B7
6B5
6B6
4M10
4B54B64B74B8
4B4
4B3
4B2
4B1
4M1
6B4
6B3
6B2
6M4
6M3
4B10
4B114B12
4B13
4B16
5B12
4B154B14
5B13
5B14
5B15
5B16
6B1
6B15
6B14
6B136B126B11
6M11
6B16
6B10
8B6
9M15
9B15
9B14
9B13
9B10
9B9
9B12
9B11
8B9
8B11 8B10
8M10
8B4
8B58B8 8B7
8B1
8B2
8M1
8B3
9B8
7B9
9B7
9B5
9B6
7M10
7B5 7B6 7B7 7B8
7B4
7B3
7B2
7B1
7M1
9B4
9B3
9B2
9M4
9M3
9M11
7B10
7B11 7B12
7B13
7B16
8B12
7B15 7B14
8B13
8B14
8B15
8B16
9B1
9B16
11B6
12B911B9
11B11 11B10
11M10
11B4
11B511B8 11B7
11B1
11B2
11M1
11B3
12B8
10B9
12B7
12B5
12B6
10M10
10B5 10B6 10B7 10B8
10B4
10B3
10B2
10B1
10M1
12B4
12B3
12B2
12M4
12M3
10B10
10B11 10B12
10B13
10B16
11B12
10B15 10B14
11B13
11B14
11B15
11B16
12B1
12B15
12B14
12B13 12B12 12B11
12M11
12B16
12B10
14B6
15B15
15B14
15B13
15B10
15B9
15B12
15B11
14B9
14B11 14B10
14M10
14B4
14B514B8 14B7
14B1
14B2
14M1
14B3
15B8
13B9
15B7
15B5
15B6
13M10
13B5 13B6 13B7 13B8
13B4
13B3
13B2
13B1
13M1
15B4
15B3
15B2
15M4
15M3
15M11
13B10
13B11 13B12
13B13
13B16
14B12
13B15 13B14
14B13
14B14
14B15
14B16
15B1
15B16
17B6
18B9 17B9
17B11 17B10
17M10
17B4
17B517B8 17B7
17B1
17B2
17M1
17B3
18B8
16B9
18B7
18B5
18B6
16M10
16B5 16B6 16B7 16B8
16B4
16B3
16B2
16B1
16M1
18B4
18B3
18B2
18M4
18M3
16B10
16B11 16B12
16B13
16B16
17B12
16B15 16B14
17B13
17B14
17B15
17B16
18B1
18B15
18B14
18B13 18B12 18B11
18M11
18B16
18B10
New MILESTONE as of JUNE 2009
# of bus 330
# of gen.
42(+1)
# of load
90
# of DPL
517
Product versus Performance Requirements
April 19, 2023IPST 2009, Kyoto - Japan 23
20 KHz50 us
40 kHz25 us
8
20
Nu
mb
er o
f C
PU
Model Sampling Rate and Step Requirement(when OPAL-RT interpolation algorithm is used)
SlowDynamics
Medium to Fast Dynamics & Transients Very Fast TransientsUltra- fast
Transients
100
10 kHz100 us
2
4
1
1 kHz1000 us
RT-LABMP4
Mechatronics
xPC
RT-LABMP8
100 kHz10 us
1 MHz1us
250 kHz5 us
RT-XSGFPGA
CHALLENGES & SOLUTIONS
April 19, 2023IPST 2009, Kyoto - Japan 24
Computational powerFast and low latency IOs
INTEL/AMD CPU FPGA (250 nanosecond time step)
MULTI-CORE CPU PCI Express switches 20 nanoseconds/signal
ARTEMiS L-STABLE ORDER 5 SOLVER
Finite Element Analysis for motors
Fast and low latency interprocessor communication
Precise and stable network solver
OK
OK
Scalability to simulate very large systems
100 3-phase busses on one PC! Cluster of 8-CPU Supercomputers PCI Express Switches Hundreds of 3-phase busses
OK
April 19, 2023 25
Blocksets
StateflowStateflowStateflowStateflowStateflowToolboxes
StateflowStateflowRTW
RT-LAB – QNX – LINUX – XSG -ORCHESTRA
RT PC CLUSTER IO Interface FPGA
RT-LAB Open Software Architecture
April 19, 2023 26
Blocksets
StateflowStateflowSimPowerSystemsStateflowStateflowStateflow
Toolboxes
StateflowStateflowRTW
RT-LAB – QNX – LINUX – XSG -ORCHESTRA
RT PC CLUSTER IO Interface FPGA
ARTEMiS
RT-LAB Open Software Architecture
SIMPLE R-L-C CIRCUIT SWITCHING (ARTEMiS (L-Stable) vs Trapezoidal Integration Method)
April 19, 2023IPST 2009, Kyoto - Japan 27
ART5 -100 us &Trapezoidal-10us
Trapezoidal-100us(phase shift)
Fr: 1125 Hz(890 us)
Effect of Solvers– PI Line Switching
141 states / 141 nodes running real-time on 1 CPUwith a fixed time-step of 50 us
100 km of line modelized with 10 pi segments
100 km of line modelized with 10 pi segments
10 km of line modelized with 1 pi segments
Line energization transients
3
Out3
2
Out2
1
Out1
i+ -
i l ine3
i+ -
i l ine2
i+ -
i l ine1
Xc2
Xc1
Xc
Xb2
Xb1
Xb
Xa2
Xa1
Xa
v+-
Voltage Measurement
PulseGenerator
c
12
c
1 2
c
1 2
Line-end terminal voltage comparison of 3 different solvers:
Simulink/SPS fixed-step TUSTIN (Order 2), EMTP-RV (Tustin / Euler),
and L-Stable (Order 5 ARTEMiS )141 states, 141 nodes, 50 us time step
IPST 2009, Kyoto - Japan April 19, 2023 28
PI Line Switching – Simulink/SimPowerSystems
0.03 0.04 0.05 0.06 0.07 0.08 0.09-50
0
50
100
150
200
250
Simulink/SimPowerSystems fixed-step discrete solver (Tustin)
0.718 0.72 0.722 0.724 0.726 0.728 0.73
0
50
100
150
200 10 us
Reference at 1 us 50 us 100 us
2 ms
April 19, 2023 29IPST 2009, Kyoto - Japan
PI Line Switching
0.72 0.721 0.722 0.723 0.724 0.725 0.726
0
50
100
150
200
0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2
0
50
100
150
200
250ARTEMIS art5 fixed-step discrete solver
10 us
Reference at 1 us
50 us
100 us
1 ms
0.72 0.721 0.722 0.723 0.724 0.725 0.726
0
50
100
150
200
0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2
0
50
100
150
200
250EMTP-RV (trapezoidal and euler backward)
10 usReference at 1 us
50 us
100 us
– ARTEMiS art5
TUSTIN
April 19, 2023 30IPST 2009, Kyoto - Japan
ENTERGY Power Grid (New Orleans, USA)
April 19, 2023IPST 2009, Kyoto - Japan 31
86 3-phase busses86 lines23 loads
(7 CPU, < 50 us)
COMPLEX CIRCUIT (ENTERGY – New Orleans, USA)(ARTEMiS (L-Stable) vs Trapezoidal Integration Method)
April 19, 2023IPST 2009, Kyoto - Japan 32
ARTEMiS5 & EMTP-RV -1us and 50 us
ARTEMiS5 &EMTP-RV -50 us
ARTEMiS5 & EMTP-RV -1us
Complex Power Grid Simulated with 7 CPUs 3-Phase Fault Application CasePaper presented at IPST Japan, June 2009
April 19, 2023 33
Blocksets
StateflowStateflowSimPowerSystemsStateflowStateflowStateflow
Toolboxes
StateflowStateflowRTW
RT-LAB – QNX – LINUX – XSG -ORCHESTRA
RT PC CLUSTER IO Interface FPGA
ARTEMiSRT-EVENTSTestDRIVE GUI
Test Manager
IO Lib
JMAG-RT
XILINX SG
RT-LAB Open Software Architecture
EMTP-RV INTERFACE
JMAG-2: Finite Element-based Motor Model JMAG-RT
April 19, 2023IPST 2009, Kyoto - Japan 34
Stator Outer Diameter [mm]
112
Rotor Outer Diameter [mm]
55
Poles 4
Slots 24 Cross-section of IPM machine
In many cases,motor current and torque distortion
caused by saturation effects and rotor construction cannot be neglected.
Finite Element-based models enable simulation of these distortions.
HIL System with the capability to execute
these models in Real-Time enables testing of control algorithms designed to cancel these
effects
JMAG-3: PMSM Model: Comparison of FEM, JMAG-RT & dq Ideal Models
Courtesy of Japan Research Institute, Limited
0.0
0.5
1.0
1.5
2.0
2.5
3.0
0.00 0.02 0.04 0.06 0.08 0.10
Time[s]
Torq
ue[N
m]
JMAG-RT
FEA (JMAG)
Ld, Lq
April 19, 2023 35IPST 2009, Kyoto - Japan
CHALLENGES & SOLUTIONS
April 19, 2023IPST 2009, Kyoto - Japan 36
Computational powerFast and low latency IOs
INTEL/AMD CPU FPGA ( 250 nanosecond time step)
MULTI-CORE CPU PCI Express switches 20 nanoseconds/signal
ARTEMiS L-STABLE ORDER 5 SOLVER
FEA for motors
Fast and low latency interprocessor communication
Precise and stable network solver
OK
OK
Scalability to simulate very large systems
100 3-phase bus on one PC! Cluster of 8-CPU Supercomputers PCI Express Switches Hundreds of 3-phase busses
OK
OK Simulation of IGBT switching
Interpolation (since 2000 in theautomotive industry!)
Comparison … analog set up
April 19, 2023 37
Blocksets
StateflowStateflowSimPowerSystemsStateflowStateflowStateflow
Toolboxes
StateflowStateflowRTW
RT-LAB – QNX – LINUX – XSG -ORCHESTRA
RT PC CLUSTER IO Interface FPGA
ARTEMiSRT-EVENTSTestDRIVE GUI
Test Manager
IO Lib
JMAG-RT
XILINX SG
RT-LAB Open Software Architecture
EMTP-RV INTERFACE
Applications of RT-Events and TSB: Inverter
Accurate simulation with Time Stamped Bridge with Tcarrier/Ts=10
(This case: 10 kHz carrier & 10 us time step)
SimPowerSystems
TSB RT-Events and the TSB
Interpolating inverter models (Time Stamped Bridges) are special high-speed, high accuracy switching inverter models from the RTeDrive blockset
• Use interpolation to obtain a fixed step simulation with an accuracy equivalent to a variable-step solver
38
Test Results: Effect of Carrier Frequency
April 19, 2023IPST 2009, Kyoto - Japan 39
0 0.003 0.006 0.009 0.012-20
-10
0
10
20
Motor Current [A]
Time [sec]
0 0.003 0.006 0.009 0.012-20
-10
0
10
20Motor Current [A]
Time [sec]
0 0.003 0.006 0.009 0.012-20
-10
0
10
20
Motor Current [A]
Time [sec]
0 0.003 0.006 0.009 0.012-20
-10
0
10
20
Motor Current [A]
Time [sec]
0 0.003 0.006 0.009 0.012-20
-10
0
10
20
Motor Current [A]
Time [sec]
0 0.003 0.006 0.009 0.012-20
-10
0
10
20
Motor Current [A]
Time [sec]
HIL Simulation Actual Physical System
© Opal-RT
© Opal-RT
PWM freq. = 4.5 kHz
(PWM freq. = 9.0 kHz
PWM freq. = 2.25 kHz
PWM = 9 kHz>> 111 micros1% DF = 1.1 usTstep = 10 us!!
Test results are courtesy of
Mitsubishi Electric
April 19, 202340
Test Results: Effect of Dead Time
0 0.003 0.006 0.009 0.012-20
-10
0
10
20
Motor Current [A]
Time [sec]
0 0.003 0.006 0.009 0.012-20
-10
0
10
20
Motor Current [A]
Time [sec]
0 0.003 0.006 0.009 0.012-20
-10
0
10
20Motor Current [A]
Time [sec]
0 0.003 0.006 0.009 0.012-20
-10
0
10
20
Motor Current [A]
Time [sec]
HIL Simulation Actual Physical System
(b) Dead Time = 8.4 µs
(a) Dead Time = 4.2 µs
© Opal-RT
PWM = 9 kHz>> 111 micros
DT = 4 us or lessTstep = 10 us!!
IPST 2009, Kyoto - Japan
Test results are courtesy of
Mitsubishi Electric
Test Results: Effects of Interpolation
April 19, 2023IPST 2009, Kyoto - Japan 41
Off-line simulation:• SimPowerSystems• Native Simulink• Ts = 1 µs
0 0.05 0.1 0.15 0.2-25
-20
-15
-10
-5
0
5
10
15
20
25
Time [sec]
Motor Current [A]
0 0.05 0.1 0.15 0.20
5
10
15
20
25
Time [sec]
Motor Torque [Nm]
RT simulation:• RT-Events
with interpolation• Ts = 10 µs
0 0.05 0.1 0.15 0.2-25
-20
-15
-10
-5
0
5
10
15
20
25
Time [sec]
Motor Current [A]
0 0.05 0.1 0.15 0.20
5
10
15
20
25
Time [sec]
Motor Torque [Nm]
0 0.05 0.1 0.15 0.2-25
-20
-15
-10
-5
0
5
10
15
20
25
Time [sec]
Motor Current [A]
0 0.05 0.1 0.15 0.20
5
10
15
20
25
Time [sec]
Motor Torque [Nm]Off-line simulation:
• SimPowerSystems• Native Simulink• Ts = 10 µs
Currents Torque
RESULTS VERY CLOSETO THEORETICAL RESPONSE
WITH 9kHz (110 us) and10 us time step
: BAD
: Betterbut 1 us Tstep
250 nanosis required
Test results are courtesy of
Mitsubishi Electric
CHALLENGES & SOLUTIONS
April 19, 2023IPST 2009, Kyoto - Japan 42
Computational powerFast and low latency IOs
INTEL/AMD CPU FPGA ( 250 nanosecond time step)
MULTI-CORE CPU PCI Express switches 20 nanoseconds/signal
ARTEMiS L-STABLE ORDER 5 SOLVER
FEA for motors
Fast and low latency inter processor communication
Precise and stable network solver
OK
OK
Scalability to simulate very large systems
100 3-phase bus on one PC! Cluster of 8-CPU Supercomputers PCI Express Switches Hundreds of 3-phase busses
OK
OK Simulation of IGBT switching
Simulation of MANY IGBTs and Thyristors
Interpolation (since 2000 in theautomotive industry!)
Comparison … analog set upOK
Tests with hundreds of switches
3-level STATCOM with 72 Switches
A simplified STATCOM schematic (72-switch)
*extracted from ‘Operating performance of the
STATCOM in the Kanzaki substation’, CIGRE
2005, by H. Yonezawa, et al. MITSUBISHI
Electric
20 microseconds, 3 CPUs with controller
1000 times faster than conventional simulation software
Reference model In EMTP/RV (3us)
vs Simulink/SPS/ RT-LAB (50 us)
IPST 2009, Kyoto - Japan April 19, 2023 43
Multi-Terminal HVDC System (114 Thyristors in Total)
SVC
SVC
SVC
SVC
SVC
SVC
8 x 12-pulse AC-DC Converters and Controllers 6 SVCs and Controllers Several AC Filters 24 DC Filter Banks and DC Lines Several AC Machines and Controls
IPST 2009, Kyoto - Japan April 19, 2023 44
System Decoupling for Multi-CPU Parallel Simulation
SVC
SVC
SVC
SVC
SVC
SVC
SS1
SS2SS3SS4
SS5 SS6 SMHVDC controllerSM controller
50 micros, 6 CPUs for Power System (XEON 2.3 GHz)
100 us, 1 CPU for Controllers
IPST 2009, Kyoto - Japan
8 x 12-pulse AC-DC Converters & Controllers
6 SVCs & Controllers Several AC Filters 24 DC Filter Banks & DC Lines Several AC Machines & Controls
April 19, 2023 45
10 micros possiblefor 12-pulse Converters(estimation with 3.2 GHz CPU)
Advanced Firing Technique for HVDC
April 19, 2023IPST 2009, Kyoto - Japan 46
SLOW Idc RAMP
ARTEMiS 10us and 50 uswith compensation
ARTEMiS 10us NO compensation
ARTEMiS 50us NO compensation
April 19, 2023IPST 2009, Kyoto - Japan 47
ARTEMiS 10us and 50 uswith compensation
ARTEMiS 10us NO compensation
ARTEMiS 50us NO compensation
SLOW Idc modulation
Advanced Firing Technique for HVDC-2
Ex:41 buses71 lines11 plants3 HVDC15 loads
Applications - eMEGAsim: Large Power Grids with Converters
April 19, 2023IPST 2009, Kyoto - Japan 48
Regional GridRegional GridEmbedded DistributionEmbedded Distribution
Ships, Aircraft, SpacecraftShips, Aircraft, Spacecraft
Train Traction Systems and AC Network Feeding Systems
Train Traction Systems and AC Network Feeding Systems
Very Large Wind FarmsVery Large Wind Farms
7 CPUs, 46 microseconds7 CPUs, 20 microseconds
Farm of 10 Wind Turbines and a Power Grid
IPST 2009, Kyoto - Japan
All doubly-fed generators and IGBT switching is simulated in detail (no average models)
Simulation time using one CPU and conventional solvers for 60 s:
200 min or 3.2 hrs
Doubly-fed IM
2 CPU 144 micros RT Offline(6.45 ms) 3 CPU 72 micros RT Offline(6.22 ms) 4 CPU 42 micros RT Offline(6.16 ms)
6 CPU 28 micros RTDual - Quad core (8cpu) No IO.
200 times faster than simulation using one CPU and conventional solvers
April 19, 2023 49
OK
CHALLENGES & SOLUTIONS
April 19, 2023IPST 2009, Kyoto - Japan 50
Computational powerFast and low latency IOs
INTEL/AMD CPU FPGA ( 250 nanosecond time step)
MULTI-CORE CPU PCI Express switches 20 nanoseconds/signal
ARTEMiS L-STABLE ORDER 5 SOLVER
FEA for motors
Fast and low latency inter processor communication
Precise and stable network solver
OK
OK
Scalability to simulate very large systems
100 3-phase bus on one PC! Cluster of 8-CPU Supercomputers PCI Express Switches Hundreds of 3-phase busses
OK
OK Simulation of IGBT switching
Simulation of MANY IGBTs and Thyristors and large power systems
Interpolation (since 2000 in theautomotive industry!)
Comparison … analog set upOK
Tests with hundreds of switches
MATLAB > 20 years, >1M users SPS> Hydro-Québec> 14 years OPAL-RT: 12 yrs experience, Customers
in all markets (aero, auto, power)
Technologies trusted by large organizations
OK
OPAL-RT Customers – Electric & Power Electronics
CONVERTEAM
TRAIN & HVDC
More Electrical Aircraft
French Navy
WUHAN & SHANGHAIElectrical Ship Institutes
IPST 2009, Kyoto - Japan April 19, 2023 51
OPAL-RT Customers – Educational Market
Université Saint-Joseph
GuatalajaraMexico
LEGGrenoble
LILLEFrance
Universidad del Pais Vasco - Spain
IITINDIA
April 19, 2023 52IPST 2009, Kyoto - Japan
OPAL-RT Customers – Defense & Aerospace
IPST 2009, Kyoto - Japan April 19, 2023 53
CHALLENGES & SOLUTIONS
April 19, 2023IPST 2009, Kyoto - Japan 54
Computational powerFast and low latency IOs
INTEL/AMD CPU FPGA ( 250 nanosecond time step)
MULTI-CORE CPU PCI Express switches 20 nanosecond/signal
ARTEMiS L-STABLE ORDER 5 SOLVER and FEA for motors
Fast and low latency interprocessor communication
Precise and stable network solver
OK
OK
Scalability to simulate very large systems
100 3-phase busses on one PC! Cluster of 8-CPU Supercomputers PCI Express Switches Hundreds of 3-phase busses
OK
OK Simulation of IGBT switching Simulation of MANY IGBTs
Interpolation (since 2000 in theautomotive industry!)
Comparison … analog set up Tests with hundreds of IGBT MATLAB > 20 years, >1M users SPS> Hydro-Québec> 14 years OPAL-RT: 12 yrs experience, customers
in all markets (aero, auto, power)
Technologies trusted by large organizations
OK
OK
Affordability From 25 k$ and up Price will go down with volume
CONCLUSIONS
April 19, 2023IPST 2009, Kyoto - Japan 55
High-accuracy parallel Real-Time Simulators with sub-microsecond reaction time are becoming affordable commodity equipment
Such simulators are now used by leading power electronic equipment manufacturers such as Mitsubishi, Hitachi, GE, Toyota and by military & institutional R&D centers and
universities.
… for Power Grids, Ships, Aircraft, Hybrid Vehicles, Wind Farms …
ALL EQUIPMENT AND SYSTEMS THAT CAN BE MANUFACTURED AND <IMAGINED> …
… WITH TECHNOLOGIES THAT WILL BE AVAILABLE OVER THE NEXT 10 – 20 YEARS …
… CAN BE DESIGNED AND TESTED WITH FULLY DIGITAL HIGH-BANDWIDTH SIMULATORS …
… USING COMMERCIAL TECHNOLOGIES …
… AFFORDABLE FOR SMALL COMPANIES AND UNIVERSITIES
POWER ELECTRONIC DEVICES FOR AUTOMOTIVE, AIRCRAFT, SHIPS, TRAINS, POWER GRIDS AND MICRO GRIDS ARE CONVERGING
… SIMULATORS ARE TOO!
THANK YOU
Live demonstration of
eMEGAsim Fully Digital Real Time Digital Simulator
is available at meeting room #4
Kyoto International Community House
3, 4, 5 June 2009, from 13:00 to 20:00
April 19, 2023IPST 2009, Kyoto - Japan 56
ADVANTAGES OF A MODERN AND OPEN
DIGITAL REAL-TIME SIMULATORARCHITECTURE OVER CUSTOM SIMULATORS
April 19, 202357IPST 2009, Kyoto - Japan
Advantages – Standard Hardware
MORE POWERFUL CPU EVERY 6 MONTHS FULL MULTI-CORE SUPPORT:
INTEL and AMD (no recompiling required) 2 X 4-Core, now, 2 X 8 Core soon in 2009 Large on-chip fast shared cache memory Very-large on-board shared memory
DOLPHIN 10 Gbits/s communication fabric 5 microsecond worst case latency Very high bandwidth Large cluster of 8-core Supercomputers
COMMERCIAL IO AND FIREWIRE BOARDS OP5000 RECONFIGURABLE FPGA IOs
April 19, 202358IPST 2009, Kyoto - Japan
Standard Operating Systems
QNX NEUTRINO RTOS Used for more than 30 years Embedded Medical, Trains and Cars POSIX Compliant Multi-core support FPGA embedded processor support
OPTIONAL LINUX RTOS based on REDHAT CPU SHIELDED TECHNOLOGIES 1 to 3 us latency
STANDARD MULTI-CORE WINDOWS PCs FAST PARALLEL OFF-LINE SIMULATION OFF-LINE simulation server
April 19, 202359IPST 2009, Kyoto - Japan
Advantages – Standard Software
FULL AND NATIVE INTEGRATION WITH MATLAB, SIMULINK and SimPowerSystems
AUTOMATIC CODE GENERATION WITH RTW Large power grid equipment manufacturers now
generate C code for protection and control systems with RTW
ORCHESTRA PUBLISH & SUBSCRIBE INTERFACE INTEGRATION WITH INDUSTRY-STANDARD
MULTI-DOMAIN SIMULATION SOFTWARE AMEsim, DYMOLA, TMW toolboxes
April 19, 202360IPST 2009, Kyoto - Japan
Advantages – FPGA Processors
USER PROGRAMABLE XILINX FPGA CHIPS XILINX System Generator HDL code generator SIMULINK graphical editor
DIRECT INTERFACE WITH IO CONVERTERS Synchronized data acquisition from user code Minimum latency
MULTIRATE MODELS Fast SG FPGA subsystems communicate with Subsystems running on INTEL multi-core with RTW
SUB-MICROSECOND APPLICATIONS Fast power electronic controllers, PWM, PLL, Fast protection system Filters, Sensors and actuators emulators …
April 19, 202361IPST 2009, Kyoto - Japan
Advanntages – FPGA Motor Drive Models
HIGH-ACCURACY MOTOR DRIVE MODELS IGBT convertor and motor 250 nanosecoond model
loop time Finite Element-based motor model (JMAG) Phase-domain model to simulate inductor and back
EMF non linearity (precise torque simulation) DESIGN, PROTOTYPE AND TEST OF ASIC
CONTROLLER AND FAST IGBT PROTECTION 1.3 us latency max between gate signals and motor
current output A/D can be synchronized on PWM signal Dead time monitoring Overcurrents and over temperature and other …
April 19, 202362IPST 2009, Kyoto - Japan
Advantages – VSC with 100+ Switches
TYPICAL VSC APPLICATIONS MULTI-DRIVE TRACTION SYSTEMS FACTS, STATCOM, HVDC & SVC LIGHT IN-LAND DISTRIBUTED ENERGY SYSTEMS MORE ELECTRICAL AIRCRAFT ON-SHIP ENERGY GENERATION, DISTRIBUTION
AND PROPULSION SYSTEM Hybrid Vehicles: Commercial, Off-highway, Military…
CHALLENGES Very large number of switches Two-level and Multi-level Very fast switching and small dead time Switching events occur between model time steps
April 19, 202363IPST 2009, Kyoto - Japan
Advantages – VSC with 100+ Switches
STATE-SPACE AND SIMULINK MODELS Unlimited number of IGBT switches and converters Direct model integration in SIMULINK Simulation of dead-time and time delays smaller than
the simulation time steps Library of single- and multi-level IGBT converter
models with direct integration (interpolation) Pioneer of interpolation in use for 5 years Capability to simulate faults and open circuits
STATE-SPACE IS BETTER THAN CONVENTIONAL NODAL TECHNIQUES FOR VSC No MATRIX recalculation; easier to interpolate
April 19, 202364IPST 2009, Kyoto - Japan
Advantages – VSC with 100+ Switches
VALIDATED WITH ANALOG SETUP MITSUBISHI and TOYOTA CONVERTEAM (France) is replacing their analog
test benches with RT-LAB Simulators UNDER EVALUATION BY MAJOR POWER
ELECTRONIC EQUIPMENT MANUFACTURERS Applications with hundreds of IGBTs MITSUBISHI, TOSHIBA, FUJI
REQUIRE ALSO FAST DIO (FPGA) REQUIRE RT-EVENTS TO SIMULATE PWM
CONTROLS
April 19, 202365IPST 2009, Kyoto - Japan