hmi antivirus testing presentation (ms powerpoint) - joe falco
DESCRIPTION
TRANSCRIPT
Intelligent Systems DivisionManufacturing Engineering Laboratory
Performance Testing: The Effects of Antivirus Software on the
Operation of PC Based HMI Software
Joe Falco Manufacturing Engineering Laboratory
National Institute of Standards and Technology
February 18, 2004
NIST Industrial Control Security Testbed Architecture
Bottling Plant Simulation
• DeviceNet I/O network
• Three controller options
• PC-based software PLC
• Modicon hardware PLC
• DeltaV Hybrid Controller
• SQL database for data logging
Water Distribution SCADA Simulation
• Ultrasonic Level Transmitters• Analog Flow Meters• Liquid Level Switches• Centrifugal Pumps
• MTU Allen-Bradley ControlLogix/Flex IO• RTUs Allen-Bradley SLC500• DNP 3.0 Serial• Ethernet
Performance Testing
Provide performance measures of PC based control software execution vs. modes of operation of concurrently executing security software
Note: Any results will be reported in aggregate,
or with any vendor-identifying information
removed.
Antivirus vs. HMI Performance
• Map functionality of both antivirus software packages.• Configure HMI software at upper and lower bounds.• Record antivirus installation and default configurations.• Test procedures least intrusive to most intrusive.• Design test procedures to be repeatable.• Monitor PC system resources (CPU, Network Traffic).• Monitor communication packets from HMI to PLC.• Compare loads with and without antivirus software.• Inject test viruses from available access points.• Include testing during virus definition updates.
Antivirus/HMI Test Matrix
HMI-1 vs. AV-1 HMI-2 vs. AV-1
HMI-1 vs. AV-2 HMI-2 vs. AV-2
HMI-1 HMI-2
AV-1
AV-2
Current Status
• Antivirus application functionality mapping completed
• HMI-1 programmed for lower end operation• Performed preliminary testing between
HMI-1, AV-1 and AV-2 applications
Initial Testing
• Manual Scanning of Hard Drive• Manual Scanning of Floppy Drive• Active Scanning• AV1 Manual Scan of Hard Drive over different
CPU priority settings• Data packets collected over 1 minute period• Analyze single data variable packet – calculate
time between consecutive messages.• Baseline• Antivirus mode of operation/ no virus• Antivirus mode of operation/ virus present
Manual Scan of Hard Drive (HMI1/AV1&AV2)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.71 19 37 55 73 91 109
127
145
163
181
199
217
235
253
271
289
307
325
343
361
379
Message Count (1 minute of data collection)
Tim
e B
etw
ee
n C
on
se
cu
tiv
e M
es
sa
ge
s (
se
co
nd
s) Baseline - no scanning
AV1 scanning - no viruses
AV1 scanning - 3 viruses quarantined
AV2 scanning - no viruses
AV2 Scanning - 3 viruses quarantined
Start scan
End scan AV2 End scan
AV1
Directory size : 2.3Gb
Virus Files used: eicar.com eicar_com.zip eicarcom2.zip
More message delays due to AV2 result in fewer messages sent
Manual Scan: Hard Drive
Manual Scan Floppy Drive (HMI1/AV1&AV2)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.71 16 31 46 61 76 91 106
121
136
151
166
181
196
211
226
241
256
271
286
301
316
331
346
361
376
391
Message Count (1 minute of data collection)
Tim
e B
etw
een
Co
nse
cuti
ve M
essa
ges
(se
con
ds)
Baseline - no scanning
AV1 scanning - no viruses
AV1 scanning - 3 viruses quarantined
AV2 scanning - no viruses
AV2 scanning - 3 viruses quarantined
Start scan
Virus Files used: eicar.com eicar_com.zip eicarcom2.zip
Note: In all cases the floppy contained a 1Mb uninfected file
Manual Scan: Floppy Drive
Active Scanning Enabled (HMI1/AV1&AV2)
0
0.05
0.1
0.15
0.2
0.25
0.31 16 31 46 61 76 91 106
121
136
151
166
181
196
211
226
241
256
271
286
301
316
331
346
361
376
391
Message Count (1 minute of data collection)
Tim
e B
etw
een
Co
nse
cuti
ve M
essa
ges
(se
con
ds)
Baseline - no scanning
File copy - no scanning
File copy - AV1 scanning (1 virus quarantined)
File copy - AV2 scanning (1 virus quarantined)
Directory containing 1Mb file and the eicar.com file are copied to the hard drive
while active scanning is enabled.
Initiate File
Active Scanning
Manual Scanning of Hard Drive: CPU Priority Settings (HMI1/AV1)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.51 18 35 52 69 86 103
120
137
154
171
188
205
222
239
256
273
290
307
324
341
358
375
Message Count (1 minute of data collection)
Tim
e B
etw
een
Co
nse
cuti
ve M
essa
ges
(se
con
ds)
Zero CPU Priority
20% CPU Priority
60% CPU Priority
100% CPU Priority
18.0
Large message delay results in fewer messages sent
Time between consecutive messagesexcedes 18 seconds
at higher priority settings
Directory size : 2.3Gb
Start scan
AV1 : CPU Priority Settings
Next Steps
• Program HMI-1 application at an upper end.• Program HMI-2 application at lower and upper
end.• Document a set of performance test methods
based on results of initial testing.• Perform testing across test methods.• Continue efforts using other security
applications such as personal firewalls and control applications such as software PLCs
Summary
• Introduction to the NIST Process Control Security Testbed.
• Development of performance methods to assess the effects of security software on the performance of PC based control software.
• Presented initial test results for effects of antivirus software on the performance of HMI software.
• Discussed future activities in this area.