Resource-Efficiency Testing
Introduction to Resource‐Efficiency Testing
• Introduction
• Metrics Acquisition - Windows Performance Monitor
• OPC Server test plan model
• OPC Client test plan model
• Network Failure Simulation Batch File
• Test Reports
• Next Steps
Resource-Efficiency TestingContents
Resource-Efficiency TestingIntroduction (1 of 3)
What is it? 36-hour stress-test to verify no resource leaks exist
in your product, e.g. CPU, memory, threads etc.
More info? Windows Performance Monitor is used in the lab; the
performance counter details are available online:www.opcfoundation.org/ua/ctt/help/testcases/server/all/testcases.html#Efficiency
technet.microsoft.com/en-us/library/cc768048.aspx
What is tested? UA Servers/Clients, OPC Classic Servers/Clients
Easy to test? Yes, but several other OPC products are required to
generate a load on the product being tested.
4/19/2012 © 2012 OPC Foundation 3
You
Resource-Efficiency TestingIntroduction (2 of 3)
4/19/2012 © 2012 OPC Foundation 4
How it works: UA and Classic products are tested the same way:
Servers:- multiple device (PLC) connections; 1 is intermittent
- 5 or more OPC Client connections, 1 is intermittent
Clients: - 5 or more OPC Server connections, 1 is intermittent
Load is placed on the product for stress-testing.
Resources are measured over 36-hours.
Test authors? Compliance Working Group
Resource-Efficiency TestingIntroduction (3 of 3)
How do I pass? No resource leaks or performance degradation.
Am I Certified? No. It means that your product is a good candidate
for certification testing in a Test Lab.
Lab tools used? Windows Performance Monitor, network fault
simulation BAT file, also:
Servers: 2 or more PLCs; 5 or more OPC Clients.
Clients: 5 or more OPC Servers.
Test when? As frequently as possible, and on any official build
whether it is alpha, beta, or release etc.
4/19/2012 © 2012 OPC Foundation 5
Resource-Efficiency TestingWindows Performance Monitor (1 of 6)
4/19/2012 © 2012 OPC Foundation6
Configure the Windows Performance monitor to record the performance counters for 36‐hour duration• Click Start, click in the Search box, type perfmon.msc, and then press ENTER.
Perfmon.msc
NP4
Resource-Efficiency TestingWindows Performance Monitor (2 of 6)
4/19/2012 © 2012 OPC Foundation7
Create a Data Collector Set to store the data values recorded for the 36‐hour test.
Specify the directory you want to store the
file
Create a new Data collector set to record the data
Click finish to create the Data collector
set
Resource-Efficiency TestingWindows Performance Monitor (4 of 6)
4/19/2012 © 2012 OPC Foundation8
Configure the Process, Processor, Threads, Handles, and other counters for the instances of your product:
Step 3: Pick the Client/Server Process
(must be running to appear in the this list)
Step 4: Choose the performance counters
from the list
Step 1: Double click “Performance Counter ” to select
the counters
Step 2: Click “Add” to specify the resources to monitor
Step 5: Select the log format. Test lab uses comma separated format to record the values for the counters
selected.
Resource-Efficiency TestingWindows Performance Monitor (5 of 6)
4/19/2012 © 2012 OPC Foundation9
Setup a 36‐hour stop condition in the Data Collector Set properties to stop recording:
Setup the duration to 36
hours
Click OK to apply the changes
Resource-Efficiency TestingWindows Performance Monitor (6 of 6)
4/19/2012 © 2012 OPC Foundation10
Setup the stop condition in the Data Collector Set properties to stop recording the data values after 36 hours.
Measurements can be displayed in a chart, as well as being logged to a file (as previously configured).
Click the Triangle icon to start monitoring…
Resource-Efficiency TestingTest Plan Model – Server (1 of 2)
4/19/2012 © 2012 OPC Foundation11
Device 1(e.g. PLC)
Device 2(e.g. PLC)
Remote PC1Classic/UA Client
with max no of Tags Supported
• An OPC Server is configured with 2 devices (PLCs)• Typically, 5 Clients are connected, with approx. 1000+ items each• Make 1 Device and 1 Client go online/offline every 2‐minutes
Remote PC2Classic/UA Client
with max no of Tags Supported
Remote PC3Classic/UA Client
with max no of Tags Supported
Remote PC4Classic/UA Client
with max no of Tags Supported
Remote PC5Classic/UA Client
with max no of Tags Supported
Local Client (if available)UA/Classic Server
NP5
Resource-Efficiency TestingTest Plan Model – Server (1 of 2)
4/19/2012 © 2012 OPC Foundation12
Remote PC1Classic/UA Server with max no of Tags
Supported
• An OPC Client is connected to 5 or more OPC Servers, each approx. 1000+ items• Make 1 Server go online/offline every 2‐minutes
Remote PC2Classic/UA Server with max no of Tags
Supported
Remote PC3Classic/UA Server with max no of Tags
Supported
Remote PC4Classic/UA Server with max no of Tags
Supported
Remote PC5Classic/UA Server with max no of Tags
Supported
Local Server (if available)UA/Classic Client
NP5
@echo offSET DTIME=120CLSECHO Delay time is: %DTIME%ECHO Network Connection Enabler/Disabler started at: %date% %time%
:startECHO Disabling network adapter at %time%netsh interface set interface "Local Area Connection" DISABLED
timeout /t %DTIME%ECHO Enabling network adapter at %date% %time%netsh interface set interface "Local Area Connection" ENABLED
choice /c:yn /m "Repeat network adapter recycle (y/n)?" /t %DTIME% /d yECHO %ERRORLEVEL%IF ERRORLEVEL 2 GOTO endIF ERRORLEVEL 1 GOTO start
:endECHO.ECHO Network adapter recycle complete at %date% %time%
@echo offSET DTIME=120CLSECHO Delay time is: %DTIME%ECHO Network Connection Enabler/Disabler started at: %date% %time%
:startECHO Disabling network adapter at %time%netsh interface set interface "Local Area Connection" DISABLED
timeout /t %DTIME%ECHO Enabling network adapter at %date% %time%netsh interface set interface "Local Area Connection" ENABLED
choice /c:yn /m "Repeat network adapter recycle (y/n)?" /t %DTIME% /d yECHO %ERRORLEVEL%IF ERRORLEVEL 2 GOTO endIF ERRORLEVEL 1 GOTO start
:endECHO.ECHO Network adapter recycle complete at %date% %time%
Resource-Efficiency TestingBatch File
4/19/2012 © 2012 OPC Foundation13
• To toggle the enabled state of the “Local Area Connection” network adapter in a windows environment use the below batch file.
• Replicate the following text into file “NIC.BAT” and run it using Administrator/elevated permission.
NP6
Resource-Efficiency TestingReports
4/19/2012 © 2012 OPC Foundation14
In Certification testing, the Test Lab will generate a resource Data report that is generated from the data collected by the Windows Performance Monitor:
NP7
Resource-Efficiency TestingNext Steps…
If no growth is detected in your Classic/UA product then you are ready for Lab Certification:
• Request Certification in the Test Lab:www.opcfoundation.org/certification/schedule-testing
4/19/2012 © 2012 OPC Foundation 15
Certification: www.opcfoundation.org/certification
Lab Testing: www.opcfoundation.org/certification/schedule-testing
Bug Tracker: www.opcfoundation.org/mantis
Contact: [email protected]
Resource-Efficiency TestingThe End!