spirent testcenter results reporter · 2014-05-16 · 13 proprietary and confidential user queries...

PROPRIETARY AND CONFIDENTIAL

Results Reporter

Spirent TestCenter

2 PROPRIETARY AND CONFIDENTIAL

Agenda

Result Reporter Basics

Exporting Results

Understanding Database Tables

User Queries

Template Editor

Customizing Column Headers

Tips and Tricks


Result Reporter (RR) Basics

Java Application • Menu -> Help -> System Information

Results Location Win 7 => “C:\Users\<uname>\Documents\Spirent\TestCenter

x.xx\Results”

Templates Location Win 7 “C:\Program Files (x86)\Spirent Communications\Spirent

TestCenter x.xx\Spirent TestCenter Application\Results

Reporter\templates “

RR Logs Win 7 => “C:\Users\<uname>\Documents\Spirent\TestCenter

x.xx\ResultsReporter”

Export results • From

●RR Application ●Command Line ●API • Supported Formats

●PDF ●HTML ●CSV ●XLS


RR Basics

Filter

• Apply Filter to narrow down the results based on requirements


Export Results : RR Application

From the Menu

From the Icons

From the Graph


Export Results: Command Line

Windows: ResultsReporterCLI.bat -o MyResults.db -f pdf -d MyResults.pdf -t

templates\TemplateName.rtp

Linux: ResultsReporterCLI.sh -o MyResults.db -f pdf -d MyResults.pdf -t

templates/TemplateName.rtp

Usage The full list of options is printed on the console if you run the script with no

parameters:usage: Spirent TestCenter Results Reporter CLI

-d,--dest <file> the path to save to

-f,--format <pdf|pdf-tree|html|csv|csv-tree|jpeg> the format to save as

-h,--help print this message

-o,--open <file> open a result file at the given path

-t,--template <file> applies a template located at the given path

Example: Export RFC2544 Thtoughput summary Results to PDF

ResultsReporterCLI.bat -o 2544-Tput-Summary-2_2013-11-12_11-29-20.db -f pdf -d

MyResults.pdf -t templates\Rfc2544ThroughputStats.rtp


Export Results: API

ExportDbResultsCommand


Export Results: API

ExportDbResultsCommand

package require SpirentTestCenter

set hTempName "C:/Program Files (x86)/Spirent Communications/Spirent TestCenter 4.39/Spirent

TestCenter Application/Results Reporter/templates/Rfc2544ThroughputStats.rtp"

set hResultsdatabase "2544-Tput-Summary-5_2014-04-09_17-30-30.db"

array set hResult [stc::perform ExportDbResultsCommand -TemplateUri $hTempName -

ResultFileName "ExportFromAPI.csv" -Format CSV -ResultDbFile $hResultsdatabase ]


Export Results: API

D


Export Results: API - DbGetCommand

DbGetCommand


array set hRfcResults [stc::perform DbGet -DatabaseConnectionString 2544-Tput-Summary-

5_2014-04-09_17-30-30.db\

-AttributeList [list FrameSize,Result, ThroughputRate, MinLatency, AvgLatency, MaxLatency ] \

-Table Rfc2544ThroughputPerLoadResult]

parray hRfcResults

# Display results.

puts "\n Display Results Method 3"

puts "RFC Results"

foreach szLine [split $hRfcResults(-Output) \n] {

puts \t$szLine

}


Export Results: API – TCL Shell

From TCL Shell

tclsh

cd "C:\Program Files (x86)\Spirent Communications\Spirent TestCenter 4.39\Spirent TestCenter Application\Results Reporter"


ResultsReporterCLI.bat -o 2544-Tput-Summary-2_2013-11-12_11-29-20.db -f pdf -d ExportFromAPI.pdf -t templates/Rfc2544ThroughputStats.rtp


Understanding Database Tables

DB Explorer

Result Nodes


User Queries

Create simple queries using “Create Query”

• select FrameSize,Result, ThroughputRate, MinLatency, AvgLatency, MaxLatency from Rfc2544ThroughputPerLoadResult

• select DISTINCT FrameSize from Rfc2544ThroughputPerLoadResult

• select FrameSize,Result, ThroughputRate, MinLatency, AvgLatency, MaxLatency from Rfc2544ThroughputPerLoadResult Where Result="Passed“

• select FrameSize,Result, ThroughputRate, MinLatency, AvgLatency, MaxLatency from Rfc2544ThroughputPerLoadResult Where Result="Passed“ AND FrameSize=64

• select FrameSize,Result, ThroughputRate, MinLatency, AvgLatency, MaxLatency from Rfc2544ThroughputPerLoadResult Order By MaxLatency ASC|DESC


User Queries

How to edit the Asymmetric per side results SQL to show all iterations? Currently it only shows the final pass iteration


User Queries cont..

How to edit the Asymmetric per side results SQL to show all iterations? Currently it only shows the final pass iteration

SELECT IterationNum AS Trial, LoadIterationNum, FrameSize, Profile, IntendedLoad, ROUND(OfferedLoad,4) AS OfferedLoad, Result, TxFrameCount, RxFrameCount, SUM(TxFrameCount) AS 'TotalTxFrameCount', SUM(RxFrameCount) AS 'TotalRxFrameCount', SUM(FrameLoss) AS 'TotalFrameLoss', round((SUM(TxFrameCount)-SUM(RxFrameCount))*100.0/TxFrameCount, 5) AS 'PercentLoss' from Rfc2544AsymmetricThroughputPerLoadResult GROUP BY Profile, DataSetId ORDER BY DataSetId


Queries from the Results

Create queries from the Results


Template Editor

RFC Templates

• Use Case 1 : RFC 2544 template to show FcsErrorFrameCount and OutOfSeqCount stats in

Test Summary Result Table

Advanced Test Summary Result Table

Trial Summary Result Table


Template Editor Cont..

Use Case 1: Test Summary Result table

1. Find the table(s) associated to the test

Template Editor -> DB Explorer -> Rfc2544ThroughputPerFrameSizeResult

2. Right click on the requested result columns and Select “Add to Reported”

3. Check the added variable(s) in the “Reporter Stats”

4. Two new variables added to the bottom of the Reported Stats

5. Click on a Variable name and “Update” button -> Result will be updated

select FcsErrorFrameCount from Rfc2544ThroughputPerFrameSizeResult where DataSetId between =$@StartDataSetId$ and =$@EndDataSetId$;

select OutOfSeqCount from Rfc2544ThroughputPerFrameSizeResult where DataSetId between =$@StartDataSetId$ and =$@EndDataSetId$;


Template Editor Cont…

6. Select the “Tables” column and add the newly added Variables

7. Add column headers for the table <tr>

<td bgcolor="#eeeeee">Total\nTrials</td>

<td bgcolor="#eeeeee">Number of\nPassed Trials</td>

<td bgcolor="#eeeeee">=$FrameSizeTitle$</td>

<td bgcolor="#eeeeee">Intended\nLoad (=$DisplayLoadUnit$)</td>

<td bgcolor="#eeeeee">Offered\nLoad (=$DisplayLoadUnit$)</td>

<td bgcolor="#eeeeee">Throughput\n(%)</td>

<td bgcolor="#eeeeee">Aggregated\nThroughput (fps)</td>

<td bgcolor="#eeeeee">Aggregated\nTheoretical Max (fps)</td>

<td bgcolor="#eeeeee">Aggregated\nThroughput (Mbps)</td>

<td bgcolor="#eeeeee">Aggregated\nTheoretical Max (Mbps)</td>

<td bgcolor="#eeeeee">FCS Error\n Count</td>

<td bgcolor="#eeeeee">Out Of sequnce\n Count</td>


Template Editor Cont…

8. Add Variables </tr>

<tr iterate="=$TestSum.Fs$">

<td halign="right" bgcolor="#eeeeee">=$TestSum.TrialNum$</td>

<td halign="right" bgcolor="#eeeeee">=$TestSum.PassedTrialNum$</td>

<td halign="right" bgcolor="#eeeeee">=$TestSum.Fs$</td>

<td halign="right" bgcolor="#ffffff">=$TestSum.Intended$</td>

<td halign="right" bgcolor="#ffffff">=$TestSum.Offered$</td>

<td halign="right" bgcolor="#ffffff">=$TestSum.ThruPct$</td>

<td halign="right" bgcolor="#ffffff">=$TestSum.ThruRate$</td>

<td halign="right" bgcolor="#ffffff">=$TestSum.TheoRate$</td>

<td halign="right" bgcolor="#ffffff">=$TestSum.OfferedMbpsLoad$</td>

<td halign="right" bgcolor="#ffffff">=$TestSum.TheoMbpsRate$</td>

<td halign="right" bgcolor="#ffffff">=$TS_FcsErrorFrameCount$</td>

<td halign="right" bgcolor="#ffffff">=$TS_OutOfSeqCount$</td>

</tr>

9. Click Render and Validate the data poplated

10. Check the Template level and verify this works



Use Case 2: Advanced Test Summary Result table

1. Find the table(s) associated to the test

Template Editor -> DB Explorer -> Rfc2544ThroughputPerLoadResult

2. Repeat the Steps from 2 – 5 from use case 1

3. Select the table “AggLatencyJitter”

4. Follow the Steps 7 -10 from use case 1



Use Case 3: Trial Summary Result table

• This table is slightly different other tables

• Queries are located in Reported Stats column and Computed Stats Columns

• Select Reported Stats Column

Select the TrialSum

Copy the Query and paste in a wordPad

Rename the TrailSum Variable to TrialSumORG

Create a new Variable and name this as TrialSum

Modify the Query to add new results columns

Copy from the WordPad and paste to the new TrailSum Variable



Use Case 3: Trial Summary Result table SELECT Datasetid ||'^'|| IterationNum ||'^'|| FrameSize ||'^'|| Result||'^'|| IntendedLoad||'^'|| OfferedLoad||'^'||

ThroughputRate||'^'|| PortName||'^'|| Framerate ||'^'|| Round(TheoFramerate,3)||'^'|| MbpsLinerate ||'^'|| Round(TheoMbpsLineRate,3) ||'^'||PercentLoss||'^'||MaxLatencyThresholdExceeded||'^'||OutOfSeqThresholdExceeded||'^'||OutOfSeqCount FROM (select F.DatasetId as Datasetid, F.IterationNum IterationNum, (CASE WHEN F.FrameSizeType = 'Random' THEN 'Random' WHEN F.FrameSizeType = 'iMIX' THEN F.FrameLengthDistribution WHEN F.ConfiguredFrameSize <> 0 AND F.FrameSize <> F.ConfiguredFrameSize THEN "Configured-" || F.ConfiguredFrameSize || " Avg-" ||F.FrameSize ELSE F.FrameSize END) As FrameSize ,Result , Intendedload , Coalesce(Offeredload,'N/A') AS Offeredload ,Coalesce(Throughputrate,'N/A') Throughputrate,' ' As PortName , CASE ThroughputRate WHEN 0 THEN 0 ELSE F.FrameRate END FrameRate ,T.FrameRate TheoFramerate , CASE ThroughputRate WHEN 0 THEN 0 ELSE F.MbpsLinerate END MbpsLinerate ,T.MbpsLineRate TheoMbpsLineRate, PercentLoss , CASE MaxLatencyThresholdExceeded WHEN 0 THEN 'False' ELSE 'True' END MaxLatencyThresholdExceeded ,CASE OutOfSeqThresholdExceeded WHEN 0 THEN 'False' ELSE 'True' END OutOfSeqThresholdExceeded, F.OutOfSeqCount from Rfc2544ThroughputPerFrameSizeResult F LEFT JOIN Theoreticalmaxlinerate T ON T.DataSetId=F.DataSetId WHERE F.DataSetId BETWEEN =$@StartDataSetId$ AND =$@EndDataSetId$ UNION ALL select FP.DatasetId as DatasetId,' ' IterationNum,' ' FrameSize ,' ' Result, ' ' IntendedLoad , ' ' OfferedLoad, ' ' ThroughputRate, FP.PortName PortName, CASE WHEN FP.FrameRate IS NULL THEN 'N/A' else FP.FrameRate END Framerate,TP.FrameRate TheoFramerate, CASE WHEN FP.MbpsLinerate IS NULL THEN 'N/A' else FP.MbpsLinerate END MbpsLinerate, TP.MbpsLineRate TheoMbpsLineRate , ' ' PercentLoss,' ' MaxLatencyThresholdExceeded ,' ' OutOfSeqThresholdExceeded,' ' OutOfSeqCount from Rfc2544ThroughputPerFrameSizeResultByPort FP LEFT JOIN TheoreticalMaxLineRatePerPort TP ON TP.Datasetid=FP.Datasetid AND FP.PortName =TP.Port WHERE FP.DataSetId BETWEEN =$@StartDataSetId$ AND =$@EndDataSetId$ ORDER BY 1)



Use Case 3: Trial Summary Result table

• Select Computed Stats

• TrailSum.fs and add new columns

catch { array unset TrialSumData }

set cols [list Datasetid IterationNum Fs Result Intended Offered ThruPct PortName ThruRate TheoRate MbpsRate TheoMbpsRate Percentloss maxlatency outofsequence ooscount fcscount]



Create Table from SQL queries

• Preexisting tables

Create New Graphs

Create Templates from Scratch with new Logos


Additional use cases

We only need the 3 tables in the bottom – throughput, latency and jitter.

1. Is there any template that can show only summary? (I also tried CustomStats.rtp, but the output is the same)

2. Throughput results are displayed in L1 throughput and %. Since we need L2 throughput we would need additional calculation to get L2 throughput. However, t the software team do not want to do data manipulation. Is there a way we can get L2 throughput?

Example, 68B, throughput is 4971 fps. L2 throughput is 2.7Mbps (4971*68*8/1M)

Tech-Talk - Customize Column Headers in Results Reporter

I created a Filtered Stream Results View based upon VLAN ID. Within STC GUI, that column is correctly named with the VLAN ID.

However, when I save the results, that column is named “Comp16_4.” Does anyone know a way to customize that header to VLAN ID (either within STC App or Results Reporter)?


SQL link

http://www.w3schools.com/sql/default.asp

http://www.w3schools.com/sql/default.asp


Tips and Tricks

RFC or Iteration Save related long term Tests

• Disable the Automatic launch of Result Reporter before start the test in the STC GUI

• Just open the Summary result DB file first

• Avoid opening all the Iterations at once

Unable to Launch the RR or RR hangs

• Try change the default RR java to Local PC java

Edit the STCResultsReporter.bat in the RR install dir

From .\_jvm\bin\java.exe -mx768m -Dcom.caw.analyzer.skipNodePruneCheck=true -Djava.library.path=bin;.. -cp %CLASSPATH% com.caw.analyzer.sqlite.STCExploreFrame %*

To .\C:\Program Files (x86)\Java\jre6\binjvm\bin\java.exe -mx768m -Dcom.caw.analyzer.skipNodePruneCheck=true -Djava.library.path=bin;.. -cp %CLASSPATH% com.caw.analyzer.sqlite.STCExploreFrame %*

• Edit the rrSettings.ini and set the loadTestCenter=true to loadTestCenter=false

• Try Adding "-Dswin.noxp=true“

.\_jvm\bin\java.exe -mx768m -Dswin.noxp=true -Dcom.caw.analyzer.skipNodePruneCheck=true -Djava.library.path=bin;.. -cp %CLASSPATH% com.caw.analyzer.sqlite.STCExploreFrame %*


Thank You