![Page 1: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/1.jpg)
Comparison of Two Dimension-Reduction M th d f N t k Si l ti M d lMethods for Network Simulation Models
image generated using http://www.wordle.net/
Kevin Mills (ANTD) & Jim Filliben (SED)Complex Systems Study GroupComplex Systems Study Group
NIST SED/ANTD Seminar 9/22/11 222/A326millsjjfsedtalk092211.pptx
![Page 2: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/2.jpg)
Comparison of Two Dimension-Reduction M th d f N t k Si l ti M d lMethods for Network Simulation Models
image generated using http://www.wordle.net/
Kevin Mills (ANTD) & Jim Filliben (SED)Complex Systems Study GroupComplex Systems Study Group
NIST SED/ANTD Seminar 9/22/11 222/A326millsjjfsedtalk092211.pptx
![Page 3: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/3.jpg)
Motivation for this Talk
1. Generate useful internet MesoNet modelingconclusions & insightconclusions & insight
2. Show stat framework/approach &methodology + beginning-to-”end” demo
3. Show dimension reduction dependency onp yDesign of Experiment & Sensitivity Analysis
3
![Page 4: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/4.jpg)
Outline• CxS: Complex System IMS Project• Goal – Problem – Solution
St t F k• Stat Framework• Overview of Candidate MesoNet Factors & Responses• Experiment Designg• Sensitivity Analysis• Dimension Reduction
via Correlation Analysis with Clusteringvia Correlation Analysis with Clusteringvia Principal Components Analysis
• Comparison of Dimension Reduction TechniquesC l i• Conclusions
4
![Page 5: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/5.jpg)
IMS Project: Measurement Science for Complex Information SystemInformation System
http://www.nist.gov/itl/antd/emergent_behavior.cfm
This project aims to develop and evaluate a coherent set of methods to understand behavior in complex information systems such as theto understand behavior in complex information systems, such as the Internet, computational grids and computing clouds.
Such large distributed systems exhibit global behavior arising from independent decisions made by many simultaneous actors, which adapt their behavior based on local measurements of system state.
Actor adaptations shift the global system state influencingActor adaptations shift the global system state, influencing subsequent measurements, leading to further adaptations.
This continuous cycle of measurement and adaptation drives a time-varying global behavior.
For this reason, proposed changes in actor decision algorithms must be examined/understood at large spatiotemporal scale in order to
5
be examined/understood at large spatiotemporal scale in order to predict ( and control) system behavior.
![Page 6: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/6.jpg)
CxS Project
What is the problem? No one understands how to measure, predict or control macroscopic behavior in complex information systems: (1) threatening our nation’s security and (2) costing billions of dollars.
“[Despite] society’s profound dependence on networks, fundamental knowledge about them is primitive. [G]lobalcommunication … networks have quite advanced qtechnological implementations but their behavior under stress still cannot be predicted reliably.… There is no science today that offers the fundamental knowledgescience today that offers the fundamental knowledge necessary to design large complex networks [so] that their behaviors can be predicted prior to building them.” (above quote from Network Science 2006, a National
6
(above quote from Network Science 2006, a National Research Council report)
![Page 7: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/7.jpg)
Project Goal – Problem – Solution• Goal – understand internet congestion and compare proposed
Internet congestion control algorithms under a wide range of controlled, repeatable conditions, as simulated by selecting , p , y gcombinations of parameter values for MesoNet, a 11- to 20-parameter network simulator.
• Problem – how to determine which MesoNet core responses to analyze when characterizing model behavior.
• Solution – apply experiment design techniques to generate an affordable but representative data sample, and carry out the subsequent response variable evaluation via three data analysis approaches:approaches:
1. sensitivity analysis 2. correlation analysis with clustering & 3 principal components analysis3. principal components analysis
7
![Page 8: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/8.jpg)
Abilene Network (3-Tier MesoNet Topology)
8
![Page 9: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/9.jpg)
General Problem-Solving Framework
Expert
12
5Problem Solution2
34
DataDEX = g(k,n) 1. Graphical
2. Quantitative
![Page 10: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/10.jpg)
General Problem-Solving Framework
Expert
P bl
12
5Problem Solution2
34
1 Ch t i i 1 # Di t ib tiData
1-FAT
Graphical
Quantitative
1. Characterizing
2. Sensitivity
3. Optimizing
4 Modeling
1. #, Distribution
2. List: Ranked Factors
3. Vector: (x1,…,xk)
4 fR lit
DEX = g(k,n)
Monte Carlo
Latin HC
Orthogonal
4. Modeling
5. Comparing
6. Predicting
7 Uncertainty
4. f
5. Y/N
6 #
7 SD(#)
Reality
Lab
Computational
Orthogonal
RespSurface
7. Uncertainty
8. Verifying
9. Validating
7. SD(#)
8. Y/N, Vector: (x1, …,xk)
9. Y/N, Vector: (x1,
![Page 11: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/11.jpg)
General Problem-Solving Framework
Expert
12
5Problem Solution2
34
DataDEX = g(k,n) 1. Graphical
2. Quantitative
Q1. Response Dimension?
Q2. Important Factors?
Q3 Best Factor Settings?
A1. # & Set {...}
A2. List (Ranked)
A3 Vector (x1 x2 xk)Q3. Best Factor Settings?
Q4. Improvement over TCP?A3. Vector (x1,x2,...,xk)
A4. Y/N & Best/Worst
![Page 12: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/12.jpg)
The Starting Point: Generic ModelSystem Behavior Y = f(X1, X2, ..., Xk)
1. Y = f(X1, X2, ..., Xk) Comparative
2. Y = f(X1, X2, ..., Xk) Screening
3. Y = f(X1, X2, ..., Xk) Regression
4. Y = f(X1, X2, ..., Xk) Optimization
5. Y = f(X1, X2, ..., Xk) = c Consensus
6 Y = f(X1 X2 Xk) Dimension Red12
6. Y = f(X1, X2, ..., Xk) Dimension Red.
![Page 13: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/13.jpg)
The Starting Point: Generic ModelSystem Behavior Y = f(X1, X2, ..., Xk)
1. Y = f(X1, X2, ..., Xk) Comparative
2. Y = f(X1, X2, ..., Xk) Screening
3. Y = f(X1, X2, ..., Xk) Regression
4. Y = f(X1, X2, ..., Xk) Optimization
5. Y = f(X1, X2, ..., Xk) = c Consensus
6 Y = f(X1 X2 Xk) Dimension Red13
6. Y = f(X1, X2, ..., Xk) Dimension Red.
![Page 14: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/14.jpg)
The Starting Point: Generic Model (Part 2)
System Behavior Y = f(X1, X2, ..., Xk)
System Behavior Yi = fi(X1, X2, ..., Xk) (i = 1, 2,..., m)
14Unknowns: (k=?,n=?,m=?)
![Page 15: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/15.jpg)
Factor Groups Affecting System Behavior
1 Network Factors1. Network Factors2. User Factors3. Source & Receiver Factors4. Protocol Factors
![Page 16: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/16.jpg)
Factors Xi Affecting System BehaviorYi = fi(X1 X2 Xk)
Networkx1 Propagation delayx2 Network speed
Yi fi(X1, X2, ..., Xk)
Factors x2 Network speedx3 Buffer sizing
Userx4 Average file size for web pagesx5 Average think time between web clicks
Factors x5 Average think time between web clicksx6 Probability a user opts to transfer a larger file
x7 Probability a source or receiver is on a fasthost
Source & Receiver Factors
host
x8 Scaling factor for number of sources &receivers
x9 Distribution of sourcesx10 Distribution of receivers
ProtocolFactors x11 Initial TCP slow-start threshold
(k=11,n=?,m=?)
![Page 17: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/17.jpg)
n <= 100Affordable Number of Runs n = ?
n <= 100
4 Ways to Reduce DEX Full Factorial Design n:1. Reduce # Factors (but scope reduced)2 R d N b f L l ( 2?)2. Reduce Number of Levels (=> 2?)3. Reduce Number of Reps4 Fractional Factorial Design4. Fractional Factorial Design
(k=11,n <= 100,m=?)
![Page 18: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/18.jpg)
n <= 100Affordable Number of Runs n = ?
n <= 100
4 Ways to Reduce DEX Full Factorial Design n:1. Reduce # Factors (but scope reduced)2 R d N b f L l (2?)2. Reduce Number of Levels (2?)3. Reduce Number or Reps4 Fractional Factorial Design4. Fractional Factorial Design
(k=11,n <= 100,m=?)
![Page 19: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/19.jpg)
Affordable Number of Runs n = ?
Additional Desirable Feature of the Design:Good Estimates for (at least) theGood Estimates for (at least) theMain Effects & 2-Term Interactions(Resolution)(Resolution)11 + 11-choose-2 = 11 + 55 = 66(66+1) = 67 64 26 211-5
Fi l D i 211 5 O th l 2 L l F ti lFinal Design: 211-5 Orthogonal 2-Level Fractional Factorial Design (k=11,n=64)
(k=11,n=64,m=?)
![Page 20: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/20.jpg)
MesoNet Factors (k=11) & Levels (2)Category Factor Code Definition Level 1: - Level 2: +
x1 PDM Propagation delay 1 2
NetworkFactors
x1 PDM Propagation delay 1 2x2 BRS (s) Network speed 800 p/ms 400 p/ms
x3 QSA Buffer sizing RTTxC/SQRT(n) RTTxC
UserFactors
x4 AvFSWO Average file size forweb pages 50 packets 100 packets
x5 AvThT Average think timebetween web clicks 2000 ms 5000 ms
Probabilit a ser optsx6 PrLF Probability a user optsto transfer a larger file 0.02 0.01
x7 PrFHProbability a sourceor receiver is on a fasthost
0.4 0.2
Source &
Receiver Factors
x8 SFSRScaling factor fornumber of sources &receivers
2 3
x9 SDist Distribution ofsources WEB P2Psources
x10 RDist Distribution ofreceivers WEB P2P
ProtocolFactors x11 SST Initial TCP slow-start
threshold 43 packets 1.07x109
packetsFactors threshold packets
![Page 21: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/21.jpg)
211-5 Orthogonal Fractional Factorial Design(k = 11 n = 64)(k 11, n 64)
Generators:Generators:X7 = X3*X4*X5X8 = X1*X2*X3*X4X9 = X1*X2*X6X9 X1 X2 X6X10 = X2*X4*X5*X6X11 = X1*X4*X5*X6
Resolution IV
Reference: Box, Hunter, & Hunter, “Statistics forExperimenters”, 2nd Edition, 2005, Wiley, p. 272
21
![Page 22: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/22.jpg)
211-5 Fractional Factorial Design (k=11,n=64) (2to11m5.xls)
Inde X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11Index X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X111 -1 -1 -1 -1 -1 -1 -1 +1 -1 +1 +12 +1 -1 -1 -1 -1 -1 -1 -1 +1 +1 -13 -1 +1 -1 -1 -1 -1 -1 -1 +1 -1 +14 +1 +1 -1 -1 -1 -1 -1 +1 -1 -1 -15 -1 -1 +1 -1 -1 -1 1 -1 -1 +1 +16 +1 -1 +1 -1 -1 -1 1 +1 +1 +1 -17 -1 +1 +1 -1 -1 -1 1 +1 +1 -1 +18 +1 +1 +1 -1 -1 -1 1 -1 -1 -1 -19 -1 -1 -1 +1 -1 -1 1 -1 -1 -1 -1
10 +1 -1 -1 +1 -1 -1 1 +1 +1 -1 +111 -1 +1 -1 +1 -1 -1 1 +1 +1 +1 -111 -1 +1 -1 +1 -1 -1 1 +1 +1 +1 -112 +1 +1 -1 +1 -1 -1 1 -1 -1 +1 +113 -1 -1 +1 +1 -1 -1 -1 +1 -1 -1 -114 +1 -1 +1 +1 -1 -1 -1 -1 +1 -1 +115 -1 +1 +1 +1 -1 -1 -1 -1 +1 +1 -116 +1 +1 +1 +1 -1 -1 -1 +1 -1 +1 +117 -1 -1 -1 -1 +1 -1 1 +1 -1 -1 -118 +1 -1 -1 -1 +1 -1 1 -1 +1 -1 +119 -1 +1 -1 -1 +1 -1 1 -1 +1 +1 -120 +1 +1 -1 -1 +1 -1 1 +1 -1 +1 +121 -1 -1 +1 -1 +1 -1 -1 -1 -1 -1 -122 +1 -1 +1 -1 +1 -1 -1 +1 +1 -1 +123 -1 +1 +1 -1 +1 -1 -1 +1 +1 +1 -124 +1 +1 +1 -1 +1 -1 -1 -1 -1 +1 +125 -1 -1 -1 +1 +1 -1 -1 -1 -1 +1 +126 +1 -1 -1 +1 +1 -1 -1 +1 +1 +1 -127 -1 +1 -1 +1 +1 -1 -1 +1 +1 -1 +128 +1 +1 1 +1 +1 1 1 1 1 1 1
22
28 +1 +1 -1 +1 +1 -1 -1 -1 -1 -1 -129 -1 -1 +1 +1 +1 -1 1 +1 -1 +1 +130 +1 -1 +1 +1 +1 -1 1 -1 +1 +1 -131 -1 +1 +1 +1 +1 -1 1 -1 +1 -1 +132 +1 +1 +1 +1 +1 -1 1 +1 -1 -1 -1
![Page 23: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/23.jpg)
33 -1 -1 -1 -1 -1 +1 -1 +1 +1 -1 -134 +1 -1 -1 -1 -1 +1 -1 -1 -1 -1 +135 -1 +1 -1 -1 -1 +1 -1 -1 -1 +1 -136 +1 +1 1 1 1 +1 1 +1 +1 +1 +136 +1 +1 -1 -1 -1 +1 -1 +1 +1 +1 +137 -1 -1 +1 -1 -1 +1 1 -1 +1 -1 -138 +1 -1 +1 -1 -1 +1 1 +1 -1 -1 +139 -1 +1 +1 -1 -1 +1 1 +1 -1 +1 -140 +1 +1 +1 -1 -1 +1 1 -1 +1 +1 +141 -1 -1 -1 +1 -1 +1 1 -1 +1 +1 +141 1 1 1 +1 1 +1 1 1 +1 +1 +142 +1 -1 -1 +1 -1 +1 1 +1 -1 +1 -143 -1 +1 -1 +1 -1 +1 1 +1 -1 -1 +144 +1 +1 -1 +1 -1 +1 1 -1 +1 -1 -145 -1 -1 +1 +1 -1 +1 -1 +1 +1 +1 +146 +1 -1 +1 +1 -1 +1 -1 -1 -1 +1 -147 -1 +1 +1 +1 -1 +1 -1 -1 -1 -1 +148 +1 +1 +1 +1 -1 +1 -1 +1 +1 -1 -149 -1 -1 -1 -1 +1 +1 1 +1 +1 +1 +150 +1 -1 -1 -1 +1 +1 1 -1 -1 +1 -151 -1 +1 -1 -1 +1 +1 1 -1 -1 -1 +152 +1 +1 -1 -1 +1 +1 1 +1 +1 -1 -153 -1 -1 +1 -1 +1 +1 -1 -1 +1 +1 +154 +1 -1 +1 -1 +1 +1 -1 +1 -1 +1 -155 -1 +1 +1 -1 +1 +1 -1 +1 -1 -1 +156 +1 +1 +1 -1 +1 +1 -1 -1 +1 -1 -157 -1 -1 -1 +1 +1 +1 -1 -1 +1 -1 -158 +1 -1 -1 +1 +1 +1 -1 +1 -1 -1 +159 -1 +1 -1 +1 +1 +1 -1 +1 -1 +1 -160 +1 +1 -1 +1 +1 +1 -1 -1 +1 +1 +161 -1 -1 +1 +1 +1 +1 1 +1 +1 -1 -162 1 1 1 1 1 1 1 1 1 1 1
23
62 +1 -1 +1 +1 +1 +1 1 -1 -1 -1 +163 -1 +1 +1 +1 +1 +1 1 -1 -1 +1 -164 +1 +1 +1 +1 +1 +1 1 +1 +1 +1 +1
345 1234 126 2456 1456
![Page 24: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/24.jpg)
What does this design look like? Why use it? (k=11,n=64) (k=7,n=8)
(k=7,n=8) 27-4Orthogonal Fractional Factorial Design
X5
+
X5
+
X4
+
X1
X2
X3
+
+
__
_
_
+ X4
+
X1
X2
X3
+
+
__
_
_
+
+ +
X5
+
X1
X2
X3
+
+
__
_
X5
+
X1
X2
X3
+
+
__
_
X4_ + X4
_ +
![Page 25: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/25.jpg)
What does this design not look like?
(k=7,n=8) 1FAT Fractional Factorial Design
X5
+
X5
+
X4
+
X1
X2
X3
+
+
__
_
_
+ X4
+
X1
X2
X3
+
+
__
_
_
+
+ +
X5
+
X1
X2
X3
+
+
__
_
X5
+
X1
X2
X3
+
+
__
_
X4_ + X4
_ +
![Page 26: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/26.jpg)
Measures of System Behavior(Response Variables)(Response Variables)Yi = fi(X1, X2, ..., Xk)
1. Characterizing Macroscopic Behavior
2. Characterizing Instantaneous Throughput for Active Flows by Flow Class (User)
(k=11,n=64,m=?)
![Page 27: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/27.jpg)
16 Responses Characterizing Macroscopic Behavior
Response Definitiony1 Active Flows – flows attempting to transfer datay gy2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement intervaly5 Loss Rate: y4/(y3+y4)y y (y y )y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement intervaly9 Connection-Failure Rate: y8/(y8+y1)y9 y (y y )y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly13 Negative Acknowledgments per Flow per measurement intervaly13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)
27
![Page 28: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/28.jpg)
6 Responses Characterizing Instantaneous Throughput for Active Flows by Flow Classfor Active Flows by Flow Class
Response Definition (Throughput in packets/second)y17 Average Throughput for Active DD Flows
y18 Average Throughput for Active DF Flows
y19 Average Throughput for Active DN Flows
y20 Average Throughput for Active FF Flows
y21 Average Throughput for Active FN Flows
y22 Average Throughput for Active NN Flowsy g g p
Router Type Speed
Backbone 2sBackbone 2s
PoP 25 % of s
D-class Access 25 % of s
F-class Access 5 % of s
28
N-class Access 2.5 % of s
![Page 29: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/29.jpg)
MesoNet 22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows flows attempting to transfer datay1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement interval5 L R t 4/( 3+ 4)y5 Loss Rate: y4/(y3+y4)
y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement interval9 C ti F il R t 8/( 8 1)y9 Connection-Failure Rate: y8/(y8+y1)
y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
29
y20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 30: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/30.jpg)
General Problem-Solving Framework
Expert
12
5Problem Solution2
34
DataDEX =
g(k=11,n=64)1. Graphical
2. Quantitative
Q1. Response Dimension?
Q2. Important Factors?
Q3 Best Factor Settings?
A1. # & Set {...}
A2. List (Ranked)
A3 Vector (x1 x2 xk)Q3. Best Factor Settings?
Q4. Improvement over TCP?A3. Vector (x1,x2,...,xk)
A4. Y/N & Best/Worst
![Page 31: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/31.jpg)
Data: 64 x 22 Multivariate Data Set Resulting from a211-5 Orthogonal Fractional Factorial Experiment Design211-5 Orthogonal Fractional Factorial Experiment Design
Run y1 y2 … y21 y22
1 4680.619 0.168126 … 92.034 89.785
2 6654.512 0.239371 … 72.596 57.738
3 9431 405 0 339259 … 29 569 13 9633 9431.405 0.339259 29.569 13.963
4 11565.81 0.415439 … 23.427 19.882… … … … … …
61 10319.55 0.247471 … 87.969 41.573
62 1738.469 0.093668 … 159.298 161.602
63 1783.509 0.096094 … 148.395 161.36
64 21467.6 0.514811 … 26.159 9.981
31
![Page 32: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/32.jpg)
General Problem-Solving Framework
Expert
12
5Problem Solution2
34
DataDEX =
g(k=11,n=64)1. Graphical
2. Quantitative
Q1. Response Dimension?
Q2. Important Factors?
Q3 Best Factor Settings?
A1. # & Set {...}
A2. List (Ranked)
A3 Vector (x1 x2 xk)Q3. Best Factor Settings?
Q4. Improvement over TCP?A3. Vector (x1,x2,...,xk)
A4. Y/N & Best/Worst
![Page 33: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/33.jpg)
Sensitivity Analysis
33
![Page 34: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/34.jpg)
Sensitivity Analysis
Q1. Of the 11 factors, what are most/leastimportant (including interactions)?
Q2 Robust over the 22 responses?Q2. Robust over the 22 responses?
34
![Page 35: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/35.jpg)
Analysis: For each of the 22 responses ...Response Definition
Example 1: Y10 = Retransmission Ratey1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement intervaly5 Loss Rate: y4/(y3+y4)y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement intervaly py9 Connection-Failure Rate: y8/(y8+y1)y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly p py13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)y16 Relative queuing delay: y15/(x1x41)
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
35
y19 Average Throughput for Active DN Flowsy20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 36: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/36.jpg)
Y10: Retransmission RateMain Effects Plot (Augmented)
![Page 37: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/37.jpg)
Y10: Retransmission RateMain Effects Plot (Augmented)
![Page 38: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/38.jpg)
Y10: Retransmission RateMain Effects Plot (Augmented)
Means: (+ - + - + + + - - + -)
![Page 39: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/39.jpg)
Y10: Retransmission RateInteraction Effects Matrix
![Page 40: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/40.jpg)
Y10: Retransmission RateInteraction Effects Matrix
![Page 41: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/41.jpg)
Y10: Retransmission RateInteraction Effects Matrix
http://www.itl.nist.gov/div898/handbook/pri/section5/pri59.htm
![Page 42: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/42.jpg)
Y10: Retransmission RateOrdered Data Plot
![Page 43: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/43.jpg)
Y10: Retransmission RateOrdered Data Plot
++ ‐ +x1 11 9x2 4 16x3 16 4x4 7 13
‐ +7 13 x115 5 x24 16 x316 4 x4 x4 7 13
x5 14 6x6 10 10x7 11 9x8 8 12
6 14 x510 10 x69 11 x713 7 x814 6 9 x8 8 12
x9 5 15x10 11 9x11 10 10
14 6 x910 10 x1010 10 x11
![Page 44: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/44.jpg)
Y10: Retransmission RateOrdered Data Plot
++ ‐ +x1 11 9x2 4 16x3 16 4x4 7 13
‐ +7 13 x115 5 x24 16 x316 4 x4
Left:(+ + + + )
x4 7 13x5 14 6x6 10 10x7 11 9x8 8 12
6 14 x510 10 x69 11 x713 7 x814 6 9
Means:(+ - + - + + + - - + -)
Left:(+ - + - + . + - - . .) Right:(+ - + - + . + - - + .)
x8 8 12x9 5 15x10 11 9x11 10 10
14 6 x910 10 x1010 10 x11
![Page 45: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/45.jpg)
Response Definition
Example 2: Y17 = Ave. TP for Active DD Flows
y1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement intervaly5 Loss Rate: y4/(y3+y4)y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement intervaly py9 Connection-Failure Rate: y8/(y8+y1)y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly p py13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)y16 Relative queuing delay: y15/(x1x41)
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
45
y19 Average Throughput for Active DN Flowsy20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 46: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/46.jpg)
Y17: Average Throughput for Active DD Flows Main Effects Plot (Augmented)
![Page 47: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/47.jpg)
Main Effects Plot (Augmented)Y17: Average Throughput for Active DD Flows
![Page 48: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/48.jpg)
Main Effects Plot (Augmented)Y17: Average Throughput for Active DD Flows
Means: (- - + + - - - + . + +)
![Page 49: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/49.jpg)
Y17: Average Throughput for Active DD Flows Interaction Effects Matrix
![Page 50: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/50.jpg)
Y17: Average Throughput for Active DD Flows Interaction Effects Matrix
![Page 51: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/51.jpg)
Y17: Average Throughput for Active DD Flows Interaction Effects Matrix
![Page 52: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/52.jpg)
Ordered Data PlotY17: Average Throughput for Active DD Flows
![Page 53: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/53.jpg)
Ordered Data PlotY17: Average Throughput for Active DD Flows
+ +‐ +0 20 x110 10 x211 9 x316 4 x4
‐ +x1 19 1x2 10 10x3 9 11x4 3 1716 4 x4
11 9 x59 11 x610 10 x711 9 x8
x4 3 17x5 11 9x6 12 8x7 11 9x8 9 1111 9 x8
11 9 x910 10 x1012 8 x11
x8 9 11x9 11 9x10 8 12x11 9 11
![Page 54: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/54.jpg)
Ordered Data PlotY17: Average Throughput for Active DD Flows
+ +‐ +0 20 x110 10 x211 9 x316 4 x4
‐ +x1 19 1x2 10 10x3 9 11x4 3 1716 4 x4
11 9 x59 11 x610 10 x711 9 x8
x4 3 17x5 11 9x6 12 8x7 11 9x8 9 11Left:( + +)11 9 x8
11 9 x910 10 x1012 8 x11
x8 9 11x9 11 9x10 8 12x11 9 11Means:(- - + + - - - + . + +)
Left:(- . . + . . . . . . +) Right:(- . . + . - . . . + .)
![Page 55: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/55.jpg)
Robustness Assessment: Stacked Main Effects Plot
55
![Page 56: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/56.jpg)
X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11
Robustness Assessment: (1-Way) ANOVA CDF Values (unordered)
X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11PDM BRS QSA AvFSW AVThT PrLF PrFH SFSR SDist RDist SST
Y1 51.16 98.66 37.27 99.83 100 20.6 13.47 99.93 99.98 15.78 14.39Y2 62.1 99.84 35.47 99.99 100 31.51 18.87 91.42 96.85 22.06 14.69Y3 42.48 100 15.96 99.97 100 28.33 7.84 97.49 87.44 33.63 38.25Y4 23.87 100 30.78 99.88 99.99 27.37 9.48 94.56 74.98 42.21 42.83Y5 86.91 99.99 99.98 96.28 97.66 13.56 23.51 94.83 98.87 54.01 18.52Y6 14.99 100 47.49 99.99 99.99 29.52 19.27 92.44 71.04 36.43 36.74Y7 84.55 99.99 41.43 100 99.82 24.79 16.99 94.31 99.37 27.9 57.22Y8Y8 83.44 98.98 99.06 70.34 95.79 45.54 18.13 95.79 99.25 44.83 42.18Y9 91.84 99.57 99.89 49.3 95.69 20.05 13.19 88.83 99.21 62.88 45.21Y10 86.67 99.97 99.97 95.67 95.87 21.7 29.64 94.35 99.21 48.61 26.76Y11 22.45 99.94 99.09 17.5 98.91 45.27 48.81 80.41 98.37 62.46 98.93Y12 87 12 99 99 71 44 96 85 99 91 3 49 23 44 87 87 99 4 38 95 98 02Y12 87.12 99.99 71.44 96.85 99.91 3.49 23.44 87.87 99.4 38.95 98.02Y13 99.47 96.76 100 93.93 95.28 31.3 42.08 55.53 83.6 22.11 43.18Y14 99.68 99.32 100 70.85 95.1 2.42 44.48 81.68 95.31 30.49 2.75Y15 100 88.52 100 83.64 76 18.17 8.34 71.77 69.49 5.28 8.59
Y16 81.89 91.56 100 87.83 82.66 22.07 4.41 76.31 79.34 13.34 0.82Y17 100 16.78 3.28 100 21.28 27.89 24.66 34.01 2.89 16.24 27.46Y18 100 99.09 67.06 99.45 94.98 47.51 11.33 84.16 99.41 42.36 62.51Y19 95.05 100 70.38 43.16 99.94 10.71 30.51 95.02 99.94 66.59 53.33Y20 99.98 99.71 70.05 95.48 98.11 33.15 0.96 85.65 99.85 47.06 73.11Y21 93 100 73 21 59 53 99 98 17 79 32 17 97 03 98 34 34 56 61 62
56
Y21 93 100 73.21 59.53 99.98 17.79 32.17 97.03 98.34 34.56 61.62
Y22 83.79 100 69.13 69.1 99.96 12.49 30.32 95.01 99.95 59.86 63.94
Sum 7 19 9 13 18 0 0 11 15 0 2
![Page 57: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/57.jpg)
X2 X5 X9 X4 X8 X3 X1 X11 X10 X7 X6
Robustness Assessment: (1-Way) ANOVA CDF Values (ordered)
X2 X5 X9 X4 X8 X3 X1 X11 X10 X7 X6BRS AVThT SDist AvFSW SFSR QSA PDM SST RDist PrFH PrLF
Y1 98.66 100 99.98 99.83 99.93 37.27 51.16 14.39 15.78 13.47 20.6Y2 99.84 100 96.85 99.99 91.42 35.47 62.1 14.69 22.06 18.87 31.51Y3 100 100 87.44 99.97 97.49 15.96 42.48 38.25 33.63 7.84 28.33Y4 100 99.99 74.98 99.88 94.56 30.78 23.87 42.83 42.21 9.48 27.37Y5 99.99 97.66 98.87 96.28 94.83 99.98 86.91 18.52 54.01 23.51 13.56Y6 100 99.99 71.04 99.99 92.44 47.49 14.99 36.74 36.43 19.27 29.52Y7 99.99 99.82 99.37 100 94.31 41.43 84.55 57.22 27.9 16.99 24.79Y8Y8 98.98 95.79 99.25 70.34 95.79 99.06 83.44 42.18 44.83 18.13 45.54Y9 99.57 95.69 99.21 49.3 88.83 99.89 91.84 45.21 62.88 13.19 20.05Y10 99.97 95.87 99.21 95.67 94.35 99.97 86.67 26.76 48.61 29.64 21.7Y11 99.94 98.91 98.37 17.5 80.41 99.09 22.45 98.93 62.46 48.81 45.27Y12 99 99 99 91 99 4 96 85 87 87 71 44 87 12 98 02 38 95 23 44 3 49Y12 99.99 99.91 99.4 96.85 87.87 71.44 87.12 98.02 38.95 23.44 3.49Y13 96.76 95.28 83.6 93.93 55.53 100 99.47 43.18 22.11 42.08 31.3Y14 99.32 95.1 95.31 70.85 81.68 100 99.68 2.75 30.49 44.48 2.42Y15 88.52 76 69.49 83.64 71.77 100 100 8.59 5.28 8.34 18.17
Y16 91.56 82.66 79.34 87.83 76.31 100 81.89 0.82 13.34 4.41 22.07Y17 16.78 21.28 2.89 100 34.01 3.28 100 27.46 16.24 24.66 27.89Y18 99.09 94.98 99.41 99.45 84.16 67.06 100 62.51 42.36 11.33 47.51Y19 100 99.94 99.94 43.16 95.02 70.38 95.05 53.33 66.59 30.51 10.71Y20 99.71 98.11 99.85 95.48 85.65 70.05 99.98 73.11 47.06 0.96 33.15Y21 100 99 98 98 34 59 53 97 03 73 21 93 61 62 34 56 32 17 17 79
57
Y21 100 99.98 98.34 59.53 97.03 73.21 93 61.62 34.56 32.17 17.79
Y22 100 99.96 99.95 69.1 95.01 69.13 83.79 63.94 59.86 30.32 12.49
Sum 19 18 15 13 11 9 7 2 0 0 0
![Page 58: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/58.jpg)
Robustness Assessment: Multiplot of (1-Way) ANOVA CDF Values
58
![Page 59: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/59.jpg)
Robustness Assessment: Multiplot of (1-Way) ANOVA CDF Values
59
![Page 60: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/60.jpg)
Robustness Assessment: Multiplot of (1-Way) ANOVA CDF Values
1
60
![Page 61: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/61.jpg)
Robustness Assessment: Multiplot of (1-Way) ANOVA CDF Values
1
22
61
![Page 62: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/62.jpg)
Robustness Assessment: Multiplot of (1-Way) ANOVA CDF Values
1
22
362
3
![Page 63: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/63.jpg)
Robust Sensitivity Analysis Ranking (Criterion 1)
Major Factors (ordered) influencing MesoNet behavior:X2: Network SpeedX5: Think TimeX5: Think TimeX9: Distribution of SourcesX4: File SizeX8: Number of Sources
Minor Factor influencing MesoNet behavior:X3: Buffer Size – small buffer sizes reduces delay variability &
large buffer size has greater effect underg ghigh network speed
X1: Propagation Delay
Non FactorsNon-FactorsX11: Initial TCP Slow-Start ThresholdX10: Distribution of ReceiversX7: Probability a Source or Receiver is on a Fast Host
63
X6: Probability a User Opts to Transfer a Larger File
![Page 64: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/64.jpg)
Robust Sensitivity Analysis Ranking (Criterion 2)
Major Factors (ordered) influencing MesoNet behavior:X2: Network SpeedX4: File SizeX4: File SizeX5: Think TimeX8: Number of SourcesX1: Propagation DelayX9 Distrib tion of So rcesX9: Distribution of Sources
Minor Factor influencing MesoNet behavior:X3: Buffer Size – small buffer sizes reduces delay variability &y y
large buffer size has greater effect underhigh network speed
Non-FactorsX11: Initial TCP Slow Start ThresholdX11: Initial TCP Slow-Start ThresholdX10: Distribution of ReceiversX7: Probability a Source or Receiver is on a Fast HostX6: Probability a User Opts to Transfer a Larger File
64
![Page 65: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/65.jpg)
D fi iti L l 1 L l 2
Robust Sensitivity Analysis Ranking (Criterion 2)
Category Factor Code Definition Level 1: - Level 2: +
NetworkFactors
x1 PDM Propagation delay 1 2x2 BRS (s) Network speed 800 p/ms 400 p/ms
3 QSA B ff i i RTT C/SQRT( ) RTT Cx3 QSA Buffer sizing RTTxC/SQRT(n) RTTxC
User
x4 AvFSWO Average file size forweb pages 50 packets 100 packets
x5 AvThT Average think time 2000 ms 5000 msFactors x5 AvThT gbetween web clicks 2000 ms 5000 ms
x6 PrLF Probability a user optsto transfer a larger file 0.02 0.01
x7 PrFHProbability a sourceor receiver is on a fast 0 4 0 2
Source &
Receiver
x7 PrFH or receiver is on a fasthost
0.4 0.2
x8 SFSRScaling factor fornumber of sources &receivers
2 3
Factors x9 SDist Distribution ofsources WEB P2P
x10 RDist Distribution ofreceivers WEB P2P
ProtocolFactors x11 SST Initial TCP slow-start
threshold 43 packets 1.07x109
packets
![Page 66: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/66.jpg)
Di i R d tiDimension ReductionAnalysisy
66
![Page 67: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/67.jpg)
We Applied Two Different Techniques
Principal Components
A l i
Principal Components
A l i
Principal Components
A l i
m responses (y1, … ym)
Analysis
m – d1 responsesDomainExpertise
m responses (y1, … ym)
Analysis
m – d1 responsesDomainExpertise
m responses (y1, … ym)
Analysis
m – d1 responsesDomainExpertise
Correlation
m – d2 responses Expertise
m – d3 responses
Correlation
m – d2 responses Expertise
m – d3 responses
Correlation
m – d2 responses Expertise
m – d3 responsesAnalysis responses
SCIENTIFICDATA
Analysis responsesAnalysis responses
SCIENTIFICSCIENTIFICDATADATA
& Clustering
SCIENTIFIC DOMAIN EXPERTISE
ANALYSISEXPERTISE
SCIENTIFIC DOMAIN EXPERTISE
SCIENTIFIC DOMAIN EXPERTISE
ANALYSISEXPERTISEANALYSISEXPERTISE
67
![Page 68: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/68.jpg)
Abilene Network (3-Tier MesoNet Topology)
68
![Page 69: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/69.jpg)
22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement intervaly5 Loss Rate: y4/(y3+y4)y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement intervaly py9 Connection-Failure Rate: y8/(y8+y1)y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly p py13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)y16 Relative queuing delay: y15/(x1x41)
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
69
y19 Average Throughput for Active DN Flowsy20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 70: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/70.jpg)
Data: 64 x 22 Multivariate Data Set Resulting from a211-5 Orthogonal Fractional Factorial Experiment Design211-5 Orthogonal Fractional Factorial Experiment Design
Run y1 y2 … y21 y22
1 4680.619 0.168126 … 92.034 89.785
2 6654.512 0.239371 … 72.596 57.738
3 9431 405 0 339259 … 29 569 13 9633 9431.405 0.339259 29.569 13.963
4 11565.81 0.415439 … 23.427 19.882… … … … … …
61 10319.55 0.247471 … 87.969 41.573
62 1738.469 0.093668 … 159.298 161.602
63 1783.509 0.096094 … 148.395 161.36
64 21467.6 0.514811 … 26.159 9.981
70
![Page 71: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/71.jpg)
Method 1: CorrelationAnalysis & Clustering
71
![Page 72: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/72.jpg)
Matrix of Pair-wise Scatter Plots & Correlation Coefficients
72Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
![Page 73: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/73.jpg)
Sorted Correlations
73
![Page 74: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/74.jpg)
Sorted Correlations
74
![Page 75: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/75.jpg)
Matrix of Pair-wise Scatter Plots & Correlation Coefficients (Ordered)
75Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
![Page 76: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/76.jpg)
R d 80 | | 100 100 Bl 30 | | 100 80 G | | 100 30Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
(a) Pair-wise Correlation Matrix (b) Histogram: bins where |r| > 0.65 highlighted in red
76
![Page 77: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/77.jpg)
Frequency Distribution of Absolute Value of Correlation Coefficients for All Response Pairs
Select a threshold for |r| such that correlations above that threshold will be further considered
We chose |r| > 0.65
77
![Page 78: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/78.jpg)
Response Index-Index Plot where |ri,j| > 0.65 Clustered into Mutual Correlations
78
![Page 79: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/79.jpg)
Response Index-Index Plot where |ri,j| > 0.65 Clustered into Mutual Correlations
79Plot suggests MesoNet exhibits 7 distinct behaviors
![Page 80: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/80.jpg)
22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows flows attempting to transfer datay1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement interval5 L R t 4/( 3+ 4)y5 Loss Rate: y4/(y3+y4)
y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement interval9 C ti F il R t 8/( 8 1)y9 Connection-Failure Rate: y8/(y8+y1)
y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
80
y20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 81: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/81.jpg)
Matrix of Pair-wise Scatter Plots & Correlation Coefficients (Ordered)
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
81
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
![Page 82: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/82.jpg)
22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows flows attempting to transfer datay1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement interval5 L R t 4/( 3+ 4)y5 Loss Rate: y4/(y3+y4)
y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement interval9 C ti F il R t 8/( 8 1)y9 Connection-Failure Rate: y8/(y8+y1)
y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
82
y20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 83: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/83.jpg)
Matrix of Pair-wise Scatter Plots & Correlation Coefficients (Ordered)
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
83
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
![Page 84: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/84.jpg)
22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows flows attempting to transfer datay1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement interval5 L R t 4/( 3+ 4)y5 Loss Rate: y4/(y3+y4)
y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement interval9 C ti F il R t 8/( 8 1)y9 Connection-Failure Rate: y8/(y8+y1)
y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
84
y20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 85: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/85.jpg)
Matrix of Pair-wise Scatter Plots & Correlation Coefficients (Ordered)
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
85
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
![Page 86: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/86.jpg)
22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement intervaly5 Loss Rate: y4/(y3+y4)y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement intervaly9 Connection-Failure Rate: y8/(y8+y1)y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly p py13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)y q g y y ( )
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
86
y19 Average Throughput for Active DN Flowsy20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 87: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/87.jpg)
Matrix of Pair-wise Scatter Plots & Correlation Coefficients (Ordered)
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
87
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
![Page 88: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/88.jpg)
22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement intervaly5 Loss Rate: y4/(y3+y4)y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement intervaly9 Connection-Failure Rate: y8/(y8+y1)y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly p py13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)y q g y y ( )
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
88
y19 Average Throughput for Active DN Flowsy20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 89: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/89.jpg)
Matrix of Pair-wise Scatter Plots & Correlation Coefficients (Ordered)
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
89
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
![Page 90: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/90.jpg)
22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement intervaly5 Loss Rate: y4/(y3+y4)y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement intervaly9 Connection-Failure Rate: y8/(y8+y1)y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly p py13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)y q g y y ( )
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
90
y19 Average Throughput for Active DN Flowsy20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=22)
![Page 91: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/91.jpg)
Matrix of Pair-wise Scatter Plots & Correlation Coefficients (Ordered)
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
91
Red 80 > |r|x100 < 100 Blue 30 > |r|x100 < 80 Green |r|x100 < 30
![Page 92: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/92.jpg)
Summary: Response Index-Index Plot where |ri,j| > 0.65 Clustered into Mutual Correlations
2 responses uncorrelated(1) throughput on DD flows(2) flow completion rate
25 correlationpairs reflecting congestion
(2) flow completion rate
14 correlationpairs reflectingpacket lossespacket losses
3 pair-wise correlations:(1) throughput on flows constrained by F-class routers(2) net ork dela(2) network delay(3) packets entering and leaving the network
92Plot suggests MesoNet exhibits 7 distinct behaviors
![Page 93: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/93.jpg)
Summary of Correlation Results
Correlation AnalysisDimension ResponsesCongestion y1, y2, y7, y11, y12,
y19, y21, y22Losses y5, y8, y9, y10, y13,
14y14Delay y15, y16F-class TP y18, y20D-class TP y17Packet TP y3, y4Flow TP y6Flow TP y6
93
![Page 94: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/94.jpg)
Summary of Correlation Results
Correlation AnalysisDimension ResponsesCongestion y1, y2, y7, y11, y12,
y19, y21, y22Losses y5, y8, y9, y10, y13,
14y14Delay y15, y16F-class TP y18, y20D-class TP y17Packet TP y3, y4Flow TP y6Flow TP y6
94
![Page 95: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/95.jpg)
22 Responses: 16 Macro + 6 ThroughputResponse Definition
y1 Active Flows – flows attempting to transfer datay2 Proportion of potential flows that were active: Active Flows/All Sourcesy3 Data packets entering the network per measurement intervaly4 Data packets leaving the network per measurement intervaly5 Loss Rate: y4/(y3+y4)y6 Flows Completed per measurement intervaly7 Flow-Completion Rate: y6/(y6+y1)y8 Connection Failures per measurement intervaly9 Connection-Failure Rate: y8/(y8+y1)y10 Retransmission Rate (ratio)y11 Congestion Window per Flow (packets)y12 Window Increases per Flow per measurement intervaly p py13 Negative Acknowledgments per Flow per measurement intervaly14 Timeouts per Flow per measurement intervaly15 Smoothed Round-Trip Time (ms)y16 Relative queuing delay: y15/(x1x41)y q g y y ( )
y17 Average Throughput for Active DD Flowsy18 Average Throughput for Active DF Flowsy19 Average Throughput for Active DN Flows
95
y19 Average Throughput for Active DN Flowsy20 Average Throughput for Active FF Flowsy21 Average Throughput for Active FN Flowsy22 Average Throughput for Active NN Flows
(k=11,n=64,m=227)
![Page 96: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/96.jpg)
Correlation Analysis & Clustering Suggests MesoNet Behavior Reflected in Only 7 ResponsesReflected in Only 7 Responses
Response DefinitionAverage number of packet output per measurement intervaly4 Average number of packet output per measurement interval(network throughput in packets/sec)
y6 Average number of flows completed per measurement interval(network throughput in flows/sec)
y10 Average retransmission rate (packet loss)y10 Average retransmission rate (packet loss)y15 Average smoothed round-trip time (network delay)
y17 Average instantaneous throughput for DD flows(throughput in packets/sec for the most advantaged users)A erage instantaneo s thro ghp t for FF flo sy20 Average instantaneous throughput for FF flows(throughput in packets/sec for 2nd most advantaged users)
y22 Average instantaneous throughput for NN flows(network congestion)
96
![Page 97: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/97.jpg)
Q. Why is the Scatter Plot of Y7 vs Y22 Bifurcated?
97
![Page 98: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/98.jpg)
Q. Why is the Scatter Plot of Y7 vs Y22 Bifurcated?
98
![Page 99: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/99.jpg)
Method 2: Principal Components Analysis
99
![Page 100: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/100.jpg)
Principal Components Analysis of 22 MesoNet Responses
Most response variance appears to be accounted for by the first 4 components100
![Page 101: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/101.jpg)
Weight Vectors for the first 4 Components
![Page 102: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/102.jpg)
|Weight| Vectors for the first 4 Components
0.75
1
cale
d M
AX =
1
PC1
0.75
1PC2
scal
ed M
AX =
1
0.25
0.5
ardi
zed
Wei
ght|
sc
0.25
0.5
dard
ized
Wei
ght|
s
0
0 2 4 6 8 10 12 14 16 18 20 22
Response Identifier (y1-y22)
|Sta
nda
00 2 4 6 8 10 12 14 16 18 20 22
Response Identifier (y1-y22)
|Sta
n
1PC3
AX =
1
1
AX =
1 PC4
0.5
0.75
Wei
ght|
scal
ed M
A
0.5
0.75
d W
eigh
t| sc
aled
MA
0
0.25
0 2 4 6 8 10 12 14 16 18 20 22Response Identifier (y1-y22)
|Sta
ndar
dize
d
0
0.25
0 2 4 6 8 10 12 14 16 18 20 22|Sta
ndar
dize
d
Response Identifier (y1-y22)
102
![Page 103: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/103.jpg)
Significant Responses in PC1 (congestion) Significant Responses in PC2 (delay)
Response Definitiony1 Average number of active flowsy2 Proportion of possible flows that are activey5 Loss rate7 Fl l ti t
Significant Responses in PC1 (congestion)
Response Definitiony15 Smoothed round-trip timey16 Relative queuing delay
Significant Responses in PC2 (delay)
y7 Flow-completion ratey8 Connection failuresy9 Connection-failure rate
y10 Retransmission ratey11 Average congestion windowy12 Window-increase ratey13 Negative-acknowledgment ratey13 Negative acknowledgment ratey14 Timeout ratey19 Average instantaneous throughput for DN flowsy21 Average instantaneous throughput for FN flowsy22 Average instantaneous throughput for NN flows
Response Definition
Significant Responses in PC3 (throughput for advantaged users)
Response Definition
Significant Responses in PC4(network throughput in flows/second)
y3 Packets inputy4 Packets output
y17 Average instantaneous throughput for DD flowsy18 Average instantaneous throughput for DF flowsy20 Average instantaneous throughput for FF flows
y3 Packets inputy4 Packets outputy6 Flows completed per measurement interval
103
![Page 104: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/104.jpg)
Summary of PCA Results
PCA AnalysisDimension Responses
1 2 5 7 8 9PC1:Congestion
y1, y2, y5, y7, y8, y9, y10, y11, y12, y13, y14, y19, y21, y22
PC2: Delay y15 y16PC2: Delay y15, y16
PC3:D-class &F class TP
y3, y4, y17, 18, y20F-class TP
PC4: Flow TP y3, y4, y6
104
![Page 105: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/105.jpg)
105
![Page 106: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/106.jpg)
Comparing Correlation & PCA Results
Correlation Analysis PCA
Dimension Responses Dimension Responses
Congestion y1, y2, y7, y11, y12, y19, y21, y22 PC1:
Congestion
y1, y2, y5, y7, y8, y9, y10, y11, y12, y13, y14, y19, y21, y22Losses y5, y8, y9, y10, y13, y14
Delay y15, y16 PC2: Delay y15, y16
F-class TP y18, y20PC3:D-class & y3, y4, y17, 18, y20D-class TP y17F-class TP
Packet TP y3, y4
Flow TP y6 PC4: Flow TP y3, y4, y6
The results show good alignment:PCA1 merges congestion + losses;PCA2 & Correlation identical for delay;PCA3 merges D-class & F-class Throughput;PCA4 splits Packet TP acrosstwo dimensions (D- & F-class TP and Flow TP)
106
![Page 107: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/107.jpg)
Identifying Significant Response y g g pDimensions for MesoNet:
4 or 7 or something between?4 or 7 or something between?
107
![Page 108: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/108.jpg)
HIGHER CONGESTION ISLOWER TP: -X2, -X5, +X9, +X8, +X1
y22 – NN TP Note X2 is miscoded so I reverse +/- for X2 PC1
PC+ IS: -X2, -X5, +X9, +X8, +X4, -X3
PC1
Note that PC interpretation is possible only byresorting to cross-mapping with response variables
PC1
y10 – Retransmission Rate
I THINK LOSS & CONGESTION SHOULDBE SEPARATE – SIMILAR CAUSES BUT
108HIGHER IS -X2, -X3, +X9, +X4, -X5, +X8, -X1
SUBTLE DIFFERENCES
![Page 109: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/109.jpg)
HIGHER TP: -X1, +X9, +X2, +X5, +X4y20 – FF TP PC3
PC- IS: +X4, +X2, -X1, +X9, +X3
PC3PC3
y17 – DD TP
I THINK D-class & F-class THROUGHPUTSHOULD BE SEPARATE – ONLY TWO
109HIGHER TP IS –X1, +X4
INPUT FACTORS INFLUENCED-class THROUGHPUT
![Page 110: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/110.jpg)
HIGHER PO: +X2, -X5, +X4y4 – Packet Output Rate PC4
PC- IS: +X2, -X5, -X4, -X3
PC4
THIS SEEMS A BETTER MATCH FORFLOW COMPLETION RATE
PC4
y6 – Flow Completion Rate
I THINK FLOW COMPLETE RATE &I THINK FLOW COMPLETE RATE &PACKET THROUGHPUT RATE SHOULDBE KEPT SEPARATE BECAUSE FLOWCOMPLETE IS HIGHER WITH SMALL
110HIGHER FC IS +X2, -X4, -X5
FILE SIZE & PACKET OUTPUT IS HIGHERWITH LARGE FILE SIZE
![Page 111: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/111.jpg)
Note: The Domain Analyst Sides With the Correlation Analysis Resultsthe Correlation Analysis Results
Dimension Definition1 Congestion2 Loss3 Delay4 Throughput for the most advantaged users4 Throughput for the most advantaged users5 Throughput for the somewhat advantaged users6 Network-wide Packet Throughput7 Network-wide Flow Throughput
111
![Page 112: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/112.jpg)
Pros & Cons of the 2Dimension Reduction
TechniquesTechniques
112
![Page 113: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/113.jpg)
Pros/Cons of Correlation Analysis & Clustering
• Provided effective dimension reduction (22 → 7) through
Pros• Provided effective dimension reduction (22 → 7) through
correlations that could be vetted by a domain expert• Examining response correlations helped to validate MesoNet
Uncovered nuanced differences between flow and packet• Uncovered nuanced differences between flow and packet throughput rates in a network
Cons• A second 211-5 OFF experiment with different level settings revealed
some (valid) differences in correlations – thus separate correlation f ff
Cons
analyses must be conducted for different level settings
113
![Page 114: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/114.jpg)
Pros/Cons of Principal Components Analysis
• Provided greater dimension reduction (22 → 4) than correlation analysis & clustering
Pros
analysis & clustering
There is no specific domain interpretation of even the top 2 or 3Cons
• There is no specific domain interpretation of even the top 2 or 3 principal components – in the case shown here we were able to arrive at a reasonable interpretation; in other cases, we were notPrincipal components take on + and values which present domain• Principal components take on + and – values, which present domain analysts with difficulty assigning meaning – we had to infer meaning of components by comparing them with meaning derived from analyzing individual responsesanalyzing individual responses
• Principal components proved coarser than corresponding groupings generated by clustering mutual correlations
• A second 211-5 OFF experiment with different level settings revealed• A second 211-5 OFF experiment with different level settings revealed some differences in principal components – such differences are difficult to understand without assistance from other analyses 114
![Page 115: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/115.jpg)
Summary: Correlation Analysis or PCA?• If limited to one technique, correlation analysis provides results
easier for a domain analyst to comprehendP i i l t t k + d l hi h t d i• Principal components take on + and – values, which present domain analysts with difficulty assigning meaning – we had to infer meaning by comparing main effects plots of principal components with main effects plots from responses chosen from groupings established byeffects plots from responses chosen from groupings established by correlation analysis
• Principal components proved coarser than corresponding groupings generated by clustering mutual correlationsgenerated by clustering mutual correlations
• PCA provides a reasonable complement to correlation analysis by giving a separate view of the data, which should be consistent with correlation results thus helping to validate a modelcorrelation results, thus helping to validate a model
115
![Page 116: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/116.jpg)
MesoNet Conclusions• We investigated correlation and PC analyses as two
techniques to reduce the dimension of responses from MesoNet, a network simulator,
• We demonstrated that both techniques can significantly reduce the dimension of response dataWe also showed that both techniques could be used to• We also showed that both techniques could be used to validate a model, but that PCA is more suited as a complement to correlation analysis
• We found that PCA results are difficult for a domain analyst to interpret without comparison to analyses of individual responses
• We also found that results from correlation and PC analyses with one set of parameter values cannotnecessarily be extrapolated to a different set of values y p
116
![Page 117: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/117.jpg)
Stat Conclusions
1. Stat Framework/Approach & Methodology: pp gyDemo beginning-to-”end”
2. Critical importance of domain expert2. Critical importance of domain expert
3. Dimension Reduction dependency onDEX & Sensitivity AnalysisDEX & Sensitivity Analysis
4. Internet Modeling Conclusions & Insight
117
![Page 118: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/118.jpg)
Methodology Applications
1. MesoNet Analysis #1 (k=11,n = 64,m=22 7) S iti it & Di i R d ti A l i t d ’ t lkSensitivity & Dimension-Reduction Analysis <today’s talk>
2. MesoNet Analysis #2 (k=20,n=256,m=22)Sensitivity AnalysisSensitivity Analysis
3. MesoNet TCP Congestion/Control Alg. Comparison (k=6,n=32)(5)
4. Cloud Computing Analysis (k=11,n=64,m=42 => 8) (Koala)Sensitivity & Dimension-Reduction Analysis
5. Cloud Computing VM Placement Alg. Comparison (k=6,n=32) (Koala)
118
![Page 119: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/119.jpg)
Graphical Methods
1. Main Effects Plots2 Interaction Effects Matrix2. Interaction Effects Matrix3. Ordered Data Plots4. Pairwide Scatter Plot Matrix (Unordered)5 Pairwise Scatter Plot Matrix (Ordered)5. Pairwise Scatter Plot Matrix (Ordered)6. Stacked Main Effects Plot7. Multiplot of (1-Way) ANOVA CDF Values8. Index-Index Cluster Plot9. Character Plots
10. PCA Weights Plot10. PCA Weights Plot
119
![Page 120: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/120.jpg)
PresentationsJ. Filliben, "Sensitivity Analysis Methodology for a Complex System Computational Model", 39th Symposium on the Interface: Computing Science and Statistics, Philadelphia, PA, May 26, 2007.
K Mill d J Fillib "A Effi i t S iti it A l i M th d f M i N t k M d l "K. Mills and J. Filliben, "An Efficient Sensitivity Analysis Method for Mesoscopic Network Models", Complex Systems Study Group, NIST, February 2, 2010.
K. Mills and J. Filliben, "Comparing Two Dimension-Reduction Methods for Network Simulation Models", Winter Simulation Conference (WSC 2010), Baltimore, Maryland, Dec. 6, 2010., ( ), , y , ,
K. Mills and J. Filliben, "Using Sensitivity Analysis to Identify Significant Parameters in a Network Simulation", Winter Simulation Conference (WSC 2010), Baltimore, Maryland, Dec. 6, 2010.
K Mills J Filliben D Y Cho and E Schwartz "Predicting Macroscopic Dynamics in LargeK. Mills, J. Filliben, D.-Y. Cho and E. Schwartz, Predicting Macroscopic Dynamics in Large Distributed Systems", LSN Seminar on Complex Networks and Information Systems, Gaithersburg, Maryland, June 30, 2011.
K. Mills, J. Filliben and C. Dabrowski, "An Efficient Sensitivity Analysis Method for Large Cloud Simulations", IEEE Cloud 2011, Washington, D.C., July 8, 2011.
K. Mills, J. Filliben, D.-Y. Cho and E. Schwartz, "Predicting Macroscopic Dynamics in Large Distributed Systems", American Society of Mechanical Engineers2011 Conference on Pressure Vessels & Piping Baltimore MD July 21 2011
120
2011 Conference on Pressure Vessels & Piping, Baltimore, MD, July 21, 2011.
![Page 121: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/121.jpg)
ReferencesK. Mills, “Measurement Science for Complex Information Systems”, NIST/ITL Web Page for the Complex Systems Project: http://www.nist.gov/itl/antd/emergent_behavior.cfm
K. Mills, J. Filliben, D. Cho, E. Schwartz and D. Genin, "Study of Proposed Internet Congestion Control Mechanisms“, NIST Special Publication 500-282, May 2010,534 pages. http://www.nist.gov/itl/antd/Congestion_Control_Study.cfm
K. Mills, J. Filliben and C. Dabrowski, "An Efficient Sensitivity Analysis Method for Large Cloud Simulations“, Proceedings of the 4th International Cloud Computing Conference, IEEE, Washington, D.C., July 5-9, 2011.
K. Mills, J. Filliben, D-Y. Cho and E. Schwartz, "Predicting Macroscopic Dynamics in Large Distributed Systems“, Proceedings of ASME 2011 Conference on Pressure Vessels & Piping, Baltimore, MD, July 17-22, 2011.
K. Mills and J. Filliben, "Comparison of Two Dimension-Reduction Methods for Network Simulation Models“, Journal of Research of the National Institute of Standards and Technology, 116-5, September-October 2011, in press.
121
K. Mills, J. Filliben and C. Dabrowski, "Comparing VM-Placement Algorithms for On-Demand Clouds“, (submitted to IEEE CloudCom 2011, under review.
![Page 122: Comparison of Two Dimension-Reduction Mthd f Nt kSi lti](https://reader035.vdocument.in/reader035/viewer/2022071612/61570655a097e25c76503e2a/html5/thumbnails/122.jpg)
Web ReferencesComplex Systems Projecthttp://www.nist.gov/itl/antd/emergent_behavior.cfm
NIST SP 500-282 (534 pages) http://www.nist.gov/itl/antd/Congestion_Control_Study.cfm
NIST/SEMATECH Engineering Statistics Handbookhttp://www.itl.nist.gov/div898/handbook/
Dataplothttp://www.itl.nist.gov/div898/software/dataplot/
This Talkhttp://stat.nist.gov/~filliben/fillibenmillsnistsedtalk092211.pdfhttp://www.nist.gov/itl/antd/upload/millsjjfsedtalk092211.pdf
122