doc.: ieee 802.11-13/1051r0 submission september 2013 ron porat, broadcom evaluation methodology...
DESCRIPTION
doc.: IEEE /1051r0 SubmissionRon Porat, Broadcom Simulation Methodology - General Concept We propose to have three tools to assess proposed HEW techniques performance and gain relative to 11ac, each having its own advantages : –PER simulations – typically used for new PHY features for assessing point to point performance –PHY System simulations – provide system-wise (multi-BSS) performance assessment with emphasis on PHY abstraction accuracy and simplified MAC –MAC System simulations - provides system-wise (multi-BSS) performance assessment with emphasis on MAC accuracy and simplified PHY While PER simulations have been used extensively in the past, system simulations have not been used nearly as much. However, based on the discussions in HEW so far, the main effort revolves around techniques for improving multi-BSS gain and not higher point to point throughput. Hence the need for system simulations. PHY/MAC system simulations may be used separately in order to: –Simplify (using abstraction) some of the MAC/PHY details respectively –Speed up development (by reducing dependency of PHY on MAC and vice versa) –Improve insight as to the specific reason for performance gains/losses (a more difficult thing to assess in multi-STA multi-BSS environments that include all possible PHY and MAC details). –Some proposed techniques may not require all PHY/MAC details in order to evaluate gains but are nevertheless used over the same simulation scenarios as defined in [10][11] September 2013 Slide 3TRANSCRIPT
doc.: IEEE 802.11-13/1051r0
Submission
September 2013
Ron Porat, Broadcom
Evaluation Methodology
Date: 2013-09-16Authors:
Name Affiliations Address Phone email
Ron Porat Broadcom 16340 West Bernardo Dr., San Diego, CA
92127
858-521-5409 [email protected]
Matt Fischer Broadcom
Slide 1
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Outline
• An evaluation methodology document for HEW is required in order to assess gains relative to 11ac.
• We provide here a description of several topics:– Simulation methodology – Metrics– Traffic models
September 2013
Slide 2
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Simulation Methodology - General Concept
• We propose to have three tools to assess proposed HEW techniques performance and gain relative to 11ac, each having its own advantages :– PER simulations – typically used for new PHY features for assessing point to point performance– PHY System simulations – provide system-wise (multi-BSS) performance assessment with emphasis
on PHY abstraction accuracy and simplified MAC – MAC System simulations - provides system-wise (multi-BSS) performance assessment with emphasis
on MAC accuracy and simplified PHY
• While PER simulations have been used extensively in the past, system simulations have not been used nearly as much. However, based on the discussions in HEW so far, the main effort revolves around techniques for improving multi-BSS gain and not higher point to point throughput. Hence the need for system simulations.
• PHY/MAC system simulations may be used separately in order to:– Simplify (using abstraction) some of the MAC/PHY details respectively– Speed up development (by reducing dependency of PHY on MAC and vice versa)– Improve insight as to the specific reason for performance gains/losses (a more difficult thing to assess
in multi-STA multi-BSS environments that include all possible PHY and MAC details).– Some proposed techniques may not require all PHY/MAC details in order to evaluate gains
but are nevertheless used over the same simulation scenarios as defined in [10][11]
September 2013
Slide 3
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
System Simulation - General Description
• A system simulation is comprised of multiple drops and multiple TXOP. – A drop is defined by a known AP and STA locations. Simulating multiple drops is important in
order to randomize STA location to better assess issues such as OBSS interference and intra-BSS throughput (as these issues are highly dependent on the relative path loss between AP and STA).
– In a TXOP a set of transmissions occur. Multiple TXOP (with typical aggregate duration 1-10[sec] beyond a warm-up time) are required to assess performance of a given set of APs and STAs configuration. Each BSS may have different start time, duration and end time for its TXOP but time alignment (start, duration, end) of the TXOP in the system may be an outcome of a proposed MAC protocol.
– A’warm-up’ period may be used for some parameters to converge:• MCS selection based on the past • Offered load - if all flows start at T0, then the offered load goes from 0 to X instantaneously, and over
some simulation time that follows, the backoff mechanism will effectively reduce the total offered load by increasing CW at nodes, until the offered load is at Y < X
September 2013
Slide 4
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Cont.
• General simulation structure:• For drop=1:N {
– APs and STA are dropped (randomly or in fixed locations) in a given area following certain placement rules (specifics depend on the scenario). • If simulation time is an issue, multiple drops’ locations can be agreed beforehand in order to
minimize the number of drops and enable multiple companies’ results comparison. – STAs are associated with APs (typically based on path loss but could use other rules if some
AP are not be available (e.g. belonging to a different apartment or operator) in some scenarios.– For TXOP=1:M {
• Note - For MAC simulations, one can count time, ensuring that enough time has passed to see M TXOP
• Perform transmissions consistent with all the parameters defined and collect metrics. Parameters to be defined:
– For DL and UL – channel access rules: EDCA, centralized control, restricted access windows etc..– For DL (and UL if centralized control is proposed): how the AP allocates air time to users -e.g. proportional
fair, round robin. – CCA rules– Various other EDCA parameters as specified in the MAC simulations section
– }• }
September 2013
Slide 5
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Simulation Methodology I - PER Simulations
• PHY PER simulations are used to verify point to point performance or aspects that need this type of simulation, especially new PHY features or preamble performance.
• PHY impairments such as PA non-linearity, phase noise, channel estimation error, non-linear receivers are more readily incorporated into PER simulations and may be needed if some techniques are adversely affected by those impairments [6][9].
• Other impairments such as the impact of OBSS interference or inter-symbol interference should also be tested by PER simulations before being abstracted in a system simulation.
• PER simulations should also be used to investigate potential error floors that are not well captured by system simulations due to PHY abstraction inaccuracies.
September 2013
Slide 6
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Simulation Methodology II - PHY System Sims
• As discussed the emphasis here is on more accurate representation of the PHY using PHY abstraction
• For simplicity one might consider an assumption of time aligned TXOPs.
• PHY parameters that need to be defined per AP and per STA:– Transmission BW (could be different in different BSS)– Number of antennas (could be different for different STA)– Transmission scheme- SU OL, SU BF, MU– Channel model (multipath fading, Doppler, path loss, shadowing etc..)
• Link adaptation needs to be described as well – – MCS choice (genie could be used to assess upper performance bound but not realistic performance). – Transmission mode selection SU OL vs. BF vs. MU should be described (transmission mode can be fixed or
dynamic).
• A subset of the full set of MAC features can be assumed:– E.g. CCA levels and RTS-CTS can be specified for the purpose of determining which transmission can
simultaneously occur in multiple BSS– No need to simulate management frames but rather a fixed amount of overhead can be assumed
September 2013
Slide 7
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
PHY Abstraction
• The objective of PHY abstraction is to accurately predict PER simulation results in a computationally simple way to enable running system simulations in a timely manner.
• The underlying principle is to calculate an effective average SINR (SINReff ) in a given OFDM symbol. This quantity then acts as a link between AWGN PER and multipath channel PER for a given coding type, block size and MCS level.
• Effective SINR (SINReff ) is typically calculated as follows
where SINRn is the post processing SINR at the n-th subcarrier, N is the number of subcarriers in a coded block and Φ is a mapping function.
• Several mapping functions can be used such as Constrained Capacity, EESM, MMIB, etc. [12]
September 2013
Slide 8
1
1
1 ( )N
eff nn
SINR SINRN
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Cont.
• A general description is as follows:– Start from an agreed upon per-MCS required SNR in AWGN assuming e.g. 1000bytes SISO
10% PER point.– With one receive antenna:
1. Compute SINR per tone – the ‘S’ term is a function of the Tx power and channel. The ‘I’ term is due to OBSS, intra-BSS interference or MU-MIMO related interference. Note that ‘I’ could vary during a packet due to shorter interfering packet than the desired packet or start of new interfering packet midway through the desired packet.
2. If constraint capacity is used, calculate the per-tone capacity log2(1+SINR), this could be constrained to 256QAM capacity, and average across all data tones used for transmission. If MMIB is used, calculate the average per-bit capacity as described in [13]
3. From 2, derive the averaged SINR per tone.4. Transform back to MCS using the AWGN MCS table
– With multiple receive antennas:1. SINR should reflect the receive combining output from all antennas and the combining method should be
indicated2. For MIMO simulation, linear per-stream SINR can be computed.
September 2013
Slide 9
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Simulation Methodology III - MAC System Sims
• As discussed the emphasis here is on more accurate representation of the MAC• MAC parameters that need to be described:
– Beacon periodicity– Aggregation policy– Usage of RTS-CTS or CTS2SELF– EDCA parameters (AC, CWmin, CWmax, AIFSN)– CSMA backoff procedures, energy and preamble detection– Basic rate set
• If other MAC schemes are proposed they need to be described (e.g. RAW as defined in 11ah or HCCA)
• Link adaptation needs to be described – – MCS choice (genie could be used to assess upper performance bound but not realistic performance).
• Frequency re-use needs to be described
• PHY parameters may be simplified to a SISO configuration running over AWGN.– Accurate path loss and penetration should be modeled
• In the event that some techniques require both accurate MAC and PHY details to be simulated, this simulation can be augmented by adding more detailed PHY abstraction used for simulation methodology II to provide a complete system simulation tool
September 2013
Slide 10
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Simulation Methodology Choice
• Proponents of different techniques should also provide justification for their proposed simulation tool used to justify the technique’s gains.
• Examples:– PHY PER simulation:
• New PHY FFT size – as was done in 11ac (256,512) and 11ah (32) a PER simulation is typically sufficient in order to decide the number of pilots, interleaver parameters etc.
• Preamble performance• Implementation losses of current and new PHY modes.
– PHY System simulation:• Impact of number of antennas on multi-BSS performance• Impact of PHY techniques in the context of multi-BSS• Impact of frequency re-use in multi-BSS• Impact of CCA levels on PHY performance
– MAC System simulation:• Impact of MAC scheduler – EDCA vs. RAW (as proposed in 11ah) vs. HCCA vs. other techniques• Impact of frequency re-use in multi-BSS• Impact of CCA levels
September 2013
Slide 11
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Metrics
• For PER simulations the typical metric is dB gain/loss in waterfall curves.• For system simulations we propose to look at several metrics [2]-[9]:
– Aggregate area throughput [bps/m2] - this metric directly relates to the average throughput per BSS and can more accurately compare different deployment densities and heterogeneous deployments. For example:
• How is area throughput impacted as a function of inter AP distance and number of STA?• How is area throughput impacted with flat deployments vs. heterogeneous?
– Average per-STA throughput in all participating BSS– 5% point in the throughput CDF curve – this metric measures cell edge performance– More information may be provided such as the entire CDF curve and MCS histogram [3]
September 2013
Slide 12
doc.: IEEE 802.11-13/1051r0
Submission Ron Porat, Broadcom
Traffic Models• Two main traffic models may be simulated:
– Full buffer – users always have DATA to send– Poisson arrival – more realistic traffic model emulating more realistic interference scenario
• Packet size – mix of small and large packets should be evaluated in order to test realistic assumptions on system performance
• Higher layer QoS metrics may be developed [4]• Video transmission specific metrics may be developed [5]• Other specific types of traffic models may be used [6][7]
September 2013
Slide 13
doc.: IEEE 802.11-13/1051r0
Submission
References
[1] 11-13-0657-02-0hew-hew-sg-usage-models-and-requirements-liaison-with-wfa[2] 11-13-0486-01-0hew-metrics-targets[3] 11-13-0847-01-0hew-evaluation-criteria-and-simulation-scenarios[4] 11-13-0869-00-0hew-simulation-scenarios-and-metrics-for-hew[5] 11-13-0850-00-0hew-quantitative-qoe-requirements-for-hew[6] 11-13-0722-01-0hew-hew-evaluation-methodology[7] 11-13-0723-00-0hew-hew-sg-evaluation-methodology-overview[8] 11-13-0786-00-0hew-hew-sls-methodology[9] 11-13-0837-00-0hew-considerations-on-hew-evaluation-methodology[10] 11-13-1000-00-0hew-simulation-scenarios[11] 11-13-1001-00-0hew-simulation-scenarios-document-template[12] IEEE 802.16m-08/004r5, IEEE 802.16m Evaluation Methodology Document (EMD) Section 4 http://ieee802.org/16/tgm/core.html#08_004[13] 11-13-1059-00-0hew-PHY abstraction for HEW evaluation methodology
September 2013
Ron Porat, BroadcomSlide 14