sip performance benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01...
TRANSCRIPT
![Page 1: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/1.jpg)
SIP Performance Benchmarking
draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01
March 22, 2010
Prof. Carol Davids, Illinois Inst. of Tech. Dr. Vijay Gurbani, ALU
Scott Poretsky, Allot Communications
1IETF 77 – Anaheim, BMWG
![Page 2: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/2.jpg)
Status• Working Group last call in progress• Needs reviewers and comments• IIT Masters Candidates working to implement the
methodology on open source SIP servers – results to be available soon
2IETF 77 – Anaheim, BMWG
![Page 3: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/3.jpg)
Summary of the ContentsTerminology
• SIP Benchmarking Terminology provides 4 sets of definitions:
– Protocol Components – defines the signaling, media and control planes; sessions with and without associated media, Invite-initiated sessions and Non-Invite initiated sessions
– Test Components – defines parts of the test agent
– Test Setup Parameters – defines a Session Attempt Rate, Establishment Threshold Time, and other parameters that must be recorded before entering a test cycle
– Benchmarks – defines seven test parameters
3IETF 77 – Anaheim, BMWG
![Page 4: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/4.jpg)
Benchmarks• 3.4.1. Registration Rate
Definition: The maximum number of registrations that can be successfully completed by the DUT/SUT in a given time period.
• 3.4.2. Session Establishment RateDefinition: The maximum average rate at which the DUT/SUT can successfully establish sessions.
• 3.4.3. Session Capacity Definition: The maximum number of Established Sessions that can exist simultaneously on the DUT/SUT until Session Attempt Failure occurs.
• 3.4.4. Session Overload Capacity Definition: The maximum number of Established Sessions that can exist simultaneously on the DUT/SUT until it stops responding to Session Attempts.
• 3.4.5. Session Establishment Performance Definition: The percentage of Session Attempts that become Established Sessions over the duration of a benchmarking test.
• 3.4.6. Session Attempt Delay Definition: The average time measured at the Emulated Agent for a Session Attempt to result in an Established Session.
• 3.4.7 IM Rate
Definition: Maximum number of IM messages completed by the DUT/SUT.
4IETF 77 – Anaheim, BMWG
![Page 5: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/5.jpg)
Reporting Format
Test Setup
SIP Transport Protocol = ____________________
Session Attempt Rate = _____________________
IS Media Attempt Rate = ____________________
Total Sessions Attempted = __________________
Media Streams Per Session = ________________
Associated Media Protocol = _________________
Media Packet Size = ________________________
Media Offered Load = _______________________
Media Session Hold Time = __________________
Establishment Threshold Time = _______________
Loop Detecting Option = _____________________
Forking Option = ___________________________
Number of endpoints request sent to = ________
Type of forking = __________________________
5
Benchmarks for IS
Session Capacity = __________________________
Session Overload Capacity = __________________
Session Establishment Rate = _________________
Session Establishment Performance = __________
Session Attempt Delay = _____________________
Session Disconnect Delay = __________________
Benchmarks for NS
IM Rate = _______________________________
Registration Rate = _________________________
Re-registration Rate = ________________________
IETF 77 – Anaheim, BMWG
![Page 6: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/6.jpg)
Summary of Changes
• Updated and clarified some definitions and test cases based on clarification from implementation in IIT VoIP Research Lab.
6IETF 77 – Anaheim, BMWG
![Page 7: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/7.jpg)
Next Steps• Reviewers and comments please!• Create -02 incorporating lessons learned in lab testing
as well as reviewers’ comments.
7IETF 77 – Anaheim, BMWG
![Page 8: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/8.jpg)
BACKUP
8IETF 77 – Anaheim, BMWG
![Page 9: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/9.jpg)
Scope – DUT/SUT
• The DUT must be a RFC 3261 capable network equipment. This is referred to as the "Signaling Server".
– This may be a Registrar, Redirect Server, Stateless Proxy or Stateful Proxy. A DUT MAY also include a B2BUA, SBC, or P-CSCF functionality.
– The DUT MAY be a multi-port SIP-to-switched network gateway implemented as a SIP UAC or UAS
– The DUT or SUT MUST NOT be end user equipment.
• The DUT MAY have an internal SIP ALG, Firewall, and/or a NAT. This is referred to as the "SIP Aware Stateful Firewall.“
The Tester acts as multiple "Emulated Agents" that initiate (or respond) to SIP messages as session endpoints and source (or receive) “Associated Media” for established connections.
Terminology defines SIP Control Plane performance benchmarks for black-box measurements of SIP signaling of networking devices
– Stress and debug scenarios are not addressed in this work item
9
Signaling
SUT
IETF 77 – Anaheim, BMWG
![Page 10: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/10.jpg)
Scope – Signaling and Media• Control signaling is benchmarked
• Media performance is not benchmarked in this work item
• It is RECOMMENDED that control plane benchmarks are performed with media present, but this is optional.
• The SIP INVITE requests MUST always include the SDP body
• The type of DUT dictates whether the associated media streams traverse the DUT or SUT. Both scenarios are within the scope of this work item.
DUT or SUT –Calling UE –
TesterCalled UE –
Tester
Signaling Signaling
Associated Media
DUT or SUT –Calling UE –Tester
Called UE –Tester
Signaling Signaling
Associated Media
Associated Media
10IETF 77 – Anaheim, BMWG
![Page 11: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/11.jpg)
Session Terms• 3.1.6. Session Attempt
Definition: A SIP Session for which the Emulated Agent has sent the SIP INVITE or SUBSCRIBE NOTIFY and has not yet received a message response from the DUT/SUT.
• 3.1.7. Established Session Definition: A SIP session for which the Emulated Agent acting as the UE/UA has received a 200OK message from the DUT/SUT.
• 3.1.8. Invite-initiated Session (IS) Definition: A Session that is created by an exchange of messages in the Signaling Plane, the first of which is a SIP INVITE request.
• 3.1.9. Non-INVITE-initiated Session (NS) Definition: A session that is created by an exchange of messages in the Signaling Plane that does not include an initial SIP INVITE message.
• 3.1.10. Session Attempt Failure Definition: A session attempt that does not result in an Established Session.
• 3.1.11. Standing Sessions <added and the definition needs to be changed – as below>
Definition: A SIP session that is currently an established session.
11IETF 77 – Anaheim, BMWG
![Page 12: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/12.jpg)
Scope - Scenarios• Session Establishment performance is benchmarked• Both INVITE and non-INVITE scenarios (such as IM) are addressed• Different transport mechanisms -- such as UDP, TCP, SCTP, or
TLS -- may be used; – Transport mechanism MUST be noted as a condition of the test
as the performance of SIP devices may vary accordingly• Looping and forking options are also considered
– Impacts processing at SIP proxies• REGISTER and INVITE requests may be challenged or remain
unchallenged for authentication purpose as this may impact the performance benchmarks. – Any observable performance degradation due to authentication
is considered to be of interest to the SIP community
12IETF 77 – Anaheim, BMWG
![Page 13: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/13.jpg)
Scope - Overload
• SIP Overload is considered within the scope of this work item:– Considerations on how to handle overload are deferred to work
progressing in the SIPPING working group. • The normal response to an overload stimulus -- sending a 503 response -- is
considered inadequate.
– Vendors are free to implement their specific overload control behavior as the expected test outcome if it is different from the IETF recommendations. However, such behavior MUST be documented and interpreted appropriately across multiple vendor implementations.
• This will make it more meaningful to compare the performance of different SIP overload implementations.
– This draft now has a dependency on the strategy of the overload work in SIPPING
13IETF 77 – Anaheim, BMWG
![Page 14: SIP Performance Benchmarking draft-ietf-bmwg-sip-bench-term-01 draft-ietf-bmwg-sip-bench-meth-01 March 22, 2010 Prof. Carol Davids, Illinois Inst. of Tech](https://reader036.vdocument.in/reader036/viewer/2022082821/5697bfdf1a28abf838cb2ec1/html5/thumbnails/14.jpg)
Out of Scope Scenarios• SIP Control performance benchmarking is the focus of this work item.
– Media performance is not benchmarked in this work item – Stress and Steady-State benchmarking is not considered in scope. This could be
covered in an Appendix if preferred.• Re-INVITE requests are not considered in scope• Benchmarking SIP Presence is not considered in scope
• IMS-specific scenarios are not considered, but test cases can be applied with 3GPP-specific SIP signaling and the P-CSCF as a DUT
• Session disconnect is not considered in scope– Only session establishment is considered for the performance
benchmarks. – Disconnect is a lightweight transaction to release resources for steady-state. – Has no performance benchmark because dependent on INVITE– posted on SIPPING for feedback
14IETF 77 – Anaheim, BMWG