www.symmetricom.com QoE DivisionYves Cognet

CTO

Quality of Experience metrics for IPTV

QoS as perceived by an end-user is called QoE (Quality of Experience)

QoE can be measured and benchmarked

QoE relates how an application is perceived by an end-user.

QoE metrics are content dependant and are impacted by network impairments as well as

content impairments

QoS versus QoE (highlight)

Quality of ExperienceQuality of Service Describes how IP can be used for offering Differentiated Services

Describes how Differentiated Services can improve Quality of Services (aka for offering Class Of Services)

QoS describes how a network is performing for a specific Class Of Service. QoS metrics are network centric metrics.

“Operators want to know the video performance as perceived by the customer through the IP TV network….”.

Key Challenges to solve

Assure service delivery (end2end monitoring)

Monitor service quality (Quality of Experience)

Reduce time to fix (identify the problem and diagnose the source)

Speed up deployment

Deliver what the customer is paying for

Reduce customer churns

Reduce cost of support

Secure revenue streams (competition)

What was missing

QoS for IPTV is well known but QoE for IPTV is in its infancy. The V-Factor™ patented technology is recognized as the “critical missing link” between IPTV QoE and IPTV QoS.

V-Factor technology takes into account Network impairments (delay, jitter, loss, miss ordering,..) as defined

by ITU Y1540/41and IETF RFC2330 Transport impairments (MPEG2 TS streamer jitter) as defined by

ETSI TR 101290 Content impairments (is codec dependant and measure the

quantization, size of macroblocks, …),

The V-Factor provides a score that measures the impact of losses/jitter on I, B or P frames/slices by simulating the Human Vision System.

IPTV QoE methodologies

Subjective QoE testing

Average opinion of many people concerning

quality of Video

Not scalable

Objective QoE testing

Score delivered by a system that rate the

quality of the video as perceived by a user

Human Human Vision Vision

System System ModelModel

Human Human Vision Vision

System System ModelModel

Video Quality Measurement Full Reference (FR)

Full Reference

Processor Processor

Original Video Received Video

Original Video

Transmitted Video Signal

ITU-T J.144

Video Quality Measurement Reduced Reference (RR)

Reduced-Reference •Spatial (Pixel) and Temporal (Motion) Information

Reduced-Reference Signal

Processor Processor

Transmitted Video Received Video

ITU-T J.143

Video Quality Measurement No-Reference (NR)

Processor Processor

Transmitted Video Received Video

V-Factor™

No-Reference• V-Factor – Moving Picture Quality Metrics (MPQM) based on the Human Vision System

What is V-Factor?

Based on MPQM (Moving Picture Quality Metrics) Considers both Network based metrics and Content based metrics

Measures Perceived Video and Audio Quality over time

What the V-Factor does 1/2

Transmitted Video Received Video

Multiplexer/streamer STB

MPEG PES packet analysisI,B,P frames and slices analysisAudio PESVideo PES : Slices, Macro block, Block, Quantizer

V-Factor analyses all PES video and audio packets in real time and collect statistics about I,B,P frames and slices. PES packets analysis is codec dependant (MPEG2, H264,..).

What the V-Factor does 2/2

Transmitted Video Received Video

IP/RTP/Transport Stream header inspectionPCR (jitter)Packet lossBandwidth

Multiplexer/streamer STB

V-Factor collects network statistics from IP header or RTP MPEG2TS header, Network packets analysis is transport dependant (RTP only, MPEG2TS over UDP, MPEG2TS over RTP over UDP, ..).

IP packets (6 to 7 PES)

The MPQM model is fed from network impairments at the IP level (as defined by ITU U1540/41 and RFC2330) and also information collected from the content of the compressed video layer (GOP, Quantizer,..)The PLR is computed from network impairments, the Program Clock Reference (PCR) Jitter and Network Jitter.

Loss Loss modelmodel

Mar

kov

mo

del

sM

arko

v m

od

elsBursty

Loss Episode

Single Loss

MP

QM

mo

del

Packet Loss Rate (PLR) probability

Codec type , I/B/P/S, Quantizer information, bandwidth

V-Factor

Human Vision System (HVS)

En

tro

py

En

tro

py Γ

Patented

Au

dio

Au

dio Sound

MOS

TR 101 290

Co

nte

nt

Imp

air

me

nts

Co

nte

nt

Imp

air

me

ntsPLC (FEC)

De

Jit

ter

Bu

ffe

rD

e J

itte

r B

uff

er

Network Jitter

Buffer size

Network impairments

Codec impairments

MPQM

V-Factor Measurements

Transport Stream analysis (PCR, Out

of sequence)

V-Factor Parameters

All parameters are “per sample” counters unless otherwise notedPlease see MPEG-2 System Specifications and ETSI TR101290 for more detailed explanations of these parameters

Network related parameters Jitter Discards – Number of frames discarded due to jitter Network Loss Probability – Statistically accurate predictive calculation of frame loss at the PES

level Out of Sequence – Number of miss-ordered frames delivered Max Loss Episode Length – Maximum number of lost frames per episode Max Loss Episodes – Cumulative count of loss episodes since first observation In Sequence – Number of properly ordered frames delivered PLC – packet loss concealment (such as FEC or other mechanisms) ETSI TR 101 290 group 1 and 2 – standard DVB parameters for MPEG2 TS

Content related parameters Program Rate – Transport stream rate in kilobits per second as observed Program Clock Rate Overall Jitter – Jitter of synchronization stream in microseconds (from the

streamer) I frame Count – Number of “I” (Intra) frames/slices for this sample period P frame Count – Number of “P” (Predictive) frames/slices for this sample period B frame Count – Number of “B” (Bi-directional) frames/slices for this sample period Compression Ratio – Ratio of I frames to I+P+B frames QUANT_SCALE – Worst case value of the QUANT_SCALE per reporting period V-Factor – QoSmetrics specific calculation of user perceived video quality, details elsewhere Sound codec Video codec (H264/CABAC – MPEG2 ,…) Multicast Join Time – Actual time the stream was joined in “epoch” seconds Multicast First Time – Actual time the first data arrived in “epoch” seconds

IP TV – QoE metric

Test 1 - 0% packet loss / 0% jitter / 0% dup Test 3 - 1% packet loss / higher jitter / 1% dup

Test 2 - 0,1% packet loss / some jitter / 0,1% dup

Test 1 Test 2 Test 3

Deploying the V-Factor

Secure revenues by keeping customer satisfaction

Integrate with NOC & customer support workflows

Accelerate and secure IPTV deployment during rollout Keep customer support costs under control

By deploying probes before and after encryption, the solution overcomes most situation when encryption is used for DRM purpose.

IPTV in Service Monitoring Measure Performance by Channel and by Geographic Region

"One of the major demands in the emerging IP video market was the need for a video scoring metric suitable for the IPTV market. With the announcement of V-Factor™, which is the first product suitable for large-scale deployment of IPTV, QoSmetrics eliminated one of the major stumbling blocks for successful deployment."

“QoSmetrics’ V-Factor™ is a sophisticated service analysis portfolio that maximizes video quality, giving service providers what they need to avoid network performance impairments – setting the company apart from its competitors,” said Brian Dolan, editor of Fierce-IPTV. “The company’s customer list is very impressive, and will only get better as U.S. deployments continue, making them a Fierce15 winner.”

Awards & Recognition

The NetAgent product has been named a Best of Interop Finalist for this week’s Networld+Interop exhibition held annually in Las Vegas.

Who is using V-Factor

ITPV Customers/Trials OEM/Distribution Partners

Other alternatives