Download - Takács György 12. előadás 2009. 05. 13
![Page 1: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/1.jpg)
T.Gy. Intrernetes médiakommunikáció. 2009.05.
13.
1
Internetes médiakommunikáció
Az Internetes Médiakommunikáció minőségének megítélése, mérése, a
rendszer tervezési alapjai
Takács György 12. előadás2009. 05. 13.
![Page 2: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/2.jpg)
What is QoE?
The overall acceptability of an application or service, as perceived subjectively by the end user.
NOTES1 Quality of Experience includes the complete end-
to-end system effects (client, terminal, network, services infrastructure, etc).
2 Overall acceptability may be influenced by user expectations and context. ~From ITU-T Recommendation P.10/G.100~
![Page 3: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/3.jpg)
![Page 4: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/4.jpg)
![Page 5: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/5.jpg)
Factors affecting QoE of IPTV services
![Page 6: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/6.jpg)
![Page 7: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/7.jpg)
Why QoE evaluation?
Recent survey indicates that…. • 84% of IPTV providers think video quality monitoring is
critical or very important• 77% say video quality is a main reason for customer
churn• 78% say video quality problems are a main reason for
high volumes of customer service calls~From http://telephonyonline.com/iptv/news/iptv-video-
quality-0219/~
There surely is a market need for QoE evaluation methods/tools!
![Page 8: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/8.jpg)
Quality monitoring as well as quality design
Conventional telephone services over PSTN are relatively stable after a call is connected.
• In PSTN services, service providers established telephone systems
• with static quality design, then, monitor/manage the “availability” and “functionality”
IP networks (core/access/home-NWs) and IP-based terminals are time-variant systems.
• In IPTV services, media quality varies depending on coding performance at H/E, traffic conditions in networks, and terminal performance
![Page 9: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/9.jpg)
What to measure?Pre-transmission domain• Encode quality• System failure (blackout, frame freezing etc.)Network domain• • Packet loss• • Delay• • Delay jitterPost-transmission domain• Error recovery performance such as FEC (Forward Error
Correction)• Decode quality (including PLC: Packet Loss Concealment)• D/A conversion performance• Color management performance• Display performance
![Page 10: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/10.jpg)
When to measure?
QoE measurement is necessary in the following phases:
Before the service is developed:• Individual quality factors (coding distortion etc.)• Quality planningAfter the service is developed, but before the service is
delivered:• Off-service quality checkAfter the service is delivered:• In-service quality monitoring at user end• In-service network mid-point monitoring• On-site quality check (on claim)
![Page 11: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/11.jpg)
Where to measure?
![Page 12: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/12.jpg)
How to measure?
• Subjective quality assessment is time consuming and expensive.
• It cannot be applied to on-site and/or real-time measurement, which is essential in quality monitoring.
• Objective quality assessment, which estimates QoE by subjective testing, is the solution!
![Page 13: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/13.jpg)
Overview of objective quality assessment models
From the viewpoint of input information into a model, objective quality assessment models can be categorized as follows:
• Media-layer models• Parametric packet-layer models• Bitstream-layer models• Parametric planning models• Hybrid models
![Page 14: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/14.jpg)
Media-layer models
![Page 15: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/15.jpg)
Current standards of media-layer models (video)
• ITU-T standardized Rec. J.144, which recommends four fullreference models for Standard Definition TV, in 2004.– Coding distortion only (no transmission errors such as
packet loss)
• VQEG finalized its evaluation on full-reference and reducedreference models applicable to QCIF ~ VGA. The draft Recommendation has been consented at ITU-T SG9 as Rec. J.247, which includes four models, and Rec. J.246, respectively in May 2008.– Packet-loss degradation as well as coding distortion is in
the scope of Recommendation.
• ITU-T SG9 considered that current no-reference technologies seemed to be premature in terms of quality-estimation accuracy.
![Page 16: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/16.jpg)
Parametric packet-layer models
• Only uses packet-header information.• Light-weight implementation.• Can be applied even if payload is encrypted.
![Page 17: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/17.jpg)
Bitstream-layer models
• Use up to coded bitstream (payload), but not decoder output.
• Can take into account the content dependence of the impact of packet loss.
![Page 18: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/18.jpg)
Standardization activities for packet-layer andbitstream-layer models
Parametric packet-layer models• ITU-T SG12 Q.14 & VQEG• • P.NAMS (non-intrusive parametric model
for the assessment of performance of multimedia streaming)
Bitstream-layer models• ITU-T SG12 Q.14 & VQEG• • P.NBAMS (non-intrusive parametric
bitstream model for the assessment of performance of multimedia streaming)
![Page 19: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/19.jpg)
Parametric planning models
• Use quality planning parameters as input.• ITU-T Recommendation G.1070 for videophone applications.• ITU-T SG12 Q.13 is working on G.OMVAS (opinion model for video
and audio streaming applications).
![Page 20: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/20.jpg)
Present state of multimedia qualityassessment methods
Future trends of quality assessmentresearch
• Multimodality
• Multiparty
• Wideband
• Examples of NTT’s studies on multimedia quality assessment for audiovisual communication services
![Page 21: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/21.jpg)
Relevant ITU-T/R Recommendationson Multimedia Quality Assessment
• Multimedia quality assessment is at a reasonably advanced stage.
![Page 22: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/22.jpg)
Existing Subjective Multimedia QualityAssessment Methods (P.911)
Multimedia quality is assessed in a similar way to individual audio/video qualities.
• Absolute Category Rating (ACR)• Degradation Category Rating (DCR)• Pair Comparison Method (PC)• Single Stimulus Continuous Quality• Evaluation (SSCQE)Assessment paying attention to the cross-modal
influences is important.• Interactions between differing quality levels in
different modalities
![Page 23: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/23.jpg)
![Page 24: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/24.jpg)
![Page 25: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/25.jpg)
Existing Subjective Multimedia QualityAssessment Methods (P.920)
Several category judgment scales are used to evaluate multimedia quality.
• Overall audiovisual quality• Individual audio/video qualities• Effort needed to interrupt• Communication difficulty• Acceptability of communicationCommunication quality depends on tasks used in
conversational test.Assessment considering interactivity and usability
is also important.
![Page 26: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/26.jpg)
Existing Concept Model of ObjectiveMultimedia Quality Evaluation (J.148)
Basic components of an objective multimedia quality model are defined.
![Page 27: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/27.jpg)
Examples of Multimedia QualityEvaluation Model
![Page 28: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/28.jpg)
Framework for ConductingQuality Assessment Research
![Page 29: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/29.jpg)
Key Words for Future Trends ofMultimedia Quality Assessment (1/3)
Multimodality
• Combination of multiple media such as audio, video, text, graphics, fax, and telephony in the communication of information
• Key points of quality assessment
√ Intermedia synchronization
√ Intermedia quality balance
![Page 30: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/30.jpg)
Key Words for Future Trends ofMultimedia Quality Assessment (2/3)
• Multiparty
• Communication style extending from 1:1 to N:N (e.g., instant messaging, teleconferencing, and distributed collaboration services)
• Key points of quality assessment
√ Interdestination synchronization
√ Interdestination quality balance
![Page 31: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/31.jpg)
Key Words for Future Trends ofMultimedia Quality Assessment (3/3)
• Wideband• Telecommunications applications having more
bandwidth available for higher-quality multimodal services
• Key points of quality assessment
√ Psychological factors: We need to assess the richness of high-quality services not only on a one-dimensional scale, like MOS, but also on a multi-dimensional scale.
![Page 32: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/32.jpg)
Examples of NTT’s Studies on QualityAssessment for “Multimodality” (1/3)
![Page 33: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/33.jpg)
Examples of NTT’s Studies on QualityAssessment for “Multimodality” (2/3)
![Page 34: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/34.jpg)
Examples of NTT’s Studies on QualityAssessment for “Multimodality” (3/3)
![Page 35: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/35.jpg)
Examples of NTT’s Studies on QualityAssessment for “Multiparty” (1/2)
Quality imbalance is one of the multiparty quality degradation factors.
![Page 36: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/36.jpg)
Examples of NTT’s Studies on QualityAssessment for “Multiparty” (2/2)
• Overall quality is strongly affected by the inferior quality at another point.
• Other-point audiovisual quality depends on conversation task or roles.
![Page 37: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/37.jpg)
Examples of NTT’s Studies on QualityAssessment for “Wideband” (1/3)
• Interactive multimodal quality can be evaluated using a multi-dimensional scale of psychological factors.
![Page 38: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/38.jpg)
Examples of NTT’s Studies on QualityAssessment for “Wideband” (2/3)
Psychological factors were extracted by using the semantic differential (SD) technique and factor analysis.
• Subject’s impression of an audiovisual communication service was evaluated on the basis of 25 pairs of bipolar adjectives on a seven-grade comparison scale.
![Page 39: Takács György 12. előadás 2009. 05. 13](https://reader035.vdocument.in/reader035/viewer/2022062518/56813f58550346895daa2513/html5/thumbnails/39.jpg)
Examples of NTT’s Studies on QualityAssessment for “Wideband” (3/3)
• Multimedia quality was formulated as a function of two psychological factors expressing an aesthetic feeling and a feeling of activity.