quality of experience prediction for streaming videozduanmu/icme16qoe/paper/... · video streaming...
TRANSCRIPT
![Page 1: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/1.jpg)
QUALITY OF EXPERIENCE PREDICTIONFOR STREAMING VIDEO
Zhengfang Duanmu
Joint work with Abdul Rehman, Kai Zeng, and Zhou Wang
July 13, 2016
![Page 2: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/2.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Outline
1 Video Streaming and Quality of ExperienceVideo StreamingQuality of ExperienceObjective QoE Prediction
2 Streaming Quality Index (SQI)Objective QoE Prediction ModelExperimental Results
3 Conclusions
2 / 26
![Page 3: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/3.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Outline
1 Video Streaming and Quality of ExperienceVideo StreamingQuality of ExperienceObjective QoE Prediction
2 Streaming Quality Index (SQI)Objective QoE Prediction ModelExperimental Results
3 Conclusions
3 / 26
![Page 4: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/4.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
The Age of Streaming
4 / 26
![Page 5: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/5.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
The Age of Streaming
Factors of Picking Streaming ServiceQuality of Experience;
Content;
Price;
Advertisement.
5 / 26
![Page 6: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/6.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Quality of Experience
DefinitionThe degree of delight or annoyance of the user of an application orservice. [Callet, 2013]
6 / 26
![Page 7: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/7.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Quality of Experience
Influencing FactorsPlayback smoothness
7 / 26
![Page 8: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/8.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Quality of Experience
Influencing FactorsDuration of initial buffering
8 / 26
![Page 9: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/9.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Quality of Experience
Influencing FactorsVideo presentation quality
9 / 26
![Page 10: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/10.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Interaction between Presentation Quality and StallingExperience
0 20 40 60 80 1000
5
10
15
20
25
30
35
SSIMplus of stalling frames
MO
S dr
op
initial bufferingfitted initial bufferingstallingfitted stalling
Figure: SRCC = 0.79
10 / 26
![Page 11: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/11.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Subjective QoE Prediction
Subjective QoE Assessment
Enormous video space
11 / 26
![Page 12: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/12.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Existing QoE models: Quality of Service-based
PhilosophyThere exists a causal relationship between generic QoS problems andgeneric QoE problems.
FactorsThroughput -> Delivered video quality
Stalling duration -> Waiting experience
Existing ModelsLinear mapping [Mok, 2011];
Exponential mapping [Hoßfeld, 2012];
Logrithmic mapping [Rodriguez, 2012];
12 / 26
![Page 13: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/13.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Existing QoE models: Signal Fidelity-based
PhilosophyQoE can be measured by the distance from test video to the prestinevideo in the video space.
Existing ModelsPSNR;
SSIM [Wang, 2004];
MS-SSIM [Wang, 2003];
VQM [Pinson, 2004];
SSIMplus [Rehman, 2015];
13 / 26
![Page 14: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/14.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Video StreamingQuality of ExperienceObjective QoE Prediction
Our method: Hybrid
MotivationQoS-based: not directly related to human perception;
Signal fidelity-based: only work for static videos;
No modeling on the interaction between video presentation quality andstalling.
14 / 26
![Page 15: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/15.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Objective QoE Prediction ModelExperimental Results
Outline
1 Video Streaming and Quality of ExperienceVideo StreamingQuality of ExperienceObjective QoE Prediction
2 Streaming Quality Index (SQI)Objective QoE Prediction ModelExperimental Results
3 Conclusions
15 / 26
![Page 16: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/16.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Objective QoE Prediction ModelExperimental Results
Presentation Quality
Source
Bitrate 1
Bitrate 2
Bitrate 3
V
Q
A
S
E
G
M
E
N
T
E
R
Seg1 Seg2 Seg3
Seg1 Seg2 Seg3
Seg3Seg1 Seg2
T
R
A
N
S
C
O
D
I
N
G
16 / 26
![Page 17: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/17.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Objective QoE Prediction ModelExperimental Results
Presentation Quality
0 500 1000 150075
80
85
90
95
100
Frame #
Stat
ic v
ideo
qua
lity
Static video qualityStalling position
0 500 1000 150075
80
85
90
95
100
Frame # with stallings
Stre
amin
g vi
deo
qual
ity (
P)
PStalling positionRecovery position
17 / 26
![Page 18: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/18.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Objective QoE Prediction ModelExperimental Results
Stalling Experience Quantification
Stalling Experience Quantification
Sk(t) =
Pik−1
(−1 + exp
{−(
tf − ikT0
)})ikf ≤ t ≤ ik+lk
f
Pik−1
(−1 + exp
{−(
lkT0
)})·(
exp{−(
tf − ik − lkT1
)})t > ik+lk
f
0 otherwise
(1)
18 / 26
![Page 19: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/19.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Objective QoE Prediction ModelExperimental Results
Stalling Experience Quantification
Parameters
Parameter DescriptionT0 rate of dissatisfaction in stalling eventT1 strength of memory in stalling event
T init0 rate of dissatisfaction in initial buffering event
T init1 strength of memory in initial buffering eventP0 expectation on initial quality of the video
19 / 26
![Page 20: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/20.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Objective QoE Prediction ModelExperimental Results
Presentation Quality
0 500 1000 150075
80
85
90
95
100
Frame #
Stat
ic v
ideo
qua
lity
Static video qualityStalling position
0 500 1000 150075
80
85
90
95
100
Frame # with stallings
Stre
amin
g vi
deo
qual
ity (
P)
PStalling positionRecovery position
0 500 1000 1500−100
−80
−60
−40
−20
0
Frame # with stallings
Stal
ling
expe
rien
ce (
S nk )
Sn1
Sn2
Stalling positionRecovery position
0 500 1000 15000
20
40
60
80
100
Frame # with stallings
QoE
(Q
)
QStalling positionRecovery position
20 / 26
![Page 21: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/21.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Objective QoE Prediction ModelExperimental Results
Experimental Results
Performance comparison of QoE models on streaming video QoEdatabase.
PLCC MAE SRCC KRCCFTW [Hoßfeld, 2012] 0.3313 14.9455 0.3154 0.2583
PSNR 0.6663 10.7254 0.6715 0.4697SSIM [Wang, 2004] 0.8432 7.6039 0.8177 0.6070
SSIMplus [Rehman, 2015] 0.8350 7.6934 0.8024 0.5924SQI-PSNR 0.7391 9.8445 0.7492 0.5434SQI-SSIM 0.9015 5.8941 0.9009 0.7238
SQI-SSIMplus 0.9026 5.8330 0.9007 0.7213
21 / 26
![Page 22: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/22.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Objective QoE Prediction ModelExperimental Results
Experimental Results
40 60 80 1000
20
40
60
80
100
FTW
MO
S
no stallinginitial bufferingstalling
0.85 0.9 0.95 10
20
40
60
80
100
SSIM
MO
S
no stallinginitial bufferingstalling
20 40 60 80 1000
20
40
60
80
100
SSIMplus
MO
S
no stallinginitial bufferingstalling
0.5 0.6 0.7 0.8 0.9 10
20
40
60
80
100
VsQM
MO
S
no stallinginitial bufferingstalling
0.8 0.85 0.9 0.95 10
20
40
60
80
100
SQI−SSIM
MO
S
no stallinginitial bufferingstalling
0 20 40 60 80 1000
20
40
60
80
100
SQI−SSIMplusM
OS
no stallinginitial bufferingstalling
22 / 26
![Page 23: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/23.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Outline
1 Video Streaming and Quality of ExperienceVideo StreamingQuality of ExperienceObjective QoE Prediction
2 Streaming Quality Index (SQI)Objective QoE Prediction ModelExperimental Results
3 Conclusions
23 / 26
![Page 24: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/24.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Conclusions
ContributionProposed an objective QoE model for video streaming that considerspresentation quality and its interaction with stalling;
Achieved the best performance in predicting subject opinions.
Future WorkConstruct comprehensive database;
Investigate other QoE-related factors;
Improve the QoE model.
24 / 26
![Page 25: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/25.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
References
P. L. Callet, S. MÃuller, and A. Perkis “Qualinet White Paper on Definitions of Quality ofExperience”, European Network on Quality of Experience in Multimedia Systems and Services,2013.
R. Mok, E. Chan, and R. Chang, “Measureing the quality of experience of HTTP video streaming,”IFIP/IEEE Int. Sym. Integrated Network Management, 2011.
T. Hoßfeld, S. Egger, R. Schatz, K. Masuch, and C. Lorentzen, “Initial delay vs. interruption:between the devil and the deep blue sea,” IEEE QoMEX, 2012.
D. Rodriguez, J. Abrahao, D. Begazo, R. Rosa, G. Bressan, “Quality metric to assess videostreaming service over TCP considering temporal location of pauses,” IEEE TCE, 2012.
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From errorvisibility to structural similarity,” IEEE TIP, 2004.
Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image qualityassessment,” IEEE Asilomar, 2003.
M. H. Pinson, S. Wolf, “A new standardized method for objectively measuring video quality,” IEEETrans. Broadcasting, 2004.
A. Rehman, K. Zeng, and Z. Wang, “Display device-adapted video quality-of-experienceassessment,” SPIE Electronic Imaging, 2015.
25 / 26
![Page 26: QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE](https://reader033.vdocument.in/reader033/viewer/2022050414/5f8ad502a2dea24a6155358b/html5/thumbnails/26.jpg)
Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)
Conclusions
Thank you
26 / 26