_2d6471bfe6baaa1adc53e3f8e5668699_c5m2

7/26/2019 _2d6471bfe6baaa1adc53e3f8e5668699_C5M2

1/52

Internet of Things:Multimedia Technologies

Harinath Garudadri and Gan

Chockalingam

Qualcomm Institute of Calit2

University of California, San Die


2/52

Lesson 1 Multimedia Processing

Lesson 2 Novel Sensing

Module Introduction


3/52

Multimedia Processing

Lesson 1


4/52

1 A Generic Model

2 Audio Processing

Lesson 1 |Multimedia Processing

3 Image Processing

4 Video Processing

5 Quality Evaluation


5/52

1 - A Generic Model


6/52

Multimedia data rates and Internet servic

ide!and Audio " 1# $%& ' 1( !its)sam*le + 2 , Audio " ((.1 $%& ' 1# !its)sam*le ' / c0an1!41 Mbs

Still Images " 1/(21/( image ' 3 !its eac0 4and 7 + 3!14 M"#tes

5a8 Video " 1/329/ ' /( !its)*i2el : /( 4*sMbs

;0ese rates are not com*ati!le 8it0 t0e Internet se

e need to com*ress data.

1 |A Generic Model


7/52

% &eneric 'omression "lock (iagra

1 |A Generic Model


8/52

The )irit of *rocessing at +ncoder

Time,-re.uenc# (ecomosition Identi4y Statistical Redundancy in signals E2*loit Perceptual Irrelevancy< re=uires *rior $no8

/uanti0ation

>ossy !itrate reduction < 4e8er !its to less relevantcom*onents

+ntro# 'oding >oss-less !itrate reduction < 4e8er !its to 4re=uent

1 |A Generic Model


9/52

Time,-re.uenc# (ecomosition

S0ort-time ?ourier ;rans4orm

0tt*s@))en.8i$i*edia.org)8i$i)S0ort-time?ouriertrans4orm Ga!or ;rans4orm 0tt*s@))en.8i$i*edia.org)8i$i)Ga!ortrans4orm avelet ;rans4orm 0tt*s@))en.8i$i*edia.org)8i$i)avelettrans4orm Modi4ied iscrete ,osine ;rans4orm

0tt*s@))en.8i$i*edia.org)8i$i)Modi4ieddiscretecosinetrans4orm

1 |A Generic Model
https://en.wikipedia.org/wiki/Short-time_Fourier_transformhttps://en.wikipedia.org/wiki/Gabor_transformhttps://en.wikipedia.org/wiki/Wavelet_transformhttps://en.wikipedia.org/wiki/Modified_discrete_cosine_transformhttps://en.wikipedia.org/wiki/Modified_discrete_cosine_transformhttps://en.wikipedia.org/wiki/Wavelet_transformhttps://en.wikipedia.org/wiki/Gabor_transformhttps://en.wikipedia.org/wiki/Short-time_Fourier_transform


10/52

+ntro# 'oding

%u44man ,oding 0tt*s@))en.8i$i*edia.org)8i$i)%u44mancoding

Arit0metic coding 0tt*s@))en.8i$i*edia.org)8i$i)Arit0meticcoding

)#mbol

a1

a/ 1

aB 11

a( 111

1 |A Generic Model
https://en.wikipedia.org/wiki/Huffman_codinghttps://en.wikipedia.org/wiki/Arithmetic_codinghttps://en.wikipedia.org/wiki/Arithmetic_codinghttps://en.wikipedia.org/wiki/Huffman_coding


11/52

/ - Audio Processing


12/52

?astl and C8ic$er6 DPsyc0oacoustics < ?acts and Models6 Brdedition /9

2 |Audio Processing


13/52

The Miracle 13d" demo "randenburg and im ohnst

1$$ d" )78 7oise is 1$$$$$ times belo9 th

signal13 d" )78 7oise is 2$ times belo9 the sign

A. Friginal Signal

7. Friginal 0ite noise : -1B d7

,. Friginal S0a*ed noise : -1B d7

. i44erence o4 A and ,

2 |Audio Processing


14/52

?astl and C8ic$er6 DPsyc0oacoustics < ?acts and Models6 Brdedition /

2 |Audio Processing


15/52

%ig0 5esolution;-?

decom*osition

Psyc0oacousticModel

2 |Audio Processing


16/52

B - Image Processing


17/52

)till Images

HPEG < Hoint P0otogra*0ic E2*erts Grou*6 a oGrou* GJ in ISF)IE,

' International Standardi&ation Frgani&ation ISF' International Electrotec0nical ,ommission IE,

?or a good revie8 0tt*s@))en.8i$i*edia.org)8i$i)H

3 |Image Processing
https://en.wikipedia.org/wiki/JPEGhttps://en.wikipedia.org/wiki/JPEG


18/52

3 |Image Processing


19/52

KLV is same as K,7,5

K lumaJ is gamma corrected >uminance6 !ut O is not usu

0tt*s@))msdn.microso4t.com)en-us)li!rary)8indo8s)des$to*)!!B1

3 |Image Processing


20/52

Visuali&ation o4 323 ,; !asis 4unctions ;0e &ig-&ag scanning

3 |Image Processing


21/52

3 |Image Processing

3 |


22/52

3 |Image Processing


23/52

( - Video Processing

4 | Vid P i


24/52

MotionEstimation

Predicted ?ramemotion

com*ensatedJ Motion

in4ormationP and 74ramesJ

4 |Video Processing

4 | Vid P i


25/52

0tt*@))8e!.stan4ord.edu)class)eeB3a)0andouts)lectures)EEB3aMotionEstimatio

4 |Video Processing

4 | Vid P i


26/52

4 |Video Processing

4 | Video Processing


27/52

4 |Video Processing

4 | Video Processing


28/52

4 |Video Processing


29/52

- Quality Evaluation

5 | Quality Evaluation


30/52

Multimedia data rates and Internet service

ide!and Audio " 1# $%& ' 1( !its)sam*le + 22

; 12 kbs , Audio " ((.1 $%& ' 1# !its)sam*le ' / c0ann

Mbs


31/52

)ub>ective +valuation of /ualit#%"? Test A is original 7 is encoded and decoded Is R close to A or 7

%"'@A8 Test A and 7 are original and encoded)decoded in random order , is al8ays original Identi4y *rocessed signal and assign su!Tective =uality score

MB)A8% MLlti*le Stimuli 8it0 %idden 5e4erence and Anc0or 5e4erence is $no8n tests include a 0idden version o4 re4erence a

anc0ors o4 $no8n assessedJ =uality

5 |Quality Evaluation

5 | Quality Evaluation


32/52

Cb>ective +valuation of /ualit#

*+%/ *ercetual +valuation of %udio /ua

5 |Quality Evaluation


33/52

Novel Sensing

Lesson 2

Lesson 2 | Novel Sensing


34/52

Lesson 1 Pi2el ,amera

Lesson 2 ?all 5is$ Assessment

Lesson 3 An F!Tective 7alance ;est

Lesson 2 |Novel Sensing


35/52

1 - Pi2el ,amera

1 | Pi2el ,amera


36/52

htt:@@inilabs!com@roducts@d#namic,vision,sensors@

1 |Pi2el ,amera

1 | Pi2el ,amera


37/52

!

1 |Pi2el ,amera

VIEFU P>EASE


38/52

/ - ?all 5is$ Assessment

2 |?all 5is$ Assessment


39/52

'(' +stimates for 'ost of -alls in B

B( 7illion /1BJ

7illion //J

| a s ssess e t



40/52

Ina!ility to Maintain Postural 7al

W

Greater 5is$ o4 ?alls

"2#"$% %&

|



41/52

|



42/52

|



43/52

|


44/52

B - An F!Tective 7alance ;est

3 |An F!Tective 7alance ;est


45/52

,urrent DGold Standards

Intervie8 andF!servational

Su!Tective and non-re*eata!le

Gait >a!s andPlates

F!Tective6 !ut e2*



46/52

,amera 7ased Assessment

X Integrates 8it0 current 8or$ 4lo8s

X ynamic Vision Sensor < *i2el c0anges at 1 Ys

X ,onsidered similar to visual corte2 V1



47/52

F44 7ody Sensing@ ynamic Vision Sensor VSJ,amera

-igure 2:Images 4rom ynamic Vision Sensor system during a !alanc

%!7alci et! al! D% 'omuter Eision "ased 'andidate for -unctional "alance TestF I+++ +M"' %

Ital#!

VIEFUP>EASE



48/52



49/52



50/52

5 |Multimedia ;ec0nologies


51/52

-urther )tud#


52/52

"ack u slides

_2d6471bfe6baaa1adc53e3f8e5668699_c5m2

Documents