0#12% - massachusetts institute of technologypeople.csail.mit.edu/khosla/slides/ ·...
TRANSCRIPT
!"#$%&#'()%#*%+&#,(%-.-/0#12%
34+$5#%6".)0#% 37)"%8#)%9#1&#% :#;#5%<#&+4%
=#*%>(%-1(4+?$%>"#$%>+00%@(?.&(%-.-/0#12%
!"#$%#$&'()$*(+,-"#&
!"#$%&'($)*$+,-'./0
!"#$%&'($)*$+,-'./0
.)%/0%&)%/%!-"#1&
!"#$%&'($)*$+,-'./0
2'3%*-)(#4&
!"#$%&'($)*$+,-'./0
!"#$%#$&'()$*(+,-"#&
.)%/0%&)%/%!-"#1&
2'3%*-)(#4& 3('%"&),552*(62-"#&
5"'(78(#4&9"9,/2*($8&
,#'%*)$2#'(#4&,)%*&+%:23("*&
Related work
• Social networks - Gladwell, 2002 - Rogers, 2003 - Romero et al, 2011
• Video popularity - Pinto et al, 2013 - Shamma et al, 2011 - Nwana et al, 2013
• High-‐level aJributes - Dhar et al, 2011 - Isola et al, 2011 - PaJerson et al, 2012 - Khosla et al, 2013
• Text popularity - Petrovic et al, 2011 - Hong et al, 2011
Overview
• What is image popularity?
• Predict popularity using image content?
• How does this compare to social factors?
• What makes an image popular?
• Conclusions and future work
Overview
• What is image popularity?
• Predict popularity using image content?
• How does this compare to social factors?
• What makes an image popular?
• Conclusions and future work
!",*$)($)+,7'$8&892,.)*#/0
!",*$)($)+,7'$8&892,.)*#/0
!",*$)($)+,7'$8&892,.)*#/0
!",*$)($)+,7'$8&892,.)*#/0
What is image popularity?
• Context dependent
Facebook newsfeed
Flickr search results
Flickr user
:&892,.)*#$%,*,('*0
3(%<)&
K2$2)%$T&@UC&5(//("#&S/(!=*&(524%)&
V)%*I;9%!(0!&%U4U&(#'(3(',2/&,)%*)&
:&892,.)*#$;&<*'=*(0
W#%I9%*I,)%*&%U4U&S/(!=*&)%2*!:&*%),/$)&
V)%*I5(O&%U4U&S2!%+""=&#%<)7%%'&
Popularity metric
Popularity metric
Popularity metric
Task: predict log normalized views
Measure: rank correla7on (ρ) Szabo and Huberman, 2010
Overview
• What is image popularity?
• Predict popularity using image content?
• How does this compare to social factors?
• What makes an image popular?
• Conclusions and future work
:.'%);><7$8&892,.)*#$9()<7$)+,7'$;&<*'<*0
1.2 2.7
0.7 4.5
3.6 5.3
X&
$*2(#(#4&(524%)&
7&
?
@UC&
:.'%);><7$8&892,.)*#$9()<7$)+,7'$;&<*'<*0
•! ;(59/%&(524%&7%2$,*%T&(524%&(#$%#)($8&
norm
aliz
ed v
iew
s
intensity mean 0 0.5 1 0
9
18
1.2
AUDC&
:.'%);><7$8&892,.)*#$9()<7$)+,7'$;&<*'<*0
•! ;(59/%&(524%&7%2$,*%T&(524%&(#$%#)($8&
norm
aliz
ed v
iew
s
intensity mean 0 0.5 1 0
9
18
1.2
AUDC&
3.6
AUYC&
X&
:.'%);><7$8&892,.)*#$9()<7$)+,7'$;&<*'<*0
•! ;(59/%&(524%&7%2$,*%T&(524%&(#$%#)($8&
1.2
AUDC&
3.6
AUYC&
X&rank corr = -0.00
intensity mean 0 0.5 1 0
9
18
norm
aliz
ed v
iew
s
:.'%);><7$8&892,.)*#$9()<7$)+,7'$;&<*'<*0
•! ;(59/%&(524%&7%2$,*%T&(524%&(#$%#)($8&
rank corr = -0.00 rank corr = 0.01 rank corr = -0.02
norm
aliz
ed v
iew
s
norm
aliz
ed v
iew
s
norm
aliz
ed v
iew
s
intensity mean intensity variance intensity skewness 0 0.5 1 0
9
18
0 0.125 0.25 0
9
18
-5
8
20
-50 50 150
:.'%);><7$8&892,.)*#$9()<7$)+,7'$;&<*'<*0
•! ;(59/%&(524%&7%2$,*%)T&H;E&!"/"*&)92!%&
Predic>ng popularity using image content
• Simple image features: HSV color space
rank corr = -0.05
norm
aliz
ed v
iew
s
mean hue
norm
aliz
ed v
iew
s
norm
aliz
ed v
iew
s
rank corr = 0.02 rank corr = 0.01
mean saturation mean value 0 0.5 1 0
10
20
0 0.5 1 0
10
20
0 0.5 1 0
10
20
:.'%);><7$8&892,.)*#$9()<7$)+,7'$;&<*'<*0
I*-/$%%I&#,(%
<JK%KI9A% 9ILA%=.&-/$(1%%D+)+.*%M(#$/1()%
9/--.1$%N(?$.1%
:(,1())+.*%
DUZ& I&#,(%-.-/0#1+$5%
:.'%);><7$8&892,.)*#$9()<7$)+,7'$;&<*'<*0
>*2'(%#$&%U4U&HW>&[K2/2/&@AAY\&
>()$&%U4U&>L;N&[W/(32&@AAD\&
N%O$,*%&%U4U&]^F&[W_2/2&@AA@\&
R"/"*&%U4U&[32#&'%&P%(_%*&@AAZ\&
K%%9&/%2*#(#4&%U4U&[M*(6:%3)=8&@AD@\&
DAAA&W+_%!$)&%U4U&[M*(6:%3)=8&@AD@\&
2(*9/2#%&+(!8!/%&+(*'&+"2$&+"J/%&"*2#4%&+%#!:&
X&9%*)"#&):%%9&$*2(#&$%/%3()("#&'"4&7"O&
*%5"$%&!2*&)$"3%&$2+/%&299/%&!2*$&*28&
Predic>ng popularity using image content
0
0.1
0.2
0.3
0.4
User-‐specific
rank correla-o
n
Predic>ng popularity using image content
0
0.1
0.2
0.3
0.4
One-‐per-‐user
User-‐mix
User-‐specific
rank correla-o
n
Predic>ng popularity using image content
Predic>ng popularity using image content
Overview
• What is image popularity?
• Predict popularity using image content?
• How does this compare to social factors?
• What makes an image popular?
• Conclusions and future work
Predic>ng popularity using social cues
• User informa-on • Mean views • Photo count • Contacts • Groups • Group members • Member dura-on • Is pro
• Image informa-on • Tags • Title length • Descrip-on length
Predic>ng popularity using social cues
-‐0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
One-‐per-‐user
User-‐mix
User-‐specific
rank correla-o
n
Predic>ng popularity using image + social cues
0
0.2
0.4
0.6
Content only Social only Combined
One-‐per-‐user
User-‐mix
User-‐specific
rank correla-o
n
Predic>ng popularity using image + social cues
0
0.2
0.4
0.6
0.8
1
Content only Social only Combined
One-‐per-‐user
User-‐mix
User-‐specific
rank correla-o
n
Predic>ng popularity using image + social cues
0
0.2
0.4
0.6
0.8
1
Content only Social only Combined
One-‐per-‐user
User-‐mix
User-‐specific
rank correla-o
n
Overview
• What is image popularity?
• Predict popularity using image content?
• How does this compare to social factors?
• What makes an image popular?
• Conclusions and future work
!",*$+,4'($,<$)+,7'$8&892,./0
32#&'%&P%(_%*&%$&2/?&REFB&@AAc&
R"/"*&9
%*!%#$24
%&
!",*$+,4'($,<$)+,7'$8&892,./0
I*-/$%%I&#,(%
=.0.1%"+)$.,1#&%
9/--.1$%N(?$.1%
:(,1())+.*%
DUZ& I&#,(%-.-/0#1+$5%
What makes an image popular?
colors
colo
r im
port
ance
!",*$+,4'($,<$)+,7'$8&892,./0
•!F*%'(!-#4&"+_%!$&!2$%4"*(%)&
M*(6:%3)=8&%$&2/?&GLF;&@AD@&
L524%& W+_%!$&&!2$%4"*8&
What makes an image popular?
• tench, Tinca -nca • goldfish, Carassius auratus • great white shark • -ger shark, Galeocerdo cuvieri • hammerhead, hammerhead shark
• electric ray, crampfish, numbfish, torpedo
• s-ngray • cock • hen • ostrich, Struthio camelus
• brambling, Fringilla mon-fringilla
• goldfinch, Carduelis carduelis • house finch, linnet, Carpodacus mexicanus
• junco, snowbird • indigo bun-ng • robin, American robin
• bulbul
• jay • magpie
• chickadee • water ouzel, dipper • kite • bald eagle, American eagle, Haliaeetus leucocephalus
• vulture • great grey owl, great gray owl, Strix nebulosa • European fire salamander, Salamandra salamandra
• common newt, Triturus vulgaris
• eg • spoJed salamander, Ambystoma maculatum
• axolotl, mud puppy, Ambystoma mexicanum
• bullfrog, Rana catesbeiana • tree frog, tree-‐frog • tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui • loggerhead, loggerhead turtle, CareJa careJa
!",*$+,4'($,<$)+,7'$8&892,./0
I*-/$%%I&#,(%
J@O(?$%%0+'(0+"..4%
9/--.1$%N(?$.1%
:(,1())+.*%
DUZ& I&#,(%-.-/0#1+$5%
a%'(,5&9")(-3%&(592!$&&
!",*$+,4'($,<$)+,7'$8&892,./0
,+#*$%-#*4#% 0#45@/,% @#)'($@#00%
-0.>% ?"(($#"% 00#&#%
;$*"#4&9")(-3%&(592!$&&
&
!",*$+,4'($,<$)+,7'$8&892,./0
@1#))+(1(% 1(D.0D(1% &+*+)'+1$%
&#+00.$% @+'+*+% ?/-%
G%42-3%&(592!$&&
&
!",*$+,4'($,<$)+,7'$8&892,./0
)-#$/0#% -0/*,(1% 0#-$.-%
!",*$+,4'($,<$)+,7'$8&892,./0
;$*"#4&9")(-3%&&
•! 5(#()=(*$&•! 52(//"$&•! +(=(#(&•! !,9&•! +*2))(%*%&•! *%3"/3%*&
a%'(,5&9")(-3%&&
•! !:%%$2:&•! 4(2#$&92#'2&•! +2)=%$+2//&•! //252&•! 9/"<&•! /2'8+,4&
G%42-3%&&
•! )92$,/2&•! 9/,#4%*&•! /29$"9&•! 4"/7!2*$&•! )92!%&:%2$%*&
Overview
• What is image popularity?
• Predict popularity using image content?
• How does this compare to social factors?
• What makes an image popular?
• Conclusions and future work
Conclusion
• Image popularity can be predicted to a reasonable extent using content alone!
• Both image content and social cues are important for popularity predic-on
• Content/social cues help to a different degree in different contexts
• We iden-fy some factors that make an image popular
Future work
• Predic-ng other no-ons of popularity e.g. shareability, likeability, comments
• Temporal evolu-on of popularity
• Effect of high-‐level aJributes such as emo-ons
• Automa-cally modifying image popularity
hPp://popularity.csail.mit.edu
Thank you!