![Page 1: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/1.jpg)
Music Video Redundancy and Half-Life
in YouTube
Matthias Prellwitz and Michael L. [email protected]
TPDL 2011Berlin, Germany
9/26/11
![Page 2: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/2.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 2
Linking to a particular copy“Rolling Stones - Satisfaction”
![Page 3: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/3.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 3
Metadata lost when YouTube video disappears
video title The Rolling Stones – Satisfactionurl http://www.youtube.com/watch?v=214szPQBUYc
![Page 4: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/4.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 4
Metadata hard to recover fromSearch Engines
![Page 5: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/5.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 5
But nearly 300 copies remain in YouTube
![Page 6: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/6.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 6
Linking music-related URIs
‣ Transparent URI semantics
‣ http://www.last.fm/music/John+Lennon/_/Imagine
‣ http://www.ilike.com/artist/The+Cribs/track/I%27m+A+Realist
‣ http://www.last.fm/music/Johnny+Cash/_/Highwayman‣ Opaque URI semantics
‣ http://vids.myspace.com/index.cfm?fuseaction=vids.individual&videoid=5168491
‣ http://www.youtube.com/watch?v=VST2KKIYn50
![Page 7: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/7.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 7
Popular Music US Top 40 Singles Charts of 9/25/10
![Page 8: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/8.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 8
Popular MusicSelected Music Blogs
![Page 9: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/9.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 9
Popular MusicThe 500 Greatest Songs of all Time
![Page 10: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/10.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 10
Total Result Size RangeUS Top 40 Singles Charts of 9/25/10
123,23983,298
43,945
663526
Lady GagaAlejandro
Selena Gomez & The SceneA Year Without Rain
![Page 11: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/11.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 11
Total Result Size RangeSelected Music Blogs
264,753256,205
232,936
Lady GagaBad Romance
000
Mariah Carey featuring Juelz Santana & Bone Thugs-n-HarmonyDon't Forget About Us
![Page 12: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/12.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 12
Total Result Size RangeThe 500 Greatest Songs of all Time
174,088162,937
145,076
Michael JacksonBillie Jean
000
The Isley BrothersThat Lady (Part 1 and 2)
![Page 13: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/13.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson
URI Unavailability Rooted from a selected collection
13
![Page 14: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/14.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 14
URI Unavailability Expected Half-life
![Page 15: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/15.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 15
URI Publication and Removal Rate
![Page 16: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/16.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 16
Lifetimes of unavailable videos
Years
Month
![Page 17: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/17.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 17
Reasons for no unavailable videos
![Page 18: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/18.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 18
When a YouTube video disappears
‣ video title The Rolling Stones - Satisfaction‣ url http://www.youtube.com/watch?v=214szPQBUYc‣ Published 2009-06-13 13:44 Removed 2010-04-09 (300 days
online)
HTTP/1.1 404 Not FoundContent-Type: text/html; charset=utf-8
![Page 19: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/19.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 19
Metadata purged from YouTube Databases
‣ Video feedcurl -I "http://gdata.youtube.com/feeds/api/videos/214szPQBUYc"HTTP/1.1 404 Not FoundContent-Type: text/html; charset=UTF-8Private video
‣ Related videos curl -I "http://gdata.youtube.com/feeds/api/videos/214szPQBUYc/related"HTTP/1.1 404 Not Found Content-Type: text/html; charset=UTF-8Parent Video not found
‣ Video comments curl -I "http://gdata.youtube.com/feeds/api/videos/214szPQBUYc/comments"HTTP/1.1 200 OK Content-Type: application/atom+xml; charset=UTF-8
![Page 20: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/20.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 20
Metadata Normalization
Dereferencing ASIN via amazon.com Webservice: Artist: Michael Jackson Title: Billie Jean (Single Version)
![Page 21: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/21.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 21
Availability of music-related metadata
‣ parsed out only at the first time a URI showed up in the result list for the first time
‣ YouTube crawling restrictions
‣ Remaining portion
‣ query video title against music related services via search engines
‣ Google/Yahoo! with site parameter www.last.fm/music
![Page 22: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/22.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 22
Retrieving and preserving a video’s metadata
‣ Active preservation attempt once a video copy is available
‣ Parse HTML out for structured music-related metadata
‣ YouTube generated meta data
‣ AmazonMP3 affiliate link
‣ search engines with free-form video title against music-related websites
‣ Preserving metadata into the public web infrastructure
![Page 23: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/23.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 23
Preservation Prototype
![Page 24: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/24.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 24
Metadata preservationExample: twitter
![Page 25: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/25.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 25
Pointing to a Resolver service
‣ http://ytresolve.cs.odu.edu/r/http://www.youtube.com/watch?v=214szPQBUYc/
‣ Author-side approach
‣ content creator points directly to a resolver service‣ Server-side approach
‣ Plugin/Renderer class automatically rewrites YouTube video watch URIs to resolver service
‣ Client-side approach
‣ Web-Browser plugin intercepts click on Youtube video watch URIsand redirects to resolver service
![Page 26: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/26.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 26
YouTube Resolver service
redirect
HTTP/1.1 200 OK
search for preserved metadata
‣ in list of designated accounts
query YouTube API with those
HTTP/1.1 404 Not Found
HTTPHTTPStatus Status CodeCode
http://www.youtube.com/watch?v=214szPQBUYchttp://www.youtube.com/v/214szPQBUYchttp://www.youtu.be/214szPQBUYchttp://www.youtube.com/user/WEASELxLOVER#p/a/u/2/214szPQBUYc
http://www.youtube.com/watch?v=214szPQBUYchttp://www.youtube.com/v/214szPQBUYchttp://www.youtu.be/214szPQBUYchttp://www.youtube.com/user/WEASELxLOVER#p/a/u/2/214szPQBUYc
HTTP/1.1 303 See Others *
*)http://www.youtube.com/verify_controversy...http://www.youtube.com/verify\_age...https://www.google.com/accounts/ServiceLogin...http://www.youtube.com/das_captcha..
exact best available granularity
Provided (and evaluate) alternative copies
![Page 27: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/27.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 27
Future Work
‣ Evaluation of preservation and retrieval quality of chosen services
‣ exchange services‣ additional automation of preservation process
‣ once YT URI was passed for resolving‣ Evaluation of retrieved available copies
‣ redirect to best copy instead of returning a list to choose‣ Consider international requesters
‣ taking requester’s location (country) into account
![Page 28: Music Video Redundancy and Half-Life in YouTube Matthias Prellwitz and Michael L. Nelson matthias.prellwitz@googlemail.com mln@cs.odu.edu TPDL 2011 Berlin,](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f155503460f94c2b4c4/html5/thumbnails/28.jpg)
TPDL 20119/26/11
Music Video Redundancy and Half-Life in YouTubeMatthias Prellwitz and Michael L. Nelson 28
Summary
‣ Pointing to a specific YouTube video copy by its URI has a risk of disappearance
‣ alternative copies over time available
‣ YouTube URIs unlikely to be cached once gone
‣ YouTube metadata only reliable for available URIs
‣ active preservation attempt
‣ Introducing a level of indirection: Resolver service
‣ check URI status and location header
‣ search the public web for injected metadata
‣ query for alternative copies