ensemblue: integrating distributed storage and consumer electronics daniel peek and jason flinn...
TRANSCRIPT
EnsemBlue: Integrating Distributed
Storage and Consumer Electronics
Daniel Peek and Jason Flinn
University of Michigan
University of Michigan 2
Bringing Distributed Systems Home
• Distributed systems ignore Consumer Electronic Devices (CEDs)
• Much user interaction is through CEDs
• Integrate CEDs into distributed systems
University of Michigan 3
CED Integration Difficult
• Application-specific resources
• Heterogeneous interfaces• Closed platforms
– Cannot execute our code
University of Michigan 4
Today
University of Michigan 5
• Hypothesis: Distributed file systems can help manage CEDs and multimedia
EnsemBlue
My Data
My Devices
University of Michigan 6
EnsemBlue
• Based on BlueFS [Nightingale 04]– Client-server design– Single namespace– Supports mobile clients– Designed for small group of users
FileServer
University of Michigan 7
Outline
• Motivation• Device-Specific Namespaces• Device-Specific Functionality• Disconnected Collections of Devices
• Evaluation• Conclusion
University of Michigan 8
Device-Specific Protocol
Connecting CEDs
• CEDs cannot perform DFS protocol
DFS Protocol?
Distributed File System
Device-Specific Protocol
• Attach: computer speaks for CED• Works with disconnected clients
DFS Protocol
University of Michigan 9
Device-Specific Namespaces
• User prefers one organization• CED requires another organization
Classical
Jazz
F01
F02
F03
F04
University of Michigan 10
• Where to store the mapping?
Device-Specific Namespaces
• Cannot store in DFS due to partial caching
• Make CEDs self-describing
University of Michigan 11
Receipts
• What should mapping look like?
Object 1.123.46F2
~dpeek/classical/song.mp3
/iPod/f27/ABCD.mp3
• Path lookups may fail with partial caching
• Receipts bypass path resolution
University of Michigan 12
Device-Specific Execution
• Updating indexes• Transcoding• Type-specific caching• Organizers
University of Michigan 13
Device-Specific Execution
• Cannot execute on CED• Leverage general computers
– Take action when files change
• Problem: Need notification of file changes
University of Michigan 14
Notification Via Persistent Queries
• Don’t need new mechanism• Leverage cache consistency mechanism
– Structure notifications as file system object
– Robust to crash– Handles disconnected operation
updates
truncate
read
process*.mp3
University of Michigan 15
MP3
M4A
Persistent Queries• Example: M4A to MP3 transcoder
FileServer
M4A to MP3Transcoder *.m4a
M4A
M4A
MP3
*.m4a MP3
University of Michigan 16
Disconnected Devices
• Server gives safety, availability, consistency
• Server mediates communication
FileServer
University of Michigan 17
Disconnected Devices
• Disconnected devices cannot interact!
FileServer
University of Michigan 18
Disconnected Devices
• Peer-to-Peer better supports mobility
• CEDs store and forward updates
FileServer
University of Michigan 19
Ensembles
• Ensembles can avoid these problems
• Select a pseudo file server– Consistent view of all files– Improved data access– Update propagation
Pseudo FileServer
?
?
University of Michigan 20
Outline
• Motivation• Device-Specific Namespaces• Device-Specific Functionality• Disconnected Collections of Devices
• Evaluation• Conclusion
University of Michigan 21
Microbenchmarks
• Persistent query execution is fast– <200 ms for 1,000s of matches, GBs of data
• Persistent queries have low overhead– Negligible even for 10,000 queries
• Ensemble formation as fast as 20 ms– Proportional to data transfer size
University of Michigan 22
Organizing My Photos
PhotoOrganizer
iCal
*.jpg
FileServer
University of Michigan 23
Managing My Music
iPod DBUpdater
MP3
MP3 DB
DB
MP3 DB
MP3 DB
*.mp3
FileServer
MP3
University of Michigan 24
Conclusion
• Added CEDs to distributed systems
• Device-specific data organization
• Device-specific functionality• Supports TiVo, iPod, digital cameras, media players, cell phones, PDAs
University of Michigan 25
Questions
?