2015-07-02martin bisson1 stereoscopic video and gstreamer by martin bisson
TRANSCRIPT
23-04-19 Martin Bisson 1
Stereoscopic video and GStreamer
By Martin Bisson
Presentation outline
IntroductionUse casesImplementationRoad ahead
23-04-19 Martin Bisson 2
Presentation outline
IntroductionUse casesImplementationRoad ahead
23-04-19 Martin Bisson 3
Stereoscopy
23-04-19 Martin Bisson 4
Recent popularity
3D-ready hardware
3D moviesAvatar => ~2.8 billion USD
23-04-19 Martin Bisson 5
Presentation outline
IntroductionUse casesImplementationRoad ahead
23-04-19 Martin Bisson 6
Use cases
1) Merge 2 video streams into a stereo stream
2) Convert a (double) video stream into a stereo stream
23-04-19 Martin Bisson 7
Use cases
3) Convert a stereo stream into a « regular » video streamCan be displayed or encoded by normal plugins
4) Generate an anaglyph video stream from a stereo stream
5) Output a stereo stream on various 3D display hardware
23-04-19 Martin Bisson 8
Presentation outline
IntroductionUse casesImplementationRoad ahead
23-04-19 Martin Bisson 9
Implementation
What is a « stereo stream »?Buffer flags?New caps
video/x-raw-{rgb,yuv,gray}-stereolayout = { 1122 , 1212 }fourcc = { YUY2, YVYU, UYVY , Y800 , GREY , Y8 , Y16 }
23-04-19 Martin Bisson 10
Memory layout
23-04-19 Martin Bisson 11
1122Memory-consecutive
1212Row-interleaved
Supported formats
Only packed, no planarRGB : RGBx, xRGB, BGRx, xBGR, RGB,
BGR, RGB_16, RGB_15YUV : YUY2, YVYU, UYVY, Y800,
GREY, Y8, Y16GRAY : GRAY8, GRAY16
23-04-19 Martin Bisson 12
Plugins
video3dmergeGstCollectPadsmode : memory-consecutive or row-interleaved
23-04-19 Martin Bisson 13
Plugins
video3dconvertGstVideoFiltermode : left-right or top-bottom
23-04-19 Martin Bisson 14
Plugins
video3dpresentGstVideoFiltermode : a lot of them…
23-04-19 Martin Bisson 15
video3dpresent modes
Left, RightLeft/Right, Right/Left, Top/Bottom, Bottom/TopAnaglyph
Gray/Gray, Gray/Color, Color/Gray, Color/ColorRed/Cyan, Green/Magenta, Blue/Yellow, and vice-versa
23-04-19 Martin Bisson 16
Anaglyph implementation
« C templates »Inspired by ffmpegcolorspace (imgconvert.c)
Orc optimizationOnly RGBx/BGRx/xRGB/xBGR output formatsYUY2/YVYU/UYVY/Y800/GREY/Y8/Y16 input
formats for gray-gray anaglyphRGBx/BGRx/xRGB/xBGR input formats for all
anaglyph types
23-04-19 Martin Bisson 17
23-04-19 Martin Bisson 18
Example application
23-04-19 Martin Bisson 19
Presentation outline
IntroductionUse casesImplementationRoad ahead
23-04-19 Martin Bisson 20
Integration
Not in main GStreamer tree (in a branch at http://www.gitorious.org/video3d/video3d/)
Talks about how to handle stereo video:Buffer flagsMetadataNew caps
23-04-19 Martin Bisson 21
Integration
New caps (video/x-raw-*-stereo)Caps negotiation and all that jazz…
Buffer flags/metadataBuffers can be used in existing plugins
23-04-19 Martin Bisson 22
Integration
23-04-19 Martin Bisson 23
3D hardware
Implement stereo sinksNVIDIA 3D VisionQuad Buffered OpenGLiZ3DEtc…
See Stereoscopic Player (http://www.3dtv.at/)
23-04-19 Martin Bisson 24
Stereo codecs
Multiview Video Coding (H.264/MPEG-4 AVC)Uses temporal AND inter-view predictionExisting implementation by Nokia
http://research.nokia.com/page/4988
23-04-19 Martin Bisson 25
23-04-19 Martin Bisson 26
Questions?