webkit, html5 media and gstreamer on multiple platforms
TRANSCRIPT
![Page 1: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/1.jpg)
WebKit, HTML5 mediaand GStreamer onmultiple platforms
Spreading GStreamerawesome in WebKit
Philippe Normand
![Page 2: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/2.jpg)
Talk outlineAbout WebKitHTML5 Media playback in WebKit withGStreamerWebKit/GStreamer: opening the door to moreplatformsEnd goal: spread the usage of GStreameron more and more WebKit-basedapplications!
![Page 3: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/3.jpg)
About WebKitA content rendering/editing engineNot a browser!Highly standards-compliant
![Page 4: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/4.jpg)
WebKit EcosystemApple, Nokia, Google, RIM, Adobe, Sony-EriccsonSamsung, Palm, Motorola, Electronic Arts, ...=> lots of devices using WebKit in the market=> wide range of applications using thevarious WebKit ports
![Page 5: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/5.jpg)
High level WebKitArchitecture
JavaScriptCore: Javascript engine andplatform-specific bitsWebCore: platform-independant blocks andtheir counter part platform implementationsfor:
parsing, layoutingnetwork, paintingmedia playback...
WebKit: platform-specific APIs => used byapplication developersPorts: GTK+, Qt, EFL, Chromium, Mac,WinCairo, Apple Windows, ...
![Page 6: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/6.jpg)
HTML5 Mediaplayback in WebKit
3 layers: DOM/HTML level(HTMLMediaElement), WebCore/graphics(MediaPlayer), platform-dependantimplementation (MediaPlayerPrivate)Some MediaPlayerPrivate implementations:
GStreamer: GTK+ and EFL portsQt/Phonon: Qt portQuicktime: Apple Mac/Win ports
![Page 7: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/7.jpg)
GStreamer mediaplayer
playbin2 of course!Custom video sink and source elementsBasic trick-modes supportOn-disk bufferingFullscreen video display
![Page 8: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/8.jpg)
The GStreamer playerin other WebKit ports?
GStreamer cross-platform already :)
the MediaPlayer layer allows multiple Privateinstances
choice based on media mime-type andcodecs:
<video src=test.ogv type="video/ogg; codecs="theora,vorbis&qu
Targets: Mac and WinCairo ports
![Page 9: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/9.jpg)
Porting check-listGStreamer platform packagesGLib main context integrationwebkit src elementwebkit video sinkFullscreen supportWebKit build :)
![Page 10: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/10.jpg)
GStreamer platformpackages
on Windows: OSSBuildon Mac OSX: OSSBuild too! soon!For now on Mac: scripts developed based ongtk-osx-build
![Page 11: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/11.jpg)
GStreamer build onMac
jhbuild to the rescueadd -headerpad_max_install_names toLDFLAGSuse install_name_tool to change theshared library paths in the various .dylib filesabsolute paths -> relative paths to@loader_path
![Page 12: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/12.jpg)
GLib main contextintegration
GstBus signal watch used in the player =>need periodic calls ofg_main_context_iteration()on Mac: NSRunLoop observer triggering theglib main context iterationson Windows: integration in the message loop
![Page 13: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/13.jpg)
WebKit src elementBased on appsrc, work by Sebastian Dröge!using WebCore Resource loader (=> HTTPsession data: cookies, referer)Contains some platform-specific code topause/resume data downloadFor now: #ifdefs :( Better approach: useWebCore resource defersLoading API.
![Page 14: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/14.jpg)
Video paintingvideo sink exports buffers to the player("repaint-requested" signal)ImageGStreamer: GstBuffer ->WebCore::Imageactual painting deferred to theImageGStreamer implementations:
ImageGStreamerCairo: used by GTK+ andWinCairo portsImageGStreamerCG: CoreGraphicsimplementation for the Mac port
![Page 15: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/15.jpg)
Fullscreen support(Overview)
webkitEnterFullscreen() VideoElement DOM API=> WebKit WebView private API=> => FullscreenVideoController
enterFullscreen() hooks intoMediaPlayer::platformMedia()MediaPlayerPrivate-specifics: to create awidget where the video is paintedController gets the widget, adds controls,goes fullscreen
![Page 16: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/16.jpg)
Fullscreen support(GStreamer side)
Our playbin2 video-sink is a bin!
fullscreen off:
tee name=t t. ! queue ! webkitvideosink
fullscreen on:
tee name=t t. ! queue ! webkitvideosink t. ! queue !ffmpegcolorspace ! videoscale ! autovideosink
![Page 17: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/17.jpg)
Fullscreen support(Video overlay)
GstXOverlayPlatformVideoWindow : window on whichautovideosink overlays the video
PlatformVideoWindowGtkPlatformVideoWindowEFLPlatformVideoWindowCocoa...
![Page 18: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/18.jpg)
WebKit buildmodifications
(note: WTF here means Web TemplateFramework, obviously!)add JavaScriptCore/wtf/gobject/ inyour buildaddWebCore/platform/graphics/gstreamer/in your buildenable WTF_USE_GSTREAMER andENABLE_GLIB_SUPPORT definesport-specific FullscreenVideoControlleradaptations
![Page 19: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/19.jpg)
Implementationstatus : Mac OSX
Most of the patches merged upstreamalready, excepted:
XCode config additionsgst webkit src patchPlatformVideoWindowCocoaimplementationFullscreenVideoController patch
Packaging: WebKit-nightly DMG includingGStreamer and its dependencies.
![Page 20: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/20.jpg)
Implementationstatus : WinCairo
HTML5 media implemented from scratch forWinCairo:
WinCairo still using Visual Studio 5 => bigblocker :(Patches in a separate public git repository
Basic media controlsFullscreen video
![Page 21: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/21.jpg)
Recap!1. GStreamer build2. Periodic calls to
g_main_context_iteration()3. Patching gst webkit src element and/or your
ResourceHandle implementation4. ImageGStreamerMyPort implementation5. PlatformVideoWindowMyPort
implementation6. Patching your WebKit
FullscreenVideoController7. Build WebKit!
![Page 22: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/22.jpg)
Some bit of adviceUse playbin2 :-)Facade design-pattern for your app platform-specific bitsSingle build-system == less maintenanceburden (gyp, cmake, ...)Continuous integration on multiple platforms:BuildBot
![Page 23: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/23.jpg)
DemoWebM playback, in Safari! With GStreamer! :-)
![Page 24: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/24.jpg)
Future plansFinish upstream merge of the patchesWebKit/GStreamer build slavesFullscreen video improvementsWebKitGTK+ API to allow fine-grained controlon the player like:
per-webview volume controlcodec installer
![Page 25: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/25.jpg)
Luke, may the sourcebe with yougst-mac and win-gst WebKit branches:
http://git.igalia.com/cgi-bin/gitweb.cgi?p=webkit.githttp://trac.webkit.org/wiki/GStreamerOnMac
![Page 26: WebKit, HTML5 media and GStreamer on multiple platforms](https://reader034.vdocument.in/reader034/viewer/2022052303/554d2168b4c905ab268b4944/html5/thumbnails/26.jpg)
That would be all,folks
Work sponsored by:IgaliaR&D funds from Galicia governmenta private Igalia client
Contact:http://[email protected]: philn-tp on #gstreamer, #webkit and#webkit-gtk
Questions?