video streaming: from the native android player to uncoventional devices
DESCRIPTION
Getting a streaming video in your Android smartphone or tablet is no longer enough.In the latest period, Google shows how to push this concept forward to new appliances. Chromecast and Android TV are two of the most promising gadgets for upsetting the way users enjoy video streaming. This talk we’ll give you an overview about the streaming in Android. Starting from video streaming on mobile devices, we will guide you into the evolution of the development through Chromecast up to Android TV. Matteo Bonifazi and Alessandro Martellucci will be illustrating this talk with their experiences developing mobile television applications for the main Italian broadcaster providers.TRANSCRIPT
![Page 1: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/1.jpg)
Video Streaming from the native Android player
to uncoventional devices
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
![Page 2: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/2.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
• Reply is today a leading IT Services Company
• Operates in Italy, Germany, UK, Benelux, USA and Brasil.
• Open Reply is the company of Reply Group focused on open source software,
multichannel web solutions and mobile applications.
• Based in Rome, Open Reply’s Android Lab is a young team of over 20 engineers 100%
focused on Android development.
• We are specialised in broadcasting, banking and Android OS Customisation.
Open Reply & Android Lab
![Page 3: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/3.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
• Playing a content on your handheld device.
• Entertaiment tailored for the user with
• Easiest way to enjoy online video and music on user TV on
Presentation Millestones
![Page 4: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/4.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Android multimedia framework
• Android support for playing several media types from media file stored inside the
application (raw resources, standalone files) or for OTT streaming.
• Documentation of the media framework is enough just for simple test case.
• Underneath MediaPlayer Framework documentation is kind of nightmare (lot of
events and error are not documented at all).
![Page 5: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/5.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Supported type
HttpLiveStreaming Protocol 3 is supported by Android 4.0. Best implementation is in Kitkat.
Google seems putting all his efforts to support new media type ( SmoothStreaming, Dynamic adaptive streaming)
![Page 6: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/6.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Playing a video
VideoView
Three different approaches
Mediaplayer &
SurfaceView
ExoPlayer
![Page 7: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/7.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
//1. Find the view from the layout VideoView myVideoView = (VideoView)findViewById(R.id.myvideoview); //2. Setup video url myVideoView.setVideoURI(Uri.parse(SrcPath)); //3. Setup Video controller myVideoView.setMediaController(new MediaController(this)); //4. Start playing myVideoView.requestFocus(); myVideoView.start();
android.widget.VideoView
MediaPlayer code example
Programmer doesn’t directly handle the MediaPlayer
![Page 8: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/8.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
//0. Get SurfaceView and its holder mPreview = (SurfaceView)findViewById(R.id.surfaceView); holder = mPreview.getHolder(); //1. Create MediaPlayer object: mp = new MediaPlayer(); //2. Add SurfaceHolder callback - Aware when SurfaceView is created holder.addCallback(new SurfaceHolder.Callback(){ .... @Override public void surfaceCreated(SurfaceHolder holder) { //3. Attach the surface to the player mp.setDisplay(holder); try { mp.setDataSource(filepath);
//4. Prepare the Mediaplayer in sync or async mode ( prepareAsync() ) mp.prepare(); } catch (Exception e) {// Catch the exception} //5. Start the player mp.start(); }... });
android.media.Mediaplayer &
android.view.SurfaceView
MediaPlayer code example
Application creates SurfaceView and MediaPlayer. More control on MediaPlayer state
![Page 9: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/9.jpg)
Droidcon London – 2014 – Matteo Bonifazi e Alessandro Martellucci
// 1. Instantiate the player. player = ExoPlayer.Factory.newInstance(RENDERER_COUNT); // 2. Construct renderers. MediaCodecVideoTrackRenderer videoRenderer = … MediaCodecAudioTrackRenderer audioRenderer = ... // 3. Inject the renderers through prepare. player.prepare(videoRenderer, audioRenderer); // 4. Pass the surface to the video renderer. player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface); // 5. Start playback. player.setPlayWhenReady(true); ... player.release(); // Release when everything is done!
ExoPlayer
MediaPlayer code example
Pre-built player that can be extend. Implement features not currently
supported in the normal mediaplayer.
![Page 10: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/10.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
What kind of Mediaplayer exist?
• Built-in players
ü AwesomePlayer (default player selected)
ü NuPlayer (Apple HLS) • Extra player factories can be registered • DIY mediaplayer
ü Demuxing: android.media.mediaExtractor
ü Decoding: android.media.MediaCodec
ü Video rendering: android.media.MediaCodec
ü Audio rendering: android.media.AudioTrack
ü Implement the interface frameworks/av/include/media/MediaPlayerInterface.h
![Page 11: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/11.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Android TV Entertainment tailored for you
Source:h)p://www.televisedrevolu4on.com/
![Page 12: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/12.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Nexus Player
Source:h)p://www.televisedrevolu4on.com/
1.8GHz Quad Core, Intel® Atom™ Imagination PowerVR Series 6
Graphics 2D/3D Engine
1GB RAM 8GB storage
Asus device – 235 g
![Page 13: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/13.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Android TV app features
• Android TV devices have Android Lollipop 5.0 on board.
• Android TV has inside the same Android mul4media framework of normal
devices.
• Android TV app can be built on API 17 towards.
• Based on Leanback Android Support library
![Page 14: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/14.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
10 feet experience
Smartphone 5” 320 dp
TV Full HD 30” 320 dpi
10 feet
16 inches
![Page 15: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/15.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Manifest features Support landscape
Portrait activity are forbidden
![Page 16: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/16.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Manifest features No touch screen
<uses-feature android:name="android.hardware.touchscreen" android:required="false"/>
![Page 17: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/17.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Manifest features Limit sensor
<uses-feature android:name="android.hardware.sensor.accelerometer” android:required="false" />
![Page 18: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/18.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Keep calm and lean back!!
Provides built-in tailored for 10 feet experience
![Page 19: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/19.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Leanback support library
<uses-feature android:name="android.software.leanback" android:required="true" /> <application android:allowBackup="false”
android:label="@string/app_name” android:theme="@style/Theme.Leanback" >
<activity android:name="MainActivity”
android:screenOrientation="landscape" > <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LEANBACK_LAUNCHER" /> </intent-filter> </activity>
![Page 20: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/20.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Leanback support library Model View Presenter
Presenter Model View
![Page 21: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/21.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Leanback UI component BroswerFragment
allows the developer “to create a primary layout for browsing categories and rows of media items [with a minimum of code]”
![Page 22: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/22.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Leanback UI component DetailFragment
Display information about the content that the user has selected
![Page 23: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/23.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Chromecast a cast-ready device for multi-screen experience
source: www.google.it
![Page 24: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/24.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Chromecast hardware specifications
source: www.pcworld.com
![Page 25: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/25.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Chromecast components
• Google Cast technology
• Multi-Screen experiece • Google Cast SDK
• Sender Application • Android app • iOS app • Chrome app
• Receiver Application • Default Media Receiver • Styled Media Receiver • Custom Media Receiver
![Page 26: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/26.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Android Client Application library dependencies
• Minimum SDK version supported by Google Cast is 9 (Gingerbread)
• MediaRouter API of android-support-v7
• Google Play Services
• AppCompat API of android-support-v7
![Page 27: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/27.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Android Client Application typical sender application flow
• Sender app starts MediaRouter device discovery: MediaRouter.addCallback • MediaRouter informs sender app of the route the user selected: MediaRouter.Callback.onRouteSelected • Sender app retrieves CastDevice instance: CastDevice.getFromBundle • Sender app creates and uses GoogleApiClient: GoogleApiClient.Builder
• Sender app launches the receiver app: Cast.CastApi.launchApplication • Sender app creates a communication channel: Cast.CastApi.setMessageReceivedCallbacks • Sender sends a message to the receiver over the communication channel: Cast.CastApi.sendMessage
source: developers.google.com
![Page 28: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/28.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Cast-Ready Device Discovery capabilities
Remote Playback Live Audio Live Video
MediaRouteSelector.Builder mediaRouteSelectorBuilder = new MediaRouteSelector.Builder();
mediaRouteSelectorBuilder.addControlCategory(MediaControlIntent.CATEGORY_REMOTE_PLAYBACK);
mediaRouteSelectorBuilder.addControlCategory(MediaControlIntent.CATEGORY_LIVE_AUDIO);
mediaRouteSelectorBuilder.addControlCategory(MediaControlIntent.CATEGORY_LIVE_VIDEO);
MediaRouterSelector mediaRouterSelector = mediaRouterSelectorBuilder.build();
![Page 29: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/29.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Media Cast Button easy approach for discovering
source: developers.google.com source: developers.google.com
public boolean onCreateOptionsMenu(Menu menu) { MenuItem mediaRouteMenuItem = menu.findItem(R.id.media_route_menu_item); MediaRouteActionProvider mediaRouteActionProvider = (MediaRouteActionProvider) MenuItemCompat.getActionProvider(mediaRouteMenuItem); mediaRouteActionProvider.setRouteSelector(mMediaRouteSelector); …. }
![Page 30: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/30.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Receiver Application what a mistery?
What is?
HTML5 and Javascript application
What does it do? Display the media content on TV
Message handling
Which type?
Default Media Receiver
Styled Media Receiver Custom Media Receiver
![Page 31: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/31.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Default Media Receiver simplest
• Off-the-shelf
• No UI customization
• No registration Source: developers.google.com
![Page 32: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/32.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Styled Media Receiver simple and customizable
• Similar to Default Media Player
• CSS UI customization
• Registration
Source: developers.google.com
![Page 33: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/33.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Custom Media Receiver (1/3) whatever you want
• Fully Web Applicaiton
• Debug(able) at 9222
• Registration
Source: developers.google.com
![Page 34: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/34.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Custom Media Receiver (2/3) example
<html> <head> <title>Example minimum receiver</title> <script src="//www.gstatic.com/cast/sdk/libs/receiver/2.0.0/cast_receiver.js"></script> </head> <body> <video id='media'/> <script> window.mediaElement = document.getElementById('media'); window.mediaManager = new cast.receiver.MediaManager(window.mediaElement); window.castReceiverManager = cast.receiver.CastReceiverManager.getInstance(); window.castReceiverManager.start(); </script> </body> </html>
![Page 35: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/35.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Custom Media Receiver (3/3) advanced features
• Video Codification/Decodification • H.264 High Profile Level 4.1, 4.2 and 5 • VP8
• Adaptive Bitrate Streaming
• HTTP Live Streaming (HLS) • Dynamic Adaptive Streaming over HTTP (MPEG-DASH) • Smooth Streaming
• Digital Rights Management • Play Ready DRM • Widevine DRM
• Media Player Library
![Page 36: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/36.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Channel, Namespace and Protocol (1/2) communication
• Protocol: a set of well-known messages • Namespace: a labeled protocol • Channel: the communication layer
class CustomChannel implements Cast.MessageReceivedCallback { public String getNamespace() { return “urn:x-cast:com.channel.custom”; } @Override public void onMessageReceiver(CastDevice castDevice, String namespace, String message) { … } } … Cast.CastApi.setMessageReceivedCallbacks(mApiClient, mCustomChannel.getNamespace(), mCustomChannel); …
![Page 37: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/37.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Channel, Namespace and Protocol (2/2) communication
• Media Namespace: urn:x-cast:com.google.media.cast • RemoteMediaPlayer • MediaManager
RECEIVER MEDIA CHANNEL
window.mediaManager = new cast.receiver.MediaManager(window.mediaElement); …
CLIENT MEDIA CHANNEL Cast.CastApi.setMessageReceivedCallbacks(mApiClient,
mRemoteMediaPlayer.getNamespace(), mCustomChannel); …
![Page 38: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/38.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Google Api Client and Media Route Provider manufacturing
Source: developers.google.com
CastDevice selectedCastDevice = CastDevice.getFromBundle(selectedRouteInfo.getExtras); Cast.CastOptions.Builder apiOptionsBuilder = new Cast.CastOptions.Builder(selectedCastDevice, …); googleApiClient = new GoogleApiClient.Builder().addApi(Cast.API, apiOptionsBuilder.build()).build(); googleApiClient.connect();
Cast.CastApi.launchApplication(googleApiClient, applicationId, launchOptions); Cast.CastApi.joinApplication(googleApiClient); Cast.CastApi.stopApplication(googleApiClient); Cast.CastApi.leaveApplication(googleApiClient);
![Page 39: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/39.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
RemoteMediaPlayer …finally
MediaInfo mediaInfo = new MediaInfo.Builder("http://your.server.com/video.mp4") .setContentType("video/mp4") .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) .setMetadata(mediaMetadata) .build();
… mRemoteMediaPlayer = new RemoteMediaPlayer(); mRemoteMediaPlayer.load(mApiClient, mediaInfo, true); …
![Page 40: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/40.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Enjoy the video
Source: www.huffingtonpost.ca
Thank you to all of you
![Page 41: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/41.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Resources
• Google Cast -> https://developers.google.com/cast • Chromecast App -> http://www.google.it/chrome/devices/chromecast/apps.html • Google Cast Downloads -> https://developers.google.com/cast/docs/downloads • Github -> https://github.com/googlecast
• Android TV -> https://developer.android.com/tv • Nvidia Guideline -> https://developer.nvidia.com/android-tv-developer-guide • Github -> https://github.com/googlesamples/androidtv-Leanback.git
• Android Mediaplayer-> http://developer.android.com/guide/topics/media/mediaplayer.html • ExoPlayer -> https://github.com/google/ExoPlayer
![Page 42: Video Streaming: from the native Android player to uncoventional devices](https://reader034.vdocument.in/reader034/viewer/2022052412/559443001a28ab01308b46a3/html5/thumbnails/42.jpg)
Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Contacts
a.martellucci[at]reply.it
@martellux
+AlessandroMartellucci
m.bonifazi[at]reply.it
@mbonifazi
+MatteoBonifazi
http://www.reply.eu http://www.gaiareply.com