or how to handle multimedia on linux
TRANSCRIPT
![Page 1: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/1.jpg)
Solving the puzzle of multimedia on linux
or how to handle multimedia on linux
Muriel Moscardini
Fluendo SA
![Page 2: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/2.jpg)
Agenda
1. Introduction 2. Multimedia 3. Linux as a solution 4. Stakes of multimedia on linux 5. What solutions do we have 6. In practice… 7. Market solutions 8. Conclusion
![Page 3: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/3.jpg)
1. Introduction A talk for all, Distributors and Users, companies, IT directors, device
makers, software vendors, developers, product managers,…to understand how to cope with patented multimedia formats
Not a talk about legitimacy of patent on software but solutions to live
with it Here to inform, each being responsible for taking his own decisions
Share experience, give tips, explain, make it clearer
The talk is open to all the feedback you may have to share.
![Page 4: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/4.jpg)
2 - Multimedia
![Page 5: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/5.jpg)
2.1. Multimedia everywhere
![Page 6: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/6.jpg)
+31%
2.2. Market Figures
Multimedia Consumption : A Vibrant Market
-11%
$8 billion by 2014*
70% watch online video*
*source Digital Media Post
VOD and Streaming 2010/2011
Digital Downloads +
On-line subscription
Among all online consumers,
*source PricewaterhouseCooper's *source Nielsen
DVD and BluRay rentals
2010/2011
![Page 7: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/7.jpg)
2.3 -Multimedia in the enterprise
Multimedia content
IN HOUSE PRODUCTION Recording Training Recording events Marketing productions Corporate webcasts
CONSUMPTION E learning Business content All company divisions Social share
EXTERNAL CONTENT On line or not Streamed On demand or Live
Content Delivery Network Internal or external
![Page 8: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/8.jpg)
Multimedia is everywhere we need to care. It is important for your
“customers” whether you sell them devices or software, or whether
they are your employees
![Page 9: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/9.jpg)
3 – Linux as a solution
![Page 10: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/10.jpg)
3.1 Why Linux is a good answer? For device makers and enterprises
Low cost
Ease of customization
Very large choice of projects available, high potential of differentiation
Stable kernel
Ported to numerous platforms
Innovation stimulated by the open community
Very large ecosystem of software development company providing support and project services
Allow a mix between free and proprietary solutions to take advantage of both
![Page 11: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/11.jpg)
Linux has been chosen for Embedded solutions a long time ago and drove the innovation, we even didn’t know.
Final users acceptance has taken longer time, but thanks to new types of man-machine interfaces (smartphone, tablets) Linux based OS have become a solution for millions of people.
3.2 Solution recognized for Embedded and for final users
![Page 12: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/12.jpg)
3.3 Challenges of Linux in the enterprise
Proprietary OS
- Pay by unit - All included - No or few custom - Used for years - Usage habits - Pay for upgrades - Ecosystem of proprietary solutions
Linux distribution
- Free, and “a la carte” set up of paying apps
- Large functional scope available - Free and Proprietary applications
available - Diverse - New - Paying support services available
Ensure users acceptation Maintain efficiency Maintain scope coverage
Multimedia may be a major gap
![Page 13: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/13.jpg)
4 – Stakes of multimedia on linux
![Page 14: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/14.jpg)
4.1 Generic model
Multimedia Application
Linux OS (Fedora, Suse, Mandriva, Ubuntu, customer OS,…)
Multimedia Framework open: Gstreamer, VLC, Phonon,
Patented Formats
Free or proprietary license
Free license
Free Formats A/V Codecs
![Page 15: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/15.jpg)
4.2 Why it can become a constraint?
• Some countries recognize these patents • Ie. US, Germany recognize patent on software
• Use/Distribute patented format would require to
be patent licensee
• Many of the patent licenses are incompatible with most free software licenses so it may be incompatible to distribute both together.
• Ie. Software codec under a GPL license together with a patent license
![Page 16: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/16.jpg)
4.3 Licenses incompatibility
• Some terms in patent licenses are incompatible with some terms in open licenses*
• Mainly the fact that open source (free software) licenses require to give the right to use, copy, modify, distribute
• Patent licenses often forbid copy, reverse engineering, release
source code and often bear per-copy running royalties and therefore request tracking
• The main impact may be to be obliged to get some
software with compatible license (restricted rights – not GPL)
*ref. Section 11 of the GNU LGPL (version 2)
![Page 17: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/17.jpg)
4.4 Why we didn’t care before?
BEFORE • Use of proprietary OS • Pay for license • All in one • We didn’t know or look for what was included in
the price • Patent where included in the license price when
the format was supported • If not we were even not used to wonder why • Deal as a black box
![Page 18: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/18.jpg)
4.5 Now what has changed?
NOW, what is changing with Linux
• We build and choose the modules of our solution, no more a black box
• we don’t pay upfront, nothing patented is included a priori
• If we decide to include an open and free project, there is no business model (specific for companies or resellers) allowing you to distribute these pieces of software (patented codecs) in an obvious way
![Page 19: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/19.jpg)
4.6 Why should we care?
• Linux concerns millions of people
• Linux is growing and interest is also growing
• In the patent world, Patent holder claim infringment if not licensed
• Now patent holders care about linux
![Page 20: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/20.jpg)
4.7 Why it impacts companies?
In countries where patents on software are recognized, infringing the patent can lead to negative legal consequences
• Companies distributing thousands of units may be
concerned about being sued
• Companies who have patents themselves may be concerned about respecting patent licenses
• Enterprises deploying thousand of workstations /desktops may not want to endorse responsibility
![Page 21: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/21.jpg)
4.8 First level of impact When choosing a solution, implementing a sofware, distributing it Pay more attention and analyse what we use,
distribute, bundle on a licensing point of view
Find and accept new ways to build the solutions to comply licenses, accept to make compromises Ie. Proprietary mix with OS
Be Geography sensible
![Page 22: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/22.jpg)
4.9 Second level of impact
The solution may not be completely free anymore
• Be patent license compliant often means - be licensee, sign agreements - may imply some royatly to be paid to the patent holder, - tracking units, tracking distribution/use - may imply choices of specific software to comply with the
patent license terms
Require new skills and help from experts in licensing
![Page 23: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/23.jpg)
5 – What solutions do we have?
![Page 24: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/24.jpg)
Safer way is to use the free formats available (open and free of patent and copyright)
When you can control the source of content, it is the best option, with no constraints
5.1 Use free formats
![Page 25: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/25.jpg)
Examples of free formats
Audio codec Video codec
ALAC— lossless audio codec, previously a proprietary format of Apple Inc. CMML — timed metadata and subtitles DAISY Digital Talking Book— a talking book format FLAC — lossless audio codec Musepack — an audio codec ADPCM audio LPCM audio ILBC audio XSPF — a playlist format for multimedia Speex — speech codec WavPack — "Hybrid" (lossless/lossy) audio codec
Theora - video PNG — a raster image format standardized by ISO/IEC SMIL — a media playlisting format and multimedia integration language SVG — a vector image format standardized by W3C VRML/X3D — realtime 3D data formats standardized by ISO/IEC WebM — a video/audio format JPEG 2000— an image format standardized by ISO/IEC DASH demuxer, for adaptive streaming HLS demuxer for adaptive streaming MNG — moving pictures, based on PNG
Containers
Ogg — container for Vorbis, FLAC, Speex and Opux (audio formats) & Theora
Matroska (mkv) — container for all type of multimedia formats (audio, video, images, subtitles)
![Page 26: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/26.jpg)
Bundle and resell: choose not to provide the support directly but rely upon the fact the final user can get it: integrate some “plug-in” management program, where the user will be directed to a source providing solutions to get them, or rely upon source of content providing their own (direct user dowload)
Company: block the content of any source that you don’t cover, and rely upon content that don’t need patent license to be read
5.2 Don’t provide directly the support for patented formats
![Page 27: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/27.jpg)
• As it is not always possible to choose the formats to be supported depending on the source of content you want to support.
• In this case, the approach is to use proprietary implementations and and get the license for the patent.
Bundle with software under compatible licenses
5.3 Use implementation with patent license
![Page 28: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/28.jpg)
5.3 Use implementation with patent license
= READY
TO SHIP/USE PRODUCT
•User satisfaction (internal/external) •Cope with laws in country of Distribution or use •Reduce patent infrigment risk and law pursuit
Codec pack and other multimedia application from
the market answering licensing
requirements Patent licenses...
+
![Page 29: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/29.jpg)
Some of the patented formats
Format Known patent licensing entities/ patent holders
Windows Media Audio
Windows Media Video
Windows Media network protocol
Smooth Streaming demuxer
H264 /MPEG4part10/AVC) video
MPEG2 video
MPEG4part2 video
AAC/HEAAC audio
DD/AC3 audio
DD+/EAC3 audio
TrueHD audio
Dolby Headphones audio
Pulse audio
AMR NB WB audio
MP3/MPEG1LayerIII audio
H265 video
SILK audio
DTS audio
RMV video
RMA audio
Microsoft
Microsoft
Microsoft
Microsoft
MPEGLA
MPEGLA
MPEGLA
Via Licensing (Dolby)
Dolby
Dolby
Dolby
Dolby
Dolby
Voice Age
Thomson
Not yet defined
Skype
DTS
Real networks
Real networks
Containers
ASF
3GPP
Microsoft
3GPP
![Page 30: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/30.jpg)
5.4 What if we deal with hardware codecs
•To cope with multimedia requirements, harware manufacturers more and more include the encoding(decoding directly in the platform
•They may be implementation licensee but will never be final products
•Ie. If you resell an APP considering you will use the hardware decoder of the platform, you are responsible
![Page 31: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/31.jpg)
7 – In practice…
![Page 32: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/32.jpg)
7.1 Generic matters when you start Define the need and scope:
- for device makers define the target and identify the multimedia consumption behavior of your target
- for companies identify the need for each category of worker and ensure where multimedia must be supported
- choose the OS/distro the most adapted to your business
Define the technical configuration
- define the multimedia framework to be used and list players available
- list the codecs needed to be supported
![Page 33: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/33.jpg)
7.2 Define if you are eligible
Main patent licensors will require the final OEM or final user to be the licensee.
Some patent licensors will provide 2 levels of license
- One for implementation: when you provide the software (codec)
- One for distribution
You need to identify where you are in the chain and what you provide
![Page 34: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/34.jpg)
7.3 How does that work? Step 1
1. You identify your needs and possible providers
- You define the codec list and identify possible vendors of implementation
- For each format you identify known licensors. Nobody will never
commit to an exhaustive list of patent holders, as some may come up and nobody knows it. Be sure to identify the ones that count for you.
![Page 35: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/35.jpg)
7.3 How does that work? Step 2
2. You get the codec under the appropriate license
You buy the implementation from a codec vendor
You develop or make develop for you
- Some codecs may require to acquire a specific license for implementation different from the one for use or distribute
- Some may be developped following public specs even if it needs then a license to be distributed
You buy from codec licensors
- Some licensors provide their own implementation
![Page 36: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/36.jpg)
7.3 How does that work? Step3
3. You get the appropriate patent license
Main patent licensors will require you first to define
- The device: hand held, Desktop like, embedded
- The target : internal use, bundle and resell, ODM, OEM, Software vendor,…
- For the entire system or for only one application?
Depending on this you may have different licenses. Be careful you may have to sign different licenses if you want to cover different use
![Page 37: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/37.jpg)
7.4 Codec implementation Business model
It completely depends on the software vendors
It can be a fix fee or a per unit fee, or even a buy out
It will often come as a normal software license, with an offer on support services
It will mainly be binary
![Page 38: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/38.jpg)
7.5 Patent license Business model
It is usually based upon a per quarter / semester / year royalty reporting per unit (reporting and payment after sales)
Some propose % of revenue generated on a service (for profesionnal application / streaming servers,…)
Some propose annual caps or volume discount (ie. MPEGLA) that can be paid upfront
![Page 39: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/39.jpg)
7.4 Patent license costs
It depends on Formats
Sometimes it depends on the use Systeme wide or per application (ie: Via licensing)
It globally goes from $0,10 to $2 per unit
Price per unit is usually a price per machine or per application
Price may differ depending on the type of device (ie: Desktop like, hand held, ...)
![Page 40: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/40.jpg)
7.4 Patent license costs
Some require up front fee, or minimum fee per year whatever the units sold
Some propose a one year fee for one application for PC software (ie: Via licensing)
Some propose free units for a certain volume
![Page 41: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/41.jpg)
7.6 What process to follow? STEP 1 DEFINE THE SCOPE Device makers Enterprise
Target users Market target Employees
Users behaviour Mobile, home, work, perso,….
Work, which division, perso too?
HDW/OS/Framework/Player used
Cross platform? Hardare codecs?
Linux distro by default?
Source of content Depend on market May depend on division
Features supported Decode? Encode? Decode? Encode?
Formats needed User consumption depend on known sources
User consumption fixed by the company
Quantity
Distribution volume forecast
Number of users
Country Distribution Sites (if multinational)
![Page 42: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/42.jpg)
7.6 What process to follow?
STEP 2 TESTING Device makers Enterprise
Evaluation Testing in target environment
Test or buy trial versions
STEP3 FORMALIZE
Product license
Reseller agreement to bundle and distribute Or pay for development
Site licensing agreement for use
Business model options Per unit royalty fee Annual fee Buy out
One time fee license + support agreement
![Page 43: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/43.jpg)
8 – Market solutions
![Page 44: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/44.jpg)
Solution providers
License a full proprietary solution (application): Corel, Cyberlink, Realnetworks,
Get proprietary codecs implementations: Fluendo, Corecodec, Rovi (main concept), Xvid, Packet video, Dolby, DTS, H264 licensing, Enthropy wave,…
+ Get patent licenses: Microsoft, MPEGLA, Via licensing, Dolby, DTS, VoicAge,Technicolor,…
Get a packaged product one-stop-shop on Gstreamer: Fluendo
![Page 45: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/45.jpg)
Example: Gstreamer
Hardware platform
Operating system
Gstreamer
Media Players (A/V, DVD,…)
Streaming, Adaptive streaming, transcoding
Codecs, containers, muxers, demuxers
Specific elements HDW acceleration
Streaming server Video editors
Rendering Post processing, audio/video edition
Video conferencing
Stable, supported
Modular
Cross platform (Gstreamer SDK)
Wide coverage
Widely used
LGPL V2
Gstreamer.net and gstreamer.com
![Page 46: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/46.jpg)
Conclusion Linux growth introduces specific consideration of multimedia
When talking about legal issues on linux, we often open Pandora’s box, but it should not be a big deal, and solutions can be found.
Now solutions are available meanwhile things on pure legal side may take time to evolve.
The most important may be as usual to know what we do, to rely upon experts who exist, and to be able to measure the risks we take.
To feel comfortable, work with lawyers and developers and believe the ecosystem we are part of, brings value, and helps also make things evolve enabling all types of needed configurations. So get in touch with the appropriate partner and you will find solutions.
![Page 47: or how to handle multimedia on linux](https://reader030.vdocument.in/reader030/viewer/2022012704/61a65aa31c0e2f24c52ea374/html5/thumbnails/47.jpg)
Contact
Muriel Paumier-Moscardini
Skype: muriel.moscardini
Download slides on http://events.linuxfoundation.org/events/collaboration-summit/slides