navi applications and davinci™ technology
TRANSCRIPT
01001000100001001000100000110000001000001100
NAVI Applications andDaVinci™ Technology
(1) Networked Audio and Video Innovation
Jean-Michel MercierApplication [email protected]
2
Agenda
Introduction
DaVinci™ Technology and NAVI Requirements
Software design for NAVI on DaVinci
Conclusion
3
Introduction
Market requirementsGood quality and low bandwidthLow latencySupport of any type of videos sourceStandard streaming protocolsIntelligence on the deviceEasy software designFast track to market
4
IntroductionDaVinci is a perfect chip for NAVI
DSP for video compression and analyticsARM for streaming, recording and user interface
Example of existing application on mixed architecture
101011…
101011…
ARM
5
Agenda
Introduction
DaVinci™ and NAVI Requirements
Software design for NAVI on DaVinci
Conclusion
6
Codec Offer from TI and 3PVideo
H.263 prof. 0 Encoder + DecoderMPEG-2 MP@ML Encoder + DecoderMPEG-4 SP and ASP Encoder + DecoderH.264 BP/MP Encoder + DecoderWM9V MP EncoderWM9V MP/AP/VC-1 DecoderJPEG/MJPEG Encoder + Decoder
AudioMPEG Audio Layer 1,2,3 Encoder + DecoderAAC LC/HE Encoder + DecoderWM9A / WM8A Encoder + DecoderG.7xx Encoder + Decoder
OthersDe-interlace filterResize filter
7
MPEG-4 / H.264 ComparisonPSNR vs Bit rate
30.00
32.00
34.00
36.00
38.00
40.00
42.00
44.00
46.00
48.00
0.00 500.00 1000.00 1500.00 2000.00 2500.00 3000.00 3500.00 4000.00
Bit rate (bps)
PSN
D (d
B)
h264mpeg-4
2000 kbps1600 kbps
20% gain in efficiency
43dB41.8dB
8
MPEG-4 / H.264 ComparisonEncoding Time vs PSNR
0.00
5.00
10.00
15.00
20.00
25.00
0.00 10.00 20.00 30.00 40.00 50.00 60.00
PSNR
Enco
ding
Tim
e
h264mpeg-4
41.8dB
9,5 ms
18 ms
9
Codec Choice: Warnings
Not all implementations are equivalent
Performance is « easy » to haveWhat has to be sacrificed then?
DecoderCompliance
EncoderQuality (motion estimation, decision, regulation is
heart of know-how)
10
Interlace issues
original
11
Interlace issues
MPEG-4 SP
12
Interlace Issues
de-interlaced MPEG-4
original MPEG-4
13
Interlace Issues
Video
14
Do I Need to De-Interlace?
Display is 90% PC Progressive displayVery disturbing artifact (comb artifact)
on PC: overload PC (lot of channels)at Encoder: improve efficiency (MPEG-4)
No needPCEncode inH.264
Encoderimprove coding
Encoderimprove coding
Encode inMPEG-4 SP
Display on TVDisplay on PC
15
Noise Filtering
Encoder will try to encode noiseMore data to go through same bandwidthQuantize more lower qualityRemove noise enhance coding efficiencySimple 3x3/5x5 not much efficient3D = spatial + temporal
• Remove noise• Remove flicker Video
16
Causes for Latency
Pipeline unit is frame
Capture N
Encode N
Transmit N
Decode N
Display N
N+1
N+1
N+1
N+1
N+1
N+2
N+2
N+2
N+2
N+2
N+3
N+3
N+3
N+3
Total latency = few frames
17
Causes for Latency
B-frames encoding is non-causal
N N+1 N+2 N+3 N+4 N+5 N+6 N+7
I B B P B B P B
N N+3 N+1 N+2 N+6 N+4
N+8 N+9 N+10
B I B
N+5 N+9 N+7
N N+1 N+2 N+3 N+4 N+5 N+6 N+7
Total latency = 3 frames
18
Causes for LatencyI frames are bigger than P frames
N N+1 N+2 N+3 N+4 N+5 N+6 N+7 N+8 N+9 N+10
Num
ber o
f bits
per
fram
e
Target bit rate / frame rate
Effective line bandwidth
Transmission latency
Enco
der o
utpu
tD
ecod
er
N N+1 N+2 N+3 N+4 N+5 N+6 N+7 N+8 N+9
19
Error ResilienceFor how long can you stop watching?
CausesNetwork Loose packets (UDP)Bit errors (radio)
ResultCorrupted imageStalled video
Wait for I to reconstructI period trades-off: better resilience, less efficiency
Key P P P P P P P Key P P P P
Key P Key P P P PStalled video
20
Error Resilience ToolsMPEG-4
RVLC: Reverse VLCData partitioningResync markers & Video Packet
H.264Slices
Video
Normal encoded frame Slice/packets encoded frameLost or damaged
decoded not decoded
Lost or damaged
decoded decoded
21
Agenda
Introduction
DaVinci™ and NAVI Requirements
Software design for NAVI on DaVinci
Conclusion
22
Software Design
Dual coreShared memoryShared peripherals
23
Software Design
Codec EngineDaVinci made easy for everyone
Assumption #1Don’t touch the DSP if you don’t need
Stay on LinuxAll peripherals use standard drivers: sockets, file system, serial, V4L2 …
DSP is a “black-box” coprocessorForget about it, call API
Use open framework for A/V applicationPlatform portability, reference applications
24
xDAIS Algorithm Standard
Standard API for resources management• algNumAlloc : how many resources?• algAlloc : describe resources• algInit : init from those resources
Proprietary API for processing• Proprietary configuration and parameters• Proprietary function call
Do not ease switching from MPEG-4 to H.264Do not ease to switch from VENDA to VENDB
AAC DecxDAIS Prop
MP3 DecxDAIS Prop
MP4 DecxDAIS Prop
MP4 DecxDAIS Prop
Vendor A Vendor B Vendor A Vendor B
25
xDM – Digital Media ExtensionDefine 8 standard classes of algos:
Video – Imaging – Speech – Audio: VISAEncoders and decoders
For each class, defines:Configuration structureFunction calls with full prototype
Application writer benefits:Plug’n’Play architectureEach change of codec/provider
AAC DecxDAIS AUDDEC
Vendor A
MP4 DecxDAIS VIDDEC
Vendor A
MP3 DecxDAIS AUDDEC
Vendor B
MP4 DecxDAIS VIDDEC
Vendor B
26
Codec EnginePre-built codec combo/serverUse DSP algs from ARM at no painTransparent Remote Procedure CallTransparent mix of host (ARM) or DSP algsDeal only with Linux programming
XDM AUDDEC
MP3 Dec
XDM VIDDEC
MP4 Dec
XDM VIDDEC
MP3 Dec
XDM AUDDEC
AAC Dec
Frameworkcomponents
ACPY3
DMAN3
DSKT2
Node
DSP/Bios
DSP Server
Application
VIDDEC API
XDM SPHDEC
G711 Dec
AUDDEC API
Codec Engine
ARM Combo
SPHDEC API
DSP/Link
CMEMdriver
27
Codec Engine
All you have to remember is
Application
VIDDEC API
AUDDEC API
Codec Engine
ARM Combo
SPHDEC API
CMEMdriver
28
Using Codec EngineCERuntime_init();myCE = Engine_open( “myengine”,…)myEnc = VIDENC_create(myCE, “mpeg4”,…)allocate_buffers()do {capture_frame( &frame );VIDENC_process( myEnc, &frame, &bits…)send_stream( &bits )
} while (!end);VIDENC_delete( myEnc );Engine_close( myCE );
29
See Live Code in Action
Browse a sample applicationRun the demoChange MPEG-4 to H.264Rebuild and run again
30
Need for an A/V Framework
Codec Engine handles DSP callsNeed a solution for
A/V SyncStreaming/Container handling
Linux on ARM give access to a lot of solutions
OpenML http://www.khronos.org/openml/GStreamer http://gstreamer.net
31
What is GStreamer?
Pipeline oriented frameworkGraph based design: connect boxes
Content agnosticGStreamer core is the enginePlug-ins handles the workRuns on desktop or embedded Linux Only depends on gLibc
Already used by a lot of desktop applications
32
How Does It Work?Source filters get data from the real world
File parsers: AVI, MP4, ASF readersNetwork client: UDP, TCP, RTP, RTSPCapture: V4L2, OSS
Sink push data to the real worldFile writersNetwork server/streamerRendering: Frame buffer, OSS
Nodes process dataElements are connected through padsData types are negotiatedBuffers are transported through pointersCopy is avoided as much as possible
33
A/V Sync
Flexible clock schemeSink and Source can be master or slaveExample: network player
Demux is masterSinks are slaves
Networksource
TSDemux
Video Dec
Audio Dec
FB sink
OSS sink
clock
Elementary stream packets with timestamps
raw network stream
34
Tools and Utilities
gst-editorGraphical graph editorExport design in XMLPlay graph in GST-launch
gst-launchCommand line utilityCreate and play a graph by command line or XML
gst-launch “mysong.mp3” ! mp3dec ! osssink
gst-inspectList installed plug-ins
gst-registerRegister plug-ins capabilities
35
Agenda
Introduction
DaVinci™ and NAVI Requirements
Software design for NAVI on DaVinci
Conclusion
36
ConclusionCodec Engine makes using DaVinci easierFramework helps for fast track to marketSkilled 3P will help you go thoughATEME specifics for DaVinci
HW and SW expertiseOwn enhanced codec or complementary IPs
• H.264 Main Profile Encoder• MPEG-4 ASP Encoder• MPEG-4 ASP & DivX Decoder• Streaming protocols
A/V application and GStreamer expertise
