vic development national center of high-performance computing barz hsu...
TRANSCRIPT
VIC Development
National Center of High-performance ComputingBarz Hsu ([email protected])
15 January 2007
Start Point
Intend to develop high visual quality video tool
Based on AccessGrid vic-2.8ucl1.1.3-3 Initial release of NCHC-VIC with
MPEG4 codec Up to NTSC/PAL resolution High-efficient render Fullscreen mode Linux version only
Timeline
Develop NCHC-VIC from January 2004 to August 2005
Develop GStreamer-based media tools from June to August 2006
Merge NCHC-VIC to UCL code base from September to December 2006
Outlines
Review NCHC-VIC Porting Issues Demos GStreamer based new media tools
NCHC-VIC Features
Codec MPEG4 H.264
Render Xvideo Swscale Fullscreen
Postprocess Deinterlace Deblocking Colorspace conversion
MPEG4
FFmpeg encoder/decoder ME (motion estimation)
EPZS algorithm QPEL
CBR (constant bit rate) GOP (group of picture): 15 Low latency
No B-frame rtp_callback Old_latency = encoding_latency + tramsmission_latency New_latency =~ encoding_latency/5 + tramsmission_latency
Low CPU usage
MPEG4 (cont’d)
IETF RFC 3640 RTP payload type 45 Rearrangement RTP packet buffer for only on
e frame Error resilience/concealment Compatible with VP Handy to use other codec such as
MPEG2, H.261, H.263, H.263+, DV, MJPEG, Theora
H.264
X264 encoder Turn off CABAC
save computation up to 30%, increasing coding bit rate around 15%
Inter-ME X264_ANALYSE_PSUB16x16 RC_METHOD X264_RC_ABR i_direct_mv_pred X264_DIRECT_PRED_NONE Deblocking filter No B-frame GOP is 20~50 frames
H.264 (cont’d)
FFmpeg encoder FFmpeg decoder Each RTP packet is a collection of NAL units IETF RFC 3894? Compatibility?
Quicktime、 VLC、 inSORS Contributed by Champ Yeh and Barz Hsu fro
m NCHC
Advanced UI
Contributed by Morris Lin from NCHC
Accelerated Render
XVideo+Shared memory Software scaling CPU capability runtime detection Win32/Win64, Linux, MacOSX Colorspace Conversion
SWSCALE: Software Scaling
Libswscale Shipping with FFmpeg and MPlayer
Arbitrary resizing Bilinear interpolation Colorspace conversion
YUV420 to RGB32/RGB24/RGB16 for render YUYV to YUV420 for grabber
CPU multimedia instructions Intel: MMX/MMX2/SSE/SSE2 AMD: 3D Now/3D Now2 PowerPC: Altivec
Runtime CPU capability detection Speed up to 10X than pure C code
Xvideo Extension: Graphic Accelerated Render XFree86 X11 Extension Supported by Nvidia and ATI graphic card Direct YUV420 render
No colorspace conversion needed Perform image scaling and colorspace conversion b
y graphics Most efficient way Up to 40~100 videos at same time Automatically use swscale if xvideo no longer availa
ble
Fullscreen
Boardless Light weight If graphics accelerated Dual head is fine Only for Windows and Linux
Fullscreen
Postprocess
Provide better visual quality Remove coding artifact by specific encoders Deinterlace Deblocking Dering Automatic level
Use higher level of postprocessing if CPU usage is not full
Linux Port
Nothing special! File size of statically linking binary is around
10MB Most robust and efficient version V4l grabber bugfix
Windows Port
Visual Studio 2003 Inline assembly syntax different from GCC
AT&T v.s. INTEL Create DLLs and LIBs (import library) under
Mingw/Cygwin ffmpeg, x264, libswscale, libpostproc Is it possbile to use sttatical linking?
S-video, Composite input support correct framerate for win32 DS grabber
MacOSX Port
Merge AG Aqua branch http://www.vislab.uq.edu.au/research/accessgrid/software/aquavic/
Aqua and Quicktime framework Without X11 Use Aqua Tcl/Tk 8.4 shipped with MacOSX Native look Grabber-macosx
iSight, QuickCam support More robust than AG Aqua branch
MacOSX Port (cont’d)
Compilation Steps (Linux)
Check out source code svn co https://frostie.cs.ucl.ac.uk/repos/mmedia/vic/branches/mpeg4
Configuration ./configure
Check out FFmpeg with revision 7110 (Dec, 2007) svn co -r 7110 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg ./configure --enable-gpl --enable-swscaler --enable-pp
Check out X264 with revision 600 (Dec, 2007) svn co -r 600 svn://svn.videolan.org/x264/trunk x264
Options --with-ffmpeginc --with-ffmpeglib --with-x264inc --with-x264lib
Build make
Compilation Steps (MacOSX)
Install TclTkAquaBI for Tcl/Tk 8.4.10 http://tcltkaqua.sourceforge.net/
Configuration and Building Steps are same as Linux platform
Compilation Steps (Windows) Configuration
./configure Check out FFmpeg with revision 7110 (Dec, 2007)
svn co -r 7110 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg ./configure --enable-mingw32 --enable-memalign-hack --enable-swscaler
--enable-gpl --enable-shared --disable-static Check out X264 with revision 600 (Dec, 2007)
svn co -r 600 svn://svn.videolan.org/x264/trunk x264 ./configure --enable-shared Patch Makefile for generating DLL add the 3-6 lines in SONAME targe $(SONAME): .depend $(OBJS) $(OBJASM) $(CC) -shared -o $@ $(OBJS) $(OBJASM) -Wl,-soname,$(SONAME)
$(LDFLAGS) ifeq ($(SYS),MINGW) $(CC) -shared -Wl,--output-def,libx264.def -o libx264.dll $(OBJS) $(O
BJASM) lib /machine:i386 /def:libx264.def endif
Compilation Steps (Windows) (cont’d)
Copy LIBs for VS2003 avcodec-51.lib avutil-49.lib libx264.lib postproc-51.lib
swscale-0.lib Copy DLLs to the path of excuteable
avcodec-51.dll avutil-49.dll libx264.dll postproc-51.dll swscale-0.dll
Tcl/Tk 8.4
Multi-threading Tcl_init.tcl and tk_init.tcl files required Further work needed to embed the init scripts
to binary It’s critical to call Tk_FindExecuteable() to loc
ate init scripts for Windows platform
Open Issues?