Aug 25, 2005page1 Aug 25, 2005
Integration of Advanced Video/Speech Codecs into AccessGridNational Center for High Performance Computing
Speaker: Barz Hsu
Email: [email protected]
Aug 25, 2005page2
Outline
• Overview
• NCHC-VIC:– Introduction to MPEG4/H.264 codec– Codec Optimization
• NCHC-RAT:– Introduction to Speex codec– Echo Cancellation and Noise Control
• Future Work
Aug 25, 2005page3
Overview
• AccessGrid is scalable collaborative environment.• AccessGrid is complicated and not easy to use.• Major complexity comes from the two core media
– vic (video) and rat (speech)
• Most of video/speech codecs inside AccessGrid are out-of-date.
• Integrate state-of-art MPEG4/H.264 codecs for video coding, and Speex codec for speech coding.
• High quality and low latency
Aug 25, 2005page4
MPEG4 Codec
• Related Open Source Project– FFmpeg, xvid, and sklmp4
• FFmpeg/libavcodec– High performance on CPU usage– Support various codec
• MPEG4, MPEG2, MPEG1• H.263++, H.263, H.261
– Provide error resilience tools• slices, data partition
– Cross Platform• Linux, Windows, and Mac OS
Aug 25, 2005page5
H.264 Codec
• NCHC-VIC based on X264 project of VideoLAN.• X264 is widely used by other OpenSource projects,
ex., FFMpeg, Mplayer, VideoLan.• X264 is the only free software among current H.264
codecs.• The project is still under aggressive development.
– http://developers.videolan.org/x264.html
Aug 25, 2005page6
H.264 Codec (cont’d)
• Adopt new techniques to effectively reduce bandwidth requirement and provide better visual quality– Smaller block division– Multiple prediction modes– Multiple reference frames– Finer motion estimation (Quarter-Pel)– Deblocking filter
• More computation intensive (5 to 10 times of MPEG4 ASP)
Visual Comparison between H.264 and MPEG4 (static video with 128kbps)Visual Comparison between H.264 and MPEG4 (static video with 128kbps)
Aug 25, 2005page7
Codec Optimization
• Multithread Encoding– Take advantage of symmetric multiple processor
architecture
• Pipeline the latency of transmission and encoding
• SIMD Instruction Sets– Optimized by MMX, SSE/SSE2/SSE3
Aug 25, 2005page8
H.264 Optimization
• Due to the need of huge computation for H.264 encoding, we close some advanced features to meet the requirement of the real-time video conference– Using CAVLC instead of CABAC– Larger block division– Simpler block mode prediction
Aug 25, 2005page9
FFMpeg MPEG4 vs x264 (RD Curve)
x264: cavlc, only 16x16 block
Aug 25, 2005page10
FFMpeg MPEG4 vs x264 (Speed)
x264: cavlc, only 16x16 blockCPU: P4 2.8GHzRAM: 512MB
Aug 25, 2005page11
Visual Comparison between H.264 and MPEG4 (static video with 128kbps)
H.264 MPEG4
Aug 25, 2005page12
Visual Comparison between H.264 and MPEG4 (dynamic video with 128kbps)
H.264 MPEG4
Aug 25, 2005page13
Full Screen Snapshot
Aug 25, 2005page14
Experimental Video
Aug 25, 2005page15
NCHC-VIC for Win32 Port
• Based on Linux Version• Update IPv6 code• Compiled by Visual Studio 6• Some parts are compiled by gcc and then link.
Aug 25, 2005page16
NCHC VIC for Win32 Screenshot
Aug 25, 2005page17
Speex Codec for Speech
• NCHC-RAT adopts Speex open source project• Paten-free audio compression format• Narrowband (8 kHz), wideband (16 kHz), and ultra-
wideband (32 kHz) • Subband coding to reduce computation overhead• Packet loss concealment• Intensity stereo encoding• Project website,
– http://www.speex.org/
Aug 25, 2005page18
Echo Cancellation and Noise Control
• Adaptive filter to model acoustic echo path– Multidelay Block Frequency Domain Adaptive Filter
• Double Talk Detector– Multidelay Double-Talk Detector Combined with the MDF Adaptive
Filter
• Noise Control– Mexico-hat low pass filter
Aug 25, 2005page19
Echo Model
Aug 25, 2005page20
NLMS Algorithm
• Normalize Least Mean Square Algorithm
Aug 25, 2005page21
Future Work
• Session Description Protocol• Session Announcement Protocol • Advanced User Interface• Zero-Configure Cluster Decoding
Aug 25, 2005page22
Thanks for
your attendance