mpeg4 codec for access grid
TRANSCRIPT
1/23/2005page1 11/11/2004
MPEG4 Codec for Access GridsNational Center for High Performance Computing
Speaker: Barz Hsu
Email: [email protected]
1/23/2005page2
Outline
• Definition of HDTV• Brief Introduction to MPEG-4 Standard• History and Problems of VIC• Contributions
1/23/2005page3
What’s High Definition Video?
• Frame Resolution: >= 720x480 or 640x480
• Frame Rate >= 20 fps
• Peak Signal to Noise Ratio (PSNR)>= 30 dB
1/23/2005page4
Another Key Factors for Access Grids
• Real-Time– Minimize latency & jitter
• Error Resilience– RTP over UDP is unreliable
• Large Scale Video Conferencing– Selection of suitable coding tools– Optimization
• Adaptive– network bandwidth– computing resources
1/23/2005page5
Status of MPEG4
• Part 2: Advanced Simple Profile (ASP)• Part 10: Advanced Video Coding (AVC or H.264)
• Match MPEG-2 quality at up to half data rate• Deliver excellent video quality across entire
bandwidth spectrum – from 3G to HD– (from 40Kbps to 10MBps)
1/23/2005page6
Performance Comparison
1/23/2005page7
Interoperability Issues
• Communicate with Polycom, iChat AV (Tiger)• Transport Stream
– RFC 3016: RTP Payload Format for MPEG-4 Audio/Visual Streams.– RFC 3640: RTP Payload Format for Transport of MPEG-4
Elementary Streams– MPEG2TS
1/23/2005page8
VIC: Video Conferencing Tool
• Developed by Network Research Group in U.C. Berkeley
• Mbone Tools used by Access Grid, inSORS, and VRVS– vic for video and rat for audio
• Based on RTP/RTCP to provide real-time video• Support Multicast/Unicast Protocol• Hybrid of Tcl/Tk and C++ • Supported codecs
– H.261, H.263, H.263++, MJPEG, nv, and etc.
1/23/2005page9
The Problems of VIC
• Hard to maintain
• Lack of high-quality video codec– only H.261 is relative useable
• Limitation on frame resolution– Only QCIF (176x144), CIF (352x288)
• Limitation on viewing resolution• Lack of exhaustive optimization on codec, render, grabber,
and etc
1/23/2005page10
Our Goal
• Broadcast in Television Quality– Leverage with Existing Open Source Resources
• VLC, MPlayer, and FFmpeg– Providing “Sense of Presence”– As Real-Time As Possible– High-Definition, High Quality– Incorporate with up-to-date codecs such as MPEG2, MPEG4
(ASP), H.264– Less Compression Artifact– Error Resilience/Concealment
1/23/2005page11
MPEG4 Codec
• Related Open Source Project– FFmpeg, xvid, and sklmp4
• FFmpeg/libavcodec– High performance of CPU usage– Support various codec
• MPEG4, MPEG2, MPEG1• H.263++, H.263, H.261
– Provide error resilience tools• slices, data partition
– Cross Platform• Linux, Windows, and Mac OS
1/23/2005page12
Benefits from MPEG4 codec
• Support arbitrary sizes of a video stream – in contrast to H.261– the largest size depends on your video grabber card and camera
• Less obvious block artifacts
1/23/2005page13
MPEG4 Performance
• Typical bit-rate is 1Mbps– 720x480 (NTSC) with 25 frames/sec and PSNR value is about 40 dB
• Consistent bit-rate– adjusting bit-rate only affects to the quality; not influence the frame-
rate• Up to 40 CIF, or 10 NTSC video streams.
1/23/2005page14
Deinterlace
• Convert interlaced video into progressive one
1/23/2005page15
Efficient Color Conversion
• VIC didn’t give correct color representation.
• Efficient Color Conversion– Grabber (YUYV) to Encoder (YUV420)– Decoder (YUV420) to Display (RGB15/RGB16/RGB24)
1/23/2005page16
Scaling Viewing Windows
• Original VIC– only support limited sizes of viewing windows– based on unrealistic pixel replication method
• NCHC-VIC– scaling viewing windows arbitrarily without too much overhead– interpolation by 9 tap FIR filter– hardware assisted by Xvideo extension in XFree86– full screen mode
1/23/2005page17
Full Screen Snapshot
1/23/2005page18
Real-Time Video Streaming
• Real-time– the compressed data are transferred at a speed that matches the
coded video source rate.
• Latency, Jitter– average end-to-end delay– delay variation
1/23/2005page19
Error Resilience
• Error Progratation
1/23/2005page20
Error Resilience (cont’d)
• Resynchronization marker– the data between the error location and the first following
resynchronization marker would have to be discarded.
• Data Partition– motion vectors and related syntactic elements are transmitted first,
followed by syntactic elements like MB headers, motion vectors, and DCT coefficients.
1/23/2005page21
Minimize Latency
• Multithread Encoding– Take advantage of symmetric multiple processor architecture
• Pipeline the latency of transmission and encoding
• SIMD Instruction Ses– Optimized by SSE2/SSE, MMX
1/23/2005page22
Future Work
• Support H.264 codec• Support MPEG-4 High Efficient AAC (HE-AAC)
1/23/2005page23
Conclusion
• Project Website:
• http://vic.nchc.org.tw
1/23/2005page24
Thanks for
your attendance