error control techniques for interactive low-bit rate video transmission over the internet
DESCRIPTION
Error Control Techniques for Interactive Low-bit Rate Video Transmission over the Internet. Injong Rhee Department of Computer Science North Carolina State University. Video Conferencing over Packet-Switching Networks. Packet Loss. ¡¡ Compressed video frame is packetized and transmitted. - PowerPoint PPT PresentationTRANSCRIPT
Error Control Techniques Error Control Techniques for Interactive Low-bit for Interactive Low-bit Rate Video Transmission Rate Video Transmission over the Internet.over the Internet.
Error Control Techniques Error Control Techniques for Interactive Low-bit for Interactive Low-bit Rate Video Transmission Rate Video Transmission over the Internet.over the Internet.Injong RheeDepartment of Computer ScienceNorth Carolina State University
Video Conferencing over Packet-Video Conferencing over Packet-Switching Networks.Switching Networks.Video Conferencing over Packet-Video Conferencing over Packet-Switching Networks.Switching Networks.
Packet Loss
•¡¡ Compressed video frame is packetized and transmitted.•¡¡ Packet loss can occur quite frequently due to congestion •and user mobility.
Motion Compensated CodingMotion Compensated CodingMotion Compensated CodingMotion Compensated Coding
IntraFrame(I-frame)
InterFrame(P-frame)
Temporal Dependency
• Most video codecs use it.• Each P-frame depends on its previous frames.• I-frame is periodically transmitted.
When a packet is lost,….When a packet is lost,….When a packet is lost,….When a packet is lost,….
IntraFrame(I-frame)
InterFrame(P-frame)
Packet Loss
Error Propagation
• Conventional Solution : Transmit I-frame more often.• However, compression efficiency reduces.
Myth: retransmission is not Myth: retransmission is not useful for interactive videouseful for interactive videoMyth: retransmission is not Myth: retransmission is not useful for interactive videouseful for interactive video
retransmit
NACK
lost
F0 F1 F2 F3 F1 (retransmit)
F0 F1 F2 F3
TIME
(F1 arrives after its display)
Conventionally, late packets are discarded.
Retransmission stops error Retransmission stops error propagation.propagation.Retransmission stops error Retransmission stops error propagation.propagation.
retransmit
NACK
lost
F0 F1 F2 F3 F1 (retransmit)
F0 F1 F2 F3
TIME
F1 is recovered here
Don’t discard: Use late packets to recover their reference frames
Handling retransmitted ‘late’ Handling retransmitted ‘late’ packets in decoderpackets in decoderHandling retransmitted ‘late’ Handling retransmitted ‘late’ packets in decoderpackets in decoder
Old Reference Frame
ReferenceFrameCurrent
Diff.erenceFrame +
Current FrameInfo
Retransmitted Information.
Inverse Quantization/DCTInverse Quantization/DCT
Motion Prediction Compensation
Motion Prediction Compensation
Display
Error PropagationError PropagationError PropagationError Propagation
Recovery from Error Recovery from Error PropagationPropagationRecovery from Error Recovery from Error PropagationPropagation
Error Recovery TechniqueError Recovery TechniqueError Recovery TechniqueError Recovery Technique
•Recovery from Error Spread using Continuous Update (RESCU)
•Focus on error spread/propagation
•Continuous Update– Retransmission (this paper)– Forward error correction
•Combined with Layer coding (this paper)
Extending Packet DeadlineExtending Packet DeadlineExtending Packet DeadlineExtending Packet Deadline
1Add more frame buffers at the decoder.
2Adjust Temporal Dependency Distance (TDD) of a frame.– TDD: frame intervals from that
frame to its temporally dependent frame.
Add more reference frame Add more reference frame buffers.buffers.Cascaded BufferingCascaded Buffering
Add more reference frame Add more reference frame buffers.buffers.Cascaded BufferingCascaded Buffering
•Minus: More computation and buffers at the decoder.
•Plus : No change in the codec and picture pattern.
Base Reference Frame
Reference Frame 1
Contains Full ImagePrediction Errors + MVsPrediction Errors + MVs
Reference Frame 2
Adjust Temporal Dependency Adjust Temporal Dependency
DistanceDistance Periodic TDDs (PTDD)Periodic TDDs (PTDD)
Adjust Temporal Dependency Adjust Temporal Dependency
DistanceDistance Periodic TDDs (PTDD)Periodic TDDs (PTDD)
•Changing Dependency.
• f
• PTDD can be dynamically adjusted.
+Reduced Buffers & computations.
—No protection for non-periodic frame.
Intra Inter Inter Periodic
Continuous UpdatesContinuous UpdatesRetransmission & Forward Error CorrectionRetransmission & Forward Error CorrectionContinuous UpdatesContinuous UpdatesRetransmission & Forward Error CorrectionRetransmission & Forward Error Correction
•Retransmission – Retransmit lost packets within
PTDD.– Retransmission delay requires
larger PTDD.
•Forward Error Correction– send parity packets over a PTDD
period.
Periodic frame Periodic frame
Parity Packet for periodic frame
Protecting non-periodic Protecting non-periodic framesframes RESCU + Quality Assurance LayeringRESCU + Quality Assurance Layering
Protecting non-periodic Protecting non-periodic framesframes RESCU + Quality Assurance LayeringRESCU + Quality Assurance Layering
• .
•HP stream is sent with forward error correction.
•RESCU contributes to reducing bit rates.
HP
LP
HP
LP
HP
LP
HP
LP
protecting non-periodic protecting non-periodic frames frames Change temporal dependencyChange temporal dependency
protecting non-periodic protecting non-periodic frames frames Change temporal dependencyChange temporal dependency
•Errors in non-periodic frames do not propagate.
•As PTDD increases,– Compression efficiency decreases.
Periodic Inter Inter Periodic
Compression EfficiencyCompression Efficiency(MPEG-4 Class A, AKIYO)(MPEG-4 Class A, AKIYO)Compression EfficiencyCompression Efficiency(MPEG-4 Class A, AKIYO)(MPEG-4 Class A, AKIYO)
AVG Bytes/Frame
AV
G P
SN
R (
dB
) Intra-H261
RESCU + QAL
RESCU
H.261
Error ResilienceError Resilience (H.261 v.s. RESCU) (H.261 v.s. RESCU)Error ResilienceError Resilience (H.261 v.s. RESCU) (H.261 v.s. RESCU)
RESCU + QALRESCU + QALRESCU + QALRESCU + QAL
Error ResilienceError Resiliencevarious loss groupsvarious loss groupsError ResilienceError Resiliencevarious loss groupsvarious loss groups
H.261
Intra-H.261
Loss Rate
PS
NR
RESCU + QAL
RESCU
ConclusionsConclusionsConclusionsConclusions
•RESCU accommodates delays in recovery (retransmission or FEC).
•error resilience for interactive video.
•high compression efficiency.– need I-frames less frequently.
•Good for bandwidth-constrained, lossy networks (e.g., mobile wireless networks).
•Much more work to be done.
Error ResilienceError ResilienceRetransmission over various network Retransmission over various network delaysdelays
Error ResilienceError ResilienceRetransmission over various network Retransmission over various network delaysdelays
50 ms100 ms
150 ms
200 ms
250 ms300 ms