On the use of hierarchical prediction structures for efficient summary generation of H.264/AVC

bitstreams

Luis Herranz, Jose´ M. Martı´nez

Image Communication 2009

Outline

• Introduction• System Framework– Skimming Unit (SU)– Coding order– Generation of the summary– Presentation of the summary

• Experiment

Introduction

• MPEG-2 prediction structure

Introduction

• Hierarchical prediction structure in H.264/AVC using P frames (delay 0)

Introduction

• Hierarchical prediction structure in H.264/AVC using B frames (delay 8)

System FrameworkAnalyzing input bitstream Generating the summarized bitstream

# frames selected

Skimming unit (SU)

-The selection of SUs-Temporal level

Skimming Units

Skimming Unit (SU)

Temporal layerIndex

𝑆𝑠𝑜𝑢𝑟𝑐𝑒 𝐹𝑟𝑎𝑚𝑒

1

𝑆𝑠𝑜𝑢𝑟𝑐𝑒=(𝐺0𝑇 ,𝐺1

𝑇 ,…,𝐺𝑀−1𝑇 )

Skimming Units

Overlapped SUs

Generation of the Summary

• Bit stream adaptation guided by skimming parameters

Presentation of the Summary

• Constant frame rate• Variable frame rate– Signaling the changes of frame rate– Using a predefined set of SPSs

Summarization Modalities

• Image storyboard: • Video Skim: • Frame dropping• Fast preview

Frame selection Skimming curve

Image Storyboard

• Clustered using K-means algorithm

Video Skimming• Face detection• Moving median filter

(a) Frames with face detected

(b) Skimming parameters -- 32 frames per GOP

(c) Skimming parameters -- 2 frames per GOP

Fast Preview

• Motion activity• Median filter

(a) Filtered activity

(b) Skimming parameters -- 32 frames GOP

(c) Skimming parameters -- 2 frames GOP

Subjective Evaluation

Visual distortion Semantic distortion

Performance Evaluation

• Coding efficiency and processing time

Performance Evaluation

• Bit stream extraction– Low computational cost– Faster generation of the bit stream

skim storyboard

Fast playback