Digital Video Processing and

Analysis : Introduction

Cheolkon Jung

School of Electronic Engineering

Xidian University, China

8 PhD & 17 Master Students

(Alumni: 3 PhD, 44 Masters)

Research Lab (2010~Present) 2

Advanced digital media

High dynamic range (HDR) video

Ultra high definition (UHD) video

Stereoscopic 3D (S3D) video

Multimedia big data

Surveillance video

Main research topics

Image restoration & Super-resolution

S3D video processing & communications

Computational photography & Display enhancement

Multimedia annotation, retrieval & security

H.265 and perceptual video coding

Deep learning for computer vision (2015~)

Research Interests 3

Instructor Information 4

• Instructor: Cheolkon Jung

• Office: #203, 100 Building

• Email: zhengzk@xidian.edu.cn

• Webpage: http://web.xidian.edu.cn/zhengzk

http://see.xidian.edu.cn/media

• Time:

• Formal: Wednesday PM 4:30-6:00 (Every week);

• Informal: Appointed time;

• Place: #210, 100 Building

http://web.xidian.edu.cn/zhengzk

Images 5

Pixel

(Picture Element)

Image: 2D vector

Videos 6 Image sequences

Difference Between Images and Videos 7

• Images • Shape

• Color

• Texture

• Videos • Shape

• Color

• Texture

• Motion

Motion 8 Motion Estimation

Motion Compensation

3D Videos 9

2009

3DTV

At Home

Theater

3D Videos 10

3D Display With Glasses

• Images • Shape

• Color

• Texture

• Videos • Shape

• Color

• Texture

• Motion

• Depth in 3D videos

Difference Between 2D and 3D Videos 11

• Human 3D: perceive depth information from two slightly different views by two

eyes.

Depth Perception

Depth 12 Depth Estimation

3D Reconstruction

• Exactness • Perfect reproduction without degradation

• Perfect duplication of processing result

• Convenient & powerful computer-aided processing • Can perform rather sophisticated processing through hardware or Software

• Even kids can do it!

• Easy storage and transmission • 1 CD can store 1 video or hundreds of family photos!

• Paperless transmission of high quality photos through network within seconds

Films Sensors

Digital Videos 13

• Source of videos: • Captured with an analog or digital TV/movie camera

• Time-based sequences: radar, satellite, or drones

• Video units:

Digital Videos 14

• Popular mobile imaging devices:

• Smart phone, video camera, personal media player (PMP), etc

• Storage size expansion:

• HDD, CD, DVD, flash memory, etc

• Digital videos everywhere:

• Social, commercial, industry and military applications

• Entertainment, education, medicine, databases, security, etc.

• High-speed computing environments in most digital devices

Digital Videos 15

Digital Video Processing & Analysis

• Video compression

• Video enhancement

• Video surveillance

• Video content analysis

• Video indexing & retrieval

• Object detection & recognition

• Motion estimation

• Stereo video processing

16

IEEE ACCESS 2018

Low Light Video Enhancement 17

HDR Video Compression

IEEE VCIP 2015 & 2016

SIGNAL PROCESS: IMAGE (2018)

Free energy

principle

Perceptual Block Merging

Whole Framework of the Proposed Method

Perceptual Encoder

Calculate

FEJNDCU &

FEJNDframe

Adjust RD cost

minimization FEJND map Block merging

Perceptual

weight

18

IEEE TMM (2014)

Example

Learning

2D to 3D Video Conversion 19

IEEE ICASSP 2016

IEEE TCSVT (2018)

Ambient Light Adaptive Video Enhancement

Edge Guided Depth Image Super-resolution (DISR) Using CNN

DISR by HR edge prediction, instead of HR texture prediction IEEE ICASSP 2018

Deep Learning for Machine Intelligence 21

Main Contents 22

• Video Basics

• Video Modeling

• Deep Leaning for Visual Understanding

• Motion Estimation

• HEVC for Video Compression

• Stereo and 3D Videos (S3D)

• 360 Videos

• Text books: TPAMI/IJCV/TIP/TMM/TCSVT/CVPR/ICCV/ECCV

Video Processing and Communications (Wang et al.)

Student Projects:

- Paper selection

- One-page proposal

- Proposal presentation

- Reproduction of methods

- Final presentation

- Demonstration

- Technical report and manual

Source codes should not be open!

* Quiz: 5 times