technology day italy 4k broadcast chain - rohde & …± user datagram protocol: simple “send...

David A. Smith

February 2017

[email protected]

Technology Day Italy

4K Broadcast Chain

4k Broadcast Chain : RSI Technology Day

David A. Smith : February 2017 2

Overview

ı UHD Resolutions

WCG, HFR, and bits/pixel

ı High Dynamic Range Principles

Perceptual Quantisation and Hybrid Log Gamma

ı IP in the Studio

Sony Live, AIMS, ASPEN

ı 4k/HDR Workflow and Broadcast Solutions



Beyond Resolution

Ultra HDTV & HDR

ı With millions of pixels a few inches from your face

Better be glad that the signal's not interlaced

Don't you know it needs a big screen size

Real wide

Alright

You love the spectral distribution

Well you know

It's just green and blue and red

You think that it's a revolution

Well you know

It's just the latest hype instead



EBU UHD Standards



DC

I 4K: 4

096 x

2160

4k x 2k Ultra HDTV Format: UHD-1

ı UHD-1: 3840 x 2160

4 x HD Resolution (8 Megapixels)

ı DCI 4K: 4096 x 2160

Digital Cinema Initiative for equivalent resolution as 35mm Film

UHD-1: 3840 x 2160

HD: 1920 x 1080



UHD-1 and the “WOW” Factor

ı Resolution not the only issue for better picture quality

ı Improved Audio

Greater immersion in sound field (3D and Object Based)

ı Reduction of Temporal (Flicker/Judder) Effects

Higher Frame Rates

ı Increased Colour Space (Gamut)

More bits/pixel to reduce contouring and banding effects

More and brighter colours to improve realism



More Than Just Resolution

ı 4K on large TVs can appear similar to HD on smaller TVs

Minimal “WOW!” Factor



Theoretical Resolution: Screen Size v Viewing Distance



(Reality) Limits of Visual Acuity

ı Eye structure is NOT regular

Eye has rods (B/W) and cones (Colour)

Photoreceptors vary in size and shape across eye

=> One Pixel per Photoreceptor never applies

ı Eye is Never Still (Saccadic Motion)

Brain “builds” picture from multiple views

=> Higher equivalent resolution than camera

Eye + Brain is NOT a Camera



Beyond Resolution:

HDR+

(>8bit + WCG + HFR + HDR)

ı With millions of pixels a few inches from your face

Better be glad that the signal's not interlaced

Don't you know it needs a big screen size

Real wide

Alright

You love the spectral distribution

Well you know

It's just green and blue and red

You think that it's a revolution

Well you know

It's just the latest hype instead



Higher Frame Rates (HFR) with UHD TV

ı UHD will support frame rates up to 300fps

100fps in Europe and 120 in USA most likely (?)

ı HFR Reduces:

Judder on scrolling text/titles

Motion blur

=> Arguably 2K@48fps is better than 4k@24fps



Shutter Speeds: 60fps

ı Faster Shutter speeds freezes motion better

ı Slower shutter speeds better for low light applications

Limitations for live sports and outside broadcast filming



Increased Colour Depth with UHDTV

l More bits to reduce contouring/banding

l Rec2020: up to 12 Bits/pixel



BT2020 Extended Color Space

ı UHDTV: Rec 2020

HDTV: Rec 709

ı Wider Colour Gamut

8, 10 and 12 bit colour spaces

ı 2015: UHDTVs were at best Rec709

ı 2017: UHDTVs now support subset of REC2020

Much better than Rec 709

Only professional monitors support full Rec2010



Wide Colour Gamut (WCG) Examples



Expanded Colour Volume (Brightness) with BT2020



Real World Brightness

ı Extreme light and dark part of everyday experience

ı Measured in candela/m2 (“nits”)

80,000 Nits

10 Nits

150,000 Nits

15,000 Nits

1 Nits

3000 Nits



Brightness Ranges



SDR v HDR Brightness Range



Different TVs technologies have different max brightness

100 Nits 100 Nits 400 Nits 1000 Nits 1400 Nits

ı “Gamma” Curve needed to map to each display type

Two main systems proposed for UHD: PQ and HLG

Increasing display brightness



Standard Dynamic Range (SDR) Gamma Curve (BT.1866)

ı Standard CRT TVs max brightness around 100 Nits

Mapped to 8 bits in CCIR601

ı Non-linear Response of Camera/CRT requires Gamma correction



High Dynamic Range (HDR) “Gamma” Curves:

l HDR: Opto-Electrical Transfer Functions (OETF) replace Gamma curves

l Non-linear response of camera, panel and eye to brightness

l Additional metadata (SEI) transmitted to identify exact EOTF to be used by TV



HDR Levels: Different Content and Display Brightness



Hybrid Log Gamma (HLG) EOTF (“Gamma”) Curve

Scene Referred (as captured by camera)

Relative Standard

(to display’s capabilities)

NO metadata

100%

120%

Target ~1000 nits



Perceptual Quantiser (PQ) EOTF (“Gamma”) Curve

Display Referred (as set in post production)

Absolute Standard

(needs metadata)

Brightness ALWAYS mapped up to 10 00nits



PQ: Dynamic or Static Metadata

ı Metadata needed to convert picture for display

Dolby Vision uses metadata for each frame or scene (not possible for live TV)

HDR10 has one metadata set for entire film (Blu-ray)



ITU-R BT.2100-0 July 2016: HLG and PQ

ı PQ: Perceptual Quantisation

ı Maps non-linear PQ to display light

E = non-linear colour

F = linear luminance

Y = linear colour

ı Static or dynamic metadata required

ı HLG: Hybrid Log Gamma (BBC)

ı Maps linear light to non-linear signal

E = signal value for R, G, B at scene

E’= non-linear R’, G’, B’ signal

ı No Metadata Required



BT.709 v BT.2020 v BT.2100



SDR: Few Sources, Formats and Colour Spaces



HDR: Multiple Sources, Colour Spaces and Display Formats



Scene to Screen: Maximise Quality, Brightness, Colour…

Source: The Light Illusion



4K/HDR over IP in the Studio



Legacy Studio Formats



Serial Digital Interconnect (SDI)

ı SDI well established as the studio interconnect since 1982

SDI: 270Mbits/s up to 576i30

HD-SDI: 1.485Gbits/s up to 1080i30

3G-SDI: 2,97Gbits/s up to 1080p60

ı Reliable system carrying audio, SD/HD video and metadata on a single coax cable

Single format and point to point cabling simplifies system installation and management

ı Widespread availability of compatible routers, switchers, mixing desks etc.

ı Future high speed SDI cabling in development for UHD and beyond:

6G-SDI, 12G-SDI, 24G-SDI, ……



Why Change From SDI??



IP has many advantages over SDI

ı Reduced costs

standard-IT solutions

less cables

ı Improved flexibility & agility

UPnP (Universal Plug and Play)

scalability, virtual resources

remote production, non-central routing

ı Multiple formats on a single network

SD, HD, 4k, 8k

MPEG-2, H.264, H265 (HEVC), J2K , AAC

ı One cable for multiple A, V, and metadata

bidirectional

central sync over network



IP Complexities

ı Multiple formats and standards possible

Combined audio & video

Separate audio and video streams

Compressed or uncompressed

ı Identification and management of multiple sources

Synchronisation across a network

ı Bandwidth management

Raw 4K bit rates > 10Gbit

Intelligent routing of sources

ı …….



IP Setup and Interoperabilityı Single elementary (“Essence”) Streams OR

ı Combined audio/video/metadata Streams

ı Synchronisation across complete network

ı What devices are in the network

ı How are they found

ı Quality of service across network

ı Software Defined Networking (SDN)

ı Interoperability of encoding

Cloud or local

Timing

Identity

Media Transport

Discovery & registration

Flow Control

Flow Switching

Compression



Video/Audio over IP Formats



IP Protocols

Transport Specific

Datagram



UDP or TCP over IP Datagram Format

ı Internet Protocol Layer Contains Routing Information Source and Destination IP Addresses

ı User Datagram Protocol: Simple “send and forget” system

Port Number and payload

No Acknowledgements or retries

ı Transport Control Protocol: Connection based system Resend on packet loss possible but no FEC

UDP or TCPIP

Length (MTU) typically 1500 bytes



RTP/UDP/IP Datagram Format

ı Real Time Protocol used to carry e.g. video or audio Supports synchronisation and packet re-ordering

ı RFC4175 for uncompressed video via RTP Up to 32k x 32k resolution!

ı RFC2733 Defines Forward Error Correction for RTP packets

RTPUDPIPTime

Stamp

Sequence

Number

Source

ID Payload

Length (MTU) typically 1500 bytes



AES67: Audio Over IP

ı Transport: RTP/UDP/IP

Unicast

Multicast & IGMPv2

ı Encoding

16/24 bit linear

48 (44.1/96)kHz

Channel Count: 1…8 (more possible)

ı Synchronisation: IEEE 1588-2008

Media clock synchronous to PTP clock



SMPTE 2022-2: Transport Stream over RTP/UDP/IP

ı RTP payload must contain complete 188 byte TS Packets

ı Typically 7 TS Packets per RTP Packet

ı Typically used for IPTV services

UDPIP RTPTS

Packet

TS

Packet

TS

Packet

TS

Packet

TS

Packet

TS

Packet

TS

Packet



Key Video over IP standards

ı SMPTE 2022

2022-1 FEC for MPEG2 TS over IP

2022-2 CBR MPEG2 TS over IP

2022-3 VBR MPEG2 TS over IP

2022-4 Non-piecewise CVBR MPEG2 TS over IP

2022-5 FEC for High Bit Rate Media over IP

2022-6 High Bit Rate Media (uncompressed HD-SDI) over IP

(complete SDI stream with video, audio, ancillary data etc.)

2022-7 Seamless Switching SMPTE2022 streams

RTPUDPIPTime

Stamp

Sequence

Number

Source

ID Payload



IP in the Studio

Sony Live IP

AIMS

ASPEN



Core IP in the Studio Standards



Three Competing Studio Systems



Sony Live IP

ı Based on SMPTE2022-6 SDI over IP (video, audio, data together)

But separate video, audio and data essence option proposed

ı SMPTE2059/IEE1588 for synchronisation

ı Does not require “special” router hardware

ı LLVC compression reduces network traffic

4Kp60 in <10GB Ethernet

ı LLVC is proprietary codec from Sony

Submitted to SMPTE for standardisation



ASPEN – Adaptive Sample Picture Encapsulation

ı Based on SMPTE2022-2 (MPEG2 TS over IP)

Uses PCR and PTS in transport stream for synchronisation

ı Driven by Evertz but with some open aspects



AIMS Alliance for IP Media Solutions

ı Based on TR-03 and TR-04 from the Video Services Forum (VSF) with AES67 audio

TR-03: Uncompressed elementary streams over IP using RFC4175

TR-04: video, audio and meta data using separate SMPTE2022-6 IP streams

SMPTE2059/IEE1588 synchronisation



Network Management: Registration and Discovery

ı AMWA NMOS:

Advanced Media Work Flow Association (AMWA)

Network Media Open Specification (NMOS)

Registration and Discovery

Uses standard internet technology (DNS, TTP, JSON…)

ı Video Services Forum (VSF)

ı Joint Taskforce for Networked Media:

Combining AMWA and VSF

Drafting SMPTE2110



Converging Standards: SMPTE2110 (in Draft)



Studio IP Network

Video

Audio

ProductionVideo/Audio

Compression

& Storage

Distribution

Package

Creation

Network

Video/Audio

Monitoring

Video/Audio

IP Input

SDI to IP

IP

Switch

(COTS)

Video/Audio

Storage/

Playout

Network

Manager &

Timing

IPTV, OTT,

Sat, DTT,

Headend



R&S IP Studio Solutions

IP

Switch

(COTS)

Software

Defined

Network

Manager

Venice 4K AVS100 Clipster

AVHE100

Prismon

SpycerCell

SpycerStorage

Video/Audio

IP Input

SDI to IP



4K/HDR Workflow



Scene to Screen: Typical SDR/HDR Workflow



HDR Workflow Issues:

ı Framerate:

50p/60p becoming 100p/120p

High Bandwidth Record/Play

ı Colour Space/Volume Conversion

P3, DCI, XYZ, Rec709, Rec2020

HDR10, PQ, HLG, Dolby Vision…

High Performance FP Processing Needed

ı HDR Metadata (SEI)

Static or Dynamic

Must track with content from Scene to Screen



Rohde & Schwarz HDR Solutions: Clipster

IMFInteroperable

Master Format



R&S HDR Broadcast Solutions: Venice/Clipster and AVHE100

R&S®AVHE100R&S®TLU9

HDR

ı First SDI-based HDR/SDR signalization

VENICE 4K / CLIPSTER to AVHE100

ı Frame by Frame Switching

UHD/HEVC HDR

Encoding

R&S®VENICE 4K

HDR or SDR

Playout



Rohde & Schwarz: End to End HDR & SDR Broadcast Solutions

Live ProductionHDR capable camera /

converter box

mastering

Contribution

4Kp120

4:4:4

HEVC UHD

Encoding and

Stat-Mux

4 x 3G SDI

with HDR Metadata

Uplink

TS with SEI

HDR Metadata

ReceptionHDR capable decoder

and display

4 x 3G SDI

with HDR Metadata

Clipster Venice 4K AVHE100

BT2100

PQ and HLG

OTT

IPTV

SAT

DTT

Colour Volume Conversion



Thank You!Rohde & Schwarz

Caring for Your Content

Every Frame Counts Every Pixel Counts

technology day italy 4k broadcast chain - rohde & …± user datagram protocol: simple “send...

Documents