frequency and time synchronization in packet based … · • drawback : hardware upgrades all...

Cisco Confidential 1 © 2010 Cisco and/or its affiliates. All rights reserved.

Frequency and Time Synchronization In Packet Based Networks Peter Gaspar, Consulting System Engineer

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2

•  Synchronization Problem Statement

•  Overview of the Standardization Works

•  Frequency Transfer: techniques and deployment Synchronous Ethernet Adaptive Clock Recovery

•  Time Synchronization Two-Way Transfer Time Protocols

•  Overview of IEEE Std 1588-2008 for Telecom

•  Summary

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID 3

Problem Statement What and Why Do We Care About?


Why and How are Packet Switched Networks Involved? •  Transition from TDM to Ethernet networks.

•  Connect consumers requiring Frequency and/or Time (F&T) synchronization.

•  PSN is built with network elements that May have to support F&T distribution May be consumers of F&T

Aggregation

Subscriber Access

MSE

TDM / ATM

Ethernet

WiMAX

OLT xPON

xDSL DSLAM

M-CMTS

DVB-T/H 3GPP/2

DOCSIS

Backbone

Hub & Spoke or Ring

P P Internet

PE PE

MSA

PE

Peer ISP

Mesh P

TDM / ATM

P P

Identity Address Mgmt

Portal Subscriber Database

Monitoring Policy Definition

Billing

Service Exchange

VoD Content Network

TV SIP

Mobile user

Femto-cell

Mobile TV

Enterprise

Residential SoHO


•  Single domain vs. multiple domains Internet is a multi-domain network.

Wholesale Ethernet virtual link

•  Frequency and time could use different distribution methods.

•  Operators may provide synchronization services to their customers.

Aggregation

Subscriber Access

MSE

TDM / ATM

Ethernet

WiMAX

OLT xPON

xDSL

DSLAM

M-CMTS

DVB-T/H 3GPP/2

DOCSIS

Backbone

Hub & Spoke or Ring

P P

Internet

PE PE

MSA

PE

Peer ISP

Mesh P

TDM / ATM

P P

VoD

Content Network TV SIP

Mobile user

Femto-cell

Mobile TV

Enterprise

Residential SoHO


•  Frequency TDM interoperability and Co-existence: Circuit Emulation, TDM, MSAN (MGW) Access: Wireless Base Stations, PON, DSL

•  Time and Phase alignment Wireless Base Stations SLA and Performance Measurements

BS : Base Station PON : Passive Optical Network DSL : Digital Subscriber Line SLA : Service Level Agreement


•  Inter-CO/LAN (WAN)

•  Intra-CO, LAN

•  Intra-node, -platform

External Integrated Time and Frequency Server


The Leading Requirements Application Frequency Phase Alignment

Time Synchronization

TDM support (e.g. CES, SDH transformation), Access

PRC-traceability, jitter & wander limitations ITU-T G.8261/G.823/G.824/G.825

Mobile Base Stations

GSM, WCDMA and LTE FDD

Frequency assignment (fractional frequency accuracy) shall be better than •  ± 50ppb (macrocells)

•  ± 100ppb (micro- & pico-cells) •  ± 250ppb (femtocells)

N/A (except for MBMS and SFN)

UMTS TDD Phase alignment between base stations must be < ±2.5µs

TD-SCDMA Phase alignment between base stations must be < ±3µs

CDMA2K Time alignment error should be less than 3 µs and shall be less than 10 µs

LTE TDD Phase alignment between base stations from ±0.5µs to ±50µs (service degradation)

WiMAX Mobile Shall be better than ± 15 ppb Phase alignment between base stations must be < ±1µs

DVB-S/H/T2 SFN TBD Cell synchronization accuracy for SFN support must be < ± 3µs

MB SFN Service Phase/time alignment between base stations requirement can vary but in order of µs

One-way delay and jitter Performance Measurement

To improve precision << 1 ms for 10 to 100µs measurement accuracy need ± 1 µs to ± 10µs ToD accuracy


•  Cost

•  Limited utilization Locations Regulatory & Politics

•  Reliability Geography Vulnerability

https://www.gsw2008.net/files/Civ%20Vulnerabilities_GSW2008.pdf

746th Test Squadron

Use of GPS (and GNSS alternatives) raises some concerns:

GPS : Global Positioning System

GNSS : Global Navigation Satellite System


•  As Replacement or Backup

•  Alternative Radio Navigation LORAN-C ELORAN

•  Atomic Clock Cheap Scale Atomic Clock Molecular Clock

•  Network Clock Main topic of this session!

LORAN : LOng Range Aid to Navigation


Standardization Development Organizations Who’s doing what?


•  Frequency transfer Parallel (overlay) SDH/SONET network Radio Navigation (e.g., GPS, LORAN) PHY-layer mechanisms Packet-based solutions

•  Time transfer (relative and absolute) Radio Navigation (e.g., GPS, LORAN…) Packet-based solutions


SDO Techno Status Scope Market

ITU-T SG15 Q13

Synchronous Ethernet

G.8261(2008) G.8262(2007)+Amend.1

G.8264(2008) G.781 (2008)

PHY-layer frequency transfer

Service Provider (SP) Metro & Core

Ethernet

Packet-based timing

G.8261 (2006)

Multiple working items: profile, metrics,

modeling…

CES performance

Packet-based frequency, phase and time transfer

Service Provider (SP)

IEEE 1588 PTP

IEEE1588-2002 IEEE1588-2008

No “Telecom” profile

Precise time distribution

Enterprise: Time SP: Frequency,

phase and time ITU-T & IETF

802.1AS Based on PTP Ballot Precise time

distribution Residential

IETF NTP NTP

NTPv3 Standard NTPv4 (CY09)

Time distribution Internet

SP domain

TICTOC NTPv5 PTP Profile(s)

New WG (approved March 08)

Frequency and time transfer

Internet Specific SP areas


IETF TICTOC

ITU-T Q13/15

IEEE1588-2008

(PTPv2) IEEE

802.1AS

IETF NTP

AVB Profile(s)

ATIS Telcordia

ProfiNet: IEC 61158 Type10 DeviceNet: IEC 62026-3

ControlNet: IEC 61158 Type2

IEC Profiles

Telecom Profile(s) On-going

IEEE 802.3 Timestamping


Frequency Transfer Distribution of Frequency Reference


•  Physical layer options Ex: SONET/SDH, SDSL, GPON, Synchronous Ethernet Pros: “carrier-class”, well defined, guaranteed results Cons: node by node link bit timing, requires HW changes

•  Packet-based options Ex: SAToP, CESoPSN, NTP, PTP (protocol of IEEE Std 1588) Pros: flexible, looks simple, some can do time as well Cons: the network and the network traffic, not so simple!


•  The task of network synchronization is to distribute the reference signal from the PRC to all network elements requiring synchronization.

•  The method used for propagating the reference signal in the network is the master-slave method.

•  Slave clock must be slaved to clock of higher (or equal) stability. hierarchical model

PRC : Primary Reference Clock

Source: ETSI EG 201 793 “Synchronization network engineering”


•  Synchronization equipments PRC (PRS) and SSU (BITS) do not belong to the Transport network.

•  SEC (SDH/SONET Equipment Clock) belong to Transport network. They are embedded in Network Element : NE.


•  Synchronization information is transmitted through the network via synchronization network connections.

•  Synchronization network connections are unidirectional and generally point-to-multipoint.

Stratum 1 level

Stratum 2 level

NE (Stratum level ≥ 3)

CO


Core Network

Aggregation and Access Networks

PRC : Primary Reference Clock (≈ PRS) SSU : Synchronization Supply Unit (≈ BITS) SEC : SDH Equipment Clock



Receiver for synchronization reference signal



Figure 4-2. Recommended BITS Implementation with SONET Timing Distribution

NE’s External Timing Output

NE’s External Timing Input a.k.a. BITS IN

Source: Telcordia GR-436-CORE . Digital Network Synchronization Plan


SSU/BITS

NE

PRC/PRS

SSU/BITS

NE NE NE NE NE

Intra-office

Intra-office

Intra-office

Inter-office Inter-office

BITS

NE

PRS

BITS

NE NE NE

PRS

Intra-office

Intra-office

NE NE

Inter-office

Intra-office

Inter-office


•  Some of these synchronized trail contain a communication channel, the Synchronization Status Message (SSM) transporting a quality identifier, the QL (quality level) value.

This is a 4-bit field in SDH/SONET frame overhead.

•  Purpose: Traceability (and help in prevention of timing loops)

Stratum 1 level

Stratum 2 level

NE level

What clock quality do I get? Is that the best source I can use?


SSM Allows Source Traceability

PRC synchronization network connection

SSU synchronization network connection SEC synchronization network connection

Representation of the PRC network connection

X

Fault Representation of the synchronization network connection in case of failure

Example of restoration of the synchronization


•  PHY-layer frequency transfer solution for IEEE802.3 links

•  Well-known design rules and metrics Best fit for operators running SONET/SDH

•  Fully specified at ITU-T Working Group 15 Question 13 For both 2.048 and 1.544 kbps hierarchies

•  Expected to be fundamental to high quality time transfer

•  Drawback : hardware upgrades All timing chain shall be SyncE capable.


PLL

Synchronous Ethernet capable

Line Card

IEEE802.3 ± 100ppm

ITU-T G.8261 SyncE interface jitter & wander

ITU-T G.8262 (EEC): Synchronous Ethernet

Equipment Clock

ITU-T G.781: Clock Selection Process

Synchronous Ethernet capable

Line Card

Frequency distribution

traces

External timing interface inputs

External Equipment BITS/SSU)

External timing interface inputs

PRC-traceable signal from BITS/SSU

ITU-T G.8264 ESMC and SSM-QL

External timing interface outputs

Synchronous Ethernet capable Equipment


•  Ethernet Synchronization Messaging Channel Use OSSP from IEEE802.3ay (a revision to IEEE Std 802.3-2005)

•  Key purpose: transmit SSM (QL) Outcome: Simple and efficient

But designed to support extensions

•  Protocol model: Event-driven with TLVs

•  Two message types Event message sent when QL value change Information message sent every second

•  TLVs QL-TLV is currently the unique defined TLV. Other functions can be developed.

OSSP : Organization Specific Slow Protocol


0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | Slow Protocols MAC Address | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | Slow Protocol MAC Addr (cont) | Source MAC Addr | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | Source MAC Address (continued) | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| |Slow Protocols Ethertype 0x8809| Subtype (10) | ITU-OUI Oct 1 | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | ITU-OUI Octets 2/3 (0x0019A7) | ITU Subtype (0x0001)* | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | Vers. |C| Reserved | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | Type: 0x01 | Length | Resvd | QL | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | Future TLV #n (extension TLV) | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | | | Padding or Reserved | | | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | FCS | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|

* Allocated by TSB

IEEE 802.3 OSSP

ITU-T OUI Header

ESMC Header

QL-TLV

Future TLV Extension Payload

OSSP


Assuring The Continuity at PHY Layer

•  Extension or replacement of SDH/SONET synchronization chain

•  Inherit from previous ITU-T (and Telcordia) recommendations

•  Difference: frequency transfer path engineering will define the necessary upgrades.

Only the NE part of the engineered timing chain needs SyncE upgrades.

ITU-T G.8262 (EEC) Node

BITS/SSU

SONET/SDH PHY SyncE

BITS/SSU PRC/PRS BITS/SSU

PHY SyncE





•  Three key steps: Generation: from signal to packet Transfer: packet transmission over packet network(s) Recovery: from packet to signal

Reference Clock Recovered

Clock PSN


•  ITU-T Recommendation G.8261 (2008) Adaptive Clock Recovery Definition

“In this case the timing recovery process is based on the (inter-) arrival time of the packets (e.g., timestamps or CES packets). The information carried by the packets could be used to support this operation. Two-way or one-way protocols can be used.”

ACR Protocol / Method One-Way Two-Way Timestamp

CES (SAToP, CESoPSN) X

IETF NTP (X) X X

IEEE Std 1588-2008 PTP X X X

IETF RTP X X X


Independent Timing Stream

TDM TDM

IWF IWF

Recovered TDM timing based on the adaptive clock recovery

ACR Packet Stream Reference

Clock

TDM PW bit stream

Clocking method a.k.a. “out-of-band” (here, used for CES clocking)

TDM TDM IWF &

PEC

IWF &

PEC

ACR Packet Stream

TDM PW bit stream

PEC

Reference Clock


Source: Diagram from “Time Domain Representation of Oscillator Performance”,

Marc A. Weiss, Ph.D. NIST


•  Frequency Accuracy ≤ ±50ppb at base station radio interface (specified) Turns into ≤ ± 16ppb at base station traffic interface (not specified*)

•  Frequency Stability For T1, it shall comply to G.824 traffic mask (specification; 3GPP Rel8) Sometimes* G.824 synchronization mask preferred

* Note: real requirements are variable as they are dependent on base station clock servo.


•  Phase measurement Measure signal under test against a reference signal

•  Phase deviation plot TIE : Time Interval Error

•  Analysis MTIE TDEV


Step 1 : Phase Measurements

•  At a certain signal threshold, time stamp the edges of timing signal.

•  Signal edges are the significant instants.

•  PHY-layer signals have high frequency (e.g., 1544 kHz)

-0.1 -0.2

+0.1

-0.2

+0.1

Signal

Ref.


Step 2 : Phase Deviation

•  Phase deviation or TIE (Time Interval Error)


Step 3: Analysis •  Analysis cover different aspects of the

Clock (oscillator) e.g. in free-running or holdover mode

Signal

•  Primary used measurement analysis are: Phase (TIE) Frequency (fractional frequency offset) Frequency accuracy MTIE TDEV


Signal with jitter and wander present


Jitter: Filter out low-frequency components with high-pass filter

Frequency Jitter range 10 Hz


Wander: Filter out high-frequency components with low-pass filter

Frequency Wander range 10 Hz


•  Both MTIE and TDEV are measures of wander over ranges of values. From very short-term wander to long-term wander

•  MTIE and TDEV analysis shows comparison to standard requirements. Defined by ATIS/ANSI, Telcordia/Bellcore, ETSI & ITU-T E.g., ITU-T G.824, ANSI T1.101 or Telcordia GR-253-CORE

•  MTIE is a peak detector: simple peak-to-peak analysis.

•  TDEV is a highly averaged “rms”-type of calculation.


Frequency Accuracy (Frequency Offset)

ITU-T G.823 Traffic Interface (MRTIE mask)

ITU-T G.823 Synchronization Interface (MTIE mask)


•  Physical layer signals can be characterized.

•  Recommendations exist for node clock and interface limits.

•  Synchronous Ethernet Equipment Clock (EEC) inherits from SONET NE clock specifications.

•  The performance of SyncE-capable NE and SyncE interface are fully specified and metrics exist.


•  How to guarantee the packet-based recovered clock quality?

PSN

Reference Clock

Recovered Clock

Slave/ Client

Master/ Server ?

OK

Packet Delay Variation is key impairment factor for timing.

DS1 DS1


•  TIE is still a valid measurement for characterizing the packet-based servo (slave).

Oscillators and timing interfaces

•  How can the PSN behavior be characterized? Algorithms use minTDEV value Need sufficient numbers of minimal latency packets Packet Delay Variation (PDV) as metric?

•  First approach is to reuse known tools to PDV analysis/measurement. Some can be applied to PDV as to TIE.


10 Switches, 40% Load

10 Switches, 80% Load

minTDEV


•  One metric would not be sufficient characterizing the various possible conditions.

Reference Clock

Recovered Clock

Classification (metric)

Master/ Server

PSN

Common, generic PSN metrics for timing performance

characterization?

  Today, very close relationship between metric (packet classification) and implementation specific algorithm.


•  Protocol parameters

•  Influenced by : the PSN design, the HW & SW NE configuration, the traffic.

•  Master implementation

PSN

Reference Clock

Recovered Clock

Slave/ Client Master/

Server

PSN Metrics

? ?

  Slave implementation

  minTDEV used in algorithms, but still not adopted as metric

  Even with (still to be agreed) metrics, other parameters will remain critical.

?


1.  PHY-layer Synchronization Distribution guarantees the quality.

2.  Packet-based Synchronization Distribution provides the flexibility.

3.  Mixing the option for getting best of both solutions.

PRC/PRS Thru BITS/SSU

EEC

EEC

EEC

EEC

Consumer

Non-capable PHY Layer Synchronization Network

SEC

PHY-layer method e.g., SDH/SONET, SyncE

Packet-based method (ACR)

PHY-layer Freq Transfer



e.g. SyncE


e.g. SyncE

SyncE consumer

Packet-based

consumer


Time Synchronization What Specific Challenges Does the

Time Distribution Introduce?


•  Transmitting time reference can be absolute (from national standards) or relative (bounded timekeeping system).

•  Time synchronization is one way achieving phase synchronization. Phase alignment does not mandate giving a time value.


•  This is not phase locking which is often a result of a PLL in a physical timing transfer.

Phase locking implies frequency synchronization and allows phase offset.

•  The term phase synchronization (or phase alignment) implies that all associated nodes have access to a reference timing signal whose significant events occur at the same instant (within the relevant phase accuracy requirement).

Figure xxx/G.8266 – Phase Synchronization


Target from ±1µs to tens of µs (alignment between BS)

Target from ≤ ±0.5µs to tens of µs (from common reference)

Time Source


•  Strictly speaking, the term synchronization applies to alignment of time and the term syntonization applies to alignment of frequency.

•  The master/server and slave/client clocks each have their own time-base and own wall-clock and the intent is to make the slave/client “equal” to the master/server.

•  The notion of frequency synchronization (or syntonization) is making the time-bases “equal”, allowing a fixed (probably unknown) offset in the wall-clocks. The notion of time synchronization is making the wall-clocks “equal”.


NTP vs. PTP Message Exchange

NTP

PTP

As part of time recovery, there’s always a frequency recovery process.


•  Forward and backward delays and delay variations are not identical.


•  Each Node and Link can introduce asymmetry.

•  There are various sources of asymmetry.


•  Link Link delays and asymmetry Asymmetric (upstream/downstream) link techniques Physical layer clock

•  Node Different link speed (forward / reverse) Node design LC design Enabled features

•  Network Traffic path inconsistency Interface speed change


Summary and Introduction to IEEE Std 1588 •  Basis of all packet time transfer protocols (NTP, IEEE1588) is the two

way time transfer mechanism.

•  TWTT consists of a time transfer mechanism and a time delay “radar”.

•  Assumes path symmetry and path consistency.

•  IEEE1588 incorporates some in-network correction mechanisms to improve the quality of the transfer.

•  IEEE1588 has the concept of asymmetry correction. But the correction values are not dynamically measured - they need to be statically configured.


IEEE Std 1588-2008 for Telecom Challenges of IEEE 1588-2008 applied

in Service Provider networks


•  A set of event messages consisting of:

- Sync - Delay_Req - Pdelay_Req - Pdelay_Resp

•  A set of general messages consisting of:

- Follow_Up - Delay_Resp - Pdelay_Resp_Follow_Up - Announce - Management - Signaling

  Transmission modes: either unicast or multicast (can be mixed)

  Encapsulations: L2 Ethernet, IPv4, IPv6 (others possible)

  Multiple possible values or range of values, TLVs (possible extensions), …


MASTER SLAVE

Delay_Resp

t1

t3

t4

Timestamps known by slave

t1, t2, t3, t4

SM_Delay

Master time = TM Slave time = TS

t2

t1, t2, t3

t1, t2

SYNC

Delay_Req

MS_Delay


SYNC

MASTER SLAVE

Delay_REQ

Delay_RESP

MAC/PHY MAC/PHY µP µP

Hardware assistance necessary to prevent insertion of errors or inaccuracies.

t1

t2

t3

t4

t4

t1

t2

t3

Need to inject the timestamp into the payload at the time the packet gets out.


t1, t2, t3, t4

t1, t2, t3

t1, t2


SYNC()

MASTER SLAVE

Delay_REQ()

Delay_RESP(t4)

MAC/PHY MAC/PHY µP µP


Follow_Up(t1)

t1

t2

t4

t3

Two-step clock mode Vs. One-step (a.k.a. “on-the-fly”) clock mode

t1, t2, t3, t4

t1, t2, t3

t1, t2

t2


•  Five basic types of PTP devices (“clocks”) Ordinary clock (master or slave) Boundary clock (“master and slave”) End-to-end Transparent clock Peer-to-peer Transparent clock Management node

•  All five types implement one or more aspects of the PTP protocol


•  BC and TC aims correcting delay variation into intermediate nodes between OCs.

•  Can correct link asymmetry if known.

Ref. Clock

Recovered Clock

Ordinary Slave

Ordinary Master

TC BC

Transparent Clock

Boundary Clock


•  Can help on scalability when using unicast.

•  Equivalent to NTP Stratum (>1) Server UTC

•  Node by node: BC slave function is critical

Ref. Clock

Recovered Clock

Ordinary Slave

Ordinary Master

BC

Boundary Clock

BC

Boundary Clock


•  TC calculates Residence Time (forward / reverse intra node delays).

•  TC are supposed to be transparent but: One-step clock issue

Ref. Clock

Recovered Clock

Ordinary Slave

Ordinary Master

Transparent Clock

Transparent Clock

TC TC


•  If IEEE 1588-2008 is not planned node to node, with every node IEEE 1588 aware and in unique domain…

•  Multiple interface types IEEE 802.3, ITU-T G.709, …

•  Multiple interface frequencies 10GE, 100GE, STM64, STM192…

•  Multiple encapsulations Ethernet, IP MPLS, MPLS-TP, PBB-TE…


Ref. Clock

Recovered Clock

Ordinary Slave

Ordinary Master

TC BC

Wholesale Boundary Clock

TC BC

•  Who owns the master?

•  Who owns the slaves?

•  Who owns the intermediate nodes?


•  How to guarantee the recovered clock quality?

PSN

Ref. Clock

Recovered Clock

Slave/ Client

Master/ Server

? ?

?

TC

? ?

BC

Objective: accuracy and stability from reference


•  IEEE Std 1588-2008 is actually a “toolbox” !

What does “support of IEEE 1588” really mean ?

•  IEEE Std 1588 itself is not sufficient for telecom operator operations. Node characterization, modeling, performance, metrics…

•  For phase & time support, it is expected any telecom standardization would take time.


Summary


•  Timing is a new service many networks shall have to support.

•  Different solutions are necessary to cover disparate requirements, network designs and conditions.

Physical layer solutions required to upgrade routers and switches. Packet-based solutions are more flexible but less deterministic.

•  Whatever the timing protocol, it must deal with the same network constraints.

•  Each network is different

•  Synchronization Experts are welcome to enter the packet based networks and assist with the designs

Thank you.


Registrujte se za Cisco Live Networkers u Londonu ili Bahreinu! Više informacija na: http://www.ciscolive.com/

frequency and time synchronization in packet based … · • drawback : hardware upgrades all...

Documents