Key Global Application Drivers in Asia and Korea

1st Global Research Platform Workshop

Key Global Application Drivers in Asia and Korea

1st Global Research Platform Workshop

High Energy Physics & Fusion Energy Science

Astronomy & Climate

Changes

Education & e-Culture

Future Internet &

Construction

Advanced Scientific

Computing Research

Bio/Genome Research & Medical Science

Enabling Science Discovery- New particle, “Higgs” in CERN LHC- Gravitational Wave in LIGO- Blackhole Event Horizon in Astronomy

Supermassive black hole in the heart of galaxy M87

High Energy Physics

Private network connecting Tier0 and Tier1s

LHC Optical Private Network

█ = Alice █ = Atlas █ = CMS █ = LHCbedoardo.martelli@cern.ch 20190516

T0-T1 and T1-T1 trafficT1-T1 traffic only

10Gbps20Gbps30Gbps40Gbps100Gbps

Ref: https://twiki.cern.ch/twiki/bin/view/LHCOPN/OverallNetworkMaps

TW-ASGCAS24167

█ █

CA-TRIUMFAS36391

█

US-T1-BNLAS43

█

US-FNAL-CMSAS3152

█

FR-CCIN2P3AS789

█ █ █ █

ES-PICAS43115█ █ █

IT-INFN-CNAFAS137

█ █ █ █

DE-KITAS58069█ █ █ █

UK-T1-RALAS43475█ █ █ █

NDGFAS39590

█ █

KR-KISTIAS17579

█CH-CERN

AS 513█ █ █ █

RRC-KI-T1AS59624█ █ █

RRC-JINR-T1AS2875

█

NL-T1AS1162, 1104

█ █ █

TEIN

NOTES1) ONLY links involved in LHCONE are shown

2) LHCOPN links are not shown on this diagram

3) For map explanation see “Rationale Behind the LHCONE Map” at

https://www.dropbox.com/sh/padxfo58j0j1raz/AADsB5K8fISH9FfhCjA4eCtea?dl=0

GÉANT,Europe

LHCONE Map Ver. 5.1, May 30, 2019 – WEJohnston, ESnet, wej@es.net

LHCONE L3VPN: A global infrastructure for High Energy Physics data analysis (LHC, Belle II, Pierre Auger Observatory, NOvA, XENON)

TIFRNKN, India

toCE

RN

GÉANTOpen

London

CESNETCzechia

ESnet

Geneva

RedIRISSpain

RoEduNetRomania

UAIC xx, ISS,

NIHAM,

ITIM, NIPNEx3

JANETUK

IC

UChi

(MWT2)

ARNESSlovenia

SiGNET xx

AGLT2

MSU

AGLT2

UM

Budapest

LHC1-RENATERFrance, AS2091

LPNHE

APC,

LAL

IPNHO

LLR,

CEA-

IRFU

LPC,

CPPM,

IPHC,

SUBATECH

Korea

CANA

RIE

To: Internet2

ESnet, CalREN, U Fl

Caltech

MIT

Pa

cif

icW

av

e

(dis

trib

ute

d e

xch

an

ge

)

Bucharest

TO CSTNET

Via TEIN and

CNGI-CERNET

UWisc

Madison

To: CERN

RNP/IPĔ / ANSPBrazil

SAMPA

(U. São Paulo)

Atla

ntic

Wa

ve

(distrib

ute

d e

xch

an

ge

)

UIUC

(MWT2)

UCSD

To

MANLAN

to G

ÉANT

Ge

ne

va

,CE

RN

Internet2USA

KREO

Net2

Global Research Platform Network (GRPnet)

Pacif

icWav

e/PN

WG

Madrid

CANARIE

to G

ÉANT

(Lo

nd

on

via

Ori

en

t+)

FCCNPortugal

London

SPRACE

Prague

Brussels

Lisbon

ESnetUSA

FNAL-T1prague_cesnet_lcg2,

FZU/praguelcg2

DFNGermany

GSI

Frankfurt

to C

ER

N

NORDUnet

CANARIE

Peer: ASGC, ESnet, Internet2, CANARIE,

KREONet2;------------

GÉANT, GARR, CSTNET

UT

Arlington

Kharkov-KIPT

URANUkraine

Hamburg

Beijing/TEIN North

CSTNETChina

China Next

Generation

Internet,

Internet

Exchange

CIEMAT-LCG2,

UAM-LCG2

AARNetAustralia

UMel

PERNPakistan

NCP-LCG2

PK-CIIT

PNNL, SLAC, ORNL

ASGCTaiwanASGC-T1

IndianaUSA

UNL

KREO

Net2

Milan

Peers:

ESnet,

Internet2

CERNGeneva

CERN-T1

CERN-T0

NET2 / BU

(North East Tier 2)

UCSB

ESnet, Internet2 via WIX,

AmLight

RedCALARA

To: SINET,

ESnet, Internet2

CANARIE/CANETCanada

Montreal

Harvard

GÉANT to TEINSingapore

SANETSlovakia

FMPh-UNIBA xx

IEPSAS-Kosice

GA

RR

Amsterdam

Peers: ASGC,

CERNLight,

NORDUnet2,

GÉANT, RU-

VRF/KIAE, SINET,

NL-T1;

----------------

SURFsara, ESnet,

PSNC,

Vienna

PIONIERPoland PSNC

AGH Kraków,

U.Warsaw

CERNLight

Geneva

GRNETGreece

UIC

BT

AA

Omni

PoP

(Ch

ica

go

)

Peers: UOK, UFL, CANET,

IU, PURDUE, IUPUI,

ESnet, CALTECH, MIT,

UOC, NORDUNET,

GÉANT, AARNet,

(same VRF asESnet Europe)

ND

WIX(Washington)

AMPath/AMLight

NAP of Americas

(Miami)

PTTANAP do Brasil(São Paulo)

McGill

ASGC2

SINETJapan

Hirosh.,

Tsukuba,

Tokyo

CUDIMexico

UNAM

Poznan

SINETTokyo

To SINET,

ESnet

via

Seattle

Hong Kong

CNGI-CERNETChina

KREONet2Korea

PNUKISTI –T1

KNU, KCMS

GÉANT,

CANARIE,

NORDUnet,

SINET

ESnet

MOXY(Montreal)

To RU-VRF,

GÉANT,

MAN LAN,

KRLightDaejeon

MREN(Chicago)

GÉANT to Beijingv

ia O

rien

t+

to G

ÉANT

Lo

nd

on

REUNAChile

to

JGN

KREONet

CENI

C/Ca

lREN

HEPNET-JJapanKEK T1

UTNETICEPP

U Tokyo

Tokyo

CBPF

(Rio de Janeiro)

TANet2Taiwan

NCU, NTU

SOX(Atlanta)

BelnetBelgium

BEgrid-ULB-VUB,

Begrid-UCL, IIHE

DE-KIT-T1RWTH

DESY

HEPGrid

(UERJ)

To

ES

ne

t

IUPUI

(MWT2)

Purdue

SURFsaraNetherlands

CEA

Saclay

CC-IN2P3-T1

DNIC

HEPCAS,

IHEP Beijing

CMS ATLAS

FAPESP

Amparo

SimFraU

BCNet

UBC

UTorontoUVic

TRIUMF-T1

NL-T1

Nikhef

ASGC Taiwan

via Singapore

To ASGC,

Taiwan

BNL-T1

BNL-T1ANL

ERX-ERNETIndia

To:

KERONet,

ASGC,

TEIN

AS

GC

to

GÉA

NT A

mst

erd

am

, CER

N

TIFR

Mumbai

ESnet To

TEIN, via

JGN

To

GÉ

AN

T

To ASGC,

KREONet2,

CANARIE

RAL

Cyfronet

POZMAN

ICM-PUB

Ioannina

ThaiRENThailand

THAISARN,PUBNET-TH,NSTDA-TH

UNINET-TH,SUT-TH

UNINET,ERX-CHULANET

PIC-T1

Marseille

ASGC to

NetherLight

AS

GC

toS

ing

ap

ore

ASGC to

Singapore

ASGC to

Amsterdam

ASGC

ASGC

ASGC

SINET,

To MANLAN

SINET

To LA

SINET to Tokyo

SINET

To Tokyo

SINE

T to

Amst

erda

m

SINET to

TOKYO

To

London

Perth

Paris

Amsterdam

London

AN

A-4

00

Peers: SINET, ESnet,

Internet2, TransPAC

UTFSM

SingARENSingapore

Peers:ASGC, SINET, JGN,

TEIN;

Singapore

MAN LAN(New York)

Internet2, CERN,

ESnet, NORDUnet,

CANARIE

NORDUnetNordic

NDGF-T1,

NDGF-T1b

NDGF-T1c

Helsinki

MyRENMalaysia

GARR LHCONE VRF domain/aggregator- A provider network.

Provider network PoP router

NREN/site router at exchange point

London

Connector network – provides,e.g., an L2 path between VRFs.

ANSP

CANARIE

CUDI

Communication links:

1/10, 20/30/40, and 100Gb/s or N x 100G

Underlined link information

indicates link provider, not use

Exchange point

Not currently connected to

LHCONE

JGN –

SingAREN/NSCC

NDGF-T1

RAL

Dotted outline indicates distributed site

Dashed outline indicating a WLCG federation site not currently on LHCONE

Dublin

PacWave /PNWG

(Seattle)Peer:

ESnet, KREONet2

IU

GÉ

AN

T, C

ER

NE

T

International infrastructure by provider/collaboration

GÉANT

JGN, SingAREN (+ TIENCC, SurfNet, NORDUnet,GÉANT on the London link)

SINET, Japan

ASGC, Taiwan

KREONet2, Korea

ESnet transatlantic, USA

ANA-400 - Various links provided by CANARIE, ESnet,

GÉANT, Internet2, NORDUnet, SURFnet, SINET,IU/NSF

NORDUnetvarious

KIAE, Russia

KIAE-TRANSIT(RU-VRF)

Russia

Moscow

GridPNPI

INR

(Troitsk)

BINP IHEPProtvino

JINR-T1/T2

SarFTI

RRC-KI T1/T2

GARRItaly

INFN

Bari, Catania,

Frascati, Legnaro,

Milano, Roma1,

Torino

INFN

Pisa, Napoli

CNAF-T1

PacWave(Los Angeles)

NKNIndiaVECC

Kolkata

Peers:Internet2, RU-VRF,

NORDUnet,ESnet

Starlight(Chicago)

UChi

Belle II Tier 1/2

Yellow outline indicates LHC+Belle II site

KEK

LHC Tier 2/3 ATLAS and CMS

LHC Tier 1 ATLAS and CMSCNAF-T1 Sites that are standalone VRFs

UNLPNU LHC ALICE or LHCb site

JGNJapan

HongKong

TransPACUS/NSF/IU

HongKong

HKIXHong Kong

GOREX(U of Guam)

Guam

PacWave(Sunnyvale)

Peers: ESnet,

UCSD,UCSD

Waterloo

NetherLight(Amsterdam)

- 20 R&E networks

- 14 Tier1s and ~70 Tier2s in 5 continents

- ~250 perfSONAR instances

- Collaboration with Bell II experiment, DUNE experiment

Asia North America

Europe

South AmericaOceania

Layer3 (routed) Virtual Private Network

GEANTEU AS20965

TEINHK: AS24490SG: AS24490

ASGCNetTW AS24167

CERNlightCH AS20641

ThaiRENTH AS24475

SINETJP AS2907

ESnetUS AS293

TIFRIN AS2697

Not Asia-Pacifc

Asia-Pacific

LHCONE connection

Connection in place, but no LHCONEMissing connection

JGNJP AS17934

AARnetAU AS7575

CERnetCN AS23911

CSTnetCN AS7497

Internet2US AS11537

MyRENMY

CANARIECA AS6509

NKNIN AS55824

LHCONE mutual transit

Asia-Pacific VRFs – Current Status

PERNPK

INHERENTID AS18007

KREONet2KR AS17579

Localize bulk data in Cloud Service

Daniele Cesini , Patrick Fuhrmann, “The Data Lake model for the scientific computing of the next decade“, 2018

- Data Organization, Management, Access (DOMA)- Storage consolidation - Cashing and data access- Data Transfer among Storage in distributed Regional Storage (DRS) or among DRSs- QoS

*Asia Tier Center Forum

AsiaSite B

Site A

SUT, TH

GSDC, KR

Site C

America

Site A GSDC, KRTbps Tbps

Gbps

Gbps

EU-LakeAisa-Lake

Open Storage Network(OSG)

Europe

Fusion Energy : KSTAR & ITER

1) Engineering Validation and Engineering Design Activities for the International Fusion Materials Irradiation Facility (IFMIF/EVEDA:2007-2017)

2) International Fusion Energy Research Centre (IFERC: 2007-2017), project leader is N.Nakajima

a) DEMO Design and R&D coordination Centreb) Computational Simulation Centre c) ITER Remote Experimentation Centre

- Estimation for the ITER data amount in 2016

-Total archive capacity: 90 - 220 TB/day

3) Satellite Tokamak Programme (2007-2019) Participation to upgrade of JT-60 Tokamak to JT-60SA and its exploitation.

ITER Phase Total DAN* archive rate Plasma duration Time Data Volume per Shot

Initial 2 GB/s (16Gbps) 400 – 500 sec (6.7 – 8.3 min) 800 GB – 1 TB

Final 50 GB/s (400Gbps) 1000 s (16.7 min) 50 TB

H. Nakanishi et al, “High-Performance Data Transfer for Full Data Replication between ITER and the Remote Experimentation Centre”, 26th IAEA Fusion Energy Conference, 2016

11

Data storage and Analyses

//

Proposal and setting discharge parameter by remote user

Setting discharge parameter

Data collection and analyses

ITER Remote Experimentation Centre (Rokkasho, Japan)

Send discharge parameter

Monitor of the operation,Transfer results of experiment

Validation of safety of the facility and Operation

ITER Organization (Saint-Paul-les-Durance)

Broad band internet

The ITER remote experimentation centre will be made to participate the experiment remotely, where the experimental proposal and data collection and analyses will be done through the broadband network.

JW Farthing, “Progress on ITER remote experimentation Center”, 2017

Russian Participation Center

Korean Participation Center

AstronomyWritten by colaboration with Dr. Taehyun Jung, Korea Astronomy & Space Science Institute

Global VLBI Communities

North America Europe

East-Asian

Oceania

• 25m x 10 radio telescopes operated by NRAO (National Radio Astronomical Observatory), US

• Observing wavelength: 90cm –3mm

• Correlator: NRAO in Socorro, USA

• ~ 24 radio telescopes in Europe, Asia, South Africa and 10 radio telescopes in Puerto Rico

• Observing wavelength: 90cm – 7mm

• Correlator: The joint Institute for VLBI ERIC (JIVE) in Dwingeloo, Netherland

• ~ 20 radio telescopes in China, Japan and S. Korea

• Observing wavelength: 5cm – 7mm

• Correlator: KASI in Daejeon, South Korea

• ~ 11 radio telescopes in Australia operated by CSIRO

• Observing wavelength: 20cm – 3mm

• Correlator: PawseySupercomputing Centre in Perth, Australia

Global VLBI Communities

North America Europe

East-Asian

Oceania

Global VLBI Alliance Initiative• “Earth VLBI alliance” : VLBA + EVN + EAVN + LBA + …• Provide simpler the global VLBI access for astronomers in

order to pursue the breakthrough science by maximizing the power of VLBI in worldwide

Fast Radio Burst (FRB) Gravitational Wave

• VLBI imaging can help § to clarify the astrophysics§ precise localizations of the origin § to Identify potential persistent radio counterparts • e.g. Transients, Neutron star mergers, Gravitational Wave, Fast

Radio Burst etc.• e-VLBI is greatly important for rapid response sciences and multi-

messenger astronomy

• Data Intensive Science§ High performance networking and computing, Big data§ Large-scale instruments : telescope§ Data Center and HPC involved

• Event-driven (or scheduled) Rapid Response Science§ Fast response to request for observation§ Real-time, more robust against failure § Immediate feedback on the state of variable and transient phenomena§ Noticed immediately and addressed instantaneously§ Dynamically addressable to telescope, network and correlator§ Real-time correlation (saving the storage cost)

• Collaborative Science§ Globally Multi-partners and multi-sites of telescopes§ Sometime a telescope is located at an out-of-the-way place§ large-scale global collaboration

• End-to-end high performance network§ Large bandwidth network with 100Gbps§ Science DMZ + DTN + PerfSONAR + Science Engagement§ (Static) allocation of lightpaths§ Virtual Private Network

• Dynamically provisioning and optimizing network§ Locally and internationally (inter-continent) connected§ Dynamic allocation of lightpaths§ Bandwidth on demand§ Automated GOLE§ Programmable capability

• Automated operation enabled by networkers and operated by astronomer

Data-Intensive Science

Rapid Response Science Collaborative Science

Why not? Automated Dynamic Virtual Private Network

• Global VLBI Alliance (GVA) demonstration with the most common frequency (22GHz/13mm)§ Number of telescopes

- VLBA (10), EVN (14), EAVN (15), LBA (5) - Total 44 radio telescopes (RT)

§ Data rate - 1Gbps / 2Gbps / 4Gbps / 8Gbps (Currently)- 16Gbps / 32Gbps / 64Gbps … (Future)

§ Realtime e-VLBI capabilities- EVN/LBA: real-time e-VLBI correlation - VLBA/EAVN: not supported yet

EAVN correlation center(Daejeon, South Korea)

VLBA correlation center(Socorro, USA)

LBA correlation center(Perth, Australia)

EVN correlation center(Dwingeloo, Netherland)

LA

Australia

Singapore

New Zealand

Hawaii

BrazilRNP

New York

Montreal

Washington

ChicagoSeattle

Miami

Daejeon

(Korea)

TokyoHong

Kong

Taipei

Amsterdam

CERNParis

London

KREONet2, KOREN, Internet2/SingAREN, TransPAC, SXTransport, SINET, AMLight Exp, AGGCnet, APR(SingAREN/JGN), NEAAR,

ANA(Internet2, NORDUnet, SURFnet, CANARIE, GEANT), Esnet, CAE-1, GRP

Realtime observation and correlation using telescopes available all over the world

Automated GOLEs

NREN BNREN C

NREN A

Science DMZ

DTN

Science DMZ

DTN

Science DMZ

DTN

Science DMZ

DTN

Science DMZ

DTN

Observatory 1. Observatory 2. Observatory 3.

Observatory 4.

Observatory 5.

ps

ps

ps

ps

N*100Gbps

N*100Gbps

N*100Gbps

Science DMZ

DTN

Supercomputer

ps

Cloud ProviderScience DMZ

DTN ps

Science DMZ

DTN ps

Data Center B(Correlator)

Data Center A(Correlator)

KREONet2

Yonsei Astronomy Observatory @Seoul

Seoul

Jeju

Daejeon

Sejong

Korea-JapanCorrelation

Center (KJCC)@Daejeon

Tamna Astronomy Observatory@ Jeju

Ulsan Astronomy Observatory @Ulsan

100G

100G40G (+1G)

10G(+1G) 10G(+1G)

§ e-KVN: Korean e-VLBI(electronic Very Long Baseline Interferometry) Network

Ulsan

VLBI antenna @ Sejong Space Geodetic Observation Center

10G

@East Asia @Europe

@Australia

=> Toward Real-time e-VLBI observation with real-time correlation among observatories

Biomedical Science and Genomics

ICGC-PCAWG (ICGC-TCGA Whole Genome Pan-Cancer Analysis Working Group ) : 795.14 TB

TCGA (Cancer Genome Altas) :1.05PB/Sequence Read Archive

1000 Genome Project: 200TB/European Genome-Phenome Archive (EGA)

Cancer Cell LineEncylopedia : 30TB

CPTAC

Cancer Proteogenomics Project

Biomedical Science Community, KR

China

United States of America

Europe

Hong Kong

Seattle

Chicago

Amsterdam

CERN

KREONET POPKRLIGHT POP

10G

100G

Jeju

Changwon

IX Internet eXchange – up to 10G

Jeonju

Busan

Ulsan

Pohang

Gangneung

Incheon Seoul

LHCO

PN 1

0G

300G100

G

Gwangju

Daegu

Suwon

Cheonan

Sejong Ochang

Daejeon

100G

30G100G

100G

100G

100G100G

20G

30G

40G

40G 30G

20G

40G

20G

20G

10G

20G20G

Chuncheon

10G

- National-wide 100Gbps National Research Network- 17 Domestic GigaPoPs, 4 International GigaPoPs- 100G upgrade for international network: Seattle, Hong Kong- New GigaPoP in Korea: Chuncheon- User: about 200 national research institutes, universities etc.(500,000 users)

KREONET/KREONet2KOREA RESEARCH ENVIRONMENT OPEN NETWORK

DaeJeon

Seattle

Chicago

South-East/Middle Asia

▷ (Japan) KEK Bell Experiment▷ (Asia) CERN Tier2/3 Center▷ APEC, ASEAN Climate Center▷ (Japan) TEMDEC MediScience▷ (Korea) NFRI KSTAR▷ East Asia VLBI network

South America

▷ (Chille) CTIO KMTNet(KASI Alien Planetary System)

▷ (Chille) GMT (Giant Magellan Telescope) ▷ (Chille) LSST (Large Synoptic

Survey Telescope)

<Internet2(MoU )>

Europe

▷ (Swiss) CERN LHC▷ (France) ITER Grid▷ JIVE: European VLBI Network▷ (Norway) Satellie Data Receiver▷ (UK) SKA (Square Kilometre

Array)

Africa

▷ HartRao (Hartebeesthoek Radio Astronomy Observatory)

▷ SAAO (South Africa Astro-Obser.)- KMTNet (KASI)

▷ (South Africa) SKA (Square Kilometre Array)

Oceania

▷ (Autra.) SKA (Square KilometreArray) and SKA Pathfinder

▷ (Austra.) CSRIR/ATNF▷ (Austra.) SSO KMTNet

(KASI Alien Planetary System)

<PacWave (Partnership) >

<StarLight (Partnership) >

Middle East▷ (UAE) SKSH-SNU. Hospi.

Hybrid Diagnosis(Medical Korean wave)

North America

▷ (USA) NOAA (Ocean.-Atmos.▷ (USA) NASA (AeroSpace)▷ (CANADA) McGill University

– GBRAIN project▷ (USA) NERSC/BNL▷ (USA) Stanley institute : NGS ▷ (USA) LIGO Experiment▷ (CANADA) ICGC Genome

100G(+10G)

10G(+10G)

10G(+10G)

Amsterdam 10G

<Asi@Connect/SINET/CSTNET/CERNET(MoU ) >

100G(+10G)

100G(+10G)

GEANT (Agreement)

SURFnet (MoU)

Hong Kong

• Key Global Application Drivers in Korea and Asia• High Energy Physics & Fusion Energy• Astronomy

§ Global VLBI Alliance Initiative- Accessible to astronomers for all available world-wide telescope- Enabling real-time correlation as a global VLBI array

§ Rapid Response Science- Automated Dynamic Virtual Private Network

§ SKA, GMT, LSST, …• Biomedical Science and Genomics

§ ICGC, TCGA, ICGC-TCGA PCAWG, ..• AI enabled (HPC) Datacenter

bscho@kisti.re.kr