05 (idnog01) evolution of ixes and peering in japan by seiichi kawamura
DESCRIPTION
TRANSCRIPT
Evolu&on of peering and Internet Exchanges in Japan
[IDNOG 1]
Seiichi Kawamura BIGLOBE Inc.
as2518.peeringdb.com
copyright (c) 2014 BIGLOBE Inc. 1
copyright (c) 2014 BIGLOBE Inc. 2
Congratula&ons on the first mee&ng!
from
In 2012 One ques&on was asked
• This was a long forgoUen ques&on. ( in our community)
• It used to be easy. – You peered, because you needed routes.
• Now, the Internet is a given. – You can buy full routes easily...
• A stagnant market… – IX ports cost 20,000usd/month – The top ISPs never peer
• So why peer?
copyright (c) 2014 BIGLOBE Inc. 3
What does peering do for you?
copyright (c) 2014 BIGLOBE Inc. 4 photo by Megartworks hUp://www.flickr.com/photos/mgeartworks/10183970615/ used under crea&ve commons license 2.0 hUps://crea&vecommons.org/licenses/by/2.0/
2014 what is happening now (in JP)
• An evolving ecosystem
– Rise of peering communi&es – Cloud, Content, mobile driven
• Price destruc&on – 2 years ago 10G was 20,000usd -‐> has become close to interna&onal pricing level
• More technical experiments
copyright (c) 2014 BIGLOBE Inc. 5
Finally star&ng to catch up with the reali&es of the Internet
but first a bit of history
copyright (c) 2014 BIGLOBE Inc. 6
The history of Internet in Japan
• hUps://www.nic.ad.jp/&meline/en/
copyright (c) 2014 BIGLOBE Inc. 7
copyright (c) 2014 BIGLOBE Inc. 8
BIGLOBE
March 1994 The first IX in Japan!
1997 JANOG
• Peering between licensed telecoms required gov permission
• No domes&c peering – All traffic send to US
copyright (c) 2014 BIGLOBE Inc. 9
1) Before the IX 2) NSPIXP1
• Ini&ally layer3 IX based on peering with AS2500 (WIDE) – AS2500 was NOT considered a
telecom and could peer with anyone
• Ini&al traffic about 192kbps • Repeaters, Cisco2501, and
BGP3!!! – IIJ, Infoweb, SPIN, WIDE 3) on to NSPIXP2
shim to a Layer2 IX • Relocate to a real DC • Change to switching hubs
warning! history more than 20 years ago
Internet eXchanges in Japan now
copyright (c) 2014 BIGLOBE Inc. 10
Major city exchange
Global exchange
Equinix
Regional exchange
Akita IX, Okinawa IXand others…
Inter-‐region (MPLS-‐IX)
dis&x (dissolved)
Distribu&on throughout the country
copyright (c) 2014 BIGLOBE Inc. 11
Tokyo: -‐ JPNAP/JPNAP2 -‐ BBIX -‐ JPIX -‐ Equinix -‐ dix-‐ie (nspixp2)
Osaka: -‐ JPNAP -‐ BBIX -‐ JPIX -‐ NSPIXP3
Niigata: -‐ Echigo-‐IX
Akita: -‐ Akita-‐IX
Nagoya: -‐ BBIX -‐ JPIX
Fukuoka: -‐ BBIX
Cable Landing Sta&ons
Okinawa: -‐ OIX
Traffic in Japan
copyright (c) 2014 BIGLOBE Inc. 12
Source: hUp://www.soumu.go.jp/main_content/000244628.pdf
Download traffic at 2,275Gbps (28% from overseas)
Major Internet Exchange Architectures
copyright (c) 2014 BIGLOBE Inc. 13
op&cal switch
L2 Switch L2 Switch
L2 Switch L2 Switch
redundant rings
op&cal switch
Fully Redundant MPLS connected
Distributed MPLS IX Core
other IX ISP MPLS
Router
ISP MPLS Router
LSP
LSP
ISP Router
ISP Router
Changes coming from Mobile and Cloud area
• More mobile and cloud traffic – Broadband at 1.3x growth while mobile at 1.8x growth
• Content providers have different requirements compared to ISPs – semi-‐full route – simple route servers – latency aware – mtu9000 – fast detec&on using BFD – DoS protec&on as a service
copyright (c) 2014 BIGLOBE Inc. 14
Evolu&on of peering community
• Basically we had none – PAST: People disliked talking about peering at JANOG
• Peering relied heavily on ISPs
copyright (c) 2014 BIGLOBE Inc. 15
• BoFs, study councils, talks at NOGs, and joint efforts by the IXPs and the community – more focus on content/dc and its requirements
Star&ng an AS
1. get an AS number 2. get an engineer that has experience with BGP 3. buy transit from an upstream ISP 4. connect to an IX 5. peer with other AS 6. maintain the peer
copyright (c) 2014 BIGLOBE Inc. 16
The underlined steps have always been the problem, and this was because we did not have a
community to help out new comers
community ac&vi&es • Google Groups : Peering in Japan
– local Japanese language only – discussion on latest peering issues – no IX personnel on list – host Peering BoFs
• Tutorials – IX companies provide low cost tutorials (available regularly)
– free tutorials at JANOG (not always available) • CloudIX Study Council
– Group of members(BGP operators) in BBIX doing technical experiments
– sharing skills and helping out each other to peer
copyright (c) 2014 BIGLOBE Inc. 17
community effort example : PeeringDB
• What is PeeringDB? – hUp://www.peeringdb.com/ – A common place to publish informa&on about your AS (AS number, exchanges and facili&es you are at, your peering policy, your contact info, etc.)
– DO NOT use it to collect contacts for sales or spam you will be penalized
• This REALLY REALLY helps you to peer and maintain your network
• Good presenta&on here – hUps://fileshare.tools.isoc.org/mwangi/public/AXIS/Technical
%20Aspects%20Workshop/Day-‐5/Session-‐3/PeeringDB-‐and-‐Role-‐of-‐Peering-‐Coordinator.pdf
copyright (c) 2014 BIGLOBE Inc. 18
copyright (c) 2014 BIGLOBE Inc. 19
4 years ago nobody had a PeeringDB record in Japan
community efforts to
promote and help each other create records
Now everyone that I peer with has a record
and we are much much happier!
JANOG where we all come together
copyright (c) 2014 BIGLOBE Inc. 20
• Founded 1997 • 6400 mailing list members • Meetings
– Held twice a year, usually January and July. – 2 days of plenary sessions
• mostly focus on technical and operational discussions
– Run by 12 Committee members + Meeting volunteers – Presentations and discussions are in Japanese
• English acceptable but rough translation is recommended • around 10 non-native attendees
per meeting – ~800 attendees at Tokyo – ~400 attendees at other Cities
Plenary BoF
NOC Sake
Recent peering trends
copyright (c) 2014 BIGLOBE Inc. 22
Emerging Internet Exchange Architectures
copyright (c) 2014 BIGLOBE Inc. 23
SDN? Simple DIY?
OF Switch OF Switch
ISP Router
OF Switch
OF controller
ISP Router
virtual Router
L2 Switch
ISP Router
ISP Router
ISP Router
facility hosted by ISP: at your own risk IX WIDE Project
PIX-‐IE (SDN IX)
• What it aims to accomplish – Using OpenFlow and Cumulus Linux – Fine grain path control across an AS boundary – DDoS protec&on func&on – Value added services with Func&on Virtualiza&on – Experiment on what cannot be done with the legacy L2/L3 IX technology
• Interested? contact [sekiya at wide.ad.jp]
copyright (c) 2014 BIGLOBE Inc. 24
Recent topics being discussed in the community
• decentraliza&on from Tokyo • Full service IX and LCC IX • u&lizing SDN and API to interconnect clouds • how to reach MTU 9000 • more open peering discussions • which DC is best for peering • BCOP (best current opera&onal prac&ce) on peering document – hUp://www.internetsociety.org/deploy360/projects/bcop/
copyright (c) 2014 BIGLOBE Inc. 25
Summary : The lessons that we learned
in the past few years • An ac&ve community is key driver for an ac&ve peering landscape
• We’re not a “customer” of an IX, but rather are members and need to contribute to the community
• There’s s&ll gaps between ISPs and content, but it’s easier than before to talk about a solu&on, not just seUle with payment
• Being more open, invi&ng new people, will keep the community moving and it will lead to beUer opera&on prac&ces.
copyright (c) 2014 BIGLOBE Inc. 26
Acknowledgements
• Prof. Akira KATO and Prof. Yuji SEKIYA from WIDE Project
• Shingo KUDO from Sombank • Tom PASEKA from Cloudflare
copyright (c) 2014 BIGLOBE Inc. 27
The following persons helped review the content. Thank you!!!