aws re:invent 2016: from one to many: evolving vpc design (arc302)
TRANSCRIPT
![Page 1: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Rob Alexander, Principal Solutions Architect
December 2, 2016
ARC302
From One to ManyEvolving VPC Design
![Page 2: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/2.jpg)
Disclaimer:
Do Try This at Home!
![Page 3: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/3.jpg)
Assuming you’ve heard of…
Route Table
Elastic
Network
Interface
Amazon VPC
Internet
Gateway
Customer
Gateway Virtual
Private
Gateway
VPN
Connection
VPC subnet
Network ACL
Security group
Enhanced
Networking
VPC
Peering
AWS Direct
Connect
![Page 4: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/4.jpg)
Related Sessions
NET201 – Creating Your Virtual Data Center: VPC
Fundamentals and Connectivity Options
NET305 – Extending Datacenters to the Cloud:
Connectivity Options and Considerations for Hybrid
Environments
![Page 5: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/5.jpg)
From one…
Subnet
Availability Zone A
Subnet
Availability Zone B
VPC
![Page 6: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/6.jpg)
us-east-2
VPCVPC
VPC
VPCTransit VPC
VPC
us-west-2
VPC
VPC
VPC
eu-west-1
VPCVPC
VPC
VPCTransit VPC
VPC
Branch Branch
NA
HQ
VPC
VPC
VPC
VPC
VPC
VPC
Chicago DX
AP
HQ
London DX
ap-northeast-1
VPC
VPC
VPC
VPCTransit VPC
VPC
EU
HQ
Tokyo DX … to many
![Page 7: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/7.jpg)
VPC/16
Choose a CIDR
• CIDR fixed on VPC
creation
• /16 down to /28
• Go Big
![Page 8: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/8.jpg)
VPC IPv4 space design
• Plan for expansion to additional Availability
Zones or regions
• Consider connectivity to corporate networks
• Don’t overlap IP space
• Save space for the future
• IPv4 space is required, but …
![Page 9: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/9.jpg)
IPv6 now supported in VPC
• Optionally enable IPv6 on VPC
• /56 of Amazon’s Global Unicast Address (GUA) per VPC
• /64 CIDR block per subnet
• IPv6 completely independent from IPv4
• Enabled per subnet or per instance (per ENI)
• Supported by Security Groups, Route Tables, NACLs, VPC
Peering, IGW, DX, Flow Logs and DNS Resolution
![Page 10: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/10.jpg)
Availability Zone A
VPC
• Even distribution of IP
space across AZs
• Use at least 2 AZs
• Subnets are AZ
specific
• How big? How many?
Create subnets
Subnet
Availability Zone B
Subnet
Availability Zone C
Subnet
/16
![Page 11: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/11.jpg)
Availability Zone A
Subnet
VPC
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet Subnet Subnet Subnet Subnet Subnet
Subnet Subnet Subnet Subnet Subnet
Subnet Subnet Subnet Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
/16
![Page 12: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/12.jpg)
VPC subnet design
• Traditional switching limitations do not apply
• Consider large, mixed use subnets
• Use security groups to enforce isolation
• Use tags for grouping resources
• Use subnets as containers for routing policy
![Page 13: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/13.jpg)
Related Sessions
NET401 – Another Day, Another Billion Packets
![Page 14: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/14.jpg)
Availability Zone A
Public subnet
Private subnet
Availability Zone B
Public subnet
Private subnet
VPC/16
Availability Zone C
Public subnet
Private subnet
/22 /22 /22
/20 /20 /204091 IPs
1019 IPs
![Page 15: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/15.jpg)
VPC/16
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
Availability Zone C
Private subnet
Public subnet
Private subnet
/22 /22 /22
/20
/20
/20
/20
/20
/20
![Page 16: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/16.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
.1
VPC
.1
.1 .1
.1 .1
Routing Policy
Main Route Table
Destination Target
10.1.0.0/16 Local
VPC CIDR 10.1.0.0/16
![Page 17: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/17.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
Routing Policy
AWS Region
Internet
Public Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 IGW
![Page 18: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/18.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
Routing Policy
AWS Region
Internet
Private Route Table
Destination Target
10.1.0.0/16 Local
Corp CIDR VGW
![Page 19: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/19.jpg)
Availability Zone A
Public subnet
Private subnet
Availability Zone B
VPC/54
Availability Zone C
/64
/64
18 MILLION,
Public subnet
Private subnet
Public subnet
Private subnet
What about IPv6?
/64
/64
/64
/64
TRILLIONIPs
![Page 20: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/20.jpg)
Availability Zone A
Private subnet
Public subnet
Availability Zone B
Private subnet
Public subnet
VPC
IPv6
Routing Policy
AWS Region
Internet
Public Route Table
Destination Target
10.1.0.0/16 Local
2001:db8:1234:1a00::/56 Local
0.0.0.0/0 IGW
::/0 IGW
![Page 21: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/21.jpg)
Availability Zone A
Private subnet
Public subnet
Availability Zone B
Private subnet
Public subnet
VPC
IPv6
Routing Policy
AWS Region
Internet
Public Route Table
Destination Target
10.1.0.0/16 Local
2001:db8:1234:1a00::/56 Local
Corp CIDR VGW
::/0 EIGW
Egress-Only IGW
![Page 22: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/22.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
Routing Policy
AWS Region
Internet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 ???
Corp CIDR VGW
![Page 23: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/23.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
Routing Policy
AWS Region
Internet
Why go outside?
• AWS API endpoints
• Regional services
• Third-party services
![Page 24: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/24.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
Routing Policy
AWS Region
Internet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT Instance
Corp CIDR VGW
NAT
Instance
![Page 25: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/25.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
Routing Policy
AWS Region
Internet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT Instance
Corp CIDR VGW
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 Black Hole
Corp CIDR VGW
NAT
Instance
![Page 26: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/26.jpg)
Scalable and Available NAT
![Page 27: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/27.jpg)
Evolving design requirements
• Public subnets for resources reachable from Internet
• Private subnets with egress only access to public network
• Scalable, highly available NAT
• One AWS account
• One VPC
• One region
![Page 28: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/28.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
AWS Region
Internet
NAT
Instance
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 Black Hole
Corp CIDR VGW
![Page 29: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/29.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
AWS Region
Internet
Deploy a
NAT Gateway
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT Gateway
Corp CIDR VGW
NAT
Gateway
![Page 30: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/30.jpg)
Why a NAT Gateway?
10.1.1.112:54318 52.27.192.88:35678
NAT Instance
Source IP: Port NAT’d Source IP:Port
Security Updates
Package Repos
NTP
VPC
Public Network
![Page 31: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/31.jpg)
Why a NAT Gateway?
10.1.1.112:54318 52.27.192.88:35678
Source IP: Port NAT’d Source IP:Port
VPC
Source IP is the same
Source Port must be
unique
Destination
IP and Port
are the same
NAT Instance
Public Network
52.27.192.88:33622
52.27.192.88:38438
52.27.192.88:48132
52.27.192.88:29754Security Update
![Page 32: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/32.jpg)
Why a NAT Gateway?
10.1.1.112:54318 52.27.192.88:35678
Source IP: Port NAT’d Source IP:Port
VPC
Source IP is the same
Source Port must be
unique
Destination
IP and Port
are the same
Public Network
52.27.192.88:33622
52.27.192.88:38438
52.27.192.88:48132
52.27.192.88:29754
NAT Gateway
Security Update
![Page 33: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/33.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
AWS Region
Internet
Deploy a
NAT Gateway
NAT
Gateway
• Still need IGW
• Separate subnets
• Requires EIP
• AZ specific
• Burst to 10 Gbps
![Page 34: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/34.jpg)
1
NAT Gateway: Securing Access
NAT Gateway ENI:
Network ACL
Public subnet
NAT
GatewayNetwork ACLs still apply
![Page 35: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/35.jpg)
NAT Gateway: Securing Access
Use routing
policy to control
access to NAT
Gateway
Private subnet
Public subnet
Private subnet
NAT Enabled
no-NAT
no-NAT Private Route Table
Destination Target
10.1.0.0/16 Local
NAT Enabled Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT Gateway
NAT
Gateway2
![Page 36: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/36.jpg)
NAT Gateway: Securing Access
Outbound Rules
Type Protocol Port Range Destination
All traffic All 0 - 65535 0.0.0.0/0
Use security groups
to restrict outbound
access for instances
Default VPC security group:
3
![Page 37: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/37.jpg)
NAT Gateway: Securing Access
Outbound Rules
Type Protocol Port Range Destination
All traffic All 0 - 65535 10.2.0.0/16
Outbound Rules
Type Protocol Port Range Destination
All traffic All 0 - 65535 0.0.0.0/0
Use security groups
to restrict outbound
access for instances
Default VPC security group:
NAT Enabled VPC security group:
3
![Page 38: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/38.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
AWS Region
Internet
Deploy a
NAT Gateway
NAT
Gateway
NAT Enabled
no-NAT
NAT Enabled
no-NAT
![Page 39: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/39.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Availability Zone B
Private subnet
Public subnet
Private subnet
VPC
AWS Region
Internet
Deploy a
NAT Gateway
NAT
Gateway
NAT
Gateway
NAT Enabled
no-NAT
NAT Enabled
no-NAT
![Page 40: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/40.jpg)
• Drop in replacement for NAT instance
• Fully managed
• Highly available and fault tolerant
• Scalable to 10 Gbps burst per gateway
• Supports VPC Flow Logs
• No higher level functions like IPS, UTM,
URL Filtering, packet inspection, etc
• Cannot associate security group to
gateway
Pro & Con: NAT Gateway
![Page 41: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/41.jpg)
AWS
Region
Considering multiple VPCs
Public-facing
web apps
Internal
company
apps
What’s next?
VPN
connection
VPC VPC VPC
Customer
network
![Page 42: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/42.jpg)
One VPC, Two VPC
![Page 43: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/43.jpg)
VPC
Why not 1 big VPC?
Why not 1 AWS Account?
• Blast radius
• Account Limits
• API Limits
![Page 44: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/44.jpg)
Considerations for one or many VPCs
AWS Region
ProdNot
Prod
VPCVPC
![Page 45: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/45.jpg)
Considerations for one or many VPCs
AWS Region
PCI
Apps
VPC VPC
Non
Regulated
Apps
![Page 46: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/46.jpg)
Considerations for one or many VPCs
AWS Region
Prod
VPC
AWS Region
Disaster
Recovery
VPC
![Page 47: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/47.jpg)
Considerations for one or many VPCs
AWS RegionVPC
Audit
Logging &
Analytics
AWS
CloudTrail
AWS
Config
VPC Flow
Logs
VPC
Legal
VPC
Finance
VPC
Sales
App Logs,
S3 Access Logs,
ELB Logs
Amazon
Redshift
Amazon
EMR
S3
![Page 48: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/48.jpg)
AWS Region
Internal application to VPC
Public-facing
web app
Internal
company
app
VPN
connection
VPCVPC
Customer
network
![Page 49: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/49.jpg)
Availability Zone A
Private subnet Private subnet
AWS Region
Virtual
Private
Gateway
VPN
connection
Intranet
app
Intranet
app
Availability Zone B
Internal customers
Private Route Table
Destination Target
10.1.0.0/16 Local
Corp CIDR VGW
VPC
Internal application to VPC
Customer
network
![Page 50: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/50.jpg)
But apps will make heavy use of …
Amazon S3
…as a primary data store
![Page 51: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/51.jpg)
VPC Egress Control
![Page 52: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/52.jpg)
Evolving design requirements
• VPN connectivity to private-only VPC
• No egress in the VPC to public networks
• Private IP access to Amazon S3
• Content-specific access controls
• One AWS account
• One VPC
• One region
![Page 53: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/53.jpg)
Availability Zone A
Private subnet Private subnet
AWS
Region
Virtual
Private
Gateway
VPN
connection
Intranet
app
Intranet
app
Availability Zone B
You really don’t want to do this:
Amazon
S3
Internet
Customer
border router
Customer VPN
Internet
VPC
Customer
network
![Page 54: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/54.jpg)
Availability Zone A
Private subnet Private subnet
AWS
Region
Virtual
Private
Gateway
Intranet
app
Intranet
app
Availability Zone B
So do this instead:
Amazon
S3
VPC
VPN
connection
VPC Endpoints
• No IGW
• No NAT
• No public IPs
• Free
• Robust access
control
Customer
network
![Page 55: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/55.jpg)
Creating S3 VPC endpoint
aws ec2 create-vpc-endpoint
--vpc-id vpc-40f18d25
--service-name com.amazonaws.us-west-2.s3
--route-table-ids rtb-2ae6a24f rtb-61c78704
Private subnet
VPCRoute Table
Destination Target
10.1.0.0/16 Local
Corp CIDR VGW
Prefix List for S3 us-west-2 VPCE
![Page 56: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/56.jpg)
Creating S3 VPC endpoint
aws ec2 create-vpc-endpoint
--vpc-id vpc-40f18d25
--service-name com.amazonaws.us-west-2.s3
--route-table-ids rtb-2ae6a24f rtb-61c78704
Public subnet
VPCRoute Table
Destination Target
10.1.0.0/16 Local
0.0.0.0 IGW
Prefix List for S3 us-west-2 VPCE
![Page 57: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/57.jpg)
Creating S3 VPC endpoint
Private subnet
VPC
Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0 NAT Gateway
Prefix List for S3 us-west-2 VPCE
Public subnet
NAT
Gateway
![Page 58: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/58.jpg)
Prefix lists
aws ec2 describe-prefix-lists
PREFIXLISTS pl-68a54001 com.amazonaws.us-west-2.s3
CIDRS 54.231.160.0/19
CIDRS 52.218.128.0/18
• Logical route destination target
• Dynamically translates to service IPs
• S3 IP ranges change over time
• S3 prefix lists abstract change
![Page 59: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/59.jpg)
Prefix lists
… and use them in your outbound security group rules!
![Page 60: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/60.jpg)
Private subnet
Controlling VPC access to Amazon S3
AWS Identity & Access
Management (IAM) policy
on VPCE:
VPC
{
"Statement": [
{
"Sid": "vpce-restrict-to-backup-bucket",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:PutObject”
],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::backups-reinvent",
"arn:aws:s3:::backups-reinvent/*"]
}
]
}
Backups bucket?
![Page 61: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/61.jpg)
Private subnet
Controlling VPC access to Amazon S3
S3 bucket policy:
VPC
From
vpce-bc42a4e5?
{
"Statement": [
{
"Sid": "bucket-restrict-to-specific-vpce",
"Principal": "*",
"Action": "s3:*",
"Effect": "Deny",
"Resource": ["arn:aws:s3:::backups-reinvent",
"arn:aws:s3:::backups-reinvent/*"],
"Condition": {
"StringNotEquals": {
"aws:sourceVpce": "vpce-bc42a4e5”
}
}
}
]
}
![Page 62: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/62.jpg)
Controlling VPC access to Amazon S3
Recap on security layers:
1. Route table association
2. VPCE policy
3. Bucket policy
4. Security groups with prefix list
Private subnet
VPC
1.
2.
3.
4.
![Page 63: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/63.jpg)
Private subnet Private subnet
AWS
Region
Intranet
apps
Compliance
app
Endpoints in action
VPC
Compliance Backups
VPCE1 VPCE2
Private subnet
Intranet
apps
![Page 64: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/64.jpg)
Private subnet Private subnet
AWS
Region
Intranet
apps
Compliance
app
Endpoints in action
VPC
Compliance Backups
VPCE1 VPCE2
Private subnet
Intranet
appsPrivate subnet Private subnet
Private subnet
Logs Analytics
![Page 65: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/65.jpg)
• Secure, highly scalable and highly
available access to S3
• Fine grained control of access to
content in S3 from VPC
• Control which VPCs/VPCEs can
access which S3 buckets
• No public IPs required, source IPs kept
private
• Bucket policy restricted to specific
VPCs (or VPCEs) will disable S3
Console access
• Requires Amazon DNS enabled on
VPC
Pro & Con: VPC Endpoints
![Page 66: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/66.jpg)
AWS Region
Public-facing
web apps
Internal-
only
apps
What’s next?
VPN
connection
VPC VPC VPC
Customer
networkCustomer Gateway
(CGW)
![Page 67: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/67.jpg)
Shared Service Hubs
![Page 68: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/68.jpg)
AWS
Region
VPC
VPC
VPC
VPC VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPCVPC
Customer
network
Public
apps
Internal
apps
![Page 69: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/69.jpg)
AWS Region
VPC
HA VPN Pair
Availability Zone A
HA VPN
To
VPC
iBGP
eBGP
Customer CIDRs or Default Route
eBGP
AWS ASN 7224
Re-advertise VPC CIDR via IGP
VGW
VPC CIDR
Customer ASN (Public or Private)
CGW1 CGW2
VPN1
Tun1
VPN1
Tun2
Availability Zone A
VPN2
Tun1
VPN2
Tun2
Reuse your CGW Public IP
to connect to more VPCs
Customer
network
MED
MED
![Page 70: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/70.jpg)
AWS
Region
VPCVPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
• DNS
• Directory
• Logging
• Monitoring
• SecurityShared services
Customer
network
![Page 71: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/71.jpg)
Evolving design requirements
• Centralize network connectivity to and from cloud
• Centralize management, security, and common services
• Account owners in control of own VPC resources
• Many AWS accounts
• Many VPCs
• One region
![Page 72: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/72.jpg)
AWS
RegionVPC
VPC
VPC
VPC
VPC
VPC
• DNS
• Directory
• Logging
• Monitoring
• SecurityShared services
Hub and
Spoke
with
Peering
VPC
Shared
services
VPC
VPC
Customer
network
Spoke VPC
Spoke VPC
Spoke VPCSpoke VPC
Spoke VPC
Spoke VPC
VPC
![Page 73: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/73.jpg)
Customer
network
AWS Region
VPC
Hub VPC
Private subnet
VPC
Spoke VPC
Public subnet
10.2.0.0/1610.1.0.0/16
Private subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 PCX-1
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.11.0/24 PCX-1
VPC peering
Shared services
10.2.22.0/24
10.1.11.0/24
![Page 74: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/74.jpg)
AWS Region
VPC
Hub VPC
Private subnet
VPC
Spoke VPC
Public subnet
10.2.0.0/1610.1.0.0/16
Private subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 PCX-1
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.11.0/24 PCX-1
172.16.0.0/16 PCX-1
Edge-to-edge routing
Shared services
10.2.22.0/24
10.1.11.0/24
172.16.0.0/16Customer
network
![Page 75: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/75.jpg)
AWS Region
VPC
Hub VPC
Private subnet
VPC
Spoke VPC
Proxy
subnets
10.2.0.0/1610.1.0.0/16
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.0.0/16 PCX-1
Edge-to-edge via proxy
PCX-1 10.2.22.0/24
Internal
ELB
Proxy
fleet
Internet
Public
services
S3VPC
Customer
network
Proxy Route Table
Destination Target
10.1.0.0/16 local
10.2.0.0/16 PCX-1
172.16.0.0/16 VGW
Proxy Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 PCX-1
172.16.0.0/16 VGW
0.0.0.0/0 IGW
S3 Prefix List VPCE
![Page 76: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/76.jpg)
Customer
network
Availability Zone A
Private subnet
Public subnet
Private subnet
Elastic
Load
Balancer
Shared
services
AWS Region
Internet
VPC
Auto Scaling
proxy
fleet
Public
servicesS3
PCX-1
Availability Zone B
Private subnet
Public subnet
Private subnet
Elastic
Load
Balancer
Shared
services
Auto Scaling
proxy
fleet
Spoke VPC
VPC
Private subnet
Proxy in practice
Hub VPC
![Page 77: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/77.jpg)
Availability Zone A
Private subnet
Public subnet
Private subnet
Elastic
Load
Balancer
Shared
services
AWS Region
Internet
VPC
Auto Scaling
proxy
Fleet
Public
servicesS3
PCX-1
Availability Zone B
Private subnet
Public subnet
Private subnet
Elastic
Load
Balancer
Bastion
host
Auto Scaling
proxy
fleet
Spoke VPC
VPC
Private subnet
Proxy in practice
Hub VPC
Customer
network
![Page 78: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/78.jpg)
Shared Services Hub: To-Do List
• Use IAM to restrict spoke AWS accounts from altering network
• Create a NetOps IAM role in all accounts:https://aws.amazon.com/blogs/security/how-to-assign-permissions-using-new-aws-managed-policies-for-job-functions/
• Enable AWS CloudTrail, AWS Config, and VPC Flow Logs for all accounts
• Integrate CloudTrail with CloudWatch Logs and create alarms:https://aws.amazon.com/blogs/aws/cloudtrail-integration-with-cloudwatch-now-available-in-four-more-regions
![Page 79: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/79.jpg)
• Minimizes on premises network change
• Reduces latency, cost of cloud
applications accessing common services
• Provides spoke accounts control over
own resources
• But controls and secures egress traffic
from spokes
• Security Groups work across peers
• Cost and management of central proxy
layer
• Not a transparent proxy
• Configuring end devices to use proxy
• Restricted to HTTP/S
• No transitive networking
• Peering data transfer cost
Pro & Con: Shared Services Hub and Spoke
![Page 80: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/80.jpg)
AWS Region
VPCVPC
VPC
VPC
VPC
• DNS
• Directory
• Logging
• Monitoring
• Security
VPC
VPC
VPC
VPC
VPC VPC
VPC
VPC
VPC
VPC
Shared services
Customer
network
Dev hubProd hub
Data
services
hub
![Page 81: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/81.jpg)
AWS Region
VPCVPC
VPC
VPC
VPC
• DNS
• Directory
• Logging
• Monitoring
• Security
VPC
VPC
VPC
VPC
VPC VPC
VPC
VPC
VPC
VPC
Shared services
Customer
network
Dev hubProd hub
Data
services
hub
VPC
VPC
![Page 82: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/82.jpg)
Customer
network
AWS Region
Availability Zone A
Private subnet
VPC
Availability Zone B
Private subnet
AWS Lambda
Amazon API Gateway
Elastic
Network
Interface
VPCVPC
VPC
VPC
Prod hub
VPC
Internet
Hybrid
Serverless
Amazon
AuroraReplica
Mobile Application VPC
![Page 83: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/83.jpg)
Legacy
Apps
Customer
network
AWS Region
Availability Zone A
Private subnet
VPC
Availability Zone B
Private subnet
AWS Lambda
Amazon API Gateway
Elastic
Network
Interface
VPCVPC
VPC
VPC
Prod hub
VPC
Internet
Hybrid
Serverless
Amazon
AuroraReplica
Mobile Application VPC
![Page 84: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/84.jpg)
us-east-2 region
VPC VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC VPC
VPC
VPC
VPC
VPC
eu-west-1 region
VPC VPC
VPC
VPC
VPC
VPC
![Page 85: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/85.jpg)
VPC Mass Transit
![Page 86: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/86.jpg)
Evolving design requirements
• Centralize and minimize network connections
• Allow end to end routing from cloud to existing networks
• Minimal operational overhead
• Leverage AWS network
• Many AWS accounts
• Many VPCs
• Many regions
![Page 87: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/87.jpg)
Availability Zone A
Public subnet
VPC
Transit
VPC
Availability Zone B
Public subnet
AWS
Region
EC2 VPN EC2 VPN
![Page 88: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/88.jpg)
Availability Zone A
Public subnet
VPC
Transit
VPC
Availability Zone B
Public subnet
EC2 VPN EC2 VPN
AWS
RegionVPC
Spoke VPC
Transit VPC
VPCSpoke VPC
VPCSpoke VPC
![Page 89: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/89.jpg)
AWS
RegionVPC
VPC
VPC
VPC
VPC
VPC
VPCTransit VPC
Customer
network
Spoke VPC
Spoke VPC
Spoke VPCSpoke VPC
Spoke VPC
Spoke VPC
Branches
Transit
VPC
![Page 90: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/90.jpg)
https://aws.amazon.com/answers/networking/transit-vpc/
Transit VPC
![Page 91: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/91.jpg)
Transit VPC
Built using Cisco Cloud Services Router (CSR) 1000V
• Available on the AWS Marketplace
• A virtualized ASR with full IOS-XE software stack
• BYOL or Pay-as-you-Go license models
![Page 92: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/92.jpg)
Availability Zone A
Public subnet
VPC
Availability Zone B
Public subnet
CSR1 CSR2
AWS Region
Transit VPC
S3 Bucket
for
VPN Config
Route Table
Destination Target
100.64.127.224/27 Local
0.0.0.0 IGW
Prefix List for S3 VPCE
100.64.127.224 / 27
Transit VPC:
Creation
![Page 93: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/93.jpg)
What is EC2 Auto Recovery?
RECOVER Instance
Instance ID
Instance metadata
Private IP addresses
Elastic IP addresses
EBS volume attachments
Instance retains:
* Supported on C3, C4, M3, M4, P2, R3, T2, and X1 instance types with EBS-only storage
StatusCheckFailed_System
Amazon CloudWatch
per-instance metric alarm:
When alarm triggers?
![Page 94: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/94.jpg)
Availability Zone A
Public subnet
VPC
Availability Zone B
Public subnet
CSR1 CSR2
AWS Region
Transit VPC
S3 Bucket
for
VPN Config
VPCSpoke VPC
AWS Lambda
Cisco
Configurator
AWS Lambda
VGW Poller
transitvpc:spoke = true
Transit VPC:
Add Spoke
SSH Only to CSR Security Group
![Page 95: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/95.jpg)
Availability Zone A
Public subnet
VPC
Availability Zone B
Public subnet
AWS Region
Transit VPC
VPCSpoke VPCTransit VPC:
Preferred
Route
Spoke VPC Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0 VGW
Transit VPC Route Table
Destination Target
100.64.127.224/27 Local
0.0.0.0 IGW
Prefix List for S3 VPCE
Active / Active
![Page 96: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/96.jpg)
Availability Zone A
Public subnet
VPC
Availability Zone B
Public subnet
AWS Region
Transit VPC
VPCSpoke VPC
transitvpc:preferred-path = CSR1
Transit VPC:
Preferred
Route
Spoke VPC Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0 VGW
Transit VPC Route Table
Destination Target
100.64.127.224/27 Local
0.0.0.0 IGW
Prefix List for S3 VPCE
Spoke VGW Tag
Active / Passive
![Page 97: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/97.jpg)
Transit VPC: Preferred route spoke configuration
From CSR2:
!
address-family ipv4 vrf vpn-8a23d2e3
neighbor 169.254.35.57 remote-as 7224
neighbor 169.254.35.57 timers 10 30 30
neighbor 169.254.35.57 activate
neighbor 169.254.35.57 as-override
neighbor 169.254.35.57 soft-reconfiguration inbound
neighbor 169.254.35.57 route-map rm-vpn-8a23c7e3 out
exit-address-family
!
route-map rm-vpn-8a23c7e3 permit 10
set as-path prepend 64512 64512
!
BGP AS override
configured by default
![Page 98: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/98.jpg)
Availability Zone A
Public subnet
VPC
Availability Zone B
Public subnet
CSR1 CSR2
AWS Region
Transit VPC
S3 Bucket
for
VPN Config
VPCSpoke VPC
AWS Lambda
Cisco
Configurator
AWS Lambda
VGW Poller
transitvpc:spoke = false
Transit VPC:
Remove Spoke
![Page 99: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/99.jpg)
AWS
RegionVPC
VPC
VPC
VPC
VPC
VPC
VPC
Transit VPC
Customer
network
Spoke VPC
Spoke VPC
Spoke VPCSpoke VPC
Spoke VPC
Spoke VPC
Branches
Transit
VPC
Internet
Public
services
![Page 100: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/100.jpg)
Customer
network
VPCTransit VPC
us-east-2
us-west-2
VPC
VPC
Spoke VPC
Spoke
VPC
VPCTransit VPC
eu-west-1
eu-central-1
VPC
VPC
Spoke VPC
Spoke
VPC
AWS Network
Backbone
Internet
![Page 101: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/101.jpg)
• End to End routing between VPCs in all
regions and any other non-AWS network
• Central transit routers can perform higher
level networking and security functions
• Spoke VGWs are HA by default
• Minimizes on premises networking changes
• Can minimize cost if replacing on premises
or colo networking hardware
• Availability and management of transit router
instances
• Licensing costs
• Cost of data transfer between transit, spokes
and other networks
Pro & Con: Transit VPC
![Page 102: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/102.jpg)
AWS
Region VPC
VPC
VPC
VPC
VPCTransit VPC
Spoke VPC
Spoke VPCSpoke VPC
Spoke VPC
Transit VPC
with
AWS Direct
Connect
(DX)
Detached
VGW
transitvpc:spoke = true
Customer
network
AWS Direct Connect
location
Private virtual interface (VIF) to
detached VGW• 1 PVI per VGW
• 1 BGP ASN
• 1 802.1Q VLAN Tag
• 1 BGP MD5 key
Private fiber connectionOne or multiple
50 – 500 Mbps,
1 Gbps or 10 Gbps pipes
![Page 103: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/103.jpg)
AWS
Region VPC
VPC
VPC
VPC
VPCTransit VPC
Spoke VPC
Spoke VPCSpoke VPC
Spoke VPC
Customer
network
AWS Direct Connect
location
Private DX VIF to
dedicated VGW
100.64.127.224 / 27
Private Virtual Interface 1
VLAN Tag 101
BGP ASN 7224
BGP Announce 100.64.127.224/27
Interface IP 169.254.251.5/30
Customer Interface 0/1.101
VLAN Tag 101
BGP ASN 65001
BGP Announce Customer Internal
Interface IP 169.254.251.6/30
![Page 104: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/104.jpg)
AWS
Region VPC
VPC
VPC
VPC
VPCTransit VPC
Spoke VPC
Spoke VPCSpoke VPC
Spoke VPC
Customer
network
AWS Direct Connect
location
Public DX VIF to
dedicated VGW
Public EIPs
Public Virtual Interface 1
VLAN Tag 501
BGP ASN 7224
BGP Announce AWS Regional
Public CIDRs
Interface IP Public /30 Provided
Customer Interface 0/1.501
VLAN Tag 501
BGP ASN 65501 (or Public)
BGP Announce Customer Public
Interface IP Public /30 Provided
NAT + Security layer
![Page 105: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/105.jpg)
Equinix Chicago
Customer
network
us-west-2
VPC
VPC
VPCVPC
Transit VPC
VPC
us-east-2
VPC
VPC
VPCVPC
Transit VPC
VPC
AWS Direct Connect Inter-Region Connectivity
A single DX Public interface can reach all US regions
![Page 106: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/106.jpg)
• Be selective in your public network announcements
• Filtering public prefix announcements if necessary
• Authoritative AWS public IP list available:
https://ip-ranges.amazonaws.com/ip-ranges.json
• For notification of IP changes, subscribe to SNS topic:
arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged
AWS Direct Connect Public Interface
![Page 107: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/107.jpg)
Related Sessions
NET402 – Deep Dive: AWS Direct Connect and VPNs
![Page 108: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/108.jpg)
Leverage corporate network
Headquarters
Branch
Branch
DX Location
Provider Edge (PE)Customer Edge (CE)
eBGP
Provider
MPLS
Network
PECE
PE
CE
eBGP
AWS Region
MPLS / IPVPN
PE DX
eBGP
CE PE
![Page 109: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/109.jpg)
Headquarters
Branch
Branch
Chicago DX Location
eBGP
Provider
MPLS
Network
PECE
PECE
AWS
Ohio
region
Multi-region DX
PE DX
eBGP
CE PE
London DX Location
AWS
Ireland
region
PE DX
eBGP
Going global
AS 7224
AS 7224
100 BGP Route Max
100 BGP Route Max
![Page 110: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/110.jpg)
• Private network, no Internet dependencies
• Predictable latency on DX connections
• Dedicated bandwidth to AWS
• Access to public networks of all US regions
over single US based DX connection
• Public DX BGP announcements may require
filtering
• For large networks, 100 route per VPC limit
may require summarization or default routes
• Cost of provider network and DX connections
Pro & Con: Transit VPC with DX
![Page 111: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/111.jpg)
us-east-2
VPCVPC
VPC
VPCTransit VPC
VPC
us-west-2
VPC
VPC
VPC
eu-west-1
VPCVPC
VPC
VPCTransit VPC
VPC
AWS Network
Backbone
Provider
MPLS
Network
Branch Branch
NA
HQ
VPC
VPC
VPC
VPC
VPC
VPC
Chicago DX
AP
HQ
London DX
ap-northeast-1
VPC
VPC
VPC
VPC
Transit VPC
VPC
EU
HQ
Tokyo DX
![Page 112: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/112.jpg)
Thank you!
![Page 113: AWS re:Invent 2016: From One to Many: Evolving VPC Design (ARC302)](https://reader031.vdocument.in/reader031/viewer/2022030317/586f90421a28ab54768b78f1/html5/thumbnails/113.jpg)
Remember to complete
your evaluations!