© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Leo ZhadanovskyPrincipal Solutions Architect

Amazon Web Services

November 29, 2016

How Harvard University Improves Scalable Cloud Network

Security, Visibility, and Automation

SAC326

Thomas VachonManager of Cloud Architecture

Harvard University

What to expect from the session

Learn how Harvard designed and deployed the platform,

utilizing serverless architecture to orchestrate the solution

from within to protect their most sensitive data and afford

students, faculty, and staff the flexibility of cloud computing.

Connecting your on-premises

networks to Amazon VPCs

How to connect to your VPC

• Bastion host

• Site-to-site VPN

• AWS Direct Connect

virtual private

cloud

corporate data

center

How to connect to your VPC

• Bastion host

• Needs Elastic IP address

• Adds extra hop

• Single point of failure

• Simple

virtual private cloudcorporate data center

Bastion HostServer

How to connect to your VPC

• Site-to-site VPN

• AWS: Virtual private gateway

• On-premises: Customer gateway

• IKE, IPSec v2, BGP (optional but preferred)

• Can run into bandwidth limit with on-premises VPN devices

virtual private cloud

customer

gateway

VPN

gateway

VPN

connection

corporate data center

How to connect to your VPC

• AWS Direct Connect

• Dedicated, fiber connection between AWS and on-premises

• Available in 1 Gbps, 10 Gbps

• Many PoPs around the world

• Public and private VIFs available

• Transit over AWS backbone for US regions

• Routing priority

Virtual private cloud

customer

gateway

VPN

gateway

Corporate data center

AWS Direct

Connect

Network security options

Controlling network access in a VPC

• Security groups

• Network ACLs

• Routing tables

• Internet gateway

• NAT gateway

• S3 private endpoint

Internet

gateway

Route table

Security

group

VPC subnet

Network visibility

• AWS CloudTrail

• VPC Flow Logs

• Amazon S3 bucket logs

• Elastic Load Balancing logs

• AWS Config Flow logsAWS

CloudTrail

AWS

Config

IDS/IPS

• Agent-based solutions

• Available in AWS Marketplace

• Examples: Trend Micro Deep Security, Alert Logic Threat

Manager

• Costs usually scale by number of hosts

• Inline solutions

• Available in AWS Marketplace

• Examples: Cisco, Brocade, Fortinet, Palo Alto

• Single point of failure

IDS/IPS

• Egress through Direct Connect

• Use on-premises IDS/IPS devices

• There should be redundant Direct Connects

• Ideally, also diverse paths

• On-premises network becomes single point of failure for AWS

Internet connectivity

• Makes DNS more interesting

Harvard Cloud Shield

What is Cloud Shield?

• Network security platform

• Traffic aggregation and

inspection points

• Redundant and

geographically diverse

points of presence

Goals and alternatives

Solution overview: Design goals

• Provide highly available network access to the cloud

• Provide visibility of traffic into, out of, and between

applications

• Provide next-gen firewall protections such as IPS and

antivirus

• Provide simpler configuration through inline filtering

Security agents

• Easier configuration

• No additional overhead costs

• More expensive for customers

• Reactive response

Solution overview: Other options

Inline virtual firewalls

• Proactive response

• Cheaper for customers

• Very high overhead costs

• Complex VPC routing

Technical design overview

Network connectivity

Connectivity (2015)

Connectivity 2016 proposed

Connectivity 2016 actual

Network connectivity: Overview

• Four connections to AWS over Direct Connect

• Two private links between Harvard’s campus and

Virginia network point of presence

• No common spans or buildings between any links

Network design

Routing in detail

Routing in detail: Direct Connect

config router bfd

config neighbor

edit 10.254.1.4

set interface ”vlan10"

edit "10.254.1.4"

set advertisement-interval 1

set activate6 disable

set bfd enable

set prefix-list-in "vpc-cidr-network"

set remote-as 7224

set route-map-out "prepend-ASN"

set send-community6 disable

end

Routing in detail: Upstream router

template peer-policy cs-aws-peering

default-originate

advertisement-interval 0

send-community exit-peer-policy

template peer-session cs-aws-peering

timers 10 30

fall-over bfd

exit-peer-session

neighbor 10.254.1.2 remote-as 64816

neighbor 10.254.1.2 inherit peer-session cs-aws-peering

neighbor 10.254.1.2 description EBGP to atsdev1

address-family ipv4

aggregate-address 198.54.100.0 255.255.255.0 summary-only

Routing in detail: Key route filtering

config router prefix-list

edit “pub-nets”

set prefix 198.54.100.0 255.255.255.0

set le 32

end

edit "vpc-cidr-network”

set prefix 10.0.0.0 255.255.240.0

unset ge

unset le

end

Network orchestration

Network orchestration: Overview

• Developed a server-less architecture for a manager of

managers

• Built on Python and overlays 5 different network

management products or networking devices

• Utilize a schema-less managed NoSQL database to

pass state between different components

Lessons learned

Lessons learned: Business

• Ensure network security is

in place first

• Align with your technology

providers and vendors

• Have key business

sponsors

• Constant communication is

essential

Lessons learned: Network design

• Stateful failover isn't

practical

• Failing over sites

periodically is a must

• Network interoperability is a

myth

Lessons learned: Routing

• iBGP and eBGP function

differently

• Graceful restart is not

always ideal

• Use BFD on every network

hop

• Terminate public peering at

each network PoP

Lessons learned: Connectivity

• Path selection is critical and

hard

• The price of a service does

not imply quality of a

service

• Use multiple Direct Connect

endpoints

Lessons learned: Orchestration

• Not all APIs are created

equal (or exist)

• Network vendors are not

software engineers

• Ensure all values are

externally configurable

Thank you!

Remember to complete

your evaluations!