openshift container netzwerk aus sicht der workloadpeople.redhat.com/~llange/ocp_netzwerk.pdf ·...
TRANSCRIPT
Wolfram Richter | Red Hat
OpenShift Container Netzwerkaus Sicht der Workload
Why this session?
“OpenShift is great for web applications,
but we want to do X… will this work?“
X ∈ {“Analytics”, “Non-HTTP”, “High-
Performance Computing”, “Big Data”,
“Object storage”, “NAS”, “Replicated
Databases”, …}
Let’s take a look from a networking PoV!
Agenda
• What is OpenShift?
• How plain Docker Networking works
and what OpenShift does differently
• Container Networking across nodes
• Kubernetes Services
• Ingress: OpenShift Router
• Egress Pods and Network Policy
OpenShift
● Platform & Container as a Service
● Built for both traditional and cloud-native applications
● An integrated hybrid cloud application platform for application development and deployment
● Develop, build, and manage container based applications
● Easily turn source code into running applications with source-to-image capabilities
OpenShift High-Level Architecture
Container Networking
Problem Statements
• As an X,
I want my
containerized
applications
to be able to
connect to other
services, so that
they can perform
meaningful work.
• As an X,
I want my
containerized
applications
to be accessible
externally,
so that a
wide range of users
can use them.
RFC1918 IP
assigned by
docker daemon
Docker host
Docker bridge
Host NIC
Container
So we can use the
“ip” command
Container
Indicates which IF
the veth device is
connected to
Docker host
Endpoint of the
container’s veth
device
Source IP
appears to be
Node IP
Docker host
Inbound traffic is
forwarded to
container
Outbound traffic is
masqueraded
Container Networking
Problem Statement
• As an X, I want to use network attached
storage from within my container, so I
can provide stateful services (*).
• (*) and storage traffic shouldn’t share application network bandwith
Container Networking
Problem Statement
• As a X, I want name resolution to work
inside the container like they would on a
dedicated machine, so that I don’t have
to care about them.
/etc/hosts
/etc/hostname
/etc/resolv.conf
Container
Files in the
container fs are
overwritten
OpenShift Networking
Problem Statement
• As a X, I want container networking to
work seamlessly across multiple nodes,
so that I don’t have to worry where
which containers run (*).
• (*) while still maintaining compatibility with plain docker containers
OpenShift node
Host NICs
Docker bridge
docker bridge <-> ovs bridge
ovs bridge
ovs bridge <-> host NICs
OpenShift node
Look, there’s no
container
connected
OpenShift node
Container veth
endpoints on ovs
bridge
OpenShift node
Pod IP address
Container
Ping works from
container on same
host
Container
Ping fails from
container on
different host
Each node has
specific IP
range
OpenShift node
IP Range node 1
IP Range node 2
IP packets
destined for pod
on other node is
encapsulated via
VXLAN…
... and sent out
via the node‘s IP
stack
(MTU impact!)
OpenShift node
Port 1 is VXLAN
OpenShift node
Flow rules that
trigger VXLAN
encapsulation
Destination node
IP address
OpenShift node
Two pods in the
same namespace
on different
nodes…
OpenShift pod
…can
communicate with
each other
OpenShift Networking
Problem Statement
• As a X, I want to ensure that a rogue
pod cannot access pods in another
project, so that I have a base level of
security.
Project-specific
VXLAN ID
OpenShift pod
Pod in a different
namespace…
…cannot be
reached
OpenShift Networking
Problem Statement
• As a X, I want to be able to connect to
other containerized services using a
stable endpoint, so I don’t have to
reconfigure my application when other
containers come and go.
OpenShift node
Service IP
Address
OpenShift pod
Name resolution
via OpenShift
dnsmasq on node
Namespace in
search suffix list
OpenShift pod
Service name is
resolved into IP
adress
OpenShift pod
Communication
via service IP
Kubernetes Service Modes
• User-space mode– IPTables rules forward packages destined to the
service IP address to the kube-proxy
– Kube-proxy will in turn initiate connections to the actual destination IP and proxy between the two endpoints
– Key advantage: can detect non-responding pods and retry connection to other pods
• IPTables mode– kube-proxy continuously updates the node’s IPTables
rules
– forward packets directly to one of the target pod’s IP
– Key advantage: increased throughput
OpenShift Networking
Problem Statement
• As an X, I want my containerized
applications to be accessible externally,
so that a wide range of users can use
them (*).
• (*) without having to care on which node a container/pod runs
Ingress router
pod bound to
host port
Host Port = port
exposed on the
node
(containerized)
haproxy
OpenShift Routing
• Layer 7 Routing: HTTP(S), TLS-SNI
• To properly route other protocols,
deploy dedicated customized routers
• Alternatively instrument external load
balancers such as F5, etc.
OpenShift Networking
Problem Statement
• As an operator, I want to be able to fall
back to the known working version of a
service when deploying a new version
so I have a safety net (blue/green
deployments)
Router reconfiguration allows
blue/green deployments
oc patch route/api-gateway -p '{ "spec": {
"to": { "name": "api-gateway-green" }}}’
oc patch route/api-gateway -p '{ "spec": {
"to": { "name": "api-gateway-blue" }}}'
OpenShift Networking
Problem Statement
• As an operator, I want my containerized
applications to use specific source IP
addresses to access external services,
so I can restrict service access via
(external) firewall rules.
Egress source IP
Egress default
GWEgress target IP
(points to external
service “IPA”)
Points to egress-1
pod
OpenShift pod
Retrieving from
egress-1 service
works
External service
Egress
source IP
Egress
target IP
if2: node’s eth0
OpenShift egress pod
OpenShift Networking
Problem Statement
• As an operator, I want to control which
services my containerized applications
can access, so I can limit access via
internal means.
Egress Network Policy
{"kind": "EgressNetworkPolicy",
"apiVersion": "v1",
"metadata": { "name": "default” },
"spec": {
"egress": [{ "type": "Allow", "to": { "cidrSelector": "1.2.3.0/24” }
},{ "type": "Deny", "to": { "cidrSelector": "0.0.0.0/32” } }
]}
}
Summary
If the question is “OpenShift is great for
web applications, but we want to do X…
will this work?“, the answer is most likely
yes (from a networking point of view) (*).
(*) keep in mind that there is an MTU impact, multiple processing hops
which impact latency, etc…