++k8s/ingress blog post
This commit is contained in:
Nico Schottelius
parent
3f6cc3bfc7
commit
227db05303
2 changed files with 172 additions and 33 deletions
|
|
@ -1,33 +0,0 @@
|
||||||
title: Kubernetes does not need a LoadBalancer nor ingress
|
|
||||||
---
|
|
||||||
pub_date: 2021-06-09
|
|
||||||
---
|
|
||||||
author: ungleich
|
|
||||||
---
|
|
||||||
twitter_handle: ungleich
|
|
||||||
---
|
|
||||||
_hidden: no
|
|
||||||
---
|
|
||||||
_discoverable: no
|
|
||||||
---
|
|
||||||
abstract:
|
|
||||||
|
|
||||||
---
|
|
||||||
body:
|
|
||||||
|
|
||||||
## TL;DR
|
|
||||||
|
|
||||||
You might be running services just fine without a loadbalancer
|
|
||||||
in Kubernetes. Using public ClusterIP is very easy, especially with
|
|
||||||
IPv6 only clusters.
|
|
||||||
|
|
||||||
## Background
|
|
||||||
|
|
||||||
Typically in loadbalancer
|
|
||||||
ingress for public
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## DNS
|
|
||||||
|
|
||||||
## Letsencrypt
|
|
||||||
172
content/u/blog/kubernetes-without-ingress/contents.lr
Normal file
172
content/u/blog/kubernetes-without-ingress/contents.lr
Normal file
|
|
@ -0,0 +1,172 @@
|
||||||
|
title: Kubernetes without Ingress
|
||||||
|
---
|
||||||
|
pub_date: 2021-06-09
|
||||||
|
---
|
||||||
|
author: ungleich
|
||||||
|
---
|
||||||
|
twitter_handle: ungleich
|
||||||
|
---
|
||||||
|
_hidden: no
|
||||||
|
---
|
||||||
|
_discoverable: no
|
||||||
|
---
|
||||||
|
abstract:
|
||||||
|
|
||||||
|
---
|
||||||
|
body:
|
||||||
|
|
||||||
|
## Introduction
|
||||||
|
|
||||||
|
On our journey to build and define IPv6 only kubernetes clusters we
|
||||||
|
came accross some principles that seem awkward in the IPv6 world. Let
|
||||||
|
us today have a look at the *LoadBalancer* and *Ingress* concepts.
|
||||||
|
|
||||||
|
## Ingress
|
||||||
|
|
||||||
|
Let's have a look at the [Ingres
|
||||||
|
definition](https://kubernetes.io/docs/concepts/services-networking/ingress/)
|
||||||
|
definiton from the kubernetes website:
|
||||||
|
|
||||||
|
```
|
||||||
|
Ingress exposes HTTP and HTTPS routes from outside the cluster to
|
||||||
|
services within the cluster. Traffic routing is controlled by rules
|
||||||
|
defined on the Ingress resource.
|
||||||
|
```
|
||||||
|
|
||||||
|
So the ingress basically routes from outside to inside. But, in the
|
||||||
|
IPv6 world, services can already be publicly reachable, depending on
|
||||||
|
your network policy.
|
||||||
|
|
||||||
|
## Services
|
||||||
|
|
||||||
|
Let's have a look at how services in IPv6 only clusters look like:
|
||||||
|
|
||||||
|
```
|
||||||
|
% kubectl get svc
|
||||||
|
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||||
|
etherpad ClusterIP 2a0a:e5c0:13:e2::a94b <none> 9001/TCP 19h
|
||||||
|
nginx-service ClusterIP 2a0a:e5c0:13:e2::3607 <none> 80/TCP 43h
|
||||||
|
postgres ClusterIP 2a0a:e5c0:13:e2::c9e0 <none> 5432/TCP 19h
|
||||||
|
...
|
||||||
|
```
|
||||||
|
All these services are world reachable, so a theorethical ingress does
|
||||||
|
actually not do any work. Services are routed to the pods as usual.
|
||||||
|
|
||||||
|
## ServiceTypes
|
||||||
|
|
||||||
|
While we are at looking at the k8s primitives, let's have a closer
|
||||||
|
look at the **Service**, specifically at 3 of the **ServiceTypes**
|
||||||
|
supported by k8s, including it's definition:
|
||||||
|
|
||||||
|
### ClusterIP
|
||||||
|
|
||||||
|
The k8s website says
|
||||||
|
|
||||||
|
```
|
||||||
|
Exposes the Service on a cluster-internal IP. Choosing this value
|
||||||
|
makes the Service only reachable from within the cluster. This is the
|
||||||
|
default ServiceType.
|
||||||
|
```
|
||||||
|
|
||||||
|
So in the context of IPv6, this sounds wrong. There is nothing that
|
||||||
|
makes an global IPv6 address be "internal", besides possible network
|
||||||
|
policies. The concept is probably coming from the strict difference of
|
||||||
|
RFC1918 space usually used in k8s clusters and not public IPv4.
|
||||||
|
|
||||||
|
This difference does not make a lot of sense in the IPv6 world though
|
||||||
|
and seeing services as public by default, makes more sense.
|
||||||
|
|
||||||
|
### NodePort
|
||||||
|
|
||||||
|
Let's have a look at the definition first again:
|
||||||
|
|
||||||
|
```
|
||||||
|
Exposes the Service on each Node's IP at a static port (the
|
||||||
|
NodePort). A ClusterIP Service, to which the NodePort Service routes,
|
||||||
|
is automatically created. You'll be able to contact the NodePort
|
||||||
|
Service, from outside the cluster, by requesting <NodeIP>:<NodePort>.
|
||||||
|
```
|
||||||
|
|
||||||
|
Conceptually this can be similarily utilised in the IPv6 only world
|
||||||
|
like it does in the IPv4 world. However given that there are enough
|
||||||
|
addresses available with IPv6, this might not be such an interesting
|
||||||
|
ServiceType anymore.
|
||||||
|
|
||||||
|
|
||||||
|
### LoadBalancer
|
||||||
|
|
||||||
|
Before we have a look at this type, let's take some steps back
|
||||||
|
first to ...
|
||||||
|
|
||||||
|
|
||||||
|
## ... Load Balancing
|
||||||
|
|
||||||
|
There are a variety of possibilities to do load balancing. From simple
|
||||||
|
round robin, to ECMP based load balancing, to application aware,
|
||||||
|
potentially weighted load balancing.
|
||||||
|
|
||||||
|
So for load balancing, there is usually more than one solution and
|
||||||
|
there is likely not one size fits all.
|
||||||
|
|
||||||
|
So with this said, let.s have a look at the
|
||||||
|
**ServiceType LoadBalancer** definition:
|
||||||
|
|
||||||
|
```
|
||||||
|
Exposes the Service externally using a cloud provider's load
|
||||||
|
balancer. NodePort and ClusterIP Services, to which the external load
|
||||||
|
balancer routes, are automatically created.
|
||||||
|
```
|
||||||
|
|
||||||
|
So whatever the cloud provider offers, can be used, and that is a good
|
||||||
|
thing. However, let's have a look at how you get load balancing for
|
||||||
|
free in IPv6 only clusters:
|
||||||
|
|
||||||
|
## Load Balancing in IPv6 only clusters
|
||||||
|
|
||||||
|
So what is the most easy way of reliable load balancing in network?
|
||||||
|
ECMP (equal cost multi path) comes to the mind right away. Given that
|
||||||
|
kubernetes nodes can BGP peer with the network (upstream or the
|
||||||
|
switches), this basically gives load balancing to the world for free:
|
||||||
|
|
||||||
|
```
|
||||||
|
[ The Internet ]
|
||||||
|
|
|
||||||
|
[ k8s-node-1 ]-----------[ network ]-----------[ k8s-node-n]
|
||||||
|
[ ECMP ]
|
||||||
|
|
|
||||||
|
[ k8s-node-2]
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
In the real world on a bird based BGP upstream router
|
||||||
|
this looks as follows:
|
||||||
|
|
||||||
|
```
|
||||||
|
[18:13:02] red.place7:~# birdc show route
|
||||||
|
BIRD 2.0.7 ready.
|
||||||
|
Table master6:
|
||||||
|
...
|
||||||
|
2a0a:e5c0:13:e2::/108 unicast [place7-server1 2021-06-07] * (100) [AS65534i]
|
||||||
|
via 2a0a:e5c0:13:0:225:b3ff:fe20:3554 on eth0
|
||||||
|
unicast [place7-server4 2021-06-08] (100) [AS65534i]
|
||||||
|
via 2a0a:e5c0:13:0:225:b3ff:fe20:3564 on eth0
|
||||||
|
unicast [place7-server2 2021-06-07] (100) [AS65534i]
|
||||||
|
via 2a0a:e5c0:13:0:225:b3ff:fe20:38cc on eth0
|
||||||
|
unicast [place7-server3 2021-06-07] (100) [AS65534i]
|
||||||
|
via 2a0a:e5c0:13:0:224:81ff:fee0:db7a on eth0
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
|
||||||
|
We know, a TL;DR at the end is not the right thing to do, but hey, we
|
||||||
|
are at ungleich, aren't we?
|
||||||
|
|
||||||
|
In a nutshell, with IPv6 the concept of **Ingress**,
|
||||||
|
**Service** and the **LoadBalancer** ServiceType
|
||||||
|
types need to be revised, as IPv6 allows direct access without having
|
||||||
|
to jump through hoops.
|
||||||
|
|
||||||
|
If you are interesting in continuing the discussion,
|
||||||
|
we are there for you in
|
||||||
|
[the #hacking:ungleich.ch Matrix channel](https://chat.with.ungleich.ch).
|
||||||
Loading…
Reference in a new issue