[k8s] blog update, +dns

This commit is contained in:
Nico Schottelius 2021-06-13 20:39:22 +02:00
parent 74df18d145
commit 97d9ad7886
2 changed files with 178 additions and 2 deletions

View file

@ -0,0 +1,166 @@
title: Making kubernetes kube-dns publicly reachable
---
pub_date: 2021-06-13
---
author: ungleich
---
twitter_handle: ungleich
---
_hidden: no
---
_discoverable: no
---
abstract:
Looking into IPv6 only DNS provided by kubernetes
---
body:
## Introduction
If you have seen our
[article about running kubernetes
Ingress-less](/u/blog/kubernetes-without-ingress/), you are aware that
we are pushing IPv6 only kubernetes clusters at ungleich.
Today, we are looking at making the "internal" kube-dns service world
reachable using IPv6 and global DNS servers.
## The kubernetes DNS service
If you have a look at your typical k8s cluster, you will notice that
you usually have two coredns pods running:
```
% kubectl -n kube-system get pods -l k8s-app=kube-dns
NAME READY STATUS RESTARTS AGE
coredns-558bd4d5db-gz5c7 1/1 Running 0 6d
coredns-558bd4d5db-hrzhz 1/1 Running 0 6d
```
These pods are usually served by the **kube-dns** service:
```
% kubectl -n kube-system get svc -l k8s-app=kube-dns
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 2a0a:e5c0:13:e2::a <none> 53/UDP,53/TCP,9153/TCP 6d1h
```
As you can see, the kube-dns service is running on a publicly
reachable IPv6 address.
## IPv6 only DNS
IPv6 only DNS servers have one drawback: they cannot be reached via DNS
recursions, if the resolver is IPv4 only.
At [ungleich we run a variety of
services](https://redmine.ungleich.ch/projects/open-infrastructure/wiki)
to make IPv6 only services usable in the real world. In case of DNS,
we are using **DNS forwarders**. They are acting similar to HTTP
proxies, but for DNS.
So in our main DNS servers, dns1.ungleich.ch, dns2.ungleich.ch
and dns3.ungleich.ch we have added the following configuration:
```
zone "k8s.place7.ungleich.ch" {
type forward;
forward only;
forwarders { 2a0a:e5c0:13:e2::a; };
};
```
This tells the DNS servers to forward DNS queries that come in for
k8s.place7.ungleich.ch to **2a0a:e5c0:13:e2::a**.
Additionally we have added **DNS delegation** in the
place7.ungleich.ch zone:
```
k8s NS dns1.ungleich.ch.
k8s NS dns2.ungleich.ch.
k8s NS dns3.ungleich.ch.
```
## Using the kubernetes DNS service in the wild
With this configuration, we can now access IPv6 only
kubernetes services directly from the Internet. Let's first discover
the kube-dns service itself:
```
% dig kube-dns.kube-system.svc.k8s.place7.ungleich.ch. aaaa
; <<>> DiG 9.16.16 <<>> kube-dns.kube-system.svc.k8s.place7.ungleich.ch. aaaa
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23274
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: f61925944f5218c9ac21e43960c64f254792e60f2b10f3f5 (good)
;; QUESTION SECTION:
;kube-dns.kube-system.svc.k8s.place7.ungleich.ch. IN AAAA
;; ANSWER SECTION:
kube-dns.kube-system.svc.k8s.place7.ungleich.ch. 27 IN AAAA 2a0a:e5c0:13:e2::a
;; AUTHORITY SECTION:
k8s.place7.ungleich.ch. 13 IN NS kube-dns.kube-system.svc.k8s.place7.ungleich.ch.
;; Query time: 18 msec
;; SERVER: 192.168.4.188#53(192.168.4.188)
;; WHEN: Sun Jun 13 20:32:41 CEST 2021
;; MSG SIZE rcvd: 146
```
As you can see, the **kube-dns** service in the **kube-system**
namespace resolves to 2a0a:e5c0:13:e2::a, which is exactly what we
have configured.
At the moment, there is also an etherpad test service
named "ungleich-etherpad" running:
```
% kubectl get svc -l app=ungleichetherpad
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ungleich-etherpad ClusterIP 2a0a:e5c0:13:e2::b7db <none> 9001/TCP 3d19h
```
Let's first verify that it resolves:
```
% dig +short ungleich-etherpad.default.svc.k8s.place7.ungleich.ch aaaa
2a0a:e5c0:13:e2::b7db
```
And if that works, well, then we should also be able to access the
service itself!
```
% curl -I http://ungleich-etherpad.default.svc.k8s.place7.ungleich.ch:9001/
HTTP/1.1 200 OK
X-Powered-By: Express
X-UA-Compatible: IE=Edge,chrome=1
Referrer-Policy: same-origin
Content-Type: text/html; charset=utf-8
Content-Length: 6039
ETag: W/"1797-Dq3+mr7XP0PQshikMNRpm5RSkGA"
Set-Cookie: express_sid=s%3AZGKdDe3FN1v5UPcS-7rsZW7CeloPrQ7p.VaL1V0M4780TBm8bT9hPVQMWPX5Lcte%2BzotO9Lsejlk; Path=/; HttpOnly; SameSite=Lax
Date: Sun, 13 Jun 2021 18:36:23 GMT
Connection: keep-alive
Keep-Alive: timeout=5
```
(attention, this is a test service and might not be running when you
read this article at a later time)
## More of this
We are discussing
kubernetes and IPv6 related topics in
**the #hacking:ungleich.ch Matrix channel**
([you can signup here if you don't have an
account](https://chat.with.ungleich.ch)) and will post more about our
k8s journey in this blog. Stay tuned!

View file

@ -39,6 +39,16 @@ So the ingress basically routes from outside to inside. But, in the
IPv6 world, services are already publicly reachable. It just IPv6 world, services are already publicly reachable. It just
depends on your network policy. depends on your network policy.
### Update 2021-06-13: Ingress vs. Service
As some people pointed out (thanks a lot!), a public service is
**not the same** as an Ingress. Ingress has also the possibility to
route based on layer 7 information like the path, domain name, etc.
However, if all of the traffic from an Ingress points to a single
IPv6 HTTP/HTTPS Service, effectively the IPv6 service will do the
same, with one hop less.
## Services ## Services
Let's have a look at how services in IPv6 only clusters look like: Let's have a look at how services in IPv6 only clusters look like:
@ -51,8 +61,8 @@ nginx-service ClusterIP 2a0a:e5c0:13:e2::3607 <n
postgres ClusterIP 2a0a:e5c0:13:e2::c9e0 <none> 5432/TCP 19h postgres ClusterIP 2a0a:e5c0:13:e2::c9e0 <none> 5432/TCP 19h
... ...
``` ```
All these services are world reachable, so a theorethical ingress does All these services are world reachable, depending on your network
actually not do any work. Services are routed to the pods as usual. policy.
## ServiceTypes ## ServiceTypes