From 97d9ad7886f5cb0e4974c0f434cf05e61fc6b794 Mon Sep 17 00:00:00 2001 From: Nico Schottelius Date: Sun, 13 Jun 2021 20:39:22 +0200 Subject: [PATCH] [k8s] blog update, +dns --- .../contents.lr | 166 ++++++++++++++++++ .../kubernetes-without-ingress/contents.lr | 14 +- 2 files changed, 178 insertions(+), 2 deletions(-) create mode 100644 content/u/blog/kubernetes-making-dns-publicly-reachable/contents.lr diff --git a/content/u/blog/kubernetes-making-dns-publicly-reachable/contents.lr b/content/u/blog/kubernetes-making-dns-publicly-reachable/contents.lr new file mode 100644 index 0000000..0de0b75 --- /dev/null +++ b/content/u/blog/kubernetes-making-dns-publicly-reachable/contents.lr @@ -0,0 +1,166 @@ +title: Making kubernetes kube-dns publicly reachable +--- +pub_date: 2021-06-13 +--- +author: ungleich +--- +twitter_handle: ungleich +--- +_hidden: no +--- +_discoverable: no +--- +abstract: +Looking into IPv6 only DNS provided by kubernetes +--- +body: + +## Introduction + +If you have seen our +[article about running kubernetes +Ingress-less](/u/blog/kubernetes-without-ingress/), you are aware that +we are pushing IPv6 only kubernetes clusters at ungleich. + +Today, we are looking at making the "internal" kube-dns service world +reachable using IPv6 and global DNS servers. + +## The kubernetes DNS service + +If you have a look at your typical k8s cluster, you will notice that +you usually have two coredns pods running: + +``` +% kubectl -n kube-system get pods -l k8s-app=kube-dns +NAME READY STATUS RESTARTS AGE +coredns-558bd4d5db-gz5c7 1/1 Running 0 6d +coredns-558bd4d5db-hrzhz 1/1 Running 0 6d +``` + +These pods are usually served by the **kube-dns** service: + +``` +% kubectl -n kube-system get svc -l k8s-app=kube-dns +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +kube-dns ClusterIP 2a0a:e5c0:13:e2::a 53/UDP,53/TCP,9153/TCP 6d1h +``` + +As you can see, the kube-dns service is running on a publicly +reachable IPv6 address. + +## IPv6 only DNS + +IPv6 only DNS servers have one drawback: they cannot be reached via DNS +recursions, if the resolver is IPv4 only. + +At [ungleich we run a variety of +services](https://redmine.ungleich.ch/projects/open-infrastructure/wiki) +to make IPv6 only services usable in the real world. In case of DNS, +we are using **DNS forwarders**. They are acting similar to HTTP +proxies, but for DNS. + +So in our main DNS servers, dns1.ungleich.ch, dns2.ungleich.ch +and dns3.ungleich.ch we have added the following configuration: + +``` +zone "k8s.place7.ungleich.ch" { + type forward; + forward only; + forwarders { 2a0a:e5c0:13:e2::a; }; +}; +``` + +This tells the DNS servers to forward DNS queries that come in for +k8s.place7.ungleich.ch to **2a0a:e5c0:13:e2::a**. + +Additionally we have added **DNS delegation** in the +place7.ungleich.ch zone: + +``` +k8s NS dns1.ungleich.ch. +k8s NS dns2.ungleich.ch. +k8s NS dns3.ungleich.ch. +``` + +## Using the kubernetes DNS service in the wild + +With this configuration, we can now access IPv6 only +kubernetes services directly from the Internet. Let's first discover +the kube-dns service itself: + +``` +% dig kube-dns.kube-system.svc.k8s.place7.ungleich.ch. aaaa + +; <<>> DiG 9.16.16 <<>> kube-dns.kube-system.svc.k8s.place7.ungleich.ch. aaaa +;; global options: +cmd +;; Got answer: +;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23274 +;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 + +;; OPT PSEUDOSECTION: +; EDNS: version: 0, flags:; udp: 4096 +; COOKIE: f61925944f5218c9ac21e43960c64f254792e60f2b10f3f5 (good) +;; QUESTION SECTION: +;kube-dns.kube-system.svc.k8s.place7.ungleich.ch. IN AAAA + +;; ANSWER SECTION: +kube-dns.kube-system.svc.k8s.place7.ungleich.ch. 27 IN AAAA 2a0a:e5c0:13:e2::a + +;; AUTHORITY SECTION: +k8s.place7.ungleich.ch. 13 IN NS kube-dns.kube-system.svc.k8s.place7.ungleich.ch. + +;; Query time: 18 msec +;; SERVER: 192.168.4.188#53(192.168.4.188) +;; WHEN: Sun Jun 13 20:32:41 CEST 2021 +;; MSG SIZE rcvd: 146 +``` + +As you can see, the **kube-dns** service in the **kube-system** +namespace resolves to 2a0a:e5c0:13:e2::a, which is exactly what we +have configured. + +At the moment, there is also an etherpad test service +named "ungleich-etherpad" running: + +``` +% kubectl get svc -l app=ungleichetherpad +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +ungleich-etherpad ClusterIP 2a0a:e5c0:13:e2::b7db 9001/TCP 3d19h +``` + +Let's first verify that it resolves: + +``` +% dig +short ungleich-etherpad.default.svc.k8s.place7.ungleich.ch aaaa +2a0a:e5c0:13:e2::b7db +``` + +And if that works, well, then we should also be able to access the +service itself! + +``` +% curl -I http://ungleich-etherpad.default.svc.k8s.place7.ungleich.ch:9001/ +HTTP/1.1 200 OK +X-Powered-By: Express +X-UA-Compatible: IE=Edge,chrome=1 +Referrer-Policy: same-origin +Content-Type: text/html; charset=utf-8 +Content-Length: 6039 +ETag: W/"1797-Dq3+mr7XP0PQshikMNRpm5RSkGA" +Set-Cookie: express_sid=s%3AZGKdDe3FN1v5UPcS-7rsZW7CeloPrQ7p.VaL1V0M4780TBm8bT9hPVQMWPX5Lcte%2BzotO9Lsejlk; Path=/; HttpOnly; SameSite=Lax +Date: Sun, 13 Jun 2021 18:36:23 GMT +Connection: keep-alive +Keep-Alive: timeout=5 +``` + +(attention, this is a test service and might not be running when you +read this article at a later time) + +## More of this + +We are discussing +kubernetes and IPv6 related topics in +**the #hacking:ungleich.ch Matrix channel** +([you can signup here if you don't have an +account](https://chat.with.ungleich.ch)) and will post more about our +k8s journey in this blog. Stay tuned! diff --git a/content/u/blog/kubernetes-without-ingress/contents.lr b/content/u/blog/kubernetes-without-ingress/contents.lr index fad6f6f..3f8276e 100644 --- a/content/u/blog/kubernetes-without-ingress/contents.lr +++ b/content/u/blog/kubernetes-without-ingress/contents.lr @@ -39,6 +39,16 @@ So the ingress basically routes from outside to inside. But, in the IPv6 world, services are already publicly reachable. It just depends on your network policy. +### Update 2021-06-13: Ingress vs. Service + +As some people pointed out (thanks a lot!), a public service is +**not the same** as an Ingress. Ingress has also the possibility to +route based on layer 7 information like the path, domain name, etc. + +However, if all of the traffic from an Ingress points to a single +IPv6 HTTP/HTTPS Service, effectively the IPv6 service will do the +same, with one hop less. + ## Services Let's have a look at how services in IPv6 only clusters look like: @@ -51,8 +61,8 @@ nginx-service ClusterIP 2a0a:e5c0:13:e2::3607 5432/TCP 19h ... ``` -All these services are world reachable, so a theorethical ingress does -actually not do any work. Services are routed to the pods as usual. +All these services are world reachable, depending on your network +policy. ## ServiceTypes