227 lines
6.7 KiB
Markdown
227 lines
6.7 KiB
Markdown
title: Making kubernetes kube-dns publicly reachable
|
|
---
|
|
pub_date: 2021-06-13
|
|
---
|
|
author: ungleich
|
|
---
|
|
twitter_handle: ungleich
|
|
---
|
|
_hidden: no
|
|
---
|
|
_discoverable: yes
|
|
---
|
|
abstract:
|
|
Looking into IPv6 only DNS provided by kubernetes
|
|
---
|
|
body:
|
|
|
|
## Introduction
|
|
|
|
If you have seen our
|
|
[article about running kubernetes
|
|
Ingress-less](/u/blog/kubernetes-without-ingress/), you are aware that
|
|
we are pushing IPv6 only kubernetes clusters at ungleich.
|
|
|
|
Today, we are looking at making the "internal" kube-dns service world
|
|
reachable using IPv6 and global DNS servers.
|
|
|
|
## The kubernetes DNS service
|
|
|
|
If you have a look at your typical k8s cluster, you will notice that
|
|
you usually have two coredns pods running:
|
|
|
|
```
|
|
% kubectl -n kube-system get pods -l k8s-app=kube-dns
|
|
NAME READY STATUS RESTARTS AGE
|
|
coredns-558bd4d5db-gz5c7 1/1 Running 0 6d
|
|
coredns-558bd4d5db-hrzhz 1/1 Running 0 6d
|
|
```
|
|
|
|
These pods are usually served by the **kube-dns** service:
|
|
|
|
```
|
|
% kubectl -n kube-system get svc -l k8s-app=kube-dns
|
|
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
|
kube-dns ClusterIP 2a0a:e5c0:13:e2::a <none> 53/UDP,53/TCP,9153/TCP 6d1h
|
|
```
|
|
|
|
As you can see, the kube-dns service is running on a publicly
|
|
reachable IPv6 address.
|
|
|
|
## IPv6 only DNS
|
|
|
|
IPv6 only DNS servers have one drawback: they cannot be reached via DNS
|
|
recursions, if the resolver is IPv4 only.
|
|
|
|
At [ungleich we run a variety of
|
|
services](https://redmine.ungleich.ch/projects/open-infrastructure/wiki)
|
|
to make IPv6 only services usable in the real world. In case of DNS,
|
|
we are using **DNS forwarders**. They are acting similar to HTTP
|
|
proxies, but for DNS.
|
|
|
|
So in our main DNS servers, dns1.ungleich.ch, dns2.ungleich.ch
|
|
and dns3.ungleich.ch we have added the following configuration:
|
|
|
|
```
|
|
zone "k8s.place7.ungleich.ch" {
|
|
type forward;
|
|
forward only;
|
|
forwarders { 2a0a:e5c0:13:e2::a; };
|
|
};
|
|
```
|
|
|
|
This tells the DNS servers to forward DNS queries that come in for
|
|
k8s.place7.ungleich.ch to **2a0a:e5c0:13:e2::a**.
|
|
|
|
Additionally we have added **DNS delegation** in the
|
|
place7.ungleich.ch zone:
|
|
|
|
```
|
|
k8s NS dns1.ungleich.ch.
|
|
k8s NS dns2.ungleich.ch.
|
|
k8s NS dns3.ungleich.ch.
|
|
```
|
|
|
|
## Using the kubernetes DNS service in the wild
|
|
|
|
With this configuration, we can now access IPv6 only
|
|
kubernetes services directly from the Internet. Let's first discover
|
|
the kube-dns service itself:
|
|
|
|
```
|
|
% dig kube-dns.kube-system.svc.k8s.place7.ungleich.ch. aaaa
|
|
|
|
; <<>> DiG 9.16.16 <<>> kube-dns.kube-system.svc.k8s.place7.ungleich.ch. aaaa
|
|
;; global options: +cmd
|
|
;; Got answer:
|
|
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23274
|
|
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
|
|
|
|
;; OPT PSEUDOSECTION:
|
|
; EDNS: version: 0, flags:; udp: 4096
|
|
; COOKIE: f61925944f5218c9ac21e43960c64f254792e60f2b10f3f5 (good)
|
|
;; QUESTION SECTION:
|
|
;kube-dns.kube-system.svc.k8s.place7.ungleich.ch. IN AAAA
|
|
|
|
;; ANSWER SECTION:
|
|
kube-dns.kube-system.svc.k8s.place7.ungleich.ch. 27 IN AAAA 2a0a:e5c0:13:e2::a
|
|
|
|
;; AUTHORITY SECTION:
|
|
k8s.place7.ungleich.ch. 13 IN NS kube-dns.kube-system.svc.k8s.place7.ungleich.ch.
|
|
```
|
|
|
|
As you can see, the **kube-dns** service in the **kube-system**
|
|
namespace resolves to 2a0a:e5c0:13:e2::a, which is exactly what we
|
|
have configured.
|
|
|
|
At the moment, there is also an etherpad test service
|
|
named "ungleich-etherpad" running:
|
|
|
|
```
|
|
% kubectl get svc -l app=ungleichetherpad
|
|
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
|
ungleich-etherpad ClusterIP 2a0a:e5c0:13:e2::b7db <none> 9001/TCP 3d19h
|
|
```
|
|
|
|
Let's first verify that it resolves:
|
|
|
|
```
|
|
% dig +short ungleich-etherpad.default.svc.k8s.place7.ungleich.ch aaaa
|
|
2a0a:e5c0:13:e2::b7db
|
|
```
|
|
|
|
And if that works, well, then we should also be able to access the
|
|
service itself!
|
|
|
|
```
|
|
% curl -I http://ungleich-etherpad.default.svc.k8s.place7.ungleich.ch:9001/
|
|
HTTP/1.1 200 OK
|
|
X-Powered-By: Express
|
|
X-UA-Compatible: IE=Edge,chrome=1
|
|
Referrer-Policy: same-origin
|
|
Content-Type: text/html; charset=utf-8
|
|
Content-Length: 6039
|
|
ETag: W/"1797-Dq3+mr7XP0PQshikMNRpm5RSkGA"
|
|
Set-Cookie: express_sid=s%3AZGKdDe3FN1v5UPcS-7rsZW7CeloPrQ7p.VaL1V0M4780TBm8bT9hPVQMWPX5Lcte%2BzotO9Lsejlk; Path=/; HttpOnly; SameSite=Lax
|
|
Date: Sun, 13 Jun 2021 18:36:23 GMT
|
|
Connection: keep-alive
|
|
Keep-Alive: timeout=5
|
|
```
|
|
|
|
(attention, this is a test service and might not be running when you
|
|
read this article at a later time)
|
|
|
|
## IPv6 vs. IPv4
|
|
|
|
Could we have achived the same with IPv4? The answere here is "maybe":
|
|
If the kubernetes service is reachable from globally reachable
|
|
nameservers via IPv4, then the answer is yes. This could be done via
|
|
public IPv4 addresses in the kubernetes cluster, via tunnels, VPNs,
|
|
etc.
|
|
|
|
However, generally speaking, the DNS service of a
|
|
kubernetes cluster running on RFC1918 IP addresses, is probably not
|
|
reachable from globally reachable DNS servers by default.
|
|
|
|
For IPv6 the case is a bit different: we are using globally reachable
|
|
IPv6 addresses in our k8s clusters, so they can potentially be
|
|
reachable without the need of any tunnel or whatsoever. Firewalling
|
|
and network policies can obviously prevent access, but if the IP
|
|
addresses are properly routed, they will be accessible from the public
|
|
Internet.
|
|
|
|
And this makes things much easier for DNS servers, which are also
|
|
having IPv6 connectivity.
|
|
|
|
The following pictures shows the practical difference between the two
|
|
approaches:
|
|
|
|
|
|
|
|
## Does this make sense?
|
|
|
|
That clearly depends on your use-case. If you want your service DNS
|
|
records to be publicly accessible, then the clear answer is yes.
|
|
|
|
If your cluster services are intended to be internal only
|
|
(see [previous blog post](/u/blog/kubernetes-without-ingress/), then
|
|
exposing the DNS service to the world might not be the best option.
|
|
|
|
## Note on security
|
|
|
|
CoreDNS inside kubernetes is by default configured to allow resolving
|
|
for *any* client that can reach it. Thus if you make your kube-dns
|
|
service world reachable, you also turn it into an open resolver.
|
|
|
|
The following coredns configuration **does** correctly block
|
|
requests, **IF your coredns version is new enough**:
|
|
|
|
|
|
```
|
|
Corefile: |
|
|
.:53 {
|
|
acl k8s.place7.ungleich.ch {
|
|
allow net ::/0
|
|
}
|
|
acl . {
|
|
allow net 2a0a:e5c0:13::/48
|
|
block
|
|
}
|
|
forward . /etc/resolv.conf {
|
|
max_concurrent 1000
|
|
}
|
|
...
|
|
```
|
|
|
|
We tested this with
|
|
[coredns-1.8.4](https://github.com/coredns/coredns/issues/4697) in
|
|
which the ACL behaviour is fixed.
|
|
|
|
## More of this
|
|
|
|
We are discussing
|
|
kubernetes and IPv6 related topics in
|
|
**the #hacking:ungleich.ch Matrix channel**
|
|
([you can signup here if you don't have an
|
|
account](https://chat.with.ungleich.ch)) and will post more about our
|
|
k8s journey in this blog. Stay tuned!
|