Merge branch 'master' of code.ungleich.ch:ungleich-public/ungleich-staticcms
1
.gitignore
vendored
|
@ -1,3 +1,4 @@
|
|||
venv/
|
||||
.DS_Store
|
||||
.idea/
|
||||
.history
|
||||
|
|
BIN
assets/u/image/cards/infrastructure-availability.jpg
Normal file
After Width: | Height: | Size: 261 KiB |
BIN
assets/u/image/cards/service-hours.jpg
Normal file
After Width: | Height: | Size: 264 KiB |
BIN
assets/u/image/cards/sla-levels.jpg
Normal file
After Width: | Height: | Size: 201 KiB |
BIN
assets/u/image/cards/support-packages.jpg
Normal file
After Width: | Height: | Size: 275 KiB |
BIN
assets/u/image/k8s-v6-v4-dns.png
Normal file
After Width: | Height: | Size: 88 KiB |
BIN
assets/u/image/product.jpg
Normal file
After Width: | Height: | Size: 3.8 MiB |
|
@ -9,3 +9,25 @@
|
|||
.headlinebold {
|
||||
font-family: "Nimbus Sans L";
|
||||
}
|
||||
|
||||
.colored-table {
|
||||
text-align: center;
|
||||
background: #E5EDF1;
|
||||
}
|
||||
|
||||
.colored-table td, .colored-table th {
|
||||
border: 1px solid #fff;
|
||||
}
|
||||
|
||||
.colored-table thead th {
|
||||
border-bottom: 2px solid #fff;
|
||||
}
|
||||
|
||||
.bg-offer {
|
||||
background-color: #c0dcf3;
|
||||
}
|
||||
.table-responsive {
|
||||
font-family: "Rubik", sans-serif;
|
||||
font-size: 13px;
|
||||
line-height: 22px;
|
||||
}
|
|
@ -25,6 +25,20 @@ header h1 {
|
|||
font-size: 42px;
|
||||
}
|
||||
|
||||
// Sanghee, 2021-09-20
|
||||
h2 {
|
||||
font-size: 1.65rem;
|
||||
line-height: 2;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
// Sanghee, 2021-09-20
|
||||
h3 {
|
||||
font-size: 1rem;
|
||||
text-transform: uppercase;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
header nav ul {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
|
|
|
@ -0,0 +1,132 @@
|
|||
title: Bye, bye netboot
|
||||
---
|
||||
pub_date: 2021-08-31
|
||||
---
|
||||
author: ungleich infrastructure team
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: yes
|
||||
---
|
||||
abstract:
|
||||
Data Center Light servers are switching to disk based boot
|
||||
---
|
||||
body:
|
||||
|
||||
## Introduction
|
||||
|
||||
Since the very beginning of the [Data Center Light
|
||||
project](/u/projects/data-center-light) our servers have been
|
||||
*somewhat stateless* and booted from their operating system from the
|
||||
network.
|
||||
|
||||
From today on this changes and our servers are switched to boot from
|
||||
an disk (SSD/NVMe/HDD). While this first seems counter intuitive with
|
||||
growing a data center, let us explain why this makes sense for us.
|
||||
|
||||
## Netboot in a nutshell
|
||||
|
||||
There are different variants of how to netboot a server. In either
|
||||
case, the server loads an executable from the network, typically via
|
||||
TFTP or HTTP and then hands over execution to it.
|
||||
|
||||
The first option is to load the kernel and then later switch to an NFS
|
||||
based filesystem. If the filesystem is read write, you usually need
|
||||
one location per server or you mount it read only and possibly apply
|
||||
an overlay for runtime configuration.
|
||||
|
||||
The second option is to load the kernel and an initramfs into memory
|
||||
and stay inside the initramfs. The advantage of this approach is that
|
||||
no NFS server is needed, but the whole operating system is inside the
|
||||
memory.
|
||||
|
||||
The second option is what we used in Data Center Light for the last
|
||||
couple of years.
|
||||
|
||||
## Netboot history at Data Center Light
|
||||
|
||||
Originally all our servers started with IPv4 PXE based
|
||||
netboot. However as our data center is generally speaking IPv6 only,
|
||||
the IPv4 DHCP+TFTP combination is an extra maintenance and also a
|
||||
hindrance for network debugging: if you are in a single stack IPv6
|
||||
only network, things are much easier to debug. No need to look for two
|
||||
routing tables, no need to work around DHCP settings that might
|
||||
interfere with what one wants to achieve via IPv6.
|
||||
|
||||
As the IPv4 addresses became more of a technical debt in our
|
||||
infrastructure, we started flashing our network cards with
|
||||
[ipxe](https://ipxe.org/), which allows even older network cards to
|
||||
boot in IPv6 only networks.
|
||||
|
||||
Also in an IPv6 only netboot environment, it is easier to run
|
||||
active-active routers, as hosts are not assigned DHCP leases. They
|
||||
assign addresses themselves, which scales much nicer.
|
||||
|
||||
## Migrating away from netbooting
|
||||
|
||||
So why are we migrating away from netbooting, even after we migrated
|
||||
to IPv6 only networking? There are multiple aspects:
|
||||
|
||||
On power failure, netbooted hosts lose their state. The operating
|
||||
system that is loaded is the same for every server and needs some
|
||||
configuration post-boot. We have solved this using
|
||||
[cdist](https://www.cdi.st/), however the authentication-trigger
|
||||
mechanism is non-trivial, if you want to keep your netboot images and
|
||||
build steps public.
|
||||
|
||||
The second reason is state synchronisation: as we are having multiple
|
||||
boot servers, we need to maintain the same state on multiple
|
||||
machines. That is solvable via CI/CD pipelines, however the level of
|
||||
automation on build servers is rather low, because the amount of OS
|
||||
changes are low.
|
||||
|
||||
The third and main point is our ongoing migration towards
|
||||
[kubernetes](https://kubernetes.io/). Originally our servers would
|
||||
boot up, get configured for providing ceph storage or to be a
|
||||
virtualisation host. The amount of binaries to keep in our in-memory
|
||||
image was tiny, in the best case around 150MB. With the migration
|
||||
towards kubernetes, every node is downloading the containers, which
|
||||
can be comparable huge (gigabytes of data). The additional pivot_root
|
||||
workarounds that are required for initramfs usage are just an
|
||||
additional minor point that made us question our current setup.
|
||||
|
||||
## Automating disk based boot
|
||||
|
||||
We have servers from a variety of brands and each of them comes with a
|
||||
variety of disk controllers: from simple pass-through SATA controllers
|
||||
to full fledged hardware raid with onboard cache and battery for
|
||||
protecting the cache - everything is in the mix.
|
||||
|
||||
So it is not easily possible to generate a stack of disks somewhere
|
||||
and then add them, as the disk controller might add some (RAID0) meta
|
||||
data to it.
|
||||
|
||||
To work around this problem, we insert the disk that is becoming the
|
||||
boot disk in the future into the netbooted servers, install the
|
||||
operating system from the running environment and at the next
|
||||
maintenance window ensure that the server is actually booting from it.
|
||||
|
||||
If you are curious on how this works, you can checkout the script that
|
||||
we use for
|
||||
[Devuan/Debian](https://code.ungleich.ch/ungleich-public/ungleich-tools/-/blob/master/debian-devuan-install-on-disk.sh)
|
||||
and
|
||||
[Alpine Linux](https://code.ungleich.ch/ungleich-public/ungleich-tools/-/blob/master/alpine-install-on-disk.sh)
|
||||
|
||||
## The road continues
|
||||
|
||||
While a data center needs to be stable, it also needs to adapt to
|
||||
newer technologies or different flows. The disk based boot is our
|
||||
current solution for our path towards kubernetes migration, but who
|
||||
knows - in the future things might look different again.
|
||||
|
||||
If you want to join the discussion, we have a
|
||||
[Hacking and Learning
|
||||
(#hacking-and-learning:ungleich.ch)](/u/projects/open-chat/) channel
|
||||
on Matrix for an open exchange.
|
||||
|
||||
Oh and in case [you were wondering what we did
|
||||
today](https://twitter.com/ungleich/status/1432627966316584968), we
|
||||
switched to disk based booting - that case is full of SSDs, not 1'000
|
||||
CHF banknotes.
|
After Width: | Height: | Size: 1 MiB |
199
content/u/blog/glamp-1-2021/contents.lr
Normal file
|
@ -0,0 +1,199 @@
|
|||
title: GLAMP #1 2021
|
||||
---
|
||||
pub_date: 2021-07-17
|
||||
---
|
||||
author: ungleich
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: yes
|
||||
---
|
||||
abstract:
|
||||
The first un-hack4glarus happens as a camp - Thursday 2021-08-19 to Sunday 2021-08-22.
|
||||
---
|
||||
body:
|
||||
|
||||
## Glamp 2021 has been CANCELLED
|
||||
|
||||
Due to the rising number of Coronavirus infections, travel
|
||||
restrictions and travel uncertainties,
|
||||
we have decided to CANCEL the GLAMP2021.
|
||||
|
||||
|
||||
## Tl;DR
|
||||
|
||||
Get your tent, connect it to power and 10Gbit/s Internet in the midst
|
||||
of the Glarner mountains. Happenening Thursday 2021-08-19 to Sunday 2021-08-22.
|
||||
Apply for participation by mail (information at the bottom of the page).
|
||||
|
||||
## Introduction
|
||||
|
||||
It has been some time since our
|
||||
[last Hack4Glarus](https://hack4glarus.ch) and we have been missing
|
||||
all our friends, hackers and participants. At ungleich we have been
|
||||
watching the development of the Coronavirus world wide and as you
|
||||
might know, we have decided against a Hack4Glarus for this summer, as
|
||||
the Hack4Glarus has been an indoor event so far.
|
||||
|
||||
## No Hack4Glarus = GLAMP
|
||||
|
||||
However, we want to try a different format that ensures proper
|
||||
safety. Instead of an indoor Hack4Glarus in Linthal, we introduce
|
||||
the Glarus Camp (or GLAMP in short) to you. An outdoor event with
|
||||
sufficient space for distancing. As a camping site we can use the
|
||||
surrounding of the Hacking Villa, supported by the Hacking Villa
|
||||
facilities.
|
||||
|
||||
Compared to the Hack4Glarus, the GLAMP will focus more on
|
||||
*relaxation*, *hangout* than being a hackathon. We think times are
|
||||
hard enough to give everyone a break.
|
||||
|
||||
## The setting
|
||||
|
||||
Many of you know the [Hacking Villa](/u/projects/hacking-villa/) in
|
||||
Diesbach already. Located just next to the pretty waterfall and the amazing
|
||||
Legler Areal. The villa is connected with 10 Gbit/s to the
|
||||
[Data Center Light](/u/projects/data-center-light/) and offers a lot
|
||||
of fun things to do.
|
||||
|
||||
## Coronavirus measures beforehand
|
||||
|
||||
To ensure safety for everyone, we ask everyone attending to provide a
|
||||
reasonable proof of not spreading the corona virus with one of the
|
||||
following proofs:
|
||||
|
||||
* You have been vaccinated
|
||||
* You had the corona virus and you are symptom free for at least 14
|
||||
days
|
||||
* You have been tested with a PCR test (7 days old at maximum) and the
|
||||
result was negative
|
||||
|
||||
All participants will be required to take an short antigen test on
|
||||
site.
|
||||
|
||||
**Please do not attend if you feel sick for the safety of everyone else.**
|
||||
|
||||
## Coronavirus measures on site
|
||||
|
||||
To keep the space safe on site as well, we ask you to follow these
|
||||
rules:
|
||||
|
||||
* Sleep in your own tent
|
||||
* Wear masks inside the Hacking Villa
|
||||
* Especially if you are preparing food shared with others
|
||||
* Keep distance and respect others safety wishes
|
||||
|
||||
## Hacking Villa Facilities
|
||||
|
||||
* Fast Internet (what do you need more?)
|
||||
* A shared, open area outside for hacking
|
||||
* Toilets and bath room located inside
|
||||
|
||||
## What to bring
|
||||
|
||||
* A tent + sleeping equipment
|
||||
* Fun stuff
|
||||
* Your computer
|
||||
* Wifi / IoT / Hacking things
|
||||
* If you want wired Internet in your tent: a 15m+ Ethernet cable
|
||||
* WiFi will be provided everywhere
|
||||
|
||||
## What is provided
|
||||
|
||||
* Breakfast every morning
|
||||
* A place for a tent
|
||||
* Power to the tent (Swiss plug)
|
||||
* WiFi to the tent
|
||||
* Traditional closing event spaghetti
|
||||
|
||||
## What you can find nearby
|
||||
|
||||
* A nearby supermarket (2km) reachable by foot, scooter, bike
|
||||
* A waterfall + barbecue place (~400m)
|
||||
* Daily attractions such as hacking, hiking, biking, hanging out
|
||||
|
||||
## Registration
|
||||
|
||||
As the space is limited, we can accomodate about 10 tents (roughly 23
|
||||
people). To register, send an email to support@ungleich.ch based on
|
||||
the following template:
|
||||
|
||||
```
|
||||
Subject: GLAMP#1 2021
|
||||
|
||||
For each person with you (including yourself):
|
||||
|
||||
Non Coronavirus proof:
|
||||
(see requirements on the glamp page)
|
||||
|
||||
Name(s):
|
||||
(how you want to be called)
|
||||
|
||||
Interests:
|
||||
(will be shown to others at the glamp)
|
||||
|
||||
Skills:
|
||||
(will be shown to others at the glamp)
|
||||
|
||||
Food interests:
|
||||
(we use this for pooling food orders)
|
||||
|
||||
What I would like to do:
|
||||
(will be shown to others at the glamp)
|
||||
|
||||
```
|
||||
|
||||
The particaption fee is 70 CHF/person (to be paid on arrival).
|
||||
|
||||
## Time, Date and Location
|
||||
|
||||
* Arrival possible from Wednesday 2021-08-18 16:00
|
||||
* GLAMP#1 starts officially on Thursday 2021-08-19, 1000
|
||||
* GLAMP#1 closing lunch Sunday 2021-08-22, 1200
|
||||
* GLAMP#1 ends officially on to Sunday 2021-08-22, 1400
|
||||
|
||||
Location: [Hacking Villa](/u/projects/hacking-villa/)
|
||||
|
||||
|
||||
## FAQ
|
||||
|
||||
### Where do I get Internet?
|
||||
|
||||
It is available everywhere at/around the Hacking Villa via WiFi. For
|
||||
cable based Internet bring a 15m+ Ethernet cable.
|
||||
|
||||
### Where do I get Electricity?
|
||||
|
||||
You'll get electricity directly to the tent. Additionally the shared
|
||||
area also has electricity. You can also bring solar panels, if you
|
||||
like.
|
||||
|
||||
### Where do I get food?
|
||||
|
||||
Breakfast is provided by us. But what about the rest of the day?
|
||||
There are a lot of delivery services available, ranging from Pizza,
|
||||
Tibetan, Thai, Swiss (yes!), etc. available.
|
||||
|
||||
Nearby are 2 Volg supermarkets, next Coop is in Schwanden, bigger
|
||||
Migros in Glarus and very big Coop can be found in Netstal. The Volg
|
||||
is reachable by foot, all others are reachable by train or bike.
|
||||
|
||||
There is also a kitchen inside the Hacking Villa for cooking.
|
||||
There is also a great barbecue place just next to the waterfall.
|
||||
|
||||
### What can I do at the GLAMP?
|
||||
|
||||
There are
|
||||
[alot](http://hyperboleandahalf.blogspot.com/2010/04/alot-is-better-than-you-at-everything.html)
|
||||
of opportunities at the GLAMP:
|
||||
|
||||
You can ...
|
||||
|
||||
* just relax and hangout
|
||||
* hack on project that you post poned for long
|
||||
* hike up mountains (up to 3612m! Lower is also possible)
|
||||
* meet other hackers
|
||||
* explore the biggest water power plant in Europe (Linth Limmern)
|
||||
* and much much more!
|
BIN
content/u/blog/glamp-1-2021/diesback-bg-small.jpg
Normal file
After Width: | Height: | Size: 380 KiB |
|
@ -46,13 +46,10 @@ But you cannot get a certicate that you need for HTTPS without a name.
|
|||
|
||||
At the last [Hack4Glarus](https://hack4glarus.ch) we were
|
||||
brainstorming and testing solutions on how to solve this problem. How
|
||||
can we give **any** IPv6 address a name? At the Hackathon our
|
||||
participants invited a cool [stateful
|
||||
solution](https://redmine.ungleich.ch/issues/7379)
|
||||
that is now even reachable at [weneedaname](https://weneeda.name/).
|
||||
can we give **any** IPv6 address a name?
|
||||
|
||||
After the hackathon our team was continuing to brainstorm on how to
|
||||
solve this problem, but in a stateless way.
|
||||
solve this problem in a stateless way.
|
||||
|
||||
## Knot to the rescue
|
||||
|
||||
|
@ -100,3 +97,10 @@ On popular request, we have added support for **has-aaaa.name**,
|
|||
too. So you can for instance reach
|
||||
*2a0a-e5c0-0000-0002-0400-b3ff-fe39-795c.has-aaaa.name*, which is the
|
||||
IPv6 address of [ungleich.ch](https://ungleich.ch).
|
||||
|
||||
## Update 2021-08-12
|
||||
|
||||
* The stateful project domain as originally developed at the
|
||||
[Hack4Glarus is not in use
|
||||
anymore](https://redmine.ungleich.ch/issues/7379). This does *not*
|
||||
affect the has-a.name or has-aaa.name domains which are run by ungleich.ch.
|
||||
|
|
After Width: | Height: | Size: 167 KiB |
|
@ -0,0 +1,123 @@
|
|||
title: Configuring bind to only forward DNS to a specific zone
|
||||
---
|
||||
pub_date: 2021-07-25
|
||||
---
|
||||
author: ungleich
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: yes
|
||||
---
|
||||
abstract:
|
||||
Want to use BIND for proxying to another server? This is how you do it.
|
||||
---
|
||||
body:
|
||||
|
||||
## Introduction
|
||||
|
||||
In this article we'll show you an easy solution to host DNS zones on
|
||||
IPv6 only or private DNS servers. The method we use here is **DNS
|
||||
forwarding** as offered in ISC BIND, but one could also see this as
|
||||
**DNS proxying**.
|
||||
|
||||
## Background
|
||||
|
||||
Sometimes you might have a DNS server that is authoritative for DNS
|
||||
data, but is not reachable for all clients. This might be the case for
|
||||
instance, if
|
||||
|
||||
* your DNS server is IPv6 only: it won't be directly reachable from
|
||||
the IPv4 Internet
|
||||
* your DNS server is running in a private network, either IPv4 or IPv6
|
||||
|
||||
In both cases, you need something that is publicly reachable, to
|
||||
enable clients to access the zone, like show in the following picture:
|
||||
|
||||
![](dns-proxy-forward.png)
|
||||
|
||||
## The problem: Forwarding requires recursive queries
|
||||
|
||||
ISC Bind allows to forward queries to another name server. However to
|
||||
do so, it need to be configured to allow handling recursive querying.
|
||||
However, if we allow recursive querying by any client, we basically
|
||||
create an [Open DNS resolver, which can be quite
|
||||
dangerous](https://www.ncsc.gov.ie/emailsfrom/DDoS/DNS/).
|
||||
|
||||
## The solution
|
||||
|
||||
ISC Bind by default has a root hints file compiled in, which allows it
|
||||
to function as a resolver without any additional configuration
|
||||
files. That is great, but not if you want to prevent it to work as
|
||||
forwarder as described above. But we can easily fix that problem. Now,
|
||||
let's have a look at a real world use case, step-by-step:
|
||||
|
||||
### Step 1: Global options
|
||||
|
||||
In the first step, we need to set the global to allow recursion from
|
||||
anyone, as follows:
|
||||
|
||||
```
|
||||
options {
|
||||
directory "/var/cache/bind";
|
||||
|
||||
listen-on-v6 { any; };
|
||||
|
||||
allow-recursion { ::/0; 0.0.0.0/0; };
|
||||
};
|
||||
```
|
||||
|
||||
However as mentioned above, this would create an open resolver. To
|
||||
prevent this, let's disable the root hints:
|
||||
|
||||
### Step 2: Disable root hints
|
||||
|
||||
The root hints are served in the root zone, also know as ".". To
|
||||
disable it, we give bind an empty file to use:
|
||||
|
||||
```
|
||||
zone "." {
|
||||
type hint;
|
||||
file "/dev/null";
|
||||
};
|
||||
```
|
||||
|
||||
Note: in case you do want to allow recursive function for some
|
||||
clients, **you can create multiple DNS views**.
|
||||
|
||||
### Step 3: The actual DNS file
|
||||
|
||||
In our case, we have a lot of IPv6 only kubernetes clusters, which are
|
||||
named `xx.k8s.ooo` and have a world wide rachable CoreDNS server built
|
||||
in. In this case, we want to allow the domain c1.k8s.ooo to be world
|
||||
reachable, so we configure the dual stack server as follows:
|
||||
|
||||
```
|
||||
zone "c1.k8s.ooo" {
|
||||
type forward;
|
||||
forward only;
|
||||
forwarders { 2a0a:e5c0:2:f::a; };
|
||||
};
|
||||
```
|
||||
|
||||
### Step 4: adjusting the zone file
|
||||
|
||||
In case you are running an IPv6 only server, you need to configure the
|
||||
upstream DNS server. In our case this looks as follows:
|
||||
|
||||
```
|
||||
; The domain: c1.k8s.ooo
|
||||
c1 NS kube-dns.kube-system.svc.c1
|
||||
|
||||
; The IPv6 only DNS server
|
||||
kube-dns.kube-system.svc.c1 AAAA 2a0a:e5c0:2:f::a
|
||||
|
||||
; The forwarding IPv4 server
|
||||
kube-dns.kube-system.svc.c1 A 194.5.220.43
|
||||
```
|
||||
|
||||
## DNS, IPv6, Kubernetes?
|
||||
|
||||
If you are curious to learn more about either of these topics, feel
|
||||
[free to join us on our chat](/u/projects/open-chat/).
|
After Width: | Height: | Size: 154 KiB |
210
content/u/blog/ipv6-link-local-support-in-browsers/contents.lr
Normal file
|
@ -0,0 +1,210 @@
|
|||
title: Support for IPv6 link local addresses in browsers
|
||||
---
|
||||
pub_date: 2021-06-14
|
||||
---
|
||||
author: ungleich
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: yes
|
||||
---
|
||||
abstract:
|
||||
Tracking the progress of browser support for link local addresses
|
||||
---
|
||||
body:
|
||||
|
||||
## Introduction
|
||||
|
||||
Link Local addresses
|
||||
([fe80::/10](https://en.wikipedia.org/wiki/Link-local_address)) are
|
||||
used for addressing devices in your local subnet. They can be
|
||||
automatically generated and using the IPv6 multicast address
|
||||
**ff02::1**, all hosts on the local subnet can easily be located.
|
||||
|
||||
However browsers like Chrome or Firefox do not support **entering link
|
||||
local addresses inside a URL**, which prevents accessing devices
|
||||
locally with a browser, for instance for configuring them.
|
||||
|
||||
Link local addresses need **zone identifiers** to specify which
|
||||
network device to use as an outgoing interface. This is because
|
||||
**you have link local addresses on every interface** and your network
|
||||
stack does not know on its own, which interface to use. So typically a
|
||||
link local address is something on the line of
|
||||
**fe80::fae4:e3ff:fee2:37a4%eth0**, where **eth0** is the zone
|
||||
identifier.
|
||||
|
||||
Them problem is becoming more emphasised, as the world is moving more
|
||||
and more towards **IPv6 only networks**.
|
||||
|
||||
You might not even know the address of your network equipment anymore,
|
||||
but you can easily locate iit using the **ff02::1 multicast
|
||||
address**. So we need support in browsers, to allow network
|
||||
configurations.
|
||||
|
||||
## Status of implementation
|
||||
|
||||
The main purpose of this document is to track the status of the
|
||||
link-local address support in the different browsers and related
|
||||
standards. The current status is:
|
||||
|
||||
* Firefox says whatwg did not define it
|
||||
* Whatwg says zone id is intentionally omitted and and reference w3.org
|
||||
* w3.org has a longer reasoning, but it basically boils down to
|
||||
"Firefox and chrome don't do it and it's complicated and nobody needs it"
|
||||
* Chromium says it seems not to be worth the effort
|
||||
|
||||
Given that chain of events, if either Firefox, Chrome, W3.org or
|
||||
Whatwg where to add support for it, it seems likely that the others
|
||||
would be following.
|
||||
|
||||
## IPv6 link local address support in Firefox
|
||||
|
||||
The progress of IPv6 link local addresses for Firefox is tracked
|
||||
on [the mozilla
|
||||
bugzilla](https://bugzilla.mozilla.org/show_bug.cgi?id=700999). The
|
||||
current situation is that Firefox references to the lack of
|
||||
standardisation by whatwg as a reason for not implementing it. Quoting
|
||||
Valentin Gosu from the Mozilla team:
|
||||
|
||||
```
|
||||
The main reason the zone identifier is not supported in Firefox is
|
||||
that parsing URLs is hard. You'd think we can just pass whatever
|
||||
string to the system API and it will work or fail depending on whether
|
||||
it's valid or not, but that's not the case. In bug 1199430 for example
|
||||
it was apparent that we need to make sure that the hostname string is
|
||||
really valid before passing it to the OS.
|
||||
|
||||
I have no reason to oppose zone identifiers in URLs as long as the URL
|
||||
spec defines how to parse them. As such, I encourage you to engage
|
||||
with the standard at https://github.com/whatwg/url/issues/392 instead
|
||||
of here.
|
||||
|
||||
Thank you!
|
||||
```
|
||||
|
||||
## IPv6 link local address support in whatwg
|
||||
|
||||
The situation at [whatwg](https://whatwg.org/) is that there is a
|
||||
[closed bug report on github](https://github.com/whatwg/url/issues/392)
|
||||
and [in the spec it says](https://url.spec.whatwg.org/#concept-ipv6)
|
||||
that
|
||||
|
||||
Support for <zone_id> is intentionally omitted.
|
||||
|
||||
That paragraph links to a bug registered at w3.org (see next chapter).
|
||||
|
||||
|
||||
## IPv6 link local address support at w3.org
|
||||
|
||||
At [w3.org](https://www.w3.org/) there is a
|
||||
bug titled
|
||||
[Support IPv6 link-local
|
||||
addresses?](https://www.w3.org/Bugs/Public/show_bug.cgi?id=27234#c2)
|
||||
that is set to status **RESOLVED WONTFIX**. It is closed basically
|
||||
based on the following statement from Ryan Sleevi:
|
||||
|
||||
```
|
||||
Yes, we're especially not keen to support these in Chrome and have
|
||||
repeatedly decided not to. The platform-specific nature of <zone_id>
|
||||
makes it difficult to impossible to validate the well-formedness of
|
||||
the URL (see https://tools.ietf.org/html/rfc4007#section-11.2 , as
|
||||
referenced in 6874, to fully appreciate this special hell). Even if we
|
||||
could reliably parse these (from a URL spec standpoint), it then has
|
||||
to be handed 'somewhere', and that opens a new can of worms.
|
||||
|
||||
Even 6874 notes how unlikely it is to encounter these in practice -
|
||||
"Thus, URIs including a
|
||||
ZoneID are unlikely to be encountered in HTML documents. However, if
|
||||
they do (for example, in a diagnostic script coded in HTML), it would
|
||||
be appropriate to treat them exactly as above."
|
||||
|
||||
Note that a 'dumb' parser may not be sufficient, as the Security Considerations of 6874 note:
|
||||
"To limit this risk, implementations MUST NOT allow use of this format
|
||||
except for well-defined usages, such as sending to link-local
|
||||
addresses under prefix fe80::/10. At the time of writing, this is
|
||||
the only well-defined usage known."
|
||||
|
||||
And also
|
||||
"An HTTP client, proxy, or other intermediary MUST remove any ZoneID
|
||||
attached to an outgoing URI, as it has only local significance at the
|
||||
sending host."
|
||||
|
||||
This requires a transformative rewrite of any URLs going out the
|
||||
wire. That's pretty substantial. Anne, do you recall the bug talking
|
||||
about IP canonicalization (e.g. http://127.0.0.1 vs
|
||||
http://[::127.0.0.1] vs http://012345 and friends?) This is
|
||||
conceptually a similar issue - except it's explicitly required in the
|
||||
context of <zone_id> that the <zone_id> not be emitted.
|
||||
|
||||
There's also the issue that zone_id precludes/requires the use of APIs
|
||||
that user agents would otherwise prefer to avoid, in order to
|
||||
'properly' handle the zone_id interpretation. For example, Chromium on
|
||||
some platforms uses a built in DNS resolver, and so our address lookup
|
||||
functions would need to define and support <zone_id>'s and map them to
|
||||
system concepts. In doing so, you could end up with weird situations
|
||||
where a URL works in Firefox but not Chrome, even though both
|
||||
'hypothetically' supported <zone_id>'s, because FF may use an OS
|
||||
routine and Chrome may use a built-in routine and they diverge.
|
||||
|
||||
Overall, our internal consensus is that <zone_id>'s are bonkers on
|
||||
many grounds - the technical ambiguity (and RFC 6874 doesn't really
|
||||
resolve the ambiguity as much as it fully owns it and just says
|
||||
#YOLOSWAG) - and supporting them would add a lot of complexity for
|
||||
what is explicitly and admittedly a limited value use case.
|
||||
```
|
||||
|
||||
This bug references the Mozilla Firefox bug above and
|
||||
[RFC3986 (replaced by RFC
|
||||
6874)](https://datatracker.ietf.org/doc/html/rfc6874#section-2).
|
||||
|
||||
## IPv6 link local address support in Chrome / Chromium
|
||||
|
||||
On the chrome side there is a
|
||||
[huge bug
|
||||
report](https://bugs.chromium.org/p/chromium/issues/detail?id=70762)
|
||||
which again references a huge number of other bugs that try to request
|
||||
IPv6 link local support, too.
|
||||
|
||||
The bug was closed by cbentzel@chromium.org stating:
|
||||
|
||||
```
|
||||
There are a large number of special cases which are required on core
|
||||
networking/navigation/etc. and it does not seem like it is worth the
|
||||
up-front and ongoing maintenance costs given that this is a very
|
||||
niche - albeit legitimate - need.
|
||||
```
|
||||
|
||||
The bug at chromium has been made un-editable so it is basically
|
||||
frozen, besides people have added suggestions to the ticket on how to
|
||||
solve it.
|
||||
|
||||
## Work Arounds
|
||||
|
||||
### IPv6 link local connect hack
|
||||
|
||||
Peter has [documented on the IPv6 link local connect
|
||||
hack](https://website.peterjin.org/wiki/Snippets:IPv6_link_local_connect_hack)
|
||||
to make firefox use **fe90:0:[scope id]:[IP address]** to reach
|
||||
**fe80::[IP address]%[scope id]**. Checkout his website for details!
|
||||
|
||||
### IPv6 hack using ip6tables
|
||||
|
||||
Also from Peter is the hint that you can also use newer iptable
|
||||
versions to achieve a similar mapping:
|
||||
|
||||
"On modern Linux kernels you can also run
|
||||
|
||||
```ip6tables -t nat -A OUTPUT -d fef0::/64 -j NETMAP --to fe80::/64```
|
||||
|
||||
if you have exactly one outbound interface, so that fef0::1 translates
|
||||
to fe80::1"
|
||||
|
||||
Thanks again for the pointer!
|
||||
|
||||
## Other resources
|
||||
|
||||
If you are aware of other resources regarding IPv6 link local support
|
||||
in browsers, please join the [IPv6.chat](https://IPv6.chat) and let us
|
||||
know about it.
|
200
content/u/blog/ipv6-vpn-dns-entries/contents.lr
Normal file
|
@ -0,0 +1,200 @@
|
|||
title: IPv6, VPN and DNS entries
|
||||
---
|
||||
pub_date: 2021-10-13
|
||||
---
|
||||
author: Nico Schottelius
|
||||
---
|
||||
twitter_handle: NicoSchottelius
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: yes
|
||||
---
|
||||
abstract:
|
||||
Looking at how the patterns of VPN and DNS names changes with IPv6
|
||||
---
|
||||
body:
|
||||
|
||||
## TL; DR
|
||||
|
||||
With IPv6, DNS management of protected networks can be
|
||||
simplified. IPv6 VPNs can use simplified DNS configurations to
|
||||
simplify the network configurations by just using public, restricted
|
||||
DNS entries.
|
||||
|
||||
## VPN and DNS in the IPv4 world
|
||||
|
||||
VPNs in the IPv4 world are often used to create site-to-site tunnels,
|
||||
allowing different networks to talk to each other. A typical case is
|
||||
that organisation A needs to access protected resources of
|
||||
organisation B and maybe even vice-versa. So a typical VPN looks like
|
||||
this:
|
||||
|
||||
```
|
||||
Organisation A
|
||||
--------------
|
||||
|
||||
Protected Host A ---------- Router/VPN gateway
|
||||
(10.0.0.42/24) |
|
||||
|
|
||||
|
|
||||
Organisation B (Internet)
|
||||
-------------- |
|
||||
|
|
||||
|
|
||||
Protected Host B ---------- Router/VPN gateway
|
||||
(10.20.0.42/24)
|
||||
Host name: lakeside.int.org-b.example.com
|
||||
```
|
||||
|
||||
Now if the Protected Host A and Protected Host B want to communicate
|
||||
with each other on IP basis, this is no problem (I am not elaborating
|
||||
on the problems of IP collisions in this article, a follow up article
|
||||
will follow soon).
|
||||
|
||||
However if Protected Host A wants to reach the Protected Host B via
|
||||
its internal DNS name **lakeside.int.org-b.example.com**, this is
|
||||
usually a problem, for multiple reasons:
|
||||
|
||||
* Protected Host A might not know the right internal DNS server to
|
||||
query for int.org-b.example.com.
|
||||
* Protected Host A might know the right internal DNS server to
|
||||
query for int.org-b.example.com, but might not have access to it via
|
||||
the VPN
|
||||
* The DNS records for int.org-b.example.com often are intentionally
|
||||
not published to public DNS for multiple reasons: privacy related or
|
||||
because administrators don't like to publish RFC1918 records into
|
||||
public DNS records
|
||||
|
||||
|
||||
## VPN and DNS in the IPv6 world
|
||||
|
||||
There are multiple ways of how VPNs can be built in the IPv6 world,
|
||||
including usage of the private IPv4 addresses equivalent named Unique
|
||||
Local Address (ULA). However instead of using ULA, I will today show
|
||||
an approach that is more "IPv6 native", using Global Unique Addresses
|
||||
(GUA), or what is simply known as "public IPv6 address".
|
||||
|
||||
While you might have heard it, I will repeat nonetheless: there are
|
||||
enough IPv6 addresses for every practical use case that we imagine at
|
||||
the moment. This is important, because we can use **globally unique
|
||||
IPv6 addresses** inside the VPN.
|
||||
|
||||
Isn't that a problem? Publicly reachable IPv6 addresses inside a VPN?
|
||||
It would, if the addresses were **globally reachable**. In the IPv6
|
||||
world nothing speaks against having **globally unique, but non-routed
|
||||
IPv6 addresses**. This is actually a perfect match and much better
|
||||
than we can do in the IPv4 world:
|
||||
|
||||
* Both organisations A and B can acquire globally unique
|
||||
addresses. Let's say they organisation A acquires 2001:db8:0::/48 and
|
||||
organisation B acquires 2001:db8:1::/48.
|
||||
* Both organisations have two options: they can announce their IPv6
|
||||
range to the Internet and block access to their internal network or
|
||||
* both they can even consider not to announce their network at all
|
||||
(there is not route in the Internet for it)
|
||||
|
||||
In either case, both organisations will usually select a sub network
|
||||
of size /64 for the resources they want to expose via the VPN. Let's
|
||||
say organisation A chooses 2001:db8:0:cafe::/64 and organisation B
|
||||
chooses 2001:db8:1:7ea::/64. Putting this in context, their VPN now
|
||||
looks like this:
|
||||
|
||||
```
|
||||
Organisation A
|
||||
--------------
|
||||
|
||||
Protected Host A ---------- Router/VPN gateway
|
||||
(2001:db8:0:cafe::42/64) |
|
||||
|
|
||||
|
|
||||
Organisation B (Internet)
|
||||
-------------- |
|
||||
|
|
||||
|
|
||||
Protected Host B ---------- Router/VPN gateway
|
||||
(2001:db8:1:7ea::42/64) |
|
||||
Host name: lakeside.int.org-b.example.com
|
||||
```
|
||||
|
||||
Now, how does this change the DNS server situation? Because we are
|
||||
using IPv6, we have many more options:
|
||||
|
||||
* a) We can publish the DNS records of the domain
|
||||
int.org-b.example.com globally. While access to the network
|
||||
2001:db8:1:7ea::/64 is only possible via VPN, nothing speaks against
|
||||
having the records in a public DNS server. However, some
|
||||
administrators advocate to not publish them publicly for privacy
|
||||
reasons. That is the same logic as publishing or not publish the
|
||||
RFC1918 (10.x.y.z) addresses in the IPv4 world.
|
||||
* b) We can publicly/globally delegate the domain
|
||||
int.org-b.example.com to a nameserver that is only reachable via the
|
||||
VPN.
|
||||
* c) We can proceed the same as in the IPv4 world and have a
|
||||
disconnect, internal DNS server that is responsible for
|
||||
int.org-b.example.com.
|
||||
|
||||
Option (a) is often seen as a security risk and it can be debated
|
||||
whether someone who can already guess the correct hostname and
|
||||
retrieve it's IP address is really a significant higher security
|
||||
thread than anybody just guessing IP addresses.
|
||||
|
||||
Option (c) is the typical case for IPv4 based VPNs and is causing
|
||||
above illustrated issues.
|
||||
|
||||
Option (b) is the one that makes IPv6 VPNs much more interesting than
|
||||
IPv4 based VPNs:
|
||||
|
||||
* The world can know that there is an internal domain
|
||||
**int.org-b.example.com** and find out which DNS servers are
|
||||
responsible for it.
|
||||
* However an attacker easily guesses that internal networks exist
|
||||
anyway.
|
||||
|
||||
Let's have a look at sample nameserver entries in detail:
|
||||
|
||||
```
|
||||
int.org-b.example.com. NS ns-int1.org-b.example.com.
|
||||
int.org-b.example.com. NS ns-int2.org-b.example.com.
|
||||
```
|
||||
|
||||
What does that mean? Anyone in the world can retrieve the information
|
||||
that int.org-b.example.com has two DNS servers. However the DNS
|
||||
servers responsible for org-b.example.com can hide the IP addresses of
|
||||
ns-int1.org-b.example.com and ns-int2.org-b.example.com for everyone,
|
||||
but hosts coming from organisation A. Or even if the IP addressses of
|
||||
ns-int1.org-b.example.com and ns-int2.org-b.example.com are world
|
||||
known, access to them can easily be prevented.
|
||||
|
||||
The measures for this can for instance be DNS views or firewall
|
||||
entries. In practice this means for VPNs in the IPv6 world:
|
||||
|
||||
|
||||
```
|
||||
Organisation A
|
||||
--------------
|
||||
|
||||
Protected Host A: what is the IP address of lakeside.int.org-b.example.com?
|
||||
DNS Server of Organisation B: 2001:db8:1:7ea::42
|
||||
|
||||
|
||||
Outside party
|
||||
-------------
|
||||
Outside Hosts: what is the IP address of lakeside.int.org-b.example.com?
|
||||
|
||||
a) DNS Server of Organisation B: there is no domain
|
||||
int.org-b.example.com (DNS view restriction)
|
||||
b) DNS Server of Organisation B: these are the nameserver for
|
||||
int.org-b.example.com, but you cannot reach them (firewall protection)
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
For IPv6 based VPNs you can get away without reconfiguring your source
|
||||
networks for DNS servers of the destination party. The target party
|
||||
always needs to ensure proper access control to internal resources, so
|
||||
there is no additional overhead.
|
||||
|
||||
DNS, correctly used in the IPv6 VPN world, is a really smooth
|
||||
operation. This is why we recommend to use
|
||||
[IPv6 as a basis for VPNs](https://ipv6vpn.ch).
|
144
content/u/blog/kubernetes-dns-entries-nat64/contents.lr
Normal file
|
@ -0,0 +1,144 @@
|
|||
title: Automatic A and AAAA DNS entries with NAT64 for kubernetes?
|
||||
---
|
||||
pub_date: 2021-06-24
|
||||
---
|
||||
author: ungleich
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: yes
|
||||
---
|
||||
abstract:
|
||||
Given a kubernetes cluster and NAT64 - how do you create DNS entries?
|
||||
---
|
||||
body:
|
||||
|
||||
## The DNS kubernetes quiz
|
||||
|
||||
Today our blog entry does not (yet) show a solution, but more a tricky
|
||||
quiz on creating DNS entries. The problem to solve is the following:
|
||||
|
||||
* How to make every IPv6 only service in kubernetes also IPv4
|
||||
reachable?
|
||||
|
||||
Let's see who can solve it first or the prettiest. Below are some
|
||||
thoughts on how to approach this problem.
|
||||
|
||||
## The situation
|
||||
|
||||
Assume your kubernetes cluster is IPv6 only and all services
|
||||
have proper AAAA DNS entries. This allows you
|
||||
[to directly receive traffic from the
|
||||
Internet](/u/blog/kubernetes-without-ingress/) to
|
||||
your kubernetes services.
|
||||
|
||||
Now to make that service also IPv4 reachable, we can deploy NAT64
|
||||
service that maps an IPv4 address outside the cluster to an IPv6 service
|
||||
address inside the cluster:
|
||||
|
||||
```
|
||||
A.B.C.D --> 2001:db8::1
|
||||
```
|
||||
|
||||
So all traffic to that IPv4 address is converted to IPv6 by the
|
||||
external NAT64 translator.
|
||||
|
||||
## The proxy service
|
||||
|
||||
Let's say the service running on 2001:db8::1 is named "ipv4-proxy" and
|
||||
thus reachable at ipv4-proxy.default.svc.example.com.
|
||||
|
||||
What we want to achieve is to expose every possible service
|
||||
inside the cluster **also via IPv4**. For this purpose we have created
|
||||
an haproxy container that access *.svc.example.com and forwards it via
|
||||
IPv6.
|
||||
|
||||
So the actual flow would look like:
|
||||
|
||||
```
|
||||
IPv4 client --[ipv4]--> NAT64 -[ipv6]-> proxy service
|
||||
|
|
||||
|
|
||||
v
|
||||
IPv6 client ---------------------> kubernetes service
|
||||
```
|
||||
|
||||
## The DNS dilemma
|
||||
|
||||
It would be very tempting to create a wildcard DNS entry or to
|
||||
configure/patch CoreDNS to also include an A entry for every service
|
||||
that is:
|
||||
|
||||
```
|
||||
*.svc IN A A.B.C.D
|
||||
```
|
||||
|
||||
So essentially all services resolve to the IPv4 address A.B.C.D. That
|
||||
however would also influence the kubernetes cluster, as pods
|
||||
potentially resolve A entries (not only AAAA) as well.
|
||||
|
||||
As the containers / pods do not have any IPv4 address (nor IPv4
|
||||
routing), access to IPv4 is not possible. There are various outcomes
|
||||
of this situation:
|
||||
|
||||
1. The software in the container does happy eyeballs and tries both
|
||||
A/AAAA and uses the working IPv6 connection.
|
||||
|
||||
2. The software in the container misbehaves and takes the first record
|
||||
and uses IPv4 (nodejs is known to have or had a broken resolver
|
||||
that did exactly that).
|
||||
|
||||
So adding that wildcard might not be the smartest option. And
|
||||
additionally it is unclear whether coreDNS would support that.
|
||||
|
||||
## Alternative automatic DNS entries
|
||||
|
||||
The *.svc names in a kubernetes cluster are special in the sense that
|
||||
they are used for connecting internally. What if coreDNS (or any other
|
||||
DNS) server would instead of using *.svc, use a second subdomain like
|
||||
*abc*.*namespace*.v4andv6.example.com and generate the same AAAA
|
||||
record as for the service and a static A record like describe above?
|
||||
|
||||
That could solve the problem. But again, does coreDNS support that?
|
||||
|
||||
## Automated DNS entries in other zones
|
||||
|
||||
Instead of fully automated creating the entries as above, another
|
||||
option would be to specify DNS entries via annotations in a totally
|
||||
different zone, if coreDNS was supporting this. So let's say we also
|
||||
have control over example.org and we could instruct coreDNS to create
|
||||
the following entries automatically with an annotation:
|
||||
|
||||
```
|
||||
abc.something.example.org AAAA <same as the service IP>
|
||||
abc.something.example.org A <a static IPv4 address A.B.C.D>
|
||||
```
|
||||
|
||||
In theory this might be solved via some scripting, maybe via a DNS
|
||||
server like powerDNS?
|
||||
|
||||
## Alternative solution with BIND
|
||||
|
||||
The bind DNS server, which is not usually deployed in a kubernetes
|
||||
cluster, supports **views**. Views enable different replies to the
|
||||
same query depending on the source IP address. Thus in theory
|
||||
something like that could be done, assuming a secondary zone
|
||||
*example.org*:
|
||||
|
||||
* If the request comes from the kubernetes cluster, return a CNAME
|
||||
back to example.com.
|
||||
* If the request comes from outside the kubernetes cluster, return an
|
||||
A entry with the static IP
|
||||
* Unsolved: how to match on the AAAA entries (because we don't CNAME
|
||||
with the added A entry)
|
||||
|
||||
|
||||
## Other solution?
|
||||
|
||||
As you can see, mixing the dynamic IP generation and coupling it with
|
||||
static DNS entries for IPv4 resolution is not the easiest tasks. If
|
||||
you have a smart idea on how to solve this without manually creating
|
||||
entries for each and every service,
|
||||
[give us a shout!](/u/contact)
|
|
@ -0,0 +1,227 @@
|
|||
title: Making kubernetes kube-dns publicly reachable
|
||||
---
|
||||
pub_date: 2021-06-13
|
||||
---
|
||||
author: ungleich
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: yes
|
||||
---
|
||||
abstract:
|
||||
Looking into IPv6 only DNS provided by kubernetes
|
||||
---
|
||||
body:
|
||||
|
||||
## Introduction
|
||||
|
||||
If you have seen our
|
||||
[article about running kubernetes
|
||||
Ingress-less](/u/blog/kubernetes-without-ingress/), you are aware that
|
||||
we are pushing IPv6 only kubernetes clusters at ungleich.
|
||||
|
||||
Today, we are looking at making the "internal" kube-dns service world
|
||||
reachable using IPv6 and global DNS servers.
|
||||
|
||||
## The kubernetes DNS service
|
||||
|
||||
If you have a look at your typical k8s cluster, you will notice that
|
||||
you usually have two coredns pods running:
|
||||
|
||||
```
|
||||
% kubectl -n kube-system get pods -l k8s-app=kube-dns
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
coredns-558bd4d5db-gz5c7 1/1 Running 0 6d
|
||||
coredns-558bd4d5db-hrzhz 1/1 Running 0 6d
|
||||
```
|
||||
|
||||
These pods are usually served by the **kube-dns** service:
|
||||
|
||||
```
|
||||
% kubectl -n kube-system get svc -l k8s-app=kube-dns
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
kube-dns ClusterIP 2a0a:e5c0:13:e2::a <none> 53/UDP,53/TCP,9153/TCP 6d1h
|
||||
```
|
||||
|
||||
As you can see, the kube-dns service is running on a publicly
|
||||
reachable IPv6 address.
|
||||
|
||||
## IPv6 only DNS
|
||||
|
||||
IPv6 only DNS servers have one drawback: they cannot be reached via DNS
|
||||
recursions, if the resolver is IPv4 only.
|
||||
|
||||
At [ungleich we run a variety of
|
||||
services](https://redmine.ungleich.ch/projects/open-infrastructure/wiki)
|
||||
to make IPv6 only services usable in the real world. In case of DNS,
|
||||
we are using **DNS forwarders**. They are acting similar to HTTP
|
||||
proxies, but for DNS.
|
||||
|
||||
So in our main DNS servers, dns1.ungleich.ch, dns2.ungleich.ch
|
||||
and dns3.ungleich.ch we have added the following configuration:
|
||||
|
||||
```
|
||||
zone "k8s.place7.ungleich.ch" {
|
||||
type forward;
|
||||
forward only;
|
||||
forwarders { 2a0a:e5c0:13:e2::a; };
|
||||
};
|
||||
```
|
||||
|
||||
This tells the DNS servers to forward DNS queries that come in for
|
||||
k8s.place7.ungleich.ch to **2a0a:e5c0:13:e2::a**.
|
||||
|
||||
Additionally we have added **DNS delegation** in the
|
||||
place7.ungleich.ch zone:
|
||||
|
||||
```
|
||||
k8s NS dns1.ungleich.ch.
|
||||
k8s NS dns2.ungleich.ch.
|
||||
k8s NS dns3.ungleich.ch.
|
||||
```
|
||||
|
||||
## Using the kubernetes DNS service in the wild
|
||||
|
||||
With this configuration, we can now access IPv6 only
|
||||
kubernetes services directly from the Internet. Let's first discover
|
||||
the kube-dns service itself:
|
||||
|
||||
```
|
||||
% dig kube-dns.kube-system.svc.k8s.place7.ungleich.ch. aaaa
|
||||
|
||||
; <<>> DiG 9.16.16 <<>> kube-dns.kube-system.svc.k8s.place7.ungleich.ch. aaaa
|
||||
;; global options: +cmd
|
||||
;; Got answer:
|
||||
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23274
|
||||
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
|
||||
|
||||
;; OPT PSEUDOSECTION:
|
||||
; EDNS: version: 0, flags:; udp: 4096
|
||||
; COOKIE: f61925944f5218c9ac21e43960c64f254792e60f2b10f3f5 (good)
|
||||
;; QUESTION SECTION:
|
||||
;kube-dns.kube-system.svc.k8s.place7.ungleich.ch. IN AAAA
|
||||
|
||||
;; ANSWER SECTION:
|
||||
kube-dns.kube-system.svc.k8s.place7.ungleich.ch. 27 IN AAAA 2a0a:e5c0:13:e2::a
|
||||
|
||||
;; AUTHORITY SECTION:
|
||||
k8s.place7.ungleich.ch. 13 IN NS kube-dns.kube-system.svc.k8s.place7.ungleich.ch.
|
||||
```
|
||||
|
||||
As you can see, the **kube-dns** service in the **kube-system**
|
||||
namespace resolves to 2a0a:e5c0:13:e2::a, which is exactly what we
|
||||
have configured.
|
||||
|
||||
At the moment, there is also an etherpad test service
|
||||
named "ungleich-etherpad" running:
|
||||
|
||||
```
|
||||
% kubectl get svc -l app=ungleichetherpad
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
ungleich-etherpad ClusterIP 2a0a:e5c0:13:e2::b7db <none> 9001/TCP 3d19h
|
||||
```
|
||||
|
||||
Let's first verify that it resolves:
|
||||
|
||||
```
|
||||
% dig +short ungleich-etherpad.default.svc.k8s.place7.ungleich.ch aaaa
|
||||
2a0a:e5c0:13:e2::b7db
|
||||
```
|
||||
|
||||
And if that works, well, then we should also be able to access the
|
||||
service itself!
|
||||
|
||||
```
|
||||
% curl -I http://ungleich-etherpad.default.svc.k8s.place7.ungleich.ch:9001/
|
||||
HTTP/1.1 200 OK
|
||||
X-Powered-By: Express
|
||||
X-UA-Compatible: IE=Edge,chrome=1
|
||||
Referrer-Policy: same-origin
|
||||
Content-Type: text/html; charset=utf-8
|
||||
Content-Length: 6039
|
||||
ETag: W/"1797-Dq3+mr7XP0PQshikMNRpm5RSkGA"
|
||||
Set-Cookie: express_sid=s%3AZGKdDe3FN1v5UPcS-7rsZW7CeloPrQ7p.VaL1V0M4780TBm8bT9hPVQMWPX5Lcte%2BzotO9Lsejlk; Path=/; HttpOnly; SameSite=Lax
|
||||
Date: Sun, 13 Jun 2021 18:36:23 GMT
|
||||
Connection: keep-alive
|
||||
Keep-Alive: timeout=5
|
||||
```
|
||||
|
||||
(attention, this is a test service and might not be running when you
|
||||
read this article at a later time)
|
||||
|
||||
## IPv6 vs. IPv4
|
||||
|
||||
Could we have achived the same with IPv4? The answere here is "maybe":
|
||||
If the kubernetes service is reachable from globally reachable
|
||||
nameservers via IPv4, then the answer is yes. This could be done via
|
||||
public IPv4 addresses in the kubernetes cluster, via tunnels, VPNs,
|
||||
etc.
|
||||
|
||||
However, generally speaking, the DNS service of a
|
||||
kubernetes cluster running on RFC1918 IP addresses, is probably not
|
||||
reachable from globally reachable DNS servers by default.
|
||||
|
||||
For IPv6 the case is a bit different: we are using globally reachable
|
||||
IPv6 addresses in our k8s clusters, so they can potentially be
|
||||
reachable without the need of any tunnel or whatsoever. Firewalling
|
||||
and network policies can obviously prevent access, but if the IP
|
||||
addresses are properly routed, they will be accessible from the public
|
||||
Internet.
|
||||
|
||||
And this makes things much easier for DNS servers, which are also
|
||||
having IPv6 connectivity.
|
||||
|
||||
The following pictures shows the practical difference between the two
|
||||
approaches:
|
||||
|
||||
![](/u/image/k8s-v6-v4-dns.png)
|
||||
|
||||
## Does this make sense?
|
||||
|
||||
That clearly depends on your use-case. If you want your service DNS
|
||||
records to be publicly accessible, then the clear answer is yes.
|
||||
|
||||
If your cluster services are intended to be internal only
|
||||
(see [previous blog post](/u/blog/kubernetes-without-ingress/), then
|
||||
exposing the DNS service to the world might not be the best option.
|
||||
|
||||
## Note on security
|
||||
|
||||
CoreDNS inside kubernetes is by default configured to allow resolving
|
||||
for *any* client that can reach it. Thus if you make your kube-dns
|
||||
service world reachable, you also turn it into an open resolver.
|
||||
|
||||
At the time of writing this blog article, the following coredns
|
||||
configuration **does NOT** correctly block requests:
|
||||
|
||||
```
|
||||
Corefile: |
|
||||
.:53 {
|
||||
acl k8s.place7.ungleich.ch {
|
||||
allow net ::/0
|
||||
}
|
||||
acl . {
|
||||
allow net 2a0a:e5c0:13::/48
|
||||
block
|
||||
}
|
||||
forward . /etc/resolv.conf {
|
||||
max_concurrent 1000
|
||||
}
|
||||
...
|
||||
```
|
||||
|
||||
Until this is solved, we recommend to place a firewall before your
|
||||
public kube-dns service to only allow requests from the forwarding DNS
|
||||
servers.
|
||||
|
||||
|
||||
## More of this
|
||||
|
||||
We are discussing
|
||||
kubernetes and IPv6 related topics in
|
||||
**the #hacking:ungleich.ch Matrix channel**
|
||||
([you can signup here if you don't have an
|
||||
account](https://chat.with.ungleich.ch)) and will post more about our
|
||||
k8s journey in this blog. Stay tuned!
|
122
content/u/blog/kubernetes-network-planning-with-ipv6/contents.lr
Normal file
|
@ -0,0 +1,122 @@
|
|||
title: Kubernetes Network planning with IPv6
|
||||
---
|
||||
pub_date: 2021-06-26
|
||||
---
|
||||
author: ungleich
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: no
|
||||
---
|
||||
abstract:
|
||||
Learn which networks are good to use with kubernetes
|
||||
---
|
||||
body:
|
||||
|
||||
## Introduction
|
||||
|
||||
While IPv6 has a huge address space, you will need to specify a
|
||||
**podCidr** (the network for the pods) and a **serviceCidr** (the
|
||||
network for the services) for kubernetes. In this blog article we show
|
||||
our findings and give a recommendation on what are the "most sensible"
|
||||
networks to use for kubernetes.
|
||||
|
||||
## TL;DR
|
||||
|
||||
|
||||
## Kubernetes limitations
|
||||
|
||||
In a typical IPv6 network, you would "just assign a /64" to anything
|
||||
that needs to be a network. It is a bit the IPv6-no-brainer way of
|
||||
handling networking.
|
||||
|
||||
However, kubernetes has a limitation:
|
||||
[the serviceCidr cannot be bigger than a /108 at the
|
||||
moment](https://github.com/kubernetes/kubernetes/pull/90115).
|
||||
This is something very atypical for the IPv6 world, but nothing we
|
||||
cannot handle. There are various pull requests and issues to fix this
|
||||
behaviour on github, some of them listed below:
|
||||
|
||||
* https://github.com/kubernetes/enhancements/pull/1534
|
||||
* https://github.com/kubernetes/kubernetes/pull/79993
|
||||
* https://github.com/kubernetes/kubernetes/pull/90115 (this one is
|
||||
quite interesting to read)
|
||||
|
||||
That said, it is possible to use a /64 for the **podCidr**.
|
||||
|
||||
## The "correct way" without the /108 limitation
|
||||
|
||||
If kubernetes did not have this limitation, our recommendation would
|
||||
be to use one /64 for the podCidr and one /64 for the serviceCidr. If
|
||||
in the future the limitations of kubernetes have been lifted, skip
|
||||
reading this article and just use two /64's.
|
||||
|
||||
Do not be tempted to suggest making /108's the default, even if they
|
||||
"have enough space", because using /64's allows you to stay in much
|
||||
easier network plans.
|
||||
|
||||
## Sanity checking the /108
|
||||
|
||||
To be able to plan kubernetes clusters, it is important to know where
|
||||
they should live, especially if you plan having a lot of kubernetes
|
||||
clusters. Let's have a short look at the /108 network limitation:
|
||||
|
||||
A /108 allows 20 bit to be used for generating addresses, or a total
|
||||
of 1048576 hosts. This is probably enough for the number of services
|
||||
in a cluster. Now, can we be consistent and also use a /108 for the
|
||||
podCidr? Let's assume for the moment that we do exactly that, so we
|
||||
run a maximum of 1048576 pods at the same time. Assuming each service
|
||||
consumes on average 4 pods, this would allow one to run 262144
|
||||
services.
|
||||
|
||||
Assuming each pod uses around 0.1 CPUs and 100Mi RAM, if all pods were
|
||||
to run at the same time, you would need ca. 100'000 CPUs and 100 TB
|
||||
RAM. Assuming further that each node contains at maximum 128 CPUs and
|
||||
at maximum 1 TB RAM (quite powerful servers), we would need more than
|
||||
750 servers just for the CPUs.
|
||||
|
||||
So we can reason that **we can** run kubernetes clusters of quite some
|
||||
size even with a **podCidr of /108**.
|
||||
|
||||
## Organising /108's
|
||||
|
||||
Let's assume that we organise all our kubernetes clusters in a single
|
||||
/64, like 2001:db8:1:2::/64, which looks like this:
|
||||
|
||||
```
|
||||
% sipcalc 2001:db8:1:2::/64
|
||||
-[ipv6 : 2001:db8:1:2::/64] - 0
|
||||
|
||||
[IPV6 INFO]
|
||||
Expanded Address - 2001:0db8:0001:0002:0000:0000:0000:0000
|
||||
Compressed address - 2001:db8:1:2::
|
||||
Subnet prefix (masked) - 2001:db8:1:2:0:0:0:0/64
|
||||
Address ID (masked) - 0:0:0:0:0:0:0:0/64
|
||||
Prefix address - ffff:ffff:ffff:ffff:0:0:0:0
|
||||
Prefix length - 64
|
||||
Address type - Aggregatable Global Unicast Addresses
|
||||
Network range - 2001:0db8:0001:0002:0000:0000:0000:0000 -
|
||||
2001:0db8:0001:0002:ffff:ffff:ffff:ffff
|
||||
```
|
||||
|
||||
A /108 network on the other hand looks like this:
|
||||
|
||||
```
|
||||
% sipcalc 2001:db8:1:2::/108
|
||||
-[ipv6 : 2001:db8:1:2::/108] - 0
|
||||
|
||||
[IPV6 INFO]
|
||||
Expanded Address - 2001:0db8:0001:0002:0000:0000:0000:0000
|
||||
Compressed address - 2001:db8:1:2::
|
||||
Subnet prefix (masked) - 2001:db8:1:2:0:0:0:0/108
|
||||
Address ID (masked) - 0:0:0:0:0:0:0:0/108
|
||||
Prefix address - ffff:ffff:ffff:ffff:ffff:ffff:fff0:0
|
||||
Prefix length - 108
|
||||
Address type - Aggregatable Global Unicast Addresses
|
||||
Network range - 2001:0db8:0001:0002:0000:0000:0000:0000 -
|
||||
2001:0db8:0001:0002:0000:0000:000f:ffff
|
||||
```
|
||||
|
||||
Assuming for a moment that we assign a /108, this looks as follows:
|
70
content/u/blog/kubernetes-production-cluster-1/contents.lr
Normal file
|
@ -0,0 +1,70 @@
|
|||
title: ungleich production cluster #1
|
||||
---
|
||||
pub_date: 2021-07-05
|
||||
---
|
||||
author: ungleich
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: no
|
||||
---
|
||||
abstract:
|
||||
In this blog article we describe our way to our first production
|
||||
kubernetes cluster.
|
||||
---
|
||||
body:
|
||||
|
||||
## Introduction
|
||||
|
||||
This article is WIP to describe all steps required for our first
|
||||
production kubernetes cluster and the services that we run in it.
|
||||
|
||||
## Setup
|
||||
|
||||
### Bootstrapping
|
||||
|
||||
* All nodes are running [Alpine Linux](https://alpinelinux.org)
|
||||
* All nodes are configured using [cdist](https://cdi.st)
|
||||
* Mainly installing kubeadm, kubectl, crio *and* docker
|
||||
* At the moment we try to use crio
|
||||
* The cluster is initalised using **kubeadm init --config
|
||||
k8s/c2/kubeadm.yaml** from the [ungleich-k8s repo](https://code.ungleich.ch/ungleich-public/ungleich-k8s)
|
||||
|
||||
### CNI/Networking
|
||||
|
||||
* Calico is installed using **kubectl apply -f
|
||||
cni-calico/calico.yaml** from the [ungleich-k8s
|
||||
repo](https://code.ungleich.ch/ungleich-public/ungleich-k8s)
|
||||
* Installing calicoctl using **kubectl apply -f
|
||||
https://docs.projectcalico.org/manifests/calicoctl.yaml**
|
||||
* Aliasing calicoctl: **alias calicoctl="kubectl exec -i -n kube-system calicoctl -- /calicoctl"**
|
||||
* All nodes BGP peer with our infrastructure using **calicoctl create -f - < cni-calico/bgp-c2.yaml**
|
||||
|
||||
### Persistent Volume Claim support
|
||||
|
||||
* Provided by rook
|
||||
* Using customized manifests to support IPv6 from ungleich-k8s
|
||||
|
||||
```
|
||||
for yaml in crds common operator cluster storageclass-cephfs storageclass-rbd toolbox; do
|
||||
kubectl apply -f ${yaml}.yaml
|
||||
done
|
||||
```
|
||||
|
||||
### Flux
|
||||
|
||||
Starting with the 2nd cluster?
|
||||
|
||||
|
||||
## Follow up
|
||||
|
||||
If you are interesting in continuing the discussion,
|
||||
we are there for you in
|
||||
**the #kubernetes:ungleich.ch Matrix channel**
|
||||
[you can signup here if you don't have an
|
||||
account](https://chat.with.ungleich.ch).
|
||||
|
||||
Or if you are interested in an IPv6 only kubernetes cluster,
|
||||
drop a mail to **support**-at-**ungleich.ch**.
|
201
content/u/blog/kubernetes-without-ingress/contents.lr
Normal file
|
@ -0,0 +1,201 @@
|
|||
title: Building Ingress-less Kubernetes Clusters
|
||||
---
|
||||
pub_date: 2021-06-09
|
||||
---
|
||||
author: ungleich
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
_discoverable: yes
|
||||
---
|
||||
abstract:
|
||||
|
||||
---
|
||||
body:
|
||||
|
||||
## Introduction
|
||||
|
||||
On [our journey to build and define IPv6 only kubernetes
|
||||
clusters](https://www.nico.schottelius.org/blog/k8s-ipv6-only-cluster/)
|
||||
we came accross some principles that seem awkward in the IPv6 only
|
||||
world. Let us today have a look at the *LoadBalancer* and *Ingress*
|
||||
concepts.
|
||||
|
||||
## Ingress
|
||||
|
||||
Let's have a look at the [Ingress
|
||||
definition](https://kubernetes.io/docs/concepts/services-networking/ingress/)
|
||||
definiton from the kubernetes website:
|
||||
|
||||
```
|
||||
Ingress exposes HTTP and HTTPS routes from outside the cluster to
|
||||
services within the cluster. Traffic routing is controlled by rules
|
||||
defined on the Ingress resource.
|
||||
```
|
||||
|
||||
So the ingress basically routes from outside to inside. But, in the
|
||||
IPv6 world, services are already publicly reachable. It just
|
||||
depends on your network policy.
|
||||
|
||||
### Update 2021-06-13: Ingress vs. Service
|
||||
|
||||
As some people pointed out (thanks a lot!), a public service is
|
||||
**not the same** as an Ingress. Ingress has also the possibility to
|
||||
route based on layer 7 information like the path, domain name, etc.
|
||||
|
||||
However, if all of the traffic from an Ingress points to a single
|
||||
IPv6 HTTP/HTTPS Service, effectively the IPv6 service will do the
|
||||
same, with one hop less.
|
||||
|
||||
## Services
|
||||
|
||||
Let's have a look at how services in IPv6 only clusters look like:
|
||||
|
||||
```
|
||||
% kubectl get svc
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
etherpad ClusterIP 2a0a:e5c0:13:e2::a94b <none> 9001/TCP 19h
|
||||
nginx-service ClusterIP 2a0a:e5c0:13:e2::3607 <none> 80/TCP 43h
|
||||
postgres ClusterIP 2a0a:e5c0:13:e2::c9e0 <none> 5432/TCP 19h
|
||||
...
|
||||
```
|
||||
All these services are world reachable, depending on your network
|
||||
policy.
|
||||
|
||||
## ServiceTypes
|
||||
|
||||
While we are at looking at the k8s primitives, let's have a closer
|
||||
look at the **Service**, specifically at 3 of the **ServiceTypes**
|
||||
supported by k8s, including it's definition:
|
||||
|
||||
### ClusterIP
|
||||
|
||||
The k8s website says
|
||||
|
||||
```
|
||||
Exposes the Service on a cluster-internal IP. Choosing this value
|
||||
makes the Service only reachable from within the cluster. This is the
|
||||
default ServiceType.
|
||||
```
|
||||
|
||||
So in the context of IPv6, this sounds wrong. There is nothing that
|
||||
makes an global IPv6 address be "internal", besides possible network
|
||||
policies. The concept is probably coming from the strict difference of
|
||||
RFC1918 space usually used in k8s clusters and not public IPv4.
|
||||
|
||||
This difference does not make a lot of sense in the IPv6 world though.
|
||||
Seeing **services as public by default**, makes much more sense.
|
||||
And simplifies your clusters a lot.
|
||||
|
||||
### NodePort
|
||||
|
||||
Let's first have a look at the definition again:
|
||||
|
||||
```
|
||||
Exposes the Service on each Node's IP at a static port (the
|
||||
NodePort). A ClusterIP Service, to which the NodePort Service routes,
|
||||
is automatically created. You'll be able to contact the NodePort
|
||||
Service, from outside the cluster, by requesting <NodeIP>:<NodePort>.
|
||||
```
|
||||
|
||||
Conceptually this can be similarily utilised in the IPv6 only world
|
||||
like it does in the IPv4 world. However given that there are enough
|
||||
addresses available with IPv6, this might not be such an interesting
|
||||
ServiceType anymore.
|
||||
|
||||
|
||||
### LoadBalancer
|
||||
|
||||
Before we have a look at this type, let's take some steps back
|
||||
first to ...
|
||||
|
||||
|
||||
## ... Load Balancing
|
||||
|
||||
There are a variety of possibilities to do load balancing. From simple
|
||||
round robin, to ECMP based load balancing, to application aware,
|
||||
potentially weighted load balancing.
|
||||
|
||||
So for load balancing, there is usually more than one solution and
|
||||
there is likely not one size fits all.
|
||||
|
||||
So with this said, let.s have a look at the
|
||||
**ServiceType LoadBalancer** definition:
|
||||
|
||||
```
|
||||
Exposes the Service externally using a cloud provider's load
|
||||
balancer. NodePort and ClusterIP Services, to which the external load
|
||||
balancer routes, are automatically created.
|
||||
```
|
||||
|
||||
So whatever the cloud provider offers, can be used, and that is a good
|
||||
thing. However, let's have a look at how you get load balancing for
|
||||
free in IPv6 only clusters:
|
||||
|
||||
## Load Balancing in IPv6 only clusters
|
||||
|
||||
So what is the most easy way of reliable load balancing in network?
|
||||
[ECMP (equal cost multi path)](https://en.wikipedia.org/wiki/Equal-cost_multi-path_routing)
|
||||
comes to the mind right away. Given that
|
||||
kubernetes nodes can BGP peer with the network (upstream or the
|
||||
switches), this basically gives load balancing to the world for free:
|
||||
|
||||
```
|
||||
[ The Internet ]
|
||||
|
|
||||
[ k8s-node-1 ]-----------[ network ]-----------[ k8s-node-n]
|
||||
[ ECMP ]
|
||||
|
|
||||
[ k8s-node-2]
|
||||
|
||||
```
|
||||
|
||||
In the real world on a bird based BGP upstream router
|
||||
this looks as follows:
|
||||
|
||||
```
|
||||
[18:13:02] red.place7:~# birdc show route
|
||||
BIRD 2.0.7 ready.
|
||||
Table master6:
|
||||
...
|
||||
2a0a:e5c0:13:e2::/108 unicast [place7-server1 2021-06-07] * (100) [AS65534i]
|
||||
via 2a0a:e5c0:13:0:225:b3ff:fe20:3554 on eth0
|
||||
unicast [place7-server4 2021-06-08] (100) [AS65534i]
|
||||
via 2a0a:e5c0:13:0:225:b3ff:fe20:3564 on eth0
|
||||
unicast [place7-server2 2021-06-07] (100) [AS65534i]
|
||||
via 2a0a:e5c0:13:0:225:b3ff:fe20:38cc on eth0
|
||||
unicast [place7-server3 2021-06-07] (100) [AS65534i]
|
||||
via 2a0a:e5c0:13:0:224:81ff:fee0:db7a on eth0
|
||||
...
|
||||
```
|
||||
|
||||
Which results into the following kernel route:
|
||||
|
||||
```
|
||||
2a0a:e5c0:13:e2::/108 proto bird metric 32
|
||||
nexthop via 2a0a:e5c0:13:0:224:81ff:fee0:db7a dev eth0 weight 1
|
||||
nexthop via 2a0a:e5c0:13:0:225:b3ff:fe20:3554 dev eth0 weight 1
|
||||
nexthop via 2a0a:e5c0:13:0:225:b3ff:fe20:3564 dev eth0 weight 1
|
||||
nexthop via 2a0a:e5c0:13:0:225:b3ff:fe20:38cc dev eth0 weight 1 pref medium
|
||||
```
|
||||
|
||||
## TL;DR
|
||||
|
||||
We know, a TL;DR at the end is not the right thing to do, but hey, we
|
||||
are at ungleich, aren't we?
|
||||
|
||||
In a nutshell, with IPv6 the concept of **Ingress**,
|
||||
**Service** and the **LoadBalancer** ServiceType
|
||||
types need to be revised, as IPv6 allows direct access without having
|
||||
to jump through hoops.
|
||||
|
||||
If you are interesting in continuing the discussion,
|
||||
we are there for you in
|
||||
**the #hacking:ungleich.ch Matrix channel**
|
||||
[you can signup here if you don't have an
|
||||
account](https://chat.with.ungleich.ch).
|
||||
|
||||
Or if you are interested in an IPv6 only kubernetes cluster,
|
||||
drop a mail to **support**-at-**ungleich.ch**.
|
|
@ -1,18 +0,0 @@
|
|||
title: something i want to talk about
|
||||
---
|
||||
pub_date: 2020-04-11
|
||||
---
|
||||
author: Sanghee Kim
|
||||
---
|
||||
twitter_handle: ungleich
|
||||
---
|
||||
_hidden: yes
|
||||
---
|
||||
_discoverable: no
|
||||
---
|
||||
abstract:
|
||||
this is test post
|
||||
---
|
||||
body:
|
||||
|
||||
This is test post
|
|
@ -1,4 +1,4 @@
|
|||
title: Accessing IPv4 only hosts via IPv4
|
||||
title: Accessing IPv4 only hosts via IPv6
|
||||
---
|
||||
pub_date: 2021-02-28
|
||||
---
|
||||
|
|
|
@ -282,7 +282,8 @@ content4_text:
|
|||
You can use [matterbridge](https://github.com/42wim/matterbridge) for
|
||||
bridging to other networks.
|
||||
Matterbridge can be set up with a one-time fee of **25 CHF per connected
|
||||
protocol** and **5 CHF/month and protocol** maintenance fee.
|
||||
protocol** and **5 CHF/month and protocol** maintenance fee
|
||||
(with the exception of WhatsApp, see below).
|
||||
Matterbridge supports the following protocols:
|
||||
|
||||
* Discord
|
||||
|
@ -297,10 +298,17 @@ Matterbridge supports the following protocols:
|
|||
* Steam
|
||||
* Telegram
|
||||
* Twitch
|
||||
* WhatsApp
|
||||
* XMPP
|
||||
* Zulip
|
||||
|
||||
### WhatsApp Bridge
|
||||
|
||||
As WhatsApp requires additional resources, the pricing for the
|
||||
WhatsApp bridge is
|
||||
|
||||
* 75 CHF one-time setup fee
|
||||
* 15 CHF/month
|
||||
|
||||
## Custom deployment consultancy
|
||||
|
||||
It is possible to integrate your Matrix instance into your other projects and apps. There are a lot of ways to best utilize the security and federation Matrix offers to best fit your usecase. Talk to us about your ideas and we will help you realise how to customize your Matrix. Get in touch with us via support -at- ungleich.ch and you will hear from our team.
|
||||
|
|
|
@ -0,0 +1,113 @@
|
|||
_discoverable: yes
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
title: ungleich infrastructure availability
|
||||
---
|
||||
subtitle: Making your services dependable
|
||||
---
|
||||
image:/u/image/cards/infrastructure-availability.jpg
|
||||
---
|
||||
headline1: ungleich
|
||||
---
|
||||
headline2: Infrastructure
|
||||
---
|
||||
headline3: Availability
|
||||
---
|
||||
header_background_color: #568BD5
|
||||
---
|
||||
header_text_color: text-light
|
||||
---
|
||||
nav_classes: navbar-dark
|
||||
---
|
||||
description1:
|
||||
|
||||
## Infrastructure availability
|
||||
|
||||
This is version **1.0 of the ungleich infrastructure availability definition**.
|
||||
|
||||
Depending on your project or business case you have different
|
||||
availbility requirements. At ungleich we accomodate the range
|
||||
from "can be offline for a while" up to "is mission critical".
|
||||
We orientate ourselves on the It standard availability definitions
|
||||
"availability per year". The following table gives an overview of
|
||||
typical availability rates:
|
||||
|
||||
|
||||
```
|
||||
| % | Accepted downtime / year |
|
||||
|--------+--------------------------|
|
||||
| 98 | 175.2h or 7.3 days |
|
||||
| 99 | 87.6h or 3.65 days |
|
||||
| 99.9 | 8.76h |
|
||||
| 99.99 | 0.876h or 52.55 minutes |
|
||||
| 99.999 | 5.25 minutes |
|
||||
```
|
||||
|
||||
## General availability
|
||||
|
||||
If not otherwise specified, the **target availability** for every
|
||||
service is 99.9%. This availability is **not guaranteed by
|
||||
default**. However we offer **guaranteed availability** for a range of
|
||||
our products.
|
||||
|
||||
## Guaranteed availability
|
||||
|
||||
We can guarantee availability of individual services depending on your
|
||||
needs. As every customer need and product is individual, you can
|
||||
[ask for an individual offer](/u/contact/) for your services.
|
||||
|
||||
## High Availability Methods
|
||||
|
||||
To improve the default availability, we offer a variety of
|
||||
add-ons. Depending on the booked service, different methods are
|
||||
applicable. In this section we describe the general methods we offer.
|
||||
|
||||
### Redundant storage
|
||||
|
||||
By default all data is replicated 3 times accross our decentralised
|
||||
storage system. This protects against disk failures like a traditional
|
||||
RAID system does. Note that this does not protect against
|
||||
unintentional deletion, for this you require a backup.
|
||||
|
||||
### Offsite backup
|
||||
|
||||
To allow disaster recovery, we offer [offsite backups](../backup/)
|
||||
for most of our services. The data is stored in a physical different
|
||||
location so that even physical damage to a data center will allow you
|
||||
to restore your data.
|
||||
|
||||
### Dedicated Internet uplinks and IP blocks
|
||||
|
||||
To protect you against DDoS and impacts on our general infrastructure,
|
||||
we offer dedicated Internet uplinks that are reserved for your
|
||||
service. This includes a /48 IPv6 address block by default for BGP
|
||||
announcement. A /24 IPv4 address block for indidivual BGP announcement
|
||||
can be added at an extra service cost.
|
||||
|
||||
### Redundant services / High availability (HA)
|
||||
|
||||
We realise high availability (HA) using redundant services. These
|
||||
services can be placed in the same geographic or different geographic
|
||||
data centers. Depending on the service architecture we offer
|
||||
Cold-Standby, Hot-Standby or active-active services.
|
||||
|
||||
With different geographic locations separate data storage is included
|
||||
by default. For services in the same location, an optional separate
|
||||
data storage is also available.
|
||||
|
||||
### Load Balancing
|
||||
|
||||
For services that allow multiple backends, we offer load balancing
|
||||
services to distribute the traffic. This can be combined with
|
||||
dedicated uplinks per location, indivual IP address blocks and
|
||||
distributed application deployments in multiple locations. You can
|
||||
also easily realise [blue green
|
||||
deployments](https://en.wikipedia.org/wiki/Blue-green_deployment) or
|
||||
[Canary releases](https://martinfowler.com/bliki/CanaryRelease.html).
|
||||
|
||||
## Related pages
|
||||
|
||||
* [ungleich SLAs](../ungleich-sla/)
|
||||
* [ungleich Service Hours](../ungleich-service-hour/)
|
||||
* [ungleich Support Packages](../ungleich-support-package/)
|
135
content/u/products/ungleich-service-hour/contents.lr
Normal file
|
@ -0,0 +1,135 @@
|
|||
_discoverable: yes
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
title: ungleich service hours
|
||||
---
|
||||
subtitle: Let us solve your challenges
|
||||
---
|
||||
image:/u/image/cards/service-hours.jpg
|
||||
---
|
||||
headline1: ungleich
|
||||
---
|
||||
headline2: Service
|
||||
---
|
||||
headline3: Hours
|
||||
---
|
||||
header_background_color: #5996CE
|
||||
---
|
||||
header_text_color: text-light
|
||||
---
|
||||
nav_classes: navbar-dark
|
||||
---
|
||||
description1:
|
||||
|
||||
You can hire ungleich staff for solving your infrastructure,
|
||||
software development and project management tasks. Below table
|
||||
summarises our standard rates. For projects exceeding the specified
|
||||
standard packages, [reach out to us individually](/u/contact/).
|
||||
|
||||
|
||||
<div class="table-responsive mt-4">
|
||||
<table class="table colored-table table-bordered">
|
||||
<tr>
|
||||
<th>Package/Feature</th>
|
||||
<th>Default rate</th>
|
||||
<th>Support package</th>
|
||||
<th>Starter package</th>
|
||||
<th>Collaboration package</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Hourly Rate (during business hours)</th>
|
||||
<td>220 CHF/h</td>
|
||||
<td>210 CHF/h</td>
|
||||
<td>200 CHF/h</td>
|
||||
<td>180 CHF/h</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Hours included during business hours</th>
|
||||
<td>0</td>
|
||||
<td>2</td>
|
||||
<td>10</td>
|
||||
<td>20</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Hourly Rate (outside of business hours)</th>
|
||||
<td>360 CHF/h</td>
|
||||
<td>343 CHF/h</td>
|
||||
<td>327 CHF/h</td>
|
||||
<td>295 CHF/h</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Hours included (outside of business hours)</th>
|
||||
<td>0</td>
|
||||
<td>0</td>
|
||||
<td>2</td>
|
||||
<td>5</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Package valid for</th>
|
||||
<td>on demand</td>
|
||||
<td>30 days</td>
|
||||
<td>30 days</td>
|
||||
<td>60 days</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Discount</th>
|
||||
<td>0%</td>
|
||||
<td>4.5%</td>
|
||||
<td>9%</td>
|
||||
<td>18%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Pricing</th>
|
||||
<td>FREE</td>
|
||||
<td>500</td>
|
||||
<td>3'000</td>
|
||||
<td>5'500</td>
|
||||
</tr>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
* Business hours are Mo-Fr, 9-17, excluding Swiss national holidays
|
||||
* Hours not consumed during the given timeframe are expired.
|
||||
* All packages are non-refundable
|
||||
* Discounts available for Open Source and climate related projects
|
||||
* For long term and bigger projects, ask for an individual quote.
|
||||
|
||||
## Related pages
|
||||
|
||||
* [ungleich Infrastructure Availability](../ungleich-infrastructure-availability)
|
||||
* [ungleich SLAs](../ungleich-sla)
|
||||
* [ungleich Support Packages](../ungleich-support-package)
|
||||
|
||||
---
|
||||
offer1_title: Support Package 500 CHF
|
||||
---
|
||||
offer1_text:
|
||||
* 2 hours included during business hours
|
||||
* Further hours during business time: 210 CHF/h
|
||||
* 4.5% discount compared to default hourly rate
|
||||
* Valid for 30 days
|
||||
---
|
||||
offer1_link: https://ungleich.ch/product/support-package-500
|
||||
---
|
||||
offer2_title: Starter Package 3'000 CHF
|
||||
---
|
||||
offer2_text:
|
||||
* 10 hours included during business hours
|
||||
* 2 hours included outside business hours
|
||||
* Further hours during business time: 210 CHF/h
|
||||
* 9% discount compared to default hourly rate
|
||||
* Valid for 30 days
|
||||
---
|
||||
offer2_link: https://ungleich.ch/product/starter-package-3000
|
||||
---
|
||||
offer3_title: Collaboration Package 5'500 CHF
|
||||
---
|
||||
offer3_text:
|
||||
* 20 hours included during business hours
|
||||
* 5 hours included outside business hours
|
||||
* Further hours during business time: 180 CHF/h
|
||||
* 18% discount compared to default hourly rate
|
||||
* Valid for 60 days
|
||||
---
|
||||
offer3_link: https://ungleich.ch/product/collaboration-package-5500
|
273
content/u/products/ungleich-sla/contents.lr
Normal file
|
@ -0,0 +1,273 @@
|
|||
_discoverable: yes
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
title: ungleich SLA levels
|
||||
---
|
||||
subtitle: ungleich service level agreements
|
||||
---
|
||||
image:/u/image/cards/sla-levels.jpg
|
||||
---
|
||||
headline1: Service
|
||||
---
|
||||
headline2: Level
|
||||
---
|
||||
headline3: Agreements
|
||||
---
|
||||
header_background_color: #61C1C0
|
||||
---
|
||||
header_text_color: text-light
|
||||
---
|
||||
nav_classes: navbar-dark
|
||||
---
|
||||
description1:
|
||||
|
||||
Our Service Level Agreements (SLAs) define the reachability as well
|
||||
as the reaction times. As every customer situation is unique, we allow
|
||||
customization of the service level per customer.
|
||||
|
||||
In this document you can find what the default
|
||||
SLA covers and which additional options you have.
|
||||
|
||||
This is **version 1.0** of the ungleich SLA levels.
|
||||
|
||||
## The standard SLA
|
||||
|
||||
If not otherwise specified in the product or service you acquired from
|
||||
us, the standard SLA will apply. This SLA covers standard operations
|
||||
and is suitable for non-critical deployments. The standard SLA covers:
|
||||
|
||||
* Best effort reaction times
|
||||
* Support via support@ungleich.ch (answered 09:00-17:00 on work days)
|
||||
* Best effort telephone support
|
||||
* Included in all products
|
||||
|
||||
### Overview of the SLA levels
|
||||
|
||||
<div class="table-responsive mt-4">
|
||||
<table class="table colored-table table-bordered">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Feature / SLA</th>
|
||||
<th>Standard SLA</th>
|
||||
<th>Business SLA</th>
|
||||
<th>Professional SLA</th>
|
||||
<th>Professional Plus SLA</th>
|
||||
<th>Critical Services SLA</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<th scope="row">Support via E-Mail</th>
|
||||
<td>yes</td>
|
||||
<td>yes</td>
|
||||
<td>yes</td>
|
||||
<td>yes</td>
|
||||
<td>yes</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Guaranteed response time via E-Mail</th>
|
||||
<td>best effort</td>
|
||||
<td><a href="#1-business-day">1 business day</a></td>
|
||||
<td><a href="#4h-business-hours">4h during business hours</a></td>
|
||||
<td><a href="#4h-extended-business-hours">4h during extended business hours</a></td>
|
||||
<td><a href="#4h-24x7">4h every day</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Support via telephone</th>
|
||||
<td>best effort</td>
|
||||
<td>best effort</td>
|
||||
<td>Mo-Fr 09:00-17:00</td>
|
||||
<td>Mo-Fr 06:00-22:00</td>
|
||||
<td>Mo-Su 00:00-23:59</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Pricing (CHF/month)</th>
|
||||
<td>Included for free</td>
|
||||
<td>450</td>
|
||||
<td>900</td>
|
||||
<td>1'800</td>
|
||||
<td>4'200</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
All prices are excluding the [service hours](../ungleich-service-hour).
|
||||
|
||||
---
|
||||
offer1_title: Business SLA: 450 CHF/month
|
||||
---
|
||||
offer1_text:
|
||||
* Guaranteed response time within 1 business day
|
||||
* Efficient support handling with support history
|
||||
* Suitable for SME and smaller projects
|
||||
---
|
||||
offer1_link: https://ungleich.ch/product/business-sla/
|
||||
---
|
||||
offer2_title: Professional SLA: 900 CHF/month
|
||||
---
|
||||
offer2_text:
|
||||
* Guaranteed response time 4h during business hours
|
||||
* Guaranteed reachability via telephone during business hours
|
||||
* Efficient support handling with support history
|
||||
* Suitable when fast response during business day is important
|
||||
---
|
||||
offer2_link: https://ungleich.ch/product/professional-sla/
|
||||
---
|
||||
offer3_title: Professional Plus SLA: 1'800 CHF/month
|
||||
---
|
||||
offer3_text:
|
||||
* Guaranteed response time 4h during **extended** business hours
|
||||
* Covering most time of the day (16h)
|
||||
* Guaranteed reachability via telephone during **extended** business hours
|
||||
* Efficient support handling with support history
|
||||
* Suitable when fast response during even outside business hours is required
|
||||
---
|
||||
offer3_link: https://ungleich.ch/product/professional-plus-sla/
|
||||
---
|
||||
offer4_title: Critical Services SLA: 4'200 CHF/month
|
||||
---
|
||||
offer4_text:
|
||||
* Guaranteed response time 4h during **every day** (including national holidays)
|
||||
* Covering the complete day (24h)
|
||||
* Guaranteed reachability at any time
|
||||
* Efficient support handling with support history
|
||||
* Suitable for mission critical services
|
||||
---
|
||||
offer4_link: https://ungleich.ch/product/critical-services-sla/
|
||||
---
|
||||
description6:
|
||||
|
||||
## Services and reaction times in detail
|
||||
|
||||
### Business hours
|
||||
|
||||
Our reguar business hours are from 09:00-17:00, Monday to Friday, with the
|
||||
exception of Swiss national holidays.
|
||||
|
||||
### Extended Business hours
|
||||
|
||||
The extended business hours are from 06:00-22:00, Monday to Friday, with the
|
||||
exception of Swiss national holidays.
|
||||
|
||||
### <a name="1-business-day"></a> 1 business day reaction times
|
||||
|
||||
The request will be answered within the next business day. The table
|
||||
below shows the details for every time of the week.
|
||||
|
||||
<div class="table-responsive mt-4">
|
||||
<table class="table colored-table table-bordered">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Request received</th>
|
||||
<th>Answer guaranteed until</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Between Monday 09:00-17:00</th>
|
||||
<td>Tuesday 17:00</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Between Tuesday 09:00-17:00</th>
|
||||
<td>Wednesday 17:00</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Between Wednesday 09:00-17:00</th>
|
||||
<td>Thursday 17:00</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Between Thursday 09:00-17:00</th>
|
||||
<td>Friday 17:00</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Between Friday 17:00 and Monday 09:00</th>
|
||||
<td>Monday 17:00</th>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
If a work day is a Swiss national holiday it is treated as Friday to
|
||||
Monday period.
|
||||
|
||||
### <a name="4h-business-hours"></a> 4h during business hours reaction times
|
||||
|
||||
The request will be answered within the 4h during business hours. The
|
||||
table below shows the details:
|
||||
<div class="table-responsive mt-4">
|
||||
<table class="table colored-table table-bordered">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Request received</th>
|
||||
<th>Answer guaranteed until</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Between 09:00-13:00 on a work day</td>
|
||||
<td>Between 13:00-17:00 on the same work day</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Between 13:00-17:00 on a work day</td>
|
||||
<td>Between 09:00-13:00 on the next work day</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
### <a name="4h-extended-business-hours"></a> 4h during extended business hours reaction times
|
||||
|
||||
The request will be answered within the 4h during business hours. The
|
||||
table below shows the details:
|
||||
|
||||
<div class="table-responsive mt-4">
|
||||
<table class="table colored-table table-bordered">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Request received</th>
|
||||
<th>Answer guaranteed until</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Between 09:00-13:00 on a work day</th>
|
||||
<td>Between 13:00-17:00 on the same work day</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Between 13:00-17:00 on a work day</th>
|
||||
<td>Between 09:00-13:00 on the next work day</th>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
### <a name="4h-24x7"></a> Every day 4h response time reaction times
|
||||
|
||||
The request will be answered within the 4h independent of the day or
|
||||
time.
|
||||
<div class="table-responsive mt-4">
|
||||
<table class="table colored-table table-bordered">
|
||||
<tr>
|
||||
<th>Request received</th>
|
||||
<th>Answer guaranteed until</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Between 00:00-23:59 every day</th>
|
||||
<td>Within 4 hours</th>
|
||||
</tr>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
### Business language
|
||||
|
||||
As our team is very international, our primary business language is
|
||||
English, secondary language is German. On request, we are also
|
||||
available in Korean and French.
|
||||
|
||||
## Related pages
|
||||
|
||||
* [ungleich Infrastructure Availability](../ungleich-infrastructure-availability)
|
||||
* [ungleich Service Hours](../ungleich-service-hour)
|
||||
* [ungleich Support Packages](../ungleich-support-package)
|
150
content/u/products/ungleich-support-package/contents.lr
Normal file
|
@ -0,0 +1,150 @@
|
|||
_discoverable: yes
|
||||
---
|
||||
_hidden: no
|
||||
---
|
||||
title: ungleich support packages
|
||||
---
|
||||
subtitle: Let us make your life easy
|
||||
---
|
||||
image:/u/image/cards/support-packages.jpg
|
||||
---
|
||||
headline1: ungleich
|
||||
---
|
||||
headline2: support
|
||||
---
|
||||
headline3: package
|
||||
---
|
||||
header_background_color: #79C161
|
||||
---
|
||||
header_text_color: text-light
|
||||
---
|
||||
nav_classes: navbar-dark
|
||||
---
|
||||
description1:
|
||||
|
||||
Every project and every business has invdiual needs in regards to
|
||||
support reachablity, infrastructure uptime and development and
|
||||
consultancy efforts.
|
||||
|
||||
We allow every project to individually select their
|
||||
SLA, consultancy packages and availability requirements. Below we have
|
||||
summarised typical selections from our customers for easy selection.
|
||||
|
||||
|
||||
<div class="table-responsive mt-4">
|
||||
<table class="table colored-table table-bordered">
|
||||
<tr>
|
||||
<th>Package/Feature</th>
|
||||
<th>Business Light</th>
|
||||
<th>Business Standard</th>
|
||||
<th>Business Pro</th>
|
||||
<th>Business Pro Plus</th>
|
||||
<th>Critical Services</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Working hour package</th>
|
||||
<td>Support package</td>
|
||||
<td>Starter package</td>
|
||||
<td>Starter package</td>
|
||||
<td>Collaboration package</td>
|
||||
<td>Collaboration package</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Working hours included (during business hours/outside
|
||||
business hours)</th>
|
||||
<td>2 / 0</td>
|
||||
<td>10 / 2</td>
|
||||
<td>10 / 2</td>
|
||||
<td>20 / 5</td>
|
||||
<td>20 / 5</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>SLA package</th>
|
||||
<td>Business SLA</td>
|
||||
<td>Business SLA</td>
|
||||
<td>Professional SLA</td>
|
||||
<td>Professional Plus SLA</td>
|
||||
<td>Critical Services SLA</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Reaction times</th>
|
||||
<td>1 business day</td>
|
||||
<td>1 business day</td>
|
||||
<td>4h during business hours</td>
|
||||
<td>4h during extended business hours</td>
|
||||
<td>4h every day</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><a href="#availiblity">Targetted Minimum Availability</a></th>
|
||||
<td>99%</td>
|
||||
<td>99%</td>
|
||||
<td>99.9%</td>
|
||||
<td>99.99%</td>
|
||||
<td>99.999%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Pricing (CHF/month)</th>
|
||||
<td>950</td>
|
||||
<td>3'450</td>
|
||||
<td>3'900</td>
|
||||
<td>7'300</td>
|
||||
<td>9'700</td>
|
||||
</tr>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
## <a name="availiblity"></a> Targetted Minimum Availability
|
||||
|
||||
The availability of our services is defined individually depending on
|
||||
the services booked. Find more information about it on the
|
||||
[ungleich infrastructure
|
||||
availability](../ungleich-infrastructure-availability) page.
|
||||
|
||||
---
|
||||
offer1_title: Business Light: 950 CHF/month
|
||||
---
|
||||
offer1_text:
|
||||
* Contains service hours Support package
|
||||
* Contains Business SLA
|
||||
---
|
||||
offer1_link: https://ungleich.ch/product/support-business-light
|
||||
---
|
||||
offer2_title: Business Standard: 3'450 CHF/month
|
||||
---
|
||||
offer2_text:
|
||||
* Contains service hours Starter package
|
||||
* Contains Business SLA
|
||||
---
|
||||
offer2_link: https://ungleich.ch/product/support-business-standard
|
||||
---
|
||||
offer3_title: Business Pro: 3'900 CHF/month
|
||||
---
|
||||
offer3_text:
|
||||
* Contains service hours Starter package
|
||||
* Contains Professional SLA
|
||||
---
|
||||
offer3_link: https://ungleich.ch/product/support-business-pro
|
||||
---
|
||||
offer4_title: Business Pro Plus: 7'300 CHF/month
|
||||
---
|
||||
offer4_text:
|
||||
* Contains service hours Collaboration package
|
||||
* Contains Professional Plus SLA
|
||||
---
|
||||
offer4_link: https://ungleich.ch/product/support-business-pro-plus
|
||||
---
|
||||
offer5_title: Critical Services: 9'700 CHF/month
|
||||
---
|
||||
offer5_text:
|
||||
* Contains service hours Collaboration package
|
||||
* Contains Critical Services SLA
|
||||
---
|
||||
offer5_link: https://ungleich.ch/product/support-critical-services
|
||||
---
|
||||
description6:
|
||||
|
||||
## Related pages
|
||||
|
||||
* [ungleich Infrastructure Availability](../ungleich-infrastructure-availability/)
|
||||
* [ungleich SLAs](../ungleich-sla/)
|
||||
* [ungleich Service Hours](../ungleich-service-hour/)
|
|
@ -140,6 +140,10 @@ Let us know if you want tracked or untracked shipping.
|
|||
|
||||
### Price
|
||||
|
||||
**Update 2021-09-14: we are sold out!**
|
||||
|
||||
Thanks to your love, we ran out of our stock of VIGIR. You can preorder now and we will start shipping our next batch of VIGIR from 2021-10-21 again.
|
||||
|
||||
**The price of the VIGIR is 250 CHF**, including the LTE modem,
|
||||
6 antennas and the power supply.
|
||||
|
||||
|
|
|
@ -270,6 +270,10 @@ for **145 CHF** (203 CHF including the VIIRB).
|
|||
|
||||
### Price
|
||||
|
||||
**Update 2021-09-14: we are sold out!**
|
||||
|
||||
Thanks to your love, we ran out of our stock of VIIRB. You can preorder now and we will start shipping our next batch of VIIRB from 2021-10-21 again.
|
||||
|
||||
**The price of the VIIRB is 58 CHF**.
|
||||
|
||||
If you need a power supply, we can ship it with a
|
||||
|
|
|
@ -91,14 +91,13 @@ OpenWrt and is pre-configured with the [IPv6VPN](https://ipv6vpn.ch).
|
|||
A free subscription for 1 year is included. This way you can plug in
|
||||
the VIWIB and just get started with IPv6.
|
||||
|
||||
## Ordering
|
||||
|
||||
The preorder is open now. To order the VIWIB, send an email with your shipping address and
|
||||
the requested quantity to **support -at- ungleich.ch**. Let us know if you want tracked or untracked shipping.
|
||||
The shipment is planned to begin in mid December 2020.
|
||||
|
||||
### Price
|
||||
|
||||
**Update 2021-09-14: we are sold out!**
|
||||
|
||||
Thanks to your love, we ran out of our stock of VIWIB. You can preorder now and we will start shipping our next batch of VIWIB from 2021-10-21 again.
|
||||
|
||||
**The price of the VIWIB is 68 CHF**.
|
||||
|
||||
If you need a power supply, we can ship it with a
|
||||
|
@ -109,8 +108,23 @@ All prices exclude VAT and shipping costs.
|
|||
|
||||
### Shipping costs
|
||||
|
||||
You can order up to 4 VIWIBs within one order. In other words:
|
||||
only 1 time shipping cost for 1,2,3 or 4 VIWIBs.
|
||||
We ship world wide. If your country of residence is not in the list below, get in touch with our support and we will let you know the shipping cost. You can order multiple VIIRBs within the same package.
|
||||
|
||||
|
||||
| Country | Economy (no tracking) | Priority (with tracking) |
|
||||
|---------------------------------|--------------------------------|------------------------------|
|
||||
| Switzerland, Liechtenstein | 10 CHF | 14 CHF|
|
||||
| Denmark, Finland, Sweden | 14 CHF | 24 CHF|
|
||||
| Germany, Austria, France | 14 CHF | 24 CHF|
|
||||
| Portugal, Italy, Spain | 14 CHF | 24 CHF|
|
||||
| Latvia, The Netherlands, Iceland | 14 CHF | 24 CHF|
|
||||
| Romania, Great Britain, Czech Republic | 14 CHF | 24 CHF|
|
||||
| South Korea | 16 CHF | 29 CHF|
|
||||
| Canada, US, India, Australia | 23 CHF | 29 CHF|
|
||||
|
||||
You can order with a credit card via following links. Payment via wire transfer is also available, get in touch with our support with your shipping address.
|
||||
|
||||
## Ordering
|
||||
|
||||
* [Order to Denmark, Finland, France, Germany, Great Britain, Iceland,
|
||||
The Netherlands,
|
||||
|
@ -120,8 +134,8 @@ only 1 time shipping cost for 1,2,3 or 4 VIWIBs.
|
|||
Romania, Spain, Sweden, Italy, Czech Republic](https://datacenterlight.ch/product/viwib-2-eu/)
|
||||
* [Order to Switzerland: +10 CHF](https://datacenterlight.ch/product/viwib-ch/)
|
||||
* [Order 2 VIWIBs to Switzerland](https://datacenterlight.ch/product/viwib-2-ch/)
|
||||
* [Order to Australia, Canada, India, South Korea, US: +16 CHF](https://datacenterlight.ch/product/viwib-us/)
|
||||
* [Order 2 VIWIBs to Australia, Canada, India, South Korea, US](https://datacenterlight.ch/product/viwib-2-us/)
|
||||
* [Order to Australia, Canada, India, US: +23 CHF](https://datacenterlight.ch/product/viwib-us/)
|
||||
* [Order 2 VIWIBs to Australia, Canada, India, US](https://datacenterlight.ch/product/viwib-2-us/)
|
||||
|
||||
|
||||
The following countries can get the VIWIB with priority shipping with tracking. If you want to receive VIIRB with tracked shipping, get in touch with our team with your shipping address.
|
||||
|
|
|
@ -76,7 +76,7 @@ offer1_text:
|
|||
* **One free IPv6 VPN** included for increased security
|
||||
* Datacenter location: Glarus, Switzerland
|
||||
* Enhanced security by limiting access to only your devices
|
||||
* Suitable for 1-5 ppeople with no additioal appps
|
||||
* Suitable for 1-5 people with no additioal appps
|
||||
* The cloud will run on a virtual machine with 1 Core, 2 GB RAM, 10 GB SSD, 100 GB HDD
|
||||
* [1 time initial setup fee 35 CHF](https://ungleich.ch/product/0carboncloud-setup/)
|
||||
|
||||
|
@ -92,11 +92,11 @@ offer2_text:
|
|||
* **One free IPv6 VPN** included for increased security
|
||||
* Datacenter location: Glarus, Switzerland
|
||||
* Enhanced security by limiting access to only your devices
|
||||
* Suitable for 1-5 ppeople with no additioal appps
|
||||
* Suitable for 1-5 people with no additioal appps
|
||||
* The cloud will run on a virtual machine with 1 Core, 2 GB RAM, 10 GB SSD, 500 GB HDD
|
||||
* [1 time initial setup fee 35 CHF](https://ungleich.ch/product/0carboncloud-setup/)
|
||||
|
||||
Recommended for your private use or for a smaller project.
|
||||
Recommended for your private use or for a smaller project.
|
||||
|
||||
---
|
||||
offer2_link: https://ungleich.ch/product/0carboncloud-s-500GB/
|
||||
|
@ -108,7 +108,7 @@ offer3_text:
|
|||
* **One free IPv6 VPNs** included for increased security
|
||||
* Datacenter location: Glarus, Switzerland
|
||||
* Enhanced security by limiting access to only your devices
|
||||
* Suitable for 5-10 ppeople with 1-2 enable appps
|
||||
* Suitable for 5-10 people with 1-2 enable appps
|
||||
* The cloud will run on a virtual machine with 2 Core, 4 GB RAM, 10 GB SSD, 500 GB HDD
|
||||
* [1 time initial setup fee 35 CHF](https://ungleich.ch/product/0carboncloud-setup/)
|
||||
|
||||
|
@ -124,7 +124,7 @@ offer4_text:
|
|||
* **Two free IPv6 VPNs** included for increased security
|
||||
* Datacenter location: Glarus, Switzerland
|
||||
* Enhanced security by limiting access to only your devices
|
||||
* Suitable for 5-10 ppeople with 1-2 enable appps
|
||||
* Suitable for 5-10 people with 1-2 enable appps
|
||||
* The cloud will run on a virtual machine with 2 Core, 4 GB RAM, 10 GB SSD, 5 TB HDD
|
||||
* [1 time initial setup fee 35 CHF](https://ungleich.ch/product/0carboncloud-setup/)
|
||||
|
||||
|
@ -137,10 +137,10 @@ offer5_title: Cloud L-1 TB @ 76 CHF/month
|
|||
---
|
||||
offer5_text:
|
||||
|
||||
* **One free IPv6 VPNs** included for increased security
|
||||
* **Two free IPv6 VPNs** included for increased security
|
||||
* Datacenter location: Glarus, Switzerland
|
||||
* Enhanced security by limiting access to only your devices
|
||||
* Suitable for 10-20 ppeople with 3-4 enable appps
|
||||
* Suitable for 10-20 people with 3-4 enable appps
|
||||
* The cloud will run on a virtual machine with 4 Core, 8 GB RAM, 10 GB SSD, 1 TB HDD
|
||||
* [1 time initial setup fee 35 CHF](https://ungleich.ch/product/0carboncloud-setup/)
|
||||
|
||||
|
@ -153,10 +153,10 @@ offer6_title: Cloud L-10 TB @ 256 CHF/month
|
|||
---
|
||||
offer6_text:
|
||||
|
||||
* **Three free IPv6 VPNs** included for increased security
|
||||
* **Five free IPv6 VPNs** included for increased security
|
||||
* Datacenter location: Glarus, Switzerland
|
||||
* Enhanced security by limiting access to only your devices
|
||||
* Suitable for 10-20 ppeople with 3-4 enable appps
|
||||
* Suitable for 10-20 people with 3-4 enable appps
|
||||
* The cloud will run on a virtual machine with 4 Core, 8 GB RAM, 10 GB SSD, 10 TB HDD
|
||||
* [1 time initial setup fee 35 CHF](https://ungleich.ch/product/0carboncloud-setup/)
|
||||
|
||||
|
|
|
@ -58,6 +58,15 @@ Checkout the [SBB
|
|||
page](https://www.sbb.ch/de/kaufen/pages/fahrplan/fahrplan.xhtml?von=Zurich&nach=Diesbach-Betschwanden)
|
||||
for the next train.
|
||||
|
||||
The address is:
|
||||
|
||||
```
|
||||
Hacking Villa
|
||||
Hauptstrasse 28
|
||||
8777 Diesbach
|
||||
Switzerland
|
||||
```
|
||||
|
||||
---
|
||||
content1_image: hacking-villa-diesbach.jpg
|
||||
---
|
||||
|
|
|
@ -45,6 +45,16 @@ Specifically for learning new technologies and to exchange knowledge
|
|||
we created the **Hacking & Learning channel** which can be found at
|
||||
**#hacking-and-learning:ungleich.ch**.
|
||||
|
||||
## Kubernetes
|
||||
|
||||
Recently (in 2021) we started to run Kubernetes cluster at
|
||||
ungleich. We share our experiences in **#kubernetes:ungleich.ch**.
|
||||
|
||||
## Ceph
|
||||
|
||||
To exchange experiences and trouble shooting for ceph, we are running
|
||||
**#ceph:ungleich.ch**.
|
||||
|
||||
## cdist
|
||||
|
||||
We meet for cdist discussions about using, developing and more
|
||||
|
@ -57,7 +67,7 @@ We discuss topics related to sustainability in
|
|||
|
||||
## More channels
|
||||
|
||||
* The main / hangout channel is **o#town-square:ungleich.ch** (also bridged
|
||||
* The main / hangout channel is **#town-square:ungleich.ch** (also bridged
|
||||
to Freenode IRC as #ungleich and
|
||||
[discord](https://discord.com/channels/706144469925363773/706144469925363776))
|
||||
* The bi-yearly hackathon Hack4Glarus can be found in
|
||||
|
|
41
content/u/projects/privacy-policy/contents.lr
Normal file
|
@ -0,0 +1,41 @@
|
|||
title: Privacy Policy
|
||||
---
|
||||
subtitle: ungleich's policy on your privacy
|
||||
---
|
||||
description1:
|
||||
|
||||
## Introduction
|
||||
|
||||
This is version 0.1 of our privacy policy from 2021-10-04.
|
||||
|
||||
## Privacy by default
|
||||
|
||||
At ungleich we are strong believers of **privacy by default**. That
|
||||
means: you don't need to opt-in for privacy and you don't need to
|
||||
opt-out for newsletters or marketing information. Privacy is a big
|
||||
concern for us and our customers.
|
||||
|
||||
## Logging and data submission
|
||||
|
||||
By default all our services are configured to a minimum amount of
|
||||
logging. We cannot claim a **zero log policy**, because for
|
||||
operational measures (spammers, denial of service attacks, for
|
||||
billing) we need to log some data.
|
||||
|
||||
We however **do not sell your data**. Our business is providing
|
||||
services, not making money of your information.
|
||||
|
||||
## Third party access
|
||||
|
||||
We minimise the amount of data that is seen by third parties. At the
|
||||
moment some of our websites use google analytics (for historic
|
||||
reasons). We plan to remove this by the beginning of 2022.
|
||||
|
||||
Services like our [data storage](/u/products/data-storage/),
|
||||
[the hosted matrix chat](/u/products/hosted-matrix-chat/),
|
||||
[zero carbon VPS hosting](/u/products/virtual-machine-hosting/),
|
||||
[zero carbon chat](/u/products/zero-carbon-chat/) and
|
||||
[zero carbon cloud](/u/products/zero-carbon-cloud/) do not send any
|
||||
data to third parties by default. There might be plugins or settings
|
||||
that allow you to enable communication with third parties, but we do
|
||||
not configure them by default. Above list is not exhaustive.
|
|
@ -1,4 +0,0 @@
|
|||
The structure of templates should be:
|
||||
|
||||
- layout.html: valid for everything
|
||||
- product.html: valid for all products
|
10
templates/README.md
Normal file
|
@ -0,0 +1,10 @@
|
|||
Installation
|
||||
|
||||
```
|
||||
$ pip install lektor
|
||||
```
|
||||
|
||||
The structure of templates should be:
|
||||
|
||||
- layout.html: valid for everything
|
||||
- product.html: valid for all products
|
|
@ -98,9 +98,9 @@
|
|||
</ul>
|
||||
<form class="form-inline my-2 my-lg-0"
|
||||
action="https://search.ungleich.ch/yacysearch.html" method="get">
|
||||
<input class="form-control mr-sm-2" type="search"
|
||||
<input class="form-control p-2 mr-sm-2" type="search"
|
||||
placeholder="Search" name="query" aria-label="Search">
|
||||
<button class="btn btn-outline-success my-2 my-sm-0" type="submit">Search</button>
|
||||
<button class="btn btn-outline-success my-2 my-sm-0 btn-sm" type="submit">Search</button>
|
||||
</form>
|
||||
</div>
|
||||
</nav>
|
||||
|
|
|
@ -4,39 +4,50 @@
|
|||
|
||||
{% block content %}
|
||||
{% if this.description1 %}
|
||||
<div class="container">
|
||||
<div class="row mb-3">
|
||||
<div class="col-md-12">
|
||||
{{ this.description1 }}
|
||||
<section class="section">
|
||||
<div class="container">
|
||||
<div class="row mb-3">
|
||||
<div class="col-md-12">
|
||||
{{ this.description1 }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
{% endif %}
|
||||
<div class="container">
|
||||
<div class="row mb-5">
|
||||
<div class="col-md-4">
|
||||
<h3 class="font-weight-normal">{{ this.feature1_title }}</h3>
|
||||
<p class="font-weight-normal">{{ this.feature1_text }}</p>
|
||||
</div>
|
||||
<div class="col-md-4">
|
||||
<h3 class="font-weight-normal">{{ this.feature2_title }}</h3>
|
||||
<p class="font-weight-normal">{{ this.feature2_text }}</p>
|
||||
</div>
|
||||
<div class="col-md-4">
|
||||
<h3 class="font-weight-normal">{{ this.feature3_title }}</h3>
|
||||
<p class="font-weight-normal">{{ this.feature3_text }}</p>
|
||||
<section class="section">
|
||||
<div class="container">
|
||||
<div class="row">
|
||||
<div class="col-sm-4 col-lg-4 mb-5 mb-lg-0">
|
||||
<div class="featured-box">
|
||||
<h3>{{ this.feature1_title }}</h3>
|
||||
<p class="text-3">{{ this.feature1_text }}</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-sm-4 col-lg-4 mb-5 mb-lg-0">
|
||||
<div class="featured-box">
|
||||
<h3>{{ this.feature2_title }}</h3>
|
||||
<p class="text-3">{{ this.feature2_text }}</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-sm-4 col-lg-4 mb-5 mb-sm-0">
|
||||
<div class="featured-box">
|
||||
<h3>{{ this.feature3_title }}</h3>
|
||||
<p class="text-3">{{ this.feature3_text }}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{% if this.description2 %}
|
||||
<div class="container">
|
||||
<div class="row mb-3">
|
||||
<div class="col-md-12">
|
||||
{{ this.description2 }}
|
||||
<section class="section">
|
||||
<div class="container">
|
||||
<div class="row mb-3">
|
||||
<div class="col-md-12">
|
||||
{{ this.description2 }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
{% endif %}
|
||||
|
||||
{% if this.content1_text %}
|
||||
|
@ -64,14 +75,14 @@
|
|||
</div>
|
||||
{% endif %}
|
||||
<div class="container">
|
||||
<div class="row mb-3">
|
||||
<div class="row">
|
||||
<div class="col-md-4">
|
||||
{% if this.offer1_title and this.offer1_text and this.offer1_link %}
|
||||
<div class="card">
|
||||
<div class="card-body">
|
||||
<h5 class="card-title">{{ this.offer1_title }}</h5>
|
||||
<p class="card-text">{{ this.offer1_text }}</p>
|
||||
<a href="{{ this.offer1_link }}" class="btn btn-primary">Order</a>
|
||||
<a href="{{ this.offer1_link }}" class="btn btn-primary btn-sm">Order</a>
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
@ -82,7 +93,7 @@
|
|||
<div class="card-body">
|
||||
<h5 class="card-title">{{ this.offer2_title }}</h5>
|
||||
<p class="card-text">{{ this.offer2_text }}</p>
|
||||
<a href="{{ this.offer2_link }}" class="btn btn-primary">Order</a>
|
||||
<a href="{{ this.offer2_link }}" class="btn btn-primary btn-sm">Order</a>
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
@ -93,22 +104,25 @@
|
|||
<div class="card-body">
|
||||
<h5 class="card-title">{{ this.offer3_title }}</h5>
|
||||
<p class="card-text">{{ this.offer3_text }}</p>
|
||||
<a href="{{ this.offer3_link }}" class="btn btn-primary">Order</a>
|
||||
<a href="{{ this.offer3_link }}" class="btn btn-primary btn-sm">Order</a>
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<hr/>
|
||||
<hr/>
|
||||
</div>
|
||||
|
||||
{% if this.description4 %}
|
||||
<div class="container">
|
||||
<div class="row mb-3">
|
||||
<div class="col-md-12">
|
||||
{{ this.description4 }}
|
||||
<section class="section">
|
||||
<div class="container">
|
||||
<div class="row mb-3">
|
||||
<div class="col-md-12">
|
||||
{{ this.description4 }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
{% endif %}
|
||||
|
||||
{% if this.content2_text %}
|
||||
|
@ -159,7 +173,7 @@
|
|||
<div class="card-body">
|
||||
<h5 class="card-title">{{ this.offer4_title }}</h5>
|
||||
<p class="card-text">{{ this.offer4_text }}</p>
|
||||
<a href="{{ this.offer4_link }}" class="btn btn-primary">Order</a>
|
||||
<a href="{{ this.offer4_link }}" class="btn btn-primary btn-sm">Order</a>
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
@ -170,7 +184,7 @@
|
|||
<div class="card-body">
|
||||
<h5 class="card-title">{{ this.offer5_title }}</h5>
|
||||
<p class="card-text">{{ this.offer5_text }}</p>
|
||||
<a href="{{ this.offer5_link }}" class="btn btn-primary">Order</a>
|
||||
<a href="{{ this.offer5_link }}" class="btn btn-primary btn-sm">Order</a>
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
@ -181,7 +195,7 @@
|
|||
<div class="card-body">
|
||||
<h5 class="card-title">{{ this.offer6_title }}</h5>
|
||||
<p class="card-text">{{ this.offer6_text }}</p>
|
||||
<a href="{{ this.offer6_link }}" class="btn btn-primary">Order</a>
|
||||
<a href="{{ this.offer6_link }}" class="btn btn-primary btn-sm">Order</a>
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
|
|
@ -3,7 +3,7 @@
|
|||
{% block subtitle %}{{ this.subtitle }}{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
|
||||
<div class="section">
|
||||
<div class="container">
|
||||
{% for childpage in this.children %}
|
||||
|
||||
|
@ -63,4 +63,5 @@
|
|||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
|
|
@ -3,4 +3,4 @@ name = ungleich: IPv6 - Linux - FOSS
|
|||
url = https://ungleich.ch
|
||||
|
||||
[packages]
|
||||
lektor-atom = 0.3
|
||||
lektor-atom = 0.4.0
|
||||
|
|