uncloud-mravi/ucloud/network/README

The network base - experimental


We want to have 1 "main" network for convience.

We want to be able to create networks automatically, once a new
customer is created -> need hooks!


Mapping:

- each network is a "virtual" network. We use vxlan by default, but
  could be any technology!
- we need a counter for vxlan mappings / network IDs -> cannot use

Model in etcd:

/v1/networks/


Tests
see
https://vincent.bernat.ch/en/blog/2017-vxlan-linux


#    local 2001:db8:1::1 \


netid=100
dev=wlp2s0
dev=wlp0s20f3
ip -6 link add vxlan${netid} type vxlan \
    id ${netid} \
    dstport 4789 \
    group ff05::${netid} \
    dev ${dev} \
    ttl 5

[root@diamond ~]# ip addr add 2a0a:e5c0:5::1/48 dev vxlan100
root@manager:~/.ssh# ip addr add 2a0a:e5c0:5::2/48 dev vxlan100
root@manager:~/.ssh# ping -c3 2a0a:e5c0:5::1
PING 2a0a:e5c0:5::1(2a0a:e5c0:5::1) 56 data bytes
64 bytes from 2a0a:e5c0:5::1: icmp_seq=1 ttl=64 time=15.6 ms
64 bytes from 2a0a:e5c0:5::1: icmp_seq=2 ttl=64 time=30.3 ms
64 bytes from 2a0a:e5c0:5::1: icmp_seq=3 ttl=64 time=84.4 ms

--- 2a0a:e5c0:5::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 15.580/43.437/84.417/29.594 ms

--> work even via wifi


--------------------------------------------------------------------------------

Creating a network:

1) part of the initialisation / demo data (?)

We should probably provide some demo sets that can easily be used.

2) manual/hook based request

- hosts might have different network interfaces (?)
  -> this will make things very tricky -> don't support it
- endpoint needs only support

--------------------------------------------------------------------------------

IPAM

IP address management (IPAM) is related to networks, but needs to be
decoupled to allow pure L2 networks.

From a customer point of view, we probably want to do something like:

- ORDERING an IPv6 network can include creating a virtual network and
  an IPAM service

Maybe "orders" should always be the first class citizen and ucloud
internally "hooks" or binds things together.

--------------------------------------------------------------------------------

testing / hacking:

- starting etcd as storage


[18:07] diamond:~% etcdctl put /v1/network/200 "{ some_network }"
OK
[18:08] diamond:~% etcdctl watch -w=json --prefix /v1/network
{"Header":{"cluster_id":14841639068965178418,"member_id":10276657743932975437,"revision":6,"raft_term":2},"Events":[{"kv":{"key":"L3YxL25ldHdvcmsvMjAw","create_revision":5,"mod_revision":6,"version":2,"value":"eyBzb21lX25ldHdvcmsgfQ=="}}],"CompactRevision":0,"Canceled":false,"Created":false}


--------------------------------------------------------------------------------

Flow for using and creating networks:

- a network is created -> entry in etcd is created
  -> we need to keep a counter/lock so that 2 processes don't create
  the same network [Ahmed]
  -> nothing to be done on the hosts
- a VM using a network is created
- a VM using a network is scheduled to some host
- the local "spawn a VM" process needs to check whether there is a
  vxlan interface existing -> if no, create it before creating the VM.
  -> if no, also create the bridge
  -> possibly adjusting the MTU (??)
  -> both names should be in hexadecimal (i.e. brff01 or vxlanff01)
  --> this way they are consistent with the multicast ipv6 address
  --> attention, ip -6 link ... id XXX expects DECIMAL input

--------------------------------------------------------------------------------
If we also supply IPAM:

- ipam needs to be created *after* the network is created
- ipam is likely to be coupled to netbox (?)
  --> we need a "get next /64 prefix" function
- when an ipam service is created in etcd, we need to create a new
  radvd instance on all routers (this will be a different service on
  BSDs)
- we will need to create a new vxlan device on the routers
- we need to create a new / modify radvd.conf
- only after all of the routers reloaded radvd the ipam service is
  available!


--------------------------------------------------------------------------------
If the user requests an IPv4 VM:

- we need to get the next free IPv4 address (again, netbox?)
- we need to create a mapping entry on the routers for NAT64
  --> this requires the VM to be in a network with IPAM
  --> we always assume that the VM embeds itself using EUI64



--------------------------------------------------------------------------------
mac address handling!

Example

--------------------------------------------------------------------------------

TODOs

- create-vxlan-on-dev.sh -> the multicast group
  needs to be ff05:: +int(vxlan_id)

--------------------------------------------------------------------------------

Python hints:

>>> vxlan_id = 3400
>>> b = ipaddress.IPv6Network("ff05::/16")
>>> b[vxlan_id]
IPv6Address('ff05::d48')

we need / should assign hex values for vxlan ids in etcd!
--> easier to read

>>> b[0x3400]
IPv6Address('ff05::3400')


--------------------------------------------------------------------------------

Bridge names are limited to 15 characters


Maximum/highest number of vxlan:

>>> 2**24
16777216
>>> (2**25)-1
33554431

>>> b[33554431]
IPv6Address('ff05::1ff:ffff')

Last interface:
br1ffffff
vxlan1ffffff

root@manager:~/ucloud/network# ip -6 link add vxlan1ffffff type vxlan id 33554431 dstport 4789 group ff05::1ff:ffff dev wlp2s0 ttl 5
Error: argument "33554431" is wrong: invalid id

root@manager:~/ucloud/network# ip -6 link add vxlanffffff type vxlan id 16777215 dstport 4789 group ff05::ff:ffff dev wlp2s0 ttl 5


# id needs to be decimal
root@manager:~# ip -6 link add vxlanff01 type vxlan id ff01 dstport 4789 group ff05::ff01 dev ttl 5
Error: argument "ff01" is wrong: invalid id
root@manager:~# ip -6 link add vxlanff01 type vxlan id 65281 dstport 4789 group ff05::ff01 dev wlp2s0 ttl 5