amal/uncloud
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
uncloud/doc/uncloud-manual-2020-08-01.org
Nico Schottelius f693dd3d18 ++notes
2020-08-09 21:10:43 +02:00

7.5 KiB
Raw Blame History

Bootstrap / Installation

Pre-requisites by operating system

Alpine 

apk add openldap-dev postgresql-dev libxml2-dev libxslt-dev

Debian/Devuan: 

apt install postgresql-server-dev-all

Creating a virtual environment / installing python requirements

Virtual env

To separate uncloud requirements, you can use a python virtual env as follows:

python3 -m venv venv
. ./venv/bin/activate

Then install the requirements

pip install -r requirements.txt

Setting up the the database

Install the database service

The database can run on the same host as uncloud, but can also run a different server. Consult the usual postgresql documentation for a secure configuration.

Alpine 
apk add postgresql-server
rc-update add postgresql
rc-service  postgresql start`
Debian/Devuan: 
apt install postgresql

Create the database

Due to the use of the JSONField, postgresql is required. To get started, create a database and have it owned by the user that runs uncloud (usually "uncloud"):

bridge:~# su - postgres
bridge:~$ psql
postgres=# create role uncloud login;
postgres=# create database uncloud owner nico;

Creating the schema 

python manage.py migrate

Bootstrap

  • Login via a user so that the user object gets created

  • Run the following (replace nicocustomer with the username)

    python manage.py bootstrap-user --username nicocustomer

Testing / CLI Access

Access via the commandline (CLI) can be done using curl or httpie. In our examples we will use httpie.

Checkout out the API 

http localhost:8000/api/

Authenticate via ldap user in password store 

http --auth nicocustomer:$(pass ldap/nicocustomer)  localhost:8000/api/

Database

uncloud clients access the data base from a variety of outside hosts

So the postgresql data base needs to be remotely accessible

Instead of exposing the tcp socket, we make postgresql bind to localhost via IPv6

::1, port 5432

Then we remotely connect to the database server with ssh tunneling

ssh -L5432:localhost:5432 uncloud-database-host

Configuring your database for SSH based remote access

host all all ::1/128 trust

URLs

  • api/ - the rest API

uncloud Products

VPN

How to add a new VPN Host

Install wireguard to the host
Install uncloud to the host
Add `python manage.py vpn –hostname fqdn-of-this-host` to the crontab
Use the CLI to configure one or more VPN Networks for this host

Example of adding a VPN host at ungleich

Create a new dual stack alpine VM
Add it to DNS as vpn-XXX.ungleich.ch
Route a /40 network to its IPv6 address
Install wireguard on it
TODO Enable wireguard on boot
TODO Create a new VPNPool on uncloud with
the network address (selecting from our existing pool)
the network size (/…)
the vpn host that provides the network (selecting the created VM)
the wireguard private key of the vpn host (using wg genkey)
http command

``` http -a nicoschottelius:$(pass ungleich.ch/nico.schottelius@ungleich.ch) http://localhost:8000/admin/vpnpool/ network=2a0a:e5c1:200:: \ network_size=40 subnetwork_size=48 vpn_hostname=vpn-2a0ae5c1200.ungleich.ch wireguard_private_key=… ```

Example http commands / REST calls

creating a new vpn pool

http -a nicoschottelius:$(pass ungleich.ch/nico.schottelius@ungleich.ch) http://localhost:8000/admin/vpnpool/ network_size=40 subnetwork_size=48 network=2a0a:e5c1:200:: vpn_hostname=vpn-2a0ae5c1200.ungleich.ch wireguard_private_key=$(wg genkey)

Creating a new vpn network

Creating a VPN pool 

http -a uncloudadmin:$(pass uncloudadmin) https://localhost:8000/v1/admin/vpnpool/ \
network=2a0a:e5c1:200:: network_size=40 subnetwork_size=48 \
vpn_hostname=vpn-2a0ae5c1200.ungleich.ch wireguard_private_key=$(wg genkey)

This will create the VPNPool 2a0a:e5c1:200::/40 from which /48 networks will be used for clients.

VPNPools can only be managed by staff.

Managing VPNNetworks

To request a network as a client, use the following call:

http -a user:$(pass user) https://localhost:8000/v1/net/vpn/ \
network_size=48 \
wireguard_public_key=$(wg genkey | tee privatekey | wg pubkey)
```

VPNNetworks can be managed by all authenticated users.

* Developer Handbook
The following section describe decisions / architecture of
uncloud. These chapters are intended to be read by developers.
** Documentation
This documentation is written in org-mode. To compile it to
html/pdf, just open emacs and press *C-c C-e l p*.
** Models
*** Bill
 Bills are summarising usage in a specific timeframe. Bills usually
 spawn one month.
*** BillRecord
 Bill records are used to model the usage of one order during the
 timeframe.
*** Order
 Orders register the intent of a user to buy something. They might
 refer to a product. (???)
 Order register the one time price and the recurring price. These
 fields should be treated as immutable. If they need to be modified,
 a new order that replaces the current order should be created.
**** Replacing orders
  If an order is updated, a new order is created and points to the
  old order. The old order stops one second before the new order
  starts.

  If a order has been replaced can be seen by its replaced_by count:
  #+BEGIN_SRC sh
>>> Order.objects.get(id=1).replaced_by.count()
1

Product and Product Children

  • A product describes something a user can buy
  • A product inherits from the uncloud_pay.models.Product model to get basic attributes

Identifiers

Problem description

Identifiers can be integers, strings or other objects. They should be unique.

Approach 1: integers

Integers are somewhat easy to remember, but also include predictable growth, which might allow access to guessed hacking (obivously proper permissions should prevent this).

Approach 2: random uuids

UUIDs are 128 bit integers. Python supports uuid.uuid4() for random uuids.

Approach 3: IPv6 addresses

uncloud heavily depends on IPv6 in the first place. uncloud could use a /48 to identify all objects. Objects that have IPv6 addresses on their own, don't need to draw from the system /48.

Possible Subnetworks

Assuming uncloud uses a /48 to represent all resources.

Network Name Description
2001:db8::/48 uncloud network All identifiers drawn from here
2001:db8:1::/64 VM network Every VM has an IPv6 address in this network
2001:db8:2::/64 Bill network Every bill has an IPv6 address
2001:db8:3::/64 Order network Every order has an IPv6 address
2001:db8:5::/64 Product network Every product (?) has an IPv6 address
2001:db8:4::/64 Disk network Every disk is identified
Tests

[15:47:37] black3.place6:~# rbd create -s 10G ssd/2a0a:e5c0:1::8

Decision

We use integers, because they are easy.