212 lines
7.5 KiB
Org Mode
212 lines
7.5 KiB
Org Mode
* Bootstrap / Installation
|
|
** Pre-requisites by operating system
|
|
*** Alpine
|
|
#+BEGIN_SRC sh
|
|
apk add openldap-dev postgresql-dev libxml2-dev libxslt-dev
|
|
#+END_SRC
|
|
*** Debian/Devuan:
|
|
#+BEGIN_SRC sh
|
|
apt install postgresql-server-dev-all
|
|
#+END_SRC
|
|
** Creating a virtual environment / installing python requirements
|
|
*** Virtual env
|
|
To separate uncloud requirements, you can use a python virtual
|
|
env as follows:
|
|
#+BEGIN_SRC sh
|
|
python3 -m venv venv
|
|
. ./venv/bin/activate
|
|
#+END_SRC
|
|
Then install the requirements
|
|
#+BEGIN_SRC sh
|
|
pip install -r requirements.txt
|
|
#+END_SRC
|
|
** Setting up the the database
|
|
*** Install the database service
|
|
The database can run on the same host as uncloud, but can also run
|
|
a different server. Consult the usual postgresql documentation for
|
|
a secure configuration.
|
|
**** Alpine
|
|
#+BEGIN_SRC sh
|
|
apk add postgresql-server
|
|
rc-update add postgresql
|
|
rc-service postgresql start`
|
|
#+END_SRC
|
|
|
|
**** Debian/Devuan:
|
|
#+BEGIN_SRC sh
|
|
apt install postgresql
|
|
#+END_SRC
|
|
*** Create the database
|
|
Due to the use of the JSONField, postgresql is required.
|
|
To get started,
|
|
create a database and have it owned by the user that runs uncloud
|
|
(usually "uncloud"):
|
|
|
|
#+BEGIN_SRC sh
|
|
bridge:~# su - postgres
|
|
bridge:~$ psql
|
|
postgres=# create role uncloud login;
|
|
postgres=# create database uncloud owner nico;
|
|
#+END_SRC
|
|
*** Creating the schema
|
|
#+BEGIN_SRC sh
|
|
python manage.py migrate
|
|
#+END_SRC
|
|
|
|
** Bootstrap
|
|
- Login via a user so that the user object gets created
|
|
- Run the following (replace nicocustomer with the username)
|
|
#+BEGIN_SRC sh
|
|
python manage.py bootstrap-user --username nicocustomer
|
|
#+END_SRC
|
|
|
|
* Testing / CLI Access
|
|
Access via the commandline (CLI) can be done using curl or
|
|
httpie. In our examples we will use httpie.
|
|
** Checkout out the API
|
|
#+BEGIN_SRC sh
|
|
http localhost:8000/api/
|
|
#+END_SRC
|
|
** Authenticate via ldap user in password store
|
|
#+BEGIN_SRC sh
|
|
http --auth nicocustomer:$(pass ldap/nicocustomer) localhost:8000/api/
|
|
#+END_SRC
|
|
* Database
|
|
** uncloud clients access the data base from a variety of outside hosts
|
|
** So the postgresql data base needs to be remotely accessible
|
|
** Instead of exposing the tcp socket, we make postgresql bind to localhost via IPv6
|
|
*** ::1, port 5432
|
|
** Then we remotely connect to the database server with ssh tunneling
|
|
*** ssh -L5432:localhost:5432 uncloud-database-host
|
|
** Configuring your database for SSH based remote access
|
|
*** host all all ::1/128 trust
|
|
|
|
* URLs
|
|
- api/ - the rest API
|
|
* uncloud Products
|
|
** VPN
|
|
*** How to add a new VPN Host
|
|
**** Install wireguard to the host
|
|
**** Install uncloud to the host
|
|
**** Add `python manage.py vpn --hostname fqdn-of-this-host` to the crontab
|
|
**** Use the CLI to configure one or more VPN Networks for this host
|
|
*** Example of adding a VPN host at ungleich
|
|
**** Create a new dual stack alpine VM
|
|
**** Add it to DNS as vpn-XXX.ungleich.ch
|
|
**** Route a /40 network to its IPv6 address
|
|
**** Install wireguard on it
|
|
**** TODO Enable wireguard on boot
|
|
**** TODO Create a new VPNPool on uncloud with
|
|
***** the network address (selecting from our existing pool)
|
|
***** the network size (/...)
|
|
***** the vpn host that provides the network (selecting the created VM)
|
|
***** the wireguard private key of the vpn host (using wg genkey)
|
|
***** http command
|
|
```
|
|
http -a nicoschottelius:$(pass
|
|
ungleich.ch/nico.schottelius@ungleich.ch)
|
|
http://localhost:8000/admin/vpnpool/ network=2a0a:e5c1:200:: \
|
|
network_size=40 subnetwork_size=48
|
|
vpn_hostname=vpn-2a0ae5c1200.ungleich.ch
|
|
wireguard_private_key=...
|
|
```
|
|
*** Example http commands / REST calls
|
|
**** creating a new vpn pool
|
|
http -a nicoschottelius:$(pass
|
|
ungleich.ch/nico.schottelius@ungleich.ch)
|
|
http://localhost:8000/admin/vpnpool/ network_size=40
|
|
subnetwork_size=48 network=2a0a:e5c1:200::
|
|
vpn_hostname=vpn-2a0ae5c1200.ungleich.ch wireguard_private_key=$(wg
|
|
genkey)
|
|
**** Creating a new vpn network
|
|
*** Creating a VPN pool
|
|
|
|
#+BEGIN_SRC sh
|
|
http -a uncloudadmin:$(pass uncloudadmin) https://localhost:8000/v1/admin/vpnpool/ \
|
|
network=2a0a:e5c1:200:: network_size=40 subnetwork_size=48 \
|
|
vpn_hostname=vpn-2a0ae5c1200.ungleich.ch wireguard_private_key=$(wg genkey)
|
|
#+END_SRC
|
|
|
|
This will create the VPNPool 2a0a:e5c1:200::/40 from which /48
|
|
networks will be used for clients.
|
|
|
|
VPNPools can only be managed by staff.
|
|
|
|
*** Managing VPNNetworks
|
|
|
|
To request a network as a client, use the following call:
|
|
|
|
#+BEGIN_SRC sh
|
|
http -a user:$(pass user) https://localhost:8000/v1/net/vpn/ \
|
|
network_size=48 \
|
|
wireguard_public_key=$(wg genkey | tee privatekey | wg pubkey)
|
|
```
|
|
|
|
VPNNetworks can be managed by all authenticated users.
|
|
|
|
* Developer Handbook
|
|
The following section describe decisions / architecture of
|
|
uncloud. These chapters are intended to be read by developers.
|
|
** Documentation
|
|
This documentation is written in org-mode. To compile it to
|
|
html/pdf, just open emacs and press *C-c C-e l p*.
|
|
** Models
|
|
*** Bill
|
|
Bills are summarising usage in a specific timeframe. Bills usually
|
|
spawn one month.
|
|
*** BillRecord
|
|
Bill records are used to model the usage of one order during the
|
|
timeframe.
|
|
*** Order
|
|
Orders register the intent of a user to buy something. They might
|
|
refer to a product. (???)
|
|
Order register the one time price and the recurring price. These
|
|
fields should be treated as immutable. If they need to be modified,
|
|
a new order that replaces the current order should be created.
|
|
**** Replacing orders
|
|
If an order is updated, a new order is created and points to the
|
|
old order. The old order stops one second before the new order
|
|
starts.
|
|
|
|
If a order has been replaced can be seen by its replaced_by count:
|
|
#+BEGIN_SRC sh
|
|
>>> Order.objects.get(id=1).replaced_by.count()
|
|
1
|
|
#+END_SRC
|
|
|
|
*** Product and Product Children
|
|
- A product describes something a user can buy
|
|
- A product inherits from the uncloud_pay.models.Product model to
|
|
get basic attributes
|
|
** Identifiers
|
|
*** Problem description
|
|
Identifiers can be integers, strings or other objects. They should
|
|
be unique.
|
|
*** Approach 1: integers
|
|
Integers are somewhat easy to remember, but also include
|
|
predictable growth, which might allow access to guessed hacking
|
|
(obivously proper permissions should prevent this).
|
|
*** Approach 2: random uuids
|
|
UUIDs are 128 bit integers. Python supports uuid.uuid4() for random
|
|
uuids.
|
|
*** Approach 3: IPv6 addresses
|
|
uncloud heavily depends on IPv6 in the first place. uncloud could
|
|
use a /48 to identify all objects. Objects that have IPv6 addresses
|
|
on their own, don't need to draw from the system /48.
|
|
**** Possible Subnetworks
|
|
Assuming uncloud uses a /48 to represent all resources.
|
|
|
|
| Network | Name | Description |
|
|
|-----------------+-----------------+----------------------------------------------|
|
|
| 2001:db8::/48 | uncloud network | All identifiers drawn from here |
|
|
| 2001:db8:1::/64 | VM network | Every VM has an IPv6 address in this network |
|
|
| 2001:db8:2::/64 | Bill network | Every bill has an IPv6 address |
|
|
| 2001:db8:3::/64 | Order network | Every order has an IPv6 address |
|
|
| 2001:db8:5::/64 | Product network | Every product (?) has an IPv6 address |
|
|
| 2001:db8:4::/64 | Disk network | Every disk is identified |
|
|
|
|
**** Tests
|
|
[15:47:37] black3.place6:~# rbd create -s 10G ssd/2a0a:e5c0:1::8
|
|
|
|
*** Decision
|
|
We use integers, because they are easy.
|