uncloud-mravi/doc/uncloud-manual-2020-08-01.org

* Bootstrap / Installation
** Pre-requisites by operating system
*** Alpine
    #+BEGIN_SRC sh
apk add openldap-dev postgresql-dev libxml2-dev libxslt-dev
#+END_SRC
*** Debian/Devuan:
    #+BEGIN_SRC sh
apt install postgresql-server-dev-all
#+END_SRC
** Creating a virtual environment / installing python requirements
*** Virtual env
    To separate uncloud requirements, you can use a python virtual
    env as follows:
    #+BEGIN_SRC sh
python3 -m venv venv
. ./venv/bin/activate
#+END_SRC
    Then install the requirements
    #+BEGIN_SRC sh
pip install -r requirements.txt
#+END_SRC
** Setting up the the database
*** Install the database service
    The database can run on the same host as uncloud, but can also run
    a different server. Consult the usual postgresql documentation for
    a secure configuration.
**** Alpine
    #+BEGIN_SRC sh
apk add postgresql-server
rc-update add postgresql
rc-service  postgresql start`
#+END_SRC

**** Debian/Devuan:
       #+BEGIN_SRC sh
   apt install postgresql
   #+END_SRC
*** Create the database
    Due to the use of the JSONField, postgresql is required.
    To get started,
    create a database and have it owned by the user that runs uncloud
    (usually "uncloud"):

    #+BEGIN_SRC sh
bridge:~# su - postgres
bridge:~$ psql
postgres=# create role uncloud login;
postgres=# create database uncloud owner nico;
#+END_SRC
*** Creating the schema
    #+BEGIN_SRC sh
python manage.py migrate
#+END_SRC

** Bootstrap
   - Login via a user so that the user object gets created
   - Run the following (replace nicocustomer with the username)
    #+BEGIN_SRC sh
    python manage.py bootstrap-user --username nicocustomer
    #+END_SRC

* Testing / CLI Access
  Access via the commandline (CLI) can be done using curl or
  httpie. In our examples we will use httpie.
** Checkout out the API
   #+BEGIN_SRC sh
   http localhost:8000/api/
   #+END_SRC
** Authenticate via ldap user in password store
   #+BEGIN_SRC sh
   http --auth nicocustomer:$(pass ldap/nicocustomer)  localhost:8000/api/
   #+END_SRC
* Database
** uncloud clients access the data base from a variety of outside hosts
** So the postgresql data base needs to be remotely accessible
** Instead of exposing the tcp socket, we make postgresql bind to localhost via IPv6
*** ::1, port 5432
** Then we remotely connect to the database server with ssh tunneling
*** ssh -L5432:localhost:5432 uncloud-database-host
** Configuring your database for SSH based remote access
*** host    all             all             ::1/128                 trust

* URLs
  - api/ - the rest API
* uncloud Products
** VPN
*** How to add a new VPN Host
**** Install wireguard to the host
**** Install uncloud to the host
**** Add `python manage.py vpn --hostname fqdn-of-this-host` to the crontab
**** Use the CLI to configure one or more VPN Networks for this host
*** Example of adding a VPN host at ungleich
**** Create a new dual stack alpine VM
**** Add it to DNS as vpn-XXX.ungleich.ch
**** Route a /40 network to its IPv6 address
**** Install wireguard on it
**** TODO [#C] Enable wireguard on boot
**** TODO [#C] Create a new VPNPool on uncloud with
***** the network address (selecting from our existing pool)
***** the network size (/...)
***** the vpn host that provides the network (selecting the created VM)
***** the wireguard private key of the vpn host (using wg genkey)
***** http command
  ```
  http -a nicoschottelius:$(pass
    ungleich.ch/nico.schottelius@ungleich.ch)
    http://localhost:8000/admin/vpnpool/ network=2a0a:e5c1:200:: \
    network_size=40 subnetwork_size=48
    vpn_hostname=vpn-2a0ae5c1200.ungleich.ch
    wireguard_private_key=...
  ```
*** Example http commands / REST calls
**** creating a new vpn pool
   http -a nicoschottelius:$(pass
   ungleich.ch/nico.schottelius@ungleich.ch)
   http://localhost:8000/admin/vpnpool/ network_size=40
   subnetwork_size=48 network=2a0a:e5c1:200::
   vpn_hostname=vpn-2a0ae5c1200.ungleich.ch wireguard_private_key=$(wg
   genkey)
**** Creating a new vpn network
*** Creating a VPN pool

   #+BEGIN_SRC sh
http -a uncloudadmin:$(pass uncloudadmin) https://localhost:8000/v1/admin/vpnpool/ \
   network=2a0a:e5c1:200:: network_size=40 subnetwork_size=48 \
   vpn_hostname=vpn-2a0ae5c1200.ungleich.ch wireguard_private_key=$(wg genkey)
   #+END_SRC

This will create the VPNPool 2a0a:e5c1:200::/40 from which /48
networks will be used for clients.

VPNPools can only be managed by staff.

*** Managing VPNNetworks

To request a network as a client, use the following call:

   #+BEGIN_SRC sh
   http -a user:$(pass user) https://localhost:8000/v1/net/vpn/ \
   network_size=48 \
   wireguard_public_key=$(wg genkey | tee privatekey | wg pubkey)
```

VPNNetworks can be managed by all authenticated users.

* Developer Handbook
  The following section describe decisions / architecture of
  uncloud. These chapters are intended to be read by developers.
** Documentation
   This documentation is written in org-mode. To compile it to
   html/pdf, just open emacs and press *C-c C-e l p*.
** Models
*** Bill
    Bills are summarising usage in a specific timeframe. Bills usually
    spawn one month.
*** BillRecord
    Bill records are used to model the usage of one order during the
    timeframe.
*** Order
    Orders register the intent of a user to buy something. They might
    refer to a product. (???)
    Order register the one time price and the recurring price. These
    fields should be treated as immutable. If they need to be modified,
    a new order that replaces the current order should be created.
**** Replacing orders
     If an order is updated, a new order is created and points to the
     old order. The old order stops one second before the new order
     starts.

     If a order has been replaced can be seen by its replaced_by count:
     #+BEGIN_SRC sh
 >>> Order.objects.get(id=1).replaced_by.count()
 1
     #+END_SRC

*** Product and Product Children
    - A product describes something a user can buy
    - A product inherits from the uncloud_pay.models.Product model to
      get basic attributes
** Identifiers
*** Problem description
    Identifiers can be integers, strings or other objects. They should
    be unique.
*** Approach 1: integers
    Integers are somewhat easy to remember, but also include
    predictable growth, which might allow access to guessed hacking
    (obivously proper permissions should prevent this).
*** Approach 2: random uuids
    UUIDs are 128 bit integers. Python supports uuid.uuid4() for random
    uuids.
*** Approach 3: IPv6 addresses
    uncloud heavily depends on IPv6 in the first place. uncloud could
    use a /48 to identify all objects. Objects that have IPv6 addresses
    on their own, don't need to draw from the system /48.
**** Possible Subnetworks
     Assuming uncloud uses a /48 to represent all resources.

     | Network         | Name            | Description                                  |
     |-----------------+-----------------+----------------------------------------------|
     | 2001:db8::/48   | uncloud network | All identifiers drawn from here              |
     | 2001:db8:1::/64 | VM network      | Every VM has an IPv6 address in this network |
     | 2001:db8:2::/64 | Bill network    | Every bill has an IPv6 address               |
     | 2001:db8:3::/64 | Order network   | Every order has an IPv6 address              |
     | 2001:db8:5::/64 | Product network | Every product (?) has an IPv6 address        |
     | 2001:db8:4::/64 | Disk network    | Every disk is identified                     |

**** Tests
     [15:47:37] black3.place6:~# rbd create -s 10G ssd/2a0a:e5c0:1::8

*** Decision
    We use integers, because they are easy.

** Milestones                                                       :uncloud:
*** 1.1 (cleanup 1)
****
*** 1.0 (initial release)
**** TODO Initial Generic product support
     - Product
***** TODO Recurring product support
****** TODO Support replacing orders for updates
****** TODO [#A] Finish split of bill creation
****** TODO [#A] Test the new functions in the Order class
***** TODO Generating bill for admins/staff
      -