Use correct app name

Command executes but still does not create bills

doc/uncloud-manual-2020-08-01.org

@@ -1,8 +1,15 @@
 * Bootstrap / Installation
 ** Pre-requisites by operating system
+*** General
+    To run uncloud you need:
+    - ldap development libraries
+    - libxml2-dev libxslt-dev
+    - gcc / libc headers: for compiling things
+    - python3-dev
+    - wireguard: wg (for checking keys)
 *** Alpine
     #+BEGIN_SRC sh
-apk add openldap-dev postgresql-dev libxml2-dev libxslt-dev
+apk add openldap-dev postgresql-dev libxml2-dev libxslt-dev gcc python3-dev musl-dev wireguard-tools-wg
 #+END_SRC
 *** Debian/Devuan:
     #+BEGIN_SRC sh
@@ -60,6 +67,21 @@ python manage.py migrate
     python manage.py bootstrap-user --username nicocustomer
     #+END_SRC
 
+** Initialise the database
+   While it is not strictly required to add default values to the
+   database, it might significantly reduce the starting time with
+   uncloud.
+
+   To add the default database values run:
+
+   #+BEGIN_SRC shell
+   # Add local objects
+   python manage.py db-add-defaults
+
+   # Import VAT rates
+   python manage.py import-vat-rates
+   #+END_SRC
+
 * Testing / CLI Access
   Access via the commandline (CLI) can be done using curl or
   httpie. In our examples we will use httpie.
@@ -84,6 +106,14 @@ python manage.py migrate
 * URLs
   - api/ - the rest API
 * uncloud Products
+** Product features
+   - Dependencies on other products
+   - Minimum parameters (min cpu, min ram, etc).
+   - Can also realise the dcl vm
+     - dualstack vm = VM + IPv4 + SSD
+   - Need to have a non-misguiding name for the "bare VM"
+   - Should support network boot (?)
+
 ** VPN
 *** How to add a new VPN Host
 **** Install wireguard to the host
@@ -95,8 +125,8 @@ python manage.py migrate
 **** Add it to DNS as vpn-XXX.ungleich.ch
 **** Route a /40 network to its IPv6 address
 **** Install wireguard on it
-**** TODO Enable wireguard on boot
+**** TODO [#C] Enable wireguard on boot
-**** TODO Create a new VPNPool on uncloud with
+**** TODO [#C] Create a new VPNPool on uncloud with
 ***** the network address (selecting from our existing pool)
 ***** the network size (/...)
 ***** the vpn host that provides the network (selecting the created VM)
@@ -119,7 +149,6 @@ python manage.py migrate
    vpn_hostname=vpn-2a0ae5c1200.ungleich.ch wireguard_private_key=$(wg
    genkey)
 **** Creating a new vpn network
-** VPN
 *** Creating a VPN pool
 
    #+BEGIN_SRC sh
@@ -148,7 +177,7 @@ VPNNetworks can be managed by all authenticated users.
 * Developer Handbook
   The following section describe decisions / architecture of
   uncloud. These chapters are intended to be read by developers.
-** Documentation
+** This Documentation
    This documentation is written in org-mode. To compile it to
    html/pdf, just open emacs and press *C-c C-e l p*.
 ** Models
@@ -211,3 +240,143 @@ VPNNetworks can be managed by all authenticated users.
 
 *** Decision
     We use integers, because they are easy.
+
+** Distributing/Dispatching/Orchestrating
+*** Variant 1: using cdist
+    - The uncloud server can git commit things
+    - The uncloud server loads cdist and configures the server
+    - Advantages
+      - Fully integrated into normal flow
+    - Disadvantage
+      - web frontend has access to more data than it needs
+      - On compromise of the machine, more data leaks
+      - Some cdist usual delay
+*** Variant 2: via celery
+    - The uncloud server dispatches via celery
+    - Every decentral node also runs celery/connects to the broker
+    - Summary brokers:
+      - If local only celery -> good to use redis    - Broker
+      - If remote: probably better to use rabbitmq
+    - redis
+      - simpler
+    - rabbitmq
+      - more versatile
+      - made for remote connections
+      - quorom queues would be nice, but not clear if supported
+        - https://github.com/celery/py-amqp/issues/302
+        - https://github.com/celery/celery/issues/6067
+      - Cannot be installed on alpine Linux at the moment
+    - Advantage
+      - Very python / django integrated
+      - Rather instant
+    - Disadvantages
+      - Every decentral node needs to have the uncloud code available
+      - Decentral nodes *might* need to access the database
+        - Tasks can probably be written to work without that
+          (i.e. only strings/bytes)
+
+**** log/tests
+     (venv) [19:54] vpn-2a0ae5c1200:~/uncloud$ celery -A uncloud -b redis://bridge.place7.ungleich.ch worker -n worker1@%h --logfile ~/celery.log -
+Q vpn-2a0ae5c1200.ungleich.ch
+
+
+*** Variant 3: dedicated cdist instance via message broker
+    - A separate VM/machine
+    - Has Checkout of ~/.cdist
+    - Has cdist checkout
+    - Tiny API for management
+    - Not directly web accessible
+    - "cdist" queue
+
+** Milestones                                                       :uncloud:
+*** 1.1 (cleanup 1)
+**** TODO [#C] Unify ValidationError, FieldError - define proper Exception
+     - What do we use for model errors
+*** 1.0 (initial release)
+**** TODO [#C] Initial Generic product support
+     - Product
+***** TODO [#C] Recurring product support
+****** TODO [#C] Support replacing orders for updates
+****** DONE [#A] Finish split of bill creation
+       CLOSED: [2020-09-11 Fri 23:19]
+****** TODO [#C] Test the new functions in the Order class
+****** Define the correct order replacement logic
+       Assumption:
+       - recurringperiods are 30days
+******* Case 1: downgrading
+        - User commits to 10 CHF for 30 days
+        - Wants to downgrade after 15 days to 5 CHF product
+        - Expected result:
+          - order 1: 10 CHF until +30days
+          - order 2: 5 CHF starting 30days + 1s
+          - Sum of the two orders is 15 CHF
+        - Question is
+          - when is the VM shutdown?
+            - a) instantly
+            - b) at the end of the cycle
+          - best solution
+            - user can choose between a ... b any time
+******* Duration
+        - You cannot cancel the duration
+        - You can upgrade and with that cancel the duration
+        - The idea of a duration is that you commit for it
+        - If you want to commit lower (daily basis for instance) you
+          have higher per period prices
+******* Case X
+        - User has VM with 2 Core / 2 GB RAM
+        - User modifies with to 1 core / 3 GB RAM
+        - We treat it as down/upgrade independent of the modifications
+
+******* Case 2: upgrading after 1 day
+        - committed for 30 days
+        - upgrade after 1 day
+        - so first order will be charged for 1/30ths
+
+******* Case 2: upgrading
+        - User commits to 10 CHF for 30 days
+        - Wants to upgrade after 15 days to 20 CHF product
+        - Order 1 : 1 VM with 2 Core / 2 GB / 10 SSD -- 10 CHF
+          - 30days period, stopped after 15, so quantity is 0.5 = 5 CHF
+        - Order 2 : 1 VM with 2 Core / 6 GB / 10 SSD -- 20 CHF
+          - after 15 days
+          - VM is upgraded instantly
+          - Expected result:
+            - order 1: 10 CHF until +15days = 0.5 units = 5 CHF
+            - order 2: 20 CHF starting 15days + 1s ... +30 days after
+              the 15 days -> 45 days = 1 unit = 20 CHF
+            - Total on bill: 25 CHF
+
+******* Case 2: upgrading
+        - User commits to 10 CHF for 30 days
+        - Wants to upgrade after 15 days to 20 CHF product
+        - Expected result:
+          - order 1: 10 CHF until +30days = 1 units = 10 CHF
+
+          - order 2: 20 CHF starting 15days + 1s = 1 unit = 20 CHF
+          - Total on bill: 30 CHF
+
+
+****** TODO [#C] Note: ending date not set if replaced by default (implicit!)
+       - Should the new order modify the old order on save()?
+****** DONE Fix totally wrong bill dates in our test case
+       CLOSED: [2020-09-09 Wed 01:00]
+       - 2020 used instead of 2019
+       - Was due to existing test data ...
+***** DONE Bill logic is still wrong
+      CLOSED: [2020-11-05 Thu 18:58]
+      - Bill starting_date is the date of the first order
+      - However first encountered order does not have to be the
+        earliest in the bill!
+      - Bills should not have a duration
+      - Bills should only have a (unique) issue date
+      - We charge based on bill_records
+        - Last time charged issue date of the bill OR earliest date
+          after that
+        - Every bill generation checks all (relevant) orders
+          - add a flag "not_for_billing" or "closed"
+          - query on that flag
+          - verify it every time
+
+
+***** TODO Generating bill for admins/staff
+      -

opennebula/migrations/0001_initial.py

@@ -1,9 +1,6 @@
-# Generated by Django 3.0.6 on 2020-08-01 16:38
+# Generated by Django 3.1 on 2020-12-13 10:38
 
-from django.conf import settings
-import django.contrib.postgres.fields.jsonb
 from django.db import migrations, models
-import django.db.models.deletion
 
 
 class Migration(migrations.Migration):
@@ -11,23 +8,14 @@ class Migration(migrations.Migration):
     initial = True
 
     dependencies = [
-        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
-        ('uncloud_pay', '0001_initial'),
     ]
 
     operations = [
         migrations.CreateModel(
             name='VM',
             fields=[
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
-                ('status', models.CharField(choices=[('PENDING', 'Pending'), ('AWAITING_PAYMENT', 'Awaiting payment'), ('BEING_CREATED', 'Being created'), ('SCHEDULED', 'Scheduled'), ('ACTIVE', 'Active'), ('MODIFYING', 'Modifying'), ('DELETED', 'Deleted'), ('DISABLED', 'Disabled'), ('UNUSABLE', 'Unusable')], default='AWAITING_PAYMENT', max_length=32)),
                 ('vmid', models.IntegerField(primary_key=True, serialize=False)),
-                ('data', django.contrib.postgres.fields.jsonb.JSONField()),
+                ('data', models.JSONField()),
-                ('order', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order')),
-                ('owner', models.ForeignKey(editable=False, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
             ],
-            options={
-                'abstract': False,
-            },
         ),
     ]

opennebula/migrations/0002_auto_20200801_2332.py

@@ -1,20 +0,0 @@
-# Generated by Django 3.0.8 on 2020-08-01 23:32
-
-from django.db import migrations, models
-import django.db.models.deletion
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0003_auto_20200801_2332'),
-        ('opennebula', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='vm',
-            name='order',
-            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order'),
-        ),
-    ]

opennebula/migrations/0003_auto_20200808_1953.py

@@ -1,23 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 19:53
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('opennebula', '0002_auto_20200801_2332'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='vm',
-            name='data',
-            field=models.JSONField(),
-        ),
-        migrations.AlterField(
-            model_name='vm',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-    ]

opennebula/models.py

@@ -10,7 +10,7 @@ storage_class_mapping = {
     'hdd': 'hdd'
 }
 
-class VM(Product):
+class VM(models.Model):
     vmid = models.IntegerField(primary_key=True)
     data = models.JSONField()
 

opennebula/views.py

@@ -1,16 +1,16 @@
 from rest_framework import viewsets, permissions
 
-from .models import VM
+#from .models import VM
-from .serializers import OpenNebulaVMSerializer
+# from .serializers import OpenNebulaVMSerializer
 
-class VMViewSet(viewsets.ModelViewSet):
+# class VMViewSet(viewsets.ModelViewSet):
-    permission_classes = [permissions.IsAuthenticated]
+#     permission_classes = [permissions.IsAuthenticated]
-    serializer_class = OpenNebulaVMSerializer
+#     serializer_class = OpenNebulaVMSerializer
 
-    def get_queryset(self):
+#     def get_queryset(self):
-        if self.request.user.is_superuser:
+#         if self.request.user.is_superuser:
-            obj = VM.objects.all()
+#             obj = VM.objects.all()
-        else:
+#         else:
-            obj = VM.objects.filter(owner=self.request.user)
+#             obj = VM.objects.filter(owner=self.request.user)
 
-        return obj
+#         return obj

requirements.txt

@@ -1,9 +1,12 @@
+# Django basics
 django
 djangorestframework
 django-auth-ldap
-stripe
+
-xmltodict
 psycopg2
+ldap3
+
+xmltodict
 
 parsedatetime
 
@@ -19,6 +22,11 @@ django-hardcopy
 pyyaml
 uritemplate
 
-# Comprehensive interface to validate VAT numbers, making use of the VIES
+# Payment & VAT
-# service for European countries.
 vat-validator
+stripe
+
+
+# Tasks
+celery
+redis

uncloud/.gitignore

@@ -1 +1,2 @@
 local_settings.py
+ldap_max_uid_file

uncloud/__init__.py

@@ -0,0 +1,254 @@
+from django.utils.translation import gettext_lazy as _
+import decimal
+from .celery import app as celery_app
+
+# Define DecimalField properties, used to represent amounts of money.
+AMOUNT_MAX_DIGITS=10
+AMOUNT_DECIMALS=2
+
+decimal.getcontext().prec = AMOUNT_DECIMALS
+
+# http://xml.coverpages.org/country3166.html
+COUNTRIES = (
+    ('AD', _('Andorra')),
+    ('AE', _('United Arab Emirates')),
+    ('AF', _('Afghanistan')),
+    ('AG', _('Antigua & Barbuda')),
+    ('AI', _('Anguilla')),
+    ('AL', _('Albania')),
+    ('AM', _('Armenia')),
+    ('AN', _('Netherlands Antilles')),
+    ('AO', _('Angola')),
+    ('AQ', _('Antarctica')),
+    ('AR', _('Argentina')),
+    ('AS', _('American Samoa')),
+    ('AT', _('Austria')),
+    ('AU', _('Australia')),
+    ('AW', _('Aruba')),
+    ('AZ', _('Azerbaijan')),
+    ('BA', _('Bosnia and Herzegovina')),
+    ('BB', _('Barbados')),
+    ('BD', _('Bangladesh')),
+    ('BE', _('Belgium')),
+    ('BF', _('Burkina Faso')),
+    ('BG', _('Bulgaria')),
+    ('BH', _('Bahrain')),
+    ('BI', _('Burundi')),
+    ('BJ', _('Benin')),
+    ('BM', _('Bermuda')),
+    ('BN', _('Brunei Darussalam')),
+    ('BO', _('Bolivia')),
+    ('BR', _('Brazil')),
+    ('BS', _('Bahama')),
+    ('BT', _('Bhutan')),
+    ('BV', _('Bouvet Island')),
+    ('BW', _('Botswana')),
+    ('BY', _('Belarus')),
+    ('BZ', _('Belize')),
+    ('CA', _('Canada')),
+    ('CC', _('Cocos (Keeling) Islands')),
+    ('CF', _('Central African Republic')),
+    ('CG', _('Congo')),
+    ('CH', _('Switzerland')),
+    ('CI', _('Ivory Coast')),
+    ('CK', _('Cook Iislands')),
+    ('CL', _('Chile')),
+    ('CM', _('Cameroon')),
+    ('CN', _('China')),
+    ('CO', _('Colombia')),
+    ('CR', _('Costa Rica')),
+    ('CU', _('Cuba')),
+    ('CV', _('Cape Verde')),
+    ('CX', _('Christmas Island')),
+    ('CY', _('Cyprus')),
+    ('CZ', _('Czech Republic')),
+    ('DE', _('Germany')),
+    ('DJ', _('Djibouti')),
+    ('DK', _('Denmark')),
+    ('DM', _('Dominica')),
+    ('DO', _('Dominican Republic')),
+    ('DZ', _('Algeria')),
+    ('EC', _('Ecuador')),
+    ('EE', _('Estonia')),
+    ('EG', _('Egypt')),
+    ('EH', _('Western Sahara')),
+    ('ER', _('Eritrea')),
+    ('ES', _('Spain')),
+    ('ET', _('Ethiopia')),
+    ('FI', _('Finland')),
+    ('FJ', _('Fiji')),
+    ('FK', _('Falkland Islands (Malvinas)')),
+    ('FM', _('Micronesia')),
+    ('FO', _('Faroe Islands')),
+    ('FR', _('France')),
+    ('FX', _('France, Metropolitan')),
+    ('GA', _('Gabon')),
+    ('GB', _('United Kingdom (Great Britain)')),
+    ('GD', _('Grenada')),
+    ('GE', _('Georgia')),
+    ('GF', _('French Guiana')),
+    ('GH', _('Ghana')),
+    ('GI', _('Gibraltar')),
+    ('GL', _('Greenland')),
+    ('GM', _('Gambia')),
+    ('GN', _('Guinea')),
+    ('GP', _('Guadeloupe')),
+    ('GQ', _('Equatorial Guinea')),
+    ('GR', _('Greece')),
+    ('GS', _('South Georgia and the South Sandwich Islands')),
+    ('GT', _('Guatemala')),
+    ('GU', _('Guam')),
+    ('GW', _('Guinea-Bissau')),
+    ('GY', _('Guyana')),
+    ('HK', _('Hong Kong')),
+    ('HM', _('Heard & McDonald Islands')),
+    ('HN', _('Honduras')),
+    ('HR', _('Croatia')),
+    ('HT', _('Haiti')),
+    ('HU', _('Hungary')),
+    ('ID', _('Indonesia')),
+    ('IE', _('Ireland')),
+    ('IL', _('Israel')),
+    ('IN', _('India')),
+    ('IO', _('British Indian Ocean Territory')),
+    ('IQ', _('Iraq')),
+    ('IR', _('Islamic Republic of Iran')),
+    ('IS', _('Iceland')),
+    ('IT', _('Italy')),
+    ('JM', _('Jamaica')),
+    ('JO', _('Jordan')),
+    ('JP', _('Japan')),
+    ('KE', _('Kenya')),
+    ('KG', _('Kyrgyzstan')),
+    ('KH', _('Cambodia')),
+    ('KI', _('Kiribati')),
+    ('KM', _('Comoros')),
+    ('KN', _('St. Kitts and Nevis')),
+    ('KP', _('Korea, Democratic People\'s Republic of')),
+    ('KR', _('Korea, Republic of')),
+    ('KW', _('Kuwait')),
+    ('KY', _('Cayman Islands')),
+    ('KZ', _('Kazakhstan')),
+    ('LA', _('Lao People\'s Democratic Republic')),
+    ('LB', _('Lebanon')),
+    ('LC', _('Saint Lucia')),
+    ('LI', _('Liechtenstein')),
+    ('LK', _('Sri Lanka')),
+    ('LR', _('Liberia')),
+    ('LS', _('Lesotho')),
+    ('LT', _('Lithuania')),
+    ('LU', _('Luxembourg')),
+    ('LV', _('Latvia')),
+    ('LY', _('Libyan Arab Jamahiriya')),
+    ('MA', _('Morocco')),
+    ('MC', _('Monaco')),
+    ('MD', _('Moldova, Republic of')),
+    ('MG', _('Madagascar')),
+    ('MH', _('Marshall Islands')),
+    ('ML', _('Mali')),
+    ('MN', _('Mongolia')),
+    ('MM', _('Myanmar')),
+    ('MO', _('Macau')),
+    ('MP', _('Northern Mariana Islands')),
+    ('MQ', _('Martinique')),
+    ('MR', _('Mauritania')),
+    ('MS', _('Monserrat')),
+    ('MT', _('Malta')),
+    ('MU', _('Mauritius')),
+    ('MV', _('Maldives')),
+    ('MW', _('Malawi')),
+    ('MX', _('Mexico')),
+    ('MY', _('Malaysia')),
+    ('MZ', _('Mozambique')),
+    ('NA', _('Namibia')),
+    ('NC', _('New Caledonia')),
+    ('NE', _('Niger')),
+    ('NF', _('Norfolk Island')),
+    ('NG', _('Nigeria')),
+    ('NI', _('Nicaragua')),
+    ('NL', _('Netherlands')),
+    ('NO', _('Norway')),
+    ('NP', _('Nepal')),
+    ('NR', _('Nauru')),
+    ('NU', _('Niue')),
+    ('NZ', _('New Zealand')),
+    ('OM', _('Oman')),
+    ('PA', _('Panama')),
+    ('PE', _('Peru')),
+    ('PF', _('French Polynesia')),
+    ('PG', _('Papua New Guinea')),
+    ('PH', _('Philippines')),
+    ('PK', _('Pakistan')),
+    ('PL', _('Poland')),
+    ('PM', _('St. Pierre & Miquelon')),
+    ('PN', _('Pitcairn')),
+    ('PR', _('Puerto Rico')),
+    ('PT', _('Portugal')),
+    ('PW', _('Palau')),
+    ('PY', _('Paraguay')),
+    ('QA', _('Qatar')),
+    ('RE', _('Reunion')),
+    ('RO', _('Romania')),
+    ('RU', _('Russian Federation')),
+    ('RW', _('Rwanda')),
+    ('SA', _('Saudi Arabia')),
+    ('SB', _('Solomon Islands')),
+    ('SC', _('Seychelles')),
+    ('SD', _('Sudan')),
+    ('SE', _('Sweden')),
+    ('SG', _('Singapore')),
+    ('SH', _('St. Helena')),
+    ('SI', _('Slovenia')),
+    ('SJ', _('Svalbard & Jan Mayen Islands')),
+    ('SK', _('Slovakia')),
+    ('SL', _('Sierra Leone')),
+    ('SM', _('San Marino')),
+    ('SN', _('Senegal')),
+    ('SO', _('Somalia')),
+    ('SR', _('Suriname')),
+    ('ST', _('Sao Tome & Principe')),
+    ('SV', _('El Salvador')),
+    ('SY', _('Syrian Arab Republic')),
+    ('SZ', _('Swaziland')),
+    ('TC', _('Turks & Caicos Islands')),
+    ('TD', _('Chad')),
+    ('TF', _('French Southern Territories')),
+    ('TG', _('Togo')),
+    ('TH', _('Thailand')),
+    ('TJ', _('Tajikistan')),
+    ('TK', _('Tokelau')),
+    ('TM', _('Turkmenistan')),
+    ('TN', _('Tunisia')),
+    ('TO', _('Tonga')),
+    ('TP', _('East Timor')),
+    ('TR', _('Turkey')),
+    ('TT', _('Trinidad & Tobago')),
+    ('TV', _('Tuvalu')),
+    ('TW', _('Taiwan, Province of China')),
+    ('TZ', _('Tanzania, United Republic of')),
+    ('UA', _('Ukraine')),
+    ('UG', _('Uganda')),
+    ('UM', _('United States Minor Outlying Islands')),
+    ('US', _('United States of America')),
+    ('UY', _('Uruguay')),
+    ('UZ', _('Uzbekistan')),
+    ('VA', _('Vatican City State (Holy See)')),
+    ('VC', _('St. Vincent & the Grenadines')),
+    ('VE', _('Venezuela')),
+    ('VG', _('British Virgin Islands')),
+    ('VI', _('United States Virgin Islands')),
+    ('VN', _('Viet Nam')),
+    ('VU', _('Vanuatu')),
+    ('WF', _('Wallis & Futuna Islands')),
+    ('WS', _('Samoa')),
+    ('YE', _('Yemen')),
+    ('YT', _('Mayotte')),
+    ('YU', _('Yugoslavia')),
+    ('ZA', _('South Africa')),
+    ('ZM', _('Zambia')),
+    ('ZR', _('Zaire')),
+    ('ZW', _('Zimbabwe')),
+)
+
+
+__all__ = ('celery_app',)

uncloud/admin.py

@@ -0,0 +1,6 @@
+from django.contrib import admin
+
+from .models import *
+
+for m in [ UncloudProvider, UncloudNetwork, UncloudTask ]:
+    admin.site.register(m)

uncloud/celery.py

@@ -0,0 +1,17 @@
+import os
+
+from celery import Celery
+
+# set the default Django settings module for the 'celery' program.
+os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'uncloud.settings')
+
+app = Celery('uncloud')
+
+# Using a string here means the worker doesn't have to serialize
+# the configuration object to child processes.
+# - namespace='CELERY' means all celery-related configuration keys
+#   should have a `CELERY_` prefix.
+app.config_from_object('django.conf:settings', namespace='CELERY')
+
+# Load task modules from all registered Django app configs.
+app.autodiscover_tasks()

uncloud/management/commands/db-add-defaults.py

@@ -0,0 +1,43 @@
+import random
+import string
+
+from django.core.management.base import BaseCommand
+from django.core.exceptions import ObjectDoesNotExist
+from django.contrib.auth import get_user_model
+from django.conf import settings
+
+from uncloud_pay.models import BillingAddress, RecurringPeriod, Product
+from uncloud.models import UncloudProvider, UncloudNetwork
+
+
+class Command(BaseCommand):
+    help = 'Add standard uncloud values'
+
+    def add_arguments(self, parser):
+        pass
+
+    def handle(self, *args, **options):
+        # Order matters, objects can be dependent on each other
+
+        admin_username="uncloud-admin"
+        pw_length = 32
+
+        # Only set password if the user did not exist before
+        try:
+            admin_user = get_user_model().objects.get(username=settings.UNCLOUD_ADMIN_NAME)
+        except ObjectDoesNotExist:
+            random_password = ''.join(random.SystemRandom().choice(string.ascii_lowercase + string.digits) for _ in range(pw_length))
+
+            admin_user = get_user_model().objects.create_user(username=settings.UNCLOUD_ADMIN_NAME, password=random_password)
+            admin_user.is_superuser=True
+            admin_user.is_staff=True
+            admin_user.save()
+
+            print(f"Created admin user '{admin_username}' with password '{random_password}'")
+
+        BillingAddress.populate_db_defaults()
+        RecurringPeriod.populate_db_defaults()
+        Product.populate_db_defaults()
+
+        UncloudNetwork.populate_db_defaults()
+        UncloudProvider.populate_db_defaults()

uncloud/migrations/0002_uncloudtasks.py

@@ -0,0 +1,19 @@
+# Generated by Django 3.1 on 2020-12-20 17:16
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='UncloudTasks',
+            fields=[
+                ('task_id', models.UUIDField(primary_key=True, serialize=False)),
+            ],
+        ),
+    ]

uncloud/migrations/0003_auto_20201220_1728.py

@@ -0,0 +1,17 @@
+# Generated by Django 3.1 on 2020-12-20 17:28
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud', '0002_uncloudtasks'),
+    ]
+
+    operations = [
+        migrations.RenameModel(
+            old_name='UncloudTasks',
+            new_name='UncloudTask',
+        ),
+    ]

uncloud/models.py

@@ -1,7 +1,11 @@
 from django.db import models
-from django.db.models import JSONField
+from django.db.models import JSONField, Q
-
+from django.utils import timezone
 from django.utils.translation import gettext_lazy as _
+from django.core.validators import MinValueValidator, MaxValueValidator
+from django.core.exceptions import FieldError
+
+from uncloud import COUNTRIES
 
 class UncloudModel(models.Model):
     """
@@ -34,3 +38,135 @@ class UncloudStatus(models.TextChoices):
     DELETED          = 'DELETED', _('Deleted')     # Resource has been deleted
     DISABLED         = 'DISABLED', _('Disabled')   # Is usable, but cannot be used for new things
     UNUSABLE         = 'UNUSABLE', _('Unusable'),  # Has some kind of error
+
+
+
+###
+# General address handling
+class CountryField(models.CharField):
+    def __init__(self, *args, **kwargs):
+        kwargs.setdefault('choices', COUNTRIES)
+        kwargs.setdefault('default', 'CH')
+        kwargs.setdefault('max_length', 2)
+
+        super().__init__(*args, **kwargs)
+
+    def get_internal_type(self):
+        return "CharField"
+
+
+class UncloudAddress(models.Model):
+    full_name = models.CharField(max_length=256)
+    organization = models.CharField(max_length=256, blank=True, null=True)
+    street = models.CharField(max_length=256)
+    city = models.CharField(max_length=256)
+    postal_code = models.CharField(max_length=64)
+    country = CountryField(blank=True)
+
+    class Meta:
+        abstract = True
+
+
+###
+# UncloudNetworks are used as identifiers - such they are a base of uncloud
+
+class UncloudNetwork(models.Model):
+    """
+    Storing IP networks
+    """
+
+    network_address = models.GenericIPAddressField(null=False, unique=True)
+    network_mask = models.IntegerField(null=False,
+                                       validators=[MinValueValidator(0),
+                                                   MaxValueValidator(128)]
+                                       )
+
+    description = models.CharField(max_length=256)
+
+    @classmethod
+    def populate_db_defaults(cls):
+        for net, desc in [
+                ( "2a0a:e5c0:11::", "uncloud Billing" ),
+                ( "2a0a:e5c0:11:1::", "uncloud Referral" ),
+                ( "2a0a:e5c0:11:2::", "uncloud Coupon" )
+                ]:
+            obj, created = cls.objects.get_or_create(network_address=net,
+                                                     defaults= {
+                                                         'network_mask': 64,
+                                                         'description': desc
+                                                         }
+                                                     )
+
+
+    def save(self, *args, **kwargs):
+        if not ':' in self.network_address and self.network_mask > 32:
+            raise FieldError("Mask cannot exceed 32 for IPv4")
+
+        super().save(*args, **kwargs)
+
+
+    def __str__(self):
+        return f"{self.network_address}/{self.network_mask} {self.description}"
+
+###
+# Who is running / providing this instance of uncloud?
+
+class UncloudProvider(UncloudAddress):
+    """
+    A class resembling who is running this uncloud instance.
+    This might change over time so we allow starting/ending dates
+
+    This also defines the taxation rules.
+
+    starting/ending date define from when to when this is valid. This way
+    we can model address changes and have it correct in the bills.
+    """
+
+    # Meta:
+    # FIXMe: only allow non overlapping time frames -- how to define this as a constraint?
+    starting_date = models.DateField()
+    ending_date = models.DateField(blank=True, null=True)
+
+    billing_network = models.ForeignKey(UncloudNetwork, related_name="uncloudproviderbill", on_delete=models.CASCADE)
+    referral_network = models.ForeignKey(UncloudNetwork, related_name="uncloudproviderreferral", on_delete=models.CASCADE)
+    coupon_network = models.ForeignKey(UncloudNetwork, related_name="uncloudprovidercoupon", on_delete=models.CASCADE)
+
+
+    @classmethod
+    def get_provider(cls, when=None):
+        """
+        Find active provide at a certain time - if there was any
+        """
+
+        if not when:
+            when = timezone.now()
+
+
+        return cls.objects.get(Q(starting_date__gte=when, ending_date__lte=when) |
+                               Q(starting_date__gte=when, ending_date__isnull=True))
+
+
+    @classmethod
+    def populate_db_defaults(cls):
+        obj, created = cls.objects.get_or_create(full_name="ungleich glarus ag",
+                                                 street="Bahnhofstrasse 1",
+                                                 postal_code="8783",
+                                                 city="Linthal",
+                                                 country="CH",
+                                                 starting_date=timezone.now(),
+                                                 billing_network=UncloudNetwork.objects.get(description="uncloud Billing"),
+                                                 referral_network=UncloudNetwork.objects.get(description="uncloud Referral"),
+                                                 coupon_network=UncloudNetwork.objects.get(description="uncloud Coupon")
+                                                 )
+
+
+    def __str__(self):
+        return f"{self.full_name} {self.country}"
+
+
+class UncloudTask(models.Model):
+    """
+    Class to store dispatched tasks to be handled
+    """
+
+    task_id = models.UUIDField(primary_key=True)

uncloud/settings.py

@@ -11,6 +11,7 @@ https://docs.djangoproject.com/en/3.0/ref/settings/
 """
 
 import os
+import re
 import ldap
 
 from django.core.management.utils import get_random_secret_key
@@ -19,8 +20,6 @@ from django_auth_ldap.config import LDAPSearch, LDAPSearchUnion
 
 LOGGING = {}
 
-
-
 # Build paths inside the project like this: os.path.join(BASE_DIR, ...)
 BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 
@@ -172,7 +171,6 @@ OPENNEBULA_URL = 'https://opennebula.example.com:2634/RPC2'
 # user:pass for accessing opennebula
 OPENNEBULA_USER_PASS = 'user:password'
 
-
 # Stripe (Credit Card payments)
 STRIPE_KEY=""
 STRIPE_PUBLIC_KEY=""
@@ -185,6 +183,55 @@ ALLOWED_HOSTS = []
 # required for hardcopy / pdf rendering: https://github.com/loftylabs/django-hardcopy
 CHROME_PATH = '/usr/bin/chromium-browser'
 
+# Username that is created by default and owns the configuration objects
+UNCLOUD_ADMIN_NAME = "uncloud-admin"
+
+LOGIN_REDIRECT_URL = '/'
+LOGOUT_REDIRECT_URL = '/'
+
+# replace these in local_settings.py
+AUTH_LDAP_SERVER_URI = "ldaps://ldap1.example.com,ldaps://ldap2.example.com"
+AUTH_LDAP_BIND_DN="uid=django,ou=system,dc=example,dc=com"
+AUTH_LDAP_BIND_PASSWORD="a very secure ldap password"
+AUTH_LDAP_USER_SEARCH = LDAPSearch("dc=example,dc=com",
+                                   ldap.SCOPE_SUBTREE,
+                                   "(uid=%(user)s)")
+
+# where to create customers
+LDAP_CUSTOMER_DN="ou=customer,dc=example,dc=com"
+
+# def route_task(name, args, kwargs, options, task=None, **kw):
+#     print(f"{name} - {args} - {kwargs}")
+# #    if name == 'myapp.tasks.compress_video':
+#     return {'queue': 'vpn1' }
+# #            'exchange_type': 'topic',
+# #            'routing_key': 'video.compress'}
+
+
+# CELERY_TASK_ROUTES = (route_task,)
+
+# CELERY_TASK_ROUTES = {
+#     '*': {
+#         'queue': 'vpn1'
+#     }
+# }
+
+
+CELERY_BROKER_URL = 'redis://:uncloud.example.com:6379/0'
+CELERY_RESULT_BACKEND = 'redis://:uncloud.example.com:6379/0'
+
+CELERY_TASK_ROUTES = {
+    re.compile(r'.*.tasks.cdist.*'): { 'queue': 'cdist' } # cdist tasks go into cdist queue
+}
+
+CELERY_BEAT_SCHEDULE = {
+    'cleanup_tasks': {
+        'task': 'uncloud.tasks.cleanup_tasks',
+        'schedule': 10
+    }
+}
+
+# CELERY_TASK_CREATE_MISSING_QUEUES = False
 
 # Overwrite settings with local settings, if existing
 try:

uncloud/tasks.py

@@ -0,0 +1,19 @@
+from celery import shared_task
+from celery.result import AsyncResult
+
+from .models import UncloudTask
+
+@shared_task(bind=True)
+def cleanup_tasks(self):
+    print(f"Cleanup time from {self}: {self.request.id}")
+    for task in UncloudTask.objects.all():
+        print(f"Pruning {task}...")
+
+        if str(task.task_id) == str(self.request.id):
+            print("Skipping myself")
+            continue
+
+        res = AsyncResult(id=str(task.task_id))
+        if res.ready():
+            print(res.get())
+            task.delete()

uncloud/templates/uncloud/index.html

@@ -0,0 +1,15 @@
+{% extends 'uncloud/base.html' %}
+{% block title %}{% endblock %}
+
+{% block body %}
+<div>
+  <h1>Welcome to uncloud</h1>
+  Welcome to uncloud, checkout the following locations:
+
+  <ul>
+    <li><a href="/api/">The API</a>
+    <li><a href="/cc/reg/">The CC registration</a>
+  </ul>
+
+</div>
+{% endblock %}

uncloud/urls.py

@@ -12,7 +12,8 @@ from django.conf.urls.static import static
 from rest_framework import routers
 from rest_framework.schemas import get_schema_view
 
-from opennebula   import views as oneviews
+#from opennebula   import views as oneviews
+from uncloud      import views as uncloudviews
 from uncloud_auth import views as authviews
 from uncloud_net  import views as netviews
 from uncloud_pay  import views as payviews
@@ -41,10 +42,6 @@ router.register(r'v1/service/matrix', serviceviews.MatrixServiceProductViewSet,
 router.register(r'v1/service/generic', serviceviews.GenericServiceProductViewSet, basename='genericserviceproduct')
 
 
-# Net
-router.register(r'v1/net/vpn', netviews.VPNNetworkViewSet, basename='vpnnetwork')
-router.register(r'v1/admin/vpnreservation', netviews.VPNNetworkReservationViewSet, basename='vpnnetreservation')
-
 
 # Pay
 router.register(r'v1/my/address', payviews.BillingAddressViewSet, basename='billingaddress')
@@ -59,16 +56,26 @@ router.register(r'v1/admin/payment', payviews.AdminPaymentViewSet, basename='adm
 router.register(r'v1/admin/order', payviews.AdminOrderViewSet, basename='admin/order')
 router.register(r'v1/admin/vmhost', vmviews.VMHostViewSet)
 router.register(r'v1/admin/vmcluster', vmviews.VMClusterViewSet)
-router.register(r'v1/admin/vpnpool', netviews.VPNPoolViewSet)
+#router.register(r'v1/admin/vpnpool', netviews.VPNPoolViewSet)
-router.register(r'v1/admin/opennebula', oneviews.VMViewSet, basename='opennebula')
+#router.register(r'v1/admin/opennebula', oneviews.VMViewSet, basename='opennebula')
 
 # User/Account
 router.register(r'v1/my/user', authviews.UserViewSet, basename='user')
 router.register(r'v1/admin/user', authviews.AdminUserViewSet, basename='useradmin')
+router.register(r'v1/user/register', authviews.AccountManagementViewSet, basename='user/register')
+
+
+################################################################################
+# v2
+
+# Net
+router.register(r'v2/net/wireguardvpn', netviews.WireGuardVPNViewSet, basename='wireguardvpnnetwork')
+router.register(r'v2/net/wireguardvpnsizes', netviews.WireGuardVPNSizes, basename='wireguardvpnnetworksizes')
+
+
 
 urlpatterns = [
     path(r'api/', include(router.urls)),
-    # web/ = stuff to view in the browser
 
     path('api-auth/', include('rest_framework.urls', namespace='rest_framework')), # for login to REST API
     path('openapi', get_schema_view(
@@ -76,5 +83,12 @@ urlpatterns = [
         description="uncloud API",
         version="1.0.0"
     ), name='openapi-schema'),
+
+    # web/ = stuff to view in the browser
+#    path('web/vpn/create/', netviews.WireGuardVPNCreateView.as_view(), name="vpncreate"),
+    path('login/', authviews.LoginView.as_view(), name="login"),
+    path('logout/', authviews.LogoutView.as_view(), name="logout"),
     path('admin/', admin.site.urls),
+    path('cc/reg/', payviews.RegisterCard.as_view(), name="cc_register"),
+    path('', uncloudviews.UncloudIndex.as_view(), name="uncloudindex"),
 ]

uncloud/views.py

@@ -0,0 +1,4 @@
+from django.views.generic.base import TemplateView
+
+class UncloudIndex(TemplateView):
+    template_name = "uncloud/index.html"

uncloud_auth/migrations/0001_initial.py

@@ -1,4 +1,4 @@
-# Generated by Django 3.0.6 on 2020-08-01 16:38
+# Generated by Django 3.1 on 2020-12-13 10:38
 
 import django.contrib.auth.models
 import django.contrib.auth.validators
@@ -12,7 +12,7 @@ class Migration(migrations.Migration):
     initial = True
 
     dependencies = [
-        ('auth', '0011_update_proxy_permissions'),
+        ('auth', '0012_alter_user_first_name_max_length'),
     ]
 
     operations = [
@@ -24,7 +24,7 @@ class Migration(migrations.Migration):
                 ('last_login', models.DateTimeField(blank=True, null=True, verbose_name='last login')),
                 ('is_superuser', models.BooleanField(default=False, help_text='Designates that this user has all permissions without explicitly assigning them.', verbose_name='superuser status')),
                 ('username', models.CharField(error_messages={'unique': 'A user with that username already exists.'}, help_text='Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.', max_length=150, unique=True, validators=[django.contrib.auth.validators.UnicodeUsernameValidator()], verbose_name='username')),
-                ('first_name', models.CharField(blank=True, max_length=30, verbose_name='first name')),
+                ('first_name', models.CharField(blank=True, max_length=150, verbose_name='first name')),
                 ('last_name', models.CharField(blank=True, max_length=150, verbose_name='last name')),
                 ('email', models.EmailField(blank=True, max_length=254, verbose_name='email address')),
                 ('is_staff', models.BooleanField(default=False, help_text='Designates whether the user can log into this admin site.', verbose_name='staff status')),

uncloud_auth/migrations/0002_auto_20200808_1953.py

@@ -1,18 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 19:53
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_auth', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='user',
-            name='first_name',
-            field=models.CharField(blank=True, max_length=150, verbose_name='first name'),
-        ),
-    ]

uncloud_auth/models.py

@@ -2,8 +2,7 @@ from django.contrib.auth.models import AbstractUser
 from django.db import models
 from django.core.validators import MinValueValidator
 
-from uncloud_pay import AMOUNT_DECIMALS, AMOUNT_MAX_DIGITS
+from uncloud import AMOUNT_DECIMALS, AMOUNT_MAX_DIGITS
-from uncloud_pay.models import get_balance_for_user
 
 class User(AbstractUser):
     """
@@ -16,10 +15,3 @@ class User(AbstractUser):
             max_digits=AMOUNT_MAX_DIGITS,
             decimal_places=AMOUNT_DECIMALS,
             validators=[MinValueValidator(0)])
-
-    # @property
-    # def primary_billing_address(self):
-
-    @property
-    def balance(self):
-        return get_balance_for_user(self)

uncloud_auth/serializers.py

@@ -1,25 +1,72 @@
 from django.contrib.auth import get_user_model
+from django.db import transaction
+from ldap3.core.exceptions import LDAPEntryAlreadyExistsResult
 from rest_framework import serializers
 
-from uncloud_pay import AMOUNT_DECIMALS, AMOUNT_MAX_DIGITS
+from uncloud import AMOUNT_DECIMALS, AMOUNT_MAX_DIGITS
 from uncloud_pay.models import BillingAddress
 
+from .ungleich_ldap import LdapManager
+
+
 class UserSerializer(serializers.ModelSerializer):
     class Meta:
         model = get_user_model()
         read_only_fields = [ 'username', 'balance', 'maximum_credit' ]
-        fields =  read_only_fields + [ 'email', 'primary_billing_address' ]
+        fields =  read_only_fields + [ 'email' ]  # , 'primary_billing_address' ]
 
     def validate(self, data):
         """
         Ensure that the primary billing address belongs to the user
         """
-
+        # The following is raising exceptions probably, it is WIP somewhere
-        if 'primary_billing_address' in data:
+        # if 'primary_billing_address' in data:
-            if not data['primary_billing_address'].owner == self.instance:
+        #     if not data['primary_billing_address'].owner == self.instance:
-                raise serializers.ValidationError("Invalid data")
+        #         raise serializers.ValidationError('Invalid data')
 
         return data
 
+    def update(self, instance, validated_data):
+        ldap_manager = LdapManager()
+        return_val, _ = ldap_manager.change_user_details(
+            instance.username, {'mail': validated_data.get('email')}
+        )
+        if not return_val:
+            raise serializers.ValidationError('Couldn\'t update email')
+        instance.email = validated_data.get('email')
+        instance.save()
+        return instance
+
+
+class UserRegistrationSerializer(serializers.ModelSerializer):
+    class Meta:
+        model = get_user_model()
+        fields = ['username', 'first_name', 'last_name', 'email', 'password']
+        extra_kwargs = {
+            'password': {'style': {'input_type': 'password'}},
+            'first_name': {'allow_blank': False, 'required': True},
+            'last_name': {'allow_blank': False, 'required': True},
+            'email': {'allow_blank': False, 'required': True},
+        }
+
+    def create(self, validated_data):
+        ldap_manager = LdapManager()
+        try:
+            data = {
+                'user': validated_data['username'],
+                'password': validated_data['password'],
+                'email': validated_data['email'],
+                'firstname': validated_data['first_name'],
+                'lastname': validated_data['last_name'],
+            }
+            ldap_manager.create_user(**data)
+        except LDAPEntryAlreadyExistsResult:
+            raise serializers.ValidationError(
+                {'username': ['A user with that username already exists.']}
+            )
+        else:
+            return get_user_model().objects.create_user(**validated_data)
+
+
 class ImportUserSerializer(serializers.Serializer):
     username = serializers.CharField()

uncloud_auth/templates/uncloud_auth/login.html

@@ -0,0 +1,13 @@
+{% extends 'uncloud/base.html' %}
+
+{% block body %}
+<div class="container">
+
+<form method="post">
+  {% csrf_token %}
+  {{ form }}
+  <input type="submit" value="Login">
+</form>
+</div>
+
+{% endblock %}

uncloud_auth/uldap.py

@@ -0,0 +1,42 @@
+import ldap
+# from django.conf import settings
+
+AUTH_LDAP_SERVER_URI = "ldaps://ldap1.ungleich.ch,ldaps://ldap2.ungleich.ch"
+AUTH_LDAP_BIND_DN="uid=django-create,ou=system,dc=ungleich,dc=ch"
+AUTH_LDAP_BIND_PASSWORD="kS#e+v\zjKn]L!,RIu2}V+DUS"
+# AUTH_LDAP_USER_SEARCH = LDAPSearch("dc=ungleich,dc=ch",
+#                                    ldap.SCOPE_SUBTREE,
+#                                    "(uid=%(user)s)")
+
+
+
+ldap_object = ldap.initialize(AUTH_LDAP_SERVER_URI)
+cancelid = ldap_object.bind(AUTH_LDAP_BIND_DN, AUTH_LDAP_BIND_PASSWORD)
+
+res = ldap_object.search_s("dc=ungleich,dc=ch", ldap.SCOPE_SUBTREE, "(uid=nico)")
+print(res)
+
+# class LDAP(object):
+#     """
+#     Managing users in LDAP
+
+#     Requires the following settings?
+
+#     LDAP_USER_DN: where to create users in the tree
+
+#     LDAP_ADMIN_DN: which DN to use for managing users
+#     LDAP_ADMIN_PASSWORD: which password to used
+
+#     This module will reuse information from djagno_auth_ldap, including:
+
+#     AUTH_LDAP_SERVER_URI
+
+#     """
+#     def __init__(self):
+#         pass
+
+#     def create_user(self):
+#         pass
+
+#     def change_password(self):
+#         pass

uncloud_auth/ungleich_ldap.py

@@ -0,0 +1,284 @@
+import base64
+import hashlib
+import logging
+import random
+
+import ldap3
+from django.conf import settings
+
+logger = logging.getLogger(__name__)
+
+
+class LdapManager:
+    __instance = None
+    def __new__(cls):
+        if LdapManager.__instance is None:
+            LdapManager.__instance = object.__new__(cls)
+        return LdapManager.__instance
+
+    def __init__(self):
+        """
+        Initialize the LDAP subsystem.
+        """
+        self.rng = random.SystemRandom()
+        self.server = ldap3.Server(settings.AUTH_LDAP_SERVER)
+
+
+    def get_admin_conn(self):
+        """
+        Return a bound :class:`ldap3.Connection` instance which has write
+        permissions on the dn in which the user accounts reside.
+        """
+        conn = self.get_conn(user=settings.LDAP_ADMIN_DN,
+                        password=settings.LDAP_ADMIN_PASSWORD,
+                        raise_exceptions=True)
+        conn.bind()
+        return conn
+
+
+    def get_conn(self, **kwargs):
+        """
+        Return an unbound :class:`ldap3.Connection` which talks to the configured
+        LDAP server.
+
+        The *kwargs* are passed to the constructor of :class:`ldap3.Connection` and
+        can be used to set *user*, *password* and other useful arguments.
+        """
+        return ldap3.Connection(self.server, **kwargs)
+
+
+    def _ssha_password(self, password):
+        """
+        Apply the SSHA password hashing scheme to the given *password*.
+        *password* must be a :class:`bytes` object, containing the utf-8
+        encoded password.
+
+        Return a :class:`bytes` object containing ``ascii``-compatible data
+        which can be used as LDAP value, e.g. after armoring it once more using
+        base64 or decoding it to unicode from ``ascii``.
+        """
+        SALT_BYTES = 15
+
+        sha1 = hashlib.sha1()
+        salt = self.rng.getrandbits(SALT_BYTES * 8).to_bytes(SALT_BYTES,
+                                                               "little")
+        sha1.update(password)
+        sha1.update(salt)
+
+        digest = sha1.digest()
+        passwd = b"{SSHA}" + base64.b64encode(digest + salt)
+        return passwd
+
+
+    def create_user(self, user, password, firstname, lastname, email):
+        conn = self.get_admin_conn()
+        uidNumber = self._get_max_uid() + 1
+        logger.debug("uidNumber={uidNumber}".format(uidNumber=uidNumber))
+        user_exists = True
+        while user_exists:
+            user_exists, _ = self.check_user_exists(
+                "",
+                '(&(objectClass=inetOrgPerson)(objectClass=posixAccount)'
+                '(objectClass=top)(uidNumber={uidNumber}))'.format(
+                    uidNumber=uidNumber
+                )
+            )
+            if user_exists:
+                logger.debug(
+                    "{uid} exists. Trying next.".format(uid=uidNumber)
+                )
+                uidNumber += 1
+        logger.debug("{uid} does not exist. Using it".format(uid=uidNumber))
+        self._set_max_uid(uidNumber)
+        try:
+            uid = user  # user.encode("utf-8")
+            conn.add("uid={uid},{customer_dn}".format(
+                    uid=uid, customer_dn=settings.LDAP_CUSTOMER_DN
+                ),
+                ["inetOrgPerson", "posixAccount", "ldapPublickey"],
+                {
+                    "uid": [uid],
+                    "sn": [lastname.encode("utf-8")],
+                    "givenName": [firstname.encode("utf-8")],
+                    "cn": [uid],
+                    "displayName": ["{} {}".format(firstname, lastname).encode("utf-8")],
+                    "uidNumber": [str(uidNumber)],
+                    "gidNumber": [str(settings.LDAP_CUSTOMER_GROUP_ID)],
+                    "loginShell": ["/bin/bash"],
+                    "homeDirectory": ["/home/{}".format(user).encode("utf-8")],
+                    "mail": email.encode("utf-8"),
+                    "userPassword": [self._ssha_password(
+                        password.encode("utf-8")
+                    )]
+                }
+            )
+            logger.debug('Created user %s %s' % (user.encode('utf-8'),
+                                                 uidNumber))
+        except Exception as ex:
+            logger.debug('Could not create user %s' % user.encode('utf-8'))
+            logger.error("Exception: " + str(ex))
+            raise
+        finally:
+            conn.unbind()
+
+
+    def change_password(self, uid, new_password):
+        """
+        Changes the password of the user identified by user_dn
+
+        :param uid: str The uid that identifies the user
+        :param new_password: str The new password string
+        :return: True if password was changed successfully False otherwise
+        """
+        conn = self.get_admin_conn()
+
+        # Make sure the user exists first to change his/her details
+        user_exists, entries = self.check_user_exists(
+            uid=uid,
+            search_base=settings.ENTIRE_SEARCH_BASE
+        )
+        return_val = False
+        if user_exists:
+            try:
+                return_val = conn.modify(
+                    entries[0].entry_dn,
+                    {
+                        "userpassword": (
+                            ldap3.MODIFY_REPLACE,
+                            [self._ssha_password(new_password.encode("utf-8"))]
+                        )
+                    }
+                )
+            except Exception as ex:
+                logger.error("Exception: " + str(ex))
+        else:
+            logger.error("User {} not found".format(uid))
+
+        conn.unbind()
+        return return_val
+
+    def change_user_details(self, uid, details):
+        """
+        Updates the user details as per given values in kwargs of the user
+        identified by user_dn.
+
+        Assumes that all attributes passed in kwargs are valid.
+
+        :param uid: str The uid that identifies the user
+        :param details: dict A dictionary containing the new values
+        :return: True if user details were updated successfully False otherwise
+        """
+        conn = self.get_admin_conn()
+
+        # Make sure the user exists first to change his/her details
+        user_exists, entries = self.check_user_exists(
+            uid=uid,
+            search_base=settings.ENTIRE_SEARCH_BASE
+        )
+
+        return_val = False
+        if user_exists:
+            details_dict = {k: (ldap3.MODIFY_REPLACE, [v.encode("utf-8")]) for
+                            k, v in details.items()}
+            try:
+                return_val = conn.modify(entries[0].entry_dn, details_dict)
+                msg = "success"
+            except Exception as ex:
+                msg = str(ex)
+                logger.error("Exception: " + msg)
+            finally:
+                conn.unbind()
+        else:
+            msg = "User {} not found".format(uid)
+            logger.error(msg)
+            conn.unbind()
+        return return_val, msg
+
+    def check_user_exists(self, uid, search_filter="", attributes=None,
+                          search_base=settings.LDAP_CUSTOMER_DN):
+        """
+        Check if the user with the given uid exists in the customer group.
+
+        :param uid: str representing the user
+        :param search_filter: str representing the filter condition to find
+                            users. If its empty, the search finds the user with
+                            the given uid.
+        :param attributes: list A list of str representing all the attributes
+                           to be obtained in the result entries
+        :param search_base: str
+        :return: tuple (bool, [ldap3.abstract.entry.Entry ..])
+                       A bool indicating if the user exists
+                       A list of all entries obtained in the search
+        """
+        conn = self.get_admin_conn()
+        entries = []
+        try:
+            result = conn.search(
+                search_base=search_base,
+                search_filter=search_filter if len(search_filter)> 0 else
+                '(uid={uid})'.format(uid=uid),
+                attributes=attributes
+            )
+            entries = conn.entries
+        finally:
+            conn.unbind()
+        return result, entries
+
+    def delete_user(self, uid):
+        """
+        Deletes the user with the given uid from ldap
+
+        :param uid: str representing the user
+        :return: True if the delete was successful False otherwise
+        """
+        conn = self.get_admin_conn()
+        try:
+            return_val = conn.delete(
+                ("uid={uid}," + settings.LDAP_CUSTOMER_DN).format(uid=uid),
+            )
+            msg = "success"
+        except Exception as ex:
+            msg = str(ex)
+            logger.error("Exception: " + msg)
+            return_val = False
+        finally:
+            conn.unbind()
+        return return_val, msg
+
+    def _set_max_uid(self, max_uid):
+        """
+        a utility function to save max_uid value to a file
+
+        :param max_uid: an integer representing the max uid
+        :return:
+        """
+        with open(settings.LDAP_MAX_UID_FILE_PATH, 'w+') as handler:
+            handler.write(str(max_uid))
+
+    def _get_max_uid(self):
+        """
+        A utility function to read the max uid value that was previously set
+
+        :return: An integer representing the max uid value that was previously
+         set
+        """
+        try:
+            with open(settings.LDAP_MAX_UID_FILE_PATH, 'r+') as handler:
+                try:
+                    return_value = int(handler.read())
+                except ValueError as ve:
+                    logger.error(
+                        "Error reading int value from {}. {}"
+                        "Returning default value {} instead".format(
+                            settings.LDAP_MAX_UID_PATH,
+                            str(ve),
+                            settings.LDAP_DEFAULT_START_UID
+                        )
+                    )
+                    return_value = settings.LDAP_DEFAULT_START_UID
+                return return_value
+        except FileNotFoundError as fnfe:
+            logger.error("File not found : " + str(fnfe))
+            return_value = settings.LDAP_DEFAULT_START_UID
+            logger.error("So, returning UID={}".format(return_value))
+            return return_value

uncloud_auth/views.py

@@ -1,9 +1,22 @@
-from rest_framework import viewsets, permissions, status
+from django.contrib.auth import views as auth_views
-from .serializers import *
+from django.contrib.auth import logout
+
 from django_auth_ldap.backend import LDAPBackend
+from rest_framework import mixins, permissions, status, viewsets
 from rest_framework.decorators import action
 from rest_framework.response import Response
 
+from .serializers import *
+
+
+class LoginView(auth_views.LoginView):
+    template_name = 'uncloud_auth/login.html'
+
+class LogoutView(auth_views.LogoutView):
+    pass
+#    template_name = 'uncloud_auth/logo.html'
+
+
 class UserViewSet(viewsets.GenericViewSet):
     permission_classes = [permissions.IsAuthenticated]
     serializer_class = UserSerializer
@@ -19,19 +32,29 @@ class UserViewSet(viewsets.GenericViewSet):
         serializer = self.get_serializer(user, context = {'request': request})
         return Response(serializer.data)
 
-    def create(self, request):
+    @action(detail=False, methods=['post'])
-        """
+    def change_email(self, request):
-        Modify existing user data
+        serializer = self.get_serializer(
-        """
+            request.user, data=request.data, context={'request': request}
-
+        )
-        user = request.user
-        serializer = self.get_serializer(user,
-                                         context = {'request': request},
-                                         data=request.data)
         serializer.is_valid(raise_exception=True)
         serializer.save()
         return Response(serializer.data)
 
+
+class AccountManagementViewSet(mixins.CreateModelMixin, viewsets.GenericViewSet):
+    serializer_class = UserRegistrationSerializer
+
+    def create(self, request, *args, **kwargs):
+        serializer = self.get_serializer(data=request.data)
+        serializer.is_valid(raise_exception=True)
+        self.perform_create(serializer)
+        headers = self.get_success_headers(serializer.data)
+        return Response(
+            serializer.data, status=status.HTTP_201_CREATED, headers=headers
+        )
+
+
 class AdminUserViewSet(viewsets.ReadOnlyModelViewSet):
     permission_classes = [permissions.IsAdminUser]
 

uncloud_net/admin.py

@@ -1,3 +1,7 @@
 from django.contrib import admin
 
-# Register your models here.
+from .models import *
+
+
+for m in [ ReverseDNSEntry, WireGuardVPNPool, WireGuardVPN ]:
+    admin.site.register(m)

uncloud_net/forms.py

@@ -0,0 +1,11 @@
+from django import forms
+
+from .models import *
+from .selectors import *
+
+class WireGuardVPNForm(forms.ModelForm):
+    network_size = forms.ChoiceField(choices=allowed_vpn_network_reservation_size)
+
+    class Meta:
+        model = WireGuardVPN
+        fields = [ "wireguard_public_key" ]

uncloud_net/migrations/0001_initial.py

@@ -1,11 +1,9 @@
-# Generated by Django 3.0.6 on 2020-08-01 16:38
+# Generated by Django 3.1 on 2020-12-13 13:42
 
 from django.conf import settings
-import django.contrib.postgres.fields.jsonb
 import django.core.validators
 from django.db import migrations, models
 import django.db.models.deletion
-import uuid
 
 
 class Migration(migrations.Migration):
@@ -14,7 +12,6 @@ class Migration(migrations.Migration):
 
     dependencies = [
         migrations.swappable_dependency(settings.AUTH_USER_MODEL),
-        ('uncloud_pay', '__first__'),
     ]
 
     operations = [
@@ -25,45 +22,41 @@ class Migration(migrations.Migration):
             ],
         ),
         migrations.CreateModel(
-            name='VPNPool',
+            name='WireGuardVPNPool',
-            fields=[
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
-                ('uuid', models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True, serialize=False)),
-                ('network', models.GenericIPAddressField(unique=True)),
-                ('network_size', models.IntegerField(validators=[django.core.validators.MinValueValidator(0), django.core.validators.MaxValueValidator(128)])),
-                ('subnetwork_size', models.IntegerField(validators=[django.core.validators.MinValueValidator(0), django.core.validators.MaxValueValidator(128)])),
-                ('vpn_hostname', models.CharField(max_length=256)),
-                ('wireguard_private_key', models.CharField(max_length=48)),
-            ],
-            options={
-                'abstract': False,
-            },
-        ),
-        migrations.CreateModel(
-            name='VPNNetworkReservation',
-            fields=[
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
-                ('address', models.GenericIPAddressField(primary_key=True, serialize=False)),
-                ('status', models.CharField(choices=[('used', 'used'), ('free', 'free')], default='used', max_length=256)),
-                ('vpnpool', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='uncloud_net.VPNPool')),
-            ],
-            options={
-                'abstract': False,
-            },
-        ),
-        migrations.CreateModel(
-            name='VPNNetwork',
             fields=[
                 ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
+                ('network', models.GenericIPAddressField(unique=True)),
-                ('status', models.CharField(choices=[('PENDING', 'Pending'), ('AWAITING_PAYMENT', 'Awaiting payment'), ('BEING_CREATED', 'Being created'), ('SCHEDULED', 'Scheduled'), ('ACTIVE', 'Active'), ('MODIFYING', 'Modifying'), ('DELETED', 'Deleted'), ('DISABLED', 'Disabled'), ('UNUSABLE', 'Unusable')], default='AWAITING_PAYMENT', max_length=32)),
+                ('network_mask', models.IntegerField(validators=[django.core.validators.MinValueValidator(0), django.core.validators.MaxValueValidator(128)])),
-                ('wireguard_public_key', models.CharField(max_length=48)),
+                ('subnetwork_mask', models.IntegerField(validators=[django.core.validators.MinValueValidator(0), django.core.validators.MaxValueValidator(128)])),
-                ('network', models.ForeignKey(editable=False, on_delete=django.db.models.deletion.CASCADE, to='uncloud_net.VPNNetworkReservation')),
+                ('vpn_server_hostname', models.CharField(max_length=256)),
-                ('order', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order')),
+                ('wireguard_private_key', models.CharField(max_length=48)),
-                ('owner', models.ForeignKey(editable=False, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+            ],
+        ),
+        migrations.CreateModel(
+            name='WireGuardVPNFreeLeases',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('pool_index', models.IntegerField(unique=True)),
+                ('vpnpool', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='uncloud_net.wireguardvpnpool')),
+            ],
+        ),
+        migrations.CreateModel(
+            name='WireGuardVPN',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('pool_index', models.IntegerField(unique=True)),
+                ('wireguard_public_key', models.CharField(max_length=48)),
+                ('owner', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+                ('vpnpool', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='uncloud_net.wireguardvpnpool')),
+            ],
+        ),
+        migrations.CreateModel(
+            name='ReverseDNSEntry',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('ip_address', models.GenericIPAddressField(unique=True)),
+                ('name', models.CharField(max_length=253)),
+                ('owner', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
             ],
-            options={
-                'abstract': False,
-            },
         ),
     ]

uncloud_net/migrations/0002_auto_20200801_2332.py

@@ -1,20 +0,0 @@
-# Generated by Django 3.0.8 on 2020-08-01 23:32
-
-from django.db import migrations, models
-import django.db.models.deletion
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0003_auto_20200801_2332'),
-        ('uncloud_net', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='vpnnetwork',
-            name='order',
-            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order'),
-        ),
-    ]

uncloud_net/migrations/0002_wireguardvpnpool_wireguard_public_key.py

@@ -0,0 +1,19 @@
+# Generated by Django 3.1 on 2020-12-13 17:04
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_net', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='wireguardvpnpool',
+            name='wireguard_public_key',
+            field=models.CharField(default='', max_length=48),
+            preserve_default=False,
+        ),
+    ]

uncloud_net/migrations/0003_auto_20200808_1953.py

@@ -1,28 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 19:53
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_net', '0002_auto_20200801_2332'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='vpnnetwork',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-        migrations.AlterField(
-            model_name='vpnnetworkreservation',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-        migrations.AlterField(
-            model_name='vpnpool',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-    ]

uncloud_net/migrations/0003_wireguardvpnpool_wg_name.py

@@ -0,0 +1,19 @@
+# Generated by Django 3.1 on 2020-12-13 17:31
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_net', '0002_wireguardvpnpool_wireguard_public_key'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='wireguardvpnpool',
+            name='wg_name',
+            field=models.CharField(default='wg0', max_length=15),
+            preserve_default=False,
+        ),
+    ]

uncloud_net/migrations/0004_auto_20201213_1734.py

@@ -0,0 +1,17 @@
+# Generated by Django 3.1 on 2020-12-13 17:34
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_net', '0003_wireguardvpnpool_wg_name'),
+    ]
+
+    operations = [
+        migrations.AddConstraint(
+            model_name='wireguardvpnpool',
+            constraint=models.UniqueConstraint(fields=('wg_name', 'vpn_server_hostname'), name='unique_interface_name_per_host'),
+        ),
+    ]

uncloud_net/migrations/0005_auto_20201220_1837.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.1 on 2020-12-20 18:37
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_net', '0004_auto_20201213_1734'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='wireguardvpn',
+            name='wireguard_public_key',
+            field=models.CharField(max_length=48, unique=True),
+        ),
+    ]

uncloud_net/migrations/0006_auto_20201224_1626.py

@@ -0,0 +1,17 @@
+# Generated by Django 3.1 on 2020-12-24 16:26
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_net', '0005_auto_20201220_1837'),
+    ]
+
+    operations = [
+        migrations.AddConstraint(
+            model_name='wireguardvpn',
+            constraint=models.UniqueConstraint(fields=('vpnpool', 'wireguard_public_key'), name='wg_key_unique_per_pool'),
+        ),
+    ]

uncloud_net/models.py

@@ -4,184 +4,189 @@ import ipaddress
 from django.db import models
 from django.contrib.auth import get_user_model
 from django.core.validators import MinValueValidator, MaxValueValidator
+from django.core.exceptions import FieldError, ValidationError
 
+from uncloud_pay.models import Order
 
-from uncloud_pay.models import Product, RecurringPeriod
+class WireGuardVPNPool(models.Model):
-from uncloud.models import UncloudModel, UncloudStatus
-
-
-class MACAdress(models.Model):
-    default_prefix = 0x420000000000
-
-class VPNPool(UncloudModel):
     """
     Network address pools from which VPNs can be created
     """
 
-    uuid = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
+    class Meta:
+        constraints = [
+            models.UniqueConstraint(fields=['wg_name', 'vpn_server_hostname' ],
+                                    name='unique_interface_name_per_host')
+            ]
+
+
+    # Linux interface naming is restricing to max 15 characters
+    wg_name = models.CharField(max_length=15)
 
     network = models.GenericIPAddressField(unique=True)
-    network_size = models.IntegerField(validators=[MinValueValidator(0),
+    network_mask = models.IntegerField(validators=[MinValueValidator(0),
                                                    MaxValueValidator(128)])
 
-    subnetwork_size = models.IntegerField(validators=[
+    subnetwork_mask = models.IntegerField(validators=[
-                                              MinValueValidator(0),
+        MinValueValidator(0),
-                                              MaxValueValidator(128)
+        MaxValueValidator(128)
-                                          ])
+    ])
-
-    vpn_hostname = models.CharField(max_length=256)
 
+    vpn_server_hostname = models.CharField(max_length=256)
     wireguard_private_key = models.CharField(max_length=48)
+    wireguard_public_key = models.CharField(max_length=48)
 
     @property
-    def num_maximum_networks(self):
+    def max_pool_index(self):
         """
-        sample:
+        Return the highest possible network / last network id
-        network_size = 40
-        subnetwork_size = 48
-        maximum_networks = 2^(48-40)
-
-        2nd sample:
-        network_size = 8
-        subnetwork_size = 24
-        maximum_networks = 2^(24-8)
         """
 
-        return 2**(self.subnetwork_size - self.network_size)
+        bits = self.subnetwork_mask - self.network_mask
+
+        return (2**bits)-1
 
     @property
-    def used_networks(self):
+    def ip_network(self):
-        return self.vpnnetworkreservation_set.filter(vpnpool=self, status='used')
+        return ipaddress.ip_network(f"{self.network}/{self.network_mask}")
 
-    @property
+    def __str__(self):
-    def free_networks(self):
+        return f"{self.ip_network} (subnets: /{self.subnetwork_mask})"
-        return self.vpnnetworkreservation_set.filter(vpnpool=self, status='free')
-
-    @property
-    def num_used_networks(self):
-        return len(self.used_networks)
-
-    @property
-    def num_free_networks(self):
-        return self.num_maximum_networks - self.num_used_networks + len(self.free_networks)
-
-    @property
-    def next_free_network(self):
-        if self.num_free_networks == 0:
-            # FIXME: use right exception
-            raise Exception("No free networks")
-
-        if len(self.free_networks) > 0:
-            return self.free_networks[0].address
-
-        if len(self.used_networks) > 0:
-            """
-            sample:
-
-            pool = 2a0a:e5c1:200::/40
-            last_used = 2a0a:e5c1:204::/48
-
-            next:
-            """
-
-            last_net = ipaddress.ip_network(self.used_networks.last().address)
-            last_net_ip = last_net[0]
-
-            if last_net_ip.version == 6:
-                offset_to_next = 2**(128 - self.subnetwork_size)
-            elif last_net_ip.version == 4:
-                offset_to_next = 2**(32 - self.subnetwork_size)
-
-            next_net_ip = last_net_ip + offset_to_next
-
-            return str(next_net_ip)
-        else:
-            # first network to be created
-            return self.network
-
-    @property
-    def wireguard_config_filename(self):
-        return '/etc/wireguard/{}.conf'.format(self.network)
 
     @property
     def wireguard_config(self):
-        wireguard_config = [
+        wireguard_config = [ f"[Interface]\nListenPort = 51820\nPrivateKey = {self.wireguard_private_key}\n" ]
-            """
-[Interface]
-ListenPort = 51820
-PrivateKey = {privatekey}
-""".format(privatekey=self.wireguard_private_key) ]
 
         peers = []
 
-        for reservation in self.vpnnetworkreservation_set.filter(status='used'):
+        for vpn in self.wireguardvpn_set.all():
-            public_key = reservation.vpnnetwork_set.first().wireguard_public_key
+            public_key = vpn.wireguard_public_key
-            peer_network = "{}/{}".format(reservation.address, self.subnetwork_size)
+            peer_network = f"{vpn.address}/{self.subnetwork_mask}"
-            owner = reservation.vpnnetwork_set.first().owner
+            owner = vpn.owner
 
-            peers.append("""
+            peers.append(f"# Owner: {owner}\n[Peer]\nPublicKey = {public_key}\nAllowedIPs = {peer_network}\n\n")
-# Owner: {owner}
-[Peer]
-PublicKey = {public_key}
-AllowedIPs = {peer_network}
-""".format(
-    owner=owner,
-    public_key=public_key,
-    peer_network=peer_network))
 
         wireguard_config.extend(peers)
 
         return "\n".join(wireguard_config)
 
 
-    def configure_wireguard_vpnserver(self):
+class WireGuardVPN(models.Model):
+    """
+    Created VPNNetworks
+    """
+    owner = models.ForeignKey(get_user_model(),
+            on_delete=models.CASCADE)
+    vpnpool = models.ForeignKey(WireGuardVPNPool,
+                                 on_delete=models.CASCADE)
+
+    pool_index = models.IntegerField(unique=True)
+
+    wireguard_public_key = models.CharField(max_length=48, unique=True)
+
+    class Meta:
+        constraints = [
+            models.UniqueConstraint(fields=['vpnpool', 'wireguard_public_key'],
+                                    name='wg_key_unique_per_pool')
+        ]
+
+
+    @property
+    def network_mask(self):
+        return self.vpnpool.subnetwork_mask
+
+    @property
+    def vpn_server(self):
+        return self.vpnpool.vpn_server_hostname
+
+    @property
+    def vpn_server_public_key(self):
+        return self.vpnpool.wireguard_public_key
+
+    @property
+    def address(self):
         """
-        This method is designed to run as a celery task and should
+        Locate the correct subnet in the supernet
-        not be called directly from the web
+
+        First get the network itself
+
         """
 
-        # subprocess, ssh
+        net = self.vpnpool.ip_network
+        subnet = net[(2**(128-self.vpnpool.subnetwork_mask)) * self.pool_index]
 
+        return str(subnet)
+
+    def __str__(self):
+        return f"{self.address} ({self.pool_index})"
+
+
+class WireGuardVPNFreeLeases(models.Model):
+    """
+    Previously used VPNNetworks
+    """
+    vpnpool = models.ForeignKey(WireGuardVPNPool,
+                                 on_delete=models.CASCADE)
+
+    pool_index = models.IntegerField(unique=True)
+
+################################################################################
+
+class MACAdress(models.Model):
+    default_prefix = 0x420000000000
+
+
+class ReverseDNSEntry(models.Model):
+    """
+    A reverse DNS entry
+    """
+    owner = models.ForeignKey(get_user_model(),
+            on_delete=models.CASCADE)
+
+    ip_address = models.GenericIPAddressField(null=False, unique=True)
+
+    name = models.CharField(max_length=253, null=False)
+
+    @property
+    def reverse_pointer(self):
+        return ipaddress.ip_address(self.ip_address).reverse_pointer
+
+    def implement(self):
+        """
+        The implement function implements the change
+        """
+
+        # Get all DNS entries (?) / update this DNS entry
+        # convert to DNS name
+        #
         pass
 
 
-class VPNNetworkReservation(UncloudModel):
+    def save(self, *args, **kwargs):
-    """
+        # Product.objects.filter(config__parameters__contains='reverse_dns_network')
-     This class tracks the used VPN networks. It will be deleted, when the product is cancelled.
+        # FIXME: check if order is still active / not replaced
-     """
-    vpnpool = models.ForeignKey(VPNPool,
-                                 on_delete=models.CASCADE)
 
-    address = models.GenericIPAddressField(primary_key=True)
+        allowed = False
+        product = None
 
-    status = models.CharField(max_length=256,
+        for order in Order.objects.filter(config__parameters__reverse_dns_network__isnull=False,
-                              default='used',
+                             owner=self.owner):
-                              choices = (
+            network = order.config['parameters']['reverse_dns_network']
-                                  ('used', 'used'),
+
-                                  ('free', 'free')
+            net = ipaddress.ip_network(network)
-                              )
+            addr = ipaddress.ip_address(self.ip_address)
-    )
+
+            if addr in net:
+                allowed = True
+                product = order.product
+                break
 
 
-class VPNNetwork(Product):
+        if not allowed:
-    """
+            raise ValidationError(f"User {self.owner} does not have the right to create reverse DNS entry for {self.ip_address}")
-    A selected network. Used for tracking reservations / used networks
-    """
-    network = models.ForeignKey(VPNNetworkReservation,
-                               on_delete=models.CASCADE,
-                               editable=False)
 
-    wireguard_public_key = models.CharField(max_length=48)
-
-    default_recurring_period = RecurringPeriod.PER_365D
-
-    @property
-    def recurring_price(self):
-        return 120
-
-
-    def delete(self, *args, **kwargs):
-        self.network.status = 'free'
-        self.network.save()
         super().save(*args, **kwargs)
-        print("deleted {}".format(self))
+
+
+    def __str__(self):
+        return f"{self.ip_address} - {self.name}"

uncloud_net/selectors.py

@@ -0,0 +1,43 @@
+from django.db import transaction
+from django.db.models import Count, F
+from .models import *
+
+def get_suitable_pools(subnetwork_mask):
+    """
+    Find suitable pools for a certain network size.
+
+    First, filter for all pools that offer the requested subnetwork_size.
+
+    Then find those pools that are not fully exhausted:
+
+    The number of available networks in a pool is 2^(subnetwork_size-network_size.
+
+    The number of available networks in a pool is given by the number of VPNNetworkreservations.
+
+    """
+
+    return WireGuardVPNPool.objects.annotate(
+        num_reservations=Count('wireguardvpn'),
+        max_reservations=2**(F('subnetwork_mask')-F('network_mask'))).filter(
+            num_reservations__lt=F('max_reservations'),
+            subnetwork_mask=subnetwork_mask)
+
+
+def allowed_vpn_network_reservation_size():
+    """
+    Find all possible sizes of subnetworks that are available.
+
+    Select all pools with free networks.
+
+    Get their subnetwork sizes, reduce to a set
+
+    """
+
+    pools = WireGuardVPNPool.objects.annotate(num_reservations=Count('wireguardvpn'),
+                                     max_reservations=2**(F('subnetwork_mask')-F('network_mask'))).filter(
+                                         num_reservations__lt=F('max_reservations'))
+
+    # Need to return set of tuples, see
+    # https://docs.djangoproject.com/en/3.1/ref/models/fields/#field-choices
+#    return set([ (pool.subnetwork_mask, pool.subnetwork_mask) for pool in pools ])
+    return set([pool.subnetwork_mask  for pool in pools ])

uncloud_net/serializers.py

@@ -5,96 +5,53 @@ from django.utils.translation import gettext_lazy as _
 from rest_framework import serializers
 
 from .models import *
+from .services import *
+from .selectors import *
+
+
+class WireGuardVPNSerializer(serializers.ModelSerializer):
+    address = serializers.CharField(read_only=True)
+    vpn_server = serializers.CharField(read_only=True)
+    vpn_server_public_key = serializers.CharField(read_only=True)
+    network_mask = serializers.IntegerField()
 
-class VPNPoolSerializer(serializers.ModelSerializer):
     class Meta:
-        model = VPNPool
+        model = WireGuardVPN
-        fields = '__all__'
+        fields = [ 'wireguard_public_key', 'address', 'network_mask', 'vpn_server',
+                   'vpn_server_public_key' ]
 
-class VPNNetworkReservationSerializer(serializers.ModelSerializer):
+        extra_kwargs = {
-    class Meta:
+            'network_mask': {'write_only': True }
-        model = VPNNetworkReservation
+        }
-        fields = '__all__'
 
 
-class VPNNetworkSerializer(serializers.ModelSerializer):
+    def validate_network_mask(self, value):
-    class Meta:
+        msg = _(f"No pool for network size {value}")
-        model = VPNNetwork
+        sizes = allowed_vpn_network_reservation_size()
-        fields = '__all__'
 
-    # This is required for finding the VPN pool, but does not
+        if not value in sizes:
-    # exist in the model
-    network_size = serializers.IntegerField(min_value=0,
-                                            max_value=128,
-                                            write_only=True)
-
-    def validate_wireguard_public_key(self, value):
-        msg = _("Supplied key is not a valid wireguard public key")
-
-        """ FIXME: verify that this does not create broken wireguard config files,
-        i.e. contains \n or similar!
-        We might even need to be more strict to not break wireguard...
-        """
-
-        try:
-            base64.standard_b64decode(value)
-        except Exception as e:
-            raise serializers.ValidationError(msg)
-
-        if '\n' in value:
             raise serializers.ValidationError(msg)
 
         return value
 
-    def validate(self, data):
+    def validate_wireguard_public_key(self, value):
+        msg = _("Supplied key is not a valid wireguard public key")
 
-        # FIXME: filter for status = active or similar
+        """
-        all_pools = VPNPool.objects.all()
+        Verify wireguard key.
-        sizes = [ p.subnetwork_size for p in all_pools ]
+        See https://lists.zx2c4.com/pipermail/wireguard/2020-December/006221.html
+        """
 
-        pools = VPNPool.objects.filter(subnetwork_size=data['network_size'])
+        try:
-
+            decoded_key = base64.standard_b64decode(value)
-        if len(pools) == 0:
+        except Exception as e:
-            msg = _("No pool available for networks with size = {}. Available are: {}".format(data['network_size'], sizes))
             raise serializers.ValidationError(msg)
 
-        return data
+        if not len(decoded_key) == 32:
+            raise serializers.ValidationError(msg)
 
-    def create(self, validated_data):
+        return value
-        """
-        Creating a new vpnnetwork - there are a couple of race conditions,
-        especially when run in parallel.
-
-        What we should be doing:
-
-        - create a reservation race free
-        - map the reservation to a network (?)
-        """
-
-        pools = VPNPool.objects.filter(subnetwork_size=validated_data['network_size'])
-
-        vpn_network = None
-
-        for pool in pools:
-            if pool.num_free_networks > 0:
-                next_address = pool.next_free_network
-
-                reservation, created = VPNNetworkReservation.objects.update_or_create(
-                    vpnpool=pool, address=next_address,
-                    defaults = {
-                        'status': 'used'
-                    })
-
-                vpn_network = VPNNetwork.objects.create(
-                    owner=self.context['request'].user,
-                    network=reservation,
-                    wireguard_public_key=validated_data['wireguard_public_key']
-                )
-
-                break
-        if not vpn_network:
-            # FIXME: use correct exception
-            raise Exception("Did not find any free pool")
 
 
-        return vpn_network
+class WireGuardVPNSizesSerializer(serializers.Serializer):
+    size = serializers.IntegerField(min_value=0, max_value=128)

uncloud_net/services.py

@@ -0,0 +1,47 @@
+from django.db import transaction
+
+from .models    import *
+from .selectors import *
+from .tasks     import *
+
+@transaction.atomic
+def create_wireguard_vpn(owner, public_key, network_mask):
+
+    pool = get_suitable_pools(network_mask)[0]
+    count = pool.wireguardvpn_set.count()
+
+    # Try re-using previously used networks first
+    try:
+        free_lease = WireGuardVPNFreeLeases.objects.get(vpnpool=pool)
+
+        vpn = WireGuardVPN.objects.create(owner=owner,
+                                           vpnpool=pool,
+                                           pool_index=free_lease.pool_index,
+                                           wireguard_public_key=public_key)
+
+        free_lease.delete()
+
+    except WireGuardVPNFreeLeases.DoesNotExist:
+
+        # First object
+        if count == 0:
+            vpn = WireGuardVPN.objects.create(owner=owner,
+                                              vpnpool=pool,
+                                              pool_index=0,
+                                              wireguard_public_key=public_key)
+
+        else: # Select last network and try +1 it
+            last_net = WireGuardVPN.objects.filter(vpnpool=pool).order_by('pool_index').last()
+
+            next_index = last_net.pool_index + 1
+
+            if next_index <= pool.max_pool_index:
+                vpn = WireGuardVPN.objects.create(owner=owner,
+                                                   vpnpool=pool,
+                                                   pool_index=next_index,
+                                                   wireguard_public_key=public_key)
+
+
+
+    configure_wireguard_server(pool)
+    return vpn

uncloud_net/tasks.py

@@ -0,0 +1,60 @@
+from celery import shared_task
+from .models import *
+
+from uncloud.models import UncloudTask
+
+import os
+import subprocess
+import logging
+import uuid
+
+log = logging.getLogger(__name__)
+
+@shared_task
+def whereami():
+    print(os.uname())
+    return os.uname()
+
+def configure_wireguard_server(wireguardvpnpool):
+    """
+    - Create wireguard config (DB query -> string)
+    - Submit config to cdist worker
+    - Change config locally on worker / commit / shared
+
+    """
+
+    config = wireguardvpnpool.wireguard_config
+    server = wireguardvpnpool.vpn_server_hostname
+
+    log.info(f"Configuring VPN server {server} (async)")
+
+    task_id = uuid.UUID(cdist_configure_wireguard_server.apply_async((config, server)).id)
+    UncloudTask.objects.create(task_id=task_id)
+
+
+@shared_task
+def cdist_configure_wireguard_server(config, server):
+    """
+    Create config and configure server.
+
+    To be executed on the cdist workers.
+    """
+
+    dirname= "/home/app/.cdist/type/__ungleich_wireguard/files/"
+    fname = os.path.join(dirname,server)
+
+    log.info(f"Configuring VPN server {server} (on cdist host)")
+    with open(fname, "w") as fd:
+        fd.write(config)
+
+    log.debug("git committing wireguard changes")
+    subprocess.run(f"cd {dirname} && git pull && git add {server} && git commit -m 'Updating config for {server}' && git push",
+                   shell=True, check=True)
+
+    log.debug(f"Configuring VPN server {server} with cdist")
+    subprocess.run(f"cdist config {server}", shell=True, check=True)
+
+    # FIXME:
+    # ensure logs are on the server
+    # ensure exit codes are known
+    return True

uncloud_net/templates/uncloud_net/wireguardvpn_form.html

@@ -0,0 +1,25 @@
+{% extends 'uncloud/base.html' %}
+
+{% block body %}
+<div class="container">
+  <div class="row">
+    <div class="col">
+      <h1>
+      <h1>Create a VPN Network</h1>
+      <p>
+        Create a new wireguard based VPN network.
+      </p>
+
+    </div>
+
+    <div class="col">
+      <form method="post">
+        {% csrf_token %}
+        {{ form }}
+        <input type="submit" value="Submit">
+      </form>
+    </div>
+  </div>
+</div>
+
+{% endblock %}

uncloud_net/tests.py

@@ -3,12 +3,19 @@ from rest_framework.test import APIRequestFactory, force_authenticate
 
 from rest_framework.reverse import reverse
 from django.contrib.auth import get_user_model
-from django.core.exceptions import ValidationError
+from django.core.exceptions import ValidationError, FieldError
 
 from .views import *
 from .models import *
 
 from uncloud_pay.models import BillingAddress, Order
+from uncloud.models import UncloudNetwork
+
+class UncloudNetworkTests(TestCase):
+    def test_invalid_IPv4_network(self):
+        with self.assertRaises(FieldError):
+            UncloudNetwork.objects.create(network_address="192.168.1.0",
+                                          network_mask=33)
 
 class VPNTests(TestCase):
     def setUp(self):
@@ -57,36 +64,37 @@ class VPNTests(TestCase):
         # No assert needed
         pool = VPNPool.objects.get(network=self.pool_network2)
 
-    def test_create_vpn(self):
+    # def test_create_vpn(self):
-        url = reverse("vpnnetwork-list")
+    #     url = reverse("vpnnetwork-list")
-        view = VPNNetworkViewSet.as_view({'post': 'create'})
+    #     view = VPNNetworkViewSet.as_view({'post': 'create'})
-        request = self.factory.post(url, { 'network_size': self.pool_subnetwork_size,
+    #     request = self.factory.post(url, { 'network_size': self.pool_subnetwork_size,
-                                           'wireguard_public_key': self.vpn_wireguard_public_key
+    #                                        'wireguard_public_key': self.vpn_wireguard_public_key
 
-        })
+    #     })
-        force_authenticate(request, user=self.user)
+    #     force_authenticate(request, user=self.user)
 
-        # we don't have a billing address -> should raise an error
-        with self.assertRaises(ValidationError):
-            response = view(request)
 
-        addr = BillingAddress.objects.get_or_create(
+    #     # we don't have a billing address -> should raise an error
-            owner=self.user,
+    #     # with self.assertRaises(ValidationError):
-            active=True,
+    #     #     response = view(request)
-            defaults={'organization': 'ungleich',
-                      'name': 'Nico Schottelius',
-                      'street': 'Hauptstrasse 14',
-                      'city': 'Luchsingen',
-                      'postal_code': '8775',
-                      'country': 'CH' }
-        )
 
-        # This should work now
+    #     addr = BillingAddress.objects.get_or_create(
-        response = view(request)
+    #         owner=self.user,
+    #         active=True,
+    #         defaults={'organization': 'ungleich',
+    #                   'name': 'Nico Schottelius',
+    #                   'street': 'Hauptstrasse 14',
+    #                   'city': 'Luchsingen',
+    #                   'postal_code': '8775',
+    #                   'country': 'CH' }
+    #     )
 
-        # Verify that an order was created successfully - there should only be one order at
+    #     # This should work now
-        # this point in time
+    #     response = view(request)
-        order = Order.objects.get(owner=self.user)
+
+    #     # Verify that an order was created successfully - there should only be one order at
+    #     # this point in time
+    #     order = Order.objects.get(owner=self.user)
 
 
     def tearDown(self):

uncloud_net/views.py

@@ -1,33 +1,70 @@
+from django.views.generic.edit import CreateView
+from django.contrib.auth.mixins import LoginRequiredMixin
+from django.contrib.messages.views import SuccessMessageMixin
+from rest_framework.response import Response
 
 from django.shortcuts import render
 
 from rest_framework import viewsets, permissions
 
-
 from .models      import *
 from .serializers import *
+from .selectors   import *
+from .services    import *
+from .forms       import *
+from .tasks       import *
 
-
+class WireGuardVPNViewSet(viewsets.ModelViewSet):
-class VPNPoolViewSet(viewsets.ModelViewSet):
+    serializer_class = WireGuardVPNSerializer
-    serializer_class = VPNPoolSerializer
-    permission_classes = [permissions.IsAdminUser]
-    queryset = VPNPool.objects.all()
-
-class VPNNetworkReservationViewSet(viewsets.ModelViewSet):
-    serializer_class = VPNNetworkReservationSerializer
-    permission_classes = [permissions.IsAdminUser]
-    queryset = VPNNetworkReservation.objects.all()
-
-
-class VPNNetworkViewSet(viewsets.ModelViewSet):
-    serializer_class = VPNNetworkSerializer
-#    permission_classes = [permissions.IsAdminUser]
     permission_classes = [permissions.IsAuthenticated]
 
     def get_queryset(self):
         if self.request.user.is_superuser:
-            obj = VPNNetwork.objects.all()
+            obj = WireGuardVPN.objects.all()
         else:
-            obj = VPNNetwork.objects.filter(owner=self.request.user)
+            obj = WireGuardVPN.objects.filter(owner=self.request.user)
 
         return obj
+
+    def create(self, request):
+        serializer = self.get_serializer(data=request.data)
+        serializer.is_valid(raise_exception=True)
+
+        vpn = create_wireguard_vpn(
+            owner=self.request.user,
+            public_key=serializer.validated_data['wireguard_public_key'],
+            network_mask=serializer.validated_data['network_mask']
+        )
+        configure_wireguard_server(vpn.vpnpool)
+        return Response(WireGuardVPNSerializer(vpn).data)
+
+
+class WireGuardVPNCreateView(LoginRequiredMixin, SuccessMessageMixin, CreateView):
+    model = WireGuardVPN
+
+    login_url = '/login/'
+    success_url = '/'
+    success_message = "%(network) was created successfully"
+
+    form_class = WireGuardVPNForm
+
+    def get_success_message(self, cleaned_data):
+        return self.success_message % dict(cleaned_data,
+                                           the_prefix = self.object.prefix)
+
+class WireGuardVPNSizes(viewsets.ViewSet):
+    def list(self, request):
+        sizes = allowed_vpn_network_reservation_size()
+        print(sizes)
+
+        sizes = [ { 'size': size } for size in sizes ]
+        print(sizes)
+
+        return Response(WireGuardVPNSizesSerializer(sizes, many=True).data)
+
+
+
+# class VPNPoolViewSet(viewsets.ModelViewSet):
+#     serializer_class = VPNPoolSerializer
+#     permission_classes = [permissions.IsAdminUser]
+#     queryset = VPNPool.objects.all()

uncloud_pay/__init__.py

@@ -1,250 +1 @@
-from django.utils.translation import gettext_lazy as _
-import decimal
 
-# Define DecimalField properties, used to represent amounts of money.
-AMOUNT_MAX_DIGITS=10
-AMOUNT_DECIMALS=2
-
-decimal.getcontext().prec = AMOUNT_DECIMALS
-
-# http://xml.coverpages.org/country3166.html
-COUNTRIES = (
-    ('AD', _('Andorra')),
-    ('AE', _('United Arab Emirates')),
-    ('AF', _('Afghanistan')),
-    ('AG', _('Antigua & Barbuda')),
-    ('AI', _('Anguilla')),
-    ('AL', _('Albania')),
-    ('AM', _('Armenia')),
-    ('AN', _('Netherlands Antilles')),
-    ('AO', _('Angola')),
-    ('AQ', _('Antarctica')),
-    ('AR', _('Argentina')),
-    ('AS', _('American Samoa')),
-    ('AT', _('Austria')),
-    ('AU', _('Australia')),
-    ('AW', _('Aruba')),
-    ('AZ', _('Azerbaijan')),
-    ('BA', _('Bosnia and Herzegovina')),
-    ('BB', _('Barbados')),
-    ('BD', _('Bangladesh')),
-    ('BE', _('Belgium')),
-    ('BF', _('Burkina Faso')),
-    ('BG', _('Bulgaria')),
-    ('BH', _('Bahrain')),
-    ('BI', _('Burundi')),
-    ('BJ', _('Benin')),
-    ('BM', _('Bermuda')),
-    ('BN', _('Brunei Darussalam')),
-    ('BO', _('Bolivia')),
-    ('BR', _('Brazil')),
-    ('BS', _('Bahama')),
-    ('BT', _('Bhutan')),
-    ('BV', _('Bouvet Island')),
-    ('BW', _('Botswana')),
-    ('BY', _('Belarus')),
-    ('BZ', _('Belize')),
-    ('CA', _('Canada')),
-    ('CC', _('Cocos (Keeling) Islands')),
-    ('CF', _('Central African Republic')),
-    ('CG', _('Congo')),
-    ('CH', _('Switzerland')),
-    ('CI', _('Ivory Coast')),
-    ('CK', _('Cook Iislands')),
-    ('CL', _('Chile')),
-    ('CM', _('Cameroon')),
-    ('CN', _('China')),
-    ('CO', _('Colombia')),
-    ('CR', _('Costa Rica')),
-    ('CU', _('Cuba')),
-    ('CV', _('Cape Verde')),
-    ('CX', _('Christmas Island')),
-    ('CY', _('Cyprus')),
-    ('CZ', _('Czech Republic')),
-    ('DE', _('Germany')),
-    ('DJ', _('Djibouti')),
-    ('DK', _('Denmark')),
-    ('DM', _('Dominica')),
-    ('DO', _('Dominican Republic')),
-    ('DZ', _('Algeria')),
-    ('EC', _('Ecuador')),
-    ('EE', _('Estonia')),
-    ('EG', _('Egypt')),
-    ('EH', _('Western Sahara')),
-    ('ER', _('Eritrea')),
-    ('ES', _('Spain')),
-    ('ET', _('Ethiopia')),
-    ('FI', _('Finland')),
-    ('FJ', _('Fiji')),
-    ('FK', _('Falkland Islands (Malvinas)')),
-    ('FM', _('Micronesia')),
-    ('FO', _('Faroe Islands')),
-    ('FR', _('France')),
-    ('FX', _('France, Metropolitan')),
-    ('GA', _('Gabon')),
-    ('GB', _('United Kingdom (Great Britain)')),
-    ('GD', _('Grenada')),
-    ('GE', _('Georgia')),
-    ('GF', _('French Guiana')),
-    ('GH', _('Ghana')),
-    ('GI', _('Gibraltar')),
-    ('GL', _('Greenland')),
-    ('GM', _('Gambia')),
-    ('GN', _('Guinea')),
-    ('GP', _('Guadeloupe')),
-    ('GQ', _('Equatorial Guinea')),
-    ('GR', _('Greece')),
-    ('GS', _('South Georgia and the South Sandwich Islands')),
-    ('GT', _('Guatemala')),
-    ('GU', _('Guam')),
-    ('GW', _('Guinea-Bissau')),
-    ('GY', _('Guyana')),
-    ('HK', _('Hong Kong')),
-    ('HM', _('Heard & McDonald Islands')),
-    ('HN', _('Honduras')),
-    ('HR', _('Croatia')),
-    ('HT', _('Haiti')),
-    ('HU', _('Hungary')),
-    ('ID', _('Indonesia')),
-    ('IE', _('Ireland')),
-    ('IL', _('Israel')),
-    ('IN', _('India')),
-    ('IO', _('British Indian Ocean Territory')),
-    ('IQ', _('Iraq')),
-    ('IR', _('Islamic Republic of Iran')),
-    ('IS', _('Iceland')),
-    ('IT', _('Italy')),
-    ('JM', _('Jamaica')),
-    ('JO', _('Jordan')),
-    ('JP', _('Japan')),
-    ('KE', _('Kenya')),
-    ('KG', _('Kyrgyzstan')),
-    ('KH', _('Cambodia')),
-    ('KI', _('Kiribati')),
-    ('KM', _('Comoros')),
-    ('KN', _('St. Kitts and Nevis')),
-    ('KP', _('Korea, Democratic People\'s Republic of')),
-    ('KR', _('Korea, Republic of')),
-    ('KW', _('Kuwait')),
-    ('KY', _('Cayman Islands')),
-    ('KZ', _('Kazakhstan')),
-    ('LA', _('Lao People\'s Democratic Republic')),
-    ('LB', _('Lebanon')),
-    ('LC', _('Saint Lucia')),
-    ('LI', _('Liechtenstein')),
-    ('LK', _('Sri Lanka')),
-    ('LR', _('Liberia')),
-    ('LS', _('Lesotho')),
-    ('LT', _('Lithuania')),
-    ('LU', _('Luxembourg')),
-    ('LV', _('Latvia')),
-    ('LY', _('Libyan Arab Jamahiriya')),
-    ('MA', _('Morocco')),
-    ('MC', _('Monaco')),
-    ('MD', _('Moldova, Republic of')),
-    ('MG', _('Madagascar')),
-    ('MH', _('Marshall Islands')),
-    ('ML', _('Mali')),
-    ('MN', _('Mongolia')),
-    ('MM', _('Myanmar')),
-    ('MO', _('Macau')),
-    ('MP', _('Northern Mariana Islands')),
-    ('MQ', _('Martinique')),
-    ('MR', _('Mauritania')),
-    ('MS', _('Monserrat')),
-    ('MT', _('Malta')),
-    ('MU', _('Mauritius')),
-    ('MV', _('Maldives')),
-    ('MW', _('Malawi')),
-    ('MX', _('Mexico')),
-    ('MY', _('Malaysia')),
-    ('MZ', _('Mozambique')),
-    ('NA', _('Namibia')),
-    ('NC', _('New Caledonia')),
-    ('NE', _('Niger')),
-    ('NF', _('Norfolk Island')),
-    ('NG', _('Nigeria')),
-    ('NI', _('Nicaragua')),
-    ('NL', _('Netherlands')),
-    ('NO', _('Norway')),
-    ('NP', _('Nepal')),
-    ('NR', _('Nauru')),
-    ('NU', _('Niue')),
-    ('NZ', _('New Zealand')),
-    ('OM', _('Oman')),
-    ('PA', _('Panama')),
-    ('PE', _('Peru')),
-    ('PF', _('French Polynesia')),
-    ('PG', _('Papua New Guinea')),
-    ('PH', _('Philippines')),
-    ('PK', _('Pakistan')),
-    ('PL', _('Poland')),
-    ('PM', _('St. Pierre & Miquelon')),
-    ('PN', _('Pitcairn')),
-    ('PR', _('Puerto Rico')),
-    ('PT', _('Portugal')),
-    ('PW', _('Palau')),
-    ('PY', _('Paraguay')),
-    ('QA', _('Qatar')),
-    ('RE', _('Reunion')),
-    ('RO', _('Romania')),
-    ('RU', _('Russian Federation')),
-    ('RW', _('Rwanda')),
-    ('SA', _('Saudi Arabia')),
-    ('SB', _('Solomon Islands')),
-    ('SC', _('Seychelles')),
-    ('SD', _('Sudan')),
-    ('SE', _('Sweden')),
-    ('SG', _('Singapore')),
-    ('SH', _('St. Helena')),
-    ('SI', _('Slovenia')),
-    ('SJ', _('Svalbard & Jan Mayen Islands')),
-    ('SK', _('Slovakia')),
-    ('SL', _('Sierra Leone')),
-    ('SM', _('San Marino')),
-    ('SN', _('Senegal')),
-    ('SO', _('Somalia')),
-    ('SR', _('Suriname')),
-    ('ST', _('Sao Tome & Principe')),
-    ('SV', _('El Salvador')),
-    ('SY', _('Syrian Arab Republic')),
-    ('SZ', _('Swaziland')),
-    ('TC', _('Turks & Caicos Islands')),
-    ('TD', _('Chad')),
-    ('TF', _('French Southern Territories')),
-    ('TG', _('Togo')),
-    ('TH', _('Thailand')),
-    ('TJ', _('Tajikistan')),
-    ('TK', _('Tokelau')),
-    ('TM', _('Turkmenistan')),
-    ('TN', _('Tunisia')),
-    ('TO', _('Tonga')),
-    ('TP', _('East Timor')),
-    ('TR', _('Turkey')),
-    ('TT', _('Trinidad & Tobago')),
-    ('TV', _('Tuvalu')),
-    ('TW', _('Taiwan, Province of China')),
-    ('TZ', _('Tanzania, United Republic of')),
-    ('UA', _('Ukraine')),
-    ('UG', _('Uganda')),
-    ('UM', _('United States Minor Outlying Islands')),
-    ('US', _('United States of America')),
-    ('UY', _('Uruguay')),
-    ('UZ', _('Uzbekistan')),
-    ('VA', _('Vatican City State (Holy See)')),
-    ('VC', _('St. Vincent & the Grenadines')),
-    ('VE', _('Venezuela')),
-    ('VG', _('British Virgin Islands')),
-    ('VI', _('United States Virgin Islands')),
-    ('VN', _('Viet Nam')),
-    ('VU', _('Vanuatu')),
-    ('WF', _('Wallis & Futuna Islands')),
-    ('WS', _('Samoa')),
-    ('YE', _('Yemen')),
-    ('YT', _('Mayotte')),
-    ('YU', _('Yugoslavia')),
-    ('ZA', _('South Africa')),
-    ('ZM', _('Zambia')),
-    ('ZR', _('Zaire')),
-    ('ZW', _('Zimbabwe')),
-)

uncloud_pay/admin.py

@@ -1,6 +1,7 @@
 from django.contrib import admin
 from django.template.response import TemplateResponse
 from django.urls import path
+from django.shortcuts import render
 from django.conf.urls import url
 
 from uncloud_pay.views import BillViewSet
@@ -10,19 +11,17 @@ from django.http import FileResponse
 from django.template.loader import render_to_string
 
 
-from uncloud_pay.models import Bill, Order, BillRecord, BillingAddress
+from uncloud_pay.models import *
 
 
 class BillRecordInline(admin.TabularInline):
-#    model = Bill.bill_records.through
     model = BillRecord
 
-# AT some point in the future: expose REPLACED and orders that depend on us
+class RecurringPeriodInline(admin.TabularInline):
-# class OrderInline(admin.TabularInline):
+    model = ProductToRecurringPeriod
-#     model = Order
+
-#     fk_name = "replaces"
+class ProductAdmin(admin.ModelAdmin):
-# class OrderAdmin(admin.ModelAdmin):
+    inlines = [ RecurringPeriodInline ]
-#     inlines = [ OrderInline ]
 
 class BillAdmin(admin.ModelAdmin):
     inlines = [ BillRecordInline ]
@@ -36,12 +35,13 @@ class BillAdmin(admin.ModelAdmin):
         pat = lambda regex, fn: url(regex, self.admin_site.admin_view(fn), name='%s_%s' % (info, fn.__name__))
 
         url_patterns = [
-            pat(r'^([0-9]+)/as_pdf/$', self.my_view),
+            pat(r'^([0-9]+)/as_pdf/$', self.as_pdf),
+            pat(r'^([0-9]+)/as_html/$', self.as_html),
         ] + super().get_urls()
 
         return url_patterns
 
-    def my_view(self, request, object_id):
+    def as_pdf(self, request, object_id):
         bill = self.get_object(request, object_id=object_id)
         print(bill)
 
@@ -59,21 +59,34 @@ class BillAdmin(admin.ModelAdmin):
 
         return response
 
-        # ...
+    def as_html(self, request, object_id):
-        context = dict(
+        bill = self.get_object(request, object_id=object_id)
-           # Include common variables for rendering the admin template.
-           self.admin_site.each_context(request),
-           # Anything else you want in the context...
-#           key=value,
-        )
 
-        #return TemplateResponse(request, "admin/change_list.html", context)
+        if bill is None:
+            raise self._get_404_exception(object_id)
+
+        return render(request, 'bill.html.j2',
+                      {'bill': bill,
+                       'bill_records': bill.billrecord_set.all()
+                      })
+
+
+        bill_html = render_to_string("bill.html.j2", {'bill': bill,
+                                                      'bill_records': bill.billrecord_set.all()
+        })
+
+        bytestring_to_pdf(bill_html.encode('utf-8'), output_file)
+        response = FileResponse(output_file, content_type="application/pdf")
+
+        response['Content-Disposition'] = f'filename="bill_{bill}.pdf"'
+
+        return HttpResponse(template.render(context, request))
+        return response
 
 
 admin.site.register(Bill, BillAdmin)
-admin.site.register(Order)
+admin.site.register(ProductToRecurringPeriod)
-admin.site.register(BillRecord)
+admin.site.register(Product, ProductAdmin)
-admin.site.register(BillingAddress)
 
-
+for m in [ Order, BillRecord, BillingAddress, RecurringPeriod, VATRate, StripeCustomer ]:
-#admin.site.register(Order, OrderAdmin)
+    admin.site.register(m)

uncloud_pay/management/commands/.gitignore

@@ -0,0 +1,2 @@
+# Customer tests
+customer-*.py

uncloud_pay/management/commands/add-opennebula-vm-orders.py

@@ -1,32 +1,38 @@
-from django.core.management.base import BaseCommand
+import datetime
-from django.contrib.auth import get_user_model
-
-from django.utils import timezone
-from datetime import datetime, timedelta
-
-from uncloud_pay.models import *
-from uncloud_vm.models import *
-
 import sys
 
+from django.contrib.auth import get_user_model
+from django.core.management.base import BaseCommand
+from django.utils import timezone
+
+from uncloud_pay.models import (
+    BillingAddress
+)
+from uncloud_vm.models import (
+    VMDiskType, VMProduct
+)
+
+
 def vm_price_2020(cpu=1, ram=2, v6only=False):
     if v6only:
         discount = 9
     else:
         discount = 0
 
-    return cpu*3 + ram*4 - discount
+    return cpu * 3 + ram * 4 - discount
+
 
 def disk_price_2020(size_in_gb, disk_type):
     if disk_type == VMDiskType.CEPH_SSD:
-        price = 3.5/10
+        price = 3.5 / 10
     elif disk_type == VMDiskType.CEPH_HDD:
-        price = 1.5/100
+        price = 1.5 / 100
     else:
         raise Exception("not yet defined price")
 
     return size_in_gb * price
 
+
 class Command(BaseCommand):
     help = 'Adding VMs / creating orders for user'
 
@@ -40,109 +46,107 @@ class Command(BaseCommand):
             owner=user,
             active=True,
             defaults={'organization': 'Undefined organisation',
-                      'name': 'Undefined name',
+                      'full_name': 'Undefined name',
                       'street': 'Undefined Street',
                       'city': 'Undefined city',
                       'postal_code': '8750',
                       'country': 'CH',
                       'active': True
-            }
+                      }
         )
 
+        # 25206 + SSD
+        vm25206 = VMProduct.objects.create(name="one-25206", cores=1,
+                                           ram_in_gb=4, owner=user)
+        vm25206.create_order_at(
+            timezone.make_aware(datetime.datetime(2020, 3, 3)))
 
-        # 25206
+        # vm25206_ssd = VMDiskProduct.objects.create(vm=vm25206, owner=user, size_in_gb=30)
-        vm25206 = VMProduct(name="OpenNebula 25206",
+        # vm25206_ssd.create_order_at(timezone.make_aware(datetime.datetime(2020,3,3)))
-                            cores=1,
-                            ram_in_gb=4,
-                            owner=user)
-
-        vm25206.create_order_at(timezone.make_aware(datetime.datetime(2020,3,3)))
-        vm25206.save()
-
-        vm25206_ssd = VMDiskProduct(vm=vm25206,
-                                    owner=user,
-                                    size_in_gb=30)
-
-        vm25206_ssd.create_order_at(timezone.make_aware(datetime.datetime(2020,3,3)))
-        vm25206_ssd.save()
-
 
         # change 1
         vm25206.cores = 2
         vm25206.ram_in_gb = 8
         vm25206.save()
-        vm25206.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,4,17)))
+        vm25206.create_or_update_order(
-
+            when_to_start=timezone.make_aware(datetime.datetime(2020, 4, 17)))
-        Bill.create_next_bill_for_user(user)
-
 
         sys.exit(0)
 
+        # change 2
+        # vm25206_ssd.size_in_gb = 50
+        # vm25206_ssd.save()
+        # vm25206_ssd.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,8,5)))
 
-        vm25206_ssd.size_in_gb = 50
+        # 25206 done.
-        vm25206_ssd.save()
-        vm25206_ssd.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,8,5)))
-
-        vm25615 = VMProduct.objects.create(name="OpenNebula 25615",
-                            cores=1,
-                            ram_in_gb=4,
-                            owner=user)
-
-        vm25615_ssd = VMDiskProduct.objects.create(vm=vm25615,
-                                    owner=user,
-                                    size_in_gb=30)
 
+        # 25615
+        vm25615 = VMProduct.objects.create(name="one-25615", cores=1,
+                                           ram_in_gb=4, owner=user)
+        vm25615.create_order_at(
+            timezone.make_aware(datetime.datetime(2020, 3, 3)))
 
+        # Change 2020-04-17
         vm25615.cores = 2
         vm25615.ram_in_gb = 8
         vm25615.save()
-        vm25615.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,4,17)))
+        vm25615.create_or_update_order(
+            when_to_start=timezone.make_aware(datetime.datetime(2020, 4, 17)))
 
-        vm25615_ssd.size_in_gb = 50
+        # vm25615_ssd = VMDiskProduct(vm=vm25615, owner=user, size_in_gb=30)
-        vm25615_ssd.save()
+        # vm25615_ssd.create_order_at(timezone.make_aware(datetime.datetime(2020,3,3)))
-        vm25615_ssd.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,8,5)))
+        # vm25615_ssd.save()
-
-
-        vm25208 = VMProduct.objects.create(name="OpenNebula 25208",
-                            cores=1,
-                            ram_in_gb=4,
-                            owner=user)
-
-        vm25208_ssd = VMDiskProduct.objects.create(vm=vm25208,
-                                    owner=user,
-                                    size_in_gb=30)
 
+        vm25208 = VMProduct.objects.create(name="one-25208", cores=1,
+                                           ram_in_gb=4, owner=user)
+        vm25208.create_order_at(
+            timezone.make_aware(datetime.datetime(2020, 3, 5)))
 
         vm25208.cores = 2
         vm25208.ram_in_gb = 8
         vm25208.save()
-        vm25208.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,4,17)))
+        vm25208.create_or_update_order(
+            when_to_start=timezone.make_aware(datetime.datetime(2020, 4, 17)))
+
+        Bill.create_next_bills_for_user(user, ending_date=end_of_month(
+            timezone.make_aware(datetime.datetime(2020, 7, 31))))
+
+        sys.exit(0)
+
+        vm25615_ssd.size_in_gb = 50
+        vm25615_ssd.save()
+        vm25615_ssd.create_or_update_order(
+            when_to_start=timezone.make_aware(datetime.datetime(2020, 8, 5)))
+
+        vm25208_ssd = VMDiskProduct.objects.create(vm=vm25208,
+                                                   owner=user,
+                                                   size_in_gb=30)
 
         vm25208_ssd.size_in_gb = 50
         vm25208_ssd.save()
-        vm25208_ssd.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,8,5)))
+        vm25208_ssd.create_or_update_order(
-
+            when_to_start=timezone.make_aware(datetime.datetime(2020, 8, 5)))
 
         # 25207
         vm25207 = VMProduct.objects.create(name="OpenNebula 25207",
-                            cores=1,
+                                           cores=1,
-                            ram_in_gb=4,
+                                           ram_in_gb=4,
-                            owner=user)
+                                           owner=user)
 
         vm25207_ssd = VMDiskProduct.objects.create(vm=vm25207,
-                                    owner=user,
+                                                   owner=user,
-                                    size_in_gb=30)
+                                                   size_in_gb=30)
 
         vm25207_ssd.size_in_gb = 50
         vm25207_ssd.save()
-        vm25207_ssd.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,8,5)))
+        vm25207_ssd.create_or_update_order(
-
+            when_to_start=timezone.make_aware(datetime.datetime(2020, 8, 5)))
 
         vm25207.cores = 2
         vm25207.ram_in_gb = 8
         vm25207.save()
-        vm25207.create_or_update_order(when_to_start=timezone.make_aware(datetime.datetime(2020,6,19)))
+        vm25207.create_or_update_order(
-
+            when_to_start=timezone.make_aware(datetime.datetime(2020, 6, 19)))
 
         # FIXES: check starting times (they are slightly different)
         # add vm 25236

uncloud_pay/management/commands/import-vat-rates.py

@@ -1,44 +1,35 @@
 from django.core.management.base import BaseCommand
 from uncloud_pay.models import VATRate
-import csv
 
+import urllib
+import csv
+import sys
+import io
 
 class Command(BaseCommand):
     help = '''Imports VAT Rates. Assume vat rates of format https://github.com/kdeldycke/vat-rates/blob/master/vat_rates.csv'''
+    vat_url = "https://raw.githubusercontent.com/ungleich/vat-rates/main/vat_rates.csv"
+
 
     def add_arguments(self, parser):
-        parser.add_argument('csv_file', nargs='+', type=str)
+        parser.add_argument('--vat-url', default=self.vat_url)
 
     def handle(self, *args, **options):
-        try:
+        vat_url = options['vat_url']
-            for c_file in options['csv_file']:
+        url_open = urllib.request.urlopen(vat_url)
-                print("c_file =  %s" % c_file)
-                with open(c_file, mode='r') as csv_file:
-                    csv_reader = csv.DictReader(csv_file)
-                    line_count = 0
-                    for row in csv_reader:
-                        if line_count == 0:
-                            line_count += 1
-                        obj, created = VATRate.objects.get_or_create(
-                            start_date=row["start_date"],
-                            stop_date=row["stop_date"] if row["stop_date"] is not "" else None,
-                            territory_codes=row["territory_codes"],
-                            currency_code=row["currency_code"],
-                            rate=row["rate"],
-                            rate_type=row["rate_type"],
-                            description=row["description"]
-                        )
-                        if created:
-                            self.stdout.write(self.style.SUCCESS(
-                                '%s. %s - %s - %s - %s' % (
-                                    line_count,
-                                    obj.start_date,
-                                    obj.stop_date,
-                                    obj.territory_codes,
-                                    obj.rate
-                                )
-                            ))
-                            line_count+=1
 
-        except Exception as e:
+        # map to fileio using stringIO
-            print(" *** Error occurred. Details {}".format(str(e)))
+        csv_file = io.StringIO(url_open.read().decode('utf-8'))
+        reader = csv.DictReader(csv_file)
+
+        for row in reader:
+#            print(row)
+            obj, created = VATRate.objects.get_or_create(
+                starting_date=row["start_date"],
+                ending_date=row["stop_date"] if row["stop_date"] != "" else None,
+                territory_codes=row["territory_codes"],
+                currency_code=row["currency_code"],
+                rate=row["rate"],
+                rate_type=row["rate_type"],
+                description=row["description"]
+            )

uncloud_pay/migrations/0002_auto_20200801_2208.py

@@ -1,18 +0,0 @@
-# Generated by Django 3.0.8 on 2020-08-01 22:08
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='billingaddress',
-            name='organization',
-            field=models.CharField(blank=True, max_length=100, null=True),
-        ),
-    ]

uncloud_pay/migrations/0003_auto_20200801_2332.py

@@ -1,33 +0,0 @@
-# Generated by Django 3.0.8 on 2020-08-01 23:32
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0002_auto_20200801_2208'),
-    ]
-
-    operations = [
-        migrations.RenameField(
-            model_name='vatrate',
-            old_name='stop_date',
-            new_name='ending_date',
-        ),
-        migrations.RenameField(
-            model_name='vatrate',
-            old_name='start_date',
-            new_name='starting_date',
-        ),
-        migrations.AlterField(
-            model_name='bill',
-            name='ending_date',
-            field=models.DateTimeField(),
-        ),
-        migrations.AlterField(
-            model_name='billrecord',
-            name='quantity',
-            field=models.DecimalField(decimal_places=10, max_digits=19),
-        ),
-    ]

uncloud_pay/migrations/0004_remove_order_one_time_price.py

@@ -1,17 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 19:53
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0003_auto_20200801_2332'),
-    ]
-
-    operations = [
-        migrations.RemoveField(
-            model_name='order',
-            name='one_time_price',
-        ),
-    ]

uncloud_pay/migrations/0005_auto_20200808_1954.py

@@ -1,18 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 19:54
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0004_remove_order_one_time_price'),
-    ]
-
-    operations = [
-        migrations.RenameField(
-            model_name='order',
-            old_name='recurring_price',
-            new_name='price',
-        ),
-    ]

uncloud_pay/migrations/0006_remove_billrecord_quantity.py

@@ -1,17 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 19:57
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0005_auto_20200808_1954'),
-    ]
-
-    operations = [
-        migrations.RemoveField(
-            model_name='billrecord',
-            name='quantity',
-        ),
-    ]

uncloud_pay/migrations/0007_remove_bill_bill_records.py

@@ -1,17 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 20:20
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0006_remove_billrecord_quantity'),
-    ]
-
-    operations = [
-        migrations.RemoveField(
-            model_name='bill',
-            name='bill_records',
-        ),
-    ]

uncloud_pay/migrations/0008_delete_orderrecord.py

@@ -1,16 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 20:36
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0007_remove_bill_bill_records'),
-    ]
-
-    operations = [
-        migrations.DeleteModel(
-            name='OrderRecord',
-        ),
-    ]

uncloud_pay/migrations/0009_auto_20200808_2113.py

@@ -1,22 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 21:13
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0008_delete_orderrecord'),
-    ]
-
-    operations = [
-        migrations.RemoveField(
-            model_name='bill',
-            name='valid',
-        ),
-        migrations.AddField(
-            model_name='bill',
-            name='is_final',
-            field=models.BooleanField(default=False),
-        ),
-    ]

uncloud_pay/serializers.py

@@ -82,7 +82,7 @@ class BillRecordSerializer(serializers.Serializer):
     description = serializers.CharField()
     one_time_price = serializers.DecimalField(AMOUNT_MAX_DIGITS, AMOUNT_DECIMALS)
     recurring_price = serializers.DecimalField(AMOUNT_MAX_DIGITS, AMOUNT_DECIMALS)
-    recurring_period = serializers.ChoiceField(choices=RecurringPeriod.choices)
+#    recurring_period = serializers.ChoiceField()
     recurring_count = serializers.DecimalField(AMOUNT_MAX_DIGITS, AMOUNT_DECIMALS)
     vat_rate = serializers.DecimalField(AMOUNT_MAX_DIGITS, AMOUNT_DECIMALS)
     vat_amount = serializers.DecimalField(AMOUNT_MAX_DIGITS, AMOUNT_DECIMALS)

uncloud_pay/templates/bill.html.j2

@@ -6,7 +6,7 @@
 
   Icons, fonts, etc. are INLINED. This is rather ugly, but as the PDF
   generation is based on a local snapshot of the HTML file, URLs are
-  screwed if they are not absolute.
+  screwed if they are not absolute to the *local* filesystem.
 
   As this document is used ONLY for bills and ONLY for downloading, I
   decided that this is an acceptable uglyness.
@@ -36,7 +36,6 @@
           font-weight: 500;
           line-height: 1.1;
           font-size: 14px;
-          width: 600px;
           margin: auto;
           padding-top: 40px;
           padding-bottom: 15px;
@@ -672,60 +671,44 @@ oAsAAAAAAACGQNAFAAAAAAAAQyDoAgAAAAAAgCEQdAEAAAAAAMAQCLoAAAAAAABgCP83AL6WQ1Y7
         <br>
     </div>
     <div class="d1">
-        {% if bill.billing_address.organization != "" %}
+        <b>{{ bill.billing_address.organization }}</b><br/>
-        <b>ORG{{ bill.billing_address.organization }}</b>
+        <b>{{ bill.billing_address.name }}</b><br/>
-        <br>{{ bill.billing_address.name }} <bill.owner.email>
+        {{ bill.owner.email }}<br/>
-        {% else %}
+        {{ bill.billing_address.street }}<br/>
-        <b>{{ bill.billing_address.name }} <bill.owner.email></b>
+        {{ bill.billing_address.country }} {{ bill.billing_address.postal_code }} {{ bill.billing_address.city }}<br/>
-        {% endif %}
+
-        <br>{{ bill.billing_address.street }}
-        <br>{{ bill.billing_address.postal_code }} {{ bill.billing_address.city }}
-        <br>{{ bill.billing_address.country }}
-            <br>
     </div>
     <div class="d4">
         <div class="b1">
-            Rechnungsdatum:
+            {{ bill.starting_date|date:"c" }} -
-            <br> Rechnungsnummer
+            {{ bill.ending_date|date:"c" }}
-            <br> Zahlbar bis
+            <br>Bill id: {{ bill }}
+            <br>Due: {{ bill.due_date }}
 
         </div>
-        <div class="b2">
-          {{ bill.creation_date.date }}<br>
-          {% if bill.billing_address.vat_number != "" %}
-          {{ bill.billing_address.vat_number
-          }}<br>
-          {% else %}
-          None<br>
-          {% endif %}
-          {{ bill.billing_address.vat_number }}<br>
-          {{ bill.due_date }}
-        </div>
     </div>
     <div style="clear: both;"></div>
     <div class="d5">
-        <h1>RECHNUNG</h1>
+        <h1>Invoice</h1>
     </div>
     <table>
       <thead>
         <tr>
           <th>Detail</th>
-          <th>Quantity</th>
           <th>Price/Unit</th>
-          <th class="tr">Total</tH>
+          <th>Units</th>
+          <th>Total price</th>
         </tr>
       </thead>
       <tbody>
         {% for record in bill_records %}
         <tr class="table-list">
           <td>{{ record.starting_date|date:"c" }}
-          {% if record.ending_date %}
           - {{ record.ending_date|date:"c" }}
-          {% endif %}
+          {{ record.order }}
-          {{ record.order.description }}
           </td>
+          <td>{{ record.price|floatformat:2 }}</td>
           <td>{{ record.quantity|floatformat:2 }}</td>
-          <td>{{ record.order.price|floatformat:2 }}</td>
           <td>{{ record.sum|floatformat:2 }}</td>
         </tr>
         {% endfor %}
@@ -733,17 +716,17 @@ oAsAAAAAAACGQNAFAAAAAAAAQyDoAgAAAAAAgCEQdAEAAAAAAMAQCLoAAAAAAABgCP83AL6WQ1Y7
     </table>
     <div class="wf th">
         <p class="ts">
-            <span class="tl">Total</span>
+            <span class="tl">Total (excl. VAT)</span>
             <span class="tr">{{ bill.amount }}</span>
         </p>
         <p class="ts">
-          <span class="tl">VAT</span>
+          <span class="tl">VAT 7.7%</span>
           <span class="tr">{{ bill.vat_amount|floatformat:2 }}</span>
         </p>
     </div>
     <div class="wf pc">
         <p class="bold">
-            <span class="tl">Total</span>
+            <span class="tl">Total amount to be paid</span>
             <span class="tr">{{ bill.sum|floatformat:2 }}</span>
         </p>
     </div>

uncloud_pay/templates/uncloud_pay/stripe.html

@@ -0,0 +1,72 @@
+{% extends 'uncloud/base.html' %}
+
+{% block header %}
+		<script src="https://js.stripe.com/v3/"></script>
+		<style>
+			#content {
+				width: 400px;
+				margin: auto;
+			}
+
+			#callback-form {
+				display: none;
+			}
+		</style>
+{% endblock %}
+
+{% block body %}
+		<div id="content">
+			<h1>Registering Stripe Credit Card</h1>
+
+			<!-- Stripe form and messages -->
+			<span id="message"></span>
+			<form id="setup-form">
+				<div id="card-element"></div>
+				<button type='button' id="card-button">
+					Save
+				</button>
+			</form>
+
+			<!-- Dirty hack used for callback to API -->
+			<form id="callback-form" action="{{ callback }}" method="post"></form>
+		</div>
+
+		<!-- Enable Stripe from UI elements -->
+		<script>
+			var stripe = Stripe('{{ stripe_pk }}');
+
+			var elements = stripe.elements();
+			var cardElement = elements.create('card');
+			cardElement.mount('#card-element');
+		</script>
+
+		<!-- Handle card submission -->
+		<script>
+			var cardButton = document.getElementById('card-button');
+			var messageContainer = document.getElementById('message');
+			var clientSecret = '{{ client_secret }}';
+
+			cardButton.addEventListener('click', function(ev) {
+
+				stripe.confirmCardSetup(
+					clientSecret,
+					{
+						payment_method: {
+							card: cardElement,
+							billing_details: {
+							},
+						},
+					}
+				).then(function(result) {
+					if (result.error) {
+						var message = document.createTextNode('Error:' + result.error.message);
+						messageContainer.appendChild(message);
+					} else {
+						// Return to API on success.
+						document.getElementById("callback-form").submit();
+					}
+				});
+			});
+		</script>
+
+{% endblock %}

uncloud_pay/tests.py

@@ -1,11 +1,72 @@
 from django.test import TestCase
 from django.contrib.auth import get_user_model
 from datetime import datetime, date, timedelta
+from django.utils import timezone
 
 from .models import *
 from uncloud_service.models import GenericServiceProduct
 
-class ProductOrderTestCase(TestCase):
+import json
+
+chocolate_product_config = {
+    'features': {
+        'gramm':
+        { 'min': 100,
+          'max': 5000,
+          'one_time_price_per_unit': 0.2,
+          'recurring_price_per_unit': 0
+         },
+    },
+}
+
+chocolate_order_config = {
+    'features': {
+        'gramm': 500,
+    }
+}
+
+chocolate_one_time_price = chocolate_order_config['features']['gramm'] * chocolate_product_config['features']['gramm']['one_time_price_per_unit']
+
+vm_product_config = {
+    'features': {
+        'cores':
+        { 'min': 1,
+          'max': 48,
+          'one_time_price_per_unit': 0,
+          'recurring_price_per_unit': 4
+         },
+        'ram_gb':
+        { 'min': 1,
+          'max': 256,
+          'one_time_price_per_unit': 0,
+          'recurring_price_per_unit': 4
+         },
+    },
+}
+
+vm_order_config = {
+    'features': {
+        'cores': 2,
+        'ram_gb': 2
+    }
+}
+
+vm_order_downgrade_config = {
+    'features': {
+        'cores': 1,
+        'ram_gb': 1
+    }
+}
+
+vm_order_upgrade_config = {
+    'features': {
+        'cores': 4,
+        'ram_gb': 4
+    }
+}
+
+
+class ProductTestCase(TestCase):
     """
     Test products and products <-> order interaction
     """
@@ -15,81 +76,227 @@ class ProductOrderTestCase(TestCase):
             username='random_user',
             email='jane.random@domain.tld')
 
-    def test_update_one_time_product(self):
+        self.ba = BillingAddress.objects.create(
+            owner=self.user,
+            organization = 'Test org',
+            street="unknown",
+            city="unknown",
+            postal_code="somewhere else",
+            active=True)
+
+        RecurringPeriod.populate_db_defaults()
+        self.default_recurring_period = RecurringPeriod.objects.get(name="Per 30 days")
+
+    def test_create_product(self):
         """
-        One time payment products cannot be updated - can they?
+        Create a sample product
         """
 
-        pass
+        p = Product.objects.create(name="Testproduct",
+                                   description="Only for testing",
+                                   config=vm_product_config)
+
+        p.recurring_periods.add(self.default_recurring_period,
+                                through_defaults= { 'is_default': True })
 
 
-class BillingAddressTestCase(TestCase):
+class OrderTestCase(TestCase):
+    """
+    The heart of ordering products
+    """
+
     def setUp(self):
         self.user = get_user_model().objects.create(
             username='random_user',
             email='jane.random@domain.tld')
 
-
+        self.ba = BillingAddress.objects.create(
-    def test_user_only_inactive_address(self):
-        """
-        Raise an error, when there is no active address
-        """
-
-        ba = BillingAddress.objects.create(
             owner=self.user,
             organization = 'Test org',
             street="unknown",
             city="unknown",
             postal_code="somewhere else",
-            active=False)
-
-        self.assertRaises(uncloud_pay.models.BillingAddress.DoesNotExist,
-                          BillingAddress.get_address_for,
-                          self.user)
-
-    def test_user_only_active_address(self):
-        """
-        Find the active address
-        """
-
-        ba = BillingAddress.objects.create(
-            owner=self.user,
-            organization = 'Test org',
-            street="unknown",
-            city="unknown",
-            postal_code="unknown",
             active=True)
 
+        self.product = Product.objects.create(name="Testproduct",
+                                   description="Only for testing",
+                                   config=vm_product_config)
 
-        self.assertEqual(BillingAddress.get_address_for(self.user), ba)
+        RecurringPeriod.populate_db_defaults()
+        self.default_recurring_period = RecurringPeriod.objects.get(name="Per 30 days")
 
-    def test_multiple_addresses(self):
+        self.product.recurring_periods.add(self.default_recurring_period,
+                                           through_defaults= { 'is_default': True })
+
+
+    def test_order_invalid_recurring_period(self):
         """
-        Find the active address only, skip inactive
+        Order a products with a recurringperiod that is not added to the product
         """
 
-        ba = BillingAddress.objects.create(
+        o = Order.objects.create(owner=self.user,
-            owner=self.user,
+                                 billing_address=self.ba,
-            organization = 'Test org',
+                                 product=self.product,
-            street="unknown",
+                                 config=vm_order_config)
-            city="unknown",
-            postal_code="unknown",
-            active=True)
-
-        ba2 = BillingAddress.objects.create(
-            owner=self.user,
-            organization = 'Test org',
-            street="unknown",
-            city="unknown",
-            postal_code="somewhere else",
-            active=False)
 
 
-        self.assertEqual(BillingAddress.get_address_for(self.user), ba)
+    def test_order_product(self):
+        """
+        Order a product, ensure the order has correct price setup
+        """
+
+        o = Order.objects.create(owner=self.user,
+                                 billing_address=self.ba,
+                                 product=self.product)
+
+        self.assertEqual(o.one_time_price, 0)
+        self.assertEqual(o.recurring_price, 16)
+
+    def test_change_order(self):
+        """
+        Change an order and ensure that
+        - a new order is created
+        - the price is correct in the new order
+        """
+        order1 = Order.objects.create(owner=self.user,
+                                 billing_address=self.ba,
+                                 product=self.product,
+                                 config=vm_order_config)
 
 
-class BillAndOrderTestCase(TestCase):
+        self.assertEqual(order1.one_time_price, 0)
+        self.assertEqual(order1.recurring_price, 16)
+
+
+class ModifyOrderTestCase(TestCase):
+    """
+    Test typical order flows like
+    - cancelling
+    - downgrading
+    - upgrading
+    """
+
     def setUp(self):
+        self.user = get_user_model().objects.create(
+            username='random_user',
+            email='jane.random@domain.tld')
+
+        self.ba = BillingAddress.objects.create(
+            owner=self.user,
+            organization = 'Test org',
+            street="unknown",
+            city="unknown",
+            postal_code="somewhere else",
+            active=True)
+
+        self.product = Product.objects.create(name="Testproduct",
+                                              description="Only for testing",
+                                              config=vm_product_config)
+
+        RecurringPeriod.populate_db_defaults()
+        self.default_recurring_period = RecurringPeriod.objects.get(name="Per 30 days")
+
+        self.product.recurring_periods.add(self.default_recurring_period,
+                                through_defaults= { 'is_default': True })
+
+
+    def test_change_order(self):
+        """
+        Test changing an order
+
+        Expected result:
+
+        - Old order should be closed before new order starts
+        - New order should start at starting data
+        """
+
+        user = self.user
+
+        starting_price = 16
+        downgrade_price = 8
+
+        starting_date = timezone.make_aware(datetime.datetime(2019,3,3))
+        ending1_date = starting_date + datetime.timedelta(days=15)
+        change1_date  = start_after(ending1_date)
+
+        bill_ending_date = change1_date + datetime.timedelta(days=1)
+
+
+        order1 = Order.objects.create(owner=self.user,
+                                     billing_address=BillingAddress.get_address_for(self.user),
+                                     product=self.product,
+                                     config=vm_order_config,
+                                     starting_date=starting_date)
+
+        order1.update_order(vm_order_downgrade_config, starting_date=change1_date)
+
+        bills = Bill.create_next_bills_for_user(user, ending_date=bill_ending_date)
+
+        bill = bills[0]
+        bill_records = BillRecord.objects.filter(bill=bill)
+
+        self.assertEqual(len(bill_records), 2)
+
+        self.assertEqual(bill_records[0].starting_date, starting_date)
+        self.assertEqual(bill_records[0].ending_date, ending1_date)
+
+        self.assertEqual(bill_records[1].starting_date, change1_date)
+
+
+
+    def test_downgrade_product(self):
+        """
+        Test downgrading behaviour:
+
+        We create a recurring product (recurring time: 30 days) and downgrade after 15 days.
+
+        We create the bill right AFTER the end of the first order.
+
+        Expected result:
+
+        - First bill record for 30 days
+        - Second bill record starting after 30 days
+        - Bill contains two bill records
+
+        """
+
+        user = self.user
+
+        starting_price = 16
+        downgrade_price = 8
+
+        starting_date = timezone.make_aware(datetime.datetime(2019,3,3))
+        first_order_should_end_at = starting_date + datetime.timedelta(days=30)
+        change1_date  = start_after(starting_date + datetime.timedelta(days=15))
+        bill_ending_date = change1_date + datetime.timedelta(days=1)
+
+        order1 = Order.objects.create(owner=self.user,
+                                     billing_address=BillingAddress.get_address_for(self.user),
+                                     product=self.product,
+                                     config=vm_order_config,
+                                     starting_date=starting_date)
+
+        order1.update_order(vm_order_downgrade_config, starting_date=change1_date)
+
+        bills = Bill.create_next_bills_for_user(user, ending_date=bill_ending_date)
+
+        bill = bills[0]
+        bill_records = BillRecord.objects.filter(bill=bill)
+
+        self.assertEqual(len(bill_records), 2)
+
+        self.assertEqual(bill_records[0].starting_date, starting_date)
+        self.assertEqual(bill_records[0].order.ending_date, first_order_should_end_at)
+
+
+class BillTestCase(TestCase):
+    """
+    Test aspects of billing / creating a bill
+    """
+
+    def setUp(self):
+        RecurringPeriod.populate_db_defaults()
+
         self.user_without_address = get_user_model().objects.create(
             username='no_home_person',
             email='far.away@domain.tld')
@@ -102,7 +309,7 @@ class BillAndOrderTestCase(TestCase):
             username='recurrent_product_user',
             email='jane.doe@domain.tld')
 
-        BillingAddress.objects.create(
+        self.user_addr = BillingAddress.objects.create(
             owner=self.user,
             organization = 'Test org',
             street="unknown",
@@ -110,7 +317,7 @@ class BillAndOrderTestCase(TestCase):
             postal_code="unknown",
             active=True)
 
-        BillingAddress.objects.create(
+        self.recurring_user_addr = BillingAddress.objects.create(
             owner=self.recurring_user,
             organization = 'Test org',
             street="Somewhere",
@@ -126,41 +333,89 @@ class BillAndOrderTestCase(TestCase):
             'description': 'One chocolate bar'
         }
 
-        self.one_time_order = Order.objects.create(
+        self.chocolate = Product.objects.create(name="Swiss Chocolate",
-                owner=self.user,
+                                              description="Not only for testing, but for joy",
-                starting_date=self.order_meta[1]['starting_date'],
+                                              config=chocolate_product_config)
-                ending_date=self.order_meta[1]['ending_date'],
-                recurring_period=RecurringPeriod.ONE_TIME,
-                price=self.order_meta[1]['price'],
-                description=self.order_meta[1]['description'],
-                billing_address=BillingAddress.get_address_for(self.user))
 
-        self.recurring_order = Order.objects.create(
+
-                owner=self.recurring_user,
+        self.vm = Product.objects.create(name="Super Fast VM",
-                starting_date=timezone.make_aware(datetime.datetime(2020,3,3)),
+                                              description="Zooooom",
-                recurring_period=RecurringPeriod.PER_30D,
+                                              config=vm_product_config)
-                price=15,
+
-                description="A pretty VM",
+
-                billing_address=BillingAddress.get_address_for(self.recurring_user)
+        RecurringPeriod.populate_db_defaults()
+        self.default_recurring_period = RecurringPeriod.objects.get(name="Per 30 days")
+
+        self.onetime_recurring_period = RecurringPeriod.objects.get(name="Onetime")
+
+        self.chocolate.recurring_periods.add(self.onetime_recurring_period,
+                                through_defaults= { 'is_default': True })
+
+        self.vm.recurring_periods.add(self.default_recurring_period,
+                                through_defaults= { 'is_default': True })
+
+
+        # used for generating multiple bills
+        self.bill_dates = [
+            timezone.make_aware(datetime.datetime(2020,3,31)),
+            timezone.make_aware(datetime.datetime(2020,4,30)),
+            timezone.make_aware(datetime.datetime(2020,5,31)),
+        ]
+
+
+    def order_chocolate(self):
+        return Order.objects.create(
+            owner=self.user,
+            recurring_period=RecurringPeriod.objects.get(name="Onetime"),
+            product=self.chocolate,
+            billing_address=BillingAddress.get_address_for(self.user),
+            starting_date=self.order_meta[1]['starting_date'],
+            ending_date=self.order_meta[1]['ending_date'],
+            config=chocolate_order_config)
+
+    def order_vm(self, owner=None):
+
+        if not owner:
+            owner = self.recurring_user
+
+        return Order.objects.create(
+            owner=owner,
+            product=self.vm,
+            config=vm_order_config,
+            billing_address=BillingAddress.get_address_for(self.recurring_user),
+            starting_date=timezone.make_aware(datetime.datetime(2020,3,3)),
         )
 
+        return Order.objects.create(
+            owner=self.user,
+            recurring_period=RecurringPeriod.objects.get(name="Onetime"),
+            product=self.chocolate,
+            billing_address=BillingAddress.get_address_for(self.user),
+            starting_date=self.order_meta[1]['starting_date'],
+            ending_date=self.order_meta[1]['ending_date'],
+            config=chocolate_order_config)
+
+
 
     def test_bill_one_time_one_bill_record(self):
         """
         Ensure there is only 1 bill record per order
         """
 
-        bill = Bill.create_next_bill_for_user(self.user)
+        order = self.order_chocolate()
 
-        self.assertEqual(self.one_time_order.billrecord_set.count(), 1)
+        bill = Bill.create_next_bill_for_user_address(self.user_addr)
+
+        self.assertEqual(order.billrecord_set.count(), 1)
 
     def test_bill_sum_onetime(self):
         """
         Check the bill sum for a single one time order
         """
 
-        bill = Bill.create_next_bill_for_user(self.user)
+        order = self.order_chocolate()
-        self.assertEqual(bill.sum, self.order_meta[1]['price'])
+        bill = Bill.create_next_bill_for_user_address(self.user_addr)
+        self.assertEqual(bill.sum, chocolate_one_time_price)
 
 
     def test_bill_creates_record_for_recurring_order(self):
@@ -168,241 +423,43 @@ class BillAndOrderTestCase(TestCase):
         Ensure there is only 1 bill record per order
         """
 
-        bill = Bill.create_next_bill_for_user(self.recurring_user)
+        order = self.order_vm()
+        bill = Bill.create_next_bill_for_user_address(self.recurring_user_addr)
 
-        self.assertEqual(self.recurring_order.billrecord_set.count(), 1)
+        self.assertEqual(order.billrecord_set.count(), 1)
         self.assertEqual(bill.billrecord_set.count(), 1)
 
-# class NotABillingTC(TestCase):
-# #class BillingTestCase(TestCase):
-#     def setUp(self):
-#         self.user = get_user_model().objects.create(
-#                 username='jdoe',
-#                 email='john.doe@domain.tld')
-#         self.billing_address = BillingAddress.objects.create(
-#                 owner=self.user,
-#                 street="unknown",
-#                 city="unknown",
-#                 postal_code="unknown")
 
-#     def test_basic_monthly_billing(self):
+    def test_new_bill_after_closing(self):
-#         one_time_price = 10
+        """
-#         recurring_price = 20
+        After closing a bill and the user has a recurring product,
-#         description = "Test Product 1"
+        the next bill run should create e new bill
+        """
 
-#         # Three months: full, full, partial.
+        order = self.order_vm()
-# #        starting_date = datetime.fromisoformat('2020-03-01')
-#         starting_date = datetime(2020,3,1)
-#         ending_date   = datetime(2020,5,8)
 
-#         # Create order to be billed.
+        for ending_date in self.bill_dates:
-#         order = Order.objects.create(
+            b = Bill.create_next_bill_for_user_address(self.recurring_user_addr, ending_date)
-#                 owner=self.user,
+            b.close()
-#                 starting_date=starting_date,
-#                 ending_date=ending_date,
-#                 recurring_period=RecurringPeriod.PER_30D,
-#                 recurring_price=recurring_price,
-#                 one_time_price=one_time_price,
-#                 description=description,
-#                 billing_address=self.billing_address)
 
-#         # Generate & check bill for first month: full recurring_price + setup.
+        bill_count = Bill.objects.filter(owner=self.recurring_user).count()
-#         first_month_bills = order.generate_initial_bill()
-#         self.assertEqual(len(first_month_bills), 1)
-#         self.assertEqual(first_month_bills[0].amount, one_time_price + recurring_price)
 
-#         # Generate & check bill for second month: full recurring_price.
+        self.assertEqual(len(self.bill_dates), bill_count)
-#         second_month_bills = Bill.generate_for(2020, 4, self.user)
-#         self.assertEqual(len(second_month_bills), 1)
-#         self.assertEqual(second_month_bills[0].amount, recurring_price)
 
-#         # Generate & check bill for third and last month: partial recurring_price.
-#         third_month_bills = Bill.generate_for(2020, 5, self.user)
-#         self.assertEqual(len(third_month_bills), 1)
-#         # 31 days in May.
-#         self.assertEqual(float(third_month_bills[0].amount),
-#                 round(round((7/31), AMOUNT_DECIMALS) * recurring_price, AMOUNT_DECIMALS))
 
-#         # Check that running Bill.generate_for() twice does not create duplicates.
-#         self.assertEqual(len(Bill.generate_for(2020, 3, self.user)), 0)
 
-#     def test_basic_yearly_billing(self):
+class BillingAddressTestCase(TestCase):
-#         one_time_price = 10
+    def setUp(self):
-#         recurring_price = 150
+        self.user = get_user_model().objects.create(
-#         description = "Test Product 1"
+            username='random_user',
+            email='jane.random@domain.tld')
 
-#         starting_date = datetime.fromisoformat('2020-03-31T08:05:23')
 
-#         # Create order to be billed.
+    def test_user_no_address(self):
-#         order = Order.objects.create(
+        """
-#                 owner=self.user,
+        Raise an error, when there is no address
-#                 starting_date=starting_date,
+        """
-#                 recurring_period=RecurringPeriod.PER_365D,
-#                 recurring_price=recurring_price,
-#                 one_time_price=one_time_price,
-#                 description=description,
-#                 billing_address=self.billing_address)
 
-#         # Generate & check bill for first year: recurring_price + setup.
+        self.assertRaises(uncloud_pay.models.BillingAddress.DoesNotExist,
-#         first_year_bills = order.generate_initial_bill()
+                          BillingAddress.get_address_for,
-#         self.assertEqual(len(first_year_bills), 1)
+                          self.user)
-#         self.assertEqual(first_year_bills[0].starting_date.date(),
-#                 date.fromisoformat('2020-03-31'))
-#         self.assertEqual(first_year_bills[0].ending_date.date(),
-#                 date.fromisoformat('2021-03-30'))
-#         self.assertEqual(first_year_bills[0].amount,
-#                 recurring_price + one_time_price)
-
-#         # Generate & check bill for second year: recurring_price.
-#         second_year_bills = Bill.generate_for(2021, 3, self.user)
-#         self.assertEqual(len(second_year_bills), 1)
-#         self.assertEqual(second_year_bills[0].starting_date.date(),
-#                 date.fromisoformat('2021-03-31'))
-#         self.assertEqual(second_year_bills[0].ending_date.date(),
-#                 date.fromisoformat('2022-03-30'))
-#         self.assertEqual(second_year_bills[0].amount, recurring_price)
-
-#         # Check that running Bill.generate_for() twice does not create duplicates.
-#         self.assertEqual(len(Bill.generate_for(2020, 3, self.user)), 0)
-#         self.assertEqual(len(Bill.generate_for(2020, 4, self.user)), 0)
-#         self.assertEqual(len(Bill.generate_for(2020, 2, self.user)), 0)
-#         self.assertEqual(len(Bill.generate_for(2021, 3, self.user)), 0)
-
-#     def test_basic_hourly_billing(self):
-#         one_time_price = 10
-#         recurring_price = 1.4
-#         description = "Test Product 1"
-
-#         starting_date = datetime.fromisoformat('2020-03-31T08:05:23')
-#         ending_date   = datetime.fromisoformat('2020-04-01T11:13:32')
-
-#         # Create order to be billed.
-#         order = Order.objects.create(
-#                 owner=self.user,
-#                 starting_date=starting_date,
-#                 ending_date=ending_date,
-#                 recurring_period=RecurringPeriod.PER_HOUR,
-#                 recurring_price=recurring_price,
-#                 one_time_price=one_time_price,
-#                 description=description,
-#                 billing_address=self.billing_address)
-
-#         # Generate & check bill for first month: recurring_price + setup.
-#         first_month_bills = order.generate_initial_bill()
-#         self.assertEqual(len(first_month_bills), 1)
-#         self.assertEqual(float(first_month_bills[0].amount),
-#                 round(16 * recurring_price, AMOUNT_DECIMALS) + one_time_price)
-
-#         # Generate & check bill for first month: recurring_price.
-#         second_month_bills = Bill.generate_for(2020, 4, self.user)
-#         self.assertEqual(len(second_month_bills), 1)
-#         self.assertEqual(float(second_month_bills[0].amount),
-#                 round(12 * recurring_price, AMOUNT_DECIMALS))
-
-# class ProductActivationTestCase(TestCase):
-#     def setUp(self):
-#         self.user = get_user_model().objects.create(
-#                 username='jdoe',
-#                 email='john.doe@domain.tld')
-
-#         self.billing_address = BillingAddress.objects.create(
-#                 owner=self.user,
-#                 street="unknown",
-#                 city="unknown",
-#                 postal_code="unknown")
-
-#     def test_product_activation(self):
-#         starting_date = datetime.fromisoformat('2020-03-01')
-#         one_time_price = 0
-#         recurring_price = 1
-#         description = "Test Product"
-
-#         order = Order.objects.create(
-#                 owner=self.user,
-#                 starting_date=starting_date,
-#                 recurring_period=RecurringPeriod.PER_30D,
-#                 recurring_price=recurring_price,
-#                 one_time_price=one_time_price,
-#                 description=description,
-#                 billing_address=self.billing_address)
-
-#         product = GenericServiceProduct(
-#                 custom_description=description,
-#                 custom_one_time_price=one_time_price,
-#                 custom_recurring_price=recurring_price,
-#                 owner=self.user,
-#                 order=order)
-#         product.save()
-
-#         # Validate initial state: must be awaiting payment.
-#         self.assertEqual(product.status, UncloudStatus.AWAITING_PAYMENT)
-
-#         # Pay initial bill, check that product is activated.
-#         order.generate_initial_bill()
-#         amount = product.order.bills[0].amount
-#         payment = Payment(owner=self.user, amount=amount)
-#         payment.save()
-#         self.assertEqual(
-#                 GenericServiceProduct.objects.get(uuid=product.uuid).status,
-#                 UncloudStatus.PENDING
-#                 )
-
-# class BillingAddressTestCase(TestCase):
-#     def setUp(self):
-#         self.user = get_user_model().objects.create(
-#                 username='jdoe',
-#                 email='john.doe@domain.tld')
-
-#         self.billing_address_01 = BillingAddress.objects.create(
-#                 owner=self.user,
-#                 street="unknown1",
-#                 city="unknown1",
-#                 postal_code="unknown1",
-#                 country="CH")
-
-#         self.billing_address_02 = BillingAddress.objects.create(
-#                 owner=self.user,
-#                 street="unknown2",
-#                 city="unknown2",
-#                 postal_code="unknown2",
-#                 country="CH")
-
-#     def test_billing_with_single_address(self):
-#         # Create new orders somewhere in the past so that we do not encounter
-#         # auto-created initial bills.
-#         starting_date = datetime.fromisoformat('2020-03-01')
-
-#         order_01 = Order.objects.create(
-#                 owner=self.user,
-#                 starting_date=starting_date,
-#                 recurring_period=RecurringPeriod.PER_30D,
-#                 billing_address=self.billing_address_01)
-#         order_02 = Order.objects.create(
-#                 owner=self.user,
-#                 starting_date=starting_date,
-#                 recurring_period=RecurringPeriod.PER_30D,
-#                 billing_address=self.billing_address_01)
-
-#         # We need a single bill since we work with a single address.
-#         bills = Bill.generate_for(2020, 4, self.user)
-#         self.assertEqual(len(bills), 1)
-
-#     def test_billing_with_multiple_addresses(self):
-#         # Create new orders somewhere in the past so that we do not encounter
-#         # auto-created initial bills.
-#         starting_date = datetime.fromisoformat('2020-03-01')
-
-#         order_01 = Order.objects.create(
-#                 owner=self.user,
-#                 starting_date=starting_date,
-#                 recurring_period=RecurringPeriod.PER_30D,
-#                 billing_address=self.billing_address_01)
-#         order_02 = Order.objects.create(
-#                 owner=self.user,
-#                 starting_date=starting_date,
-#                 recurring_period=RecurringPeriod.PER_30D,
-#                 billing_address=self.billing_address_02)
-
-#         # We need different bills since we work with different addresses.
-#         bills = Bill.generate_for(2020, 4, self.user)
-#         self.assertEqual(len(bills), 2)

uncloud_pay/views.py

@@ -1,3 +1,7 @@
+from django.contrib.auth.mixins import LoginRequiredMixin
+from django.views.generic.base import TemplateView
+
+
 from django.shortcuts import render
 from django.db import transaction
 from django.contrib.auth import get_user_model
@@ -43,6 +47,25 @@ class OrderViewSet(viewsets.ReadOnlyModelViewSet):
     def get_queryset(self):
         return Order.objects.filter(owner=self.request.user)
 
+
+class RegisterCard(LoginRequiredMixin, TemplateView):
+    login_url = '/login/'
+
+    # This is not supposed to be "static" --
+    # the idea is to be able to switch the provider when needed
+    template_name = "uncloud_pay/stripe.html"
+
+    def get_context_data(self, **kwargs):
+        customer_id = uncloud_stripe.get_customer_id_for(self.request.user)
+        setup_intent = uncloud_stripe.create_setup_intent(customer_id)
+
+        context = super().get_context_data(**kwargs)
+        context['client_secret'] = setup_intent.client_secret
+        context['username'] = self.request.user
+        context['stripe_pk'] = uncloud_stripe.public_api_key
+        return context
+
+
 class PaymentMethodViewSet(viewsets.ModelViewSet):
     permission_classes = [permissions.IsAuthenticated]
 
@@ -201,6 +224,7 @@ class BillViewSet(viewsets.ReadOnlyModelViewSet):
         Allow to download
         """
         bill = self.get_object()
+        provider = UncloudProvider.get_provider()
         output_file = NamedTemporaryFile()
         bill_html = render_to_string("bill.html.j2", {'bill': bill})
 

uncloud_service/apps.py

@@ -2,4 +2,4 @@ from django.apps import AppConfig
 
 
 class UngleichServiceConfig(AppConfig):
-    name = 'ungleich_service'
+    name = 'uncloud_service'

uncloud_service/migrations/0001_initial.py

@@ -1,52 +0,0 @@
-# Generated by Django 3.0.6 on 2020-08-01 16:38
-
-from django.conf import settings
-import django.contrib.postgres.fields.jsonb
-import django.core.validators
-from django.db import migrations, models
-import django.db.models.deletion
-
-
-class Migration(migrations.Migration):
-
-    initial = True
-
-    dependencies = [
-        ('uncloud_vm', '__first__'),
-        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
-        ('uncloud_pay', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.CreateModel(
-            name='MatrixServiceProduct',
-            fields=[
-                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
-                ('status', models.CharField(choices=[('PENDING', 'Pending'), ('AWAITING_PAYMENT', 'Awaiting payment'), ('BEING_CREATED', 'Being created'), ('SCHEDULED', 'Scheduled'), ('ACTIVE', 'Active'), ('MODIFYING', 'Modifying'), ('DELETED', 'Deleted'), ('DISABLED', 'Disabled'), ('UNUSABLE', 'Unusable')], default='AWAITING_PAYMENT', max_length=32)),
-                ('domain', models.CharField(default='domain.tld', max_length=255)),
-                ('order', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order')),
-                ('owner', models.ForeignKey(editable=False, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
-                ('vm', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.VMProduct')),
-            ],
-            options={
-                'abstract': False,
-            },
-        ),
-        migrations.CreateModel(
-            name='GenericServiceProduct',
-            fields=[
-                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
-                ('status', models.CharField(choices=[('PENDING', 'Pending'), ('AWAITING_PAYMENT', 'Awaiting payment'), ('BEING_CREATED', 'Being created'), ('SCHEDULED', 'Scheduled'), ('ACTIVE', 'Active'), ('MODIFYING', 'Modifying'), ('DELETED', 'Deleted'), ('DISABLED', 'Disabled'), ('UNUSABLE', 'Unusable')], default='AWAITING_PAYMENT', max_length=32)),
-                ('custom_description', models.TextField()),
-                ('custom_recurring_price', models.DecimalField(decimal_places=2, default=0.0, max_digits=10, validators=[django.core.validators.MinValueValidator(0)])),
-                ('custom_one_time_price', models.DecimalField(decimal_places=2, default=0.0, max_digits=10, validators=[django.core.validators.MinValueValidator(0)])),
-                ('order', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order')),
-                ('owner', models.ForeignKey(editable=False, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
-            ],
-            options={
-                'abstract': False,
-            },
-        ),
-    ]

uncloud_service/migrations/0002_auto_20200801_2332.py

@@ -1,25 +0,0 @@
-# Generated by Django 3.0.8 on 2020-08-01 23:32
-
-from django.db import migrations, models
-import django.db.models.deletion
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0003_auto_20200801_2332'),
-        ('uncloud_service', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='genericserviceproduct',
-            name='order',
-            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order'),
-        ),
-        migrations.AlterField(
-            model_name='matrixserviceproduct',
-            name='order',
-            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order'),
-        ),
-    ]

uncloud_service/migrations/0003_auto_20200808_1953.py

@@ -1,23 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 19:53
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_service', '0002_auto_20200801_2332'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='genericserviceproduct',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-        migrations.AlterField(
-            model_name='matrixserviceproduct',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-    ]

uncloud_service/models.py

@@ -3,7 +3,7 @@ from uncloud_pay.models import Product, RecurringPeriod, AMOUNT_MAX_DIGITS, AMOU
 from uncloud_vm.models import VMProduct, VMDiskImageProduct
 from django.core.validators import MinValueValidator
 
-class MatrixServiceProduct(Product):
+class MatrixServiceProduct(models.Model):
     monthly_managment_fee = 20
 
     description = "Managed Matrix HomeServer"
@@ -15,8 +15,8 @@ class MatrixServiceProduct(Product):
     domain = models.CharField(max_length=255, default='domain.tld')
 
     # Default recurring price is PER_MONT, see Product class.
-    def recurring_price(self, recurring_period=RecurringPeriod.PER_30D):
+    # def recurring_price(self, recurring_period=RecurringPeriod.PER_30D):
-        return self.monthly_managment_fee
+    #     return self.monthly_managment_fee
 
     @staticmethod
     def base_image():
@@ -24,17 +24,17 @@ class MatrixServiceProduct(Product):
 #e        return VMDiskImageProduct.objects.get(uuid="93e564c5-adb3-4741-941f-718f76075f02")
         return False
 
-    @staticmethod
+    # @staticmethod
-    def allowed_recurring_periods():
+    # def allowed_recurring_periods():
-        return list(filter(
+    #     return list(filter(
-            lambda pair: pair[0] in [RecurringPeriod.PER_30D],
+    #         lambda pair: pair[0] in [RecurringPeriod.PER_30D],
-            RecurringPeriod.choices))
+    #         RecurringPeriod.choices))
 
     @property
     def one_time_price(self):
         return 30
 
-class GenericServiceProduct(Product):
+class GenericServiceProduct(models.Model):
     custom_description = models.TextField()
     custom_recurring_price = models.DecimalField(default=0.0,
             max_digits=AMOUNT_MAX_DIGITS,

uncloud_service/serializers.py

@@ -17,8 +17,8 @@ class MatrixServiceProductSerializer(serializers.ModelSerializer):
         read_only_fields = ['order', 'owner', 'status']
 
 class OrderMatrixServiceProductSerializer(MatrixServiceProductSerializer):
-    recurring_period = serializers.ChoiceField(
+    # recurring_period = serializers.ChoiceField(
-            choices=MatrixServiceProduct.allowed_recurring_periods())
+    #         choices=MatrixServiceProduct.allowed_recurring_periods())
 
     def __init__(self, *args, **kwargs):
         super(OrderMatrixServiceProductSerializer, self).__init__(*args, **kwargs)
@@ -42,8 +42,8 @@ class GenericServiceProductSerializer(serializers.ModelSerializer):
         read_only_fields = [ 'owner', 'status']
 
 class OrderGenericServiceProductSerializer(GenericServiceProductSerializer):
-    recurring_period = serializers.ChoiceField(
+    # recurring_period = serializers.ChoiceField(
-            choices=GenericServiceProduct.allowed_recurring_periods())
+    #         choices=GenericServiceProduct.allowed_recurring_periods())
 
     def __init__(self, *args, **kwargs):
         super(OrderGenericServiceProductSerializer, self).__init__(*args, **kwargs)

uncloud_vm/migrations/0001_initial.py

@@ -1,7 +1,6 @@
-# Generated by Django 3.0.6 on 2020-08-01 16:38
+# Generated by Django 3.1 on 2020-12-13 10:38
 
 from django.conf import settings
-import django.contrib.postgres.fields.jsonb
 from django.db import migrations, models
 import django.db.models.deletion
 
@@ -12,7 +11,6 @@ class Migration(migrations.Migration):
 
     dependencies = [
         migrations.swappable_dependency(settings.AUTH_USER_MODEL),
-        ('uncloud_pay', '0001_initial'),
     ]
 
     operations = [
@@ -20,7 +18,7 @@ class Migration(migrations.Migration):
             name='VMCluster',
             fields=[
                 ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
+                ('extra_data', models.JSONField(blank=True, editable=False, null=True)),
                 ('name', models.CharField(max_length=128, unique=True)),
             ],
             options={
@@ -31,7 +29,7 @@ class Migration(migrations.Migration):
             name='VMDiskImageProduct',
             fields=[
                 ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
+                ('extra_data', models.JSONField(blank=True, editable=False, null=True)),
                 ('name', models.CharField(max_length=256)),
                 ('is_os_image', models.BooleanField(default=False)),
                 ('is_public', models.BooleanField(default=False, editable=False)),
@@ -51,13 +49,13 @@ class Migration(migrations.Migration):
             name='VMHost',
             fields=[
                 ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
+                ('extra_data', models.JSONField(blank=True, editable=False, null=True)),
                 ('hostname', models.CharField(max_length=253, unique=True)),
                 ('physical_cores', models.IntegerField(default=0)),
                 ('usable_cores', models.IntegerField(default=0)),
                 ('usable_ram_in_gb', models.FloatField(default=0)),
                 ('status', models.CharField(choices=[('PENDING', 'Pending'), ('AWAITING_PAYMENT', 'Awaiting payment'), ('BEING_CREATED', 'Being created'), ('SCHEDULED', 'Scheduled'), ('ACTIVE', 'Active'), ('MODIFYING', 'Modifying'), ('DELETED', 'Deleted'), ('DISABLED', 'Disabled'), ('UNUSABLE', 'Unusable')], default='PENDING', max_length=32)),
-                ('vmcluster', models.ForeignKey(blank=True, editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.VMCluster')),
+                ('vmcluster', models.ForeignKey(blank=True, editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.vmcluster')),
             ],
             options={
                 'abstract': False,
@@ -67,35 +65,21 @@ class Migration(migrations.Migration):
             name='VMProduct',
             fields=[
                 ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
-                ('status', models.CharField(choices=[('PENDING', 'Pending'), ('AWAITING_PAYMENT', 'Awaiting payment'), ('BEING_CREATED', 'Being created'), ('SCHEDULED', 'Scheduled'), ('ACTIVE', 'Active'), ('MODIFYING', 'Modifying'), ('DELETED', 'Deleted'), ('DISABLED', 'Disabled'), ('UNUSABLE', 'Unusable')], default='AWAITING_PAYMENT', max_length=32)),
                 ('name', models.CharField(blank=True, max_length=32, null=True)),
                 ('cores', models.IntegerField()),
                 ('ram_in_gb', models.FloatField()),
-                ('order', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order')),
+                ('vmcluster', models.ForeignKey(blank=True, editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.vmcluster')),
-                ('owner', models.ForeignKey(editable=False, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+                ('vmhost', models.ForeignKey(blank=True, editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.vmhost')),
-                ('vmcluster', models.ForeignKey(blank=True, editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.VMCluster')),
-                ('vmhost', models.ForeignKey(blank=True, editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.VMHost')),
             ],
-            options={
-                'abstract': False,
-            },
         ),
         migrations.CreateModel(
             name='VMSnapshotProduct',
             fields=[
                 ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
-                ('status', models.CharField(choices=[('PENDING', 'Pending'), ('AWAITING_PAYMENT', 'Awaiting payment'), ('BEING_CREATED', 'Being created'), ('SCHEDULED', 'Scheduled'), ('ACTIVE', 'Active'), ('MODIFYING', 'Modifying'), ('DELETED', 'Deleted'), ('DISABLED', 'Disabled'), ('UNUSABLE', 'Unusable')], default='AWAITING_PAYMENT', max_length=32)),
                 ('gb_ssd', models.FloatField(editable=False)),
                 ('gb_hdd', models.FloatField(editable=False)),
-                ('order', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order')),
+                ('vm', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='snapshots', to='uncloud_vm.vmproduct')),
-                ('owner', models.ForeignKey(editable=False, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
-                ('vm', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='snapshots', to='uncloud_vm.VMProduct')),
             ],
-            options={
-                'abstract': False,
-            },
         ),
         migrations.CreateModel(
             name='VMNetworkCard',
@@ -103,35 +87,25 @@ class Migration(migrations.Migration):
                 ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                 ('mac_address', models.BigIntegerField()),
                 ('ip_address', models.GenericIPAddressField(blank=True, null=True)),
-                ('vm', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.VMProduct')),
+                ('vm', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.vmproduct')),
             ],
         ),
         migrations.CreateModel(
             name='VMDiskProduct',
             fields=[
                 ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('extra_data', django.contrib.postgres.fields.jsonb.JSONField(blank=True, editable=False, null=True)),
-                ('status', models.CharField(choices=[('PENDING', 'Pending'), ('AWAITING_PAYMENT', 'Awaiting payment'), ('BEING_CREATED', 'Being created'), ('SCHEDULED', 'Scheduled'), ('ACTIVE', 'Active'), ('MODIFYING', 'Modifying'), ('DELETED', 'Deleted'), ('DISABLED', 'Disabled'), ('UNUSABLE', 'Unusable')], default='AWAITING_PAYMENT', max_length=32)),
                 ('size_in_gb', models.FloatField(blank=True)),
                 ('disk_type', models.CharField(choices=[('ceph/ssd', 'Ceph Ssd'), ('ceph/hdd', 'Ceph Hdd'), ('local/ssd', 'Local Ssd'), ('local/hdd', 'Local Hdd')], default='ceph/ssd', max_length=20)),
-                ('image', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.VMDiskImageProduct')),
+                ('image', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.vmdiskimageproduct')),
-                ('order', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order')),
+                ('vm', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.vmproduct')),
-                ('owner', models.ForeignKey(editable=False, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
-                ('vm', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.VMProduct')),
             ],
-            options={
-                'abstract': False,
-            },
         ),
         migrations.CreateModel(
             name='VMWithOSProduct',
             fields=[
-                ('vmproduct_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='uncloud_vm.VMProduct')),
+                ('vmproduct_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='uncloud_vm.vmproduct')),
-                ('primary_disk', models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.VMDiskProduct')),
+                ('primary_disk', models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_vm.vmdiskproduct')),
             ],
-            options={
-                'abstract': False,
-            },
             bases=('uncloud_vm.vmproduct',),
         ),
     ]

uncloud_vm/migrations/0002_auto_20200801_2332.py

@@ -1,30 +0,0 @@
-# Generated by Django 3.0.8 on 2020-08-01 23:32
-
-from django.db import migrations, models
-import django.db.models.deletion
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_pay', '0003_auto_20200801_2332'),
-        ('uncloud_vm', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='vmdiskproduct',
-            name='order',
-            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order'),
-        ),
-        migrations.AlterField(
-            model_name='vmproduct',
-            name='order',
-            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order'),
-        ),
-        migrations.AlterField(
-            model_name='vmsnapshotproduct',
-            name='order',
-            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='uncloud_pay.Order'),
-        ),
-    ]

uncloud_vm/migrations/0002_vmproduct_owner.py

@@ -0,0 +1,21 @@
+# Generated by Django 3.1.4 on 2021-04-14 10:40
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+        ('uncloud_vm', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='vmproduct',
+            name='owner',
+            field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL),
+        ),
+    ]

uncloud_vm/migrations/0003_auto_20200808_1953.py

@@ -1,43 +0,0 @@
-# Generated by Django 3.1 on 2020-08-08 19:53
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('uncloud_vm', '0002_auto_20200801_2332'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='vmcluster',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-        migrations.AlterField(
-            model_name='vmdiskimageproduct',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-        migrations.AlterField(
-            model_name='vmdiskproduct',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-        migrations.AlterField(
-            model_name='vmhost',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-        migrations.AlterField(
-            model_name='vmproduct',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-        migrations.AlterField(
-            model_name='vmsnapshotproduct',
-            name='extra_data',
-            field=models.JSONField(blank=True, editable=False, null=True),
-        ),
-    ]

uncloud_vm/migrations/0003_vmproduct_created_order_at.py

@@ -0,0 +1,20 @@
+# Generated by Django 3.1.4 on 2021-04-14 10:46
+
+import datetime
+from django.db import migrations, models
+from django.utils.timezone import utc
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_vm', '0002_vmproduct_owner'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='vmproduct',
+            name='created_order_at',
+            field=models.DateTimeField(default=datetime.datetime(2021, 4, 14, 10, 46, 14, 96330, tzinfo=utc)),
+        ),
+    ]

uncloud_vm/migrations/0004_auto_20210414_1048.py

@@ -0,0 +1,24 @@
+# Generated by Django 3.1.4 on 2021-04-14 10:48
+
+import datetime
+from django.db import migrations, models
+from django.utils.timezone import utc
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_vm', '0003_vmproduct_created_order_at'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='vmproduct',
+            name='created_order_at',
+        ),
+        migrations.AddField(
+            model_name='vmproduct',
+            name='create_order_at',
+            field=models.DateTimeField(default=datetime.datetime(2021, 4, 14, 10, 48, 6, 641056, tzinfo=utc)),
+        ),
+    ]

uncloud_vm/migrations/0005_auto_20210414_1119.py

@@ -0,0 +1,24 @@
+# Generated by Django 3.1.4 on 2021-04-14 11:19
+
+import datetime
+from django.db import migrations, models
+from django.utils.timezone import utc
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_vm', '0004_auto_20210414_1048'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='vmproduct',
+            name='create_order_at',
+        ),
+        migrations.AddField(
+            model_name='vmproduct',
+            name='created_order_at',
+            field=models.DateTimeField(default=datetime.datetime(2021, 4, 14, 11, 19, 39, 447274, tzinfo=utc)),
+        ),
+    ]

uncloud_vm/migrations/0006_auto_20210414_1122.py

@@ -0,0 +1,20 @@
+# Generated by Django 3.1.4 on 2021-04-14 11:22
+
+import datetime
+from django.db import migrations, models
+from django.utils.timezone import utc
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('uncloud_vm', '0005_auto_20210414_1119'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='vmproduct',
+            name='created_order_at',
+            field=models.DateTimeField(default=datetime.datetime(2021, 4, 14, 11, 22, 11, 352536, tzinfo=utc)),
+        ),
+    ]

uncloud_vm/models.py

@@ -1,3 +1,6 @@
+import datetime
+from django.utils import timezone
+
 from django.db import models
 from django.contrib.auth import get_user_model
 
@@ -49,7 +52,9 @@ class VMHost(UncloudModel):
 
 
 
-class VMProduct(Product):
+class VMProduct(models.Model):
+    owner = models.ForeignKey(get_user_model(), on_delete=models.CASCADE,
+                              blank=True, null=True)
     vmhost = models.ForeignKey(
         VMHost, on_delete=models.CASCADE, editable=False, blank=True, null=True
     )
@@ -58,38 +63,35 @@ class VMProduct(Product):
         VMCluster, on_delete=models.CASCADE, editable=False, blank=True, null=True
     )
 
-    # VM-specific. The name is only intended for customers: it's a pain to
-    # remember IDs (speaking from experience as ungleich customer)!
     name = models.CharField(max_length=32, blank=True, null=True)
     cores = models.IntegerField()
     ram_in_gb = models.FloatField()
+    created_order_at = models.DateTimeField(default=timezone.make_aware(datetime.datetime.now()))
 
-    # Default recurring price is PER_MONTH, see uncloud_pay.models.Product.
     @property
     def recurring_price(self):
         return self.cores * 3 + self.ram_in_gb * 4
 
-    def __str__(self):
-        if self.name:
-            name = f"{self.name} ({self.id})"
-        else:
-            name = self.id
-
-        return "VM {}: {} cores {} gb ram".format(name,
-                                                  self.cores,
-                                                  self.ram_in_gb)
-
     @property
     def description(self):
         return "Virtual machine '{}': {} core(s), {}GB memory".format(
                 self.name, self.cores, self.ram_in_gb)
 
-    @staticmethod
+    # @staticmethod
-    def allowed_recurring_periods():
+    # def allowed_recurring_periods():
-        return list(filter(
+    #     return list(filter(
-            lambda pair: pair[0] in [RecurringPeriod.PER_365D,
+    #         lambda pair: pair[0] in [RecurringPeriod.PER_365D,
-                RecurringPeriod.PER_30D, RecurringPeriod.PER_HOUR],
+    #             RecurringPeriod.PER_30D, RecurringPeriod.PER_HOUR],
-            RecurringPeriod.choices))
+    #         RecurringPeriod.choices))
+
+    def create_order_at(self, dt):
+        self.created_order_at = dt
+
+    def create_or_update_order(self, when_to_start):
+        self.created_order_at = when_to_start
+
+    def __str__(self):
+        return f"VM id={self.id},name={self.name},cores={self.cores},ram_in_gb={self.ram_in_gb}"
 
 
 class VMWithOSProduct(VMProduct):
@@ -142,7 +144,7 @@ class VMDiskType(models.TextChoices):
     LOCAL_HDD  = 'local/hdd'
 
 
-class VMDiskProduct(Product):
+class VMDiskProduct(models.Model):
     """
     The VMDiskProduct is attached to a VM.
 
@@ -164,9 +166,7 @@ class VMDiskProduct(Product):
         default=VMDiskType.CEPH_SSD)
 
     def __str__(self):
-        return "{} disk for VM '{}': {}GB".format(self.disk_type,
+        return f"Disk {self.size_in_gb}GB ({self.disk_type}) for {self.vm}"
-                                                  self.vm.name,
-                                                  self.size_in_gb)
 
     @property
     def recurring_price(self):
@@ -191,7 +191,7 @@ class VMNetworkCard(models.Model):
                                               null=True)
 
 
-class VMSnapshotProduct(Product):
+class VMSnapshotProduct(models.Model):
     gb_ssd = models.FloatField(editable=False)
     gb_hdd = models.FloatField(editable=False)
 

uncloud_vm/serializers.py

@@ -101,8 +101,8 @@ class VMProductSerializer(serializers.ModelSerializer):
         read_only_fields = ['order', 'owner', 'status']
 
 class OrderVMProductSerializer(VMProductSerializer):
-    recurring_period = serializers.ChoiceField(
+    # recurring_period = serializers.ChoiceField(
-            choices=VMWithOSProduct.allowed_recurring_periods())
+    #         choices=VMWithOSProduct.allowed_recurring_periods())
 
     def __init__(self, *args, **kwargs):
         super(VMProductSerializer, self).__init__(*args, **kwargs)
@@ -133,8 +133,8 @@ class DCLVMProductSerializer(serializers.HyperlinkedModelSerializer):
     """
 
     # Custom field used at creation (= ordering) only.
-    recurring_period = serializers.ChoiceField(
+    # recurring_period = serializers.ChoiceField(
-        choices=VMProduct.allowed_recurring_periods())
+    #     choices=VMProduct.allowed_recurring_periods())
 
     os_disk_uuid = serializers.UUIDField()
     # os_disk_size =

uncloud_vm/tests.py

@@ -79,22 +79,6 @@ class VMTestCase(TestCase):
 #            msg='VMDiskProduct created with disk image whose status is not active.'
 #        )
 
-    def test_vm_disk_product_creation(self):
-        """Ensure that a user can only create a VMDiskProduct for an existing VM"""
-
-        disk_image = VMDiskImageProduct.objects.create(
-            owner=self.user, name='disk_image', is_os_image=True, is_public=True, size_in_gb=10,
-            status='active'
-        )
-
-        with self.assertRaises(ValidationError, msg='User created a VMDiskProduct for non-existing VM'):
-            # Create VMProduct object but don't save it in database
-            vm = VMProduct()
-
-            vm_disk_product = VMDiskProduct.objects.create(
-                owner=self.user, vm=vm, image=disk_image, size_in_gb=10
-            )
-
 # TODO: the logic tested by this test is not implemented yet.
 #    def test_vm_disk_product_creation_for_someone_else(self):
 #        """Ensure that a user can only create a VMDiskProduct for his/her own VM"""