diff --git a/K8S.md b/K8S.md index 714c4959..5eb1c6d8 100644 --- a/K8S.md +++ b/K8S.md @@ -1,138 +1,54 @@ -# Running dynamicweb on Kubernetes +# Deploying dynamicweb on Kubernetes -One image, `dynamicweb`, runs three roles. Pick the role with the `ROLE` environment variable: +[k8s/dynamicweb.yaml](k8s/dynamicweb.yaml) describes what runs: the web app, the Celery worker, +Redis, a volume for uploaded media, and an Ingress. PostgreSQL 18 is **not** included: use your +own database and point the app at it. -| `ROLE` | What it does | -|---|---| -| `web` (default) | migrations (unless `RUN_MIGRATIONS=False`), then gunicorn on port 8000 | -| `celery` | the Celery worker | -| `migrate` | runs the migrations and exits (for a Job) | +> Not tested on a cluster. The image itself is tested with `docker compose` (see [DOCKER.md](DOCKER.md)). -Do not set `command:` or `args:` on the containers. The image's entrypoint writes the database -settings from the environment and waits for Postgres, and `ROLE` does the rest. +## Steps -> The manifests below follow the image's behaviour but have **not been tested on a cluster**. -> The image itself is tested with `docker compose` (see [DOCKER.md](DOCKER.md)). +1. **Push the image** (build it once, from the repo): + ```sh + docker compose build + docker tag dynamicweb:latest /dynamicweb: + docker push /dynamicweb: + ``` +2. **Make an env file** from `.env.docker` with your real values. Change at least: + ``` + DEBUG=False + SEED_DUMMY_DATA=False + DJANGO_SECRET_KEY= + POSTGRES_HOST= # and POSTGRES_PORT/DB/USER/PASSWORD + BEHIND_TLS_PROXY=True # TLS ends at the ingress + UNGLEICH_SITE_CONFIGS= + ``` + plus your OpenNebula, LDAP, Stripe, email and reCAPTCHA values. Leave the Redis URLs alone; + the manifest creates a service called `redis`. +3. **Store it as a Secret:** + ```sh + kubectl create secret generic dynamicweb --from-env-file= + ``` +4. **Edit the manifest:** your image name (it appears twice) and the Ingress hosts (one rule for + each domain, with `www.` variants). Set `ingressClassName` to your controller. +5. **Deploy:** + ```sh + kubectl apply -f k8s/dynamicweb.yaml + kubectl logs -f deploy/dynamicweb-web # ready at "Listening at" + ``` + The first start runs the database migrations, so it takes a few minutes. +6. **Check:** `curl -I -H 'Host: ungleich.ch' http:///en-us/` returns 200. Admin + is at `/en-us/admin/login/` on any site's address. -## What you need outside the cluster +## Good to know -- **PostgreSQL 18** holding the restored database. The app creates nothing but the schema changes - from its migrations; pages, sites and templates come from the database. -- **Redis** (cache and Celery broker), in the cluster or outside. - -## Configuration - -Plain values go in a ConfigMap, passwords and keys in a Secret, both attached with `envFrom`. -The full list is `.env.docker`. The ones that matter: - -| Variable | Notes | -|---|---| -| `DEBUG` | `False`. Loads the production settings. | -| `SEED_DUMMY_DATA` | `False`. | -| `DJANGO_SECRET_KEY` | Secret. Required: with several pods nothing else keeps the key the same. | -| `POSTGRES_HOST/PORT/DB/USER/PASSWORD` | the database | -| `CELERY_BROKER_URL`, `CELERY_RESULT_BACKEND`, `CACHE_URL` | Redis URLs | -| `UNGLEICH_SITE_CONFIGS` | JSON: host to urlconf, one entry per site | -| `BEHIND_TLS_PROXY` | `True` when TLS ends at the ingress (see below) | -| `OPENNEBULA_*`, `LDAP*`, `STRIPE_*`, `EMAIL_*`, `RECAPTCHA_*`, `WEBHOOK_SECRET` | the real integrations | -| `GUNICORN_WORKERS`, `GUNICORN_TIMEOUT` | default 3 and 120 | -| `RUN_MIGRATIONS` | `False` on the web Deployment when a Job runs the migrations | - -## Manifests - -```yaml -apiVersion: batch/v1 -kind: Job -metadata: {name: dynamicweb-migrate} -spec: - backoffLimit: 2 - template: - spec: - restartPolicy: Never - containers: - - name: migrate - image: REGISTRY/dynamicweb:VERSION - env: [{name: ROLE, value: migrate}] - envFrom: [{configMapRef: {name: dynamicweb}}, {secretRef: {name: dynamicweb}}] ---- -apiVersion: apps/v1 -kind: Deployment -metadata: {name: dynamicweb-web} -spec: - replicas: 2 - selector: {matchLabels: {app: dynamicweb-web}} - template: - metadata: {labels: {app: dynamicweb-web}} - spec: - containers: - - name: web - image: REGISTRY/dynamicweb:VERSION - env: [{name: RUN_MIGRATIONS, value: "False"}] - envFrom: [{configMapRef: {name: dynamicweb}}, {secretRef: {name: dynamicweb}}] - ports: [{containerPort: 8000}] - readinessProbe: {tcpSocket: {port: 8000}, periodSeconds: 10} - volumeMounts: [{name: media, mountPath: /usr/src/app/media}] - volumes: - - name: media - persistentVolumeClaim: {claimName: dynamicweb-media} # ReadWriteMany if replicas > 1 ---- -apiVersion: apps/v1 -kind: Deployment -metadata: {name: dynamicweb-celery} -spec: - replicas: 1 - selector: {matchLabels: {app: dynamicweb-celery}} - template: - metadata: {labels: {app: dynamicweb-celery}} - spec: - containers: - - name: celery - image: REGISTRY/dynamicweb:VERSION - env: [{name: ROLE, value: celery}] - envFrom: [{configMapRef: {name: dynamicweb}}, {secretRef: {name: dynamicweb}}] ---- -apiVersion: v1 -kind: Service -metadata: {name: dynamicweb-web} -spec: - selector: {app: dynamicweb-web} - ports: [{port: 80, targetPort: 8000}] -``` - -Add an Ingress that sends every site's hostname (all the domains in `UNGLEICH_SITE_CONFIGS`, with -`www.` variants) to the `dynamicweb-web` Service. - -## What makes it work well - -- **Run the migrations once, as a Job.** Apply the Job and wait for it to finish before rolling out - the web Deployment. With `RUN_MIGRATIONS=False` the web pods start in seconds and several - replicas never migrate at the same time. -- **Keep the `Host` header.** The site is chosen from the request's host, so the ingress must pass - the original host through (the default for most ingress controllers). All hostnames must be in - `UNGLEICH_SITE_CONFIGS`, in the database's `django_site`/alias tables, and in `ALLOWED_HOSTS` - (`dynamicweb/settings/prod.py` lists the production domains; add new ones there). -- **TLS at the ingress:** set `BEHIND_TLS_PROXY=True`. Without it Django thinks requests are plain - http and rejects form posts (admin login, checkout) with a CSRF error. The ingress must set - `X-Forwarded-Proto`; the common ones do. -- **Static files** are collected when the container starts and served by the app (WhiteNoise); nothing - to configure. -- **Uploaded media** (the files managed in the admin) is written to `/usr/src/app/media` and served - by the app. Mount a shared volume there (`ReadWriteMany`), or run one web replica with a normal - volume. Without a volume, uploads vanish when a pod restarts. -- **Readiness** uses the TCP port: gunicorn only starts listening after start-up work is done. An - HTTP probe needs a `Host` header that is in `ALLOWED_HOSTS`, which is why it is not used. -- **Logs** go to stdout: `kubectl logs deploy/dynamicweb-web`, `kubectl logs deploy/dynamicweb-celery`. -- **Start-up needs OpenNebula reachable.** The app reads its VM templates at import, once per - gunicorn worker. If OpenNebula is down the app still starts but logs an error, and VM - creation is unavailable until a restart. -- **Celery**: keep one replica unless the tasks are known to be safe to run in parallel. - -## Checking a deployment - -```sh -kubectl logs job/dynamicweb-migrate # "No migrations to apply" or the list applied -kubectl rollout status deploy/dynamicweb-web -curl -I -H 'Host: ungleich.ch' http:///en-us/ # 200 -``` - -The admin is at `/en-us/admin/login/` on any site's address. +- **Run one web replica first.** Each web pod migrates the database when it starts. After the first + start you can scale (`kubectl scale deploy/dynamicweb-web --replicas=2`), with the media volume + changed to `ReadWriteMany` so every pod sees the same uploads. +- **Ingress must keep the original `Host` header** (the default). The site is chosen from the + host, and every domain must be in `UNGLEICH_SITE_CONFIGS` and in `ALLOWED_HOSTS` + (`dynamicweb/settings/prod.py` lists the current ones). +- **Updating:** push a new image tag, change the image in the manifest and apply it again. +- **Logs:** `kubectl logs deploy/dynamicweb-web` and `kubectl logs deploy/dynamicweb-celery`. +- **Optional hardening:** run the migrations once as a Job instead of in every pod. Use a Job with + `env: ROLE=migrate`, and add `RUN_MIGRATIONS=False` to the web Deployment. diff --git a/k8s/dynamicweb.yaml b/k8s/dynamicweb.yaml new file mode 100644 index 00000000..37c132d6 --- /dev/null +++ b/k8s/dynamicweb.yaml @@ -0,0 +1,87 @@ +# Apply with: kubectl apply -f k8s/dynamicweb.yaml (see K8S.md) +# Edit: the image (3 places) and the ingress hosts. Needs the Secret "dynamicweb". +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: {name: dynamicweb-media} +spec: + accessModes: [ReadWriteOnce] # ReadWriteMany if you run more than one web replica + resources: {requests: {storage: 5Gi}} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: {name: redis} +spec: + replicas: 1 + selector: {matchLabels: {app: redis}} + template: + metadata: {labels: {app: redis}} + spec: + containers: + - name: redis + image: redis:6-alpine + ports: [{containerPort: 6379}] +--- +apiVersion: v1 +kind: Service +metadata: {name: redis} +spec: + selector: {app: redis} + ports: [{port: 6379}] +--- +apiVersion: apps/v1 +kind: Deployment +metadata: {name: dynamicweb-web} +spec: + replicas: 1 + selector: {matchLabels: {app: dynamicweb-web}} + template: + metadata: {labels: {app: dynamicweb-web}} + spec: + containers: + - name: web + image: REGISTRY/dynamicweb:VERSION + envFrom: [{secretRef: {name: dynamicweb}}] + ports: [{containerPort: 8000}] + readinessProbe: {tcpSocket: {port: 8000}, periodSeconds: 10} + volumeMounts: [{name: media, mountPath: /usr/src/app/media}] + volumes: + - name: media + persistentVolumeClaim: {claimName: dynamicweb-media} +--- +apiVersion: v1 +kind: Service +metadata: {name: dynamicweb-web} +spec: + selector: {app: dynamicweb-web} + ports: [{port: 80, targetPort: 8000}] +--- +apiVersion: apps/v1 +kind: Deployment +metadata: {name: dynamicweb-celery} +spec: + replicas: 1 + selector: {matchLabels: {app: dynamicweb-celery}} + template: + metadata: {labels: {app: dynamicweb-celery}} + spec: + containers: + - name: celery + image: REGISTRY/dynamicweb:VERSION + env: [{name: ROLE, value: celery}] + envFrom: [{secretRef: {name: dynamicweb}}] +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: {name: dynamicweb} +spec: + ingressClassName: nginx # your ingress controller + # tls: [{secretName: dynamicweb-tls, hosts: [ungleich.ch]}] + rules: + - host: ungleich.ch # one rule per domain in UNGLEICH_SITE_CONFIGS + http: + paths: + - {path: /, pathType: Prefix, backend: {service: {name: dynamicweb-web, port: {number: 80}}}} + - host: www.ungleich.ch + http: + paths: + - {path: /, pathType: Prefix, backend: {service: {name: dynamicweb-web, port: {number: 80}}}}