forked from ungleich-public/cdist
		
	Add 'real world example' walkthrough docs chapter.
This commit is contained in:
		
					parent
					
						
							
								aa80e8f87d
							
						
					
				
			
			
				commit
				
					
						d6952543a7
					
				
			
		
					 3 changed files with 575 additions and 0 deletions
				
			
		| 
						 | 
					@ -5,6 +5,7 @@ next:
 | 
				
			||||||
	* New global explorer: os_release (Ľubomír Kučera)
 | 
						* New global explorer: os_release (Ľubomír Kučera)
 | 
				
			||||||
	* Type __docker: Update type, install docker CE (Ľubomír Kučera)
 | 
						* Type __docker: Update type, install docker CE (Ľubomír Kučera)
 | 
				
			||||||
	* Type __package_apt: Write a message when a package is installed or removed; shellcheck (Jonas Weber)
 | 
						* Type __package_apt: Write a message when a package is installed or removed; shellcheck (Jonas Weber)
 | 
				
			||||||
 | 
						* Documentation: Add 'Dive into real world cdist' walkthrough chapter (Darko Poljak)
 | 
				
			||||||
 | 
					
 | 
				
			||||||
4.10.2: 2018-09-06
 | 
					4.10.2: 2018-09-06
 | 
				
			||||||
	* Type __letsencrypt_cert: Add support for devuan ascii (Darko Poljak)
 | 
						* Type __letsencrypt_cert: Add support for devuan ascii (Darko Poljak)
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
							
								
								
									
										573
									
								
								docs/src/cdist-real-world.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										573
									
								
								docs/src/cdist-real-world.rst
									
										
									
									
									
										Normal file
									
								
							| 
						 | 
					@ -0,0 +1,573 @@
 | 
				
			||||||
 | 
					Dive into real world cdist
 | 
				
			||||||
 | 
					==========================
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Introduction
 | 
				
			||||||
 | 
					------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					This walkthrough shows real world cdist configuration example.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Sample target host is named **test.ungleich.ch**.
 | 
				
			||||||
 | 
					Just replace **test.ungleich.ch** with your target hostname.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Our goal is to configure python application hosting. For writing sample
 | 
				
			||||||
 | 
					application we will use `Bottle <http://bottlepy.org>`_ WSGI micro web-framework.
 | 
				
			||||||
 | 
					It will use PostgreSQL database and it will list items from **items** table.
 | 
				
			||||||
 | 
					It will be served by uWSGI server. We will also use the Nginx web server
 | 
				
			||||||
 | 
					as a reverse proxy and we want HTTPS.
 | 
				
			||||||
 | 
					For HTTPS we will use Let's Encrypt certificate.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					For setting up hosting we want to use cdist so we will write a new type
 | 
				
			||||||
 | 
					for that. This type will:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					- install required packages
 | 
				
			||||||
 | 
					- create OS user, user home directory and application home directory
 | 
				
			||||||
 | 
					- create PostgreSQL database
 | 
				
			||||||
 | 
					- configure uWSGI
 | 
				
			||||||
 | 
					- configure Let's Encrypt certificate
 | 
				
			||||||
 | 
					- configure nginx.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Our type will not create the actual python application. Its intention is only
 | 
				
			||||||
 | 
					to configure hosing for specified user and project. It is up to the user to
 | 
				
			||||||
 | 
					create his/her applications.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					So let's start.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating type layout
 | 
				
			||||||
 | 
					--------------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					We will create a new custom type. Let's call it **__sample_bottle_hosting**.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Go to **~/.cdist/type** directory (create it if it does not exist) and create
 | 
				
			||||||
 | 
					new type layout::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    cd ~/.cdist/type
 | 
				
			||||||
 | 
					    mkdir __sample_bottle_hosting
 | 
				
			||||||
 | 
					    cd __sample_bottle_hosting
 | 
				
			||||||
 | 
					    touch manifest gencode-remote
 | 
				
			||||||
 | 
					    mkdir parameter
 | 
				
			||||||
 | 
					    touch parameter/required
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating __sample_bottle_hosting type parameters
 | 
				
			||||||
 | 
					------------------------------------------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Our type will be configurable through the means of parameters. Let's define
 | 
				
			||||||
 | 
					the following parameters:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					projectname
 | 
				
			||||||
 | 
					    name for the project, needed for uWSGI ini file
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					user
 | 
				
			||||||
 | 
					    user name
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					domain
 | 
				
			||||||
 | 
					    target host domain, needed for Let's Encrypt certificate.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					We define parameters to make our type reusable for different projects, user and domain.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Define required parameters::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    printf "projectname\n" >> parameter/required
 | 
				
			||||||
 | 
					    printf "user\n" >> parameter/required
 | 
				
			||||||
 | 
					    printf "domain\n" >> parameter/required
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					For details on type parameters see `Defining parameters <cdist-type.html#defining-parameters>`_.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating __sample_bottle_hosting type manifest
 | 
				
			||||||
 | 
					----------------------------------------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Next step is to define manifest (~/.cdist/type/__sample_bottle_hosting/manifest).
 | 
				
			||||||
 | 
					We also want our type to currently support only Devuan. So we will start by
 | 
				
			||||||
 | 
					checking target host OS.  We will use `os <cdist-reference.html#explorers>`_
 | 
				
			||||||
 | 
					global explorer::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    os=$(cat "$__global/explorer/os")
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    case "$os" in
 | 
				
			||||||
 | 
					        devuan)
 | 
				
			||||||
 | 
					            :
 | 
				
			||||||
 | 
					        ;;
 | 
				
			||||||
 | 
					        *)
 | 
				
			||||||
 | 
					            echo "OS $os currently not supported" >&2
 | 
				
			||||||
 | 
					            exit 1
 | 
				
			||||||
 | 
					        ;;
 | 
				
			||||||
 | 
					    esac
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					If target host OS is not Devuan then we print error message to stderr
 | 
				
			||||||
 | 
					and exit. For other OS-es support we should check and change package names
 | 
				
			||||||
 | 
					we should install, because packages differ in different OS-es and in different
 | 
				
			||||||
 | 
					OS distributions like GNU/Linux distributions. There can also be a different
 | 
				
			||||||
 | 
					configuration locations (e.g. nginx config directory could be in /usr/local tree).
 | 
				
			||||||
 | 
					If we detected unsupported OS we should error out. cdist will stop configuration
 | 
				
			||||||
 | 
					process and output error message.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating user and user directories
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Then we create user and his/her home directory and application home directory.
 | 
				
			||||||
 | 
					We will use existing cdist types `__user <man7/cdist-type__user.html>`_ and `__directory <man7/cdist-type__directory.html>`_::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    user="$(cat "$__object/parameter/user")"
 | 
				
			||||||
 | 
					    home="/home/$user"
 | 
				
			||||||
 | 
					    apphome="$home/app"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # create user
 | 
				
			||||||
 | 
					    __user "$user" --home "$home" --shell /bin/bash
 | 
				
			||||||
 | 
					    # create user home dir
 | 
				
			||||||
 | 
					    require="__user/$user" __directory "$home" \
 | 
				
			||||||
 | 
					        --owner "$user" --group "$user" --mode 0755
 | 
				
			||||||
 | 
					    # create app home dir
 | 
				
			||||||
 | 
					    require="__user/$user __directory/$home" __directory "$apphome" \
 | 
				
			||||||
 | 
					        --state present --owner "$user" --group "$user" --mode 0755
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					First we define *user*, *home* and *apphome* variables. User is defined by type's
 | 
				
			||||||
 | 
					**user** parameter. Here we use **require** which is cdist's way to define dependencies.
 | 
				
			||||||
 | 
					User home directory should be created **after** user is created. And application
 | 
				
			||||||
 | 
					home directory is created **after** both user and user home directory are created.
 | 
				
			||||||
 | 
					For details on **require** see `Dependencies <cdist-manifest.html#dependencies>`_.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Installing packages
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Install required packages using existing `__package <man7/cdist-type__package.html>`_ type.
 | 
				
			||||||
 | 
					Before installing package we want to update apt package index using
 | 
				
			||||||
 | 
					`__apt_update_index <man7/cdist-type__apt_update_index.html>`_::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # define packages that need to be installed
 | 
				
			||||||
 | 
					    packages_to_install="nginx uwsgi-plugin-python3 python3-dev python3-pip postgresql postgresql-contrib libpq-dev python3-venv uwsgi python3-psycopg2"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # update package index
 | 
				
			||||||
 | 
					    __apt_update_index
 | 
				
			||||||
 | 
					    # install packages
 | 
				
			||||||
 | 
					    for package in $packages_to_install
 | 
				
			||||||
 | 
					        do require="__apt_update_index" __package $package --state=present
 | 
				
			||||||
 | 
					    done
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Here we use shell for loop. It executes **require="__apt_update_index" __package**
 | 
				
			||||||
 | 
					for each member in a list we define in **packages_to_install** variable.
 | 
				
			||||||
 | 
					This is much nicer then having as many **require="__apt_update_index" __package**
 | 
				
			||||||
 | 
					lines as there are packages we want to install.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					For python packages we use `__package_pip <man7/cdist-type__package_pip.html>`_::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # install pip3 packages
 | 
				
			||||||
 | 
					    for package in bottle bottle-pgsql; do
 | 
				
			||||||
 | 
					        __package_pip --pip pip3 $package
 | 
				
			||||||
 | 
					    done
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating PostgreSQL database
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Create PostgreSQL database using `__postgres_database <man7/cdist-type__postgres_database.html>`_
 | 
				
			||||||
 | 
					and `__postgres_role <man7/cdist-type__postgres_role.html>`_ for creating database user::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    #PostgreSQL db & user
 | 
				
			||||||
 | 
					    postgres_server=postgresql
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # create PostgreSQL db user
 | 
				
			||||||
 | 
					    require="__package/postgresql" __postgres_role $user --login --createdb
 | 
				
			||||||
 | 
					    # create PostgreSQL db
 | 
				
			||||||
 | 
					    require="__postgres_role/$user __package/postgresql" __postgres_database $user \
 | 
				
			||||||
 | 
					        --owner $user
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Configuring uWSGI
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Configure uWSGI using `__file <man7/cdist-type__file.html>`_ type::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # configure uWSGI
 | 
				
			||||||
 | 
					    projectname="$(cat "$__object/parameter/projectname")"
 | 
				
			||||||
 | 
					    require="__package/uwsgi" __file /etc/uwsgi/apps-enabled/$user.ini \
 | 
				
			||||||
 | 
					                --owner root --group root --mode 0644 \
 | 
				
			||||||
 | 
					                --state present \
 | 
				
			||||||
 | 
					                --source - << EOF
 | 
				
			||||||
 | 
					    [uwsgi]
 | 
				
			||||||
 | 
					    socket = $apphome/uwsgi.sock
 | 
				
			||||||
 | 
					    chdir = $apphome
 | 
				
			||||||
 | 
					    wsgi-file = $projectname/wsgi.py
 | 
				
			||||||
 | 
					    touch-reload = $projectname/wsgi.py
 | 
				
			||||||
 | 
					    processes = 4
 | 
				
			||||||
 | 
					    threads = 2
 | 
				
			||||||
 | 
					    chmod-socket = 666
 | 
				
			||||||
 | 
					    daemonize=true
 | 
				
			||||||
 | 
					    vacuum = true
 | 
				
			||||||
 | 
					    uid = $user
 | 
				
			||||||
 | 
					    gid = $user
 | 
				
			||||||
 | 
					    EOF
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					We require package uWSGI present in order to create **/etc/uwsgi/apps-enabled/$user.ini** file.
 | 
				
			||||||
 | 
					Installation of uWSGI also creates configuration layout: **/etc/uwsgi/apps-enabled**.
 | 
				
			||||||
 | 
					If this directory does not exist then **__file** type would error.
 | 
				
			||||||
 | 
					We also use stdin as file content source. For details see `Input from stdin <cdist-type.html#input-from-stdin>`_.
 | 
				
			||||||
 | 
					For feading stdin we use here-document (**<<** operator). It allows redirection of subsequent
 | 
				
			||||||
 | 
					lines read by the shell to the input of a command until a line containing only the delimiter
 | 
				
			||||||
 | 
					and a newline, with no blank characters in between (EOF in our case).
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Configuring nginx for Let's Encrypt and HTTPS redirection
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Next configure nginx for Let's Encrypt and for HTTP -> HTTPS redirection. For this
 | 
				
			||||||
 | 
					purpose we will create new type **__sample_nginx_http_letsencrypt_and_ssl_redirect**
 | 
				
			||||||
 | 
					and use it here::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    domain="$(cat "$__object/parameter/domain")"
 | 
				
			||||||
 | 
					    webroot="/var/www/html"
 | 
				
			||||||
 | 
					    __sample_nginx_http_letsencrypt_and_ssl_redirect "$domain" --webroot "$webroot"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Configuring certificate creation
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					After HTTP nginx configuration we will create Let's Encrypt certificate using
 | 
				
			||||||
 | 
					`__letsencrypt_cert <man7/cdist-type__letsencrypt_cert.html>`_ type.
 | 
				
			||||||
 | 
					For Let's Encrypt cert configuration ensure that there is a DNS entry for your
 | 
				
			||||||
 | 
					domain. We assure that cert creation is applied after nginx HTTP is configured
 | 
				
			||||||
 | 
					for Let's Encrypt to work::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # create SSL cert
 | 
				
			||||||
 | 
					    require="__package/nginx __sample_nginx_http_letsencrypt_and_ssl_redirect/$domain" \
 | 
				
			||||||
 | 
					        __letsencrypt_cert --admin-email admin@test.ungleich.ch \
 | 
				
			||||||
 | 
					            --webroot "$webroot" \
 | 
				
			||||||
 | 
					            --automatic-renewal \
 | 
				
			||||||
 | 
					            --renew-hook "service nginx reload" \
 | 
				
			||||||
 | 
					            --domain "$domain" \
 | 
				
			||||||
 | 
					            "$domain"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Configuring nginx HTTPS server with uWSGI upstream
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Then we can configure nginx HTTPS server that will use created Let's Encrypt certificate::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # configure nginx
 | 
				
			||||||
 | 
					    require="__package/nginx __letsencrypt_cert/$domain" \
 | 
				
			||||||
 | 
					        __file "/etc/nginx/sites-enabled/https-$domain" \
 | 
				
			||||||
 | 
					        --source - --mode 0644 << EOF
 | 
				
			||||||
 | 
					    upstream _bottle {
 | 
				
			||||||
 | 
					        server unix:$apphome/uwsgi.sock;
 | 
				
			||||||
 | 
					    }
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    server {
 | 
				
			||||||
 | 
					        listen 443;
 | 
				
			||||||
 | 
					        listen [::]:443;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        server_name $domain;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        access_log  /var/log/nginx/access.log;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        ssl on;
 | 
				
			||||||
 | 
					        ssl_certificate      /etc/letsencrypt/live/$domain/fullchain.pem;
 | 
				
			||||||
 | 
					        ssl_certificate_key  /etc/letsencrypt/live/$domain/privkey.pem;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        client_max_body_size 256m;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        location / {
 | 
				
			||||||
 | 
					            try_files \$uri @uwsgi;
 | 
				
			||||||
 | 
					        }
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        location @uwsgi {
 | 
				
			||||||
 | 
					            include uwsgi_params;
 | 
				
			||||||
 | 
					            uwsgi_pass _bottle;
 | 
				
			||||||
 | 
					        }
 | 
				
			||||||
 | 
					    }
 | 
				
			||||||
 | 
					    EOF
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Now our manifest is finished.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Complete __sample_bottle_hosting type manifest listing
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Here is complete __sample_bottle_hosting type manifest listing,
 | 
				
			||||||
 | 
					located in ~/.cdist/type/__sample_bottle_hosting/manifest::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    #!/bin/sh
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    os=$(cat "$__global/explorer/os")
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    case "$os" in
 | 
				
			||||||
 | 
					        devuan)
 | 
				
			||||||
 | 
					            :
 | 
				
			||||||
 | 
					        ;;
 | 
				
			||||||
 | 
					        *)
 | 
				
			||||||
 | 
					            echo "OS $os currently not supported" >&2
 | 
				
			||||||
 | 
					            exit 1
 | 
				
			||||||
 | 
					        ;;
 | 
				
			||||||
 | 
					    esac
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    projectname="$(cat "$__object/parameter/projectname")"
 | 
				
			||||||
 | 
					    user="$(cat "$__object/parameter/user")"
 | 
				
			||||||
 | 
					    home="/home/$user"
 | 
				
			||||||
 | 
					    apphome="$home/app"
 | 
				
			||||||
 | 
					    domain="$(cat "$__object/parameter/domain")"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # create user
 | 
				
			||||||
 | 
					    __user "$user" --home "$home" --shell /bin/bash
 | 
				
			||||||
 | 
					    # create user home dir
 | 
				
			||||||
 | 
					    require="__user/$user" __directory "$home" \
 | 
				
			||||||
 | 
					        --owner "$user" --group "$user" --mode 0755
 | 
				
			||||||
 | 
					    # create app home dir
 | 
				
			||||||
 | 
					    require="__user/$user __directory/$home" __directory "$apphome" \
 | 
				
			||||||
 | 
					        --state present --owner "$user" --group "$user" --mode 0755
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # define packages that need to be installed
 | 
				
			||||||
 | 
					    packages_to_install="nginx uwsgi-plugin-python3 python3-dev python3-pip postgresql postgresql-contrib libpq-dev python3-venv uwsgi python3-psycopg2"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # update package index
 | 
				
			||||||
 | 
					    __apt_update_index
 | 
				
			||||||
 | 
					    # install packages
 | 
				
			||||||
 | 
					    for package in $packages_to_install
 | 
				
			||||||
 | 
					        do require="__apt_update_index" __package $package --state=present
 | 
				
			||||||
 | 
					    done
 | 
				
			||||||
 | 
					    # install pip3 packages
 | 
				
			||||||
 | 
					    for package in bottle bottle-pgsql; do
 | 
				
			||||||
 | 
					        __package_pip --pip pip3 $package
 | 
				
			||||||
 | 
					    done
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    #PostgreSQL db & user
 | 
				
			||||||
 | 
					    postgres_server=postgresql
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # create PostgreSQL db user
 | 
				
			||||||
 | 
					    require="__package/postgresql" __postgres_role $user --login --createdb
 | 
				
			||||||
 | 
					    # create PostgreSQL db
 | 
				
			||||||
 | 
					    require="__postgres_role/$user __package/postgresql" __postgres_database $user \
 | 
				
			||||||
 | 
					        --owner $user
 | 
				
			||||||
 | 
					    # configure uWSGI
 | 
				
			||||||
 | 
					    require="__package/uwsgi" __file /etc/uwsgi/apps-enabled/$user.ini \
 | 
				
			||||||
 | 
					                --owner root --group root --mode 0644 \
 | 
				
			||||||
 | 
					                --state present \
 | 
				
			||||||
 | 
					                --source - << EOF
 | 
				
			||||||
 | 
					    [uwsgi]
 | 
				
			||||||
 | 
					    socket = $apphome/uwsgi.sock
 | 
				
			||||||
 | 
					    chdir = $apphome
 | 
				
			||||||
 | 
					    wsgi-file = $projectname/wsgi.py
 | 
				
			||||||
 | 
					    touch-reload = $projectname/wsgi.py
 | 
				
			||||||
 | 
					    processes = 4
 | 
				
			||||||
 | 
					    threads = 2
 | 
				
			||||||
 | 
					    chmod-socket = 666
 | 
				
			||||||
 | 
					    daemonize=true
 | 
				
			||||||
 | 
					    vacuum = true
 | 
				
			||||||
 | 
					    uid = $user
 | 
				
			||||||
 | 
					    gid = $user
 | 
				
			||||||
 | 
					    EOF
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # setup nginx HTTP for Let's Encrypt and SSL redirect
 | 
				
			||||||
 | 
					    domain="$(cat "$__object/parameter/domain")"
 | 
				
			||||||
 | 
					    webroot="/var/www/html"
 | 
				
			||||||
 | 
					    __sample_nginx_http_letsencrypt_and_ssl_redirect "$domain" --webroot "$webroot"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # create SSL cert
 | 
				
			||||||
 | 
					    require="__package/nginx __sample_nginx_http_letsencrypt_and_ssl_redirect/$domain" \
 | 
				
			||||||
 | 
					        __letsencrypt_cert --admin-email admin@test.ungleich.ch \
 | 
				
			||||||
 | 
					            --webroot "$webroot" \
 | 
				
			||||||
 | 
					            --automatic-renewal \
 | 
				
			||||||
 | 
					            --renew-hook "service nginx reload" \
 | 
				
			||||||
 | 
					            --domain "$domain" \
 | 
				
			||||||
 | 
					            "$domain"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    # configure nginx
 | 
				
			||||||
 | 
					    require="__package/nginx __letsencrypt_cert/$domain" \
 | 
				
			||||||
 | 
					        __file "/etc/nginx/sites-enabled/https-$domain" \
 | 
				
			||||||
 | 
					        --source - --mode 0644 << EOF
 | 
				
			||||||
 | 
					    upstream _bottle {
 | 
				
			||||||
 | 
					        server unix:$apphome/uwsgi.sock;
 | 
				
			||||||
 | 
					    }
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    server {
 | 
				
			||||||
 | 
					        listen 443;
 | 
				
			||||||
 | 
					        listen [::]:443;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        server_name $domain;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        access_log  /var/log/nginx/access.log;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        ssl on;
 | 
				
			||||||
 | 
					        ssl_certificate      /etc/letsencrypt/live/$domain/fullchain.pem;
 | 
				
			||||||
 | 
					        ssl_certificate_key  /etc/letsencrypt/live/$domain/privkey.pem;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        client_max_body_size 256m;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        location / {
 | 
				
			||||||
 | 
					            try_files \$uri @uwsgi;
 | 
				
			||||||
 | 
					        }
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        location @uwsgi {
 | 
				
			||||||
 | 
					            include uwsgi_params;
 | 
				
			||||||
 | 
					            uwsgi_pass _bottle;
 | 
				
			||||||
 | 
					        }
 | 
				
			||||||
 | 
					    }
 | 
				
			||||||
 | 
					    EOF
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating __sample_bottle_hosting type gencode-remote
 | 
				
			||||||
 | 
					----------------------------------------------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Now define **gencode-remote** script: ~/.cdist/type/__sample_bottle_hosting/gencode-remote.
 | 
				
			||||||
 | 
					After manifest is applied it should restart uWSGI and nginx services so that our
 | 
				
			||||||
 | 
					configuration is active. Our gencode-remote looks like the following::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    echo "service uwsgi restart"
 | 
				
			||||||
 | 
					    echo "service nginx restart"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Our **__sample_bottle_hosting** type is now finished.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating __sample_nginx_http_letsencrypt_and_ssl_redirect type
 | 
				
			||||||
 | 
					--------------------------------------------------------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Let's now create **__sample_nginx_http_letsencrypt_and_ssl_redirect** type::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    cd ~/.cdist/type
 | 
				
			||||||
 | 
					    mkdir __sample_nginx_http_letsencrypt_and_ssl_redirect
 | 
				
			||||||
 | 
					    cd __sample_nginx_http_letsencrypt_and_ssl_redirect
 | 
				
			||||||
 | 
					    mkdir parameter
 | 
				
			||||||
 | 
					    echo webroot > parameter/required
 | 
				
			||||||
 | 
					    touch manifest
 | 
				
			||||||
 | 
					    touch gencode-remote
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Edit manifest::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    domain="$__object_id"
 | 
				
			||||||
 | 
					    webroot="$(cat "$__object/parameter/webroot")"
 | 
				
			||||||
 | 
					    # make sure we have nginx package
 | 
				
			||||||
 | 
					    __package nginx
 | 
				
			||||||
 | 
					    # setup Let's Encrypt HTTP acme challenge, redirect HTTP to HTTPS
 | 
				
			||||||
 | 
					    require="__package/nginx" __file "/etc/nginx/sites-enabled/http-$domain" \
 | 
				
			||||||
 | 
					        --source - --mode 0644 << EOF
 | 
				
			||||||
 | 
					    server {
 | 
				
			||||||
 | 
					        listen *:80;
 | 
				
			||||||
 | 
					        listen [::]:80;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        server_name $domain;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        # Let's Encrypt
 | 
				
			||||||
 | 
					        location /.well-known/acme-challenge/ {
 | 
				
			||||||
 | 
					            root $webroot;
 | 
				
			||||||
 | 
					        }
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        # Everything else -> SSL
 | 
				
			||||||
 | 
					        location / {
 | 
				
			||||||
 | 
					            return 301 https://\$host\$request_uri;
 | 
				
			||||||
 | 
					        }
 | 
				
			||||||
 | 
					    }
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    EOF
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Edit gencode-remote::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    echo "service nginx reload"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating init manifest
 | 
				
			||||||
 | 
					----------------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Next create init manifest::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    cd ~/.cdist/manifest
 | 
				
			||||||
 | 
					    printf "__sample_bottle_hosting --projectname sample --user app --domain \$__target_host sample\n" > sample
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Using this init manifest our target host will be configured using our **__sample_bottle_hosting**
 | 
				
			||||||
 | 
					type with projectname *sample*, user *app* and domain equal to **__target_host**.
 | 
				
			||||||
 | 
					Here the last positional argument *sample* is type's object id. For details on
 | 
				
			||||||
 | 
					**__target_host** and **__object_id** see
 | 
				
			||||||
 | 
					`Environment variables (for reading) <cdist-reference.html#environment-variables-for-reading>`_
 | 
				
			||||||
 | 
					reference.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Configuring host
 | 
				
			||||||
 | 
					----------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Finally configure test.ungleich.ch::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    cdist config -v -i ~/.cdist/manifest/sample test.ungleich.ch
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					After cdist configuration is successfully finished our host is ready.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating python bottle application
 | 
				
			||||||
 | 
					----------------------------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					We now need to create Bottle application. As you remember from the beginning
 | 
				
			||||||
 | 
					of this walkthrough our type does not create the actual python application,
 | 
				
			||||||
 | 
					its intention is only to configure hosing for specified user and project.
 | 
				
			||||||
 | 
					It is up to the user to create his/her applications.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Become app user::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    su -l app
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Preparing database
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					We need to prepare database for our application. Create table and
 | 
				
			||||||
 | 
					insert some items::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    psql -c "create table items (item varchar(255));"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    psql -c "insert into items(item) values('spam');"
 | 
				
			||||||
 | 
					    psql -c "insert into items(item) values('eggs');"
 | 
				
			||||||
 | 
					    psql -c "insert into items(item) values('sausage');"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Creating application
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Next create sample app::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    cd /home/app/app
 | 
				
			||||||
 | 
					    mkdir sample
 | 
				
			||||||
 | 
					    cd sample
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Create app.py with the following content::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    #!/usr/bin/env python3
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    import bottle
 | 
				
			||||||
 | 
					    import bottle_pgsql
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    app = application = bottle.Bottle()
 | 
				
			||||||
 | 
					    plugin = bottle_pgsql.Plugin('dbname=app user=app password=')
 | 
				
			||||||
 | 
					    app.install(plugin)
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    @app.route('/')
 | 
				
			||||||
 | 
					    def show_index(db):
 | 
				
			||||||
 | 
					        db.execute('select * from items')
 | 
				
			||||||
 | 
					        items = db.fetchall() or []
 | 
				
			||||||
 | 
					        rv = '<html><body><h3>Items:</h3><ul>'
 | 
				
			||||||
 | 
					        for item in items:
 | 
				
			||||||
 | 
					            rv += '<li>' + str(item['item']) + '</li>'
 | 
				
			||||||
 | 
					        rv += '</ul></body></html>'
 | 
				
			||||||
 | 
					        return rv
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    if __name__ == '__main__':
 | 
				
			||||||
 | 
					        bottle.run(app=app, host='0.0.0.0', port=8080)
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Create wsgi.py with the following content::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    import os
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    os.chdir(os.path.dirname(__file__))
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    import app
 | 
				
			||||||
 | 
					    application = app.app
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					We have configured uWSGI with **touch-reload = $projectname/wsgi.py** so after
 | 
				
			||||||
 | 
					we have changed our **wsgi.py** file uWSGI reloads the application.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Our application selects and lists items from **items** table.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Openning application
 | 
				
			||||||
 | 
					~~~~~~~~~~~~~~~~~~~~
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Finally try the application::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    http://test.ungleich.ch/
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					It should redirect to HTTPS and return:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					.. container:: highlight
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					    .. raw:: html
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        <h3>Items:</h3>
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					        <ul>
 | 
				
			||||||
 | 
					            <li>spam</li>
 | 
				
			||||||
 | 
					            <li>eggs</li>
 | 
				
			||||||
 | 
					            <li>sausage</li>
 | 
				
			||||||
 | 
					        </ul>
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					What's next?
 | 
				
			||||||
 | 
					------------
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Continue reading next sections ;)
 | 
				
			||||||
| 
						 | 
					@ -16,6 +16,7 @@ Contents:
 | 
				
			||||||
   cdist-support
 | 
					   cdist-support
 | 
				
			||||||
   cdist-features
 | 
					   cdist-features
 | 
				
			||||||
   cdist-quickstart
 | 
					   cdist-quickstart
 | 
				
			||||||
 | 
					   cdist-real-world
 | 
				
			||||||
   man1/cdist
 | 
					   man1/cdist
 | 
				
			||||||
   cdist-bootstrap
 | 
					   cdist-bootstrap
 | 
				
			||||||
   cdist-configuration
 | 
					   cdist-configuration
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue