Implement preos and triggering
This commit is contained in:
Darko Poljak
parent
afa00a9094
commit
11974e5ed6
101 changed files with 10570 additions and 4 deletions
176
docs/src/cdist-preos.rst
Normal file
176
docs/src/cdist-preos.rst
Normal file
|
|
@ -0,0 +1,176 @@
|
|||
PreOS
|
||||
=====
|
||||
|
||||
Description
|
||||
-----------
|
||||
With cdist you can install and configure new machines. You can use cdist to
|
||||
create PreOS, minimal OS which purpose is to boot new machine.
|
||||
After PreOS is booted machine is ready for installing desired OS and
|
||||
then it is ready for configuration.
|
||||
|
||||
PreOS creation
|
||||
--------------
|
||||
With cdist you can create PreOS.
|
||||
Currently supported PreOS-es include:
|
||||
|
||||
* debian
|
||||
* ubuntu
|
||||
* devuan.
|
||||
|
||||
PreOS is created using cdist preos command. preos command has subcommands that
|
||||
create the desired PreOS.
|
||||
|
||||
For example, to create ubuntu PreOS:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ cdist preos ubuntu /preos/preos-ubuntu -b -C \
|
||||
-k ~/.ssh/id_rsa.pub -p /preos/pxe-ubuntu \
|
||||
-t "/usr/bin/curl 192.168.111.5:3000/install/"
|
||||
|
||||
For more info about available options see cdist manual page.
|
||||
|
||||
This will bootstrap (``-b``) ubuntu PreOS in ``/preos/preos-ubuntu`` directory, it
|
||||
will be configured (``-C``) using default built-in initial manifest and with
|
||||
specified ssh authorized key (``-k``) and with specified trigger command (``-t``).
|
||||
After bootstrapping and configuration PXE
|
||||
boot directory will be created (``-p``) in ``/preos/pxe-ubuntu``.
|
||||
|
||||
After PreOS is created new machines can be booted using created PXE (after
|
||||
proper dhcp, tftp setting).
|
||||
|
||||
Since PreOS is configured with ssh authorized key it can be accessed throguh
|
||||
ssh, i.e. it can be further installed and configured with cdist.
|
||||
|
||||
When installing and configuring new machines using cdist's PreOS concept
|
||||
cdist can use triggering for host installation/configuration, which is described
|
||||
in the previous chapter.
|
||||
|
||||
When new machine is booted with PreOS then trigger command is executed.
|
||||
Machine will connect to cdist trigger server. If the request is, for example,
|
||||
for installation then cdist trigger server will start install command for the
|
||||
client host using parameters specified at trigger server startup.
|
||||
|
||||
Implementing new PreOS sub-command
|
||||
----------------------------------
|
||||
preos command is implemented as a plugin system. This plugin system scans for
|
||||
preos subcommands in ``cdist/preos/`` distribution directory and also in
|
||||
``~/.cdist/preos/`` directory if it exists.
|
||||
|
||||
preos subcommand is a module or a class that satisfies the following:
|
||||
|
||||
* it has attribute ``_cdist_preos`` set to ``True``
|
||||
* it has function/method ``commandline``.
|
||||
|
||||
For a module based preos subcommand ``commandline`` function accepts a module
|
||||
object as its first argument and the list of command line
|
||||
arguments (``sys.argv[2:]``).
|
||||
|
||||
For a class preos subcommand ``commandline`` method should be staticmethod and
|
||||
it accepts a class object as its first argument and the list of command line
|
||||
arguments(``sys.argv[2:]``).
|
||||
|
||||
If preos scanning finds a module/class that has ``_cdist_preos`` set
|
||||
to ``True`` and it has function/method ``commandline`` then this module/class is
|
||||
registered to preos subcommands. The name of the command is set to ``_preos_name``
|
||||
attribute if it exists, otherwise it is set to the module/class name, lowercase.
|
||||
When registered preos subcommand is specified as preos command then ``commandline``
|
||||
will be called with first argument set to module/class object and second argument
|
||||
set to ``sys.argv[2:]``.
|
||||
|
||||
Example writing new dummy preos sub-command
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Module based preos:
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
#. Create directory ``~/.cdist/preos/`` if it does not exist
|
||||
#. Create ``~/.cdist/preos/netbsd.py`` with the following contents:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
_preos_name = 'netbsd'
|
||||
_cdist_preos = True
|
||||
|
||||
def commandline(cls, args):
|
||||
print("NetBSD PreOS: {}".format(args))
|
||||
|
||||
When you try to run this new preos you will get:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ cdist preos -h
|
||||
usage: cdist preos [-h] preos
|
||||
|
||||
Create PreOS
|
||||
|
||||
positional arguments:
|
||||
preos PreOS to create, one of: {'netbsd', 'debian', 'ubuntu'}
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
$ cdist preos netbsd
|
||||
NetBSD PreOS: []
|
||||
|
||||
Class based preos:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
#. Create directory ``~/.cdist/preos/`` if it does not exist
|
||||
#. Create ``~/.cdist/preos/freebsd.py`` with the following contents:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
class FreeBSD(object):
|
||||
_cdist_preos = True
|
||||
|
||||
@classmethod
|
||||
def commandline(cls, args):
|
||||
print("FreeBSD dummy preos: {}".format(args))
|
||||
|
||||
When you try to run this new preos you will get:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ cdist preos -h
|
||||
usage: cdist preos [-h] preos
|
||||
|
||||
Create PreOS
|
||||
|
||||
positional arguments:
|
||||
preos PreOS to create, one of: {'freebsd', 'debian', 'ubuntu'}
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
$ cdist preos freebsd
|
||||
FreeBSD dummy preos: []
|
||||
|
||||
In the ``commandline`` function/method you have all the freedom to actually create
|
||||
PreOS.
|
||||
|
||||
Simple tipical use case for using PreOS and trigger
|
||||
---------------------------------------------------
|
||||
Tipical use case for using PreOS and trigger command include the following steps.
|
||||
|
||||
#. Create PreOS PXE with ssh key and trigger command for installation.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ cdist preos ubuntu /preos/ubuntu -b -C \
|
||||
-k ~/.ssh/id_rsa.pub -p /preos/pxe \
|
||||
-t "/usr/bin/curl 192.168.111.5:3000/install/"
|
||||
|
||||
#. Configure dhcp server and tftp server.
|
||||
|
||||
#. On cdist host (192.168.111.5 from above) start trigger command (it will use
|
||||
default init manifest for installation).
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ cdist trigger -b -v
|
||||
|
||||
#. After all is set up start new machines (PXE boot).
|
||||
|
||||
#. New machine boots and executes trigger command, i.e. triggers installation.
|
||||
|
||||
#. Cdist trigger server starts installing host that has triggered it.
|
||||
|
||||
#. After cdist install is finished new host is installed.
|
||||
|
|
@ -67,6 +67,9 @@ cdist/inventory/
|
|||
The distribution inventory directory.
|
||||
This path is relative to cdist installation directory.
|
||||
|
||||
cdist/preos/
|
||||
The distribution PreOS plugins directory.
|
||||
|
||||
confdir
|
||||
Cdist will use all available configuration directories and create
|
||||
a temporary confdir containing links to the real configuration directories.
|
||||
|
|
|
|||
33
docs/src/cdist-trigger.rst
Normal file
33
docs/src/cdist-trigger.rst
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
Trigger
|
||||
=======
|
||||
|
||||
Description
|
||||
-----------
|
||||
cdist supports triggering for host installation/configuration using trigger command.
|
||||
This command starts trigger server at management node, for example:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ cdist trigger -b -v
|
||||
|
||||
This will start cdist trigger server in verbose mode. cdist trigger server accepts
|
||||
simple requests for configuration and for installation:
|
||||
|
||||
* :strong:`/cdist/install/.*` for installation
|
||||
* :strong:`/cdist/config/.*` for configuration.
|
||||
|
||||
Machines can then trigger cdist trigger server with appropriate requests.
|
||||
If the request is, for example, for installation (:strong:`/cdist/install/`)
|
||||
then cdist trigger server will start install command for the client host using
|
||||
parameters specified at trigger server startup. For the above example that means
|
||||
that client will be installed using default initial manifest.
|
||||
|
||||
When triggered cdist will try to reverse DNS lookup for host name and if
|
||||
host name is dervied then it is used for running cdist config. If no
|
||||
host name is resolved then IP address is used.
|
||||
|
||||
This command returns the following response codes to client requests:
|
||||
|
||||
* 200 for success
|
||||
* 599 for cdist run errors
|
||||
* 500 for cdist/server errors.
|
||||
|
|
@ -31,6 +31,8 @@ is being used in small up to enterprise grade environments.
|
|||
cdist-messaging
|
||||
cdist-parallelization
|
||||
cdist-inventory
|
||||
cdist-trigger
|
||||
cdist-preos
|
||||
cdist-integration
|
||||
cdist-reference
|
||||
cdist-best-practice
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ SYNOPSIS
|
|||
|
||||
::
|
||||
|
||||
cdist [-h] [-V] {banner,config,install,inventory,shell} ...
|
||||
cdist [-h] [-V] {banner,config,install,inventory,preos,shell,trigger} ...
|
||||
|
||||
cdist banner [-h] [-l LOGLEVEL] [-q] [-v]
|
||||
|
||||
|
|
@ -59,8 +59,41 @@ SYNOPSIS
|
|||
[-I INVENTORY_DIR] [-a] [-f HOSTFILE] [-H] [-t]
|
||||
[host [host ...]]
|
||||
|
||||
cdist preos [-h] preos
|
||||
|
||||
cdist preos debian [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-a ARCH] [-B]
|
||||
[-C] [-c CDIST_PARAMS] [-D DRIVE] [-e REMOTE_EXEC]
|
||||
[-i MANIFEST] [-k KEYFILE ] [-m MIRROR]
|
||||
[-P ROOT_PASSWORD] [-p PXE_BOOT_DIR] [-r]
|
||||
[-S SCRIPT] [-s SUITE] [-t TRIGGER_COMMAND]
|
||||
[-y REMOTE_COPY]
|
||||
target_dir
|
||||
|
||||
cdist preos devuan [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-a ARCH] [-B]
|
||||
[-C] [-c CDIST_PARAMS] [-D DRIVE] [-e REMOTE_EXEC]
|
||||
[-i MANIFEST] [-k KEYFILE ] [-m MIRROR]
|
||||
[-P ROOT_PASSWORD] [-p PXE_BOOT_DIR] [-r]
|
||||
[-S SCRIPT] [-s SUITE] [-t TRIGGER_COMMAND]
|
||||
[-y REMOTE_COPY]
|
||||
target_dir
|
||||
|
||||
cdist preos ubuntu [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-a ARCH] [-B]
|
||||
[-C] [-c CDIST_PARAMS] [-D DRIVE] [-e REMOTE_EXEC]
|
||||
[-i MANIFEST] [-k KEYFILE ] [-m MIRROR]
|
||||
[-P ROOT_PASSWORD] [-p PXE_BOOT_DIR] [-r]
|
||||
[-S SCRIPT] [-s SUITE] [-t TRIGGER_COMMAND]
|
||||
[-y REMOTE_COPY]
|
||||
target_dir
|
||||
|
||||
cdist shell [-h] [-l LOGLEVEL] [-q] [-v] [-s SHELL]
|
||||
|
||||
cdist trigger [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-C CACHE_PATH_PATTERN]
|
||||
[-c CONF_DIR] [-i MANIFEST] [-j [JOBS]] [-n]
|
||||
[-o OUT_PATH] [-R [{tar,tgz,tbz2,txz}]]
|
||||
[-r REMOTE_OUT_PATH] [--remote-copy REMOTE_COPY]
|
||||
[--remote-exec REMOTE_EXEC] [-6] [-D DIRECTORY]
|
||||
[-H HTTP_PORT] [-S SOURCE]
|
||||
|
||||
|
||||
DESCRIPTION
|
||||
-----------
|
||||
|
|
@ -432,6 +465,143 @@ List inventory database.
|
|||
all hosts that contain any of specified tags.
|
||||
|
||||
|
||||
PREOS
|
||||
-----
|
||||
Create PreOS. Currently, the following PreOS-es are supported:
|
||||
|
||||
* debian
|
||||
* ubuntu
|
||||
* devuan
|
||||
|
||||
|
||||
PREOS DEBIAN/DEVUAN
|
||||
-------------------
|
||||
|
||||
**target_dir**
|
||||
target directory where PreOS will be bootstrapped
|
||||
|
||||
**-a ARCH, --arch ARCH**
|
||||
target debootstrap architecture, by default 'amd64'
|
||||
|
||||
**-B, --bootstrap**
|
||||
do bootstrap step
|
||||
|
||||
**-b, --beta**
|
||||
Enable beta functionality.
|
||||
|
||||
**-C, --configure**
|
||||
do configure step
|
||||
|
||||
**-c CDIST_PARAMS, --cdist-params CDIST_PARAMS**
|
||||
parameters that will be passed to cdist config, by
|
||||
default '-v' is used
|
||||
|
||||
**-D DRIVE, --drive-boot DRIVE**
|
||||
create bootable PreOS on specified drive
|
||||
|
||||
**-e REMOTE_EXEC, --remote-exec REMOTE_EXEC**
|
||||
remote exec that cdist config will use, by default
|
||||
internal script is used
|
||||
|
||||
**-i MANIFEST, --init-manifest MANIFEST**
|
||||
init manifest that cdist config will use, by default
|
||||
internal init manifest is used
|
||||
|
||||
**-k KEYFILE, --keyfile KEYFILE**
|
||||
ssh key files that will be added to cdist config;
|
||||
'``__ssh_authorized_keys root ...``' type is appended to initial manifest
|
||||
|
||||
**-m MIRROR, --mirror MIRROR**
|
||||
use specified mirror for debootstrap
|
||||
|
||||
**-P ROOT_PASSWORD, --root-password ROOT_PASSWORD**
|
||||
Set specified password for root, generated by default
|
||||
|
||||
**-p PXE_BOOT_DIR, --pxe-boot-dir PXE_BOOT_DIR**
|
||||
PXE boot directory
|
||||
|
||||
**-r, --rm-bootstrap-dir**
|
||||
remove target directory after finishing
|
||||
|
||||
**-S SCRIPT, --script SCRIPT**
|
||||
use specified script for debootstrap
|
||||
|
||||
**-s SUITE, --suite SUITE**
|
||||
suite used for debootstrap, by default 'stable'
|
||||
|
||||
**-t TRIGGER_COMMAND, --trigger-command TRIGGER_COMMAND**
|
||||
trigger command that will be added to cdist config;
|
||||
'``__cdist_preos_trigger http ...``' type is appended to initial manifest
|
||||
|
||||
**-y REMOTE_COPY, --remote-copy REMOTE_COPY**
|
||||
remote copy that cdist config will use, by default
|
||||
internal script is used
|
||||
|
||||
|
||||
PREOS UBUNTU
|
||||
------------
|
||||
|
||||
**target_dir**
|
||||
target directory where PreOS will be bootstrapped
|
||||
|
||||
**-a ARCH, --arch ARCH**
|
||||
target debootstrap architecture, by default 'amd64'
|
||||
|
||||
**-B, --bootstrap**
|
||||
do bootstrap step
|
||||
|
||||
**-b, --beta**
|
||||
Enable beta functionality.
|
||||
|
||||
**-C, --configure**
|
||||
do configure step
|
||||
|
||||
**-c CDIST_PARAMS, --cdist-params CDIST_PARAMS**
|
||||
parameters that will be passed to cdist config, by
|
||||
default '-v' is used
|
||||
|
||||
**-D DRIVE, --drive-boot DRIVE**
|
||||
create bootable PreOS on specified drive
|
||||
|
||||
**-e REMOTE_EXEC, --remote-exec REMOTE_EXEC**
|
||||
remote exec that cdist config will use, by default
|
||||
internal script is used
|
||||
|
||||
**-i MANIFEST, --init-manifest MANIFEST**
|
||||
init manifest that cdist config will use, by default
|
||||
internal init manifest is used
|
||||
|
||||
**-k KEYFILE, --keyfile KEYFILE**
|
||||
ssh key files that will be added to cdist config;
|
||||
'``__ssh_authorized_keys root ...``' type is appended to initial manifest
|
||||
|
||||
**-m MIRROR, --mirror MIRROR**
|
||||
use specified mirror for debootstrap
|
||||
|
||||
**-P ROOT_PASSWORD, --root-password ROOT_PASSWORD**
|
||||
Set specified password for root, generated by default
|
||||
|
||||
**-p PXE_BOOT_DIR, --pxe-boot-dir PXE_BOOT_DIR**
|
||||
PXE boot directory
|
||||
|
||||
**-r, --rm-bootstrap-dir**
|
||||
remove target directory after finishing
|
||||
|
||||
**-S SCRIPT, --script SCRIPT**
|
||||
use specified script for debootstrap
|
||||
|
||||
**-s SUITE, --suite SUITE**
|
||||
suite used for debootstrap, by default 'xenial'
|
||||
|
||||
**-t TRIGGER_COMMAND, --trigger-command TRIGGER_COMMAND**
|
||||
trigger command that will be added to cdist config;
|
||||
'``__cdist_preos_trigger http ...``' type is appended to initial manifest
|
||||
|
||||
**-y REMOTE_COPY, --remote-copy REMOTE_COPY**
|
||||
remote copy that cdist config will use, by default
|
||||
internal script is used
|
||||
|
||||
|
||||
SHELL
|
||||
-----
|
||||
This command allows you to spawn a shell that enables access
|
||||
|
|
@ -444,6 +614,84 @@ usage. Its primary use is for debugging type parameters.
|
|||
be POSIX compatible shell.
|
||||
|
||||
|
||||
TRIGGER
|
||||
-------
|
||||
Start trigger (simple http server) that waits for connections. When host
|
||||
connects then it triggers config or install command and then cdist
|
||||
config/install is executed which configures/installs host.
|
||||
When triggered cdist will try to reverse DNS lookup for host name and if
|
||||
host name is dervied then it is used for running cdist config. If no
|
||||
host name is resolved then IP address is used.
|
||||
Request path recognizes following requests:
|
||||
|
||||
* :strong:`/cdist/config/.*` for config
|
||||
* :strong:`/cdist/install/.*` for install.
|
||||
|
||||
This command returns the following response codes to client requests:
|
||||
|
||||
* 200 for success
|
||||
* 599 for cdist run errors
|
||||
* 500 for cdist/server errors.
|
||||
|
||||
|
||||
**-6, --ipv6**
|
||||
|
||||
Listen to both IPv4 and IPv6 (instead of only IPv4)
|
||||
|
||||
**-b, --beta**
|
||||
|
||||
Enable beta functionality.
|
||||
|
||||
**-C CACHE_PATH_PATTERN, --cache-path-pattern CACHE_PATH_PATTERN**
|
||||
|
||||
Sepcify custom cache path pattern. It can also be set by
|
||||
CDIST_CACHE_PATH_PATTERN environment variable. If it is not set then
|
||||
default hostdir is used. For more info on format see
|
||||
:strong:`CACHE PATH PATTERN FORMAT` below.
|
||||
|
||||
**-c CONF_DIR, --conf-dir CONF_DIR**
|
||||
|
||||
Add configuration directory (can be repeated, last one wins)
|
||||
|
||||
**-D DIRECTORY, --directory DIRECTORY**
|
||||
Where to create local files
|
||||
|
||||
**-H HTTP_PORT, --http-port HTTP_PORT**
|
||||
|
||||
Create trigger listener via http on specified port
|
||||
|
||||
**-i MANIFEST, --initial-manifest MANIFEST**
|
||||
|
||||
path to a cdist manifest or '-' to read from stdin.
|
||||
|
||||
**-j [JOBS], --jobs [JOBS]**
|
||||
|
||||
Specify the maximum number of parallel jobs, currently
|
||||
only global explorers are supported
|
||||
|
||||
**-n, --dry-run**
|
||||
|
||||
do not execute code
|
||||
|
||||
**-o OUT_PATH, --out-dir OUT_PATH**
|
||||
|
||||
directory to save cdist output in
|
||||
|
||||
**-r REMOTE_OUT_PATH, --remote-out-dir REMOTE_OUT_PATH**
|
||||
|
||||
Directory to save cdist output in on the target host
|
||||
|
||||
**--remote-copy REMOTE_COPY**
|
||||
|
||||
Command to use for remote copy (should behave like scp)
|
||||
|
||||
**--remote-exec REMOTE_EXEC**
|
||||
|
||||
Command to use for remote execution (should behave like ssh)
|
||||
|
||||
**-S SOURCE, --source SOURCE**
|
||||
Which file to copy for creation
|
||||
|
||||
CONFIGURATION
|
||||
-------------
|
||||
cdist obtains configuration data from the following sources in the following
|
||||
|
|
@ -540,12 +788,16 @@ FILES
|
|||
~/.cdist/inventory
|
||||
The home inventory directory. If ~/.cdist exists it will be used as
|
||||
default inventory directory.
|
||||
~/.cdist/preos
|
||||
PreOS plugins directory, if existing.
|
||||
cdist/conf
|
||||
The distribution configuration directory. It contains official types and
|
||||
explorers. This path is relative to cdist installation directory.
|
||||
cdist/inventory
|
||||
The distribution inventory directory.
|
||||
This path is relative to cdist installation directory.
|
||||
cdist/preos
|
||||
The distribution PreOS plugins directory.
|
||||
/etc/cdist.cfg
|
||||
Global cdist configuration file, if exists.
|
||||
~/.cdist.cfg or $XDG_CONFIG_HOME/cdist/cdist.cfg
|
||||
|
|
@ -635,6 +887,28 @@ EXAMPLES
|
|||
# Configure all hosts from inventory db
|
||||
$ cdist config -b -A
|
||||
|
||||
# Create default debian PreOS in debug mode with config
|
||||
# trigger command
|
||||
$ cdist preos debian /preos/preos-debian -b -vvvv -C \
|
||||
-k ~/.ssh/id_rsa.pub -p /preos/pxe-debian \
|
||||
-t "/usr/bin/curl 192.168.111.5:3000/config/"
|
||||
|
||||
# Create ubuntu PreOS with install trigger command
|
||||
$ cdist preos ubuntu /preos/preos-ubuntu -b -C \
|
||||
-k ~/.ssh/id_rsa.pub -p /preos/pxe-ubuntu \
|
||||
-t "/usr/bin/curl 192.168.111.5:3000/install/"
|
||||
|
||||
# Create ubuntu PreOS on drive /dev/sdb with install trigger command
|
||||
# and set root password to 'password'.
|
||||
$ cdist preos ubuntu /mnt -b -B -C \
|
||||
-k ~/.ssh/id_rsa.pub -D /dev/sdb \
|
||||
-t "/usr/bin/curl 192.168.111.5:3000/install/" \
|
||||
-P password
|
||||
|
||||
# Start trigger in verbose mode that will configure host using specified
|
||||
# init manifest
|
||||
% cdist trigger -b -v -i ~/.cdist/manifest/init-for-triggered
|
||||
|
||||
|
||||
ENVIRONMENT
|
||||
-----------
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue