Release cdist 6.5.5
This commit is contained in:
parent
7dde837c39
commit
3c684ea5e4
445 changed files with 123344 additions and 0 deletions
|
@ -1,6 +1,20 @@
|
|||
Changelog
|
||||
---------
|
||||
|
||||
6.5.5: 2020-05-01
|
||||
~~~~~~~~~~~~~~~~~
|
||||
* Core: Fix XDG_CONFIG_HOME config file location (Joachim Desroches)
|
||||
* Type __postgres_database: Add encoding, lc-collate, lc-ctype, template parameters (Timothée Floure)
|
||||
* Type __motd: Improve documentation and support for FreeBSD (Evil Ham)
|
||||
* Type __openldap_server: Support extra config parameter (Evil Ham)
|
||||
* Type __postfix: Automagically support more OSs by not checking too much (Evil Ham)
|
||||
* New type: __pf_apply_anchor (Kamila Součková, Evil Ham)
|
||||
* Type __pf_ruleset: Refactor (Kamila Součková, Evil Ham)
|
||||
* Type __pf_apply: Deprecate type (Kamila Součková, Evil Ham)
|
||||
* Configuration: Add notes to cdist.cfg.skeleton (Evil Ham)
|
||||
* Explorers cpu_cores, memory: Improve *BSD support (Evil Ham)
|
||||
* Core: Remove debug logging noise (Evil Ham)
|
||||
|
||||
6.5.4: 2020-04-11
|
||||
~~~~~~~~~~~~~~~~~
|
||||
* Explorer init: Do not grep on non-existent init (Steven Armstrong)
|
||||
|
|
|
@ -8,6 +8,7 @@ cdist manual
|
|||
|
||||
**All versions**
|
||||
|
||||
* `6.5.5 <manual/6.5.5>`_
|
||||
* `6.5.4 <manual/6.5.4>`_
|
||||
* `6.5.3 <manual/6.5.3>`_
|
||||
* `6.5.2 <manual/6.5.2>`_
|
||||
|
|
4
src/extra/manual/6.5.5/.buildinfo
Normal file
4
src/extra/manual/6.5.5/.buildinfo
Normal file
|
@ -0,0 +1,4 @@
|
|||
# Sphinx build info version 1
|
||||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
|
||||
config: 56ebae29e61e17553beddd2d4c652e4c
|
||||
tags: 645f666f9bcd5a90fca523b33c5a78b7
|
278
src/extra/manual/6.5.5/_sources/cdist-best-practice.rst.txt
Normal file
278
src/extra/manual/6.5.5/_sources/cdist-best-practice.rst.txt
Normal file
|
@ -0,0 +1,278 @@
|
|||
Best practice
|
||||
=============
|
||||
Practices used in real environments
|
||||
|
||||
Passwordless connections
|
||||
------------------------
|
||||
It is recommended to run cdist with public key authentication.
|
||||
This requires a private/public key pair and the entry
|
||||
"PermitRootLogin without-password" in the sshd server.
|
||||
See sshd_config(5) and ssh-keygen(1).
|
||||
|
||||
|
||||
Speeding up ssh connections
|
||||
---------------------------
|
||||
When connecting to a new host, the initial delay with ssh connections
|
||||
is pretty big. As cdist makes many connections to each host successive
|
||||
connections can be sped up by "sharing of multiple sessions over a single
|
||||
network connection" (quote from ssh_config(5)). This is also called "connection
|
||||
multiplexing".
|
||||
|
||||
Cdist implements this since v4.0.0 by executing ssh with the appropriate
|
||||
options (`-o ControlMaster=auto -o ControlPath=/tmp/<tmpdir>/s -o
|
||||
ControlPersist=2h`).
|
||||
|
||||
Note that the sshd_config on the server can configure the maximum number of
|
||||
parallel multiplexed connections this with `MaxSessions N` (N defaults to 10
|
||||
for OpenSSH v7.4).
|
||||
|
||||
|
||||
Speeding up shell execution
|
||||
----------------------------
|
||||
On the source host, ensure that /bin/sh is *not* bash: bash is quite slow for
|
||||
script execution. Instead, you could use dash after installing it::
|
||||
|
||||
ln -sf /bin/dash /bin/sh
|
||||
|
||||
|
||||
Multi master or environment setups
|
||||
----------------------------------
|
||||
If you plan to distribute cdist among servers or use different
|
||||
environments, you can do so easily with the included version
|
||||
control git. For instance if you plan to use the typical three
|
||||
environments production, integration and development, you can
|
||||
realise this with git branches::
|
||||
|
||||
# Go to cdist checkout
|
||||
cd /path/to/cdist
|
||||
|
||||
# Create branches
|
||||
git branch development
|
||||
git branch integration
|
||||
git branch production
|
||||
|
||||
# Make use of a branch, for instance production
|
||||
git checkout production
|
||||
|
||||
Similar if you want to have cdist checked out at multiple machines,
|
||||
you can clone it multiple times::
|
||||
|
||||
machine-a % git clone git://your-git-server/cdist
|
||||
machine-b % git clone git://your-git-server/cdist
|
||||
|
||||
|
||||
Separating work by groups
|
||||
-------------------------
|
||||
If you are working with different groups on one cdist-configuration,
|
||||
you can delegate to other manifests and have the groups edit only
|
||||
their manifests. You can use the following snippet in
|
||||
**conf/manifests/init**::
|
||||
|
||||
# Include other groups
|
||||
sh -e "$__manifest/systems"
|
||||
|
||||
sh -e "$__manifest/cbrg"
|
||||
|
||||
|
||||
Maintaining multiple configurations
|
||||
-----------------------------------
|
||||
When you need to manage multiple sites with cdist, like company_a, company_b
|
||||
and private for instance, you can easily use git for this purpose.
|
||||
Including a possible common base that is reused across the different sites::
|
||||
|
||||
# create branches
|
||||
git branch company_a company_b common private
|
||||
|
||||
# make stuff for company a
|
||||
git checkout company_a
|
||||
# work, commit, etc.
|
||||
|
||||
# make stuff for company b
|
||||
git checkout company_b
|
||||
# work, commit, etc.
|
||||
|
||||
# make stuff relevant for all sites
|
||||
git checkout common
|
||||
# work, commit, etc.
|
||||
|
||||
# change to private and include latest common stuff
|
||||
git checkout private
|
||||
git merge common
|
||||
|
||||
|
||||
The following **.git/config** is taken from a real world scenario::
|
||||
|
||||
# Track upstream, merge from time to time
|
||||
[remote "upstream"]
|
||||
url = git://git.schottelius.org/cdist
|
||||
fetch = +refs/heads/*:refs/remotes/upstream/*
|
||||
|
||||
# Same as upstream, but works when being offline
|
||||
[remote "local"]
|
||||
fetch = +refs/heads/*:refs/remotes/local/*
|
||||
url = /home/users/nico/p/cdist
|
||||
|
||||
# Remote containing various ETH internal branches
|
||||
[remote "eth"]
|
||||
url = sans.ethz.ch:/home/services/sans/git/cdist-eth
|
||||
fetch = +refs/heads/*:refs/remotes/eth/*
|
||||
|
||||
# Public remote that contains my private changes to cdist upstream
|
||||
[remote "nico"]
|
||||
url = git.schottelius.org:/home/services/git/cdist-nico
|
||||
fetch = +refs/heads/*:refs/remotes/nico/*
|
||||
|
||||
# The "nico" branch will be synced with the remote nico, branch master
|
||||
[branch "nico"]
|
||||
remote = nico
|
||||
merge = refs/heads/master
|
||||
|
||||
# ETH stable contains rock solid configurations used in various places
|
||||
[branch "eth-stable"]
|
||||
remote = eth
|
||||
merge = refs/heads/stable
|
||||
|
||||
Have a look at git-remote(1) to adjust the remote configuration, which allows
|
||||
|
||||
|
||||
Multiple developers with different trust
|
||||
----------------------------------------
|
||||
If you are working in an environment that requires different people to
|
||||
work on the same configuration, but having different privileges, you can
|
||||
implement this scenario with a gateway host and sudo:
|
||||
|
||||
- Create a dedicated user (for instance **cdist**)
|
||||
- Setup the ssh-pubkey for this user that has the right to configure all hosts
|
||||
- Create a wrapper to update the cdist configuration in ~cdist/cdist
|
||||
- Allow every developer to execute this script via sudo as the user cdist
|
||||
- Allow run of cdist as user cdist on specific hosts on a per user/group basis.
|
||||
|
||||
- f.i. nico ALL=(ALL) NOPASSWD: /home/cdist/bin/cdist config hostabc
|
||||
|
||||
For more details consult sudoers(5)
|
||||
|
||||
|
||||
Templating
|
||||
----------
|
||||
* create directory files/ in your type (convention)
|
||||
* create the template as an executable file like files/basic.conf.sh, it will output text using shell variables for the values
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
#!/bin/sh
|
||||
# in the template, use cat << eof (here document) to output the text
|
||||
# and use standard shell variables in the template
|
||||
# output everything in the template script to stdout
|
||||
cat << EOF
|
||||
server {
|
||||
listen 80;
|
||||
server_name $SERVERNAME;
|
||||
root $ROOT;
|
||||
|
||||
access_log /var/log/nginx/$SERVERNAME_access.log
|
||||
error_log /var/log/nginx/$SERVERNAME_error.log
|
||||
}
|
||||
EOF
|
||||
|
||||
* in the manifest, export the relevant variables and add the following lines to your manifest:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# export variables needed for the template
|
||||
export SERVERNAME='test"
|
||||
export ROOT='/var/www/test'
|
||||
# render the template
|
||||
mkdir -p "$__object/files"
|
||||
"$__type/files/basic.conf.sh" > "$__object/files/basic.conf"
|
||||
# send the rendered template
|
||||
__file /etc/nginx/sites-available/test.conf \
|
||||
--state present
|
||||
--source "$__object/files/basic.conf"
|
||||
|
||||
|
||||
Testing a new type
|
||||
------------------
|
||||
If you want to test a new type on a node, you can tell cdist to only use an
|
||||
object of this type: Use the '--initial-manifest' parameter
|
||||
with - (stdin) as argument and feed object into stdin
|
||||
of cdist:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# Singleton type without parameter
|
||||
echo __ungleich_munin_server | cdist --initial-manifest - munin.panter.ch
|
||||
|
||||
# Singleton type with parameter
|
||||
echo __ungleich_munin_node --allow 1.2.3.4 | \
|
||||
cdist --initial-manifest - rails-19.panter.ch
|
||||
|
||||
# Normal type
|
||||
echo __file /tmp/stdintest --mode 0644 | \
|
||||
cdist --initial-manifest - cdist-dev-01.ungleich.ch
|
||||
|
||||
|
||||
Other content in cdist repository
|
||||
---------------------------------
|
||||
Usually the cdist repository contains all configuration
|
||||
items. Sometimes you may have additional resources that
|
||||
you would like to store in your central configuration
|
||||
repository (like password files from KeepassX,
|
||||
Libreoffice diagrams, etc.).
|
||||
|
||||
It is recommended to use a subfolder named "non-cdist"
|
||||
in the repository for such content: It allows you to
|
||||
easily distinguish what is used by cdist and what is not
|
||||
and also to store all important files in one
|
||||
repository.
|
||||
|
||||
|
||||
Notes on CDIST_ORDER_DEPENDENCY
|
||||
-------------------------------
|
||||
With CDIST_ORDER_DEPENDENCY all types are executed in the order in which they
|
||||
are created in the manifest. The current created object automatically depends
|
||||
on the previously created object.
|
||||
|
||||
It essentially helps you to build up blocks of code that build upon each other
|
||||
(like first creating the directory xyz than the file below the directory).
|
||||
|
||||
This can be helpful, but one must be aware of its side effects.
|
||||
|
||||
|
||||
CDIST_ORDER_DEPENDENCY kills parallelization
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Suppose you have defined CDIST_ORDER_DEPENDENCY and then, among other things,
|
||||
you specify creation of three, by nature independent, files.
|
||||
|
||||
**init**
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
CDIST_ORDER_DEPENDENCY=1
|
||||
export CDIST_ORDER_DEPENDENCY
|
||||
|
||||
...
|
||||
__file /tmp/file1
|
||||
__file /tmp/file2
|
||||
__file /tmp/file3
|
||||
...
|
||||
|
||||
Due to defined CDIST_ORDER_DEPENDENCY cdist will execute them in specified order.
|
||||
It is better to use CDIST_ORDER_DEPENDENCY in well defined blocks:
|
||||
|
||||
**init**
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
CDIST_ORDER_DEPENDENCY=1
|
||||
export CDIST_ORDER_DEPENDENCY
|
||||
...
|
||||
unset CDIST_ORDER_DEPENDENCY
|
||||
|
||||
__file /tmp/file1
|
||||
__file /tmp/file2
|
||||
__file /tmp/file3
|
||||
|
||||
CDIST_ORDER_DEPENDENCY=1
|
||||
export CDIST_ORDER_DEPENDENCY
|
||||
...
|
||||
unset CDIST_ORDER_DEPENDENCY
|
118
src/extra/manual/6.5.5/_sources/cdist-bootstrap.rst.txt
Normal file
118
src/extra/manual/6.5.5/_sources/cdist-bootstrap.rst.txt
Normal file
|
@ -0,0 +1,118 @@
|
|||
Bootstrap
|
||||
=========
|
||||
This document describes the usual steps recommended for a new
|
||||
cdist setup. It is recommended that you have read and understood
|
||||
`cdist quickstart <cdist-quickstart.html>`_ before digging into this.
|
||||
|
||||
|
||||
Location
|
||||
---------
|
||||
First of all, you should think about where to store your configuration
|
||||
database and who will be accessing or changing it. Secondly you have to
|
||||
think about where to configure your hosts from, which may be a different
|
||||
location.
|
||||
|
||||
For starters, having cdist (which includes the configuration database) on
|
||||
your notebook should be fine.
|
||||
Additionally an external copy of the git repository the configuration
|
||||
relies on is recommended, for use as backup as well as to allow easy collaboration
|
||||
with others.
|
||||
|
||||
For more sophisticated setups developing cdist configurations with multiple
|
||||
people, have a look at `cdist best practice <cdist-best-practice.html>`_.
|
||||
|
||||
|
||||
Setup working directory and branch
|
||||
----------------------------------
|
||||
I assume you have a fresh copy of the cdist tree in ~/cdist, cloned from
|
||||
one of the official urls (see `cdist quickstart <cdist-quickstart.html>`_ if you don't).
|
||||
Entering the command "git branch" should show you "* master", which indicates
|
||||
you are on the **master** branch.
|
||||
|
||||
The master branch reflects the latest development of cdist. As this is the
|
||||
development branch, it may or may not work. There are also version branches
|
||||
available, which are kept in a stable state. Let's use **git branch -r**
|
||||
to list all branches::
|
||||
|
||||
cdist% git branch -r
|
||||
origin/1.0
|
||||
origin/1.1
|
||||
origin/1.2
|
||||
origin/1.3
|
||||
origin/1.4
|
||||
origin/1.5
|
||||
origin/1.6
|
||||
origin/1.7
|
||||
origin/2.0
|
||||
origin/HEAD -> origin/master
|
||||
origin/archive_shell_function_approach
|
||||
origin/master
|
||||
|
||||
So **2.0** is the latest version branch in this example.
|
||||
All versions (2.0.x) within one version branch (2.0) are compatible to each
|
||||
other and won't break your configuration when updating.
|
||||
|
||||
It's up to you to decide which branch you want to base your own work on:
|
||||
master contains more recent changes, newer types, but may also break.
|
||||
The version branches are stable, but may lack the latest features.
|
||||
Your decision can be changed later on, but may result in merge conflicts,
|
||||
which you will need to solve.
|
||||
|
||||
Let's assume you want latest stuff and select the master branch as base for
|
||||
your own work. Now it's time to create your branch, which contains your
|
||||
local changes. I usually name it by the company/area I am working for:
|
||||
ethz-systems, localch, customerX, ... But this is pretty much up to you.
|
||||
|
||||
In this tutorial I use the branch **mycompany**::
|
||||
|
||||
cdist% git checkout -b mycompany origin/master
|
||||
Branch mycompany set up to track remote branch master from origin.
|
||||
Switched to a new branch 'mycompany'
|
||||
cdist-user% git branch
|
||||
master
|
||||
* mycompany
|
||||
|
||||
From now on, you can use git as usual to commit your changes in your own branch.
|
||||
|
||||
|
||||
Publishing the configuration
|
||||
----------------------------
|
||||
Usually a development machine like a notebook should be considered
|
||||
temporary only. For this reason and to enable shareability, the configuration
|
||||
should be published to another device as early as possible. The following
|
||||
example shows how to publish the configuration to another host that is
|
||||
reachable via ssh and has git installed::
|
||||
|
||||
# Create bare git repository on the host named "loch"
|
||||
cdist% ssh loch "GIT_DIR=/home/nutzer/cdist git init"
|
||||
Initialized empty Git repository in /home/nutzer/cdist/
|
||||
|
||||
# Add remote git repo to git config
|
||||
cdist% git remote add loch loch:/home/nutzer/cdist
|
||||
|
||||
# Configure the mycompany branch to push to loch
|
||||
cdist% git config branch.mycompany.remote loch
|
||||
|
||||
# Configure mycompany branch to push into remote master branch
|
||||
cdist% git config branch.mycompany.merge refs/heads/master
|
||||
|
||||
# Push mycompany branch to remote branch master initially
|
||||
cdist% git push loch mycompany:refs/heads/master
|
||||
|
||||
Now you have setup the git repository to synchronise the **mycompany**
|
||||
branch with the **master** branch on the host **loch**. Thus you can commit
|
||||
as usual in your branch and push out changes by entering **git push**.
|
||||
|
||||
|
||||
Updating from origin
|
||||
--------------------
|
||||
Whenever you want to update your cdist installation, you can use git to do so::
|
||||
|
||||
# Update git repository with latest changes from origin
|
||||
cdist% git fetch origin
|
||||
|
||||
# Update current branch with master branch from origin
|
||||
cdist% git merge origin/master
|
||||
|
||||
# Alternative: Update current branch with 2.0 branch from origin
|
||||
cdist% git merge origin/2.0
|
98
src/extra/manual/6.5.5/_sources/cdist-cache.rst.txt
Normal file
98
src/extra/manual/6.5.5/_sources/cdist-cache.rst.txt
Normal file
|
@ -0,0 +1,98 @@
|
|||
Local cache overview
|
||||
====================
|
||||
|
||||
Description
|
||||
-----------
|
||||
While executing, cdist stores data to local cache. Currently this feature is
|
||||
one way only. That means that cdist does not use stored data for future runs.
|
||||
Anyway, those data can be used for debugging cdist, debugging types and
|
||||
debugging after host configuration fails.
|
||||
|
||||
Local cache is saved under $HOME/.cdist/cache directory, one directory entry
|
||||
for each host. Subdirectory path is specified by
|
||||
:strong:`-C/--cache-path-pattern` option, :strong:`cache_path_pattern`
|
||||
configuration option or by using :strong:`CDIST_CACHE_PATH_PATTERN`
|
||||
environment variable.
|
||||
|
||||
For more info on cache path pattern see :strong:`CACHE PATH PATTERN FORMAT`
|
||||
section in cdist man page.
|
||||
|
||||
|
||||
Cache overview
|
||||
--------------
|
||||
As noted above each configured host has got its subdirectory in local cache.
|
||||
Entries in host's cache directory are as follows.
|
||||
|
||||
bin
|
||||
directory with cdist type emulators
|
||||
|
||||
conf
|
||||
dynamically determined cdist conf directory, union of all specified
|
||||
conf directories
|
||||
|
||||
explorer
|
||||
directory containing global explorer named files containing explorer output
|
||||
after running on target host
|
||||
|
||||
messages
|
||||
file containing messages
|
||||
|
||||
object
|
||||
directory containing subdirectory for each cdist object
|
||||
|
||||
object_marker
|
||||
object marker for this particular cdist run
|
||||
|
||||
stderr
|
||||
directory containing init manifest and remote stderr stream output
|
||||
|
||||
stdout
|
||||
directory containing init manifest and remote stdout stream output
|
||||
|
||||
target_host
|
||||
file containing target host of this cdist run, as specified when running
|
||||
cdist
|
||||
|
||||
typeorder
|
||||
file containing types in order of execution.
|
||||
|
||||
|
||||
Object cache overview
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
Each object under :strong:`object` directory has its own structurue.
|
||||
|
||||
code-local
|
||||
code generated from gencode-local, present only if something is
|
||||
generated
|
||||
|
||||
code-remote
|
||||
code generated from gencode-remote, present only if something is
|
||||
generated
|
||||
|
||||
explorer
|
||||
directory containing type explorer named files containing explorer output
|
||||
after running on target host
|
||||
|
||||
files
|
||||
directory with object files created during type execution
|
||||
|
||||
parameter
|
||||
directory containing type parameter named files containing parameter
|
||||
values
|
||||
|
||||
source
|
||||
this type's source (init manifest)
|
||||
|
||||
state
|
||||
this type execution state ('done' when finished)
|
||||
|
||||
stderr
|
||||
directory containing type's manifest, gencode-* and code-* stderr stream
|
||||
outputs
|
||||
|
||||
stdin
|
||||
this type stdin content
|
||||
|
||||
stdout
|
||||
directory containing type's manifest, gencode-* and code-* stdout stream
|
||||
outputs.
|
41
src/extra/manual/6.5.5/_sources/cdist-configuration.rst.txt
Normal file
41
src/extra/manual/6.5.5/_sources/cdist-configuration.rst.txt
Normal file
|
@ -0,0 +1,41 @@
|
|||
Configuration
|
||||
=============
|
||||
|
||||
Description
|
||||
-----------
|
||||
cdist obtains configuration data from the following sources in the following
|
||||
order:
|
||||
|
||||
#. command-line options
|
||||
#. configuration file specified at command-line using -g command line option
|
||||
#. configuration file specified in CDIST_CONFIG_FILE environment variable
|
||||
#. environment variables
|
||||
#. user's configuration file (first one found of ~/.cdist.cfg, $XDG_CONFIG_HOME/cdist/cdist.cfg, in specified order)
|
||||
#. in-distribution configuration file (cdist/conf/cdist.cfg)
|
||||
#. system-wide configuration file (/etc/cdist.cfg)
|
||||
|
||||
if one exists.
|
||||
|
||||
Configuration source with lower ordering number from above has a higher
|
||||
precedence. Configuration option value read from source with higher
|
||||
precedence will overwrite option value, if exists, read from source with
|
||||
lower precedence. That means that command-line option wins them all.
|
||||
|
||||
Users can decide on the local conifguration file location. It can be either
|
||||
~/.cdist.cfg or $XDG_CONFIG_HOME/cdist/cdist.cfg. Note that, if both exist,
|
||||
then ~/.cdist.cfg is used.
|
||||
|
||||
For a per-project configuration, particular environment variables or better,
|
||||
CDIST_CONFIG_FILE environment variable or -g CONFIG_FILE command line option,
|
||||
can be used.
|
||||
|
||||
Config file format
|
||||
------------------
|
||||
|
||||
cdist configuration file is in the INI file format. Currently it supports
|
||||
only [GLOBAL] section.
|
||||
|
||||
Here you can find configuration file skeleton:
|
||||
|
||||
.. literalinclude:: cdist.cfg.skeleton
|
||||
:language: ini
|
54
src/extra/manual/6.5.5/_sources/cdist-explorer.rst.txt
Normal file
54
src/extra/manual/6.5.5/_sources/cdist-explorer.rst.txt
Normal file
|
@ -0,0 +1,54 @@
|
|||
Explorer
|
||||
========
|
||||
|
||||
Description
|
||||
-----------
|
||||
Explorers are small shell scripts, which will be executed on the target
|
||||
host. The aim of each explorer is to give hints to types on how to act on the
|
||||
target system. An explorer outputs the result to stdout, which is usually
|
||||
a one liner, but may be empty or multi line especially in the case of
|
||||
type explorers.
|
||||
|
||||
There are general explorers, which are run in an early stage, and
|
||||
type explorers. Both work almost exactly the same way, with the difference
|
||||
that the values of the general explorers are stored in a general location and
|
||||
the type specific below the object.
|
||||
|
||||
Explorers can reuse other explorers on the target system by calling
|
||||
|
||||
::
|
||||
|
||||
$__explorer/<explorer_name> (general and type explorer)
|
||||
|
||||
or
|
||||
|
||||
::
|
||||
|
||||
$__type_explorer/<explorer name> (type explorer).
|
||||
|
||||
In case of significant errors, the explorer may exit non-zero and return an
|
||||
error message on stderr, which will cause cdist to abort.
|
||||
|
||||
You can also use stderr for debugging purposes while developing a new
|
||||
explorer.
|
||||
|
||||
Examples
|
||||
--------
|
||||
A very simple explorer may look like this::
|
||||
|
||||
hostname
|
||||
|
||||
Which is in practise the **hostname** explorer.
|
||||
|
||||
A type explorer, which could check for the status of a package may look like this:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
if [ -f "$__object/parameter/name" ]; then
|
||||
name="$(cat "$__object/parameter/name")"
|
||||
else
|
||||
name="$__object_id"
|
||||
fi
|
||||
|
||||
# Expect dpkg failing, if package is not known / installed
|
||||
dpkg -s "$name" 2>/dev/null || exit 0
|
48
src/extra/manual/6.5.5/_sources/cdist-features.rst.txt
Normal file
48
src/extra/manual/6.5.5/_sources/cdist-features.rst.txt
Normal file
|
@ -0,0 +1,48 @@
|
|||
Features
|
||||
========
|
||||
|
||||
But cdist ticks differently, here is the feature set that makes it unique:
|
||||
|
||||
Simplicity
|
||||
There is only one type to extend cdist called **type**
|
||||
|
||||
Design
|
||||
+ Type and core cleanly separated
|
||||
+ Sticks completely to the KISS (keep it simple and stupid) paradigm
|
||||
+ Meaningful error messages - do not lose time debugging error messages
|
||||
+ Consistency in behaviour, naming and documentation
|
||||
+ No surprise factor: Only do what is obviously clear, no magic
|
||||
+ Define target state, do not focus on methods or scripts
|
||||
+ Push architecture: Instantly apply your changes
|
||||
|
||||
Small core
|
||||
cdist's core is very small - less code, less bugs
|
||||
|
||||
Fast development
|
||||
Focus on straightforwardness of type creation is a main development objective
|
||||
Batteries included: A lot of requirements can be solved using standard types
|
||||
|
||||
Modern Programming Language
|
||||
cdist is written in Python
|
||||
|
||||
Requirements, Scalability
|
||||
No central server needed, cdist operates in push mode and can be run from any computer
|
||||
|
||||
Requirements, Scalability, Upgrade
|
||||
cdist only needs to be updated on the master, not on the target hosts
|
||||
|
||||
Requirements, Security
|
||||
Uses well-know `SSH <http://www.openssh.com/>`_ as transport protocol
|
||||
|
||||
Requirements, Simplicity
|
||||
Requires only shell and SSH server on the target
|
||||
|
||||
UNIX
|
||||
Reuse of existing tools like cat, find, mv, ...
|
||||
|
||||
UNIX, familiar environment, documentation
|
||||
Is available as manpages and HTML
|
||||
|
||||
UNIX, simplicity, familiar environment
|
||||
cdist is configured in POSIX shell
|
||||
|
145
src/extra/manual/6.5.5/_sources/cdist-hacker.rst.txt
Normal file
145
src/extra/manual/6.5.5/_sources/cdist-hacker.rst.txt
Normal file
|
@ -0,0 +1,145 @@
|
|||
Hacking
|
||||
=======
|
||||
|
||||
Welcome
|
||||
-------
|
||||
Welcome dear hacker! I invite you to a tour of pointers to
|
||||
get into the usable configuration management system, cdist.
|
||||
|
||||
The first thing to know is probably that cdist is brought to
|
||||
you by people who care about how code looks like and who think
|
||||
twice before merging or implementing a feature: Less features
|
||||
with good usability are far better than the opposite.
|
||||
|
||||
|
||||
Reporting bugs
|
||||
--------------
|
||||
If you believe you've found a bug and verified that it is
|
||||
in the latest version, drop a mail to the cdist mailing list,
|
||||
subject prefixed with "[BUG] " or create an issue on code.ungleich.ch.
|
||||
|
||||
|
||||
Coding conventions (everywhere)
|
||||
-------------------------------
|
||||
If something should be improved or needs to be fixed, add the word FIXME
|
||||
nearby, so grepping for FIXME gives all positions that need to be fixed.
|
||||
|
||||
Indentation is 4 spaces (welcome to the python world).
|
||||
|
||||
|
||||
How to submit stuff for inclusion into upstream cdist
|
||||
-----------------------------------------------------
|
||||
If you did some cool changes to cdist, which you think might be of benefit to other
|
||||
cdist users, you're welcome to propose inclusion into upstream.
|
||||
|
||||
There are some requirements to ensure your changes don't break other peoples
|
||||
work nor kill the authors brain:
|
||||
|
||||
- All files should contain the usual header (Author, Copying, etc.)
|
||||
- Code submission must be done via git
|
||||
- Do not add cdist/conf/manifest/init - This file should only be touched in your
|
||||
private branch!
|
||||
- Code to be included should be branched of the upstream "master" branch
|
||||
|
||||
- Exception: Bugfixes to a version branch
|
||||
|
||||
- On a merge request, always name the branch I should pull from
|
||||
- Always ensure **all** manpages build. Use **./build man** to test.
|
||||
- If you developed more than **one** feature, consider submitting them in
|
||||
separate branches. This way one feature can already be included, even if
|
||||
the other needs to be improved.
|
||||
|
||||
As soon as your work meets these requirements, write a mail
|
||||
for inclusion to the mailinglist **cdist-configuration-management at googlegroups.com**
|
||||
or open a merge request at https://code.ungleich.ch/ungleich-public/cdist.
|
||||
|
||||
|
||||
How to submit a new type
|
||||
------------------------
|
||||
For detailed information about types, see `cdist type <cdist-type.html>`_.
|
||||
|
||||
Submitting a type works as described above, with the additional requirement
|
||||
that a corresponding manpage named man.rst in ReSTructured text format with
|
||||
the manpage-name "cdist-type__NAME" is included in the type directory
|
||||
AND the manpage builds (`make man`).
|
||||
|
||||
Warning: Submitting "exec" or "run" types that simply echo their parameter in
|
||||
**gencode** will not be accepted, because they are of no use. Every type can output
|
||||
code and thus such a type introduces redundant functionality that is given by
|
||||
core cdist already.
|
||||
|
||||
|
||||
Example git workflow
|
||||
---------------------
|
||||
The following workflow works fine for most developers
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# get latest upstream master branch
|
||||
git clone https://code.ungleich.ch/ungleich-public/cdist.git
|
||||
|
||||
# update if already existing
|
||||
cd cdist; git fetch -v; git merge origin/master
|
||||
|
||||
# create a new branch for your feature/bugfix
|
||||
cd cdist # if you haven't done before
|
||||
git checkout -b documentation_cleanup
|
||||
|
||||
# *hack*
|
||||
*hack*
|
||||
|
||||
# clone the cdist repository on code.ungleich.ch if you haven't done so
|
||||
|
||||
# configure your repo to know about your clone (only once)
|
||||
git remote add ungleich git@code.ungleich.ch:YOURUSERNAME/cdist.git
|
||||
|
||||
# push the new branch to ungleich gitlab
|
||||
git push ungleich documentation_cleanup
|
||||
|
||||
# (or everything)
|
||||
git push --mirror ungleich
|
||||
|
||||
# create a merge request at ungleich gitlab (use a browser)
|
||||
# *fixthingsbecausequalityassurancefoundissuesinourpatch*
|
||||
*hack*
|
||||
|
||||
# push code to ungleich gitlab again
|
||||
git push ... # like above
|
||||
|
||||
# add comment that everything should be green now (use a browser)
|
||||
|
||||
# go back to master branch
|
||||
git checkout master
|
||||
|
||||
# update master branch that includes your changes now
|
||||
git fetch -v origin
|
||||
git diff master..origin/master
|
||||
git merge origin/master
|
||||
|
||||
If at any point you want to go back to the original master branch, you can
|
||||
use **git stash** to stash your changes away::
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# assume you are on documentation_cleanup
|
||||
git stash
|
||||
|
||||
# change to master and update to most recent upstream version
|
||||
git checkout master
|
||||
git fetch -v origin
|
||||
git merge origin/master
|
||||
|
||||
Similarly when you want to develop another new feature, you go back
|
||||
to the master branch and create another branch based on it::
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# change to master and update to most recent upstream version
|
||||
git checkout master
|
||||
git fetch -v origin
|
||||
git merge origin/master
|
||||
|
||||
git checkout -b another_feature
|
||||
|
||||
(you can repeat the code above for as many features as you want to develop
|
||||
in parallel)
|
182
src/extra/manual/6.5.5/_sources/cdist-install.rst.txt
Normal file
182
src/extra/manual/6.5.5/_sources/cdist-install.rst.txt
Normal file
|
@ -0,0 +1,182 @@
|
|||
How to install cdist
|
||||
====================
|
||||
|
||||
Requirements
|
||||
-------------
|
||||
|
||||
Source Host
|
||||
~~~~~~~~~~~
|
||||
|
||||
This is the machine from which you will configure target hosts.
|
||||
|
||||
* /bin/sh: A posix like shell (for instance bash, dash, zsh)
|
||||
* Python >= 3.2
|
||||
* SSH client
|
||||
* sphinx (for building html docs and/or the man pages)
|
||||
|
||||
Target Hosts
|
||||
~~~~~~~~~~~~
|
||||
|
||||
* /bin/sh: A posix like shell (for instance bash, dash, zsh)
|
||||
* SSH server
|
||||
|
||||
Install cdist
|
||||
-------------
|
||||
|
||||
From git
|
||||
~~~~~~~~
|
||||
|
||||
Cloning cdist from git gives you the advantage of having
|
||||
a version control in place for development of your own stuff
|
||||
immediately.
|
||||
|
||||
To install cdist, execute the following commands:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
git clone https://code.ungleich.ch/ungleich-public/cdist.git
|
||||
cd cdist
|
||||
export PATH=$PATH:$(pwd -P)/bin
|
||||
|
||||
From version 4.2.0 cdist tags and releases are signed.
|
||||
You can get GPG public key used for signing `here <_static/pgp-key-EFD2AE4EC36B6901.asc>`_.
|
||||
It is assumed that you are familiar with *git* ways of signing and verification.
|
||||
|
||||
You can also get cdist from `github mirror <https://github.com/ungleich/cdist>`_.
|
||||
|
||||
To install cdist with distutils from cloned repository, first you have to
|
||||
create version.py:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
./bin/build-helper version
|
||||
|
||||
Then you install it with:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make install
|
||||
|
||||
or with:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make install-user
|
||||
|
||||
to install it into user *site-packages* directory.
|
||||
Or directly with distutils:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python setup.py install
|
||||
|
||||
Note that `bin/build-helper` script is intended for cdist maintainers.
|
||||
|
||||
|
||||
Available versions in git
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
* The active development takes place in the **master** branch
|
||||
* The released versions can be found in the tags
|
||||
|
||||
Other branches may be available for features or bugfixes, but they
|
||||
may vanish at any point. To select a specific branch use
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# Generic code
|
||||
git checkout -b <localbranchname> origin/<branchname>
|
||||
|
||||
So for instance if you want to use and stay with version 4.1, you can use
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
git checkout -b 4.1 origin/4.1
|
||||
|
||||
Building and using documentation (man and html)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If you want to build and use the documentation, run:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make docs
|
||||
|
||||
Documentation comes in two formats, man pages and full HTML
|
||||
documentation. Documentation is built into distribution's
|
||||
docs/dist directory. man pages are in docs/dist/man and
|
||||
HTML documentation in docs/dist/html.
|
||||
|
||||
If you want to use man pages, run:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
export MANPATH=$MANPATH:$(pwd -P)/docs/dist/man
|
||||
|
||||
Or you can move man pages from docs/dist/man directory to some
|
||||
other directory and add it to MANPATH.
|
||||
|
||||
Full HTML documentation can be accessed at docs/dist/html/index.html.
|
||||
|
||||
You can also build only man pages or only html documentation, for
|
||||
only man pages run:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make man
|
||||
|
||||
for only html documentation run:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make html
|
||||
|
||||
You can also build man pages for types in your ~/.cdist directory:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make dotman
|
||||
|
||||
Built man pages are now in docs/dist/man directory. If you have
|
||||
some other custom .cdist directory, e.g. /opt/cdist then use:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make DOT_CDIST_PATH=/opt/cdist dotman
|
||||
|
||||
Note that `dotman`-target has to be built before a `make docs`-run, otherwise
|
||||
the custom man-pages are not picked up.
|
||||
|
||||
Python package
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
Cdist is available as a python package at
|
||||
`PyPi <http://pypi.python.org/pypi/cdist/>`_. You can install it using
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
pip install cdist
|
||||
|
||||
Installing from source with signature verification
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If you want to install cdist from signed source and verify it, first you need to
|
||||
download cdist archive and its detached signature.
|
||||
|
||||
Get both, *cdist-x.y.z.tar.gz* and *cdist-x.y.z.tar.gz.asc* from release
|
||||
notes of the desired tag *x.y.z* at
|
||||
`cdist git repository <https://code.ungleich.ch/ungleich-public/cdist/-/tags>`_.
|
||||
|
||||
Get GPG public key used for signing `here <_static/pgp-key-EFD2AE4EC36B6901.asc>`_
|
||||
and import it into GPG.
|
||||
|
||||
Now cdist source archive can be verified using `gpg`, e.g. to verify `cdist-6.2.0`:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ gpg --verify cdist-6.2.0.tar.gz.asc cdist-6.2.0.targ.gz
|
||||
gpg: Signature made Sat Nov 30 23:14:19 2019 CET
|
||||
gpg: using RSA key 69767822F3ECC3C349C1EFFFEFD2AE4EC36B6901
|
||||
gpg: Good signature from "ungleich GmbH (ungleich FOSS) <foss@ungleich.ch>" [ultimate]
|
||||
|
||||
Further steps are the same as for `installing from git <cdist-install.html#from-git>`_.
|
47
src/extra/manual/6.5.5/_sources/cdist-integration.rst.txt
Normal file
47
src/extra/manual/6.5.5/_sources/cdist-integration.rst.txt
Normal file
|
@ -0,0 +1,47 @@
|
|||
cdist integration / using cdist as library
|
||||
==========================================
|
||||
|
||||
Description
|
||||
-----------
|
||||
|
||||
cdist can be integrate with other applications by importing cdist and other
|
||||
cdist modules and setting all by hand. There are also helper functions which
|
||||
aim to ease this integration. Just import **cdist.integration** and use its
|
||||
functions:
|
||||
|
||||
* :strong:`cdist.integration.configure_hosts_simple` for configuration
|
||||
* :strong:`cdist.integration.install_hosts_simple` for installation.
|
||||
|
||||
Functions require `host` and `manifest` parameters.
|
||||
`host` can be specified as a string representing host or as iterable
|
||||
of hosts. `manifest` is a path to initial manifest. For other cdist
|
||||
options default values will be used. `verbose` is a desired verbosity
|
||||
level which defaults to VERBOSE_INFO. `cdist_path` parameter specifies
|
||||
path to cdist executable, if it is `None` then functions will try to
|
||||
find it first from local lib directory and then in PATH.
|
||||
|
||||
In case of cdist error :strong:`cdist.Error` exception is raised.
|
||||
|
||||
:strong:`WARNING`: cdist integration helper functions are not yet stable!
|
||||
|
||||
Examples
|
||||
--------
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# configure host from python interactive shell
|
||||
>>> import cdist.integration
|
||||
>>> cdist.integration.configure_hosts_simple('185.203.114.185',
|
||||
... '~/.cdist/manifest/init')
|
||||
|
||||
# configure host from python interactive shell, specifying verbosity level
|
||||
>>> import cdist.integration
|
||||
>>> cdist.integration.configure_hosts_simple(
|
||||
... '185.203.114.185', '~/.cdist/manifest/init',
|
||||
... verbose=cdist.argparse.VERBOSE_TRACE)
|
||||
|
||||
# configure specified dns hosts from python interactive shell
|
||||
>>> import cdist.integration
|
||||
>>> hosts = ('dns1.ungleich.ch', 'dns2.ungleich.ch', 'dns3.ungleich.ch', )
|
||||
>>> cdist.integration.configure_hosts_simple(hosts,
|
||||
... '~/.cdist/manifest/init')
|
211
src/extra/manual/6.5.5/_sources/cdist-inventory.rst.txt
Normal file
211
src/extra/manual/6.5.5/_sources/cdist-inventory.rst.txt
Normal file
|
@ -0,0 +1,211 @@
|
|||
Inventory
|
||||
=========
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
cdist comes with simple built-in tag based inventory. It is a simple inventory
|
||||
with list of hosts and a host has a list of tags.
|
||||
Inventory functionality is still in **beta** so it can be used only if beta
|
||||
command line flag is specified (-b, --beta) or setting CDIST_BETA env var.
|
||||
|
||||
Description
|
||||
-----------
|
||||
|
||||
The idea is to have simple tagging inventory. There is a list of hosts and for
|
||||
each host there are tags. Inventory database is a set of files under inventory
|
||||
database base directory. Filename equals hostname. Each file contains tags for
|
||||
hostname with each tag on its own line.
|
||||
|
||||
Using inventory you can now configure hosts by selecting them by tags.
|
||||
|
||||
Tags have no values, as tags are just tags. Tag name-value would in this
|
||||
context mean that host has two tags and it is selected by specifying that both
|
||||
tags are present.
|
||||
|
||||
This inventory is **KISS** cdist built-in inventory database. You can maintain it
|
||||
using cdist inventory interface or using standard UNIX tools.
|
||||
|
||||
cdist inventory interface
|
||||
-------------------------
|
||||
|
||||
With cdist inventory interface you can list host(s) and tag(s), add host(s),
|
||||
add tag(s), delete host(s) and delete tag(s).
|
||||
|
||||
Configuring hosts using inventory
|
||||
---------------------------------
|
||||
|
||||
config command now has new options, **-t**, **-a** and **-A**.
|
||||
|
||||
**-A** means that all hosts in tag db is selected.
|
||||
|
||||
**-a** means that selected hosts must contain ALL specified tags.
|
||||
|
||||
**-t** means that host specifies tag - all hosts that have specified tags are
|
||||
selected.
|
||||
|
||||
Examples
|
||||
--------
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# List inventory content
|
||||
$ cdist inventory list -b
|
||||
|
||||
# List inventory for specified host localhost
|
||||
$ cdist inventory list -b localhost
|
||||
|
||||
# List inventory for specified tag loadbalancer
|
||||
$ cdist inventory list -b -t loadbalancer
|
||||
|
||||
# Add hosts to inventory
|
||||
$ cdist inventory add-host -b web1 web2 web3
|
||||
|
||||
# Delete hosts from file old-hosts from inventory
|
||||
$ cdist inventory del-host -b -f old-hosts
|
||||
|
||||
# Add tags to specified hosts
|
||||
$ cdist inventory add-tag -b -t europe,croatia,web,static web1 web2
|
||||
|
||||
# Add tag to all hosts in inventory
|
||||
$ cdist inventory add-tag -b -t vm
|
||||
|
||||
# Delete all tags from specified host
|
||||
$ cdist inventory del-tag -b -a localhost
|
||||
|
||||
# Delete tags read from stdin from hosts specified by file hosts
|
||||
$ cdist inventory del-tag -b -T - -f hosts
|
||||
|
||||
# Configure hosts from inventory with any of specified tags
|
||||
$ cdist config -b -t web dynamic
|
||||
|
||||
# Configure hosts from inventory with all specified tags
|
||||
$ cdist config -b -t -a web dynamic
|
||||
|
||||
# Configure all hosts from inventory db
|
||||
$ cdist config -b -A
|
||||
|
||||
Example of manipulating database
|
||||
--------------------------------
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
$ python3 scripts/cdist inventory add-host -b localhost
|
||||
$ python3 scripts/cdist inventory add-host -b test.mycloud.net
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
localhost
|
||||
test.mycloud.net
|
||||
$ python3 scripts/cdist inventory add-host -b web1.mycloud.net web2.mycloud.net shell1.mycloud.net shell2.mycloud.net
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
localhost
|
||||
test.mycloud.net
|
||||
web1.mycloud.net
|
||||
web2.mycloud.net
|
||||
shell1.mycloud.net
|
||||
shell2.mycloud.net
|
||||
$ python3 scripts/cdist inventory add-tag -b -t web web1.mycloud.net web2.mycloud.net
|
||||
$ python3 scripts/cdist inventory add-tag -b -t shell shell1.mycloud.net shell2.mycloud.net
|
||||
$ python3 scripts/cdist inventory add-tag -b -t cloud
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
localhost cloud
|
||||
test.mycloud.net cloud
|
||||
web1.mycloud.net cloud,web
|
||||
web2.mycloud.net cloud,web
|
||||
shell1.mycloud.net cloud,shell
|
||||
shell2.mycloud.net cloud,shell
|
||||
$ python3 scripts/cdist inventory add-tag -b -t test,web,shell test.mycloud.net
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
localhost cloud
|
||||
test.mycloud.net cloud,shell,test,web
|
||||
web1.mycloud.net cloud,web
|
||||
web2.mycloud.net cloud,web
|
||||
shell1.mycloud.net cloud,shell
|
||||
shell2.mycloud.net cloud,shell
|
||||
$ python3 scripts/cdist inventory del-tag -b -t shell test.mycloud.net
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
localhost cloud
|
||||
test.mycloud.net cloud,test,web
|
||||
web1.mycloud.net cloud,web
|
||||
web2.mycloud.net cloud,web
|
||||
shell1.mycloud.net cloud,shell
|
||||
shell2.mycloud.net cloud,shell
|
||||
$ python3 scripts/cdist inventory add-tag -b -t all
|
||||
$ python3 scripts/cdist inventory add-tag -b -t mistake
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
localhost all,cloud,mistake
|
||||
test.mycloud.net all,cloud,mistake,test,web
|
||||
web1.mycloud.net all,cloud,mistake,web
|
||||
web2.mycloud.net all,cloud,mistake,web
|
||||
shell1.mycloud.net all,cloud,mistake,shell
|
||||
shell2.mycloud.net all,cloud,mistake,shell
|
||||
$ python3 scripts/cdist inventory del-tag -b -t mistake
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
localhost all,cloud
|
||||
test.mycloud.net all,cloud,test,web
|
||||
web1.mycloud.net all,cloud,web
|
||||
web2.mycloud.net all,cloud,web
|
||||
shell1.mycloud.net all,cloud,shell
|
||||
shell2.mycloud.net all,cloud,shell
|
||||
$ python3 scripts/cdist inventory del-host -b localhost
|
||||
$ python3 scripts/cdist inventory list -b
|
||||
test.mycloud.net all,cloud,test,web
|
||||
web1.mycloud.net all,cloud,web
|
||||
web2.mycloud.net all,cloud,web
|
||||
shell1.mycloud.net all,cloud,shell
|
||||
shell2.mycloud.net all,cloud,shell
|
||||
$ python3 scripts/cdist inventory list -b -t web
|
||||
test.mycloud.net all,cloud,test,web
|
||||
web1.mycloud.net all,cloud,web
|
||||
web2.mycloud.net all,cloud,web
|
||||
$ python3 scripts/cdist inventory list -b -t -a web test
|
||||
test.mycloud.net all,cloud,test,web
|
||||
$ python3 scripts/cdist inventory list -b -t -a web all
|
||||
test.mycloud.net all,cloud,test,web
|
||||
web1.mycloud.net all,cloud,web
|
||||
web2.mycloud.net all,cloud,web
|
||||
$ python3 scripts/cdist inventory list -b -t web all
|
||||
test.mycloud.net all,cloud,test,web
|
||||
web1.mycloud.net all,cloud,web
|
||||
web2.mycloud.net all,cloud,web
|
||||
shell1.mycloud.net all,cloud,shell
|
||||
shell2.mycloud.net all,cloud,shell
|
||||
$ cd cdist/inventory
|
||||
$ ls -1
|
||||
shell1.mycloud.net
|
||||
shell2.mycloud.net
|
||||
test.mycloud.net
|
||||
web1.mycloud.net
|
||||
web2.mycloud.net
|
||||
$ ls -l
|
||||
total 20
|
||||
-rw-r--r-- 1 darko darko 16 Jun 24 12:43 shell1.mycloud.net
|
||||
-rw-r--r-- 1 darko darko 16 Jun 24 12:43 shell2.mycloud.net
|
||||
-rw-r--r-- 1 darko darko 19 Jun 24 12:43 test.mycloud.net
|
||||
-rw-r--r-- 1 darko darko 14 Jun 24 12:43 web1.mycloud.net
|
||||
-rw-r--r-- 1 darko darko 14 Jun 24 12:43 web2.mycloud.net
|
||||
$ cat test.mycloud.net
|
||||
test
|
||||
all
|
||||
web
|
||||
cloud
|
||||
$ cat web2.mycloud.net
|
||||
all
|
||||
web
|
||||
cloud
|
||||
|
||||
For more info about inventory commands and options see `cdist <man1/cdist.html>`_\ (1).
|
||||
|
||||
Using external inventory
|
||||
------------------------
|
||||
|
||||
cdist can be used with any external inventory where external inventory is
|
||||
some storage or database from which you can get a list of hosts to configure.
|
||||
cdist can then be fed with this list of hosts through stdin or file using
|
||||
**-f** option. For example, if your host list is stored in sqlite3 database
|
||||
hosts.db and you want to select hosts which purpose is **django** then you
|
||||
can use it with cdist like:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ sqlite3 hosts.db "select hostname from hosts where purpose = 'django';" | cdist config
|
376
src/extra/manual/6.5.5/_sources/cdist-manifest.rst.txt
Normal file
376
src/extra/manual/6.5.5/_sources/cdist-manifest.rst.txt
Normal file
|
@ -0,0 +1,376 @@
|
|||
Manifest
|
||||
========
|
||||
|
||||
Description
|
||||
-----------
|
||||
Manifests are used to define which objects to create.
|
||||
Objects are instances of **types**, like in object oriented programming languages.
|
||||
An object is represented by the combination of
|
||||
**type + slash + object name**: **\__file/etc/cdist-configured** is an
|
||||
object of the type **__file** with the name **etc/cdist-configured**.
|
||||
|
||||
All available types can be found in the **cdist/conf/type/** directory,
|
||||
use **ls cdist/conf/type** to get the list of available types. If you have
|
||||
setup the MANPATH correctly, you can use **man cdist-reference** to access
|
||||
the reference with pointers to the manpages.
|
||||
|
||||
|
||||
Types in manifests are used like normal command line tools. Let's have a look
|
||||
at an example::
|
||||
|
||||
# Create object of type __package with the parameter state = absent
|
||||
__package apache2 --state absent
|
||||
|
||||
# Same with the __directory type
|
||||
__directory /tmp/cdist --state present
|
||||
|
||||
These two lines create objects, which will later be used to realise the
|
||||
configuration on the target host.
|
||||
|
||||
Manifests are executed locally as a shell script using **/bin/sh -e**.
|
||||
The resulting objects are stored in an internal database.
|
||||
|
||||
The same object can be redefined in multiple different manifests as long as
|
||||
the parameters are exactly the same.
|
||||
|
||||
In general, manifests are used to define which types are used depending
|
||||
on given conditions.
|
||||
|
||||
|
||||
Initial and type manifests
|
||||
--------------------------
|
||||
Cdist knows about two types of manifests: The initial manifest and type
|
||||
manifests. The initial manifest is used to define, which configurations
|
||||
to apply to which hosts. The type manifests are used to create objects
|
||||
from types. More about manifests in types can be found in `cdist type <cdist-type.html>`_.
|
||||
|
||||
|
||||
Define state in the initial manifest
|
||||
------------------------------------
|
||||
The **initial manifest** is the entry point for cdist to find out, which
|
||||
**objects** to configure on the selected host.
|
||||
Cdist expects the initial manifest at **cdist/conf/manifest/init**.
|
||||
|
||||
Within this initial manifest you define which objects should be
|
||||
created on which host. To distinguish between hosts, you can use the
|
||||
environment variable **__target_host** and/or **__target_hostname** and/or
|
||||
**__target_fqdn**. Let's have a look at a simple example::
|
||||
|
||||
__cdistmarker
|
||||
|
||||
case "$__target_host" in
|
||||
localhost)
|
||||
__directory /home/services/kvm-vm --parents yes
|
||||
;;
|
||||
esac
|
||||
|
||||
This manifest says: Independent of the host, always use the type
|
||||
**__cdistmarker**, which creates the file **/etc/cdist-configured**,
|
||||
with the timestamp as content.
|
||||
The directory **/home/services/kvm-vm**, including all parent directories,
|
||||
is only created on the host **localhost**.
|
||||
|
||||
As you can see, there is no magic involved, the manifest is simple shell code that
|
||||
utilises cdist types. Every available type can be executed like a normal
|
||||
command.
|
||||
|
||||
|
||||
Splitting up the initial manifest
|
||||
---------------------------------
|
||||
If you want to split up your initial manifest, you can create other shell
|
||||
scripts in **cdist/conf/manifest/** and include them in **cdist/conf/manifest/init**.
|
||||
Cdist provides the environment variable **__manifest** to reference
|
||||
the directory containing the initial manifest (see `cdist reference <cdist-reference.html>`_).
|
||||
|
||||
The following example would include every file with a **.sh** suffix::
|
||||
|
||||
# Include *.sh
|
||||
for manifest in $__manifest/*.sh; do
|
||||
# And source scripts into our shell environment
|
||||
. "$manifest"
|
||||
done
|
||||
|
||||
|
||||
Dependencies
|
||||
------------
|
||||
If you want to describe that something requires something else, just
|
||||
setup the variable "require" to contain the requirements. Multiple
|
||||
requirements can be added white space separated.
|
||||
|
||||
::
|
||||
|
||||
1 # No dependency
|
||||
2 __file /etc/cdist-configured
|
||||
3
|
||||
4 # Require above object
|
||||
5 require="__file/etc/cdist-configured" __link /tmp/cdist-testfile \
|
||||
6 --source /etc/cdist-configured --type symbolic
|
||||
7
|
||||
8 # Require two objects
|
||||
9 require="__file/etc/cdist-configured __link/tmp/cdist-testfile" \
|
||||
10 __file /tmp/cdist-another-testfile
|
||||
|
||||
|
||||
Above the "require" variable is only set for the command that is
|
||||
immediately following it. Dependencies should always be declared that way.
|
||||
|
||||
On line 4 you can see that the instantiation of a type "\__link" object needs
|
||||
the object "__file/etc/cdist-configured" to be present, before it can proceed.
|
||||
|
||||
This also means that the "\__link" command must make sure, that either
|
||||
"\__file/etc/cdist-configured" already is present, or, if it's not, it needs
|
||||
to be created. The task of cdist is to make sure, that the dependency will be
|
||||
resolved appropriately and thus "\__file/etc/cdist-configured" be created
|
||||
if necessary before "__link" proceeds (or to abort execution with an error).
|
||||
|
||||
If you really need to make all types depend on a common dependency, you can
|
||||
export the "require" variable as well. But then, if you need to add extra
|
||||
dependencies to a specific type, you have to make sure that you append these
|
||||
to the globally already defined one.
|
||||
|
||||
::
|
||||
|
||||
# First of all, update the package index
|
||||
__package_update_index
|
||||
# Upgrade all the installed packages afterwards
|
||||
require="__package_update_index" __package_upgrade_all
|
||||
# Create a common dependency for all the next types so that they get to
|
||||
# be executed only after the package upgrade has finished
|
||||
export require="__package_upgrade_all"
|
||||
|
||||
# Ensure that lighttpd is installed after we have upgraded all the packages
|
||||
__package lighttpd --state present
|
||||
# Ensure that munin is installed after lighttpd is present and after all
|
||||
# the packages are upgraded
|
||||
require="$require __package/lighttpd" __package munin --state present
|
||||
|
||||
|
||||
All objects that are created in a type manifest are automatically required
|
||||
from the type that is calling them. This is called "autorequirement" in
|
||||
cdist jargon.
|
||||
|
||||
You can find a more in depth description of the flow execution of manifests
|
||||
in `cdist execution stages <cdist-stages.html>`_ and of how types work in `cdist type <cdist-type.html>`_.
|
||||
|
||||
|
||||
Create dependencies from execution order
|
||||
-----------------------------------------
|
||||
You can tell cdist to execute all types in the order in which they are created
|
||||
in the manifest by setting up the variable CDIST_ORDER_DEPENDENCY.
|
||||
When cdist sees that this variable is setup, the current created object
|
||||
automatically depends on the previously created object.
|
||||
|
||||
It essentially helps you to build up blocks of code that build upon each other
|
||||
(like first creating the directory xyz than the file below the directory).
|
||||
|
||||
Read also about `notes on CDIST_ORDER_DEPENDENCY <cdist-best-practice.html#notes-on-cdist-order-dependency>`_.
|
||||
|
||||
In version 6.2.0 semantic CDIST_ORDER_DEPENDENCY is finally fixed and well defined.
|
||||
|
||||
CDIST_ORDER_DEPENDENCY defines type order dependency context. Order dependency context
|
||||
starts when CDIST_ORDER_DEPENDENCY is set, and ends when it is unset. After each
|
||||
manifest execution finishes, any existing order dependency context is automatically
|
||||
unset. This ensures that CDIST_ORDER_DEPENDENCY is valid within the manifest where it
|
||||
is used. When order dependency context is defined then cdist executes types in the
|
||||
order in which they are created in the manifest inside order dependency context.
|
||||
|
||||
Sometimes the best way to see how something works is to see examples.
|
||||
|
||||
Suppose you have defined **initial manifest**:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
__cycle1 cycle1
|
||||
export CDIST_ORDER_DEPENDENCY=1
|
||||
__cycle2 cycle2
|
||||
__cycle3 cycle3
|
||||
|
||||
with types **__cycle1**:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
export CDIST_ORDER_DEPENDENCY=1
|
||||
__file /tmp/cycle11
|
||||
__file /tmp/cycle12
|
||||
__file /tmp/cycle13
|
||||
|
||||
**__cycle2**:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
__file /tmp/cycle21
|
||||
export CDIST_ORDER_DEPENDENCY=1
|
||||
__file /tmp/cycle22
|
||||
__file /tmp/cycle23
|
||||
unset CDIST_ORDER_DEPENDENCY
|
||||
__file /tmp/cycle24
|
||||
|
||||
**__cycle3**:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
__file /tmp/cycle31
|
||||
__file /tmp/cycle32
|
||||
export CDIST_ORDER_DEPENDENCY=1
|
||||
__file /tmp/cycle33
|
||||
__file /tmp/cycle34
|
||||
|
||||
For the above config, cdist results in the following expected *dependency graph*
|
||||
(type *__cycleX* is shown as *cX*, *__file/tmp/cycleXY* is shown as *fcXY*):
|
||||
|
||||
::
|
||||
|
||||
c1---->fc11
|
||||
| /\
|
||||
| |
|
||||
+----->fc12
|
||||
| /\
|
||||
| |
|
||||
+----->fc13
|
||||
|
||||
c2--+--->fc21
|
||||
/\ |
|
||||
| |
|
||||
| +----->fc22
|
||||
| | /\
|
||||
| | |
|
||||
| +----->fc23
|
||||
| |
|
||||
| |
|
||||
| +----->fc24
|
||||
|
|
||||
|
|
||||
c3---->fc31
|
||||
|
|
||||
|
|
||||
+----->fc32
|
||||
|
|
||||
|
|
||||
+----->fc33
|
||||
| /\
|
||||
| |
|
||||
+----->fc34
|
||||
|
||||
Before version 6.2.0 the above configuration would result in cycle:
|
||||
|
||||
::
|
||||
|
||||
ERROR: 185.203.112.26: Cycle detected in object dependencies:
|
||||
__file/tmp/cycle11 -> __cycle3/cycle3 -> __cycle2/cycle2 -> __cycle1/cycle1 -> __file/tmp/cycle11!
|
||||
|
||||
The following manifest shows an example for order dependency contexts:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
__file /tmp/fileA
|
||||
export CDIST_ORDER_DEPENDENCY=1
|
||||
__file /tmp/fileB
|
||||
__file /tmp/fileC
|
||||
__file /tmp/fileD
|
||||
unset CDIST_ORDER_DEPENDENCY
|
||||
__file /tmp/fileE
|
||||
__file /tmp/fileF
|
||||
export CDIST_ORDER_DEPENDENCY=1
|
||||
__file /tmp/fileG
|
||||
__file /tmp/fileH
|
||||
unset CDIST_ORDER_DEPENDENCY
|
||||
__file /tmp/fileI
|
||||
|
||||
This means:
|
||||
|
||||
* C depends on B
|
||||
* D depends on C
|
||||
* H depends on G
|
||||
|
||||
and there are no other dependencies from this manifest.
|
||||
|
||||
|
||||
Overrides
|
||||
---------
|
||||
In some special cases, you would like to create an already defined object
|
||||
with different parameters. In normal situations this leads to an error in cdist.
|
||||
If you wish, you can setup the environment variable CDIST_OVERRIDE
|
||||
(any value or even empty is ok) to tell cdist, that this object override is
|
||||
wanted and should be accepted.
|
||||
ATTENTION: Only use this feature if you are 100% sure in which order
|
||||
cdist encounters the affected objects, otherwise this results
|
||||
in an undefined situation.
|
||||
|
||||
If CDIST_OVERRIDE and CDIST_ORDER_DEPENDENCY are set for an object,
|
||||
CDIST_ORDER_DEPENDENCY will be ignored, because adding a dependency in case of
|
||||
overrides would result in circular dependencies, which is an error.
|
||||
|
||||
|
||||
Examples
|
||||
--------
|
||||
The initial manifest may for instance contain the following code:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# Always create this file, so other sysadmins know cdist is used.
|
||||
__file /etc/cdist-configured
|
||||
|
||||
case "$__target_host" in
|
||||
my.server.name)
|
||||
__directory /root/bin/
|
||||
__file /etc/issue.net --source "$__manifest/issue.net
|
||||
;;
|
||||
esac
|
||||
|
||||
The manifest of the type "nologin" may look like this:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
__file /etc/nologin --source "$__type/files/default.nologin"
|
||||
|
||||
This example makes use of dependencies:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# Ensure that lighttpd is installed
|
||||
__package lighttpd --state present
|
||||
# Ensure that munin makes use of lighttpd instead of the default webserver
|
||||
# package as decided by the package manager
|
||||
require="__package/lighttpd" __package munin --state present
|
||||
|
||||
How to override objects:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# for example in the initial manifest
|
||||
|
||||
# create user account foobar with some hash for password
|
||||
__user foobar --password 'some_fancy_hash' --home /home/foobarexample
|
||||
|
||||
# ... many statements and includes in the manifest later ...
|
||||
# somewhere in a conditionally sourced manifest
|
||||
# (e.g. for example only sourced if a special application is on the target host)
|
||||
|
||||
# this leads to an error ...
|
||||
__user foobar --password 'some_other_hash'
|
||||
|
||||
# this tells cdist, that you know that this is an override and should be accepted
|
||||
CDIST_OVERRIDE=yes __user foobar --password 'some_other_hash'
|
||||
# it's only an override, means the parameter --home is not touched
|
||||
# and stays at the original value of /home/foobarexample
|
||||
|
||||
Dependencies defined by execution order work as following:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# Tells cdist to execute all types in the order in which they are created ...
|
||||
export CDIST_ORDER_DEPENDENCY=on
|
||||
__sample_type 1
|
||||
require="__some_type_somewhere/id" __sample_type 2
|
||||
__example_type 23
|
||||
# Now this types are executed in the creation order until the variable is unset
|
||||
unset CDIST_ORDER_DEPENDENCY
|
||||
# all now following types cdist makes the order ..
|
||||
__not_in_order_type 42
|
||||
|
||||
# how it works :
|
||||
# this lines above are translated to:
|
||||
__sample_type 1
|
||||
require="__some_type_somewhere/id __sample_type/1" __sample_type 2
|
||||
require="__sample_type/2" __example_type 23
|
||||
__not_in_order_type 42
|
94
src/extra/manual/6.5.5/_sources/cdist-messaging.rst.txt
Normal file
94
src/extra/manual/6.5.5/_sources/cdist-messaging.rst.txt
Normal file
|
@ -0,0 +1,94 @@
|
|||
Messaging
|
||||
=========
|
||||
|
||||
Description
|
||||
-----------
|
||||
cdist has a simple but powerful way of allowing communication between
|
||||
the initial manifest and types as well as types and types.
|
||||
|
||||
Whenever execution is passed from cdist to one of the
|
||||
scripts described below, cdist generate 2 new temporary files
|
||||
and exports the environment variables **__messages_in** and
|
||||
**__messages_out** to point to them.
|
||||
|
||||
Before handing over the control, the content of the global message
|
||||
file is copied into the file referenced by **$__messages_in**.
|
||||
|
||||
After cdist gained control back, the content of the file referenced
|
||||
by **$__messages_out** is appended to the global message file.
|
||||
|
||||
This way overwriting any of the two files by accident does not
|
||||
interfere with other types.
|
||||
|
||||
The order of execution is not defined unless you create dependencies
|
||||
between the different objects (see `cdist manifest <cdist-manifest.html>`_) and thus you
|
||||
can only react reliably on messages by objects that you depend on.
|
||||
|
||||
|
||||
Availability
|
||||
------------
|
||||
Messaging is possible between all **local** scripts:
|
||||
|
||||
- initial manifest
|
||||
- type/manifest
|
||||
- type/gencode-local
|
||||
- type/gencode-remote
|
||||
|
||||
|
||||
Examples
|
||||
--------
|
||||
When you want to emit a message use:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
echo "something" >> "$__messages_out"
|
||||
|
||||
When you want to react on a message use:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
if grep -q "^__your_type/object/id:something" "$__messages_in"; then
|
||||
echo "I do something else"
|
||||
fi
|
||||
|
||||
Some real life examples:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# Reacting on changes from block for keepalive
|
||||
if grep -q "^__block/keepalive-vrrp" "$__messages_in"; then
|
||||
echo /etc/init.d/keepalived restart
|
||||
fi
|
||||
|
||||
# Reacting on changes of configuration files
|
||||
if grep -q "^__file/etc/one" $__messages_in; then
|
||||
echo 'for init in /etc/init.d/opennebula*; do $init restart; done'
|
||||
fi
|
||||
|
||||
Restart sshd on changes
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
os="$(cat "$__global/explorer/os")"
|
||||
|
||||
case "$os" in
|
||||
centos|redhat|suse)
|
||||
restart="/etc/init.d/sshd restart"
|
||||
;;
|
||||
debian|ubuntu)
|
||||
restart="/etc/init.d/ssh restart"
|
||||
;;
|
||||
*)
|
||||
cat << eof >&2
|
||||
Unsupported os $os.
|
||||
If you would like to have this type running on $os,
|
||||
you can either develop the changes and send a pull
|
||||
request or ask for a quote at www.ungleich.ch
|
||||
eof
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
if grep -q "^__key_value/PermitRootLogin" "$__messages_in"; then
|
||||
echo $restart
|
||||
fi
|
19
src/extra/manual/6.5.5/_sources/cdist-os.rst.txt
Normal file
19
src/extra/manual/6.5.5/_sources/cdist-os.rst.txt
Normal file
|
@ -0,0 +1,19 @@
|
|||
Supported operating systems
|
||||
===========================
|
||||
|
||||
cdist was tested or is know to run on at least
|
||||
|
||||
* `Alpine Linux <https://alpinelinux.org>`_
|
||||
* `Archlinux <http://www.archlinux.org>`_
|
||||
* `CentOS <http://www.centos.org>`_
|
||||