cdist-manifest.rst 13.1 KB
Newer Older
1 2
Manifest
========
3

4
Description
Nico Schottelius's avatar
Nico Schottelius committed
5
-----------
6
Manifests are used to define which objects to create.
Tomas Pospisek's avatar
Tomas Pospisek committed
7
Objects are instances of **types**, like in object oriented programming languages.
8
An object is represented by the combination of
Tomas Pospisek's avatar
Tomas Pospisek committed
9
**type + slash + object name**: **\__file/etc/cdist-configured** is an
Darko Poljak's avatar
Darko Poljak committed
10
object of the type **__file** with the name **etc/cdist-configured**.
11

Nico Schottelius's avatar
Nico Schottelius committed
12 13
All available types can be found in the **cdist/conf/type/** directory,
use **ls cdist/conf/type** to get the list of available types. If you have
14 15 16 17 18
setup the MANPATH correctly, you can use **man cdist-reference** to access
the reference with pointers to the manpages.


Types in manifests are used like normal command line tools. Let's have a look
Darko Poljak's avatar
Darko Poljak committed
19 20 21 22
at an example::

    # Create object of type __package with the parameter state = absent
    __package apache2 --state absent
23

Darko Poljak's avatar
Darko Poljak committed
24 25
    # Same with the __directory type
    __directory /tmp/cdist --state present
26 27 28 29 30 31

These two lines create objects, which will later be used to realise the 
configuration on the target host.

Manifests are executed locally as a shell script using **/bin/sh -e**.
The resulting objects are stored in an internal database.
Nico Schottelius's avatar
Nico Schottelius committed
32 33 34 35 36 37

The same object can be redefined in multiple different manifests as long as
the parameters are exactly the same.

In general, manifests are used to define which types are used depending
on given conditions.
Nico Schottelius's avatar
Nico Schottelius committed
38 39


40
Initial and type manifests
41
--------------------------
Chris Lamb's avatar
Chris Lamb committed
42
Cdist knows about two types of manifests: The initial manifest and type
43 44
manifests. The initial manifest is used to define, which configurations
to apply to which hosts. The type manifests are used to create objects
45
from types. More about manifests in types can be found in `cdist type <cdist-type.html>`_.
46 47


48
Define state in the initial manifest
49 50
------------------------------------
The **initial manifest** is the entry point for cdist to find out, which
51
**objects** to configure on the selected host.
Tomas Pospisek's avatar
Tomas Pospisek committed
52
Cdist expects the initial manifest at **cdist/conf/manifest/init**.
53

mhameed's avatar
mhameed committed
54
Within this initial manifest you define which objects should be
55
created on which host. To distinguish between hosts, you can use the
56 57
environment variable **__target_host** and/or **__target_hostname** and/or
**__target_fqdn**. Let's have a look at a simple example::
58

Darko Poljak's avatar
Darko Poljak committed
59
    __cdistmarker
60

Darko Poljak's avatar
Darko Poljak committed
61 62 63 64 65
    case "$__target_host" in
       localhost)
            __directory /home/services/kvm-vm --parents yes
       ;;
    esac
66 67

This manifest says: Independent of the host, always use the type 
Darko Poljak's avatar
Darko Poljak committed
68
**__cdistmarker**, which creates the file **/etc/cdist-configured**,
69
with the timestamp as content.
Darko Poljak's avatar
Darko Poljak committed
70 71
The directory **/home/services/kvm-vm**, including all parent directories, 
is only created on the host **localhost**.
72 73 74

As you can see, there is no magic involved, the manifest is simple shell code that
utilises cdist types. Every available type can be executed like a normal 
75
command.
76 77


78
Splitting up the initial manifest
79 80
---------------------------------
If you want to split up your initial manifest, you can create other shell
Nico Schottelius's avatar
Nico Schottelius committed
81
scripts in **cdist/conf/manifest/** and include them in **cdist/conf/manifest/init**.
Darko Poljak's avatar
Darko Poljak committed
82
Cdist provides the environment variable **__manifest** to reference
83
the directory containing the initial manifest (see `cdist reference <cdist-reference.html>`_).
Nico Schottelius's avatar
Nico Schottelius committed
84

Darko Poljak's avatar
Darko Poljak committed
85
The following example would include every file with a **.sh** suffix::
Nico Schottelius's avatar
Nico Schottelius committed
86

Darko Poljak's avatar
Darko Poljak committed
87 88 89 90 91
    # Include *.sh
    for manifest in $__manifest/*.sh; do
        # And source scripts into our shell environment
        . "$manifest"
    done
Nico Schottelius's avatar
Nico Schottelius committed
92 93


94
Dependencies
95 96 97
------------
If you want to describe that something requires something else, just
setup the variable "require" to contain the requirements. Multiple
98 99
requirements can be added separated with (optionally consecutive)
delimiters including space, tab and newline.
100

Darko Poljak's avatar
Darko Poljak committed
101
::
102

Darko Poljak's avatar
Darko Poljak committed
103 104 105 106 107 108 109 110 111 112
     1 # No dependency
     2 __file /etc/cdist-configured
     3 
     4 # Require above object
     5 require="__file/etc/cdist-configured" __link /tmp/cdist-testfile \
     6    --source /etc/cdist-configured  --type symbolic
     7 
     8 # Require two objects
     9 require="__file/etc/cdist-configured __link/tmp/cdist-testfile" \
    10    __file /tmp/cdist-another-testfile
113 114


115
Above the "require" variable is only set for the command that is 
116
immediately following it. Dependencies should always be declared that way.
117

mhameed's avatar
mhameed committed
118
On line 4 you can see that the instantiation of a type "\__link" object needs
119 120
the object "__file/etc/cdist-configured" to be present, before it can proceed.

121
This also means that the "\__link" command must make sure, that either
Darko Poljak's avatar
Darko Poljak committed
122
"\__file/etc/cdist-configured" already is present, or, if it's not, it needs
123
to be created. The task of cdist is to make sure, that the dependency will be
124
resolved appropriately and thus "\__file/etc/cdist-configured" be created
125
if necessary before "__link" proceeds (or to abort execution with an error).
126

127 128 129 130 131
If you really need to make all types depend on a common dependency, you can
export the "require" variable as well. But then, if you need to add extra
dependencies to a specific type, you have to make sure that you append these
to the globally already defined one.

Darko Poljak's avatar
Darko Poljak committed
132
::
133

Darko Poljak's avatar
Darko Poljak committed
134 135 136 137 138 139 140
    # First of all, update the package index
    __package_update_index
    # Upgrade all the installed packages afterwards
    require="__package_update_index" __package_upgrade_all
    # Create a common dependency for all the next types so that they get to
    # be executed only after the package upgrade has finished
    export require="__package_upgrade_all"
141

Darko Poljak's avatar
Darko Poljak committed
142 143 144 145 146
    # Ensure that lighttpd is installed after we have upgraded all the packages
    __package lighttpd --state present
    # Ensure that munin is installed after lighttpd is present and after all
    # the packages are upgraded
    require="$require __package/lighttpd" __package munin --state present
147 148


149 150 151 152
All objects that are created in a type manifest are automatically required
from the type that is calling them. This is called "autorequirement" in
cdist jargon.

mhameed's avatar
mhameed committed
153
You can find a more in depth description of the flow execution of manifests
154
in `cdist execution stages <cdist-stages.html>`_ and of how types work in `cdist type <cdist-type.html>`_.
155

Darko Poljak's avatar
Darko Poljak committed
156

Darko Poljak's avatar
Darko Poljak committed
157 158 159 160 161 162 163 164 165 166
Processing preconditions
------------------------
If you want to describe that something should be processed only if specified
objects changed target's state, i.e. specified objects generated either
"code-local" or "code-remote", or both, just setup the variable "onchange" to
contain processing preconditions. "onchange" is specified in the same way as
"require", and it implies "require". If processing preconditions are not
satisfied then it means that object's explorers are not run, manifest is not
run, and gencode scripts are not run. Object processing is skipped.

167
Create dependencies from execution order
168 169 170 171 172 173 174 175 176
-----------------------------------------
You can tell cdist to execute all types in the order in which they are created 
in the manifest by setting up the variable CDIST_ORDER_DEPENDENCY.
When cdist sees that this variable is setup, the current created object
automatically depends on the previously created object.

It essentially helps you to build up blocks of code that build upon each other
(like first creating the directory xyz than the file below the directory).

177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296
Read also about `notes on CDIST_ORDER_DEPENDENCY <cdist-best-practice.html#notes-on-cdist-order-dependency>`_.

In version 6.2.0 semantic CDIST_ORDER_DEPENDENCY is finally fixed and well defined.

CDIST_ORDER_DEPENDENCY defines type order dependency context. Order dependency context
starts when CDIST_ORDER_DEPENDENCY is set, and ends when it is unset. After each
manifest execution finishes, any existing order dependency context is automatically
unset. This ensures that CDIST_ORDER_DEPENDENCY is valid within the manifest where it
is used. When order dependency context is defined then cdist executes types in the
order in which they are created in the manifest inside order dependency context.

Sometimes the best way to see how something works is to see examples.

Suppose you have defined **initial manifest**:

.. code-block:: sh

    __cycle1 cycle1
    export CDIST_ORDER_DEPENDENCY=1
    __cycle2 cycle2
    __cycle3 cycle3

with types **__cycle1**:

.. code-block:: sh

    export CDIST_ORDER_DEPENDENCY=1
    __file /tmp/cycle11
    __file /tmp/cycle12
    __file /tmp/cycle13

**__cycle2**:

.. code-block:: sh

    __file /tmp/cycle21
    export CDIST_ORDER_DEPENDENCY=1
    __file /tmp/cycle22
    __file /tmp/cycle23
    unset CDIST_ORDER_DEPENDENCY
    __file /tmp/cycle24

**__cycle3**:

.. code-block:: sh

    __file /tmp/cycle31
    __file /tmp/cycle32
    export CDIST_ORDER_DEPENDENCY=1
    __file /tmp/cycle33
    __file /tmp/cycle34

For the above config, cdist results in the following expected *dependency graph*
(type *__cycleX* is shown as *cX*, *__file/tmp/cycleXY* is shown as *fcXY*):

::

    c1---->fc11
    |      /\
    |       |
    +----->fc12
    |      /\
    |       |
    +----->fc13

    c2--+--->fc21
    /\  |
    |   |
    |   +----->fc22
    |   |      /\
    |   |       |
    |   +----->fc23
    |   |
    |   |
    |   +----->fc24
    |
    |
    c3---->fc31
    |
    |
    +----->fc32
    |
    |
    +----->fc33
    |      /\
    |       |
    +----->fc34

Before version 6.2.0 the above configuration would result in cycle:

::

    ERROR: 185.203.112.26: Cycle detected in object dependencies:
    __file/tmp/cycle11 -> __cycle3/cycle3 -> __cycle2/cycle2 -> __cycle1/cycle1 -> __file/tmp/cycle11!

The following manifest shows an example for order dependency contexts:

.. code-block:: sh

    __file /tmp/fileA
    export CDIST_ORDER_DEPENDENCY=1
    __file /tmp/fileB
    __file /tmp/fileC
    __file /tmp/fileD
    unset CDIST_ORDER_DEPENDENCY
    __file /tmp/fileE
    __file /tmp/fileF
    export CDIST_ORDER_DEPENDENCY=1
    __file /tmp/fileG
    __file /tmp/fileH
    unset CDIST_ORDER_DEPENDENCY
    __file /tmp/fileI

This means:

* C depends on B
* D depends on C
* H depends on G

and there are no other dependencies from this manifest.
297

298

299
Overrides
300 301 302
---------
In some special cases, you would like to create an already defined object 
with different parameters. In normal situations this leads to an error in cdist.
303
If you wish, you can setup the environment variable CDIST_OVERRIDE
304
(any value or even empty is ok) to tell cdist, that this object override is 
305 306
wanted and should be accepted.
ATTENTION: Only use this feature if you are 100% sure in which order 
307
cdist encounters the affected objects, otherwise this results
Tomas Pospisek's avatar
wording  
Tomas Pospisek committed
308
in an undefined situation. 
309

Tomas Pospisek's avatar
typo  
Tomas Pospisek committed
310
If CDIST_OVERRIDE and CDIST_ORDER_DEPENDENCY are set for an object,
311 312 313
CDIST_ORDER_DEPENDENCY will be ignored, because adding a dependency in case of
overrides would result in circular dependencies, which is an error.

314

315
Examples
316 317 318
--------
The initial manifest may for instance contain the following code:

Darko Poljak's avatar
Darko Poljak committed
319
.. code-block:: sh
320

Darko Poljak's avatar
Darko Poljak committed
321 322 323 324 325 326 327 328 329
    # Always create this file, so other sysadmins know cdist is used.
    __file /etc/cdist-configured

    case "$__target_host" in
       my.server.name)
          __directory /root/bin/
          __file /etc/issue.net --source "$__manifest/issue.net
       ;;
    esac
330 331 332

The manifest of the type "nologin" may look like this:

Darko Poljak's avatar
Darko Poljak committed
333 334 335
.. code-block:: sh

    __file /etc/nologin --source "$__type/files/default.nologin"
Nico Schottelius's avatar
Nico Schottelius committed
336

337 338
This example makes use of dependencies:

Darko Poljak's avatar
Darko Poljak committed
339
.. code-block:: sh
340

Darko Poljak's avatar
Darko Poljak committed
341 342 343 344 345
    # Ensure that lighttpd is installed
    __package lighttpd --state present
    # Ensure that munin makes use of lighttpd instead of the default webserver
    # package as decided by the package manager
    require="__package/lighttpd" __package munin --state present
346

Darko Poljak's avatar
Darko Poljak committed
347
How to override objects:
348

Darko Poljak's avatar
Darko Poljak committed
349
.. code-block:: sh
350

Darko Poljak's avatar
Darko Poljak committed
351
    # for example in the initial manifest
352

Darko Poljak's avatar
Darko Poljak committed
353 354
    # create user account foobar with some hash for password
    __user foobar --password 'some_fancy_hash' --home /home/foobarexample
355

Darko Poljak's avatar
Darko Poljak committed
356 357 358
    # ... many statements and includes in the manifest later ...
    # somewhere in a conditionally sourced manifest
    # (e.g. for example only sourced if a special application is on the target host)
359

Darko Poljak's avatar
Darko Poljak committed
360 361
    # this leads to an error ...
    __user foobar --password 'some_other_hash' 
362

Darko Poljak's avatar
Darko Poljak committed
363 364 365 366
    # this tells cdist, that you know that this is an override and should be accepted
    CDIST_OVERRIDE=yes __user foobar --password 'some_other_hash'
    # it's only an override, means the parameter --home is not touched 
    # and stays at the original value of /home/foobarexample
367

Darko Poljak's avatar
Darko Poljak committed
368
Dependencies defined by execution order work as following:
369

Darko Poljak's avatar
Darko Poljak committed
370
.. code-block:: sh
371

Darko Poljak's avatar
Darko Poljak committed
372 373 374 375 376 377 378 379 380
    # Tells cdist to execute all types in the order in which they are created ...
    export CDIST_ORDER_DEPENDENCY=on
    __sample_type 1
    require="__some_type_somewhere/id" __sample_type 2
    __example_type 23
    # Now this types are executed in the creation order until the variable is unset
    unset CDIST_ORDER_DEPENDENCY
    # all now following types cdist makes the order ..
    __not_in_order_type 42
381

Darko Poljak's avatar
Darko Poljak committed
382 383 384 385 386 387
    # how it works :
    # this lines above are translated to:
    __sample_type 1
    require="__some_type_somewhere/id __sample_type/1" __sample_type 2
    require="__sample_type/2" __example_type 23
    __not_in_order_type 42
Darko Poljak's avatar
Darko Poljak committed
388 389 390 391 392 393 394 395 396 397

This example makes use of processing preconditions:

.. code-block:: sh

    __foo bar
    __spam eggs
    # execute __baz type only if any of __foo/bar, __spam/eggs has changed
    # target's state, i.e. code-local or code-remote is generated (or both)
    onchange='__foo/bar __spam/eggs' __baz