forked from ungleich-public/cdist
Add documentation and mentions for polyglot scripting capabilities for configuring and extending cdist [POLYGLOT]
This commit is contained in:
Tabulo
parent
9b3505e8a1
commit
ae10ff49dd
7 changed files with 673 additions and 67 deletions
|
|
@ -3,12 +3,29 @@ Explorer
|
||||||
|
|
||||||
Description
|
Description
|
||||||
-----------
|
-----------
|
||||||
Explorers are small shell scripts, which will be executed on the target
|
Explorers are small scripts, typically written in POSIX shell,
|
||||||
host. The aim of each explorer is to give hints to types on how to act on the
|
which will be executed on the target host.
|
||||||
|
The aim of each explorer is to give hints to types on how to act on the
|
||||||
target system. An explorer outputs the result to stdout, which is usually
|
target system. An explorer outputs the result to stdout, which is usually
|
||||||
a one liner, but may be empty or multi line especially in the case of
|
a one liner, but may be empty or multi line especially in the case of
|
||||||
type explorers.
|
type explorers.
|
||||||
|
|
||||||
|
.. tip::
|
||||||
|
An :program:`explorer` can be written in **any scripting language**,
|
||||||
|
provided it is executable and has a proper **shebang**.
|
||||||
|
|
||||||
|
Nevertheless, for explorers, it is usually best to stick with the
|
||||||
|
**POSIX shell** in order to minimize
|
||||||
|
requirements on target hosts where they would need to be executed.
|
||||||
|
|
||||||
|
For executable shell code, the recommended shebang is :code:`#!/bin/sh -e`.
|
||||||
|
|
||||||
|
If an :program:`explorer` lacks `execute` permissions,
|
||||||
|
:program:`cdist` assumes it to be written in **shell** and executes it using
|
||||||
|
`$CDIST_REMOTE_SHELL`, which defaults to :code:`/bin/sh -e`.
|
||||||
|
|
||||||
|
For more details and examples, see :doc:`cdist-polyglot`.
|
||||||
|
|
||||||
There are general explorers, which are run in an early stage, and
|
There are general explorers, which are run in an early stage, and
|
||||||
type explorers. Both work almost exactly the same way, with the difference
|
type explorers. Both work almost exactly the same way, with the difference
|
||||||
that the values of the general explorers are stored in a general location and
|
that the values of the general explorers are stored in a general location and
|
||||||
|
|
@ -32,9 +49,14 @@ error message on stderr, which will cause cdist to abort.
|
||||||
You can also use stderr for debugging purposes while developing a new
|
You can also use stderr for debugging purposes while developing a new
|
||||||
explorer.
|
explorer.
|
||||||
|
|
||||||
|
|
||||||
Examples
|
Examples
|
||||||
--------
|
--------
|
||||||
A very simple explorer may look like this::
|
A very simple explorer may look like this:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
#!/bin/sh -e
|
||||||
|
|
||||||
hostname
|
hostname
|
||||||
|
|
||||||
|
|
@ -44,6 +66,8 @@ A type explorer, which could check for the status of a package may look like thi
|
||||||
|
|
||||||
.. code-block:: sh
|
.. code-block:: sh
|
||||||
|
|
||||||
|
#!/bin/sh -e
|
||||||
|
|
||||||
if [ -f "$__object/parameter/name" ]; then
|
if [ -f "$__object/parameter/name" ]; then
|
||||||
name="$(cat "$__object/parameter/name")"
|
name="$(cat "$__object/parameter/name")"
|
||||||
else
|
else
|
||||||
|
|
|
||||||
|
|
@ -22,8 +22,14 @@ Fast development
|
||||||
Focus on straightforwardness of type creation is a main development objective
|
Focus on straightforwardness of type creation is a main development objective
|
||||||
Batteries included: A lot of requirements can be solved using standard types
|
Batteries included: A lot of requirements can be solved using standard types
|
||||||
|
|
||||||
Modern Programming Language
|
Modern Programming Language (for cdist itself)
|
||||||
cdist is written in Python
|
cdist itself is written in Python
|
||||||
|
|
||||||
|
Language-agnostic / Polyglot (for the rest)
|
||||||
|
Although cdist itself is written in Python, it can be configured
|
||||||
|
and extended with any scripting language available.
|
||||||
|
|
||||||
|
(The `POSIX shell <https://en.wikipedia.org/wiki/Unix_shell>`_ is recommended, especially for any code destined to run on target hosts)
|
||||||
|
|
||||||
Requirements, Scalability
|
Requirements, Scalability
|
||||||
No central server needed, cdist operates in push mode and can be run from any computer
|
No central server needed, cdist operates in push mode and can be run from any computer
|
||||||
|
|
@ -44,5 +50,6 @@ UNIX, familiar environment, documentation
|
||||||
Is available as manpages and HTML
|
Is available as manpages and HTML
|
||||||
|
|
||||||
UNIX, simplicity, familiar environment
|
UNIX, simplicity, familiar environment
|
||||||
cdist is configured in POSIX shell
|
The ubiquitious `POSIX shell <https://en.wikipedia.org/wiki/Unix_shell>`_ is the recommended language for configuring and extending cdist.
|
||||||
|
|
||||||
|
The :program:`Cdist API` is based on simple and familiar UNIX constructs: environment variables, standard I/O, and files/directories
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,9 @@ Manifest
|
||||||
|
|
||||||
Description
|
Description
|
||||||
-----------
|
-----------
|
||||||
Manifests are used to define which objects to create.
|
Manifests are scripts that are executed *locally* (on master)
|
||||||
|
for the purpose of defining which objects to create.
|
||||||
|
|
||||||
Objects are instances of **types**, like in object oriented programming languages.
|
Objects are instances of **types**, like in object oriented programming languages.
|
||||||
An object is represented by the combination of
|
An object is represented by the combination of
|
||||||
**type + slash + object name**: **\__file/etc/cdist-configured** is an
|
**type + slash + object name**: **\__file/etc/cdist-configured** is an
|
||||||
|
|
@ -27,7 +29,7 @@ at an example::
|
||||||
These two lines create objects, which will later be used to realise the
|
These two lines create objects, which will later be used to realise the
|
||||||
configuration on the target host.
|
configuration on the target host.
|
||||||
|
|
||||||
Manifests are executed locally as a shell script using **/bin/sh -e**.
|
Manifests are executed *locally* (on master).
|
||||||
The resulting objects are stored in an internal database.
|
The resulting objects are stored in an internal database.
|
||||||
|
|
||||||
The same object can be redefined in multiple different manifests as long as
|
The same object can be redefined in multiple different manifests as long as
|
||||||
|
|
@ -36,6 +38,20 @@ the parameters are exactly the same.
|
||||||
In general, manifests are used to define which types are used depending
|
In general, manifests are used to define which types are used depending
|
||||||
on given conditions.
|
on given conditions.
|
||||||
|
|
||||||
|
.. tip::
|
||||||
|
|
||||||
|
A manifest can be written in **any scripting language**,
|
||||||
|
provided that the script is executable and has a proper **shebang**.
|
||||||
|
|
||||||
|
For executable shell code, the recommended shebang is :code:`#!/bin/sh -e`.
|
||||||
|
|
||||||
|
If :program:`manifest` lacks `execute` permissions, :program:`cdist` assumes
|
||||||
|
it to be written in **shell** and executes it using
|
||||||
|
`$CDIST_LOCAL_SHELL`, which defaults to :code:`/bin/sh -e`.
|
||||||
|
|
||||||
|
For more details and examples, see :doc:`cdist-polyglot`.
|
||||||
|
|
||||||
|
.. _cdist-manifest#initial-and-type-manifests:
|
||||||
|
|
||||||
Initial and type manifests
|
Initial and type manifests
|
||||||
--------------------------
|
--------------------------
|
||||||
|
|
|
||||||
443
docs/src/cdist-polyglot.rst
Normal file
443
docs/src/cdist-polyglot.rst
Normal file
|
|
@ -0,0 +1,443 @@
|
||||||
|
Polyglot
|
||||||
|
========
|
||||||
|
|
||||||
|
Description
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Although **cdist** itself is written in **Python**, it features a
|
||||||
|
*language-agnostic* (and hence *polyglot*) extension system.
|
||||||
|
|
||||||
|
As such, **cdist** can be extended with a mix-and-match of
|
||||||
|
**any scripting language** in addition to the usual -and recommended-
|
||||||
|
**POSIX shell** (`sh`): `bash`, `perl`, `python`, `ruby`, `node`, ... whatever.
|
||||||
|
|
||||||
|
This is true for all extension mechanisms available for **cdist**, namely:
|
||||||
|
|
||||||
|
.. list-table::
|
||||||
|
|
||||||
|
* - :doc:`manifests <cdist-manifest>`
|
||||||
|
- (including :ref:`manifest/init <cdist-manifest#initial-and-type-manifests>`
|
||||||
|
and :ref:`type manifests <cdist-type#manifest>`)
|
||||||
|
|
||||||
|
* - :doc:`explorers <cdist-explorer>`
|
||||||
|
- (both **global** and :ref:`type explorers <cdist-type#explorers>`)
|
||||||
|
|
||||||
|
* - :ref:`gencode-* scripts <cdist-type#gencode-scripts>`
|
||||||
|
- (both :program:`gencode-local` and :program:`gencode-remote`)
|
||||||
|
|
||||||
|
* - and even :ref:`generated code <cdist-type#gencode-scripts>`
|
||||||
|
- (i.e. the outputs from
|
||||||
|
:ref:`gencode-* scripts <cdist-type#gencode-scripts>`)
|
||||||
|
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>
|
||||||
|
<a>You do not have to commit to any single language...</a>
|
||||||
|
</summary>
|
||||||
|
|
||||||
|
.. container::
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
It's indeed possible (though not necessarily recommended)
|
||||||
|
to **mix-and-match** different
|
||||||
|
languages when extending **cdist**, for example:
|
||||||
|
|
||||||
|
A **type** could, in principal, have a `manifest` and an **explorer** written
|
||||||
|
in **POSIX shell**, a `gencode-remote` in **Python**
|
||||||
|
(which could generate code in **POSIX shell**) and a `gencode-local`
|
||||||
|
in **Perl** (which could generate code in **Perl**,
|
||||||
|
or some other language), while you are at it...
|
||||||
|
|
||||||
|
Just don't expect to submit such a hodge-podge as a candidate for being
|
||||||
|
distributed with **cdist** itself, though... :-)
|
||||||
|
especially if it turns out to be something that can be acheieved with
|
||||||
|
reasonable effort in **POSIX shell**.
|
||||||
|
|
||||||
|
In practise, you would at least want to enforce some consistency, if anything for
|
||||||
|
code maintainibility and your own sanity, in addition to the
|
||||||
|
the `CAVEATS`_ mentioned down below.
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
</details>
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
Needless to say, just because you *can* do something,
|
||||||
|
doesn't mean you *should* be doing it, or it's even a *good idea* to do so.
|
||||||
|
|
||||||
|
As a general rule of thumb, when extending **cdist**,
|
||||||
|
there are many good reasons in favor of sticking with the **POSIX shell**
|
||||||
|
wherever you can, and very few in favor of opting for some other
|
||||||
|
scripting language.
|
||||||
|
|
||||||
|
This is particularly true for any code that is meant to be run *remotely*
|
||||||
|
on **target hosts** (such as **explorers**),
|
||||||
|
where it is usually important to keep assumptions and requirements/dependencies
|
||||||
|
to a bare minimum. See the `CAVEATS`_ down below.
|
||||||
|
|
||||||
|
That being said, **polyglot** capabilities of **cdist** can come
|
||||||
|
quite handy for when you really need this sort of thing,
|
||||||
|
provided that you are ready to bare the consequences,
|
||||||
|
including the burden of extra dependecies
|
||||||
|
--- which is usually not that hard for code run *locally* on **master**
|
||||||
|
(`manifests`, `gencode-*` scripts, and code generated by `gencode-local`).
|
||||||
|
|
||||||
|
In any case, the mere fact of knowing we *can* escape the POSIX hatch
|
||||||
|
if we really have to, can be quite comforting for those of us suffering
|
||||||
|
from POSIX claustrophobia... which *is* of course a real health hazard
|
||||||
|
associated with high anxiety levels and all,
|
||||||
|
in case you didn't already know... ;-)
|
||||||
|
|
||||||
|
|
||||||
|
Writing polyglot extensions for **cdist**
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Whatever the kind of script (`manifest`, explorer, ...) you are writing,
|
||||||
|
you need to ensure that all 3 conditions below are met:
|
||||||
|
|
||||||
|
1. your script starts with an appropriate **shebang** line, such as::
|
||||||
|
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
.. comment: It would have been nice to make use of an extension
|
||||||
|
(such as `"sphinx_design"`) which provides a `.. dropdown::`
|
||||||
|
directive (for toggling visibility) which is the reason for
|
||||||
|
the ugly `.. raw:: html` stuff below...
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary><a>It's usually preferable to rely on the <b>env</b> program...</a></summary>
|
||||||
|
|
||||||
|
.. container::
|
||||||
|
|
||||||
|
It's usually preferable to rely on the :program:`env` program,
|
||||||
|
like in the example above, to find the interpreter by searching the PATH.
|
||||||
|
|
||||||
|
The :program:`env` program is almost guaranteed to exist even on a rudimentary
|
||||||
|
UNIX/Linux system at a pretty stable location: `/usr/bin/env`
|
||||||
|
|
||||||
|
It is, of course, also possible to write down a **hard coded** path
|
||||||
|
for the interpreter, if you are certain that it will always be
|
||||||
|
located at that location, like so::
|
||||||
|
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
This may sometimes be desirable, for example when you want to ascertain
|
||||||
|
using a specific version of an interpreter or when you are unsure about
|
||||||
|
what might get foundthrough the PATH.
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
2. your script has "*execute*" permissions set (in the Unix/Linux sense),
|
||||||
|
like so::
|
||||||
|
|
||||||
|
chmod a+x /path/to/your/script
|
||||||
|
|
||||||
|
This is essentially what matters to **cdist**, which it will take as a
|
||||||
|
clue for invoking your script *directly* (instead of passing it
|
||||||
|
to a shell as an argument).
|
||||||
|
|
||||||
|
For **generated code**, `cdist` will automatically take care of setting
|
||||||
|
*execute* permissions for you,
|
||||||
|
based on the presence of a leading **shebang** within the generated code.
|
||||||
|
|
||||||
|
3. the **interpreter** referenced by the **shebang** is available on any host(s)
|
||||||
|
where your code will run.
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>
|
||||||
|
<a>
|
||||||
|
Even for the <b>POSIX shell</b>,
|
||||||
|
it is still recommended to <b>follow the same guidelines</b> outlined above.
|
||||||
|
</a>
|
||||||
|
</summary>
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Even if you are just writing for the **POSIX shell**,
|
||||||
|
it is still recommended to follow the same guidelines outlined above.
|
||||||
|
|
||||||
|
At the very least, make sure your script has a proper **shebang**.
|
||||||
|
|
||||||
|
- If you have been following the usual **cdist** advise:
|
||||||
|
you probably already have a proper **shebang** at the very beginning
|
||||||
|
of your POSIX shell scripts.
|
||||||
|
|
||||||
|
|
||||||
|
- If (and *only* if), your POSIX shell script *does* contain a proper **shebang**:
|
||||||
|
you are also encouraged to also give it *"execute"* permissions,
|
||||||
|
so that your **shebang** will actually get honored.
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
</details>
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
|
||||||
|
That's pretty much it... except...
|
||||||
|
|
||||||
|
.. seealso:: The `CAVEATS`_ below.
|
||||||
|
|
||||||
|
|
||||||
|
CAVEATS
|
||||||
|
^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Shebang and execute permissions
|
||||||
|
"""""""""""""""""""""""""""""""""
|
||||||
|
In general, the first two conditions above are trivial to satisfy:
|
||||||
|
Just make sure you put in a **shebang** and mark your script as *executable*.
|
||||||
|
|
||||||
|
|
||||||
|
**Beware**, however, that:
|
||||||
|
|
||||||
|
.. attention::
|
||||||
|
|
||||||
|
- If your script lacks `execute` permissions (regardless of any **shebang**):
|
||||||
|
**cdist** will end up passing your script to `/bin/sh -e`
|
||||||
|
(or to `local_shell` / `remote_shell`,
|
||||||
|
if one is configured for the current context),
|
||||||
|
which may or may not be what you want.
|
||||||
|
|
||||||
|
- If your script *does* have `execute` permissions but *lacks* a **shebang**:
|
||||||
|
you can no longer be sure which interpreter (if any) will end up running your script.
|
||||||
|
|
||||||
|
What is certain, on the other hand, is that there is a wide range of
|
||||||
|
different things that could happen in such a case, depending on the OS and the chain
|
||||||
|
of execution up to that point...
|
||||||
|
|
||||||
|
It is possible (but not certain) that, in such a case, your script may
|
||||||
|
end up getting fed into `/bin/sh` or the default shell
|
||||||
|
(whatever it happens to be for the current user).
|
||||||
|
|
||||||
|
There's even a legend according to which even `csh` may get a chance to feed
|
||||||
|
on your script, and then proceed to burning your barn...
|
||||||
|
|
||||||
|
So, don't do that.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Interpreter availibility
|
||||||
|
"""""""""""""""""""""""""""""""""
|
||||||
|
|
||||||
|
For the last condition (interpreter availability),
|
||||||
|
your mileage may vary for languages other than the **POSIX shell**.
|
||||||
|
|
||||||
|
- For scripts meant to be run *locally* on the **master**, things remain relatively easy :
|
||||||
|
All you may need, if anything,
|
||||||
|
is a one time installation of stuff.
|
||||||
|
|
||||||
|
So, things should be realtively easy when it comes to: :file:`manifest` and :file:`gencode-*` scripts themselves, as well as any code generated by :file:`gencode-local`.
|
||||||
|
|
||||||
|
|
||||||
|
- For scripts meant to be run *remotely* on **target hosts**, things might get quite tricky,
|
||||||
|
depending on how likely it is
|
||||||
|
for the desired **interpreter** to be installed by default
|
||||||
|
on the **target system**.
|
||||||
|
|
||||||
|
This is an important concern for :file:`explorer` scripts
|
||||||
|
and any code generated by :file:`gencode-remote`.
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
Apart from the POSIX shell (`/bin/sh`), there aren't many interpreters out
|
||||||
|
there that are likely to have a guaranteed presence on a pristine system.
|
||||||
|
|
||||||
|
At the very least, you would have to make sure that the required interpreter
|
||||||
|
(and any extra modules/libraries your script might depend on)
|
||||||
|
are indeed available on those host(s)
|
||||||
|
before your script is invoked...
|
||||||
|
which kind of goes against the near-zero-dependency philosphy embraced
|
||||||
|
by **cdist**.
|
||||||
|
|
||||||
|
Depending on the target host OS, you might get lucky with
|
||||||
|
`bash`, `perl`, or `python` being preinstalled.
|
||||||
|
Even then, those may not necessarily be the version you expect
|
||||||
|
or have the extra modules/libraries your script might require.
|
||||||
|
|
||||||
|
**You have been warned.**
|
||||||
|
|
||||||
|
|
||||||
|
More details
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
As mentioned earlier, **cdist** itself mostly cares about the script
|
||||||
|
being marked as an *executable*, which it will take as a clue for invoking
|
||||||
|
that script *directly* (instead of passing it to a shell as an argument).
|
||||||
|
|
||||||
|
The **shebang** magic is handled by the usual process `exec` mechanisms
|
||||||
|
of the host OS (where the script is invoked) that will take over from
|
||||||
|
that point on.
|
||||||
|
|
||||||
|
|
||||||
|
Here is a simplified summary :
|
||||||
|
|
||||||
|
+-------------+---------------+------------------------------+--------------+--------------------------------------------------------+
|
||||||
|
| executable? | shebang | invocation resembles | interpreter | remarks |
|
||||||
|
+=============+===============+==============================+==============+========================================================+
|
||||||
|
| yes | `#!/bin/sh` | `/path/to/script` | `/bin/sh` | shebang **honored** by OS |
|
||||||
|
+-------------+---------------+------------------------------+--------------+--------------------------------------------------------+
|
||||||
|
| yes | `#!/bin/bash` | `/path/to/script` | `/bin/bash` | shebang **honored** by OS |
|
||||||
|
+-------------+---------------+------------------------------+--------------+--------------------------------------------------------+
|
||||||
|
| yes | | `/path/to/script` | *uncertain* | shebang **absent** |
|
||||||
|
+-------------+---------------+------------------------------+--------------+--------------------------------------------------------+
|
||||||
|
| no | `#!/bin/sh` | `/bin/sh -e /path/to/script` | `/bin/sh -e` | shebang **irrelevant** (as script is not "executable") |
|
||||||
|
+-------------+---------------+------------------------------+--------------+--------------------------------------------------------+
|
||||||
|
| no | `#!/bin/bash` | `/bin/sh -e /path/to/script` | `/bin/sh -e` | shebang **irrelevant** (as script is not "executable") |
|
||||||
|
+-------------+---------------+------------------------------+--------------+--------------------------------------------------------+
|
||||||
|
| no | | `/bin/sh -e /path/to/script` | `/bin/sh -e` | shebang **irrelevant** (as script is not "executable") |
|
||||||
|
+-------------+---------------+------------------------------+--------------+--------------------------------------------------------+
|
||||||
|
|
||||||
|
In fact, it's a little bit more involved than the above. Remember:
|
||||||
|
|
||||||
|
- As a special case, for any **generated code** (output by `gencode-*` scripts),
|
||||||
|
**cdist** will solely rely on the presence (or absence) of a leading **shebang**,
|
||||||
|
and set the executable bits accordingly, for obvious reasons.
|
||||||
|
|
||||||
|
- In the end, if a script is NOT marked as "executable",
|
||||||
|
it will simply be passed as an argument to the configured shell
|
||||||
|
that corresponds to the relevant context (i.e. `local_shell` or `remote_shell`),
|
||||||
|
if one is defined within the **cdist** configuration,
|
||||||
|
or else to `/bin/sh -e`, as a fallback in in both cases.
|
||||||
|
|
||||||
|
Well, there are also some gory implementation details
|
||||||
|
(related to how environment variables get propagated),
|
||||||
|
but those should normally have no relevance to this discussion.
|
||||||
|
|
||||||
|
|
||||||
|
The API between **cdist** and any polyglot extensions
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Conceptually, the API, based on well-known UNIX constructs,
|
||||||
|
remains exactly the same as it is for
|
||||||
|
any extension written for the **POSIX shell**.
|
||||||
|
|
||||||
|
Basically, you are all set as long as your scripting language is capable of:
|
||||||
|
|
||||||
|
- accessing **environment variables**;
|
||||||
|
- reading from and writing to the **filesystem** (files, directories, ...);
|
||||||
|
- reading from :file:`STDIN` and writing to :file:`STDOUT` (and eventually to :file:`STDERR`)
|
||||||
|
- **executing** other programs/commands;
|
||||||
|
- **exiting** with an appropriate **status code** (where 0=>success).
|
||||||
|
|
||||||
|
For all we know, no serious scripting language out there
|
||||||
|
would be missing any such basics.
|
||||||
|
|
||||||
|
The actual syntax and mechanisms will obviously be different,
|
||||||
|
the shell idioms usually being much more concise for this sort of thing,
|
||||||
|
as expected.
|
||||||
|
|
||||||
|
See the below example entitled "`Interacting with the cdist API`_".
|
||||||
|
|
||||||
|
|
||||||
|
Examples
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Interacting with the cdist API
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
As an API example, here's an excerpt from a **cdist** `type manifest`,
|
||||||
|
written for the POSIX shell, showing how one would get at the name
|
||||||
|
of the kernel on the **target host**:::
|
||||||
|
|
||||||
|
kernel_name=$(cat "${__global}/explorer/kernel_name")
|
||||||
|
|
||||||
|
# ... do something with kernel_name ...
|
||||||
|
|
||||||
|
|
||||||
|
In a nutshell, the above snippet gives the general idea about the cdist API:
|
||||||
|
|
||||||
|
Basically, we are stuffing a shell variable with the contents of a file...
|
||||||
|
which happens to contain the output from the `kernel_name` explorer...
|
||||||
|
|
||||||
|
Before invoking our `manifest` script, **cdist** would have, among other things,
|
||||||
|
run all **global explorers** on the **target host**,
|
||||||
|
collected and copied their outputs under a temporary directory on the **master**, and
|
||||||
|
set a specific environment variable (`$__global`)
|
||||||
|
to the path of a specifc subdirectory of that temporary working area.
|
||||||
|
|
||||||
|
At this point, that file (which contains the kernel name) is sitting there,
|
||||||
|
ready to be slurped... which can obviously be done from any language
|
||||||
|
that can access environment variables and read files from the filesystem...
|
||||||
|
|
||||||
|
Here's how you could do the same thing in **Python**:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
#!/usr/bin/env python
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
def read_file(path):
|
||||||
|
content = ""
|
||||||
|
try:
|
||||||
|
with open(path, "r") as fd:
|
||||||
|
content = fd.read().rstrip('\n')
|
||||||
|
except EnvironmentError:
|
||||||
|
pass
|
||||||
|
return content
|
||||||
|
|
||||||
|
kernel_name = read_file( os.environ['__global'] + '/explorer/kernel_name' )
|
||||||
|
|
||||||
|
# ... do something with kernel_name ...
|
||||||
|
|
||||||
|
|
||||||
|
And in **Perl**, it could look like:
|
||||||
|
|
||||||
|
.. code-block:: perl
|
||||||
|
|
||||||
|
#!/usr/bin/env perl
|
||||||
|
|
||||||
|
sub read_file {
|
||||||
|
my ($path) = @_;
|
||||||
|
return unless open( my $fh, $path );
|
||||||
|
local ($/);
|
||||||
|
<$fh>
|
||||||
|
}
|
||||||
|
|
||||||
|
my $kernel_name = read_file("$ENV{__global}/explorer/kernel_name");
|
||||||
|
|
||||||
|
# ... do something with kernel_name ...
|
||||||
|
|
||||||
|
|
||||||
|
Incidently, this example also helps appreciate some aspects of programming
|
||||||
|
for the shell... which were designed for this sort of thing in the first place...
|
||||||
|
|
||||||
|
A polygot type explorer (in Perl)
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Here's an imaginary type explorer written in **Perl**,
|
||||||
|
that ouputs the version of the perl interpreter running on the target host:
|
||||||
|
|
||||||
|
.. code-block:: perl
|
||||||
|
|
||||||
|
#!/usr/bin/env perl
|
||||||
|
|
||||||
|
use English;
|
||||||
|
|
||||||
|
print "${PERL_VERSION}\n";
|
||||||
|
|
||||||
|
If the path to the intended interpreter can be ascertained, you can
|
||||||
|
put that down directly on the **shebang**, like so::
|
||||||
|
|
||||||
|
#!/usr/bin/perl
|
||||||
|
|
||||||
|
However, more often than not, you would want to rely
|
||||||
|
on the `env` program (`/usr/bin/env`) to
|
||||||
|
invoke the first interpreter with the given name (`perl`, in this case)
|
||||||
|
found on the current PATH, like in the above example.
|
||||||
|
|
||||||
|
Don't forget to set *execute* permissions on the script file:::
|
||||||
|
|
||||||
|
chmod a+x ...
|
||||||
|
|
||||||
|
Or else **cdist** will feed it to a shell instance...
|
||||||
|
which may burn your barn... :-)
|
||||||
|
|
@ -98,29 +98,120 @@ If 'deprecated' marker has no content then general message is printed, e.g.:
|
||||||
|
|
||||||
How to write a new type
|
How to write a new type
|
||||||
-----------------------
|
-----------------------
|
||||||
A type consists of
|
|
||||||
|
|
||||||
- parameter (optional)
|
|
||||||
- manifest (optional)
|
|
||||||
- singleton (optional)
|
|
||||||
- explorer (optional)
|
|
||||||
- gencode (optional)
|
|
||||||
- nonparallel (optional)
|
|
||||||
|
|
||||||
Types are stored below cdist/conf/type/. Their name should always be prefixed with
|
Types are stored below cdist/conf/type/. Their name should always be prefixed with
|
||||||
two underscores (__) to prevent collisions with other executables in $PATH.
|
two underscores (__) to prevent collisions with other executables in :code:`$PATH`.
|
||||||
|
|
||||||
To implement a new type, create the directory **cdist/conf/type/__NAME**.
|
To implement a new type, create the directory :file:`cdist/conf/type/{__NAME}`,
|
||||||
|
either manually or using the helper script `cdist-new-type <man1/cdist-new-type.html>`_
|
||||||
|
which will also create the basic skeleton for you.
|
||||||
|
|
||||||
Type manifest and gencode can be written in any language. They just need to be
|
A type consists of the following elements (all of which are currently *optional*):
|
||||||
executable and have a proper shebang. If they are not executable then cdist assumes
|
|
||||||
they are written in shell so they are executed using '/bin/sh -e' or 'CDIST_LOCAL_SHELL'.
|
|
||||||
|
|
||||||
For executable shell code it is suggested that shebang is '#!/bin/sh -e'.
|
* some **markers** in the form of **plain files** within the type's directory:
|
||||||
|
|
||||||
For creating type skeleton you can use helper script
|
.. list-table::
|
||||||
`cdist-new-type <man1/cdist-new-type.html>`_.
|
|
||||||
|
|
||||||
|
* - :file:`singleton`
|
||||||
|
- *(optional)*
|
||||||
|
- A type flagged as a :file:`singleton` may be used **only
|
||||||
|
once per host**, which is useful for types that can be used only once on a
|
||||||
|
system.
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
Singleton types do not take an object name as argument.
|
||||||
|
|
||||||
|
* - :file:`nonparallel`
|
||||||
|
- (optional)
|
||||||
|
- Objects of a type flagged as :file:`nonparallel` cannot be run in parallel
|
||||||
|
when using :code:`-j` option.
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
|
||||||
|
An example of such a type is :program:`__package_dpkg` type
|
||||||
|
where :program:`dpkg` itself prevents to be run in more than one instance.
|
||||||
|
|
||||||
|
* - :file:`install`
|
||||||
|
- *(optional)*
|
||||||
|
- A type flagged as :file:`install` is used only with :command:`install` command.
|
||||||
|
With other :program:`cdist` commands, i.e. :command:`config`, such types are skipped if used.
|
||||||
|
|
||||||
|
* - :file:`deprecated`
|
||||||
|
- *(optional)*
|
||||||
|
- A type flagged as :file:`deprecated` causes
|
||||||
|
:program:`cdist` to emit a **warning** whenever that type is used.
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
If the file that corresponds to the `deprecated` marker has any content,
|
||||||
|
then this is used as a custom **deprecation message** for the warning.
|
||||||
|
|
||||||
|
* some more **metadata**:
|
||||||
|
|
||||||
|
.. list-table::
|
||||||
|
|
||||||
|
* - :file:`parameter/\*`
|
||||||
|
- *(optional)*
|
||||||
|
- A type may have **parameters**. These must be declared following a simple convention described in `Defining parameters`_, which
|
||||||
|
permits specifying additional properties for each parameter:
|
||||||
|
|
||||||
|
* required or optional
|
||||||
|
* single-value or multi-value
|
||||||
|
* string or boolean
|
||||||
|
|
||||||
|
It is also possible to give a `default` value for each optional parameter.
|
||||||
|
|
||||||
|
* and some **code** (scripts):
|
||||||
|
|
||||||
|
.. list-table::
|
||||||
|
|
||||||
|
* - :file:`manifest`
|
||||||
|
- *(optional)*
|
||||||
|
- :doc:`Type manifest <cdist-manifest>`
|
||||||
|
|
||||||
|
* - :file:`explorer/*`
|
||||||
|
- *(optional)*
|
||||||
|
- Any number of :doc:`type explorer <cdist-explorer>` scripts may exist under :file:`explorer` subdirectory.
|
||||||
|
|
||||||
|
* - :file:`gencode-local`
|
||||||
|
- *(optional)*
|
||||||
|
- A script that generates code to be executed *locally* (on master).
|
||||||
|
|
||||||
|
* - :file:`gencode-remote`
|
||||||
|
- *(optional)*
|
||||||
|
- A script that generates code to be executed *remotely* (on target host).
|
||||||
|
|
||||||
|
|
||||||
|
.. tip::
|
||||||
|
|
||||||
|
Each of the above-mentioned scripts can be written in **any scripting language**,
|
||||||
|
provided that the script is executable and has a proper **shebang**.
|
||||||
|
|
||||||
|
For executable shell code, the recommended shebang is :code:`#!/bin/sh -e`.
|
||||||
|
|
||||||
|
If a script lacks `execute` permissions, :program:`cdist` assumes
|
||||||
|
it to be written in **shell** and executes it using
|
||||||
|
`$CDIST_LOCAL_SHELL` or `$CDIST_REMOTE_SHELL`, if one is defined
|
||||||
|
for the current execution context (*local* or *remote*),
|
||||||
|
or else falling back to :code:`/bin/sh -e`.
|
||||||
|
|
||||||
|
|
||||||
|
For any code susceptible to run on remote target hosts
|
||||||
|
(i.e. **explorers** and any code generated by :code:`gencode-remote`),
|
||||||
|
it is recommended to stick to **POSIX shell**
|
||||||
|
in order to minimize requirements on target hosts where they would need to be executed.
|
||||||
|
|
||||||
|
For more details and examples, see :doc:`cdist-polyglot`.
|
||||||
|
|
||||||
|
.. seealso:: `cdist execution stages <cdist-stages.html>`_
|
||||||
|
|
||||||
Defining parameters
|
Defining parameters
|
||||||
-------------------
|
-------------------
|
||||||
|
|
@ -307,6 +398,7 @@ stdin from */dev/null*:
|
||||||
done < "$__object/parameter/foo"
|
done < "$__object/parameter/foo"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
.. _cdist-type#manifest:
|
||||||
|
|
||||||
Writing the manifest
|
Writing the manifest
|
||||||
--------------------
|
--------------------
|
||||||
|
|
@ -380,6 +472,7 @@ in your type directory:
|
||||||
|
|
||||||
For example, package types are nonparallel types.
|
For example, package types are nonparallel types.
|
||||||
|
|
||||||
|
.. _cdist-type#explorers:
|
||||||
|
|
||||||
The type explorers
|
The type explorers
|
||||||
------------------
|
------------------
|
||||||
|
|
@ -402,6 +495,7 @@ client, like this (shortened version from the type __file):
|
||||||
md5sum < "$destination"
|
md5sum < "$destination"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
.. _cdist-type#gencode-scripts:
|
||||||
|
|
||||||
Writing the gencode script
|
Writing the gencode script
|
||||||
--------------------------
|
--------------------------
|
||||||
|
|
|
||||||
|
|
@ -4,44 +4,65 @@ Why should I use cdist?
|
||||||
There are several motivations to use cdist, these
|
There are several motivations to use cdist, these
|
||||||
are probably the most popular ones.
|
are probably the most popular ones.
|
||||||
|
|
||||||
Known language
|
No need to learn a new language
|
||||||
--------------
|
-------------------------------
|
||||||
|
|
||||||
Cdist is being configured in
|
When adopting cdist, your staff does not need to learn a new
|
||||||
`shell script <https://en.wikipedia.org/wiki/Shell_script>`_.
|
|
||||||
Shell script is used by UNIX system engineers for decades.
|
|
||||||
So when cdist is introduced, your staff does not need to learn a new
|
|
||||||
`DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_
|
`DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_
|
||||||
or programming language.
|
or programming language, as cdist can be configured
|
||||||
|
and extended in **any scripting language**, the recommended one
|
||||||
|
being `shell scripts <https://en.wikipedia.org/wiki/Shell_script>`_.
|
||||||
|
|
||||||
|
Shell scripts enjoy ubiquity: they have been widely used by UNIX system engineers
|
||||||
|
for decades, and a suitable interpreter (:code:`/bin/sh`) is all but
|
||||||
|
guaranteed to be widely available on target hosts.
|
||||||
|
|
||||||
|
|
||||||
|
Easy idempotance -- without having to give up control
|
||||||
|
-----------------------------------------------------------------------------
|
||||||
|
|
||||||
|
For the sake of `idempotence <https://en.wikipedia.org/wiki/Idempotence>`_, many **contemporary SCMs** choose to ditch the power and versatality of general purpose programming languages, and adopt some form of
|
||||||
|
declarative `DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_ for describing the desired end states on target systems.
|
||||||
|
|
||||||
|
:program:`Cdist` takes a quite different approach, enabling *both* `idempotence <https://en.wikipedia.org/wiki/Idempotence>`_ *and* a decent level of programming power.
|
||||||
|
|
||||||
|
Unlike other SCMs, :program:`cdist` allows you to use a general purpose scripting language (POSIX shell is recommended) for describing the desired end states on target systems, instead of some declarative `DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_.
|
||||||
|
|
||||||
|
Unlike regular scripting, however, you are not left on your own for ensuring `idempotence <https://en.wikipedia.org/wiki/Idempotence>`_. :program:`Cdist` makes this really easy.
|
||||||
|
|
||||||
|
It does not matter how many times you "invoke" **cdist types** and in which order: :program:`cdist` will ensure that the actual code associated with each type will be executed only once (in dependency order) which, in turn, may effectively end up becoming a no-op, if the actual state is already the same as the desired one.
|
||||||
|
|
||||||
|
.. TODO: It would be great if there were an "architectural overview" page which could be referenced from here.
|
||||||
|
|
||||||
|
|
||||||
Powerful language
|
Powerful language
|
||||||
-----------------
|
--------------------
|
||||||
|
|
||||||
Not only is shell scripting widely known by system engineers,
|
Compared to a typical `DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_,
|
||||||
but it is also a very powerful language. Here are some features
|
shell scripts feature a much more powerful language.
|
||||||
which make daily work easy:
|
Here are some features which make daily work easy:
|
||||||
|
|
||||||
* Configuration can react dynamically on explored values
|
* Ability to dynamically adapt configuration based on information
|
||||||
|
*explored* from target hosts;
|
||||||
* High level string manipulation (using sed, awk, grep)
|
* High level string manipulation (using sed, awk, grep)
|
||||||
* Conditional support (**if, case**)
|
* Conditional support (**if, case**)
|
||||||
* Loop support (**for, while**)
|
* Loop support (**for, while**)
|
||||||
* Support for dependencies between cdist types
|
* Variable expansion
|
||||||
|
* Support for dependencies between cdist types and objects
|
||||||
|
|
||||||
|
If and when needed, it's always possible to simply
|
||||||
|
make use of **any other scripting language** at your disposal
|
||||||
|
*(albeit at the expense of adding a dependency on the corresponding interpreter
|
||||||
|
and libraries)*.
|
||||||
|
|
||||||
More than shell scripting
|
|
||||||
-------------------------
|
|
||||||
|
|
||||||
If you compare regular shell scripting with cdist, there is one major
|
|
||||||
difference: When using cdist types,
|
|
||||||
the results are
|
|
||||||
`idempotent <https://en.wikipedia.org/wiki/Idempotence>`_.
|
|
||||||
In practise that means it does not matter in which order you
|
|
||||||
call cdist types, the result is always the same.
|
|
||||||
|
|
||||||
Zero dependency configuration management
|
Zero dependency configuration management
|
||||||
----------------------------------------
|
-----------------------------------------
|
||||||
|
|
||||||
Cdist requires very little on a target system. Even better,
|
Cdist requires very little on a target system. Even better,
|
||||||
in almost all cases all dependencies are usually fulfilled.
|
in almost all cases all dependencies are usually already
|
||||||
|
fulfilled.
|
||||||
Cdist does not require an agent or high level programming
|
Cdist does not require an agent or high level programming
|
||||||
languages on the target host: it will run on any host that
|
languages on the target host: it will run on any host that
|
||||||
has a **ssh server running** and a POSIX compatible shell
|
has a **ssh server running** and a POSIX compatible shell
|
||||||
|
|
|
||||||
|
|
@ -30,6 +30,7 @@ It natively supports IPv6 since the first release.
|
||||||
cdist-type
|
cdist-type
|
||||||
cdist-types
|
cdist-types
|
||||||
cdist-explorer
|
cdist-explorer
|
||||||
|
cdist-polyglot
|
||||||
cdist-messaging
|
cdist-messaging
|
||||||
cdist-parallelization
|
cdist-parallelization
|
||||||
cdist-inventory
|
cdist-inventory
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue