forked from ungleich-public/cdist
		
	Add documentation and mentions for polyglot scripting capabilities for configuring and extending cdist [POLYGLOT]
This commit is contained in:
		
					parent
					
						
							
								9b3505e8a1
							
						
					
				
			
			
				commit
				
					
						ae10ff49dd
					
				
			
		
					 7 changed files with 673 additions and 67 deletions
				
			
		|  | @ -3,12 +3,29 @@ Explorer | |||
| 
 | ||||
| Description | ||||
| ----------- | ||||
| Explorers are small shell scripts, which will be executed on the target | ||||
| host. The aim of each explorer is to give hints to types on how to act on the | ||||
| Explorers are small scripts, typically written in POSIX shell, | ||||
| which will be executed on the target host. | ||||
| The aim of each explorer is to give hints to types on how to act on the | ||||
| target system. An explorer outputs the result to stdout, which is usually | ||||
| a one liner, but may be empty or multi line especially in the case of | ||||
| type explorers. | ||||
| 
 | ||||
| .. tip:: | ||||
|     An :program:`explorer` can be written in **any scripting language**, | ||||
|     provided it is executable and has a proper **shebang**. | ||||
| 
 | ||||
|     Nevertheless, for explorers, it is usually best to stick with the | ||||
|     **POSIX shell** in order to minimize | ||||
|     requirements on target hosts where they would need to be executed. | ||||
| 
 | ||||
|     For executable shell code, the recommended shebang is :code:`#!/bin/sh -e`. | ||||
| 
 | ||||
|     If an :program:`explorer` lacks `execute` permissions, | ||||
|     :program:`cdist` assumes it to be written in **shell** and executes it using | ||||
|     `$CDIST_REMOTE_SHELL`, which defaults to :code:`/bin/sh -e`. | ||||
| 
 | ||||
|     For more details and examples, see :doc:`cdist-polyglot`. | ||||
| 
 | ||||
| There are general explorers, which are run in an early stage, and | ||||
| type explorers. Both work almost exactly the same way, with the difference | ||||
| that the values of the general explorers are stored in a general location and | ||||
|  | @ -32,9 +49,14 @@ error message on stderr, which will cause cdist to abort. | |||
| You can also use stderr for debugging purposes while developing a new | ||||
| explorer. | ||||
| 
 | ||||
| 
 | ||||
| Examples | ||||
| -------- | ||||
| A very simple explorer may look like this:: | ||||
| A very simple explorer may look like this: | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     #!/bin/sh -e | ||||
| 
 | ||||
|     hostname | ||||
| 
 | ||||
|  | @ -44,6 +66,8 @@ A type explorer, which could check for the status of a package may look like thi | |||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     #!/bin/sh -e | ||||
| 
 | ||||
|     if [ -f "$__object/parameter/name" ]; then | ||||
|        name="$(cat "$__object/parameter/name")" | ||||
|     else | ||||
|  |  | |||
|  | @ -22,8 +22,14 @@ Fast development | |||
|     Focus on straightforwardness of type creation is a main development objective | ||||
|     Batteries included: A lot of requirements can be solved using standard types | ||||
| 
 | ||||
| Modern Programming Language | ||||
|     cdist is written in Python | ||||
| Modern Programming Language (for cdist itself) | ||||
|     cdist itself is written in Python | ||||
| 
 | ||||
| Language-agnostic / Polyglot (for the rest) | ||||
|     Although cdist itself is written in Python, it can be configured | ||||
|     and extended with any scripting language available. | ||||
| 
 | ||||
|     (The `POSIX shell <https://en.wikipedia.org/wiki/Unix_shell>`_ is recommended, especially for any code destined to run on target hosts) | ||||
| 
 | ||||
| Requirements, Scalability | ||||
|     No central server needed, cdist operates in push mode and can be run from any computer | ||||
|  | @ -44,5 +50,6 @@ UNIX, familiar environment, documentation | |||
|     Is available as manpages and HTML | ||||
| 
 | ||||
| UNIX, simplicity, familiar environment | ||||
|     cdist is configured in POSIX shell | ||||
|     The ubiquitious `POSIX shell <https://en.wikipedia.org/wiki/Unix_shell>`_ is the recommended language for configuring and extending cdist. | ||||
| 
 | ||||
|     The :program:`Cdist API` is based on simple and familiar UNIX constructs: environment variables, standard I/O, and files/directories | ||||
|  |  | |||
|  | @ -3,7 +3,9 @@ Manifest | |||
| 
 | ||||
| Description | ||||
| ----------- | ||||
| Manifests are used to define which objects to create. | ||||
| Manifests are scripts that are executed *locally* (on master) | ||||
| for the purpose of defining which objects to create. | ||||
| 
 | ||||
| Objects are instances of **types**, like in object oriented programming languages. | ||||
| An object is represented by the combination of | ||||
| **type + slash + object name**: **\__file/etc/cdist-configured** is an | ||||
|  | @ -27,7 +29,7 @@ at an example:: | |||
| These two lines create objects, which will later be used to realise the | ||||
| configuration on the target host. | ||||
| 
 | ||||
| Manifests are executed locally as a shell script using **/bin/sh -e**. | ||||
| Manifests are executed *locally* (on master). | ||||
| The resulting objects are stored in an internal database. | ||||
| 
 | ||||
| The same object can be redefined in multiple different manifests as long as | ||||
|  | @ -36,6 +38,20 @@ the parameters are exactly the same. | |||
| In general, manifests are used to define which types are used depending | ||||
| on given conditions. | ||||
| 
 | ||||
| .. tip:: | ||||
| 
 | ||||
|     A manifest can be written in **any scripting language**, | ||||
|     provided that the script is executable and has a proper **shebang**. | ||||
| 
 | ||||
|     For executable shell code, the recommended shebang is :code:`#!/bin/sh -e`. | ||||
| 
 | ||||
|     If :program:`manifest` lacks `execute` permissions,  :program:`cdist` assumes | ||||
|     it to be written in **shell** and executes it using | ||||
|     `$CDIST_LOCAL_SHELL`, which defaults to :code:`/bin/sh -e`. | ||||
| 
 | ||||
|     For more details and examples, see :doc:`cdist-polyglot`. | ||||
| 
 | ||||
| .. _cdist-manifest#initial-and-type-manifests: | ||||
| 
 | ||||
| Initial and type manifests | ||||
| -------------------------- | ||||
|  |  | |||
							
								
								
									
										443
									
								
								docs/src/cdist-polyglot.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										443
									
								
								docs/src/cdist-polyglot.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,443 @@ | |||
| Polyglot | ||||
| ======== | ||||
| 
 | ||||
| Description | ||||
| ----------- | ||||
| 
 | ||||
| Although **cdist** itself is written in **Python**, it features a | ||||
| *language-agnostic* (and hence *polyglot*) extension system. | ||||
| 
 | ||||
| As such, **cdist** can be extended with a mix-and-match of | ||||
| **any scripting language** in addition to the usual -and recommended- | ||||
| **POSIX shell** (`sh`): `bash`, `perl`, `python`, `ruby`, `node`, ... whatever. | ||||
| 
 | ||||
| This is true for all extension mechanisms available for **cdist**, namely: | ||||
| 
 | ||||
| .. list-table:: | ||||
| 
 | ||||
|     * - :doc:`manifests <cdist-manifest>` | ||||
|       - (including :ref:`manifest/init <cdist-manifest#initial-and-type-manifests>` | ||||
|         and :ref:`type manifests <cdist-type#manifest>`) | ||||
| 
 | ||||
|     * - :doc:`explorers <cdist-explorer>` | ||||
|       - (both **global** and :ref:`type explorers <cdist-type#explorers>`) | ||||
| 
 | ||||
|     * - :ref:`gencode-* scripts <cdist-type#gencode-scripts>` | ||||
|       - (both :program:`gencode-local` and :program:`gencode-remote`) | ||||
| 
 | ||||
|     * - and even :ref:`generated code <cdist-type#gencode-scripts>` | ||||
|       - (i.e. the outputs from | ||||
|         :ref:`gencode-* scripts <cdist-type#gencode-scripts>`) | ||||
| 
 | ||||
| 
 | ||||
| .. raw:: html | ||||
| 
 | ||||
|     <details> | ||||
|     <summary> | ||||
|         <a>You do not have to commit to any single language...</a> | ||||
|     </summary> | ||||
| 
 | ||||
| .. container:: | ||||
| 
 | ||||
|     .. note:: | ||||
| 
 | ||||
|         It's indeed possible (though not necessarily recommended) | ||||
|         to **mix-and-match** different | ||||
|         languages when extending **cdist**, for example: | ||||
| 
 | ||||
|         A **type** could, in principal, have a `manifest` and an **explorer** written | ||||
|         in **POSIX shell**, a `gencode-remote` in **Python** | ||||
|         (which could generate code in **POSIX shell**) and a `gencode-local` | ||||
|         in **Perl**  (which could generate code in **Perl**, | ||||
|         or some other language), while you are at it... | ||||
| 
 | ||||
|         Just don't expect to submit such a hodge-podge as a candidate for being | ||||
|         distributed  with **cdist** itself, though... :-) | ||||
|         especially if it turns out to be something that can be acheieved with | ||||
|         reasonable effort in **POSIX shell**. | ||||
| 
 | ||||
|         In practise, you would at least want to enforce some consistency, if anything for | ||||
|         code maintainibility and your own sanity, in addition to the | ||||
|         the `CAVEATS`_ mentioned down below. | ||||
| 
 | ||||
| .. raw:: html | ||||
| 
 | ||||
|     </details> | ||||
|     <br/> | ||||
| 
 | ||||
| Needless to say, just because you *can* do something, | ||||
| doesn't mean you *should* be doing it, or it's even a *good idea* to do so. | ||||
| 
 | ||||
| As a general rule of thumb, when extending **cdist**, | ||||
| there are many good reasons in favor of sticking with the **POSIX shell** | ||||
| wherever you can, and very few in favor of opting for some other | ||||
| scripting language. | ||||
| 
 | ||||
| This is particularly true for any code that is meant to be run *remotely* | ||||
| on **target hosts** (such as **explorers**), | ||||
| where it is usually important to keep assumptions and requirements/dependencies | ||||
| to a bare minimum. See the  `CAVEATS`_ down below. | ||||
| 
 | ||||
| That being said, **polyglot** capabilities of **cdist** can come | ||||
| quite handy for when you really need this sort of thing, | ||||
| provided that you are ready to bare the consequences, | ||||
| including the burden of extra dependecies | ||||
| --- which is usually not that hard for code run *locally* on **master** | ||||
| (`manifests`, `gencode-*` scripts, and code generated by `gencode-local`). | ||||
| 
 | ||||
| In any case, the mere fact of knowing we *can* escape the POSIX hatch | ||||
| if we really have to, can be quite comforting for those of us suffering | ||||
| from POSIX claustrophobia... which *is* of course a real health hazard | ||||
| associated with high anxiety levels and all, | ||||
| in case you didn't already know... ;-) | ||||
| 
 | ||||
| 
 | ||||
| Writing polyglot extensions for **cdist** | ||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||||
| 
 | ||||
| Whatever the kind of script (`manifest`, explorer, ...) you are writing, | ||||
| you need to ensure that all 3 conditions below are met: | ||||
| 
 | ||||
| 1.  your script starts with an appropriate **shebang** line, such as:: | ||||
| 
 | ||||
|       #!/usr/bin/env bash | ||||
| 
 | ||||
|     .. comment: It would have been nice to make use of an extension | ||||
|         (such as `"sphinx_design"`) which provides a `.. dropdown::` | ||||
|         directive (for toggling visibility) which is the reason for | ||||
|         the ugly `.. raw:: html` stuff below... | ||||
| 
 | ||||
|     .. raw:: html | ||||
| 
 | ||||
|         <details> | ||||
|         <summary><a>It's usually preferable to rely on the <b>env</b> program...</a></summary> | ||||
| 
 | ||||
|     .. container:: | ||||
| 
 | ||||
|         It's usually preferable to rely on the :program:`env` program, | ||||
|         like in the example above, to find the interpreter by searching the PATH. | ||||
| 
 | ||||
|         The :program:`env` program is almost guaranteed to exist even on a rudimentary | ||||
|         UNIX/Linux system at a pretty stable location: `/usr/bin/env` | ||||
| 
 | ||||
|         It is, of course, also possible to write down a **hard coded** path | ||||
|         for the interpreter, if you are certain that it will always be | ||||
|         located at that location, like so:: | ||||
| 
 | ||||
|             #!/bin/bash | ||||
| 
 | ||||
|         This may sometimes be desirable, for example when you want to ascertain | ||||
|         using a specific version of an interpreter or when you are unsure about | ||||
|         what might get foundthrough the PATH. | ||||
| 
 | ||||
|     .. raw:: html | ||||
| 
 | ||||
|         </details> | ||||
| 
 | ||||
| 2.  your script has "*execute*" permissions set (in the Unix/Linux sense), | ||||
|     like so:: | ||||
| 
 | ||||
|         chmod a+x /path/to/your/script | ||||
| 
 | ||||
|     This is essentially what matters to **cdist**, which it will take as a | ||||
|     clue for invoking your script *directly* (instead of passing it | ||||
|     to a shell as an argument). | ||||
| 
 | ||||
|     For **generated code**, `cdist` will automatically take care of setting | ||||
|     *execute* permissions for you, | ||||
|     based on the presence of a leading **shebang** within the generated code. | ||||
| 
 | ||||
| 3.  the **interpreter** referenced by the **shebang** is available on any host(s) | ||||
|     where your code will run. | ||||
| 
 | ||||
| .. raw:: html | ||||
| 
 | ||||
|     <details> | ||||
|     <summary> | ||||
|         <a> | ||||
|         Even for the <b>POSIX shell</b>, | ||||
|         it is still recommended to <b>follow the same guidelines</b> outlined above. | ||||
|         </a> | ||||
|     </summary> | ||||
| 
 | ||||
| .. note:: | ||||
| 
 | ||||
|     Even if you are just writing for the **POSIX shell**, | ||||
|     it is still recommended to follow the same guidelines outlined above. | ||||
| 
 | ||||
|     At the very least, make sure your script has a proper **shebang**. | ||||
| 
 | ||||
|     -   If you have been following the usual **cdist** advise: | ||||
|             you probably already have a proper **shebang** at the very beginning | ||||
|             of your POSIX shell scripts. | ||||
| 
 | ||||
| 
 | ||||
|     -   If (and *only* if), your POSIX shell script *does* contain a proper **shebang**: | ||||
|             you are also encouraged to also give it *"execute"* permissions, | ||||
|             so that your **shebang** will actually get honored. | ||||
| 
 | ||||
| .. raw:: html | ||||
| 
 | ||||
|     </details> | ||||
|     <br/> | ||||
| 
 | ||||
| 
 | ||||
| That's pretty much it... except... | ||||
| 
 | ||||
| .. seealso:: The `CAVEATS`_ below. | ||||
| 
 | ||||
| 
 | ||||
| CAVEATS | ||||
| ^^^^^^^^^^^^ | ||||
| 
 | ||||
| Shebang and execute permissions | ||||
| """"""""""""""""""""""""""""""""" | ||||
| In general, the first two conditions above are trivial to satisfy: | ||||
| Just make sure you put in a **shebang** and mark your script as *executable*. | ||||
| 
 | ||||
| 
 | ||||
| **Beware**, however, that: | ||||
| 
 | ||||
| .. attention:: | ||||
| 
 | ||||
|     -   If your script lacks `execute` permissions (regardless of any **shebang**): | ||||
|             **cdist** will end up passing your script to `/bin/sh -e` | ||||
|             (or to `local_shell` / `remote_shell`, | ||||
|             if one is configured for the current context), | ||||
|             which may or may not be what you want. | ||||
| 
 | ||||
|     -   If your script *does* have `execute` permissions but *lacks* a **shebang**: | ||||
|             you can no longer be sure which interpreter (if any) will end up running your script. | ||||
| 
 | ||||
|             What is certain, on the other hand, is that there is a wide range of | ||||
|             different things that could happen in such a case, depending on the OS and the chain | ||||
|             of execution up to that point... | ||||
| 
 | ||||
|             It is possible (but not certain) that, in such a case, your script may | ||||
|             end up getting fed into `/bin/sh` or the default shell | ||||
|             (whatever it happens to be for the current user). | ||||
| 
 | ||||
|             There's even a legend according to which even `csh` may get a chance to feed | ||||
|             on your script, and then proceed to burning your barn... | ||||
| 
 | ||||
|             So, don't do that. | ||||
| 
 | ||||
| 
 | ||||
| 
 | ||||
| 
 | ||||
| Interpreter availibility | ||||
| """"""""""""""""""""""""""""""""" | ||||
| 
 | ||||
| For the last condition (interpreter availability), | ||||
| your mileage may vary for languages other than the **POSIX shell**. | ||||
| 
 | ||||
| - For scripts meant to be run *locally* on the **master**, things remain relatively easy : | ||||
|     All you may need, if anything, | ||||
|     is a one time installation of stuff. | ||||
| 
 | ||||
|     So, things should be realtively easy when it comes to: :file:`manifest` and :file:`gencode-*` scripts themselves, as well as any code generated by :file:`gencode-local`. | ||||
| 
 | ||||
| 
 | ||||
| - For scripts meant to be run *remotely* on **target hosts**, things might get quite tricky, | ||||
|     depending on how likely it is | ||||
|     for the desired **interpreter** to be installed by default | ||||
|     on the **target system**. | ||||
| 
 | ||||
|     This is an important concern for :file:`explorer` scripts | ||||
|     and any code generated by :file:`gencode-remote`. | ||||
| 
 | ||||
|     .. warning:: | ||||
| 
 | ||||
|         Apart from the POSIX shell (`/bin/sh`), there aren't many interpreters out | ||||
|         there that are likely to have a guaranteed presence on a pristine system. | ||||
| 
 | ||||
|         At the very least, you would have to make sure that the required interpreter | ||||
|         (and any extra modules/libraries your script might depend on) | ||||
|         are indeed available on those host(s) | ||||
|         before your script is invoked... | ||||
|         which kind of goes against the near-zero-dependency philosphy embraced | ||||
|         by **cdist**. | ||||
| 
 | ||||
|         Depending on the target host OS, you might get lucky with | ||||
|         `bash`, `perl`, or `python` being preinstalled. | ||||
|         Even then, those may not necessarily be the version you expect | ||||
|         or have the extra modules/libraries your script might require. | ||||
| 
 | ||||
|         **You have been warned.** | ||||
| 
 | ||||
| 
 | ||||
| More details | ||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||||
| 
 | ||||
| As mentioned earlier, **cdist** itself mostly cares about the script | ||||
| being marked as an *executable*, which it will take as a clue for invoking | ||||
| that script *directly* (instead of passing it to a shell as an argument). | ||||
| 
 | ||||
| The **shebang** magic is handled by the usual process `exec` mechanisms | ||||
| of the host OS (where the script is invoked) that will take over from | ||||
| that point on. | ||||
| 
 | ||||
| 
 | ||||
| Here is a simplified summary : | ||||
| 
 | ||||
| +-------------+---------------+------------------------------+--------------+--------------------------------------------------------+ | ||||
| | executable? | shebang       | invocation resembles         | interpreter  | remarks                                                | | ||||
| +=============+===============+==============================+==============+========================================================+ | ||||
| | yes         | `#!/bin/sh`   | `/path/to/script`            | `/bin/sh`    | shebang **honored** by OS                              | | ||||
| +-------------+---------------+------------------------------+--------------+--------------------------------------------------------+ | ||||
| | yes         | `#!/bin/bash` | `/path/to/script`            | `/bin/bash`  | shebang **honored** by OS                              | | ||||
| +-------------+---------------+------------------------------+--------------+--------------------------------------------------------+ | ||||
| | yes         |               | `/path/to/script`            | *uncertain*  | shebang **absent**                                     | | ||||
| +-------------+---------------+------------------------------+--------------+--------------------------------------------------------+ | ||||
| | no          | `#!/bin/sh`   | `/bin/sh -e /path/to/script` | `/bin/sh -e` | shebang **irrelevant** (as script is not "executable") | | ||||
| +-------------+---------------+------------------------------+--------------+--------------------------------------------------------+ | ||||
| | no          | `#!/bin/bash` | `/bin/sh -e /path/to/script` | `/bin/sh -e` | shebang **irrelevant** (as script is not "executable") | | ||||
| +-------------+---------------+------------------------------+--------------+--------------------------------------------------------+ | ||||
| | no          |               | `/bin/sh -e /path/to/script` | `/bin/sh -e` | shebang **irrelevant** (as script is not "executable") | | ||||
| +-------------+---------------+------------------------------+--------------+--------------------------------------------------------+ | ||||
| 
 | ||||
| In fact, it's a little bit more involved than the above. Remember: | ||||
| 
 | ||||
| - As a special case, for any **generated code** (output by `gencode-*` scripts), | ||||
|   **cdist** will solely rely on the presence (or absence) of a leading **shebang**, | ||||
|   and set the executable bits accordingly, for obvious reasons. | ||||
| 
 | ||||
| - In the end, if a script is NOT marked as "executable", | ||||
|   it will simply be passed as an argument to the configured shell | ||||
|   that corresponds to the relevant context (i.e. `local_shell` or `remote_shell`), | ||||
|   if one is defined within the **cdist** configuration, | ||||
|   or else to `/bin/sh -e`, as a fallback in in both cases. | ||||
| 
 | ||||
| Well, there are also some gory implementation details | ||||
| (related to how environment variables get propagated), | ||||
| but those should normally have no relevance to this discussion. | ||||
| 
 | ||||
| 
 | ||||
| The API between **cdist** and any polyglot extensions | ||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||||
| 
 | ||||
| Conceptually, the API, based on well-known UNIX constructs, | ||||
| remains exactly the same as it is for | ||||
| any extension written for the **POSIX shell**. | ||||
| 
 | ||||
| Basically, you are all set as long as your scripting language is capable of: | ||||
| 
 | ||||
| - accessing **environment variables**; | ||||
| - reading from and writing to the **filesystem** (files, directories, ...); | ||||
| - reading from :file:`STDIN` and writing to :file:`STDOUT` (and eventually to :file:`STDERR`) | ||||
| - **executing** other programs/commands; | ||||
| - **exiting** with an appropriate **status code** (where 0=>success). | ||||
| 
 | ||||
| For all we know, no serious scripting language out there | ||||
| would be missing any such basics. | ||||
| 
 | ||||
| The actual syntax and mechanisms will obviously be different, | ||||
| the shell idioms usually being much more concise for this sort of thing, | ||||
| as expected. | ||||
| 
 | ||||
| See the below example entitled "`Interacting with the cdist API`_". | ||||
| 
 | ||||
| 
 | ||||
| Examples | ||||
| ------------------- | ||||
| 
 | ||||
| Interacting with the cdist API | ||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||||
| 
 | ||||
| As an API example, here's an excerpt from a **cdist** `type manifest`, | ||||
| written for the POSIX shell, showing how one would get at the name | ||||
| of the kernel on the **target host**::: | ||||
| 
 | ||||
|     kernel_name=$(cat "${__global}/explorer/kernel_name") | ||||
| 
 | ||||
|     # ... do something with kernel_name ... | ||||
| 
 | ||||
| 
 | ||||
| In a nutshell, the above snippet gives the general idea about the cdist API: | ||||
| 
 | ||||
| Basically, we are stuffing a shell variable with the contents of a file... | ||||
| which happens to contain the output from the `kernel_name` explorer... | ||||
| 
 | ||||
| Before invoking our `manifest` script,  **cdist** would have, among other things, | ||||
| run all **global explorers** on the **target host**, | ||||
| collected and copied their outputs under a temporary directory on the **master**, and | ||||
| set a specific environment variable (`$__global`) | ||||
| to the path of a specifc subdirectory of that temporary working area. | ||||
| 
 | ||||
| At this point, that file (which contains the kernel name) is sitting there, | ||||
| ready to be slurped... which can obviously be done from any language | ||||
| that can access environment variables and read files from the filesystem... | ||||
| 
 | ||||
| Here's how you could do the same thing in **Python**: | ||||
| 
 | ||||
| .. code-block:: python | ||||
| 
 | ||||
|     #!/usr/bin/env python | ||||
| 
 | ||||
|     import os | ||||
| 
 | ||||
|     def read_file(path): | ||||
|         content = "" | ||||
|         try: | ||||
|             with open(path, "r") as fd: | ||||
|                 content = fd.read().rstrip('\n') | ||||
|         except EnvironmentError: | ||||
|             pass | ||||
|         return content | ||||
| 
 | ||||
|     kernel_name = read_file( os.environ['__global'] + '/explorer/kernel_name' ) | ||||
| 
 | ||||
|     # ... do something with kernel_name ... | ||||
| 
 | ||||
| 
 | ||||
| And in **Perl**, it could look like: | ||||
| 
 | ||||
| .. code-block:: perl | ||||
| 
 | ||||
|     #!/usr/bin/env perl | ||||
| 
 | ||||
|     sub read_file { | ||||
|         my ($path) = @_; | ||||
|         return unless open( my $fh, $path ); | ||||
|         local ($/); | ||||
|         <$fh> | ||||
|     } | ||||
| 
 | ||||
|     my $kernel_name = read_file("$ENV{__global}/explorer/kernel_name"); | ||||
| 
 | ||||
|     # ... do something with kernel_name ... | ||||
| 
 | ||||
| 
 | ||||
| Incidently, this example also helps appreciate some aspects of programming | ||||
| for the shell... which were designed for this sort of thing in the first place... | ||||
| 
 | ||||
| A polygot type explorer (in Perl) | ||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||||
| 
 | ||||
| Here's an imaginary type explorer written in **Perl**, | ||||
| that ouputs the version of the perl interpreter running on the target host: | ||||
| 
 | ||||
| .. code-block:: perl | ||||
| 
 | ||||
|     #!/usr/bin/env perl | ||||
| 
 | ||||
|     use English; | ||||
| 
 | ||||
|     print "${PERL_VERSION}\n"; | ||||
| 
 | ||||
| If the path to the intended interpreter can be ascertained, you can | ||||
| put that down directly on the **shebang**, like so:: | ||||
| 
 | ||||
|      #!/usr/bin/perl | ||||
| 
 | ||||
| However, more often than not, you would want to rely | ||||
| on the `env` program (`/usr/bin/env`) to | ||||
| invoke the first interpreter with the given name (`perl`, in this case) | ||||
| found on the current PATH, like in the above example. | ||||
| 
 | ||||
| Don't forget to set *execute* permissions on the script file::: | ||||
| 
 | ||||
|     chmod a+x ... | ||||
| 
 | ||||
| Or else **cdist** will feed it to a shell instance... | ||||
| which may burn your barn... :-) | ||||
|  | @ -98,29 +98,120 @@ If 'deprecated' marker has no content then general message is printed, e.g.: | |||
| 
 | ||||
| How to write a new type | ||||
| ----------------------- | ||||
| A type consists of | ||||
| 
 | ||||
| - parameter    (optional) | ||||
| - manifest     (optional) | ||||
| - singleton    (optional) | ||||
| - explorer     (optional) | ||||
| - gencode      (optional) | ||||
| - nonparallel  (optional) | ||||
| 
 | ||||
| Types are stored below cdist/conf/type/. Their name should always be prefixed with | ||||
| two underscores (__) to prevent collisions with other executables in $PATH. | ||||
| two underscores (__) to prevent collisions with other executables in :code:`$PATH`. | ||||
| 
 | ||||
| To implement a new type, create the directory **cdist/conf/type/__NAME**. | ||||
| To implement a new type, create the directory :file:`cdist/conf/type/{__NAME}`, | ||||
| either manually or using the helper script `cdist-new-type <man1/cdist-new-type.html>`_ | ||||
| which will also create the basic skeleton for you. | ||||
| 
 | ||||
| Type manifest and gencode can be written in any language. They just need to be | ||||
| executable and have a proper shebang. If they are not executable then cdist assumes | ||||
| they are written in shell so they are executed using '/bin/sh -e' or 'CDIST_LOCAL_SHELL'. | ||||
| A type consists of the following elements (all of which are currently *optional*): | ||||
| 
 | ||||
| For executable shell code it is suggested that shebang is '#!/bin/sh -e'. | ||||
| * some **markers** in the form of **plain files** within the type's directory: | ||||
| 
 | ||||
| For creating type skeleton you can use helper script | ||||
| `cdist-new-type <man1/cdist-new-type.html>`_. | ||||
|     .. list-table:: | ||||
| 
 | ||||
|         * - :file:`singleton` | ||||
|           - *(optional)* | ||||
|           - A type flagged as a :file:`singleton` may be used **only | ||||
|             once per host**, which is useful for types that can be used only once on a | ||||
|             system. | ||||
| 
 | ||||
|             .. raw:: html | ||||
| 
 | ||||
|                 <br/> | ||||
| 
 | ||||
|             Singleton types do not take an object name as argument. | ||||
| 
 | ||||
|         * - :file:`nonparallel` | ||||
|           - (optional) | ||||
|           - Objects of a type flagged as :file:`nonparallel` cannot be run in parallel | ||||
|             when using :code:`-j` option. | ||||
| 
 | ||||
|             .. raw:: html | ||||
| 
 | ||||
|                 <br/> | ||||
| 
 | ||||
| 
 | ||||
|             An example of such a type is :program:`__package_dpkg` type | ||||
|             where :program:`dpkg` itself prevents to be run in more than one instance. | ||||
| 
 | ||||
|         * - :file:`install` | ||||
|           - *(optional)* | ||||
|           - A type flagged as :file:`install` is used only with :command:`install` command. | ||||
|             With other :program:`cdist` commands, i.e. :command:`config`, such types are skipped if used. | ||||
| 
 | ||||
|         * - :file:`deprecated` | ||||
|           - *(optional)* | ||||
|           - A type flagged as :file:`deprecated` causes | ||||
|             :program:`cdist` to emit a **warning** whenever that type is used. | ||||
| 
 | ||||
|             .. raw:: html | ||||
| 
 | ||||
|                 <br/> | ||||
| 
 | ||||
|             If the file that corresponds to the `deprecated` marker has any content, | ||||
|             then this is used as a custom **deprecation message** for the warning. | ||||
| 
 | ||||
| * some more **metadata**: | ||||
| 
 | ||||
|     .. list-table:: | ||||
| 
 | ||||
|         * - :file:`parameter/\*` | ||||
|           - *(optional)* | ||||
|           - A type may have **parameters**. These must be declared following a simple convention described in `Defining parameters`_, which | ||||
|             permits specifying additional properties for each parameter: | ||||
| 
 | ||||
|                 * required or optional | ||||
|                 * single-value or multi-value | ||||
|                 * string or boolean | ||||
| 
 | ||||
|             It is also possible to give a `default` value for each optional parameter. | ||||
| 
 | ||||
| * and some **code** (scripts): | ||||
| 
 | ||||
|     .. list-table:: | ||||
| 
 | ||||
|         * - :file:`manifest` | ||||
|           - *(optional)* | ||||
|           - :doc:`Type manifest <cdist-manifest>` | ||||
| 
 | ||||
|         * - :file:`explorer/*` | ||||
|           - *(optional)* | ||||
|           - Any number of :doc:`type explorer <cdist-explorer>` scripts may exist under :file:`explorer` subdirectory. | ||||
| 
 | ||||
|         * - :file:`gencode-local` | ||||
|           - *(optional)* | ||||
|           - A script that generates code to be executed *locally* (on master). | ||||
| 
 | ||||
|         * - :file:`gencode-remote` | ||||
|           - *(optional)* | ||||
|           - A script that generates code to be executed *remotely* (on target host). | ||||
| 
 | ||||
| 
 | ||||
| .. tip:: | ||||
| 
 | ||||
|     Each of the above-mentioned scripts can be written in **any scripting language**, | ||||
|     provided that the script is executable and has a proper **shebang**. | ||||
| 
 | ||||
|     For executable shell code, the recommended shebang is :code:`#!/bin/sh -e`. | ||||
| 
 | ||||
|     If a script lacks `execute` permissions,  :program:`cdist` assumes | ||||
|     it to be written in **shell** and executes it using | ||||
|     `$CDIST_LOCAL_SHELL` or `$CDIST_REMOTE_SHELL`, if one is defined | ||||
|     for the current execution context (*local* or *remote*), | ||||
|     or else falling back to :code:`/bin/sh -e`. | ||||
| 
 | ||||
| 
 | ||||
|     For any code susceptible to run on remote target hosts | ||||
|     (i.e. **explorers** and any code generated by :code:`gencode-remote`), | ||||
|     it is recommended to stick to **POSIX shell** | ||||
|     in order to minimize requirements on target hosts where they would need to be executed. | ||||
| 
 | ||||
|     For more details and examples, see :doc:`cdist-polyglot`. | ||||
| 
 | ||||
| .. seealso:: `cdist execution stages <cdist-stages.html>`_ | ||||
| 
 | ||||
| Defining parameters | ||||
| ------------------- | ||||
|  | @ -307,6 +398,7 @@ stdin from */dev/null*: | |||
|         done < "$__object/parameter/foo" | ||||
|     fi | ||||
| 
 | ||||
| .. _cdist-type#manifest: | ||||
| 
 | ||||
| Writing the manifest | ||||
| -------------------- | ||||
|  | @ -380,6 +472,7 @@ in your type directory: | |||
| 
 | ||||
| For example, package types are nonparallel types. | ||||
| 
 | ||||
| .. _cdist-type#explorers: | ||||
| 
 | ||||
| The type explorers | ||||
| ------------------ | ||||
|  | @ -402,6 +495,7 @@ client, like this (shortened version from the type __file): | |||
|        md5sum < "$destination" | ||||
|     fi | ||||
| 
 | ||||
| .. _cdist-type#gencode-scripts: | ||||
| 
 | ||||
| Writing the gencode script | ||||
| -------------------------- | ||||
|  |  | |||
|  | @ -4,44 +4,65 @@ Why should I use cdist? | |||
| There are several motivations to use cdist, these | ||||
| are probably the most popular ones. | ||||
| 
 | ||||
| Known language | ||||
| -------------- | ||||
| No need to learn a new language | ||||
| ------------------------------- | ||||
| 
 | ||||
| Cdist is being configured in | ||||
| `shell script <https://en.wikipedia.org/wiki/Shell_script>`_. | ||||
| Shell script is used by UNIX system engineers for decades. | ||||
| So when cdist is introduced, your staff does not need to learn a new | ||||
| When adopting cdist, your staff does not need to learn a new | ||||
| `DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_ | ||||
| or programming language. | ||||
| or programming language, as cdist can be configured | ||||
| and extended in **any scripting language**, the recommended one | ||||
| being `shell scripts <https://en.wikipedia.org/wiki/Shell_script>`_. | ||||
| 
 | ||||
| Shell scripts enjoy ubiquity: they have been widely used by UNIX system engineers | ||||
| for decades, and a suitable interpreter (:code:`/bin/sh`) is all but | ||||
| guaranteed to be widely available on target hosts. | ||||
| 
 | ||||
| 
 | ||||
| Easy idempotance -- without having to give up control | ||||
| ----------------------------------------------------------------------------- | ||||
| 
 | ||||
| For the sake of `idempotence <https://en.wikipedia.org/wiki/Idempotence>`_, many **contemporary SCMs**  choose to ditch the power and versatality of general purpose programming languages, and adopt some form of | ||||
| declarative `DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_ for describing the desired end states on target systems. | ||||
| 
 | ||||
| :program:`Cdist` takes a quite different approach, enabling *both* `idempotence <https://en.wikipedia.org/wiki/Idempotence>`_ *and* a decent level of programming power. | ||||
| 
 | ||||
| Unlike other SCMs, :program:`cdist` allows you to use a general purpose scripting language (POSIX shell is recommended) for describing the desired end states on target systems, instead of some declarative `DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_. | ||||
| 
 | ||||
| Unlike regular scripting, however, you are not left on your own for ensuring `idempotence <https://en.wikipedia.org/wiki/Idempotence>`_. :program:`Cdist` makes this really easy. | ||||
| 
 | ||||
| It does not matter how many times you "invoke" **cdist types** and in which order: :program:`cdist` will ensure that the actual code associated with each type will be executed only once (in dependency order) which, in turn, may effectively end up becoming a no-op, if the actual state is already the same as the desired one. | ||||
| 
 | ||||
| .. TODO: It would be great if there were an "architectural overview" page which could be referenced from here. | ||||
| 
 | ||||
| 
 | ||||
| Powerful language | ||||
| ----------------- | ||||
| -------------------- | ||||
| 
 | ||||
| Not only is shell scripting widely known by system engineers, | ||||
| but it is also a very powerful language. Here are some features | ||||
| which make daily work easy: | ||||
| Compared to a typical `DSL <https://en.wikipedia.org/wiki/Domain-specific_language>`_, | ||||
| shell scripts feature a much more powerful language. | ||||
| Here are some features which make daily work easy: | ||||
| 
 | ||||
|  * Configuration can react dynamically on explored values | ||||
|  * Ability to dynamically adapt configuration based on information | ||||
|    *explored* from target hosts; | ||||
|  * High level string manipulation (using sed, awk, grep) | ||||
|  * Conditional support (**if, case**) | ||||
|  * Loop support (**for, while**) | ||||
|  * Support for dependencies between cdist types | ||||
|  * Variable expansion | ||||
|  * Support for dependencies between cdist types and objects | ||||
| 
 | ||||
| If and when needed, it's always possible to simply | ||||
| make use of **any other scripting language** at your disposal | ||||
| *(albeit at the expense of adding a dependency on the corresponding interpreter | ||||
| and libraries)*. | ||||
| 
 | ||||
| More than shell scripting | ||||
| ------------------------- | ||||
| 
 | ||||
| If you compare regular shell scripting with cdist, there is one major | ||||
| difference: When using cdist types, | ||||
| the results are | ||||
| `idempotent <https://en.wikipedia.org/wiki/Idempotence>`_. | ||||
| In practise that means it does not matter in which order you | ||||
| call cdist types, the result is always the same. | ||||
| 
 | ||||
| Zero dependency configuration management | ||||
| ---------------------------------------- | ||||
| ----------------------------------------- | ||||
| 
 | ||||
| Cdist requires very little on a target system. Even better, | ||||
| in almost all cases all dependencies are usually fulfilled. | ||||
| in almost all cases all dependencies are usually already | ||||
| fulfilled. | ||||
| Cdist does not require an agent or high level programming | ||||
| languages on the target host: it will run on any host that | ||||
| has a **ssh server running** and a POSIX compatible shell | ||||
|  |  | |||
|  | @ -30,6 +30,7 @@ It natively supports IPv6 since the first release. | |||
|    cdist-type | ||||
|    cdist-types | ||||
|    cdist-explorer | ||||
|    cdist-polyglot | ||||
|    cdist-messaging | ||||
|    cdist-parallelization | ||||
|    cdist-inventory | ||||
|  |  | |||
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue