Parallelization
===============

Description
-----------
cdist has two modes of parallel operation.

One of them is to operate on each host in separate process. This is enabled
with :strong:`-p/--parallel` option.

The other way is to operate in parallel within one host where you specify
the number of jobs. This is enabled with :strong:`-j/--jobs` option where you
can specify the number of parallel jobs. By default,
:strong:`multiprocessing.cpu_count()` is used. For this mode global explorers,
object preparation and object run are supported and this option is still in
:strong:`beta`.

You can, of course, use those two options together. This means that each host
will be processed by its own process. Within each process cdist will operate
using specified number of parallel jobs.

For more info on those options see :strong:`cdist`\ (1).


Examples
--------

.. code-block:: sh

    # Configure hosts read from file hosts.file in parallel
    $ cdist config -p -f hosts.file

    # Configure hosts read from file hosts.file sequentially but using default
    # number of parallel jobs
    $ cdist config -b -j -f hosts.file

    # Configure hosts read from file hosts.file in parallel using 16
    # parallel jobs
    $ cdist config -b -j 16 -p -f hosts.file


Caveats
-------
When operating in parallel, either by operating in parallel for each host
(-p/--parallel) or by parallel jobs within a host (-j/--jobs), and depending
on target SSH server and its configuration you may encounter connection drops.
This is controlled with sshd :strong:MaxStartups configuration options.
You may also encounter session open refusal. This happens with ssh multiplexing
when you reach maximum number of open sessions permitted per network 
connection. In this case ssh will disable multiplexing.
This limit is controlled with sshd :strong:MaxSessions configuration
options. For more details refer to :strong:`sshd_config`\ (5).

For example, if you reach :strong:`MaxSessions` sessions you may get the
following output:

.. code-block:: sh

    $ cdist config -b -j 11 -v 78.47.116.244
    INFO: cdist: version 4.2.2-55-g640b7f9
    INFO: 78.47.116.244: Running global explorers
    INFO: 78.47.116.244: Remote transfer in 11 parallel jobs
    channel 22: open failed: administratively prohibited: open failed
    mux_client_request_session: session request failed: Session open refused by peer
    ControlSocket /tmp/tmpuah6fw_t/d886d4b7e4425a102a54bfaff4d2288b/ssh-control-path already exists, disabling multiplexing
    INFO: 78.47.116.244: Running global explorers in 11 parallel jobs
    channel 22: open failed: administratively prohibited: open failed
    mux_client_request_session: session request failed: Session open refused by peer
    ControlSocket /tmp/tmpuah6fw_t/d886d4b7e4425a102a54bfaff4d2288b/ssh-control-path already exists, disabling multiplexing
    INFO: 78.47.116.244: Running initial manifest /tmp/tmpuah6fw_t/d886d4b7e4425a102a54bfaff4d2288b/data/conf/manifest/init
    INFO: 78.47.116.244: Running manifest and explorers for __file/root/host.file
    INFO: 78.47.116.244: Generating code for __file/root/host.file
    INFO: 78.47.116.244: Finished successful run in 18.655028820037842 seconds
    INFO: cdist: Total processing time for 1 host(s): 19.159148693084717