[DISCUSSION] nonparallel vs nonparallel-self #10

New Issue

ungleich-gitea · 2021-11-20T11:24:46Z

ungleich-gitea commented

2021-11-20 11:24:46 +00:00

Conceptually there are many types that shouldn't be run in parallel with antyhing at all. Something that changes the system's global state.

On the other hand, some (most?) types actually only need to not be run in parallel with another instance of themselves, those that are accessing one well defined resource (or set of resources).

I am unsure which one the current nonparallel implementation meant to address, and whether it would make sense to address the other one.

I'd be delighted to know your thoughts...

Conceptually there are many types that shouldn't be run in parallel with _antyhing_ at all. Something that changes the system's global state. On the other hand, some (most?) types actually only need to not be run in parallel with another instance of themselves, those that are accessing one well defined resource (or set of resources). I am unsure which one the current `nonparallel` implementation meant to address, and whether it would make sense to address the other one. I'd be delighted to know your thoughts...

ungleich-gitea commented

2021-11-20 15:25:14 +00:00

I think you're pokinging a wasps nest, here.

The current nonparallel file is meant to be used with types that modify parts of one resource, e.g.

__package_*: modifies the package database,
__uci: modified the UCI database,
__sshd_config: modifies the "global" sshd_config,
__debconf_set_selections: modifies the debconf database; is currently parallel, but I'm working on a patch.)

But there are many different cases where this does not apply, e.g.:

__cron: modifies crontab, but interlocking is only required if multiple objects touch the same user.
__key_value/__line: interlocking only required if multiple objects touch the same file.
__dpkg_architecture: is nonparallel, but maybe it should also interlock with e.g. __package_apt or __apt_*?
__ssh_authorized_key: only requires interlocking if multiple objects touch the same file.

But IMHO nonparallel is actually even more complex.
There are two issues I see that are non-trivial to fix.

There are many utilities that want to acquire a lock when they are started but terminate if the lock cannot be acquired, instead of waiting for it to be become available.
Even if all cdist objects would be locked by a nonparallel-theworld cdist runs can still fail if e.g. an auto updater cron job launches and starts to upgrade packages at the same time.
Interlocking does not only apply to code-remote (as nonparallel does) but it can also apply to explorers.

EDIT: @fancsali can you make some examples of objects that should not be run with anything else in parallel?

I think you're pokinging a wasps nest, here. The current `nonparallel` file is meant to be used with types that modify parts of one resource, e.g. - `__package_*`: modifies the package database, - `__uci`: modified the UCI database, - `__sshd_config`: modifies the "global" sshd_config, - `__debconf_set_selections`: modifies the debconf database; is currently parallel, but I'm working on a patch.) But there are many different cases where this does not apply, e.g.: - `__cron`: modifies crontab, but interlocking is only required if multiple objects touch the same user. - `__key_value`/`__line`: interlocking only required if multiple objects touch the same file. - `__dpkg_architecture`: is `nonparallel`, but maybe it should also interlock with e.g. `__package_apt` or `__apt_*`? - `__ssh_authorized_key`: only requires interlocking if multiple objects touch the same file. But IMHO `nonparallel` is actually even more complex. There are two issues I see that are non-trivial to fix. * There are many utilities that want to acquire a lock when they are started but terminate if the lock cannot be acquired, instead of waiting for it to be become available. Even if all cdist objects would be locked by a `nonparallel-theworld` cdist runs can still fail if e.g. an auto updater cron job launches and starts to upgrade packages at the same time. * Interlocking does not only apply to `code-remote` (as `nonparallel` does) but it _can_ also apply to explorers. EDIT: @fancsali can you make some examples of objects that should not be run with anything else in parallel?

ungleich-gitea commented

2021-11-20 15:25:16 +00:00

changed the description