Order of execution #311

New Issue

ungleich-gitea · 2021-11-20T15:24:05Z

ungleich-gitea commented

2021-11-20 15:24:05 +00:00

Created by: jdguffey

Currently, cdist's order of execution is:
global explorers
the initial manifest
followed by object manifests
type explorers
generate and execute code locally and remotely

This can create issues if, for example, a __package provides a directory, but needs different permissions set on it after creation, so the manifest contains a __directory line that will try to mkdir on the remote machine after the __package (because otherwise the package would overwrite the permissions of the newly-created directory). This, in turn, produces an error and the manifest fails. If each type explorer was run at the same time as the gencodes for the same type, I believe that it would fix this. That is, rather than executing the explorers for __directory and __package, followed by executing the gencodes associated with those types, execute the explorer for __package, then the gencode, then the explorer for __directory, then the gencode.

The example I give is trite, I understand, but it is still a very real issue that raises issues in my manifests (and I'm sure others' as well) whenever I do anything non-trivial because of types providing things for other types (another example would be to have a __directory /usr/jail and a __jail ) because when the explorers run, neither type has provided the target yet so they both decide that they need to, but after the first one executes, the other will always fail.

*Created by: jdguffey* Currently, cdist's order of execution is: global explorers the initial manifest followed by object manifests type explorers generate and execute code locally and remotely This can create issues if, for example, a __package provides a directory, but needs different permissions set on it after creation, so the manifest contains a __directory line that will try to mkdir on the remote machine after the __package (because otherwise the package would overwrite the permissions of the newly-created directory). This, in turn, produces an error and the manifest fails. If each type explorer was run at the same time as the gencodes for the same type, I believe that it would fix this. That is, rather than executing the explorers for __directory and __package, followed by executing the gencodes associated with those types, execute the explorer for __package, then the gencode, then the explorer for __directory, then the gencode. The example I give is trite, I understand, but it is still a very real issue that raises issues in my manifests (and I'm sure others' as well) whenever I do anything non-trivial because of types providing things for other types (another example would be to have a __directory /usr/jail and a __jail <something>) because when the explorers run, neither type has provided the target yet so they both decide that they need to, but after the first one executes, the other will always fail.

ungleich-gitea added this to the future milestone 2021-11-20 15:24:05 +00:00

ungleich-gitea added the

feature

label 2021-11-20 15:24:05 +00:00

ungleich-gitea closed this issue

2021-11-20 15:24:05 +00:00

ungleich-gitea commented

2021-11-20 15:52:49 +00:00

Created by: telmich

Fixed in the master branch

*Created by: telmich* Fixed in the master branch

ungleich-gitea commented

2021-11-20 15:52:50 +00:00

Created by: telmich

Postponing this, as the 2.1 release is coming up soon and we probably need to agree on the right here.

*Created by: telmich* Postponing this, as the 2.1 release is coming up soon and we probably need to agree on the right here.

ungleich-gitea commented

2021-11-20 15:52:51 +00:00

Created by: telmich

Assuming there are 2 explorer runs, we also have two (probably different) outputs of explorers.

We could either overwrite the output of the first run ("updating") or create new files with the output of the 2nd run.

*Created by: telmich* Assuming there are 2 explorer runs, we also have two (probably different) outputs of explorers. We could either overwrite the output of the first run ("updating") or create new files with the output of the 2nd run.

ungleich-gitea commented

2021-11-20 15:52:53 +00:00

Created by: jdguffey

What do you mean by overwrite vs create new with regard to second explorer runs? I'm not sure I understand the distinction.

*Created by: jdguffey* What do you mean by overwrite vs create new with regard to second explorer runs? I'm not sure I understand the distinction.

ungleich-gitea commented

2021-11-20 15:52:54 +00:00

Created by: telmich

Hihi, there's a bug in your description - the correct order is

Currently, cdist's order of execution is:
global explorers
the initial manifest
type explorers << these ones first
followed by object manifests
generate and execute code locally and remotely

I understand the issue and the correct solution is to rerun the type explorers before the gencode-* phases again.

What was open the last time we discussed this is whether the results of the second explorer run should OVERWRITE the previous results or CREATE NEW entries.

*Created by: telmich* Hihi, there's a bug in your description - the correct order is Currently, cdist's order of execution is: global explorers the initial manifest type explorers << these ones first followed by object manifests generate and execute code locally and remotely I understand the issue and the correct solution is to rerun the type explorers _before_ the gencode-\* phases again. What was open the last time we discussed this is whether the results of the second explorer run should OVERWRITE the previous results or CREATE NEW entries.

Sign in to join this conversation.