ungleich-public/cdist
ungleich-public/cdist
7
4
Fork 10
Code Issues 58 Pull requests 15 Projects Releases Wiki Activity

Order of execution #311

New issue
Closed
opened 2021-11-20 15:24:05 +00:00 by ungleich-gitea · 5 comments
ungleich-gitea commented 2021-11-20 15:24:05 +00:00
Owner
Copy link

Created by: jdguffey

Currently, cdist's order of execution is:
global explorers
the initial manifest
followed by object manifests
type explorers
generate and execute code locally and remotely

This can create issues if, for example, a __package provides a directory, but needs different permissions set on it after creation, so the manifest contains a __directory line that will try to mkdir on the remote machine after the __package (because otherwise the package would overwrite the permissions of the newly-created directory). This, in turn, produces an error and the manifest fails. If each type explorer was run at the same time as the gencodes for the same type, I believe that it would fix this. That is, rather than executing the explorers for __directory and __package, followed by executing the gencodes associated with those types, execute the explorer for __package, then the gencode, then the explorer for __directory, then the gencode.

The example I give is trite, I understand, but it is still a very real issue that raises issues in my manifests (and I'm sure others' as well) whenever I do anything non-trivial because of types providing things for other types (another example would be to have a __directory /usr/jail and a __jail ) because when the explorers run, neither type has provided the target yet so they both decide that they need to, but after the first one executes, the other will always fail.

*Created by: jdguffey* Currently, cdist's order of execution is: global explorers the initial manifest followed by object manifests type explorers generate and execute code locally and remotely This can create issues if, for example, a __package provides a directory, but needs different permissions set on it after creation, so the manifest contains a __directory line that will try to mkdir on the remote machine after the __package (because otherwise the package would overwrite the permissions of the newly-created directory). This, in turn, produces an error and the manifest fails. If each type explorer was run at the same time as the gencodes for the same type, I believe that it would fix this. That is, rather than executing the explorers for __directory and __package, followed by executing the gencodes associated with those types, execute the explorer for __package, then the gencode, then the explorer for __directory, then the gencode. The example I give is trite, I understand, but it is still a very real issue that raises issues in my manifests (and I'm sure others' as well) whenever I do anything non-trivial because of types providing things for other types (another example would be to have a __directory /usr/jail and a __jail <something>) because when the explorers run, neither type has provided the target yet so they both decide that they need to, but after the first one executes, the other will always fail.
ungleich-gitea added this to the future milestone 2021-11-20 15:24:05 +00:00
ungleich-gitea added the
feature
 label 2021-11-20 15:24:05 +00:00
ungleich-gitea commented 2021-11-20 15:52:49 +00:00
Author
Owner
Copy link

Created by: telmich

Fixed in the master branch

*Created by: telmich* Fixed in the master branch
ungleich-gitea commented 2021-11-20 15:52:50 +00:00
Author
Owner
Copy link

Created by: telmich

Postponing this, as the 2.1 release is coming up soon and we probably need to agree on the right here.

*Created by: telmich* Postponing this, as the 2.1 release is coming up soon and we probably need to agree on the right here.
ungleich-gitea commented 2021-11-20 15:52:51 +00:00
Author
Owner
Copy link

Created by: telmich

Assuming there are 2 explorer runs, we also have two (probably different) outputs of explorers.

We could either overwrite the output of the first run ("updating") or create new files with the output of the 2nd run.

*Created by: telmich* Assuming there are 2 explorer runs, we also have two (probably different) outputs of explorers. We could either overwrite the output of the first run ("updating") or create new files with the output of the 2nd run.
ungleich-gitea commented 2021-11-20 15:52:53 +00:00
Author
Owner
Copy link

Created by: jdguffey

What do you mean by overwrite vs create new with regard to second explorer runs? I'm not sure I understand the distinction.

*Created by: jdguffey* What do you mean by overwrite vs create new with regard to second explorer runs? I'm not sure I understand the distinction.
ungleich-gitea commented 2021-11-20 15:52:54 +00:00
Author
Owner
Copy link

Created by: telmich

Hihi, there's a bug in your description - the correct order is

Currently, cdist's order of execution is:
global explorers
the initial manifest
type explorers << these ones first
followed by object manifests
generate and execute code locally and remotely

I understand the issue and the correct solution is to rerun the type explorers before the gencode-* phases again.

What was open the last time we discussed this is whether the results of the second explorer run should OVERWRITE the previous results or CREATE NEW entries.

*Created by: telmich* Hihi, there's a bug in your description - the correct order is Currently, cdist's order of execution is: global explorers the initial manifest type explorers << these ones first followed by object manifests generate and execute code locally and remotely I understand the issue and the correct solution is to rerun the type explorers _before_ the gencode-\* phases again. What was open the last time we discussed this is whether the results of the second explorer run should OVERWRITE the previous results or CREATE NEW entries.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
7.0
6.9
py3.10
ander/__package_apt_update_index
haproxy-dualstack
ander/__sed
beta
ander/os_version_debian_sid
evilham-compatibility-fixes
ander/__rsync
__snakeoil_cert
ander/update_readme
__download_improvements
feature/onchange
cleanup/string-formatting
feature/type-relationship-graph
cleanup/ssh-auth-keys-types
__letsencrypt_cert-fix-hooks
bugfix/preos-debug
bugfix/in-script-import
6.8
bugfix/sphinx-docs-build
6.7
cherry-pick-2f433a14
bugfix/make-code-consistent
6.6
regain-py3.2-support
6.5
bugfix/multiple-log-lines
matterbridge
coturn
alpinefix
matrix
new-type/network-interface
feature/process
6.4
feature/info-command
feature/libexec
6.3
preos-plugins-dir-opt
gitlab-ci
6.2
order-dep-fix
6.1
6.0
build/support-pip-from-git
feature/shell-lib
5.1
feature/support-type-deprecation
5.0
feature/python-types
4.11
4.10
shellcheck
4.9
4.8
freebsd-improvements
new-prometheus
key_value-onchange
feature/output_streams
AnotherKamila-patch-1
__letsencrypt_cert-fixes
letsencrypt-cron-fix
4.7
newtype-__letsencrypt_cert
os_explorer_devuan_fix
prometheus-exporter-fixes
daemontools-for-fbsd
type-prometheus-exporter-from-master
prometheus-more-fixes
4.6
4.5
fix-j
steven-backport
4.4
prometheus-fixes
grafana_dashboard
prometheus
daemontools
consul_improvements
feature/trigger
4.3
4.0-pre-not-stable
4.2
4.1
4.0
feature_install_and_preos
3.1
no-dot-cdist
random_dot_cdist
feature_yum_url
feature_files_export
3.0
2.3
2.2
2.1
ssh_callback
2.0
archive_shell_function_approach
1.7
1.6
1.5
1.4
1.3
1.2
1.1
1.0
7.0.0
6.9.8
6.9.7
6.9.6
6.9.5
6.9.4
6.9.3
6.9.2
6.9.1
6.9.0
6.8.0
6.7.0
6.6.0
6.5.6
6.5.5
6.5.4
6.5.3
6.5.2
6.5.1
6.5.0
6.4.0
6.3.0
6.2.0
6.1.1
6.1.0
6.0.4
6.0.3
6.0.2
6.0.1
6.0.0
5.1.3
5.1.2
5.1.1
5.1.0
5.0.2
5.0.1
5.0.0
4.11.1
4.11.0
4.10.11
4.10.10
4.10.9
4.10.8
4.10.7
4.10.6
4.10.5
4.10.4
4.10.3
4.10.2
4.10.1
4.10.0
4.9.1
4.9.0
4.8.4
4.8.3
4.8.2
4.8.1
4.8.0
4.7.3
4.7.2
4.7.1
4.7.0
4.6.1
4.6.0
4.5.0
4.4.4
4.4.3
4.4.2
4.4.1
4.4.0
4.3.2
4.3.1
4.3.0
4.2.2
4.2.1
4.2.0
4.1.0
4.0.0
3.1.13
tn1
3.1.12
3.1.11
3.1.10
3.1.9
3.1.8
3.1.7
3.1.6
4.0.0pre3
3.1.5
3.1.4
3.1.3
3.1.2
3.1.1
3.1.0
4.0.0pre2
3.0.9
3.0.8
3.0.7
3.0.6
3.0.5
3.0.4
3.0.3
4.0.0pre1
3.0.2
3.0.1
3.0.0
2.3.7
2.3.6
2.3.5
2.3.4
2.3.3
2.3.2
2.3.1
2.3.0
2.2.0
2.1.2
2.1.1
2.1.0
2.1.0pre8
2.1.0pre7
2.1.0pre6
2.0.15
2.1.0pre5
2.1.0pre4
2.1.0pre3
2.1.0pre2
2.1.0pre1
2.0.14
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.7.1
1.7.0
1.6.2
1.6.1
1.6.0
1.5.0
1.4.1
1.4.0
1.3.2
1.3.1
1.3.0
1.2.0
1.1.0
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.9.9
0.9.8
0.9.7
0.9.6
0.9.5
0.9.4
0.9.3
0.9.2
0.9.1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Labels
Clear labels   
bugfix

   
cleanup

   
discussion

   
documentation

   
doing

   
done

   
feature

   
improvement

   
packaging

   
Stale

   
testing

   
TODO
No labels
bugfix
cleanup
discussion
documentation
doing
done
feature
improvement
packaging
Stale
testing
TODO
Milestone
Clear milestone
No items
No milestone
future
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: ungleich-public/cdist#311
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?