ungleich-public/cdist
ungleich-public/cdist
7
4
Fork 10
Code Issues 58 Pull requests 15 Projects Releases Wiki Activity

Circular dependencies with export CDIST_ORDER_DEPENDENCY=1 #99

New issue
Closed
opened 2021-11-20 13:23:53 +00:00 by ungleich-gitea · 2 comments
ungleich-gitea commented 2021-11-20 13:23:53 +00:00
Owner
Copy link

Created by: darko-poljak

In mailing list the following case was reported as not working, resulting in circular dependency.

Init manifest

__foo

__foo type manifest:

export CDIST_ORDER_DEPENDENCY=1

__group ...
__user ...

Error

The requirements of the following objects could
not be resolved:
__user/... requires:
         __group/...
__user/... autorequires:

__group/... requires:
         __foo/...
__group/... autorequires:

__foo/... requires:

__foo/... autorequires:
         __user/...

__user/... requires:
__group/...
is the result of injecting last created type requirement
__group/... requires:
__foo/...
is the result of injecting last created type requirement
__foo/... autorequires:
__user/...
is the result of autorequirement.

foo autorequires group is not set since group requires foo was injected.

*Created by: darko-poljak* In mailing list the following case was reported as not working, resulting in circular dependency. Init manifest ``` __foo ``` __foo type manifest: ``` export CDIST_ORDER_DEPENDENCY=1 __group ... __user ... ``` Error ``` The requirements of the following objects could not be resolved: __user/... requires: __group/... __user/... autorequires: __group/... requires: __foo/... __group/... autorequires: __foo/... requires: __foo/... autorequires: __user/... ``` __user/... requires: __group/... is the result of injecting last created type requirement __group/... requires: __foo/... is the result of injecting last created type requirement __foo/... autorequires: __user/... is the result of autorequirement. foo autorequires group is not set since group requires foo was injected.
ungleich-gitea commented 2021-11-20 15:38:51 +00:00
Author
Owner
Copy link

Created by: darko-poljak

@telmich I will try to clarify the case better.

Let we have the following init manifest:

__foo

and foo type manifest:

export CDIST_ORDER_DEPENDENCY=1

__group ...
__user ...

So this means that we want dependency in type's manifest based on the order of types.
Here we want that __user type requires __group type.

What cdist determines is that

__group/... requires: __foo/...
__user/... requires: __group/...
__foo/... requires:  __user/...

So, __user requires __group, which requires __foo, which requires __user, and we have circle here.

How we came to that dependency tree?

  1. __foo type should depend on all types it uses in its manifest.
    When __foo type is executed, first __group type is executed.
  2. cdist sess that CDIST_ORDER_DEPENDENCY is defined.
    It looks for the previous type created. It finds __foo type and injects it as a requirement.
    And now we have __group/... requires: __foo/....
  3. Next step is auto requirement. cdist tries to define that __foo auto requires __gorup,
    but it sees that __group already requires __foo so it skips this.
  4. Next __user type is executed.
  5. cdist sess that CDIST_ORDER_DEPENDENCY is defined.
    It looks for the previous type created. It finds __group type and injects it as a requirement.
    And now we have __user/... requires: __group/....
  6. Next step is auto requirement. cdist tries to define that __foo auto requires __user,
    it sees that there is no __user requires __foo so it defines:
    __foo/... autorequires:__user/....

In the end we have this:

__group/... requires: __foo/...
__user/... requires: __group/...
__foo/... requires:  __user/...

And this is wrong!

The wrong step is in 2.. cdist should not inject requirement of type which manifest is being executed for a type in type's manifest. In the above case, requirement of __foo for type __group should not be injected!
Note that when CDIST_ORDER_DEPENDENCY in used only in init manifest, this does not happen because for init manifest, when first type is executed, previous type created is None.

*Created by: darko-poljak* @telmich I will try to clarify the case better. Let we have the following init manifest: ``` __foo ``` and foo type manifest: ``` export CDIST_ORDER_DEPENDENCY=1 __group ... __user ... ``` So this means that we want dependency in type's manifest based on the order of types. Here we want that `__user` type requires `__group` type. What cdist determines is that ``` __group/... requires: __foo/... __user/... requires: __group/... __foo/... requires: __user/... ``` So, `__user` requires `__group`, which requires `__foo`, which requires `__user`, and we have circle here. How we came to that dependency tree? 1. `__foo` type should depend on all types it uses in its manifest. When `__foo` type is executed, first `__group` type is executed. 2. cdist sess that `CDIST_ORDER_DEPENDENCY` is defined. It looks for the previous type created. It finds `__foo` type and injects it as a requirement. And now we have `__group/... requires: __foo/...`. 3. Next step is auto requirement. cdist tries to define that `__foo` auto requires `__gorup`, but it sees that `__group` already requires `__foo` so it skips this. 4. Next `__user` type is executed. 5. cdist sess that `CDIST_ORDER_DEPENDENCY` is defined. It looks for the previous type created. It finds `__group` type and injects it as a requirement. And now we have `__user/... requires: __group/...`. 6. Next step is auto requirement. cdist tries to define that `__foo` auto requires `__user`, it sees that there is no `__user` requires `__foo` so it defines: `__foo/... autorequires:__user/...`. In the end we have this: ``` __group/... requires: __foo/... __user/... requires: __group/... __foo/... requires: __user/... ``` And this is wrong! The wrong step is in **2.**. cdist **should not** inject requirement of type which manifest is being executed for a type in type's manifest. In the above case, requirement of `__foo` for type `__group` should not be injected! Note that when `CDIST_ORDER_DEPENDENCY` in used only in init manifest, this does not happen because for init manifest, when first type is executed, previous type created is **None**.
ungleich-gitea commented 2021-11-20 15:38:52 +00:00
Author
Owner
Copy link

Created by: darko-poljak

@telmich @asteven
The wrong part here is
__group/... requires:
__foo/...
injection.

When CDIST_ORDER_REQUIREMENT is set, if last created type is object whose type manifest is
currently being executed, then such requirement should not be injected.

PR follows.

*Created by: darko-poljak* @telmich @asteven The wrong part here is __group/... requires: __foo/... injection. When CDIST_ORDER_REQUIREMENT is set, if last created type is object whose type manifest is currently being executed, then such requirement should not be injected. PR follows.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
7.0
6.9
py3.10
ander/__package_apt_update_index
haproxy-dualstack
ander/__sed
beta
ander/os_version_debian_sid
evilham-compatibility-fixes
ander/__rsync
__snakeoil_cert
ander/update_readme
__download_improvements
feature/onchange
cleanup/string-formatting
feature/type-relationship-graph
cleanup/ssh-auth-keys-types
__letsencrypt_cert-fix-hooks
bugfix/preos-debug
bugfix/in-script-import
6.8
bugfix/sphinx-docs-build
6.7
cherry-pick-2f433a14
bugfix/make-code-consistent
6.6
regain-py3.2-support
6.5
bugfix/multiple-log-lines
matterbridge
coturn
alpinefix
matrix
new-type/network-interface
feature/process
6.4
feature/info-command
feature/libexec
6.3
preos-plugins-dir-opt
gitlab-ci
6.2
order-dep-fix
6.1
6.0
build/support-pip-from-git
feature/shell-lib
5.1
feature/support-type-deprecation
5.0
feature/python-types
4.11
4.10
shellcheck
4.9
4.8
freebsd-improvements
new-prometheus
key_value-onchange
feature/output_streams
AnotherKamila-patch-1
__letsencrypt_cert-fixes
letsencrypt-cron-fix
4.7
newtype-__letsencrypt_cert
os_explorer_devuan_fix
prometheus-exporter-fixes
daemontools-for-fbsd
type-prometheus-exporter-from-master
prometheus-more-fixes
4.6
4.5
fix-j
steven-backport
4.4
prometheus-fixes
grafana_dashboard
prometheus
daemontools
consul_improvements
feature/trigger
4.3
4.0-pre-not-stable
4.2
4.1
4.0
feature_install_and_preos
3.1
no-dot-cdist
random_dot_cdist
feature_yum_url
feature_files_export
3.0
2.3
2.2
2.1
ssh_callback
2.0
archive_shell_function_approach
1.7
1.6
1.5
1.4
1.3
1.2
1.1
1.0
7.0.0
6.9.8
6.9.7
6.9.6
6.9.5
6.9.4
6.9.3
6.9.2
6.9.1
6.9.0
6.8.0
6.7.0
6.6.0
6.5.6
6.5.5
6.5.4
6.5.3
6.5.2
6.5.1
6.5.0
6.4.0
6.3.0
6.2.0
6.1.1
6.1.0
6.0.4
6.0.3
6.0.2
6.0.1
6.0.0
5.1.3
5.1.2
5.1.1
5.1.0
5.0.2
5.0.1
5.0.0
4.11.1
4.11.0
4.10.11
4.10.10
4.10.9
4.10.8
4.10.7
4.10.6
4.10.5
4.10.4
4.10.3
4.10.2
4.10.1
4.10.0
4.9.1
4.9.0
4.8.4
4.8.3
4.8.2
4.8.1
4.8.0
4.7.3
4.7.2
4.7.1
4.7.0
4.6.1
4.6.0
4.5.0
4.4.4
4.4.3
4.4.2
4.4.1
4.4.0
4.3.2
4.3.1
4.3.0
4.2.2
4.2.1
4.2.0
4.1.0
4.0.0
3.1.13
tn1
3.1.12
3.1.11
3.1.10
3.1.9
3.1.8
3.1.7
3.1.6
4.0.0pre3
3.1.5
3.1.4
3.1.3
3.1.2
3.1.1
3.1.0
4.0.0pre2
3.0.9
3.0.8
3.0.7
3.0.6
3.0.5
3.0.4
3.0.3
4.0.0pre1
3.0.2
3.0.1
3.0.0
2.3.7
2.3.6
2.3.5
2.3.4
2.3.3
2.3.2
2.3.1
2.3.0
2.2.0
2.1.2
2.1.1
2.1.0
2.1.0pre8
2.1.0pre7
2.1.0pre6
2.0.15
2.1.0pre5
2.1.0pre4
2.1.0pre3
2.1.0pre2
2.1.0pre1
2.0.14
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.7.1
1.7.0
1.6.2
1.6.1
1.6.0
1.5.0
1.4.1
1.4.0
1.3.2
1.3.1
1.3.0
1.2.0
1.1.0
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.9.9
0.9.8
0.9.7
0.9.6
0.9.5
0.9.4
0.9.3
0.9.2
0.9.1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Labels
Clear labels   
bugfix

   
cleanup

   
discussion

   
documentation

   
doing

   
done

   
feature

   
improvement

   
packaging

   
Stale

   
testing

   
TODO
No labels
bugfix
cleanup
discussion
documentation
doing
done
feature
improvement
packaging
Stale
testing
TODO
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ungleich-public/cdist#99
No description provided.