Easy way to force cdist following the created order in manifest #238

New Issue

Closed

opened 2021-11-20 15:22:13 +00:00 by ungleich-gitea · 5 comments

ungleich-gitea commented 2021-11-20 15:22:13 +00:00

Owner

Copy Link

Created by: dheule

Now we configure the whole manifest like this:

require="$r" __file /home/test/hostname --state absent
r="__file/home/test/hostname"
require="$r" __directory /home/test/public_html --state absent
r="__directory/home/test/public_html"
etc ...

please implement the possibility to force cdist to honor the order in which types are definied / created without the need to specifi a require on every type line.

One idea is to do this if the user exports require="CDIST_HONOR_CREATE_ORDER" or some other special variable exported at the beginning of the manifest.

Thx.

*Created by: dheule* Now we configure the whole manifest like this: require="$r" __file /home/test/hostname --state absent r="__file/home/test/hostname" require="$r" __directory /home/test/public_html --state absent r="__directory/home/test/public_html" etc ... please implement the possibility to force cdist to honor the order in which types are definied / created without the need to specifi a require on every type line. One idea is to do this if the user exports require="CDIST_HONOR_CREATE_ORDER" or some other special variable exported at the beginning of the manifest. Thx.

ungleich-gitea closed this issue 2021-11-20 15:22:14 +00:00

ungleich-gitea commented 2021-11-20 15:49:32 +00:00

Author

Owner

Copy Link

Created by: dheule

Implemented with CDIST_ORDER_DEPENDENCY since 3.0.7

*Created by: dheule* Implemented with CDIST_ORDER_DEPENDENCY since 3.0.7

ungleich-gitea commented 2021-11-20 15:49:33 +00:00

Author

Owner

Copy Link

Created by: dheule

I will try to make a hack in the python core next week

*Created by: dheule* I will try to make a hack in the python core next week

ungleich-gitea commented 2021-11-20 15:49:34 +00:00

Author

Owner

Copy Link

Created by: dheule

Hello Nico,

One of my real world examles, red are comments which are not in the
manifest,
but for explain the workflow:

patterns welche fuer sap notwendig sind ...

require="$r" __package ulimit --state absent
r="__package/ulimit"
require="$r" __package Basis-Devel --ptype pattern
r="__package/Basis-Devel"
require="$r" __package file_server --ptype pattern
r="__package/file_server"
require="$r" __package print_server --ptype pattern
r="__package/print_server"
require="$r" __package sap_server --ptype pattern
r="__package/sap_server"
require="$r" __package x11 --ptype pattern
r="__package/x11"

Disable cups on boot

require="$r" __start_on_boot cups --state absent
r="__start_on_boot/cups"

Folgende Keys können erst gesetzt werden nach der installation von
mindestens __package/sap_server

set our values on file /etc/sysconfig/sapconf

require="$r" __key_value LIMIT_1 --file /etc/sysconfig/sapconf --value
'"@sapsys soft nofile 131200"' --delimiter '='
r="__key_value/LIMIT_1"
require="$r" __key_value LIMIT_2 --file /etc/sysconfig/sapconf --value
'"@sapsys hard nofile 131200"' --delimiter '='
r="__key_value/LIMIT_2"
require="$r" __key_value LIMIT_3 --file /etc/sysconfig/sapconf --value
'"@sdba soft nofile 131200"' --delimiter '='
r="__key_value/LIMIT_3"
require="$r" __key_value LIMIT_4 --file /etc/sysconfig/sapconf --value
'"@sdba hard nofile 131200"' --delimiter '='
r="__key_value/LIMIT_4"
require="$r" __key_value LIMIT_5 --file /etc/sysconfig/sapconf --value
'"@dba soft nofile 131200"' --delimiter '='
r="__key_value/LIMIT_5"
require="$r" __key_value LIMIT_6 --file /etc/sysconfig/sapconf --value
'"@dba hard nofile 131200"' --delimiter '='
r="__key_value/LIMIT_6"
require="$r" __key_value LIMIT_7 --file /etc/sysconfig/sapconf --value
'"@dba soft memlock infinity"' --delimiter '='
r="__key_value/LIMIT_7"
require="$r" __key_value LIMIT_8 --file /etc/sysconfig/sapconf --value
'"@dba hard memlock infinity"' --delimiter '='
r="__key_value/LIMIT_8"
require="$r" __key_value LIMIT_9 --file /etc/sysconfig/sapconf --value
'"@sapsys soft memlock infinity"' --delimiter '='
r="__key_value/LIMIT_9"
require="$r" __key_value LIMIT_10 --file /etc/sysconfig/sapconf --value
'"@sapsys hard memlock infinity"' --delimiter '='
r="__key_value/LIMIT_10"

create sap standard users

require="$r" __group sapsys --gid 200
r="__group/sapsys"
require="$r" __group sapinst --gid 210
r="__group/sapinst"
require="$r" __group dba --gid 201
r="__group/dba"
require="$r" __group oper --gid 202
r="__group/oper"
User erstellen braucht die Gruppen vorher
require="$r" __user sapadm --uid 1000 --gid sapsys --comment "SAP System
Administrator" --system --password
'password_hier_gelöscht_da_ich_diesen_hash_nicht_im_inet_möchte'
r="__user/sapadm"

SAP scripts brauchen unser und gruppen

install backup, archive and maintenance scripts

require="$r" __sfs_sles_sapscripts
r="__sfs_sles_sapscripts"

SAP Host agent braucht gruppen,user,sapscript ...

install sap host agent

require="$r" __sfs_sap_hostagent --patch 174
r="__sfs_sap_hostagent"

install sap entrys in services file

require="$r" __sfs_sap_services --version $osv
r="__sfs_sap_services"

install oracle instant client for sap

require="$r" __sfs_sap_oracle_instantclient --version '11.2.0.3.0 V1'
r="__sfs_sap_oracle_instantclient"

. $__manifest/apps/netapp.sh

We have many more examples in other files,
sometimes its a crux, on a old system the order doen't matter when
you test, but on a fresh system with minial install it does matter,
so we fix the order of all type calls with the variable $r on the moment.
This works perfekt in all cases, because we write the manifest
in the order of the installation and dependencies of the system.

I think a cool solution is if we can export a special variable in the
manifest and then all following types are recorded with auto
require="__last_type_which_was_reated" ... possibility in core ?

Greats Daniel

2014/1/17 Nico Schottelius notifications@github.com

Hey Daniel,

we actually excluded this behaviour in the early design days,
because we wanted cdist to be independent of the order of execution.

Background for that is depending on which types you use, the execution
order may be different (of objects) and thus lead to different results
and thus to incosistent configurations.

I do however see the problem you raise, though I haven't encountered
this problem much so far. As it is easy to be wrong when reading mails,
let me rephrase it, so we are both on the same track:

The creation of objects depending on each other (recursively) using
require="" is cumbersome, as it requires (sic!) to explicitly define the
requirement of the previously defined object.

So we can look into how to fix this problem. I'd actually be happy if
you could add some real world examples where it is cumbersome - if it is
just 2 lines in reality, it is probably not worth to restructure the
workflow deeply within cdist.

Cheers,

Nico

dheule [Fri, Jan 17, 2014 at 12:47:38AM -0800]:

Now we configure the whole manifest like this:

require="$r" __file /home/test/hostname --state absent
r="__file/home/test/hostname"
require="$r" __directory /home/test/public_html --state absent
r="__directory/home/test/public_html"
etc ...

please implement the possibility to force cdist to honor the order in
which types are definied / created without the need to specifi a require on
every type line.

One idea is to do this if the user exports
require="CDIST_HONOR_CREATE_ORDER" or some other special variable exported
at the beginning of the manifest.

Thx.

---

Reply to this email directly or view it on GitHub:
https://github.com/telmich/cdist/issues/262

PGP key: 7ED9 F7D3 6B10 81D7 0EC5 5C09 D7DC C8E4 3187 7DF0

—
Reply to this email directly or view it on GitHubhttps://github.com/telmich/cdist/issues/262#issuecomment-32591468
.

*Created by: dheule* Hello Nico, One of my real world examles, red are comments which are not in the manifest, but for explain the workflow: # patterns welche fuer sap notwendig sind ... require="$r" __package ulimit --state absent r="__package/ulimit" require="$r" __package Basis-Devel --ptype pattern r="__package/Basis-Devel" require="$r" __package file_server --ptype pattern r="__package/file_server" require="$r" __package print_server --ptype pattern r="__package/print_server" require="$r" __package sap_server --ptype pattern r="__package/sap_server" require="$r" __package x11 --ptype pattern r="__package/x11" # Disable cups on boot require="$r" __start_on_boot cups --state absent r="__start_on_boot/cups" Folgende Keys können erst gesetzt werden nach der installation von mindestens __package/sap_server # set our values on file /etc/sysconfig/sapconf require="$r" __key_value LIMIT_1 --file /etc/sysconfig/sapconf --value '"@sapsys soft nofile 131200"' --delimiter '=' r="__key_value/LIMIT_1" require="$r" __key_value LIMIT_2 --file /etc/sysconfig/sapconf --value '"@sapsys hard nofile 131200"' --delimiter '=' r="__key_value/LIMIT_2" require="$r" __key_value LIMIT_3 --file /etc/sysconfig/sapconf --value '"@sdba soft nofile 131200"' --delimiter '=' r="__key_value/LIMIT_3" require="$r" __key_value LIMIT_4 --file /etc/sysconfig/sapconf --value '"@sdba hard nofile 131200"' --delimiter '=' r="__key_value/LIMIT_4" require="$r" __key_value LIMIT_5 --file /etc/sysconfig/sapconf --value '"@dba soft nofile 131200"' --delimiter '=' r="__key_value/LIMIT_5" require="$r" __key_value LIMIT_6 --file /etc/sysconfig/sapconf --value '"@dba hard nofile 131200"' --delimiter '=' r="__key_value/LIMIT_6" require="$r" __key_value LIMIT_7 --file /etc/sysconfig/sapconf --value '"@dba soft memlock infinity"' --delimiter '=' r="__key_value/LIMIT_7" require="$r" __key_value LIMIT_8 --file /etc/sysconfig/sapconf --value '"@dba hard memlock infinity"' --delimiter '=' r="__key_value/LIMIT_8" require="$r" __key_value LIMIT_9 --file /etc/sysconfig/sapconf --value '"@sapsys soft memlock infinity"' --delimiter '=' r="__key_value/LIMIT_9" require="$r" __key_value LIMIT_10 --file /etc/sysconfig/sapconf --value '"@sapsys hard memlock infinity"' --delimiter '=' r="__key_value/LIMIT_10" # create sap standard users require="$r" __group sapsys --gid 200 r="__group/sapsys" require="$r" __group sapinst --gid 210 r="__group/sapinst" require="$r" __group dba --gid 201 r="__group/dba" require="$r" __group oper --gid 202 r="__group/oper" User erstellen braucht die Gruppen vorher require="$r" __user sapadm --uid 1000 --gid sapsys --comment "SAP System Administrator" --system --password 'password_hier_gelöscht_da_ich_diesen_hash_nicht_im_inet_möchte' r="__user/sapadm" SAP scripts brauchen unser und gruppen # install backup, archive and maintenance scripts require="$r" __sfs_sles_sapscripts r="__sfs_sles_sapscripts" SAP Host agent braucht gruppen,user,sapscript ... # install sap host agent require="$r" __sfs_sap_hostagent --patch 174 r="__sfs_sap_hostagent" # install sap entrys in services file require="$r" __sfs_sap_services --version $osv r="__sfs_sap_services" # install oracle instant client for sap require="$r" __sfs_sap_oracle_instantclient --version '11.2.0.3.0 V1' r="__sfs_sap_oracle_instantclient" . $__manifest/apps/netapp.sh We have many more examples in other files, sometimes its a crux, on a old system the order doen't matter when you test, but on a fresh system with minial install it does matter, so we fix the order of all type calls with the variable $r on the moment. This works perfekt in all cases, because we write the manifest in the order of the installation and dependencies of the system. I think a cool solution is if we can export a special variable in the manifest and then all following types are recorded with auto require="__last_type_which_was_reated" ... possibility in core ? Greats Daniel 2014/1/17 Nico Schottelius notifications@github.com > Hey Daniel, > > we actually excluded this behaviour in the early design days, > because we wanted cdist to be independent of the order of execution. > > Background for that is depending on which types you use, the execution > order may be different (of objects) and thus lead to different results > and thus to incosistent configurations. > > I do however see the problem you raise, though I haven't encountered > this problem much so far. As it is easy to be wrong when reading mails, > let me rephrase it, so we are both on the same track: > > The creation of objects depending on each other (recursively) using > require="" is cumbersome, as it requires (sic!) to explicitly define the > requirement of the previously defined object. > > So we can look into how to fix this problem. I'd actually be happy if > you could add some real world examples where it is cumbersome - if it is > just 2 lines in reality, it is probably not worth to restructure the > workflow deeply within cdist. > > Cheers, > > Nico > > dheule [Fri, Jan 17, 2014 at 12:47:38AM -0800]: > > > Now we configure the whole manifest like this: > > > > require="$r" __file /home/test/hostname --state absent > > r="__file/home/test/hostname" > > require="$r" __directory /home/test/public_html --state absent > > r="__directory/home/test/public_html" > > etc ... > > > > please implement the possibility to force cdist to honor the order in > > which types are definied / created without the need to specifi a require on > > every type line. > > > > One idea is to do this if the user exports > > require="CDIST_HONOR_CREATE_ORDER" or some other special variable exported > > at the beginning of the manifest. > > > > Thx. > > > > --- > > > > Reply to this email directly or view it on GitHub: > > https://github.com/telmich/cdist/issues/262 > > ## > > PGP key: 7ED9 F7D3 6B10 81D7 0EC5 5C09 D7DC C8E4 3187 7DF0 > > — > Reply to this email directly or view it on GitHubhttps://github.com/telmich/cdist/issues/262#issuecomment-32591468 > .

ungleich-gitea commented 2021-11-20 15:49:36 +00:00

Author

Owner

Copy Link

Created by: asteven

Nico Schottelius wrote on 01/17/2014 10:38 AM:

Hey Daniel,

[...]

 The creation of objects depending on each other (recursively) using
 require="" is cumbersome, as it requires (sic!) to explicitly define the
 requirement of the previously defined object.

I've also been thinking how to solve this. Something like this would be
nice:

in_order=$magic_tool_script_type_something
$in_order << DONE
__foo
__bar
__gurk
DONE

And $magic_tool_script_type_something would turn it into:
__foo
require="__foo" __bar
require="__bar" __gurk

I don't think this is difficult to implement, the interesting question
is where to implement it.
e.g.

• cdist core (python)
• some cdist helper script
• a type

Cheers,
Steven

*Created by: asteven* Nico Schottelius wrote on 01/17/2014 10:38 AM: > Hey Daniel, > > [...] > > ``` > The creation of objects depending on each other (recursively) using > require="" is cumbersome, as it requires (sic!) to explicitly define the > requirement of the previously defined object. > ``` I've also been thinking how to solve this. Something like this would be nice: in_order=$magic_tool_script_type_something $in_order << DONE __foo __bar __gurk DONE And $magic_tool_script_type_something would turn it into: __foo require="__foo" __bar require="__bar" __gurk I don't think this is difficult to implement, the interesting question is _where_ to implement it. e.g. - cdist core (python) - some cdist helper script - a type Cheers, Steven

ungleich-gitea commented 2021-11-20 15:49:37 +00:00

Author

Owner

Copy Link

Created by: telmich

Hey Daniel,

we actually excluded this behaviour in the early design days,
because we wanted cdist to be independent of the order of execution.

Background for that is depending on which types you use, the execution
order may be different (of objects) and thus lead to different results
and thus to incosistent configurations.

I do however see the problem you raise, though I haven't encountered
this problem much so far. As it is easy to be wrong when reading mails,
let me rephrase it, so we are both on the same track:

The creation of objects depending on each other (recursively) using
require="" is cumbersome, as it requires (sic!) to explicitly define the
requirement of the previously defined object.

So we can look into how to fix this problem. I'd actually be happy if
you could add some real world examples where it is cumbersome - if it is
just 2 lines in reality, it is probably not worth to restructure the
workflow deeply within cdist.

Cheers,

Nico

dheule [Fri, Jan 17, 2014 at 12:47:38AM -0800]:

Now we configure the whole manifest like this:

require="$r" __file /home/test/hostname --state absent
r="__file/home/test/hostname"
require="$r" __directory /home/test/public_html --state absent
r="__directory/home/test/public_html"
etc ...

please implement the possibility to force cdist to honor the order in which types are definied / created without the need to specifi a require on every type line.

One idea is to do this if the user exports require="CDIST_HONOR_CREATE_ORDER" or some other special variable exported at the beginning of the manifest.

Thx.

---

Reply to this email directly or view it on GitHub:
https://github.com/telmich/cdist/issues/262

PGP key: 7ED9 F7D3 6B10 81D7 0EC5 5C09 D7DC C8E4 3187 7DF0

*Created by: telmich* Hey Daniel, we actually excluded this behaviour in the early design days, because we wanted cdist to be independent of the order of execution. Background for that is depending on which types you use, the execution order may be different (of objects) and thus lead to different results and thus to incosistent configurations. I do however see the problem you raise, though I haven't encountered this problem much so far. As it is easy to be wrong when reading mails, let me rephrase it, so we are both on the same track: ``` The creation of objects depending on each other (recursively) using require="" is cumbersome, as it requires (sic!) to explicitly define the requirement of the previously defined object. ``` So we can look into how to fix this problem. I'd actually be happy if you could add some real world examples where it is cumbersome - if it is just 2 lines in reality, it is probably not worth to restructure the workflow deeply within cdist. Cheers, Nico dheule [Fri, Jan 17, 2014 at 12:47:38AM -0800]: > Now we configure the whole manifest like this: > > require="$r" __file /home/test/hostname --state absent > r="__file/home/test/hostname" > require="$r" __directory /home/test/public_html --state absent > r="__directory/home/test/public_html" > etc ... > > please implement the possibility to force cdist to honor the order in which types are definied / created without the need to specifi a require on every type line. > > One idea is to do this if the user exports require="CDIST_HONOR_CREATE_ORDER" or some other special variable exported at the beginning of the manifest. > > Thx. > > --- > > Reply to this email directly or view it on GitHub: > https://github.com/telmich/cdist/issues/262 ## PGP key: 7ED9 F7D3 6B10 81D7 0EC5 5C09 D7DC C8E4 3187 7DF0

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

master

7.0

6.9

py3.10

ander/__package_apt_update_index

haproxy-dualstack

ander/__sed

beta

ander/os_version_debian_sid

evilham-compatibility-fixes

ander/__rsync

__snakeoil_cert

ander/update_readme

__download_improvements

feature/onchange

cleanup/string-formatting

feature/type-relationship-graph

cleanup/ssh-auth-keys-types

__letsencrypt_cert-fix-hooks

bugfix/preos-debug

bugfix/in-script-import

6.8

bugfix/sphinx-docs-build

6.7

cherry-pick-2f433a14

bugfix/make-code-consistent

6.6

regain-py3.2-support

6.5

bugfix/multiple-log-lines

matterbridge

coturn

alpinefix

matrix

new-type/network-interface

feature/process

6.4

feature/info-command

feature/libexec

6.3

preos-plugins-dir-opt

gitlab-ci

6.2

order-dep-fix

6.1

6.0

build/support-pip-from-git

feature/shell-lib

5.1

feature/support-type-deprecation

5.0

feature/python-types

4.11

4.10

shellcheck

4.9

4.8

freebsd-improvements

new-prometheus

key_value-onchange

feature/output_streams

AnotherKamila-patch-1

__letsencrypt_cert-fixes

letsencrypt-cron-fix

4.7

newtype-__letsencrypt_cert

os_explorer_devuan_fix

prometheus-exporter-fixes

daemontools-for-fbsd

type-prometheus-exporter-from-master

prometheus-more-fixes

4.6

4.5

fix-j

steven-backport

4.4

prometheus-fixes

grafana_dashboard

prometheus

daemontools

consul_improvements

feature/trigger

4.3

4.0-pre-not-stable

4.2

4.1

4.0

feature_install_and_preos

3.1

no-dot-cdist

random_dot_cdist

feature_yum_url

feature_files_export

3.0

2.3

2.2

2.1

ssh_callback

2.0

archive_shell_function_approach

1.7

1.6

1.5

1.4

1.3

1.2

1.1

1.0

7.0.0

6.9.8

6.9.7

6.9.6

6.9.5

6.9.4

6.9.3

6.9.2

6.9.1

6.9.0

6.8.0

6.7.0

6.6.0

6.5.6

6.5.5

6.5.4

6.5.3

6.5.2

6.5.1

6.5.0

6.4.0

6.3.0

6.2.0

6.1.1

6.1.0

6.0.4

6.0.3

6.0.2

6.0.1

6.0.0

5.1.3

5.1.2

5.1.1

5.1.0

5.0.2

5.0.1

5.0.0

4.11.1

4.11.0

4.10.11

4.10.10

4.10.9

4.10.8

4.10.7

4.10.6

4.10.5

4.10.4

4.10.3

4.10.2

4.10.1

4.10.0

4.9.1

4.9.0

4.8.4

4.8.3

4.8.2

4.8.1

4.8.0

4.7.3

4.7.2

4.7.1

4.7.0

4.6.1

4.6.0

4.5.0

4.4.4

4.4.3

4.4.2

4.4.1

4.4.0

4.3.2

4.3.1

4.3.0

4.2.2

4.2.1

4.2.0

4.1.0

4.0.0

3.1.13

tn1

3.1.12

3.1.11

3.1.10

3.1.9

3.1.8

3.1.7

3.1.6

4.0.0pre3

3.1.5

3.1.4

3.1.3

3.1.2

3.1.1

3.1.0

4.0.0pre2

3.0.9

3.0.8

3.0.7

3.0.6

3.0.5

3.0.4

3.0.3

4.0.0pre1

3.0.2

3.0.1

3.0.0

2.3.7

2.3.6

2.3.5

2.3.4

2.3.3

2.3.2

2.3.1

2.3.0

2.2.0

2.1.2

2.1.1

2.1.0

2.1.0pre8

2.1.0pre7

2.1.0pre6

2.0.15

2.1.0pre5

2.1.0pre4

2.1.0pre3

2.1.0pre2

2.1.0pre1

2.0.14

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.7.1

1.7.0

1.6.2

1.6.1

1.6.0

1.5.0

1.4.1

1.4.0

1.3.2

1.3.1

1.3.0

1.2.0

1.1.0

1.0.4

1.0.3

1.0.2

1.0.1

1.0.0

0.9.9

0.9.8

0.9.7

0.9.6

0.9.5

0.9.4

0.9.3

0.9.2

0.9.1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

Labels

Clear labels   

bugfix

  

cleanup

  

discussion

  

documentation

  

doing

  

done

  

feature

  

improvement

  

packaging

  

Stale

  

testing

  

TODO

No Label

bugfix

cleanup

discussion

documentation

doing

done

feature

improvement

packaging

Stale

testing

TODO

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: ungleich-public/cdist#238

No description provided.