ungleich-public/cdist
ungleich-public/cdist
7
4
Fork 10
Code Issues 59 Pull requests 15 Projects Releases Wiki Activity

unexpected result using __line type #24

New issue
Open
opened 2021-11-20 11:24:54 +00:00 by ungleich-gitea · 7 comments
ungleich-gitea commented 2021-11-20 11:24:54 +00:00
Owner
Copy link

execute test

cat > /tmp/interfaces <<EOF
auto eno1
iface eno1 inet manual
auto eno2
iface eno2 inet manual
EOF

cp /tmp/interfaces /tmp/interfaces.orig

cdist config localhost -i - <<EOF
__line test_eno1 \
  --file /tmp/interfaces \
  --after 'iface eno1' \
  --line  '    mtu 9000' \
  --state present

__line test_eno2 \
  --file /tmp/interfaces \
  --after 'iface eno2' \
  --line  '    mtu 9000' \
  --state present
EOF

diff -u --color /tmp/interfaces.orig /tmp/interfaces

result:

[02:37:21] $ diff -u --color /tmp/interfaces.orig /tmp/interfaces
--- /tmp/interfaces.orig	2020-10-20 02:37:20.312333544 +0200
+++ /tmp/interfaces	2020-10-20 02:37:21.600376393 +0200
@@ -2,3 +2,4 @@
 iface eno1 inet manual
 auto eno2
 iface eno2 inet manual
+    mtu 9000
execute test ``` cat > /tmp/interfaces <<EOF auto eno1 iface eno1 inet manual auto eno2 iface eno2 inet manual EOF cp /tmp/interfaces /tmp/interfaces.orig cdist config localhost -i - <<EOF __line test_eno1 \ --file /tmp/interfaces \ --after 'iface eno1' \ --line ' mtu 9000' \ --state present __line test_eno2 \ --file /tmp/interfaces \ --after 'iface eno2' \ --line ' mtu 9000' \ --state present EOF diff -u --color /tmp/interfaces.orig /tmp/interfaces ``` result: ```diff [02:37:21] $ diff -u --color /tmp/interfaces.orig /tmp/interfaces --- /tmp/interfaces.orig 2020-10-20 02:37:20.312333544 +0200 +++ /tmp/interfaces 2020-10-20 02:37:21.600376393 +0200 @@ -2,3 +2,4 @@ iface eno1 inet manual auto eno2 iface eno2 inet manual + mtu 9000 ```
ungleich-gitea commented 2021-11-20 15:27:08 +00:00
Author
Owner
Copy link

@pedro The __hosts type has many issues of its own (cf. #826). Don't use it as an example.

@pedro The `__hosts` type has many issues of its own (cf. #826). Don't use it as an example.
ungleich-gitea commented 2021-11-20 15:27:09 +00:00
Author
Owner
Copy link

@pedro If you're interested in modifying network interfaces you might be interested in !838.
The MR is actually working, but still marked WIP because I can't make up my mind how and if dual stack configurations should be handled.

@pedro If you're interested in modifying network interfaces you might be interested in !838. The MR is actually working, but still marked WIP because I can't make up my mind how and if dual stack configurations should be handled.
ungleich-gitea commented 2021-11-20 15:27:11 +00:00
Author
Owner
Copy link

@ander that's nice, object-ids are unique, so if we are always appending automatically the object ID as a comment on each line introduced, then it would work. I saw something in __hosts type, which, oh wow, it uses __line type too in a very similar use case, and they solved it implementing what you said:

https://code.ungleich.ch/ungleich-public/cdist/-/blob/master/cdist/conf/type/__hosts/manifest#L26

so what about moving that marker in __line type so more types and custom manifests can benefit of using __line multiple times

@ander that's nice, object-ids are unique, so if we are always appending automatically the object ID as a comment on each line introduced, then it would work. I saw something in __hosts type, which, oh wow, it uses __line type too in a very similar use case, and they solved it implementing what you said: https://code.ungleich.ch/ungleich-public/cdist/-/blob/master/cdist/conf/type/__hosts/manifest#L26 so what about moving that *marker* in __line type so more types and custom manifests can benefit of using __line multiple times
ungleich-gitea commented 2021-11-20 15:27:12 +00:00
Author
Owner
Copy link

can confirm.

reason is that __line will allow only one occurence of --line X.

usecase is totally valid, but __line is not designed this usecase in mind. not a bug, but rather shortcoming.

one possible workaround is to make lines somewhat unique, for example:

__line test_eno1 \
    --file /tmp/interfaces \
    --after 'iface eno1' \
    --line  '    mtu 9000 # eno1 mtu' \
    --state present

__line test_eno2 \
    --file /tmp/interfaces \
    --after 'iface eno2' \
    --line  '    mtu 9000 # eno2 mtu' \
    --state present

but this doesn't fly with every file format.

my awk-fu isn't that good, so I can't help with a "fix" here 😿

can confirm. reason is that `__line` will allow only one occurence of `--line X`. usecase is totally valid, but `__line` is not designed this usecase in mind. not a bug, but rather shortcoming. one possible workaround is to make lines somewhat unique, for example: ``` __line test_eno1 \ --file /tmp/interfaces \ --after 'iface eno1' \ --line ' mtu 9000 # eno1 mtu' \ --state present __line test_eno2 \ --file /tmp/interfaces \ --after 'iface eno2' \ --line ' mtu 9000 # eno2 mtu' \ --state present ``` but this doesn't fly with every file format. my awk-fu isn't that good, so I can't help with a "fix" here :crying_cat_face:
ungleich-gitea commented 2021-11-20 15:27:13 +00:00
Author
Owner
Copy link

I have the same output, I hope is easy enough to test

cat > /tmp/interfaces <<EOF
auto eno1
iface eno1 inet manual
auto eno2
iface eno2 inet manual
EOF

cp /tmp/interfaces /tmp/interfaces.orig

cdist config localhost -i - <<EOF
__line test_eno1 \
  --file /tmp/interfaces \
  --after 'iface eno1' \
  --line  '    mtu 9000' \
  --state present

require='__line/test_eno1' \
  __line test_eno2 \
    --file /tmp/interfaces \
    --after 'iface eno2' \
    --line  '    mtu 9000' \
    --state present
EOF
diff -u --color /tmp/interfaces.orig /tmp/interfaces
I have the same output, I hope is easy enough to test ``` cat > /tmp/interfaces <<EOF auto eno1 iface eno1 inet manual auto eno2 iface eno2 inet manual EOF cp /tmp/interfaces /tmp/interfaces.orig cdist config localhost -i - <<EOF __line test_eno1 \ --file /tmp/interfaces \ --after 'iface eno1' \ --line ' mtu 9000' \ --state present require='__line/test_eno1' \ __line test_eno2 \ --file /tmp/interfaces \ --after 'iface eno2' \ --line ' mtu 9000' \ --state present EOF diff -u --color /tmp/interfaces.orig /tmp/interfaces ```
ungleich-gitea commented 2021-11-20 15:27:14 +00:00
Author
Owner
Copy link

race condition, because no dependency defined and parallel execution?

add require='__line/test_eno1' \ before __line test_eno2 \

race condition, because no dependency defined and parallel execution? add `require='__line/test_eno1' \` before `__line test_eno2 \`
ungleich-gitea commented 2021-11-20 15:27:16 +00:00
Author
Owner
Copy link

changed the description

changed the description
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
7.0
6.9
py3.10
ander/__package_apt_update_index
haproxy-dualstack
ander/__sed
beta
ander/os_version_debian_sid
evilham-compatibility-fixes
ander/__rsync
__snakeoil_cert
ander/update_readme
__download_improvements
feature/onchange
cleanup/string-formatting
feature/type-relationship-graph
cleanup/ssh-auth-keys-types
__letsencrypt_cert-fix-hooks
bugfix/preos-debug
bugfix/in-script-import
6.8
bugfix/sphinx-docs-build
6.7
cherry-pick-2f433a14
bugfix/make-code-consistent
6.6
regain-py3.2-support
6.5
bugfix/multiple-log-lines
matterbridge
coturn
alpinefix
matrix
new-type/network-interface
feature/process
6.4
feature/info-command
feature/libexec
6.3
preos-plugins-dir-opt
gitlab-ci
6.2
order-dep-fix
6.1
6.0
build/support-pip-from-git
feature/shell-lib
5.1
feature/support-type-deprecation
5.0
feature/python-types
4.11
4.10
shellcheck
4.9
4.8
freebsd-improvements
new-prometheus
key_value-onchange
feature/output_streams
AnotherKamila-patch-1
__letsencrypt_cert-fixes
letsencrypt-cron-fix
4.7
newtype-__letsencrypt_cert
os_explorer_devuan_fix
prometheus-exporter-fixes
daemontools-for-fbsd
type-prometheus-exporter-from-master
prometheus-more-fixes
4.6
4.5
fix-j
steven-backport
4.4
prometheus-fixes
grafana_dashboard
prometheus
daemontools
consul_improvements
feature/trigger
4.3
4.0-pre-not-stable
4.2
4.1
4.0
feature_install_and_preos
3.1
no-dot-cdist
random_dot_cdist
feature_yum_url
feature_files_export
3.0
2.3
2.2
2.1
ssh_callback
2.0
archive_shell_function_approach
1.7
1.6
1.5
1.4
1.3
1.2
1.1
1.0
7.0.0
6.9.8
6.9.7
6.9.6
6.9.5
6.9.4
6.9.3
6.9.2
6.9.1
6.9.0
6.8.0
6.7.0
6.6.0
6.5.6
6.5.5
6.5.4
6.5.3
6.5.2
6.5.1
6.5.0
6.4.0
6.3.0
6.2.0
6.1.1
6.1.0
6.0.4
6.0.3
6.0.2
6.0.1
6.0.0
5.1.3
5.1.2
5.1.1
5.1.0
5.0.2
5.0.1
5.0.0
4.11.1
4.11.0
4.10.11
4.10.10
4.10.9
4.10.8
4.10.7
4.10.6
4.10.5
4.10.4
4.10.3
4.10.2
4.10.1
4.10.0
4.9.1
4.9.0
4.8.4
4.8.3
4.8.2
4.8.1
4.8.0
4.7.3
4.7.2
4.7.1
4.7.0
4.6.1
4.6.0
4.5.0
4.4.4
4.4.3
4.4.2
4.4.1
4.4.0
4.3.2
4.3.1
4.3.0
4.2.2
4.2.1
4.2.0
4.1.0
4.0.0
3.1.13
tn1
3.1.12
3.1.11
3.1.10
3.1.9
3.1.8
3.1.7
3.1.6
4.0.0pre3
3.1.5
3.1.4
3.1.3
3.1.2
3.1.1
3.1.0
4.0.0pre2
3.0.9
3.0.8
3.0.7
3.0.6
3.0.5
3.0.4
3.0.3
4.0.0pre1
3.0.2
3.0.1
3.0.0
2.3.7
2.3.6
2.3.5
2.3.4
2.3.3
2.3.2
2.3.1
2.3.0
2.2.0
2.1.2
2.1.1
2.1.0
2.1.0pre8
2.1.0pre7
2.1.0pre6
2.0.15
2.1.0pre5
2.1.0pre4
2.1.0pre3
2.1.0pre2
2.1.0pre1
2.0.14
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.7.1
1.7.0
1.6.2
1.6.1
1.6.0
1.5.0
1.4.1
1.4.0
1.3.2
1.3.1
1.3.0
1.2.0
1.1.0
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.9.9
0.9.8
0.9.7
0.9.6
0.9.5
0.9.4
0.9.3
0.9.2
0.9.1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Labels
Clear labels   
bugfix

   
cleanup

   
discussion

   
documentation

   
doing

   
done

   
feature

   
improvement

   
packaging

   
Stale

   
testing

   
TODO
No labels
bugfix
cleanup
discussion
documentation
doing
done
feature
improvement
packaging
Stale
testing
TODO
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ungleich-public/cdist#24
No description provided.