Object conflict where there should not be one - or wrong error message #108

Closed
opened 2021-11-20 13:24:02 +00:00 by ungleich-gitea · 11 comments

Created by: telmich

[8:29] line:~% cdist config -vvc ~/vcs/ungleich-dot-cdist/ -p10 red1.place5.ungleich.ch         
VERBOSE: cdist: version 4.10.3-22-g383af673                                
WARNING: [11468]: red1.place5.ungleich.ch: Could not derive host_name for red1.place5.ungleich.ch, $host_name will be empty. Error is: [Errno 1] Unknown host
INFO: [11468]: red1.place5.ungleich.ch: Starting configuration run
ERROR: [11468]: red1.place5.ungleich.ch: Linking manifest /home/nico/vcs/ungleich-dot-cdist/manifest/.#dcl to /tmp/tmpe0cvk44v/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/.#dcl failed: [Errno 17] File exists: '/home/nico/vcs/ungleich-dot-cdist/manifest/.#dcl' -> '/tmp/tmpe0cvk44v/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/.#dcl'
VERBOSE: [11468]: config: Total processing time for 1 host(s): 0.2670779228210449
[8:29] line:~% cdist config -vvc ~/vcs/ungleich-dot-cdist/ -p10 red1.place5.ungleich.ch
VERBOSE: cdist: version 4.10.3-22-g383af673
WARNING: [11565]: red1.place5.ungleich.ch: Could not derive host_name for red1.place5.ungleich.ch, $host_name will be empty. Error is: [Errno 1] Unknown host
INFO: [11565]: red1.place5.ungleich.ch: Starting configuration run
VERBOSE: [11565]: red1.place5.ungleich.ch: Running global explorers
VERBOSE: [11565]: red1.place5.ungleich.ch: Running initial manifest /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/init
VERBOSE: [11565]: red1.place5.ungleich.ch: Preparing object __ungleich_ntp/
VERBOSE: [11565]: red1.place5.ungleich.ch: Running manifest and explorers for __ungleich_ntp/
VERBOSE: [11565]: red1.place5.ungleich.ch: Running type explorers for <CdistType __ungleich_ntp>
VERBOSE: [11565]: red1.place5.ungleich.ch: Running type manifest /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__ungleich_ntp/manifest for object __ungleich_ntp/
VERBOSE: [11565]: red1.place5.ungleich.ch: Preparing object __dcl_ceph/
VERBOSE: [11565]: red1.place5.ungleich.ch: Running manifest and explorers for __dcl_ceph/
VERBOSE: [11565]: red1.place5.ungleich.ch: Running type explorers for <CdistType __dcl_ceph>
VERBOSE: [11565]: red1.place5.ungleich.ch: Running type manifest /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest for object __dcl_ceph/
ERROR: [11565]: red1.place5.ungleich.ch: /bin/sh -e /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest: ['/bin/sh', '-e', '/tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest']

Error processing object '__dcl_ceph/'
=====================================
name: __dcl_ceph/
path: /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/object/__dcl_ceph/.cdist-0nb7pogm
source: /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/init
type: /home/nico/vcs/ungleich-dot-cdist/type/__dcl_ceph

manifest:stderr
---------------
ERROR: cdist: Object __ungleich_monit/ungleich already exists with conflicting parameters:
/tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/init: {'config': 'ungleich', 'state': 'present'}
/tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest: {'config': ['ungleich'], 'state': 'present'}


VERBOSE: [11565]: config: Total processing time for 1 host(s): 14.516885995864868
[8:29] line:~%  
*Created by: telmich* ~~~~ [8:29] line:~% cdist config -vvc ~/vcs/ungleich-dot-cdist/ -p10 red1.place5.ungleich.ch VERBOSE: cdist: version 4.10.3-22-g383af673 WARNING: [11468]: red1.place5.ungleich.ch: Could not derive host_name for red1.place5.ungleich.ch, $host_name will be empty. Error is: [Errno 1] Unknown host INFO: [11468]: red1.place5.ungleich.ch: Starting configuration run ERROR: [11468]: red1.place5.ungleich.ch: Linking manifest /home/nico/vcs/ungleich-dot-cdist/manifest/.#dcl to /tmp/tmpe0cvk44v/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/.#dcl failed: [Errno 17] File exists: '/home/nico/vcs/ungleich-dot-cdist/manifest/.#dcl' -> '/tmp/tmpe0cvk44v/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/.#dcl' VERBOSE: [11468]: config: Total processing time for 1 host(s): 0.2670779228210449 [8:29] line:~% cdist config -vvc ~/vcs/ungleich-dot-cdist/ -p10 red1.place5.ungleich.ch VERBOSE: cdist: version 4.10.3-22-g383af673 WARNING: [11565]: red1.place5.ungleich.ch: Could not derive host_name for red1.place5.ungleich.ch, $host_name will be empty. Error is: [Errno 1] Unknown host INFO: [11565]: red1.place5.ungleich.ch: Starting configuration run VERBOSE: [11565]: red1.place5.ungleich.ch: Running global explorers VERBOSE: [11565]: red1.place5.ungleich.ch: Running initial manifest /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/init VERBOSE: [11565]: red1.place5.ungleich.ch: Preparing object __ungleich_ntp/ VERBOSE: [11565]: red1.place5.ungleich.ch: Running manifest and explorers for __ungleich_ntp/ VERBOSE: [11565]: red1.place5.ungleich.ch: Running type explorers for <CdistType __ungleich_ntp> VERBOSE: [11565]: red1.place5.ungleich.ch: Running type manifest /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__ungleich_ntp/manifest for object __ungleich_ntp/ VERBOSE: [11565]: red1.place5.ungleich.ch: Preparing object __dcl_ceph/ VERBOSE: [11565]: red1.place5.ungleich.ch: Running manifest and explorers for __dcl_ceph/ VERBOSE: [11565]: red1.place5.ungleich.ch: Running type explorers for <CdistType __dcl_ceph> VERBOSE: [11565]: red1.place5.ungleich.ch: Running type manifest /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest for object __dcl_ceph/ ERROR: [11565]: red1.place5.ungleich.ch: /bin/sh -e /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest: ['/bin/sh', '-e', '/tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest'] Error processing object '__dcl_ceph/' ===================================== name: __dcl_ceph/ path: /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/object/__dcl_ceph/.cdist-0nb7pogm source: /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/init type: /home/nico/vcs/ungleich-dot-cdist/type/__dcl_ceph manifest:stderr --------------- ERROR: cdist: Object __ungleich_monit/ungleich already exists with conflicting parameters: /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/init: {'config': 'ungleich', 'state': 'present'} /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest: {'config': ['ungleich'], 'state': 'present'} VERBOSE: [11565]: config: Total processing time for 1 host(s): 14.516885995864868 [8:29] line:~% ~~~~
ungleich-gitea added the
bugfix
label 2021-11-20 13:24:02 +00:00
Author
Owner

Created by: darko-poljak

But, argparse resolves param as list and list != tuple :)

*Created by: darko-poljak* But, argparse resolves param as list and list != tuple :)
Author
Owner

Created by: darko-poljak

@telmich Although I think reading should return tuple instead of list, an unmodifiable sequence type. Do you agree?

*Created by: darko-poljak* @telmich Although I think reading should return tuple instead of list, an unmodifiable sequence type. Do you agree?
Author
Owner

Created by: darko-poljak

/home/darko/.cdist/manifest/test: {'state': 'present', 'config': 'ungleich\nungleich2'}
/home/darko/.cdist/manifest/test: {'state': 'present', 'config': ['ungleich', 'ungleich2']}

*Created by: darko-poljak* /home/darko/.cdist/manifest/test: {'state': 'present', 'config': **'ungleich\nungleich2'**} /home/darko/.cdist/manifest/test: {'state': 'present', 'config': ['ungleich', 'ungleich2']}
Author
Owner

Created by: darko-poljak

@telmich

$ ./bin/cdist config -vv -n  -i ~/.cdist/manifest/test $(cat ~/ungleich/data/opennebula-staticweb-test )
INFO: cdist: version 3.1.13
INFO: 185.203.112.26: Running global explorers
Unknown OS
Unknown OS
Unknown OS
Unknown OS
Unknown OS
Unknown OS
INFO: 185.203.112.26: Running initial manifest /home/darko/.cdist/manifest/test
ERROR: cdist: Object __ungleich_monit/ungleich already exists with conflicting parameters:
/home/darko/.cdist/manifest/test: {'state': 'present', 'config': 'ungleich\nungleich2'}
/home/darko/.cdist/manifest/test: {'state': 'present', 'config': ['ungleich', 'ungleich2']}
ERROR: 185.203.112.26: Command failed: /bin/sh -e /home/darko/.cdist/manifest/test
INFO: cdist: Total processing time for 1 host(s): 21.31067657470703
ERROR: cdist: Failed to configure the following hosts: 185.203.112.26
*Created by: darko-poljak* @telmich ``` $ ./bin/cdist config -vv -n -i ~/.cdist/manifest/test $(cat ~/ungleich/data/opennebula-staticweb-test ) INFO: cdist: version 3.1.13 INFO: 185.203.112.26: Running global explorers Unknown OS Unknown OS Unknown OS Unknown OS Unknown OS Unknown OS INFO: 185.203.112.26: Running initial manifest /home/darko/.cdist/manifest/test ERROR: cdist: Object __ungleich_monit/ungleich already exists with conflicting parameters: /home/darko/.cdist/manifest/test: {'state': 'present', 'config': 'ungleich\nungleich2'} /home/darko/.cdist/manifest/test: {'state': 'present', 'config': ['ungleich', 'ungleich2']} ERROR: 185.203.112.26: Command failed: /bin/sh -e /home/darko/.cdist/manifest/test INFO: cdist: Total processing time for 1 host(s): 21.31067657470703 ERROR: cdist: Failed to configure the following hosts: 185.203.112.26 ```
Author
Owner

Created by: darko-poljak

@telmich Randomly chosen 4.0 branch.

$ git checkout 4.0
Switched to branch '4.0'
Your branch is up to date with 'origin/4.0'.
$ ./bin/cdist config -vv -n  -i ~/.cdist/manifest/test $(cat ~/ungleich/data/opennebula-staticweb-test )
INFO: cdist: version 4.0.0-2-gd9acd2b2
INFO: 185.203.112.26: Running global explorers
INFO: 185.203.112.26: Running initial manifest /home/darko/.cdist/manifest/test
ERROR: cdist: Object __ungleich_monit/ungleich already exists with conflicting parameters:
/home/darko/.cdist/manifest/test: {'state': 'present', 'config': 'ungleich\nungleich2'}
/home/darko/.cdist/manifest/test: {'state': 'present', 'config': ['ungleich', 'ungleich2']}
ERROR: 185.203.112.26: Command failed: /bin/sh -e /home/darko/.cdist/manifest/test
INFO: cdist: Total processing time for 1 host(s): 5.966127157211304
ERROR: cdist: Failed to configure the following hosts: 185.203.112.26
*Created by: darko-poljak* @telmich Randomly chosen 4.0 branch. ``` $ git checkout 4.0 Switched to branch '4.0' Your branch is up to date with 'origin/4.0'. $ ./bin/cdist config -vv -n -i ~/.cdist/manifest/test $(cat ~/ungleich/data/opennebula-staticweb-test ) INFO: cdist: version 4.0.0-2-gd9acd2b2 INFO: 185.203.112.26: Running global explorers INFO: 185.203.112.26: Running initial manifest /home/darko/.cdist/manifest/test ERROR: cdist: Object __ungleich_monit/ungleich already exists with conflicting parameters: /home/darko/.cdist/manifest/test: {'state': 'present', 'config': 'ungleich\nungleich2'} /home/darko/.cdist/manifest/test: {'state': 'present', 'config': ['ungleich', 'ungleich2']} ERROR: 185.203.112.26: Command failed: /bin/sh -e /home/darko/.cdist/manifest/test INFO: cdist: Total processing time for 1 host(s): 5.966127157211304 ERROR: cdist: Failed to configure the following hosts: 185.203.112.26 ```
Author
Owner

Created by: telmich

Is this something that we recently changed? Because the above manifest did run without changes some time ago, I claim

*Created by: telmich* Is this something that we recently changed? Because the above manifest did run without changes some time ago, I claim
Author
Owner

Created by: darko-poljak

@telmich Something like the following:

diff --git a/cdist/util/fsproperty.py b/cdist/util/fsproperty.py
index e458fd9e..fdde8fe2 100644
--- a/cdist/util/fsproperty.py
+++ b/cdist/util/fsproperty.py
@@ -127,11 +127,17 @@ class DirectoryDict(collections.MutableMapping):
     def __getitem__(self, key):
         try:
             with open(os.path.join(self.path, key), "r") as fd:
-                return fd.read().rstrip('\n')
+                value = fd.read().rstrip('\n')
+                value_list = value.split('\n')
+                if len(value_list) == 1:
+                    return value_list[0]
+                else:
+                    return value_list
         except EnvironmentError:
             raise KeyError(key)
 
     def __setitem__(self, key, value):
*Created by: darko-poljak* @telmich Something like the following: ``` diff --git a/cdist/util/fsproperty.py b/cdist/util/fsproperty.py index e458fd9e..fdde8fe2 100644 --- a/cdist/util/fsproperty.py +++ b/cdist/util/fsproperty.py @@ -127,11 +127,17 @@ class DirectoryDict(collections.MutableMapping): def __getitem__(self, key): try: with open(os.path.join(self.path, key), "r") as fd: - return fd.read().rstrip('\n') + value = fd.read().rstrip('\n') + value_list = value.split('\n') + if len(value_list) == 1: + return value_list[0] + else: + return value_list except EnvironmentError: raise KeyError(key) def __setitem__(self, key, value): ```
Author
Owner

Created by: darko-poljak

@telmich It is reproduced easily with the following init manifest:

__ungleich_monit ungleich --config ungleich
__ungleich_monit ungleich --config ungleich

It seems that DirectoryDict does not work as it should for multiple parameters, config can be specified multiple times.
When it reads multiple parameter from file it returns file content with last \n stripped.
But when it writes multiple parameters it detects if value is a sequence.

If parameter is not multiple then file content is always one line, right?
Then when reading value in DirectoryDict we should try to split read content by \n, and if there is more then one value then we should return a list.

See the following output (additional print added to source code) for init manifest:

__ungleich_monit ungleich --config ungleich --config ungleich2
__ungleich_monit ungleich --config ungleich --config ungleich2
VERBOSE: cdist: version 4.10.3-115-g5e99219e
INFO: 185.203.112.26: Starting dry run
VERBOSE: 185.203.112.26: Running global explorers
VERBOSE: 185.203.112.26: Running initial manifest /home/darko/.cdist/manifest/test
ERROR: 185.203.112.26: /bin/sh -e /home/darko/.cdist/manifest/test: ['/bin/sh', '-e', '/home/darko/.cdist/manifest/test']

Error processing initial manifest
=================================
path: /home/darko/.cdist/manifest/test

init:stdout
-----------
value:  present
value:  ['ungleich', 'ungleich2']
parameters:  {'state': 'present', 'config': ['ungleich', 'ungleich2']}
dirdictprop value:  {'state': 'present', 'config': ['ungleich', 'ungleich2']}
dirdictprop attribute_instance:  {}
setitem state present
setitem config ['ungleich', 'ungleich2']
dirdictprop attribute_instance after update:  {'state': 'present', 'config': 'ungleich\nungleich2'}
cdist_object.parameters (after create):  {'state': 'present', 'config': 'ungleich\nungleich2'}
value:  present
value:  ['ungleich', 'ungleich2']
parameters:  {'state': 'present', 'config': ['ungleich', 'ungleich2']}
cdist_object.parameters:  {'state': 'present', 'config': 'ungleich\nungleich2'}

init:stderr
-----------
ERROR: cdist: Object __ungleich_monit/ungleich already exists with conflicting parameters:
/home/darko/.cdist/manifest/test: {'state': 'present', 'config': 'ungleich\nungleich2'}
/home/darko/.cdist/manifest/test: {'state': 'present', 'config': ['ungleich', 'ungleich2']}
*Created by: darko-poljak* @telmich It is reproduced easily with the following init manifest: ``` __ungleich_monit ungleich --config ungleich __ungleich_monit ungleich --config ungleich ``` It seems that DirectoryDict does not work as it should for multiple parameters, config can be specified multiple times. When it reads multiple parameter from file it returns file content with last \n stripped. But when it writes multiple parameters it detects if value is a sequence. If parameter is not multiple then file content is always one line, right? Then when reading value in DirectoryDict we should try to split read content by \n, and if there is more then one value then we should return a list. See the following output (additional print added to source code) for init manifest: ``` __ungleich_monit ungleich --config ungleich --config ungleich2 __ungleich_monit ungleich --config ungleich --config ungleich2 ``` ``` VERBOSE: cdist: version 4.10.3-115-g5e99219e INFO: 185.203.112.26: Starting dry run VERBOSE: 185.203.112.26: Running global explorers VERBOSE: 185.203.112.26: Running initial manifest /home/darko/.cdist/manifest/test ERROR: 185.203.112.26: /bin/sh -e /home/darko/.cdist/manifest/test: ['/bin/sh', '-e', '/home/darko/.cdist/manifest/test'] Error processing initial manifest ================================= path: /home/darko/.cdist/manifest/test init:stdout ----------- value: present value: ['ungleich', 'ungleich2'] parameters: {'state': 'present', 'config': ['ungleich', 'ungleich2']} dirdictprop value: {'state': 'present', 'config': ['ungleich', 'ungleich2']} dirdictprop attribute_instance: {} setitem state present setitem config ['ungleich', 'ungleich2'] dirdictprop attribute_instance after update: {'state': 'present', 'config': 'ungleich\nungleich2'} cdist_object.parameters (after create): {'state': 'present', 'config': 'ungleich\nungleich2'} value: present value: ['ungleich', 'ungleich2'] parameters: {'state': 'present', 'config': ['ungleich', 'ungleich2']} cdist_object.parameters: {'state': 'present', 'config': 'ungleich\nungleich2'} init:stderr ----------- ERROR: cdist: Object __ungleich_monit/ungleich already exists with conflicting parameters: /home/darko/.cdist/manifest/test: {'state': 'present', 'config': 'ungleich\nungleich2'} /home/darko/.cdist/manifest/test: {'state': 'present', 'config': ['ungleich', 'ungleich2']} ```
Author
Owner

Created by: darko-poljak

@telmich

/tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/init: {'config': 'ungleich', 'state': 'present'}
/tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest: {'config': ['ungleich'], 'state': 'present'}

{'config': 'ungleich', 'state': 'present'} vs {'config': ['ungleich'], 'state': 'present'}. The question is why list in the second one.

*Created by: darko-poljak* @telmich ``` /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/manifest/init: {'config': 'ungleich', 'state': 'present'} /tmp/tmptirtsbvg/9613ea11c03c21bc404d3d251acd217e/data/conf/type/__dcl_ceph/manifest: {'config': ['ungleich'], 'state': 'present'} ``` `{'config': 'ungleich', 'state': 'present'}` vs `{'config': ['ungleich'], 'state': 'present'}`. The question is why list in the second one.
Author
Owner

Created by: telmich

@darko-poljak ping

*Created by: telmich* @darko-poljak ping
Author
Owner

Created by: telmich

manifest/dcl: __ungleich_monit ungleich --config ungleich
type/__dcl_ceph: __ungleich_monit ungleich --config ungleich

*Created by: telmich* manifest/dcl: __ungleich_monit ungleich --config ungleich type/__dcl_ceph: __ungleich_monit ungleich --config ungleich
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: ungleich-public/cdist#108
No description provided.