[BETA] bug: error info lost with parallel (option -j) #133

New Issue

ungleich-gitea · 2021-11-20T13:24:28Z

ungleich-gitea commented

2021-11-20 13:24:28 +00:00

When running with -b -j$whatever and cdist fails, the stderr from types (NOT from the initial manifest) is not shown, which makes debugging types very hard.

What I expect

if a type fails, its stderr should be displayed, just like when running without parallelization

What happens

stderr not shown

Steps to reproduce

type/__test_type/manifest:

echo "This is my stderr" >&2
exit 47

manifest/init:

__test_type whatever

Output without parallelization / $ cdist config localhost:

ERROR: localhost: Command failed: '/bin/sh -e /tmp/tmp8q6105_m/421aa90e079fa326b6494f812ad13e79/data/conf/type/__test_type/manifest'
return code: 47
---- BEGIN stdout ----
---- END stdout ----

Error processing object '__test_type/whatever'
===============================================
name: __test_type/whatever
path: /tmp/tmp8q6105_m/421aa90e079fa326b6494f812ad13e79/data/object/__test_type/whatever/.cdist-_u9hg_tz
source: /tmp/tmp8q6105_m/421aa90e079fa326b6494f812ad13e79/data/conf/manifest/init
type: /tmp/tmp8q6105_m/421aa90e079fa326b6494f812ad13e79/data/conf/type/__test_type

---- BEGIN manifest:stderr ----
This is my stderr

---- END manifest:stderr ----

ERROR: cdist: Failed to configure the following hosts: localhost

Output with parallelization / $ cdist config -b -j4 localhost:

ERROR: [28695]: localhost: Command failed: '/bin/sh -e /tmp/tmpvbovdxyv/421aa90e079fa326b6494f812ad13e79/data/conf/type/__test_type/manifest'
return code: 47
---- BEGIN stdout ----
---- END stdout ----
ERROR: cdist: Failed to configure the following hosts: localhost

Note that no error information is shown in this case.

When running with `-b -j$whatever` and cdist fails, the stderr from types (NOT from the initial manifest) is not shown, which makes debugging types very hard. # What I expect if a type fails, its stderr should be displayed, just like when running without parallelization # What happens stderr not shown # Steps to reproduce **`type/__test_type/manifest`:** ```sh echo "This is my stderr" >&2 exit 47 ``` **`manifest/init`:** ```sh __test_type whatever ``` Output without parallelization / `$ cdist config localhost`: ``` ERROR: localhost: Command failed: '/bin/sh -e /tmp/tmp8q6105_m/421aa90e079fa326b6494f812ad13e79/data/conf/type/__test_type/manifest' return code: 47 ---- BEGIN stdout ---- ---- END stdout ---- Error processing object '__test_type/whatever' =============================================== name: __test_type/whatever path: /tmp/tmp8q6105_m/421aa90e079fa326b6494f812ad13e79/data/object/__test_type/whatever/.cdist-_u9hg_tz source: /tmp/tmp8q6105_m/421aa90e079fa326b6494f812ad13e79/data/conf/manifest/init type: /tmp/tmp8q6105_m/421aa90e079fa326b6494f812ad13e79/data/conf/type/__test_type ---- BEGIN manifest:stderr ---- This is my stderr ---- END manifest:stderr ---- ERROR: cdist: Failed to configure the following hosts: localhost ``` Output with parallelization / `$ cdist config -b -j4 localhost`: ``` ERROR: [28695]: localhost: Command failed: '/bin/sh -e /tmp/tmpvbovdxyv/421aa90e079fa326b6494f812ad13e79/data/conf/type/__test_type/manifest' return code: 47 ---- BEGIN stdout ---- ---- END stdout ---- ERROR: cdist: Failed to configure the following hosts: localhost ``` Note that no error information is shown in this case.

ungleich-gitea added the

bugfix

label 2021-11-20 13:24:28 +00:00

ungleich-gitea closed this issue

2021-11-20 13:24:29 +00:00

ungleich-gitea commented

2021-11-20 15:42:19 +00:00

Created by: darko-poljak

My cdist clone at branch https://github.com/darko-poljak/cdist/tree/bugfix/error_info_lost_with_jobs_%23632

*Created by: darko-poljak* My cdist clone at branch https://github.com/darko-poljak/cdist/tree/bugfix/error_info_lost_with_jobs_%23632

ungleich-gitea commented

2021-11-20 15:42:20 +00:00

Created by: darko-poljak

@AnotherKamila Can you test? cf36aece7d

*Created by: darko-poljak* @AnotherKamila Can you test? https://github.com/darko-poljak/cdist/commit/cf36aece7df7661ac96a30928c6b4e3c5cdf9853

ungleich-gitea commented

2021-11-20 15:42:22 +00:00

When running cdist config -b -j4 -S localhost, the stderr indeed is on stdout:

This is my stderr
ERROR: [7241]: localhost: Command failed: '/bin/sh -e /tmp/tmpewj6rpn0/421aa90e079fa326b6494f812ad13e79/data/conf/type/__test_type/manifest'
return code: 47
---- BEGIN stdout ----
---- END stdout ----
ERROR: cdist: Failed to configure the following hosts: localhost

Note that in the above, not only the stderr, but also the other info about the error is missing -- the whole section "Error processing object '__test_type/whatever'" with path info is not there.

When running `cdist config -b -j4 -S localhost`, the stderr indeed is on stdout: ``` This is my stderr ERROR: [7241]: localhost: Command failed: '/bin/sh -e /tmp/tmpewj6rpn0/421aa90e079fa326b6494f812ad13e79/data/conf/type/__test_type/manifest' return code: 47 ---- BEGIN stdout ---- ---- END stdout ---- ERROR: cdist: Failed to configure the following hosts: localhost ``` Note that in the above, not only the stderr, but also the other info about the error is missing -- the whole section "Error processing object '__test_type/whatever'" with path info is not there.

ungleich-gitea commented

2021-11-20 15:42:23 +00:00

Created by: tom-ee

Might help with narrowing this down: Does the stderr make it (to stdout sic) if the output-stream-capture is disabled with -S?

*Created by: tom-ee* Might help with narrowing this down: Does the stderr make it (to stdout *sic*) if the output-stream-capture is disabled with `-S`?

Sign in to join this conversation.