Commit graph

111 commits

Author SHA1 Message Date
Darko Poljak
23fbabe303 Further improve parallel execution. 2017-07-26 17:39:07 +02:00
Darko Poljak
d1a044cc23 multiprocessing.Pool -> concurrent.futures.ProcessPoolExecutor 2017-07-26 12:01:19 +02:00
Darko Poljak
7c7a98d083 pep8 2017-07-25 12:43:54 +02:00
Darko Poljak
0af64c01bf Add -p HOST_MAX argument. 2017-07-25 11:12:18 +02:00
Darko Poljak
060ddc2a17 Merge remote-tracking branch 'ungleich/master' into ssh-mux-sigpipe 2017-07-21 17:23:07 +02:00
Darko Poljak
e2a1519332 Merge inventory from beta branch. 2017-07-20 22:04:44 +02:00
Darko Poljak
a20b7167cd pep8 2017-07-19 07:58:14 +02:00
Darko Poljak
01d7f63fcb Merge remote-tracking branch 'ungleich/master' into ssh-mux-sigpipe 2017-07-19 07:55:19 +02:00
Darko Poljak
126a1812a5 Fix parallel object prepare and run steps. Add nonparallel type marker. 2017-07-15 22:18:20 +02:00
Darko Poljak
9c914308f6 Fix ssh connection multiplexing race condition #542
Increase ControlPersist to 2h.
After host run run ssh mux master exit command.
If custom remote exec/copy is specified then do nothing.
2017-07-06 12:43:17 +02:00
Steven Armstrong
9fe2420547 also log start of config run
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-07-05 16:40:02 +02:00
Steven Armstrong
a1d73f7281 log changing object as Processing instead of Updating
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-07-04 23:49:43 +02:00
Steven Armstrong
f6e1174adb start moving info log messages to verbose
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-07-04 23:49:43 +02:00
Darko Poljak
a37d286d67 Suppress subprocess script output in quiet mode. 2017-07-04 23:49:43 +02:00
Darko Poljak
a722f3c634 3rd iteration of logging cleanup. 2017-07-04 23:49:43 +02:00
Darko Poljak
248656b81f First iteration of logging cleanup. 2017-07-04 23:49:43 +02:00
Darko Poljak
138705e3dc Code improvements. 2017-07-02 00:32:43 +02:00
Darko Poljak
2a9bd77550 Merge custom cache path pattern from beta branch. 2017-07-01 23:59:51 +02:00
Steven Armstrong
93ccf925cd need a way to set remote.base_path from the command line
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-06-27 10:36:26 +02:00
Darko Poljak
e375bd464d Merge remote-tracking branch 'ungleich/master' into the-good-the-bad-and-the-ugly 2017-06-12 13:15:00 +02:00
Darko Poljak
29bebc4af4 Shorten ssh control path.
On macos the path is too long due to long default TMP dir.
2017-05-31 09:55:33 +02:00
Darko Poljak
e5a6599ccb Create mp_pool_run helper function for running in parallel. 2016-12-08 21:48:59 +01:00
Darko Poljak
8776a2ee06 concurrent.futures -> multiprocessing 2016-12-08 17:36:57 +01:00
Darko Poljak
e6b9fc90ba Add log messages. 2016-12-08 14:11:30 +01:00
Darko Poljak
ca3a8ddf67 no clear() in python < 3.3; if only one then do it sequentially 2016-12-08 00:47:07 +01:00
Darko Poljak
1952d43073 Initial try for parallelization. 2016-12-07 19:06:51 +01:00
Darko Poljak
258a6c7f56 Merge branch 'the-good-the-bad-and-the-ugly-gunfight' into the-good-the-bad-and-the-ugly 2016-12-07 18:57:58 +01:00
Darko Poljak
341de216a6 Fix missing vars. 2016-12-07 18:39:43 +01:00
Darko Poljak
6c1b215db8 Begin parallelizing object prepare and run. 2016-12-06 14:27:17 +01:00
Darko Poljak
e6fc74c081 ugly -> bad 2016-12-03 18:12:38 +01:00
Darko Poljak
8c53ce78f5 Started the good, the bad and the ugly - code cleanup. 2016-12-03 10:46:49 +01:00
Darko Poljak
9268062de5 Fix target_host vars in Code. 2016-11-03 13:26:50 +01:00
Darko Poljak
74652cec13 Write more informative warning messages. 2016-09-21 19:22:24 +02:00
Darko Poljak
b5262c850e Exit cleanly in case of non UTF-8 file. 2016-08-21 21:48:21 +02:00
Darko Poljak
7f1e41f769 Move hostfile line processing to new method. 2016-08-19 21:56:24 +02:00
Darko Poljak
7aa4b2d40a Support comments in hostfile, skip empty lines. 2016-08-19 13:37:57 +02:00
Darko Poljak
680806ce76 Fix getting host_name and host_fqdn. 2016-08-17 20:46:56 +02:00
Darko Poljak
2fb551131e Fix error when non-existing host is used. 2016-08-17 20:27:51 +02:00
Darko Poljak
fdf6a6570c Check for beta in scripts/cdist. 2016-08-15 16:01:39 +02:00
Darko Poljak
1c07b63f1d Add -b/--enable-beta option for enabling beta functionalities. 2016-08-15 14:20:35 +02:00
Darko Poljak
f858191976 Merge remote-tracking branch 'ungleich/master' into parallel-global-explorers-execution 2016-08-14 22:27:39 +02:00
Darko Poljak
a4c49201c0 Add jobs option for parallel execution, global explorers first. 2016-08-11 23:54:31 +02:00
Darko Poljak
45298b810c Add more comments. 2016-08-11 00:04:37 +02:00
Darko Poljak
dbcc94ab65 Add derived env vars for target hostname and fqdn. 2016-08-10 23:56:56 +02:00
Darko Poljak
6f28fc2db2 Fix ssh mux socket file error.
ssh ControlPath socket file needs to be unique for each host.
To avoid using ssh ControlPath option placeholders move socket file
to host's temp directory. Since each host has unique temp
directory then, although file name for socket file is fixed, its path
is unique.
2016-07-23 16:13:59 +02:00
Darko Poljak
64efa04599 pep8 2016-07-05 20:44:24 +02:00
Darko Poljak
fa5175fee5 Allow both hosts sources: command line args and file. 2016-05-22 09:45:08 +02:00
Darko Poljak
4fce4a631c Add -f option for reading hosts from file or stdin. 2016-05-22 09:22:39 +02:00
Nico Schottelius
73c77dd2d3 Let core use random .cdist directory for objects
Signed-off-by: Nico Schottelius <nico@freiheit.schottelius.org>
2015-03-05 15:02:26 +01:00
Nico Schottelius
ff00df72ad Merge branch 'master' into no-dot-cdist 2015-03-05 11:08:49 +01:00
Nico Schottelius
0cf170b18e prettier print when requirements are broken
Signed-off-by: Nico Schottelius <nico@freiheit.schottelius.org>
2015-02-10 21:59:39 +01:00
Nico Schottelius
c363fc24de begin to make OBJECT_MARKER dynamic
Signed-off-by: Nico Schottelius <nico@bento.schottelius.org>
2014-05-06 00:28:18 +02:00
Nico Schottelius
16d51b3cf1 backport ignoring install types in config mode from install_integration branch
Signed-off-by: Nico Schottelius <nico@bento.schottelius.org>
2014-01-29 22:40:59 +01:00
Steven Armstrong
fcfd2d0a3c refactor so that there is no interaction between code-local and code-remote
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2013-11-26 16:31:18 +01:00
Steven Armstrong
a3dea70a97 Revert "first generate and execute *-local, then *-remote"
This reverts commit cf22499538266b1b4fac1694254edfd8ba9be68d.
2013-11-26 16:31:18 +01:00
Steven Armstrong
12ef3ca4d2 first generate and execute *-local, then *-remote
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2013-11-26 16:31:17 +01:00
Steven Armstrong
589d50bdba make --dry-run work, fixes #199
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2013-09-12 21:57:22 +02:00
Nico Schottelius
760b7e7afb ++changes(2.3.2)
Signed-off-by: Nico Schottelius <nico@bento.schottelius.org>
2013-08-30 02:55:22 +02:00
Steven Armstrong
3de9b869ac there is no such thing as cdist install in master
remove all traces of it to prevent the ongoing merge issues when using the real thing

Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2013-08-29 21:56:53 +02:00
Nico Schottelius
2403fc59ee refactor commandline: merge into its own class (and add first shell code)
Signed-off-by: Nico Schottelius <nico@bento.schottelius.org>
2013-08-07 09:24:10 +02:00
Nico Schottelius
c9f728e073 lib/cdist => cdist (pypi)
Signed-off-by: Nico Schottelius <nico@brief.schottelius.org>
2012-10-25 17:21:58 +02:00
Renamed from lib/cdist/config.py (Browse further)