Commit Graph

1443 Commits

Author SHA1 Message Date
Darko Poljak 526e1c9a6d
Merge pull request #628 from tom-ee/bugfix/messaging__package_update_index
fix messaging for __package_update_index
2018-02-21 10:03:58 +01:00
Darko Poljak 1f93e7291e Fix consul message and document messaging. 2018-02-21 10:00:25 +01:00
tom a545b10538 add `--state`-parameter and messaging to __package_dpkg
- `--state` allows to remove a deb-package, if `--purge-if-absent` is
  specified the package is purged instead of "only" removed
- messaging was added
- man-page updated accordingly
2018-02-20 16:46:26 +01:00
tom 94c9d11cb8 fix messaging for __package_update_index
The message was printed unconditionally, even if no `apt-get update` had
been running.
2018-02-20 10:20:54 +01:00
Darko Poljak c0dc26a577 Fix consul message. 2018-02-15 22:03:28 +01:00
Darko Poljak 48a0fb5bd0 Add message when adding consule directly on target. 2018-02-15 21:34:54 +01:00
Darko Poljak 4d39b6af51 Add -4 and -6 params to force IPv4, IPv6 addresses respectively. 2018-02-15 20:33:36 +01:00
Darko Poljak 9a832d88b5
Add option for directly downloading on target host. 2018-02-15 15:45:31 +01:00
Darko Poljak 0074bb0564 Fix minor docs formatting. 2018-02-14 20:07:44 +01:00
Darko Poljak eb94d7a8bb
Remove update index type dep, call update index after adding new source. (#623) 2018-02-12 12:13:38 +01:00
Darko Poljak a993e0f5a9
Support disabling saving output streams 2018-02-07 18:12:15 +01:00
Darko Poljak 47399bfa9f Add --renew-hook param to letsencrypt_cert type. 2018-02-05 13:57:53 +01:00
Darko Poljak fe7da4d8b3 Update man page. 2018-01-31 20:21:48 +01:00
Darko Poljak d6ff077156 __package_apt ignores --version parameter #615 2018-01-31 20:18:11 +01:00
Darko Poljak 87558adbda
Merge pull request #620 from thriqon/130-issue-stdin
Allow reading contents of __motd and __issue from stdin
2018-01-31 20:12:10 +01:00
Darko Poljak 4d4d76bb38
Merge pull request #617 from tom-ee/maxage-for__package_update_index
Maxage for  package update index
2018-01-31 13:57:09 +01:00
Jonas Weber bfde7a6cfa Allow __motd to read from stdin 2018-01-31 12:53:50 +00:00
Jonas Weber 2069650e74 Allow filling __issue from stdin
closes #130
2018-01-31 12:53:50 +00:00
tom fa91dbfcdb (re-) copyright msg 2018-01-31 11:10:20 +01:00
tom 6a6fdac25d add error-message if OS is unsuppored; fix indentation 2018-01-31 10:37:30 +01:00
tom cb21b3679a add missing legal boilerplate to explorer 2018-01-31 09:22:19 +01:00
tom 2dc5550fa4 __package_update_index: update man-page w/ `--magage` and MESSAGING 2018-01-30 16:05:30 +01:00
Thomas Eckert 27d38a28ab __package_update_index: optional `--maxage <seconds>`-parameter for apt
`--maxage 3600` ensures that `apt-get --quiet update` is only done if
the previous run was at least 1 hour ago.

This also adds messaging
2018-01-30 13:48:25 +01:00
Jonas Weber 3545d0157f Provide `--purge-if-absent` to __package_apt
Configuration files are not purged under Debian when the package
is deinstalled. If this parameter is given, they are deleted upon
package deinstallation.
2018-01-30 12:04:19 +01:00
Jonas Weber dd188ca588 Handle stdin to __systemd_unit correctly
When directly piping in the source of a systemd unit file, cdist
errored out. This is due to not propagating the stdin file to the
underlying __config_file type (which already contains code for this
case, but this has to be duplicated in __systemd_unit).

The following example thus works:

    __systemd_unit test.service --source - <<EOF
    [Service]
    ExecStart=/bin/sleep 3600
    EOF
2018-01-21 09:12:45 +00:00
Darko Poljak 9703e0f08e Save output streams.
Implementation is 99% based on Steven's initial implementation.
2018-01-09 09:31:40 +01:00
kamila 00f524773e
__daemontools fixes
make __daemontools more robust and clean up the --install-init-script confusion
2018-01-05 22:04:13 +01:00
Michal Hanula c895be381f When installing packages on freebsd, redirect stdout and stderr to /dev/null instead of closing them.
Some pre/post-install scripts rely on them being open.

(It would be bette to leave them open and show the output, but I didn't
want to change the behaviour)
2018-01-02 18:11:37 +01:00
kamila a657af64a1 __letsencrypt_cert: add nonparallel and make admin-email required (#609) 2017-12-29 17:10:16 +01:00
uqam-fob fc8dded8e9 __line: Support regex beginning with an hyphen (fixes #607) (#608)
If `regex` begins with an hyphen, `grep` treats it as an option
and treats `file` as the regular expression. This leads to `grep`
trying to read from the standard input and making it wait infinitely.

This patch adds the missing argument breaker `--` and allows
`regex` to begin with an hyphen (provided it is called correctly).
2017-12-29 17:09:06 +01:00
lubo 3397bcbf9b __systemd_unit improvements (#606)
* __systemd_unit: Move systemctl detection to manifest

* __systemd_unit: Restart the unit if inactive

Until now, the --restart parameter caused the unit to be restarted
only when the unit file has changed. This commit modifies --restart
behavior so that the unit is also restarted when the unit is inactive.

* __systemd_unit: Do not create unit file when source is empty
2017-12-29 17:06:48 +01:00
lubo 457233f968 __sysctl: Add support for CoreOS (#605) 2017-12-12 08:50:48 +01:00
Steven Armstrong 71b858e467 use parameter defaults for chroot; re-export cdist log level to nested cdist run
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-12-12 00:15:54 +01:00
Darko Poljak eb8922ebcd Remove redundant log.error line. 2017-11-20 08:02:25 +01:00
kamila be7a99210b add --minute to __cron (#603)
We don't want this to run _every_ minute (default for `--minute` is `*`).
2017-11-19 13:31:44 +01:00
uqam-fob 3b6a471119 explorer/memory: Support OpenBSD (#602)
Adds support to detect the amount of memory available on OpenBSD
systems.
2017-11-15 07:31:17 +01:00
Darko Poljak 0bf6af6d22
Skip empty parameter. Fixes #599. (#600) 2017-11-15 07:30:35 +01:00
lubo aec63d42f1 __config_file: Fix onchange command not being executed (#596)
When a config file state changes from present to absent, onchange
command was not being run.

Fixes #595.
2017-11-10 12:22:38 +01:00
Darko Poljak 4f5dddd9d6
Fix for __line containing single quotes: #154. (#594) 2017-11-09 08:20:33 +01:00
_moep_ 4cdc4ea42c update type for Debian Stretch (#593) 2017-11-07 13:26:44 +01:00
lubo b18327c520 Add __systemd_unit type (#589) 2017-11-07 13:25:37 +01:00
Darko Poljak a8c41bfe44 Change warning message for invalid type. 2017-11-02 19:56:27 +01:00
Darko Poljak 782e662a13 Remove invalid type link from runtime conf dir at proper time. 2017-11-02 17:54:14 +01:00
Darko Poljak c14f3b68f4 Warn about invalid type and continue instead of error. 2017-11-02 17:44:47 +01:00
Darko Poljak e6c5563a16 NoSuchTypeError -> InvalidTypeError 2017-11-02 13:57:36 +01:00
kamila 4799e43516 new type: __letsencrypt_cert (#591) 2017-11-01 08:56:12 +01:00
tom 5eb478da95 Add messaging to __ssh_authorized_key. (#590) 2017-10-31 07:56:25 +01:00
Dominique Roux 2cd006de79 F/ccollect create destination (#548) 2017-10-29 17:23:35 +01:00
kamila da8f6efafc fix explorer/os for devuan ascii (#588) 2017-10-20 18:16:06 +02:00
lubo 36302f7541 Add support for CoreOS to __timezone type (#587) 2017-10-19 19:58:34 +02:00
lubo 5c4cda589c Add support for CoreOS to __hostname type (#586) 2017-10-19 19:58:25 +02:00
tom d1c3e2420f Add messaging to __process 2017-09-29 14:48:51 +02:00
Ander Punnar 8b78001c9e fix running scripts with execute bit (#583) 2017-09-29 14:44:44 +02:00
tom 7ee56d099f resolved #444 add messaging to __line-type (#580)
#444 Add messaging to __line-type
2017-09-27 12:19:07 +02:00
Darko Poljak 966bf30bee Fix \n thing. 2017-09-20 21:53:05 +02:00
Darko Poljak ea58cbd171 Better format command error output. 2017-09-20 21:19:19 +02:00
Darko Poljak 764ea49904 Log more data. 2017-09-17 20:30:07 +02:00
Darko Poljak 8883196efb Bugfix for: __ssh_authorized_keys overwrites existing keys #577 (#579)
* Fix a bug where invalid key removes all file entries.

* __ssh_authorized_key: add key validation.
2017-09-17 08:13:05 +02:00
Darko Poljak ac04edc233 Change path removal suitable for integration and normal run. 2017-09-14 09:53:16 +02:00
Darko Poljak 47d72fb83a Create control path only if needed. 2017-09-14 09:27:27 +02:00
Darko Poljak 08bc38dd0f Log ERROR to stderr and rest to stdout. (#576) 2017-09-13 13:06:06 +02:00
Darko Poljak f2614469c5 configuration: converter() -> get_converter() 2017-09-11 22:36:45 +02:00
Darko Poljak f0dc21ec0c __cdist_log_level=<log level int value>; __cdist_log_level_name=<log level name> (#574) 2017-09-11 09:06:47 +02:00
Darko Poljak 1ae5b1732e Fix missing dirs cleanup. 2017-09-10 23:08:21 +02:00
Darko Poljak 519eb60664 Fix exec/{local,remote} tests. Add exec/local test to make test. 2017-09-10 22:05:05 +02:00
kamila 1f00a35795 fixes + go version bump in __prometheus_exporter (#573) 2017-09-09 21:21:15 +02:00
Darko Poljak 3454da076f Add -l/--log-level option. Honor __cdist_loglevel env var. (#572)
Add -l/--log-level option, __cdist_loglevel -> __cdist_log_level; honor __cdist_log_level env var
2017-09-09 21:17:29 +02:00
Darko Poljak f08ac264a0 pep8 2017-09-07 16:37:49 +02:00
Darko Poljak 57f15f9cce Make __cdist_loglevel value more expressive. (#571) 2017-09-07 16:36:23 +02:00
Darko Poljak 2e4c0d3465 Improve dry run logging. 2017-09-07 16:33:22 +02:00
Darko Poljak 97f6517715 Configuration fixes. 2017-09-07 16:13:46 +02:00
Darko Poljak 28d3466e9d Fix logging dry run line. 2017-09-07 12:32:04 +02:00
Darko Poljak cb38354df3 Run cleanup commands in quiet mode for DEBUG, TRACE. 2017-09-07 12:10:17 +02:00
Darko Poljak e0a4fc4ea7 verbose -> info 2017-09-07 11:58:19 +02:00
Darko Poljak dd48c82656 Log 'Processing' line even if dry run. 2017-09-07 11:30:02 +02:00
Darko Poljak 7141e67d51 Merge pull request #570 from darko-poljak/debug-to-cdist_loglevel
__debug -> __cdist_loglevel
2017-09-07 10:09:57 +02:00
Darko Poljak 92488e83d6 __debug -> __cdist_loglevel 2017-09-06 21:59:57 +02:00
Philippe Grégoire b2fb38d1f4
__package_pkg_openbsd: fix pkg_version explorer
The pkg_version explorer will fail to properly detect if a package
is installed when a package's name is present in the description of
a package (as returned by pkg_info).

Currently, trying to install libtool fails due to incorrect parsing:

    $ pkg_info | grep libtool
    libltdl-2.4.2p1     GNU libtool system independent dlopen wrapper

Additionally, trying to install nagios results in the following output:

    $ pkg_info | grep nagios
    nagios-4.0.8p3-chroot host and service monitor
    nagios-web-4.0.8p2-chroot cgis and webpages for nagios

This commit fixes detection by properly parsing pkg_info's output.
Specifically, descriptions are ignored and package names are properly
extracted.
2017-09-04 13:07:49 -04:00
uqam-fob 0dd5411216 __package_pkg_openbsd: fix use of --name (#568)
Fixes the improper use of the file holding the value of the 'name'
option.
2017-09-03 23:27:27 +02:00
Darko Poljak fb3cd98b54 Minor fixes to man page. 2017-09-02 21:38:13 +02:00
kamila b8a1ad0ebd Improve __daemontools on FreeBSD (#567)
Improve __daemontools on freebsd
2017-09-02 21:25:17 +02:00
kamila a9f338b6a5 new type: __prometheus_exporter (#566) 2017-09-02 21:23:48 +02:00
kamila 7564038f04 fixes to __prometheus_{server,alertmanager} (#565) 2017-09-02 21:22:33 +02:00
Darko Poljak 99133219bc Add another test case. 2017-09-02 08:40:23 +02:00
Sven Wick ed55d01b35 Update machine_type (#564)
detect kvm on proxmox
2017-09-01 20:56:48 +02:00
Darko Poljak 5213f343c0 pep8 2017-09-01 14:11:58 +02:00
Darko Poljak 136f2ecd87 Add helpers for cdist config/install integration. (#551)
Implement simple integration API.
2017-09-01 14:08:50 +02:00
Darko Poljak 4af3c0c6d5 Fix empty conf_dir configuration value. 2017-08-30 23:57:47 +02:00
Darko Poljak 2f10353156 Fix configuration bugs. 2017-08-30 23:11:10 +02:00
Darko Poljak bdee7273af Configfile (#559)
Add cdist configuration/config file support.
2017-08-30 23:02:17 +02:00
Darko Poljak eae399b025 Fix removing ssh key that is last one in the file. 2017-08-30 09:27:06 +02:00
Steven Armstrong 3c3b29f710 allow hostnamectl to fail silently
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-08-29 22:50:47 +02:00
Steven Armstrong 0114001627 remember default __remote_{copy,exec} so we can use it ourself in custom __remote_{copy,exec} scripts
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-08-29 22:31:30 +02:00
Steven Armstrong 04c8415060 use __remote_{exec,copy} instead of plain ssh/scp in remote exec scripts
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-08-29 21:49:47 +02:00
Philippe Grégoire 0e3c7545cc
__user_groups: Support OpenBSD
OpenBSD's usermod(8) interface is similary to NetBSD's.
This commit makes __user_groups support it explicitly.

https://man.openbsd.org/usermod.8
http://netbsd.gw.com/cgi-bin/man-cgi?usermod++NetBSD-current
2017-08-27 08:58:16 -04:00
Darko Poljak 6751349d21 Merge pull request #561 from uqam-fob/pkg-openbsd-installurl
__package_pkg_openbsd: support using /etc/installurl
2017-08-27 14:23:11 +02:00
Philippe Grégoire d4f34a3f40
__package_pkg_openbsd: support the empty flavor
Adds support for specifying an empty flavor by passing --flavor "".
2017-08-26 10:59:04 -04:00
Philippe Grégoire 56adfd4dee
__package_pkg_openbsd: support using /etc/installurl
In 6.1, OpenBSD introduced installurl(5), which contains
the URL the sets where installed from during install or upgrade.
The content of this file is used by pkg_add(1) if PKG_PATH is
not defined.

This commit changes the behavior of __package_pkg_openbsd to omit
setting PKG_PATH to a hard-coded value if --pkg_path is not provided.
This, in turn, makes pkg_add(1) use installurl(5).
2017-08-26 09:42:36 -04:00
Mark Verboom 6f04c9bfce Explorer pipes output of apt-key list, which always generates:
Warning: apt-key output should not be parsed (stdout is not a terminal)
on stderr. Redirect stderr of apt-key to /dev/null to prevent output in
cdist run.
2017-08-25 14:31:32 +02:00
Darko Poljak c6d00019a8 Merge pull request #556 from uqam-fob/explorer-init-pgrep
explorer/init/Linux: use pgrep instead of ps
2017-08-25 11:31:42 +02:00
Darko Poljak 11a974aeff Merge pull request #557 from uqam-fob/type-user-explorers
__user: explore with /etc files
2017-08-25 11:30:32 +02:00
Darko Poljak d9cf1590b8 Fix spelling. 2017-08-25 10:50:11 +02:00
Steven Armstrong 4851c0517f inherit log level from cdist install during nested cdist config
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2017-08-24 14:43:26 +02:00
Darko Poljak f368539447 Some pylint cleaning. 2017-08-22 10:58:30 +02:00
Philippe Grégoire 31e5c97c55
use command(1) to get executable's path 2017-08-21 10:51:55 -04:00
Philippe Grégoire 997fdd8ac4
fix typo in group entry extraction 2017-08-20 15:13:01 -04:00
Philippe Grégoire a915baa73b
__user: explore with /etc files
getent(1) is a utility available where Name Service Switch (NSS)
is available. Many modern operating systems support it, but that
may not be the case of all (e.g. embedded systems).

This commit modifies the __user type explorers to check the
traditional files instead of relying solely on the availability
of getent(1).

- Makes the group explorer use /etc/group
- Makes the passwd explorer use /etc/passwd
- Makes the shadow explorer use /etc/shadow

Implementation note

"getent shadow" does not support querying an entry using a uid
since it does not store that information. Since the shadow explorer
uses __object_id, the passwd explorer does not check if __object_id
matches an entry by uid. This behavior ensures consistent, transparent
behavior of the type. The group explorer, on the other hand, handles
group names and uids; like always.
2017-08-20 12:10:54 -04:00
Philippe Grégoire ebe0c0d66d
make sure we continue on errors 2017-08-20 11:12:30 -04:00
Philippe Grégoire 5646a66f6c
explorer/init/Linux: replace ps by pgrep
BusyBox's version of ps does not support the -o option.
On Linux systems, use pgrep -P0 -l to get the name of
pid 1.
2017-08-20 11:06:07 -04:00
Philippe Grégoire 2b9bf3de24
replace grep+sed by awk 2017-08-19 18:19:17 -04:00
Philippe Grégoire e88e9c357f
fix formatting 2017-08-19 16:49:19 -04:00
Philippe Grégoire 69c6de9f9c
explorer/os: get ID from /etc/os-release
/etc/os-release was introduced by systemd[1] and is now
more and more common; even on systems without systemd (e.g. lede).
In addition to detecting the OS based on specific attributes,
this file provides the ID marker to describe the OS.

This commit adds support for OS detection via /etc/os-release.
According to [2], it is already lowercase.

[1] http://0pointer.de/blog/projects/os-release
[2] https://www.freedesktop.org/software/systemd/man/os-release.html
2017-08-19 16:41:50 -04:00
Darko Poljak ceb97fd0ee Fix for mistake in 982bb286f4. 2017-08-19 19:12:30 +02:00
Darko Poljak 982bb286f4 Cleanup options' help and cdist man page. 2017-08-17 08:24:58 +02:00
Philippe Gregoire 61b52ac806 Set default remote_cmds_cleanup_pattern
Fixes an exception when --remote-copy and --remote-exec are used.
2017-08-14 16:13:57 -04:00
Philippe Gregoire 7b0b3d9415 Fix remote cmds cleanup
When --remote-copy and --remote-exec are provided,
args.remote_cmds_cleanup_pattern is not set.

This patches fixes the evaluation of args.remote_cmds_cleanup_pattern
and prevents cdist from throwing an exception when --remote-copy
and --remote-exec are used.
2017-08-14 14:25:19 -04:00
Darko Poljak b6467e716e Merge remote-tracking branch 'ungleich/master' into jobs-locking 2017-08-13 12:27:57 +02:00
Darko Poljak 75fe3272b3 Add file locking for -j parallel execution. 2017-08-13 12:26:20 +02:00
Darko Poljak 87fe52ea61 Default logging level is WARNING. 2017-08-12 23:02:31 +02:00
Darko Poljak 314a931b84 Respect cdist log level in emulator. 2017-08-12 21:40:23 +02:00
Darko Poljak 59782ad64e Fix unit test. 2017-08-11 01:20:45 +02:00
Darko Poljak 2b0e8160cf Merge pull request #543 from darko-poljak/ssh-mux-sigpipe
Fix ssh connection multiplexing race condition #542
2017-08-10 23:42:16 +02:00
Darko Poljak 8bb70055f4 Merge branch 'master' into archiving 2017-08-10 19:11:35 +02:00
Darko Poljak ca1cc0f64a object_id = '/' is invalid 2017-08-10 19:08:44 +02:00
Darko Poljak 68cb13881f Add archiving support. 2017-08-09 16:56:02 +02:00
Nico Schottelius 260303dd14 Fix gpasswd call in __user_groups: order incorrect
From the manpage:

       -a, --add user
           Add the user to the named group.

       -d, --delete user
           Remove the user from the named group.
2017-08-03 15:11:35 +02:00
Darko Poljak 1b0f560608 Add missing args to method call. 2017-07-27 09:51:23 +02:00
Darko Poljak 0a85d913cc Optimize dir transfer. 2017-07-27 09:49:06 +02:00
Darko Poljak 0d734c872d Fix log var assignment. 2017-07-27 09:36:00 +02:00
Darko Poljak 3db987b913 Merge pull request #549 from 4nd3r/__timezone
__timezone: always exit 0
2017-07-27 09:01:15 +02:00
Ander Punnar b7b1101f11 always exit 0 2017-07-27 01:28:26 +03:00
Darko Poljak 23fbabe303 Further improve parallel execution. 2017-07-26 17:39:07 +02:00
Darko Poljak d1a044cc23 multiprocessing.Pool -> concurrent.futures.ProcessPoolExecutor 2017-07-26 12:01:19 +02:00
Darko Poljak 7c7a98d083 pep8 2017-07-25 12:43:54 +02:00
Darko Poljak 32838a096d Refine inventory logging to adhere new cdist logging. 2017-07-25 12:40:12 +02:00
Darko Poljak 0af64c01bf Add -p HOST_MAX argument. 2017-07-25 11:12:18 +02:00
Darko Poljak bb2cc68169 Fix broken text 2017-07-24 19:13:40 +02:00
Darko Poljak 4a54d1420e Remove unwanted fixme 2017-07-24 19:12:43 +02:00
Darko Poljak 9b5220a29c Merge pull request #546 from greendeath/master
Updated to docker-compose version 1.14
2017-07-21 21:49:02 +02:00
Dominique Roux 5ccfec9c18 Updated to docker-compose version 1.14 2017-07-21 21:18:16 +02:00
Darko Poljak 060ddc2a17 Merge remote-tracking branch 'ungleich/master' into ssh-mux-sigpipe 2017-07-21 17:23:07 +02:00
ander 6bfe02094d year is 2017 2017-07-21 10:23:05 +03:00
ander 2c56622eeb check file first 2017-07-21 10:22:54 +03:00
ander c706b9eefb check current timezone before doing anything 2017-07-21 10:19:02 +03:00
Darko Poljak e2a1519332 Merge inventory from beta branch. 2017-07-20 22:04:44 +02:00
Darko Poljak a20b7167cd pep8 2017-07-19 07:58:14 +02:00
Darko Poljak 01d7f63fcb Merge remote-tracking branch 'ungleich/master' into ssh-mux-sigpipe 2017-07-19 07:55:19 +02:00
Darko Poljak 2beb55be54 Remove FIXME comment due to fix appliance 2017-07-19 07:52:57 +02:00
Darko Poljak 126a1812a5 Fix parallel object prepare and run steps. Add nonparallel type marker. 2017-07-15 22:18:20 +02:00
Darko Poljak 31899b2263 Add missing shebangs. 2017-07-15 22:16:25 +02:00