romain-dartigues
/
cdist
forked from ungleich-public/cdist
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
6,754 Commits 111 Branches 0 Tags 19 MiB
Commit Graph

2038 Commits

Author SHA1 Message Date
poljakowski fde5627721 Merge branch '__download_improvements' into 'master' 
__download improvements

See merge request ungleich-public/cdist!895
 2020-07-08 12:33:41 +02:00
ander e906266286 [__download] s/variable/format specification/ 2020-07-08 00:20:55 +03:00
ander 93506d2113 [__download] curl follow redirects 2020-07-08 00:17:12 +03:00
Darko Poljak fe193ecab8 Make code consistent 
* Remove supreflous checking and warning message.
* Fix cache recording.
 2020-07-01 14:08:48 +02:00
poljakowski 727f3dbb03 Merge branch 'fix/type/__user/openbsd-shadow' into 'master' 
__user: Fix shadow explorer for OpenBSD

See merge request ungleich-public/cdist!902
 2020-06-30 22:55:33 +02:00
poljakowski c5a8004c9a Merge branch 'hosts-aliases' into 'master' 
__hosts: add --alias parameter

See merge request ungleich-public/cdist!901
 2020-06-30 22:52:52 +02:00
Dennis Camera 3860f1feea [type/{__file/__directory}] Support setuid,setguid,sticky bits 2020-06-30 15:10:30 +02:00
Dennis Camera 6467ccbdcc [type/__user] Make shellcheck happy 2020-06-30 14:31:11 +02:00
ssrq 9e33a8f42f Merge branch 'master' into 'fix/type/__user/openbsd-shadow' 
# Conflicts:
#   cdist/conf/type/__user/explorer/shadow
 2020-06-30 14:26:23 +02:00
Dennis Camera 999e7b0134 [type/__user] Fix shadow explorer for OpenBSD 2020-06-30 14:23:34 +02:00
Dennis Camera a263fdfe58 [__hosts] Add --alias parameter 
The --alias parameter allows to specify a hostname and multiple aliases on a
single /etc/hosts line.
 2020-06-30 14:05:26 +02:00
ander 99b5dcd8f0 Merge branch 'master' into __download_improvements 2020-06-28 16:57:13 +03:00
ander 996e7fc09c Merge branch 'master' into __download_improvements 2020-06-28 16:55:11 +03:00
ander b6bf90e3f1 [__download] update manual 2020-06-28 16:43:45 +03:00
ander 85614aabd6 [__download] add --download (local|remote), update manual 2020-06-28 16:38:15 +03:00
Darko Poljak 077989e8fd Remove annoying warnings 
Those warnings don't have any specail meaning and usage.
Resolve #825.
 2020-06-27 15:55:57 +02:00
poljakowski 684043bf37 Merge branch 'fix/type/__package_opkg/lock' into 'master' 
__package_opkg: Add locking

See merge request ungleich-public/cdist!896
 2020-06-25 06:31:16 +02:00
poljakowski ceedcd02f2 Merge branch 'feature/type/__locale_system/support-devuan' into 'master' 
__locale_system: "Whitelist" Devuan

See merge request ungleich-public/cdist!900
 2020-06-25 06:24:17 +02:00
Dennis Camera 5364d3bc90 [type/__package_opkg] Implement flock locking if available 2020-06-24 21:06:10 +02:00
Dennis Camera a9778965be [type/__package_opkg] Use mkdir(1) to lock instead of noclobber 
noclobber is potentially unsafe, because it relies on the underlying shell to
implement noclobber in a safe way that avoids race conditions between multiple
processes.
mkdir is safer because it is mandated by POSIX to "fail" if the target already
exists.
 2020-06-24 08:47:22 +02:00
Darko Poljak 49dde11def Remove deprecated __pf_apply 2020-06-24 07:04:32 +02:00
Dennis Camera 3649555f35 [type/__package_opkg] Do not lock execution of code-remote (revert) 
Instead, rely on `nonparallel`.
In any case cdist should never run explorer and code concurrently even if the
dependency graph would allow to do so as it would result in many more
synchronization issues than this one.
 2020-06-22 09:32:57 +02:00
ander 26dfdf37c2 [__download] support multiple checksum formats and download utilities, add --onchange and other minor changes 2020-06-21 23:39:53 +03:00
poljakowski d478bef8a6 Merge branch 'lint/py3-classes' into 'master' 
Consequently use Python 3-style classes

See merge request ungleich-public/cdist!898
 2020-06-21 17:54:56 +02:00
poljakowski e67215f93d Merge branch 'lint/no-python-shebangs' into 'master' 
Remove unnecessary Python shebangs

See merge request ungleich-public/cdist!897
 2020-06-21 17:53:44 +02:00
Dennis Camera 6aae58dea7 [type/__package_opkg] Mark lock variables readonly 2020-06-21 17:35:28 +02:00
Dennis Camera ce07021580 Do not subclass object 2020-06-21 16:53:47 +02:00
Dennis Camera a6a3fb40bf Remove unnecessary Python shebangs 2020-06-21 16:03:09 +02:00
Dennis Camera e79b26a61f [type/__package_opkg] Also lock execution of code-remote 2020-06-21 15:53:01 +02:00
Dennis Camera 97e48be39e [type/__package_opkg] Fix explorer running in parallel 2020-06-21 15:52:57 +02:00
poljakowski d59ba09d71 Merge branch '__download' into 'master' 
new type: __download

See merge request ungleich-public/cdist!892
 2020-06-17 13:39:07 +02:00
ander 201050a9e5 new type: __download 2020-06-16 20:53:31 +03:00
Darko Poljak 59b98091d7 Adapt; update docs and code style 2020-06-13 13:44:01 +02:00
Steven Armstrong 57e352cd1e log server is also usefull for cdist config 
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
 2020-06-13 13:44:01 +02:00
Steven Armstrong 831bfc822b remove unused code 
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
 2020-06-13 13:44:01 +02:00
Steven Armstrong 6e9e9ad557 implement log server to capture nested logging output 
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
 2020-06-13 13:44:01 +02:00
Darko Poljak 4167f9f60c Use proper format string with name 2020-06-11 14:22:54 +02:00
Darko Poljak 840e417eb7 Fix emulator colored logging 2020-06-11 14:16:37 +02:00
ander 7c490a703d [__clean_path] add --path parameter 2020-06-10 11:38:14 +03:00
Darko Poljak a251e53495 Merge branch 'fix_os_explorer_sles15' of https://github.com/dheule/cdist into dheule-fix_os_explorer_sles15 2020-06-08 13:43:41 +02:00
Dennis Camera 23e66e08fa Restrict colored_output value to always/never/auto. 2020-06-06 13:45:31 +02:00
Dennis Camera 89e48734bf Let config file and command line override NO_COLOR envvar 2020-06-05 12:23:36 +02:00
Dennis Camera cdb0d2be41 Patch tests 2020-06-03 23:21:50 +02:00
Dennis Camera 747c6b1076 Respect NO_COLOR environment variable 2020-06-03 22:17:52 +02:00
Daniel Heule 6a611e556a fix os explorer for sles15 2020-06-02 13:40:21 +02:00
Darko Poljak 55ebd1a4c5 Fix man build failure in newer sphinx versions 2020-06-01 20:22:40 +02:00
evilham ba77ea9edc [UX] Add option to enable LogLevel-based coloured output. 
This makes it easier for new and experienced users to run cdist with higher
verbosity levels, both to know that things are working as expected and to debug
issues.

Documentation has been modified accordingly and default behaviour is not
changed.
 2020-06-01 19:11:58 +02:00
Darko Poljak 988190363a Resolve shellcheck SC1090 2020-05-30 15:10:13 +02:00
ander 29c0180204 [__ssh_authorized_keys] add --remove-unknown parameter 2020-05-28 23:31:13 +03:00
Darko Poljak f4e1bbc87e Fix multiple log lines 
Fixes #813.
 2020-05-24 01:10:56 +02:00
ander 716d3554f3 [__user] fix user delete on freebsd 2020-05-23 00:48:35 +03:00
poljakowski dab32b0cb6 Merge branch '__group_freebsd_fix' into 'master' 
[__group] fix --gid on freebsd

See merge request ungleich-public/cdist!881
 2020-05-22 10:08:29 +02:00
ander 3bcbd95269 [__motd] debian|ubuntu|devuan use /etc/motd 2020-05-22 02:36:49 +03:00
ander bf25a18a04 [__group] fix --gid on freebsd 2020-05-22 02:31:38 +03:00
evilham ad58ea79c2 [__ssh_authorized_keys] Fix bug where --option was not multiple 
This went against both documentation and intent.
 2020-05-18 20:01:01 +02:00
poljakowski 8d639d54d0 Merge branch 'evilham-authorized_keys-man' into 'master' 
[__ssh_authorizedkey{,s}] Improve documentation.

See merge request ungleich-public/cdist!877
 2020-05-18 19:49:50 +02:00
poljakowski 2362d89976 Merge branch 'evilham-machine_type-explorer-simplify-linux-add-openbsd' into 'master' 
[explorer/machine_type] simplify linux add openbsd

See merge request ungleich-public/cdist!876
 2020-05-18 19:48:47 +02:00
poljakowski c5454afc72 Merge branch 'evilham-machine_type-explorer-freebsd' into 'master' 
[explorer/machine_type] Add support for FreeBSD.

See merge request ungleich-public/cdist!875
 2020-05-18 19:46:47 +02:00
evilham 6d502f737a [__ssh_authorizedkey{,s}] Improve documentation. 2020-05-18 18:37:16 +02:00
evilham 8b790b0a54 [explorer/machine_type] Make shellcheck happy! 2020-05-18 16:47:20 +02:00
evilham cf44c4a01b [explorer/machine_type] Simplify Linux + basic OpenBSD support. 
By abstracting away vendor-dependent pattern matching for the linux code, we can
re-use that and be reasonably sure about OpenBSD machines being virtualised when
we can identify the undelrying virtualisation technology.
It remains to be solved how to tell if an OpenBSD machine is physical; in that
case previous cdist behaviour ("unknown") remains.

For NetBSD something similar to OpenBSD could be done, with different sysctls:
hw.machine and hw.model wary of adding those without testing though, so for
NetBSD previous cdist behaviour ("unknown") remains.
https://netbsd.gw.com/cgi-bin/man-cgi?sysctl+7.i386+NetBSD-9.0
 2020-05-18 16:35:50 +02:00
evilham 7ca2bfc14a [explorer/machine_type] Add support for FreeBSD. 
More research is needed for {Net,Open}BSD support.

Indentation is left as-is for the linux code as I intend to simplify it in a
future MR, this way the diff is minimal.
 2020-05-18 16:00:23 +02:00
matze 086e683c99 [__file] add "default values" in manual for group, mode and owner 
Because at least the --mode default value may not be expected, the
manual lists the "default values". As they are not explicitly set in the
`__file` type, it is a bit more difficult, but should resolve to
following:
  - the mode comes from the umask set in the cdist code to protect file
    creation and uses the strongest umask possible.
  - the owner and group comes from the ssh user, which should always be
    the root user. (I think this can be swaped, too, but who will?)

At the owner and group parameter, it could be replaced with something
like "the ssh user and group", which would be more correct, but less
understandable.
 2020-05-16 16:33:38 +02:00
matze dcfabf9268 [__cron] add hint for default values of time-related parameters 
This should resolve some misunderstanding when leave a time-related
parameter with it's default value.
 2020-05-16 14:00:02 +02:00
matze 42f2dceeb1 [__link] fix typo in the manual 2020-05-09 18:40:44 +02:00
Nico Schottelius d4059fd29e [__letsencrypt_cert] whitelist Ubuntu 2020-05-01 15:31:23 +02:00
Nico Schottelius f58d662b32 [__pyvenv] Switch to python3 -m venv for ubuntu 2020-05-01 15:28:01 +02:00
evilham ea3bd14d8b [logging] Mute warning on return_output=True when running scripts. 
This fixes #806 which contains more information about the issue.

The TL;DR: this warning is not being useful and hinders debugging types because
it creates an innecessary line for each explorer.

An alternative proposal was #807 but was abandoned in favour of just dropping
the warning.
 2020-04-28 14:54:51 +02:00
evilham 678df1ec8a [explorers] Improve *BSD support. 
cpu_cores and memory did lacked support for other BSDs.
 2020-04-27 01:29:37 +02:00
poljakowski 8074f02bb3 Merge branch 'evilham-pf' into 'master' 
[__pf*] (~) __pf_ruleset (+)__pf_apply_anchor, deprecate __pf_apply

See merge request ungleich-public/cdist!867
 2020-04-26 09:59:24 +02:00
poljakowski 71156258a4 Merge branch 'evilham-postfix' into 'master' 
[__postfix] Automagically support more OSs by not checking too much.

See merge request ungleich-public/cdist!866
 2020-04-26 09:52:08 +02:00
poljakowski c13608fab5 Merge branch 'evilham-openldap_server' into 'master' 
[__openldap_server] Support extra config parameter.

See merge request ungleich-public/cdist!865
 2020-04-26 09:50:21 +02:00
poljakowski b40b95d758 Merge branch 'evilham-motd' into 'master' 
[__motd] Improve documentation and support for FreeBSD

See merge request ungleich-public/cdist!864
 2020-04-26 09:47:54 +02:00
poljakowski 6ec3274598 Merge branch 'postgres-database-args' into 'master' 
Add --template, --encoding, --lc-collate, --lc-support to __postgres_database type

See merge request ungleich-public/cdist!859
 2020-04-26 09:41:30 +02:00
poljakowski 80d204368a Merge branch 'fix-configuration-file-location' into 'master' 
Fix configuration file location

See merge request ungleich-public/cdist!863
 2020-04-26 09:33:38 +02:00
evilham 2928795441 [__pf_ruleset] Fix shellcheck issue. 2020-04-25 14:54:29 +02:00
evilham fefe90e9c9 [__pf*] (~) __pf_ruleset (+)__pf_apply_anchor, deprecate __pf_apply 
__pf_apply the way it exists on cdist is not really useful and __pf_ruleset does
not take advantage of other types as it should, being instead overly complex and
not as reliable.

The new __pf_ruleset is compatible with the previous one, and __pf_apply_anchors
allows for a simple and powerful way of managing pf anchors.
The functionality previously provided by __pf_apply is still possible out of the
box in __pf_ruleset.

These patches were mostly contributed by Kamila Součková and made fit for
upstreaming by Evilham.
 2020-04-25 12:23:27 +02:00
evilham 5981d0a5f1 [__postfix] Automagically support more OSs by not checking too much. 
It is quite likely that the package is going to be called postfix, rather
than trying to have an exhaustive "allow list" for this package, we can
just add special cases for OSs where that is not the case (not aware of any
atm).
 2020-04-25 00:22:28 +02:00
evilham 056c7c5400 [__openldap_server] Support extra config parameter. 
This allows the user to, e.g. manually define ACLs, while this type does not
support that.
 2020-04-25 00:12:24 +02:00
evilham de37b0ce45 [__motd] Improve documentation and support for FreeBSD 
This makes it easier to use the type just by reading the man page and also
treats FreeBSD's MOTD better.
 2020-04-24 20:26:44 +02:00
Darko Poljak 38ccdfda32 Fix newly found shellcheck issues 2020-04-22 23:21:34 +02:00
sparrowhawk 742163e38c Fix configuration file location 2020-04-15 17:10:33 +02:00
fnux c3f924d350 Add --template flag to __postgres_database type 2020-04-14 10:23:08 +02:00
Darko Poljak 704e78322e Use OrderedDict to guarantee order 
Note:
> Changed in version 3.7: Dictionary order is guaranteed to be
> insertion order. This behavior was an implementation detail of
> CPython from 3.6.
 2020-04-11 20:26:20 +02:00
Steven Armstrong e19c1bb1e0 remove duplicates from conf dirs while preserving order 
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
 2020-04-10 21:50:39 +02:00
Steven Armstrong 76d978d3d8 explorer/init: do not grep on non-existent init 
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
 2020-04-10 10:51:17 +02:00
fnux d53077f4e8 Add --encoding, --lc-collate, --lc-support to __postgres_database type 2020-04-06 09:30:01 +02:00
Darko Poljak 890c73f6bd Fix shellcheck issues 2020-04-02 20:18:04 +02:00
Nico Schottelius d034fe9369 [__pyvenv] use python3 -m venv on alpine 
Until python4 comes, this will work
 2020-04-02 18:24:13 +02:00
Nico Schottelius 0f639a9278 Make __cron nonparallel 
It fails due to removal / edit of crontab at the same time

VERBOSE: [18331]: uncloud.ungleich.ch: Running object __cron/pg_dump_cleanup
VERBOSE: [18332]: uncloud.ungleich.ch: Running object __ungleich_packages/server
VERBOSE: [18335]: uncloud.ungleich.ch: Running object __postgres_role/app
INFO: [18335]: uncloud.ungleich.ch: Processing __postgres_role/app
INFO: [18331]: uncloud.ungleich.ch: Processing __cron/pg_dump_cleanup
ERROR: [16451]: uncloud.ungleich.ch: ssh -o User=root -o ControlPath=/tmp/tmp_8eu4oep/s -o ControlMaster=auto -o ControlPersist=2h uncloud.ungleich.ch /bin/sh -c ' export __object=/var/lib/cdist/object/__cron/pg_dumpall_under_day/.cdist-mfd00npk;  export __object_id=pg_dumpall_under_day;/bin/sh -e /var/lib/cdist/object/__cron/pg_dumpall_under_day/.cdist-mfd00npk/code-remote': ['ssh', '-o', 'User=root', '-o', 'ControlPath=/tmp/tmp_8eu4oep/s', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=2h', 'uncloud.ungleich.ch', "/bin/sh -c ' export __object=/var/lib/cdist/object/__cron/pg_dumpall_under_day/.cdist-mfd00npk;  export __object_id=pg_dumpall_under_day;/bin/sh -e /var/lib/cdist/object/__cron/pg_dumpall_under_day/.cdist-mfd00npk/code-remote'"]

Error processing object '__cron/pg_dumpall_under_day'
=====================================================
name: __cron/pg_dumpall_under_day
path: /tmp/tmplaq9cwdh/6318c251013a449595327745daacf3ee/data/object/__cron/pg_dumpall_under_day/.cdist-mfd00npk
source: /tmp/tmplaq9cwdh/6318c251013a449595327745daacf3ee/data/conf/type/__ungleich_postgresql/manifest
type: /home/nico/vcs/cdist/cdist/conf/type/__cron

code-remote:stderr
------------------
crontab: can't move 'postgres.new' to 'postgres': No such file or directory

VERBOSE: [16451]: config: Total processing time for 1 host(s): 13.98031210899353
[18:09] line:~%
 2020-04-02 18:14:09 +02:00
poljakowski 1eb02f908b Merge branch 'fix/__package_update_index' into 'master' 
[__package_update_index] Fix maxage false-positives on brand-new installations

See merge request ungleich-public/cdist!858
 2020-03-29 13:53:55 +02:00
Darko Poljak b25939cdd6 Fix shellcheck 2020-03-27 13:30:52 +01:00
poljakowski 11148995d1 Merge branch 'master' into 'master' 
fix typo

See merge request ungleich-public/cdist!857
 2020-03-27 08:50:29 +01:00
Nico Schottelius 66d9905029 [__consul_agent] make conf_dir depent on the OS 2020-03-26 21:48:17 +01:00
andrewschleifer f00e4af5f0 fix typo 2020-03-26 21:17:32 +08:00
matze 358e04b2af Handle specially if no time about the last index update found. 
The explorer 'currage' now returns -1 if he can not find any value about
this. The gencode-remote script handle this value special to not exit if
-1 given as value.

This fixes the bug
  ungleich-public/cdist#803
 2020-03-14 09:58:38 +01:00
llnu fb32d6ed3f alpine uses a different getent lib which doesnt support: getent shadow 2020-03-08 16:04:02 +01:00
poljakowski 4b24d886eb Merge branch 'file/better-errors' into 'master' 
[__file] Better errors analogous to __directory

See merge request ungleich-public/cdist!854
 2020-02-26 21:50:01 +01:00
Dennis Camera 9eacba06bb [__file] Treat pre-exists as a special case in gencode-remote as it should never get there 2020-02-26 16:48:14 +01:00
Dennis Camera 46d2487f08 [__file] Give more precise error message when --state pre-exists 2020-02-26 16:40:53 +01:00