Compare commits

...

109 commits

Author SHA1 Message Date
74f340ceae [__git] Silence SC2012 as it does not apply to this case. 2020-06-18 12:01:28 +02:00
0ef94550c7 [__git] Fix quoting error, improve readability in various places. 2020-06-18 11:57:27 +02:00
a837797ee3 [__git] Simplify exporters.
Instead of using stat, we can rely on ls.
Thanks to @ssrq for pointing that out.
2020-06-18 11:39:04 +02:00
e23e5ffa3c [__git] Add a --no-updates flag to support old static behaviour.
Users relying on this would probably be better off using tags, but this enables
old behaviour with minor changes in existing manifests.
2020-06-18 11:28:20 +02:00
6e3ef60f89 [__git] Fix quoting in gencode-remote.
Echoing is hard turns out.
2020-06-18 11:26:54 +02:00
db68b04420 [__git] Fix bug introduced in explorers.
I was using explorers in the explorers >,< which is not supported and would be
awkward.
While at it, symlinked group to owner and unified that code.
2020-06-18 11:03:48 +02:00
e13b44bd5d [__git] Smart shellcheck! Fix bug 2020-06-18 00:52:46 +02:00
376fab8a66 [__git] Make shellcheck happy :-) 2020-06-18 00:50:31 +02:00
a40025d842 [__git] Implement onchange triggers and messaging.
Closes #820.
2020-06-18 00:47:16 +02:00
852ac22b13 [__git] Improve manpage regarding tags usage. 2020-06-18 00:29:34 +02:00
8f91e4aedc [__git] Keep repository up to date in subsequent runs.
This also ensure that tags and branch changes are supported.

Closes #819.
2020-06-18 00:23:41 +02:00
05c639e3b7 [__git]: Don't generate code only because owner/group is absent
This closes #821 and improves readability.
2020-06-18 00:12:02 +02:00
14f0780775 [__git]: Change default behaviour for --mode parameter
By default it only applies to the top-level directory, that way any permissions
that are specified in git are respected after cloning.

Old behaviour can be obtained with the new `--mode-recursive` parameter.

Closes #818.
2020-06-17 22:29:54 +02:00
79c3734913 [__git]: Improve understanding with a manifest comment. 2020-06-17 22:27:42 +02:00
d59e2c6440 [__git]: Fix explorers on *BSD
stat's implementation differs between BSD and GNU/Linux Operating Systems.
Tested on FreeBSD, manual-compatibility was asserted for OpenBSD and NetBSD.
2020-06-17 22:17:46 +02:00
Darko Poljak
a6543a72ad ++changelog 2020-06-17 13:40:31 +02:00
d59ba09d71 Merge branch '__download' into 'master'
new type: __download

See merge request ungleich-public/cdist!892
2020-06-17 13:39:07 +02:00
Darko Poljak
cdb998398d Release 6.6.0 2020-06-17 12:10:58 +02:00
201050a9e5 new type: __download 2020-06-16 20:53:31 +03:00
Darko Poljak
5be8437a60 ++changelog 2020-06-13 13:48:34 +02:00
7a48b30d7a Merge branch 'log-server-new-min-py-ver' into 'master'
Log server to capture nested logging output

See merge request ungleich-public/cdist!891
2020-06-13 13:46:05 +02:00
Darko Poljak
59b98091d7 Adapt; update docs and code style 2020-06-13 13:44:01 +02:00
Steven Armstrong
57e352cd1e log server is also usefull for cdist config
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2020-06-13 13:44:01 +02:00
Steven Armstrong
831bfc822b remove unused code
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2020-06-13 13:44:01 +02:00
Steven Armstrong
6e9e9ad557 implement log server to capture nested logging output
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2020-06-13 13:44:01 +02:00
Darko Poljak
eba3d0505b ++changelog 2020-06-12 06:29:39 +02:00
Darko Poljak
eec7ab8e45 Increase minimum supported Python version to 3.5 2020-06-12 06:08:56 +02:00
Darko Poljak
4167f9f60c Use proper format string with name 2020-06-11 14:22:54 +02:00
Darko Poljak
840e417eb7 Fix emulator colored logging 2020-06-11 14:16:37 +02:00
Darko Poljak
74e5d7182a ++changelog 2020-06-10 10:45:20 +02:00
b22e09e1af Merge branch '__clean_path_add_path_param' into 'master'
[__clean_path] add --path parameter

See merge request ungleich-public/cdist!889
2020-06-10 10:44:20 +02:00
7c490a703d [__clean_path] add --path parameter 2020-06-10 11:38:14 +03:00
Darko Poljak
955243a93b Update cdist man page copyright years 2020-06-09 12:51:19 +02:00
Darko Poljak
191f45eb7f ++changelog 2020-06-08 13:48:37 +02:00
Darko Poljak
58f101b8e8 Merge branch 'dheule-fix_os_explorer_sles15' 2020-06-08 13:44:11 +02:00
Darko Poljak
a251e53495 Merge branch 'fix_os_explorer_sles15' of https://github.com/dheule/cdist into dheule-fix_os_explorer_sles15 2020-06-08 13:43:41 +02:00
Darko Poljak
4a81c019e3 Merge branch 'jaakristioja-master' 2020-06-08 13:37:51 +02:00
Jaak Ristioja
7b262c0cec
[docs] Fixed capitalization of URLs in cdist-bootstrap.rst
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
2020-06-08 14:12:03 +03:00
Jaak Ristioja
dc018fdb16
[docs] Fixed typo in cdist-cache.rst
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
2020-06-08 14:12:03 +03:00
Jaak Ristioja
978e249043
[docs] Fixed capitalization of POSIX in cdist-install.rst
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
2020-06-08 14:12:03 +03:00
Jaak Ristioja
e1ff1bfdff
[docs] Fixed two typos in cdist-real-world.rst
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
2020-06-08 14:12:03 +03:00
Jaak Ristioja
46574fc577
[docs] Fixed three typos in cdist-upgrade.rst
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
2020-06-08 14:12:03 +03:00
Jaak Ristioja
3d725f12da
[docs] Fixed a typo and s/posix/POSIX/ in cdist-why.rst
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
2020-06-08 14:12:03 +03:00
Jaak Ristioja
f5630297bd
[docs] Fixed typo in cdist-configuration.rst
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
2020-06-08 14:11:58 +03:00
Darko Poljak
be47619b1e ++changelog 2020-06-08 09:11:51 +02:00
c8a98c02ff Merge branch 'no-color' into 'master'
Respect NO_COLOR environment variable

See merge request ungleich-public/cdist!887
2020-06-08 09:09:50 +02:00
Dennis Camera
89ebd7a4f7 cdist man page: update --colors metavar name 2020-06-07 19:20:10 +02:00
Dennis Camera
23e66e08fa Restrict colored_output value to always/never/auto. 2020-06-06 13:45:31 +02:00
Dennis Camera
7a570f8692 [cdist.cfg.skeleton] Update colored_output documentation based on cdist(1) 2020-06-05 13:59:17 +02:00
Dennis Camera
790c6efae9 Update colored output documentation 2020-06-05 13:56:30 +02:00
Dennis Camera
89e48734bf Let config file and command line override NO_COLOR envvar 2020-06-05 12:23:36 +02:00
Dennis Camera
cdb0d2be41 Patch tests 2020-06-03 23:21:50 +02:00
Dennis Camera
747c6b1076 Respect NO_COLOR environment variable 2020-06-03 22:17:52 +02:00
Daniel Heule
6a611e556a fix os explorer for sles15 2020-06-02 13:40:21 +02:00
Darko Poljak
48d66b0143 ++changelog 2020-06-01 22:25:15 +02:00
fc9ce280f7 Merge branch 'bugfix/sphinx-build-failure' into 'master'
Bugfix/sphinx build failure

Closes #814

See merge request ungleich-public/cdist!885
2020-06-01 22:23:50 +02:00
Darko Poljak
55ebd1a4c5 Fix man build failure in newer sphinx versions 2020-06-01 20:22:40 +02:00
Darko Poljak
9a4e3488c2 ++changelog 2020-06-01 19:17:02 +02:00
3fc36a67a1 Merge branch 'evilham-colored-output' into 'master'
[UX] Add option to enable LogLevel-based coloured output.

See merge request ungleich-public/cdist!879
2020-06-01 19:11:58 +02:00
ba77ea9edc [UX] Add option to enable LogLevel-based coloured output.
This makes it easier for new and experienced users to run cdist with higher
verbosity levels, both to know that things are working as expected and to debug
issues.

Documentation has been modified accordingly and default behaviour is not
changed.
2020-06-01 19:11:58 +02:00
Darko Poljak
988190363a Resolve shellcheck SC1090 2020-05-30 15:10:13 +02:00
Darko Poljak
b354ea6e94 ++changelog 2020-05-30 10:49:13 +02:00
b99f1eda0f Merge branch '__ssh_authorized_keys_remove_unknown' into 'master'
[__ssh_authorized_keys] add --remove-unknown parameter

See merge request ungleich-public/cdist!884
2020-05-30 10:48:02 +02:00
29c0180204 [__ssh_authorized_keys] add --remove-unknown parameter 2020-05-28 23:31:13 +03:00
Darko Poljak
abac79d4a5 Release 6.5.6 2020-05-25 11:16:48 +02:00
Darko Poljak
ba64971a56 ++changelog 2020-05-24 17:05:01 +02:00
06cc20aa28 Merge branch 'bugfix/multiple-log-lines' into 'master'
Fix multiple log lines

Closes #813

See merge request ungleich-public/cdist!883
2020-05-24 17:01:31 +02:00
Darko Poljak
f4e1bbc87e Fix multiple log lines
Fixes #813.
2020-05-24 01:10:56 +02:00
Darko Poljak
6ba73c4be6 ++changelog 2020-05-23 19:03:13 +02:00
15e4b5ee3b Merge branch '__user_remove_freebsd_fix' into 'master'
[__user] fix user delete on freebsd

See merge request ungleich-public/cdist!882
2020-05-23 19:02:31 +02:00
b7d3da443c Merge branch 'jaakristioja-master' into github 2020-05-23 01:15:09 +03:00
716d3554f3 [__user] fix user delete on freebsd 2020-05-23 00:48:35 +03:00
Jaak Ristioja
66f4421089
[docs] Fixed two typos in cdist.cfg.skeleton
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
2020-05-22 17:14:29 +03:00
Darko Poljak
226ed02c1c ++changelog 2020-05-22 10:09:05 +02:00
dab32b0cb6 Merge branch '__group_freebsd_fix' into 'master'
[__group] fix --gid on freebsd

See merge request ungleich-public/cdist!881
2020-05-22 10:08:29 +02:00
d1b73dd42b Merge branch '__motd_fix_for_debians' into 'master'
[__motd] debian|ubuntu|devuan use /etc/motd

See merge request ungleich-public/cdist!880
2020-05-22 10:05:50 +02:00
3bcbd95269 [__motd] debian|ubuntu|devuan use /etc/motd 2020-05-22 02:36:49 +03:00
bf25a18a04 [__group] fix --gid on freebsd 2020-05-22 02:31:38 +03:00
Darko Poljak
f354d80308 ++changelog 2020-05-18 21:03:42 +02:00
cc8dcf682c Merge branch 'evilham-authorized_keys-options-bug' into 'master'
[__ssh_authorized_keys] Fix bug where --option was not multiple

See merge request ungleich-public/cdist!878
2020-05-18 21:03:01 +02:00
ad58ea79c2 [__ssh_authorized_keys] Fix bug where --option was not multiple
This went against both documentation and intent.
2020-05-18 20:01:01 +02:00
Darko Poljak
f9afac4dd6 ++changelog 2020-05-18 19:51:08 +02:00
8d639d54d0 Merge branch 'evilham-authorized_keys-man' into 'master'
[__ssh_authorizedkey{,s}] Improve documentation.

See merge request ungleich-public/cdist!877
2020-05-18 19:49:50 +02:00
2362d89976 Merge branch 'evilham-machine_type-explorer-simplify-linux-add-openbsd' into 'master'
[explorer/machine_type] simplify linux add openbsd

See merge request ungleich-public/cdist!876
2020-05-18 19:48:47 +02:00
c5454afc72 Merge branch 'evilham-machine_type-explorer-freebsd' into 'master'
[explorer/machine_type] Add support for FreeBSD.

See merge request ungleich-public/cdist!875
2020-05-18 19:46:47 +02:00
d5075b49c5 Merge branch 'fix/manpages' into 'master'
Fix some type manpages

See merge request ungleich-public/cdist!874
2020-05-18 19:45:12 +02:00
6d502f737a [__ssh_authorizedkey{,s}] Improve documentation. 2020-05-18 18:37:16 +02:00
8b790b0a54 [explorer/machine_type] Make shellcheck happy! 2020-05-18 16:47:20 +02:00
cf44c4a01b [explorer/machine_type] Simplify Linux + basic OpenBSD support.
By abstracting away vendor-dependent pattern matching for the linux code, we can
re-use that and be reasonably sure about OpenBSD machines being virtualised when
we can identify the undelrying virtualisation technology.
It remains to be solved how to tell if an OpenBSD machine is physical; in that
case previous cdist behaviour ("unknown") remains.

For NetBSD something similar to OpenBSD could be done, with different sysctls:
hw.machine and hw.model wary of adding those without testing though, so for
NetBSD previous cdist behaviour ("unknown") remains.
https://netbsd.gw.com/cgi-bin/man-cgi?sysctl+7.i386+NetBSD-9.0
2020-05-18 16:35:50 +02:00
7ca2bfc14a [explorer/machine_type] Add support for FreeBSD.
More research is needed for {Net,Open}BSD support.

Indentation is left as-is for the linux code as I intend to simplify it in a
future MR, this way the diff is minimal.
2020-05-18 16:00:23 +02:00
086e683c99 [__file] add "default values" in manual for group, mode and owner
Because at least the --mode default value may not be expected, the
manual lists the "default values". As they are not explicitly set in the
`__file` type, it is a bit more difficult, but should resolve to
following:
  - the mode comes from the umask set in the cdist code to protect file
    creation and uses the strongest umask possible.
  - the owner and group comes from the ssh user, which should always be
    the root user. (I think this can be swaped, too, but who will?)

At the owner and group parameter, it could be replaced with something
like "the ssh user and group", which would be more correct, but less
understandable.
2020-05-16 16:33:38 +02:00
dcfabf9268 [__cron] add hint for default values of time-related parameters
This should resolve some misunderstanding when leave a time-related
parameter with it's default value.
2020-05-16 14:00:02 +02:00
42f2dceeb1 [__link] fix typo in the manual 2020-05-09 18:40:44 +02:00
Nico Schottelius
6f4649efc6 Reference the new cdist chat on matrix 2020-05-08 16:08:21 +02:00
Nico Schottelius
d4059fd29e [__letsencrypt_cert] whitelist Ubuntu 2020-05-01 15:31:23 +02:00
Nico Schottelius
f58d662b32 [__pyvenv] Switch to python3 -m venv for ubuntu 2020-05-01 15:28:01 +02:00
Darko Poljak
310045d9fb Release 6.5.5 2020-05-01 13:02:00 +02:00
Darko Poljak
250161e42d ++ 2020-04-28 23:08:03 +02:00
888cf54d99 Merge branch 'mute-return_output-warning' into 'master'
[logging] Mute warning on return_output=True when running scripts.

Closes #806

See merge request ungleich-public/cdist!872
2020-04-28 15:00:00 +02:00
ea3bd14d8b [logging] Mute warning on return_output=True when running scripts.
This fixes #806 which contains more information about the issue.

The TL;DR: this warning is not being useful and hinders debugging types because
it creates an innecessary line for each explorer.

An alternative proposal was #807 but was abandoned in favour of just dropping
the warning.
2020-04-28 14:54:51 +02:00
Darko Poljak
515992249d ++changelog 2020-04-27 22:55:57 +02:00
cd0c811d74 Merge branch 'evilham-explorers' into 'master'
[explorers] Improve *BSD support.

See merge request ungleich-public/cdist!869
2020-04-27 22:53:09 +02:00
965829e18a Merge branch 'evilham-cdist.cfg.skeleton' into 'master'
[docs] Improve cdist.cfg.skeleton

See merge request ungleich-public/cdist!868
2020-04-27 22:51:48 +02:00
bd66b6d948 Merge branch 'update_readme' into 'master'
update README

See merge request ungleich-public/cdist!870
2020-04-27 16:48:09 +02:00
b31e13eacf README: add bits about cdist-contrib 2020-04-27 16:30:52 +03:00
56a65518ab README: add participating section 2020-04-27 15:25:43 +03:00
0b3c417aef update README 2020-04-27 15:09:40 +03:00
678df1ec8a [explorers] Improve *BSD support.
cpu_cores and memory did lacked support for other BSDs.
2020-04-27 01:29:37 +02:00
fefc828780 [docs] Improve cdist.cfg.skeleton 2020-04-26 19:06:42 +02:00
70 changed files with 924 additions and 239 deletions

7
README
View file

@ -1,7 +0,0 @@
cdist
-----
cdist is a usable configuration management system.
For the web documentation have a look at https://www.cdi.st/
or at docs/src for reStructuredText manual.

31
README.md Normal file
View file

@ -0,0 +1,31 @@
# cdist
**cdist** is a usable configuration management system.
It adheres to the [**KISS principle**](https://en.wikipedia.org/wiki/KISS_principle)
and is being used in small up to enterprise grade environments.
For more information have a look at [**homepage**](https://cdi.st)
or at **``docs/src``** for manual in **reStructuredText** format.
## Contributing
Merge/Pull requests can be made in both
[upstream **GitLab**](https://code.ungleich.ch/ungleich-public/cdist/merge_requests)
(managed by [**ungleich**](https://ungleich.ch))
and [**GitHub** project](https://github.com/ungleich/cdist/pulls).
Issues can be made and other project management activites happen
[**only in GitLab**](https://code.ungleich.ch/ungleich-public/cdist)
(needs [**ungleich** account](https://account.ungleich.ch)).
For community-maintained types there is
[**cdist-contrib** project](https://code.ungleich.ch/ungleich-public/cdist-contrib).
## Participating
IRC: ``#cdist`` @ freenode
Matrix: ``#cdist:ungleich.ch``
Mattermost: https://chat.ungleich.ch/ungleich/channels/cdist

View file

@ -26,6 +26,7 @@ import hashlib
import cdist.log
import cdist.version
VERSION = cdist.version.VERSION
BANNER = """
@ -48,6 +49,9 @@ REMOTE_EXEC = "ssh -o User=root"
REMOTE_CMDS_CLEANUP_PATTERN = "ssh -o User=root -O exit -S {}"
MIN_SUPPORTED_PYTHON_VERSION = '3.5'
class Error(Exception):
"""Base exception class for this project"""
pass

View file

@ -5,6 +5,7 @@ import logging
import collections
import functools
import cdist.configuration
import cdist.log
import cdist.preos
import cdist.info
@ -125,6 +126,14 @@ def get_parsers():
'value.'),
action='count', default=None)
parser['colored_output'] = argparse.ArgumentParser(add_help=False)
parser['colored_output'].add_argument(
'--colors', metavar='WHEN',
help="Colorize cdist's output based on log level; "
"WHEN is 'always', 'never', or 'auto'.",
action='store', dest='colored_output', required=False,
choices=cdist.configuration.ColoredOutputOption.CHOICES)
parser['beta'] = argparse.ArgumentParser(add_help=False)
parser['beta'].add_argument(
'-b', '--beta',
@ -197,6 +206,13 @@ def get_parsers():
'supported. Without argument CPU count is used by default. '),
action='store', dest='jobs',
const=multiprocessing.cpu_count())
parser['config_main'].add_argument(
'--log-server',
action='store_true',
help=('Start a log server for sub processes to use. '
'This is mainly useful when running cdist nested '
'from a code-local script. Log server is alwasy '
'implicitly started for \'install\' command.'))
parser['config_main'].add_argument(
'-n', '--dry-run',
help='Do not execute code.', action='store_true')
@ -283,6 +299,7 @@ def get_parsers():
'host', nargs='*', help='Host(s) to operate on.')
parser['config'] = parser['sub'].add_parser(
'config', parents=[parser['loglevel'], parser['beta'],
parser['colored_output'],
parser['common'],
parser['config_main'],
parser['inventory_common'],
@ -301,6 +318,7 @@ def get_parsers():
parser['add-host'] = parser['invsub'].add_parser(
'add-host', parents=[parser['loglevel'], parser['beta'],
parser['colored_output'],
parser['common'],
parser['inventory_common']])
parser['add-host'].add_argument(
@ -315,6 +333,7 @@ def get_parsers():
parser['add-tag'] = parser['invsub'].add_parser(
'add-tag', parents=[parser['loglevel'], parser['beta'],
parser['colored_output'],
parser['common'],
parser['inventory_common']])
parser['add-tag'].add_argument(
@ -346,6 +365,7 @@ def get_parsers():
parser['del-host'] = parser['invsub'].add_parser(
'del-host', parents=[parser['loglevel'], parser['beta'],
parser['colored_output'],
parser['common'],
parser['inventory_common']])
parser['del-host'].add_argument(
@ -363,6 +383,7 @@ def get_parsers():
parser['del-tag'] = parser['invsub'].add_parser(
'del-tag', parents=[parser['loglevel'], parser['beta'],
parser['colored_output'],
parser['common'],
parser['inventory_common']])
parser['del-tag'].add_argument(
@ -398,6 +419,7 @@ def get_parsers():
parser['list'] = parser['invsub'].add_parser(
'list', parents=[parser['loglevel'], parser['beta'],
parser['colored_output'],
parser['common'],
parser['inventory_common']])
parser['list'].add_argument(
@ -430,7 +452,7 @@ def get_parsers():
# Shell
parser['shell'] = parser['sub'].add_parser(
'shell', parents=[parser['loglevel']])
'shell', parents=[parser['loglevel'], parser['colored_output']])
parser['shell'].add_argument(
'-s', '--shell',
help=('Select shell to use, defaults to current shell. Used shell'
@ -478,7 +500,12 @@ def handle_loglevel(args):
if hasattr(args, 'quiet') and args.quiet:
args.verbose = _verbosity_level_off
logging.root.setLevel(_verbosity_level[args.verbose])
logging.getLogger().setLevel(_verbosity_level[args.verbose])
def handle_log_colors(args):
if cdist.configuration.ColoredOutputOption.translate(args.colored_output):
cdist.log.CdistFormatter.USE_COLORS = True
def parse_and_configure(argv, singleton=True):
@ -492,6 +519,7 @@ def parse_and_configure(argv, singleton=True):
raise cdist.Error(str(e))
# Loglevels are handled globally in here
handle_loglevel(args)
handle_log_colors(args)
log = logging.getLogger("cdist")

View file

@ -32,6 +32,10 @@ case "$os" in
sysctl -n hw.ncpuonline
;;
"freebsd"|"netbsd")
sysctl -n hw.ncpu
;;
*)
if [ -r /proc/cpuinfo ]; then
cores="$(grep "core id" /proc/cpuinfo | sort | uniq | wc -l)"

View file

@ -2,6 +2,7 @@
#
# 2014 Daniel Heule (hda at sfs.biz)
# 2014 Thomas Oettli (otho at sfs.biz)
# 2020 Evilham (contact at evilham.com)
#
# This file is part of cdist.
#
@ -18,24 +19,66 @@
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# FIXME: other system types (not linux ...)
os=$("$__explorer/os")
if [ -d "/proc/vz" ] && [ ! -d "/proc/bc" ]; then
vendor_string_to_machine_type() {
for vendor in vmware bochs kvm qemu virtualbox bhyve; do
if echo "${1}" | grep -q -i "${vendor}"; then
if [ "${vendor}" = "bochs" ] || [ "${vendor}" = "qemu" ]; then
vendor="kvm"
fi
echo "virtual_by_${vendor}"
exit
fi
done
}
case "$os" in
"freebsd")
# FreeBSD does not have /proc/cpuinfo even when procfs is used.
# Instead there is a sysctl kern.vm_guest.
# Which is 'none' if physical, else the virtualisation.
vm_guest="$(sysctl -n kern.vm_guest 2>/dev/null || true)"
if [ -n "${vm_guest}" ]; then
if [ "${vm_guest}" = "none" ]; then
echo "physical"
exit
fi
echo "virtual_by_${vm_guest}"
exit
fi
;;
"openbsd")
# OpenBSD can also use the sysctl's: hw.vendor or hw.product.
# Note we can be reasonably sure about a machine being virtualised
# as long as we can identify the virtualisation technology.
# But not so much about it being physical...
# Patches are welcome / reach out if you have better ideas.
for sysctl in hw.vendor hw.product; do
# This exits if we can make a reasonable judgement
vendor_string_to_machine_type "$(sysctl -n "${sysctl}")"
done
;;
*)
# Defaulting to linux for compatibility with previous cdist behaviour
if [ -d "/proc/vz" ] && [ ! -d "/proc/bc" ]; then
echo openvz
exit
fi
fi
if [ -e "/proc/1/environ" ] &&
if [ -e "/proc/1/environ" ] &&
tr '\000' '\n' < "/proc/1/environ" | grep -Eiq '^container='; then
echo lxc
exit
fi
fi
if [ -r /proc/cpuinfo ]; then
if [ -r /proc/cpuinfo ]; then
# this should only exist on virtual guest machines,
# tested on vmware, xen, kvm
# tested on vmware, xen, kvm, bhyve
if grep -q "hypervisor" /proc/cpuinfo; then
# this file is aviable in xen guest systems
if [ -r /sys/hypervisor/type ]; then
@ -44,37 +87,23 @@ if [ -r /proc/cpuinfo ]; then
exit
fi
else
if [ -r /sys/class/dmi/id/product_name ]; then
if grep -q -i 'vmware' /sys/class/dmi/id/product_name; then
echo "virtual_by_vmware"
exit
elif grep -q -i 'bochs' /sys/class/dmi/id/product_name; then
echo "virtual_by_kvm"
exit
elif grep -q -i 'virtualbox' /sys/class/dmi/id/product_name; then
echo "virtual_by_virtualbox"
exit
fi
fi
if [ -r /sys/class/dmi/id/sys_vendor ]; then
if grep -q -i 'qemu' /sys/class/dmi/id/sys_vendor; then
echo "virtual_by_kvm"
exit
fi
fi
if [ -r /sys/class/dmi/id/chassis_vendor ]; then
if grep -q -i 'qemu' /sys/class/dmi/id/chassis_vendor; then
echo "virtual_by_kvm"
exit
fi
for vendor_file in /sys/class/dmi/id/product_name \
/sys/class/dmi/id/sys_vendor \
/sys/class/dmi/id/chasis_vendor; do
if [ -r ${vendor_file} ]; then
# This exits if we can make a reasonable judgement
vendor_string_to_machine_type "$(cat "${vendor_file}")"
fi
done
fi
echo "virtual_by_unknown"
exit
else
echo "physical"
exit
fi
else
echo "unknown"
fi
fi
;;
esac
echo "unknown"

View file

@ -29,7 +29,7 @@ case "$os" in
echo "$(sysctl -n hw.memsize)/1024" | bc
;;
"openbsd")
*"bsd")
echo "$(sysctl -n hw.physmem) / 1048576" | bc
;;

View file

@ -143,6 +143,11 @@ case "$uname_s" in
esac
if [ -f /etc/os-release ]; then
# after sles15, suse don't provide an /etc/SuSE-release anymore, but there is almost no difference between sles and opensuse leap, so call it suse
if grep -q ^ID_LIKE=\"suse\" /etc/os-release 2>/dev/null; then
echo suse
exit 0
fi
# already lowercase, according to:
# https://www.freedesktop.org/software/systemd/man/os-release.html
awk -F= '/^ID=/ { if ($2 ~ /^'"'"'(.*)'"'"'$/ || $2 ~ /^"(.*)"$/) { print substr($2, 2, length($2) - 2) } else { print $2 } }' /etc/os-release

View file

@ -18,7 +18,12 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
path="/$__object_id"
if [ -f "$__object/parameter/path" ]
then
path="$( cat "$__object/parameter/path" )"
else
path="/$__object_id"
fi
[ ! -d "$path" ] && exit 0

View file

@ -20,7 +20,12 @@
[ ! -s "$__object/explorer/list" ] && exit 0
path="/$__object_id"
if [ -f "$__object/parameter/path" ]
then
path="$( cat "$__object/parameter/path" )"
else
path="/$__object_id"
fi
pattern="$( cat "$__object/parameter/pattern" )"

View file

@ -10,7 +10,7 @@ DESCRIPTION
-----------
Remove files and directories which match the pattern.
Provided path (as __object_id) must be a directory.
Provided path must be a directory.
Patterns are passed to ``find``'s ``-regex`` - see ``find(1)`` for more details.
@ -29,6 +29,9 @@ pattern
OPTIONAL PARAMETERS
-------------------
path
Path which will be cleaned. Defaults to ``$__object_id``.
exclude
Pattern of files which are excluded from removal.
@ -46,6 +49,11 @@ EXAMPLES
--exclude '.+\(charset\.conf\|security\.conf\)' \
--onchange 'service apache2 restart'
__clean_path apache2-conf-enabled \
--path /etc/apache2/conf-enabled \
--pattern '.+' \
--exclude '.+\(charset\.conf\|security\.conf\)' \
--onchange 'service apache2 restart'
AUTHORS
-------

View file

@ -1,2 +1,3 @@
exclude
onchange
path

View file

@ -21,6 +21,11 @@ command
OPTIONAL PARAMETERS
-------------------
**NOTE**: All time-related parameters (``--minute``, ``--hour``, ``--day_of_month``
``--month`` and ``--day_of_week``) defaults to ``*``, which means to execute it
**always**. If you set ``--hour 0`` to execute the cronjob only at midnight, it
will execute **every** minute in the first hour of the morning all days.
state
Either present or absent. Defaults to present.
minute

View file

@ -0,0 +1,20 @@
#!/bin/sh -e
dst="/$__object_id"
# shellcheck disable=SC2059
cmd="$( printf "$( cat "$__object/parameter/cmd-sum" )" "$dst" )"
sum="$( cat "$__object/parameter/sum" )"
if [ -f "$dst" ]
then
if [ "$( eval "$cmd" )" = "$sum" ]
then
echo 'present'
else
echo 'mismatch'
fi
else
echo 'absent'
fi

View file

@ -0,0 +1,35 @@
#!/bin/sh -e
state_is="$( cat "$__object/explorer/state" )"
if [ "$state_is" = 'present' ]
then
exit 0
fi
url="$( cat "$__object/parameter/url" )"
cmd="$( cat "$__object/parameter/cmd-get" )"
tmp="$( mktemp )"
dst="/$__object_id"
printf "$cmd > %s\n" \
"$url" \
"$tmp"
if echo "$__target_host" | grep -Eq '^[0-9a-fA-F:]+$'
then
target_host="[$__target_host]"
else
target_host="$__target_host"
fi
printf '%s %s %s:%s\n' \
"$__remote_copy" \
"$tmp" \
"$target_host" \
"$dst"
echo "rm -f '$tmp'"

View file

@ -0,0 +1,66 @@
cdist-type__download(7)
=======================
NAME
----
cdist-type__download - Download file to local storage and copy it to target host
DESCRIPTION
-----------
You must use persistent storage in target host for destination file
(``$__object_id``) because it will be used for checksum calculation
in order to decide if file must be downloaded.
REQUIRED PARAMETERS
-------------------
url
URL from which to download the file.
sum
Checksum of downloaded file.
OPTIONAL PARAMETERS
-------------------
cmd-get
Command used for downloading.
Default is ``wget -O- '%s'``.
Command must output to ``stdout``.
cmd-sum
Command used for checksum calculation.
Default is ``md5sum '%s' | awk '{print $1}'``.
Command output and ``--sum`` parameter must match.
EXAMPLES
--------
.. code-block:: sh
__directory /opt/cpma
require='__directory/opt/cpma' \
__download /opt/cpma/cnq3.zip \
--url https://cdn.playmorepromode.com/files/cnq3/cnq3-1.51.zip \
--sum 46da3021ca9eace277115ec9106c5b46
require='__download/opt/cpma/cnq3.zip' \
__unpack /opt/cpma/cnq3.zip \
--move-existing-destination \
--destination /opt/cpma/server
AUTHORS
-------
Ander Punnar <ander-at-kvlt-dot-ee>
COPYING
-------
Copyright \(C) 2020 Ander Punnar. You can redistribute it
and/or modify it under the terms of the GNU General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

View file

@ -0,0 +1 @@
wget -O- '%s'

View file

@ -0,0 +1 @@
md5sum '%s' | awk '{print $1}'

View file

@ -0,0 +1,2 @@
cmd-get
cmd-sum

View file

@ -0,0 +1,2 @@
url
sum

View file

@ -50,13 +50,13 @@ state
create or modify it
group
Group to chgrp to.
Group to chgrp to. Defaults to ``root``.
mode
Unix permissions, suitable for chmod.
Unix permissions, suitable for chmod. Defaults to a very secure ``0600``.
owner
User to chown to.
User to chown to. Defaults to ``root``.
source
If supplied, copy this file from the host running cdist to the target.

View file

@ -0,0 +1,21 @@
#!/bin/sh -e
destination="/$__object_id"
state_should="$(cat "$__object/parameter/state")"
owner="$(cat "$__object/parameter/owner")"
# If the user did not provide an owner, cdist defaults to root.
git_user="${owner:-root}"
# Only do something if we are not removing the repo and it is not the first run
if [ "$state_should" = "present" ] && [ -d "$destination/.git" ]; then
# Whenever possible run git as non-root, see history of CVEs.
branch="$(su -m "$git_user" -c "git -C '$destination' rev-parse --abbrev-ref HEAD")"
if [ "$branch" != "HEAD" ]; then
echo "$branch"
else
# We are using tags
su -m "$git_user" -c "git -C '$destination' describe --always --tags --abbrev=0"
fi
fi

4
cdist/conf/type/__git/explorer/group Normal file → Executable file
View file

@ -2,4 +2,6 @@
destination="/$__object_id/.git"
stat --print "%G" "${destination}" 2>/dev/null || exit 0
# See: cdist/conf/type/__file/explorer/stat
# shellcheck disable=SC2012
ls -ld "$destination" | awk '{ print $4 }'

View file

@ -0,0 +1,30 @@
#!/bin/sh -e
if [ -f "$__object/parameter/no-updates" ]; then
# User requested explicitly not to have updates
exit
fi
destination="/$__object_id"
state_should="$(cat "$__object/parameter/state")"
branch_should="$(cat "$__object/parameter/branch")"
owner="$(cat "$__object/parameter/owner")"
# If the user did not provide an owner, cdist defaults to root.
git_user="${owner:-root}"
# Only do something if we are not removing the repo and it is not the first run
if [ "$state_should" = "present" ] && [ -d "$destination/.git" ]; then
# First fetch the remote
# Whenever possible run git as non-root, see history of CVEs.
su -m "$git_user" -c "git -C '$destination' fetch --quiet"
head="$(su -m "$git_user" -c "git -C '$destination' rev-parse HEAD")"
# Try first to get the latest commit in the remote current branch,
# if it fails try to get the commit for the expected tag name
upstream="$(su -m "$git_user" -c "git -C '$destination' rev-parse '@{u}'" 2>/dev/null ||
su -m "$git_user" -c "git -C '$destination' rev-parse '${branch_should}^{}'")"
if [ "${head}" != "$upstream" ]; then
echo "YES"
fi
fi

4
cdist/conf/type/__git/explorer/owner Normal file → Executable file
View file

@ -2,4 +2,6 @@
destination="/$__object_id/.git"
stat --print "%U" "${destination}" 2>/dev/null || exit 0
# See: cdist/conf/type/__file/explorer/stat
# shellcheck disable=SC2012
ls -ld "$destination" | awk '{ print $3 }'

View file

@ -19,13 +19,19 @@
#
#
branch_is=$(cat "$__object/explorer/branch")
state_is=$(cat "$__object/explorer/state")
owner_is=$(cat "$__object/explorer/owner")
group_is=$(cat "$__object/explorer/group")
needs_update=$(cat "$__object/explorer/needs-update")
if [ -f "$__object/parameter/no-updates" ]; then
no_updates="YES"
fi
state_should=$(cat "$__object/parameter/state")
branch=$(cat "$__object/parameter/branch")
branch_should=$(cat "$__object/parameter/branch")
source=$(cat "$__object/parameter/source")
@ -38,27 +44,70 @@ mode=$(cat "$__object/parameter/mode")
[ -f "$__object/parameter/recursive" ] && recursive='--recurse-submodules' || recursive=''
[ -f "$__object/parameter/shallow" ] && shallow='--depth 1 --shallow-submodules' || shallow=''
if { [ -n "$owner" ] && [ "$owner_is" != "$owner" ]; } || \
{ [ -n "$group" ] && [ "$group_is" != "$group" ]; }; then
needs_chown="YES"
fi
[ "$state_should" = "$state_is" ] \
&& [ "$owner" = "$owner_is" ] \
&& [ "$group" = "$group_is" ] \
&& [ -z "$needs_chown" ] \
&& [ -n "$mode" ] && exit 0
# Whenever possible run git as non-root, see history of CVEs.
if [ -n "$owner" ]; then
git_user="$owner"
else
git_user="root"
fi
case $state_should in
present)
if [ "$state_should" != "$state_is" ]; then
echo git clone --quiet "$recursive" "$shallow" --branch "$branch" "$source" "$destination"
if [ "$git_user" != "root" ]; then
# If we execute git as non-root, it is not obvious that we'll be able
# to create the output directory, so we have to ensure that.
cat << EOF
if [ ! -d '$destination' ]; then
mkdir '$destination'
fi
chown '${owner}:${group}' '$destination'
EOF
fi
if { [ -n "$owner" ] && [ "$owner_is" != "$owner" ]; } || \
{ [ -n "$group" ] && [ "$group_is" != "$group" ]; }; then
# Actually clone the repository
echo "su -m '$git_user' -c \"git clone --quiet $recursive $shallow --branch '$branch_should' '$source' '$destination'\""
echo create >> "$__messages_out"
repo_changed="YES"
elif [ "$branch_should" != "$branch_is" ]; then
# User has changed tag / branch, let's update that
echo "su -m '$git_user' -c \"git checkout --quiet '$branch_should'\""
echo "change $branch_is -> $branch_should" >> "$__messages_out"
repo_changed="YES"
elif [ -z "$no_updates" ] && [ -n "$needs_update" ]; then
# The remote has newer information than our repository.
# Fetch was done in the explorer, here we can just pull
echo "su -m '$git_user' -c \"git -C '$destination' pull --quiet\""
echo update >> "$__messages_out"
repo_changed="YES"
fi
if [ -n "$needs_chown" ]; then
echo chown -R "${owner}:${group}" "$destination"
echo "chown -R '${owner}:${group}'" >> "$__messages_out"
repo_changed="YES"
fi
if [ -f "$__object/parameter/mode-recursive" ]; then
mode_recursive="-R"
fi
if [ -n "$mode" ]; then
echo chmod -R "$mode" "$destination"
echo chmod "${mode_recursive}" "$mode" "$destination"
fi
;;
absent)
# Handled in manifest
# Handled in manifest, except for the change triggers
if [ "$state_should" != "$state_is" ]; then
echo remove >> "$__messages_out"
repo_changed="YES"
fi
;;
*)
@ -66,3 +115,7 @@ case $state_should in
exit 1
;;
esac
if [ -n "$repo_changed" ]; then
cat "$__object/parameter/onchange"
fi

View file

@ -3,12 +3,13 @@ cdist-type__git(7)
NAME
----
cdist-type__git - Get and or keep git repositories up-to-date
cdist-type__git - Get and keep git repositories up-to-date
DESCRIPTION
-----------
This cdist type allows you to clone git repositories
This cdist type allows you to clone git repositories, track specific branches
or tags and keep up to date at your own pace.
REQUIRED PARAMETERS
@ -23,7 +24,7 @@ state
Either "present" or "absent", defaults to "present"
branch
Create this branch by checking out the remote branch of this name
Check out this branch or tag to the target directory.
Default branch is "master"
group
@ -31,16 +32,45 @@ group
mode
Unix permissions, suitable for chmod.
By default it only applies to the top-level directory.
See the mode-recursive parameter as well.
onchange
The code to run if the repository is first-cloned, changes or is removed.
owner
User to chown to.
BOOLEAN PARAMETERS
------------------
mode-recursive
If present and --mode is passed, the given permissions will be applied
recursively to the working directory.
no-updates
If present the repository will not be updated after first check out.
Notice that this does not affect behaviour if you change the target
tag / branch, in which case the type will still ensure the specified one
is checked out.
recursive
Passes the --recurse-submodules flag to git when cloning the repository.
shallow
Sets --depth=1 and --shallow-submodules for cloning repositories with big history.
MESSAGES
--------
change <old_branch> -> <new_branch>
The directory tracks old_branch but should track new_branch, it will be changed
chown -R <owner>:<group>
Changed ownership
create
Freshly created the directory with the repository clone
remove
The directory with the repository exists, but state is absent, it will be removed
update
The repository tracks a branch that has been updated on the remote
EXAMPLES
--------
@ -52,15 +82,21 @@ EXAMPLES
# Checkout cdist, stay on branch 2.1
__git /home/nico/cdist --source git@code.ungleich.ch:ungleich-public/cdist.git --branch 2.1
# If at a later stage you decide to keep up with the development version,
# cdist will change the branch and keep you up to date with:
__git /home/nico/cdist --source git@code.ungleich.ch:ungleich-public/cdist.git --branch master
AUTHORS
-------
Nico Schottelius <nico-cdist--@--schottelius.org>
Evilham <cvs--@--evilham.com>
COPYING
-------
Copyright \(C) 2012 Nico Schottelius. You can redistribute it
Copyright \(C) 2012 Nico Schottelius. 2020 Evilham.
You can redistribute it
and/or modify it under the terms of the GNU General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

View file

@ -32,6 +32,7 @@ mode="$(cat "$__object/parameter/mode")"
case "$state_should" in
present)
# Handled in gencode-remote
:
;;

View file

@ -1,2 +1,4 @@
mode-recursive
no-updates
recursive
shallow

View file

@ -1,5 +1,6 @@
state
branch
group
onchange
owner
mode

View file

@ -88,7 +88,7 @@ if [ "$state" = "present" ]; then
fi
done
if [ "$os" = "freebsd" ]; then
echo pw groupadd "$@" "$name"
echo pw groupadd "$name" "$@"
else
echo groupadd "$@" "$name"
fi

View file

@ -90,6 +90,9 @@ if [ -z "${certbot_fullpath}" ]; then
__package py27-certbot
certbot_fullpath=/usr/local/bin/certbot
;;
ubuntu)
__package certbot
;;
*)
echo "Unsupported os: $os" >&2

View file

@ -18,7 +18,7 @@ source
Specifies the link source.
type
Specifies the link type: Either hard or symoblic.
Specifies the link type: Either hard or symbolic.
OPTIONAL PARAMETERS

View file

@ -22,13 +22,6 @@
os=$(cat "$__global/explorer/os")
case "$os" in
debian|ubuntu|devuan)
# Debian and Ubuntu need to be updated,
# as seen in /etc/init.d/bootlogs
echo "uname -snrvm > /var/run/motd"
echo "cat /etc/motd.tail >> /var/run/motd"
;;
freebsd)
# FreeBSD only updates /etc/motd on boot,
# as seen in /etc/rc.d/motd

View file

@ -33,10 +33,6 @@ os=$(cat "$__global/explorer/os")
case "$os" in
debian|ubuntu|devuan)
# Debian-based systems use /etc/motd.tail as a template
destination=/etc/motd.tail
;;
freebsd)
# FreeBSD uses motd.template to prepend system information on boot
# (this actually only applies starting with version 13,

View file

@ -1,6 +1,7 @@
#!/bin/sh -e
#
# 2016 Darko Poljak (darko.poljak at gmail.com)
# 2020 Nico Schotetlius (nico.schottelius at ungleich.ch)
#
# This file is part of cdist.
#
@ -45,7 +46,7 @@ then
pyvenv=$(cat "$pyvenvparam")
else
case "$os" in
alpine) # no pyvenv on alpine - I assume others will follow
alpine|ubuntu) # no pyvenv on alpine - I assume others will follow
pyvenv="python3 -m venv"
;;
*)

View file

@ -9,7 +9,7 @@ cdist-type__pyvenv - Create or remove python virtual environment
DESCRIPTION
-----------
This cdist type allows you to create or remove python virtual
environment using pyvenv.
environment using pyvenv on python3 -m venv.
It assumes pyvenv is already installed. Concrete package depends
on concrete OS and/or OS version/distribution.
Ensure this for e.g. in your init manifest as in the following example:
@ -76,4 +76,3 @@ COPYING
-------
Copyright \(C) 2016 Darko Poljak. Free use of this software is
granted under the terms of the GNU General Public License v3 or later (GPLv3+).

View file

@ -15,25 +15,27 @@ This type was created to be used by the __ssh_authorized_keys type.
REQUIRED PARAMETERS
-------------------
file
the authorized_keys file to which the given key should be added
The authorized_keys file where the given key should be managed.
key
a string containing the ssh keytype, base 64 encoded key and optional
trailing comment which shall be added to the given authorized_keys file.
The ssh key which shall be managed in this authorized_keys file.
Must be a string containing the ssh keytype, base 64 encoded key and
optional trailing comment which shall be added to the given
authorized_keys file.
OPTIONAL PARAMETERS
-------------------
comment
explicit comment instead of the one which may be trailing the given key
Use this comment instead of the one which may be trailing in the key.
option
an option to set for this authorized_key entry.
An option to set for this authorized_key entry.
Can be specified multiple times.
See sshd(8) for available options.
state
if the given keys should be 'present' or 'absent', defaults to 'present'.
If the managed key should be 'present' or 'absent', defaults to 'present'.
MESSAGES
@ -64,7 +66,7 @@ EXAMPLES
SEE ALSO
--------
:strong:`cdist__ssh_authorized_keys`\ (7), :strong:`sshd`\ (8)
:strong:`cdist-type__ssh_authorized_keys`\ (7), :strong:`sshd`\ (8)
AUTHORS

View file

@ -0,0 +1,9 @@
#!/bin/sh -e
# shellcheck disable=SC1090
file="$( . "$__type_explorer/file" )"
if [ -f "$file" ]
then
cat "$file"
fi

View file

@ -20,42 +20,48 @@ then left to the user to ensure that the file exists and that ownership and
permissions work with ssh.
REQUIRED PARAMETERS
-------------------
REQUIRED MULTIPLE PARAMETERS
----------------------------
key
the ssh key which shall be added to this authorized_keys file.
Must be a string and can be specified multiple times.
An ssh key which shall be managed in this authorized_keys file.
Must be a string containing the ssh keytype, base 64 encoded key and
optional trailing comment which shall be added to the given
authorized_keys file.
Can be specified multiple times.
OPTIONAL PARAMETERS
-------------------
comment
explicit comment instead of the one which may be trailing the given key
Use this comment instead of the one which may be trailing in each key.
file
an alternative destination file, defaults to ~$owner/.ssh/authorized_keys
An alternative destination file, defaults to ~$owner/.ssh/authorized_keys.
option
an option to set for all created authorized_key entries.
An option to set for all authorized_key entries in the key parameter.
Can be specified multiple times.
See sshd(8) for available options.
owner
the user owning the authorized_keys file, defaults to object_id.
The user owning the authorized_keys file, defaults to object_id.
state
if the given keys should be 'present' or 'absent', defaults to 'present'.
If the given keys should be 'present' or 'absent', defaults to 'present'.
BOOLEAN PARAMETERS
------------------
noparent
don't create or change ownership and permissions of the directory containing
the authorized_keys file
Don't create or change ownership and permissions of the directory containing
the authorized_keys file.
nofile
don't manage existence, ownership and permissions of the the authorized_keys
file
Don't manage existence, ownership and permissions of the the authorized_keys
file.
remove-unknown
Remove undefined keys.
EXAMPLES
@ -67,6 +73,12 @@ EXAMPLES
__ssh_authorized_keys root \
--key "$(cat ~/.ssh/id_rsa.pub)"
# same as above, but make sure your key is only key in
# root's authorized_keys file
__ssh_authorized_keys root \
--key "$(cat ~/.ssh/id_rsa.pub)" \
--remove-unknown
# allow key to login as user-name
__ssh_authorized_keys user-name \
--key "ssh-rsa AXYZAAB3NzaC1yc2..."

View file

@ -55,8 +55,12 @@ _cksum() {
echo "$1" | cksum | cut -d' ' -f 1
}
_type_and_key() {
echo "$1" | tr ' ' '\n' | awk '/^(ssh|ecdsa)-[^ ]+/ { printf $1" "; getline; printf $1 }'
}
while read -r key; do
type_and_key="$(echo "$key" | tr ' ' '\n' | awk '/^(ssh|ecdsa)-[^ ]+/ { printf $1" "; getline; printf $1 }')"
type_and_key="$( _type_and_key "$key" )"
object_id="$(_cksum "$file")-$(_cksum "$type_and_key")"
set -- "$object_id"
set -- "$@" --file "$file"
@ -72,3 +76,24 @@ while read -r key; do
# Ensure __ssh_authorized_key does not read stdin
__ssh_authorized_key "$@" < /dev/null
done < "$__object/parameter/key"
if [ -f "$__object/parameter/remove-unknown" ] &&
[ -s "$__object/explorer/keys" ]
then
while read -r key
do
type_and_key="$( _type_and_key "$key" )"
if grep -Fq "$type_and_key" "$__object/parameter/key"
then
continue
fi
__ssh_authorized_key "remove-$( _cksum "$file$key" )" \
--file "$file" \
--key "$key" \
--state absent \
< /dev/null
done \
< "$__object/explorer/keys"
fi

View file

@ -1,2 +1,3 @@
noparent
nofile
remove-unknown

View file

@ -1,5 +1,4 @@
comment
file
option
owner
state

View file

@ -0,0 +1 @@
option

View file

@ -135,10 +135,18 @@ elif [ "$state" = "absent" ]; then
if grep -q "^${name}:" "$__object/explorer/passwd"; then
#user exists, but state != present, so delete it
if [ -f "$__object/parameter/remove-home" ]; then
if [ "$os" = "freebsd" ]; then
printf "pw userdel '%s' -r >/dev/null 2>&1\\n" "${name}"
else
printf "userdel -r '%s' >/dev/null 2>&1\\n" "${name}"
fi
echo "userdel -r" >> "$__messages_out"
else
if [ "$os" = "freebsd" ]; then
printf "pw userdel '%s' >/dev/null 2>&1\\n" "${name}"
else
printf "userdel '%s' >/dev/null 2>&1\\n" "${name}"
fi
echo "userdel" >> "$__messages_out"
fi
fi

View file

@ -29,18 +29,20 @@ import time
import itertools
import tempfile
import multiprocessing
from cdist.mputil import mp_pool_run, mp_sig_handler
import atexit
import shutil
import socket
from cdist.mputil import mp_pool_run, mp_sig_handler
from cdist import core, inventory
from cdist.util.remoteutil import inspect_ssh_mux_opts
import cdist
import cdist.hostsource
import cdist.exec.local
import cdist.exec.remote
import cdist.util.ipaddr as ipaddr
import cdist.configuration
from cdist import core, inventory
from cdist.util.remoteutil import inspect_ssh_mux_opts
def graph_check_cycle(graph):
@ -195,7 +197,6 @@ class Config(object):
@classmethod
def commandline(cls, args):
"""Configure remote system"""
if (args.parallel and args.parallel != 1) or args.jobs:
if args.timestamp:
cdist.log.setupTimestampingParallelLogging()
@ -203,6 +204,7 @@ class Config(object):
cdist.log.setupParallelLogging()
elif args.timestamp:
cdist.log.setupTimestampingLogging()
log = logging.getLogger("config")
# No new child process if only one host at a time.
@ -381,10 +383,16 @@ class Config(object):
If operating in parallel then return tuple (host, True|False, )
so that main process knows for which host function was successful.
"""
log = logging.getLogger(host)
try:
if args.log_server:
# Start a log server so that nested `cdist config` runs
# have a place to send their logs to.
log_server_socket_dir = tempfile.mkdtemp()
cls._register_path_for_removal(log_server_socket_dir)
cdist.log.setupLogServer(log_server_socket_dir, log)
remote_exec, remote_copy, cleanup_cmd = cls._resolve_remote_cmds(
args)
log.debug("remote_exec for host \"{}\": {}".format(

View file

@ -27,6 +27,7 @@ import cdist.argparse
import re
import multiprocessing
import logging
import sys
class Singleton(type):
@ -246,9 +247,33 @@ class LogLevelOption(OptionBase):
return VerbosityOption().translate(val)
class ColoredOutputOption(BooleanOption):
CHOICES = ('always', 'never', 'auto')
DEFAULT = 'auto'
def get_converter(self):
return self.translate
@staticmethod
def translate(val):
if isinstance(val, bool):
return val
elif val == 'always':
return True
elif val == 'never':
return False
elif val == 'auto':
return 'NO_COLOR' not in os.environ and sys.stdout.isatty()
ColoredOutputOption.DEFAULT = ColoredOutputOption.translate(
ColoredOutputOption.DEFAULT)
_ARG_OPTION_MAPPING = {
'beta': 'beta',
'cache_path_pattern': 'cache_path_pattern',
'colored_output': 'colored_output',
'conf_dir': 'conf_dir',
'manifest': 'init_manifest',
'out_path': 'out_path',
@ -294,6 +319,7 @@ class Configuration(metaclass=Singleton):
'remote_shell': StringOption('remote_shell'),
'cache_path_pattern': StringOption('cache_path_pattern'),
'conf_dir': ConfDirOption(),
'colored_output': ColoredOutputOption('colored_output'),
'init_manifest': StringOption('init_manifest'),
'out_path': StringOption('out_path'),
'remote_out_path': StringOption('remote_out_path'),
@ -319,6 +345,7 @@ class Configuration(metaclass=Singleton):
'CDIST_REMOTE_COPY': 'remote_copy',
'CDIST_INVENTORY_DIR': 'inventory_dir',
'CDIST_CACHE_PATH_PATTERN': 'cache_path_pattern',
'CDIST_COLORED_OUTPUT': 'colored_output',
'__cdist_log_level': 'verbosity',
}
ENV_VAR_BOOLEAN_OPTIONS = ('CDIST_BETA', )
@ -327,11 +354,10 @@ class Configuration(metaclass=Singleton):
}
ARG_OPTION_MAPPING = _ARG_OPTION_MAPPING
ADJUST_ARG_OPTION_MAPPING = {
_ARG_OPTION_MAPPING[key]: key for key in _ARG_OPTION_MAPPING
}
ADJUST_ARG_OPTION_MAPPING = {v: k for k, v in _ARG_OPTION_MAPPING.items()}
REQUIRED_DEFAULT_CONFIG_VALUES = {
'GLOBAL': {
'colored_output': 'auto',
'verbosity': 0,
},
}
@ -484,8 +510,7 @@ class Configuration(metaclass=Singleton):
newconfig = self._read_config_file(config_file)
self._update_config_dict(config, newconfig)
# command line config file
if (self.args and 'config_file' in self.args and
self.args['config_file']):
if (self.args and self.args.get('config_file', None)):
newconfig = self._read_config_file(self.args['config_file'])
self._update_config_dict(config, newconfig)
# command line

View file

@ -116,6 +116,10 @@ class Code(object):
if dry_run:
self.env['__cdist_dry_run'] = '1'
if '__cdist_log_server_socket_export' in os.environ:
self.env['__cdist_log_server_socket'] = os.environ[
'__cdist_log_server_socket_export']
def _run_gencode(self, cdist_object, which):
cdist_type = cdist_object.cdist_type
script = os.path.join(self.local.type_path,

View file

@ -119,6 +119,8 @@ class Manifest(object):
'__cdist_log_level': util.log_level_env_var_val(self.log),
'__cdist_log_level_name': util.log_level_name_env_var_val(
self.log),
'__cdist_colored_log': str(
cdist.log.CdistFormatter.USE_COLORS).lower(),
}
if dry_run:

View file

@ -129,6 +129,9 @@ class Emulator(object):
# if invalid __cdist_log_level value
logging.root.setLevel(logging.WARNING)
colored_log = self.env.get('__cdist_colored_log', 'false')
cdist.log.CdistFormatter.USE_COLORS = colored_log == 'true'
self.log = logging.getLogger(self.target_host[0])
def commandline(self):

View file

@ -280,9 +280,6 @@ class Remote(object):
assert isinstance(command, (list, tuple)), (
"list or tuple argument expected, got: %s" % command)
if return_output and stdout is not subprocess.PIPE:
self.log.debug("return_output is True, ignoring stdout")
close_stdout = False
close_stderr = False
if self.save_output_streams:

View file

@ -22,6 +22,7 @@
import fcntl
import logging
import os
import cdist.log
log = logging.getLogger('cdist-flock')

View file

@ -1,7 +1,7 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# 2013 Steven Armstrong (steven-cdist at armstrong.cc)
# 2013-2019 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
@ -22,9 +22,21 @@
import cdist.config
import cdist.core
import cdist.log
class Install(cdist.config.Config):
@classmethod
def onehost(cls, host, host_tags, host_base_path, host_dir_name, args,
parallel, configuration, remove_remote_files_dirs=False):
# Always start log server during cdist install so that nested
# `cdist config` runs have a place to send their logs to.
args.log_server = True
super().onehost(host, host_tags, host_base_path, host_dir_name, args,
parallel, configuration, remove_remote_files_dirs)
def object_list(self):
"""Short name for object list retrieval.
In install mode, we only care about install objects.

View file

@ -2,6 +2,7 @@
# -*- coding: utf-8 -*-
#
# 2010-2013 Nico Schottelius (nico-cdist at schottelius.org)
# 2019-2020 Steven Armstrong
#
# This file is part of cdist.
#
@ -20,9 +21,16 @@
#
#
import logging
import sys
import datetime
import logging
import logging.handlers
import sys
import os
import asyncio
import contextlib
import pickle
import struct
import threading
# Define additional cdist logging levels.
@ -50,9 +58,32 @@ def _trace(msg, *args, **kwargs):
logging.trace = _trace
class DefaultLog(logging.Logger):
class CdistFormatter(logging.Formatter):
USE_COLORS = False
RESET = '\033[0m'
COLOR_MAP = {
'ERROR': '\033[0;31m',
'WARNING': '\033[0;33m',
'INFO': '\033[0;94m',
'VERBOSE': '\033[0;34m',
'DEBUG': '\033[0;90m',
'TRACE': '\033[0;37m',
}
FORMAT = '%(levelname)s: %(message)s'
def __init__(self, fmt):
super().__init__(fmt=fmt)
def format(self, record):
msg = super().format(record)
if self.USE_COLORS:
color = self.COLOR_MAP.get(record.levelname)
if color:
msg = color + msg + self.RESET
return msg
class DefaultLog(logging.Logger):
FORMAT = '%(levelname)s: %(name)s: %(message)s'
class StdoutFilter(logging.Filter):
def filter(self, rec):
@ -64,10 +95,15 @@ class DefaultLog(logging.Logger):
def __init__(self, name):
super().__init__(name)
self.propagate = False
formatter = logging.Formatter(self.FORMAT)
self.addFilter(self)
if '__cdist_log_server_socket' in os.environ:
log_server_socket = os.environ['__cdist_log_server_socket']
socket_handler = logging.handlers.SocketHandler(log_server_socket,
None)
self.addHandler(socket_handler)
else:
formatter = CdistFormatter(self.FORMAT)
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.addFilter(self.StdoutFilter())
@ -82,13 +118,6 @@ class DefaultLog(logging.Logger):
self.addHandler(stdout_handler)
self.addHandler(stderr_handler)
def filter(self, record):
"""Prefix messages with logger name"""
record.msg = self.name + ": " + str(record.msg)
return True
def verbose(self, msg, *args, **kwargs):
self.log(logging.VERBOSE, msg, *args, **kwargs)
@ -110,7 +139,7 @@ class TimestampingLog(DefaultLog):
class ParallelLog(DefaultLog):
FORMAT = '%(levelname)s: [%(process)d]: %(message)s'
FORMAT = '%(levelname)s: [%(process)d]: %(name)s: %(message)s'
class TimestampingParallelLog(TimestampingLog, ParallelLog):
@ -137,4 +166,42 @@ def setupParallelLogging():
logging.setLoggerClass(ParallelLog)
async def handle_log_client(reader, writer):
while True:
chunk = await reader.read(4)
if len(chunk) < 4:
return
data_size = struct.unpack('>L', chunk)[0]
data = await reader.read(data_size)
obj = pickle.loads(data)
record = logging.makeLogRecord(obj)
logger = logging.getLogger(record.name)
logger.handle(record)
def run_log_server(server_address):
# Get a new loop inside the current thread to run the log server.
loop = asyncio.new_event_loop()
loop.create_task(asyncio.start_unix_server(handle_log_client,
server_address))
loop.run_forever()
def setupLogServer(socket_dir, log=logging.getLogger(__name__)):
"""Run a asyncio based unix socket log server in a background thread.
"""
log_server_socket = os.path.join(socket_dir, 'log-server')
log.debug('Starting logging server on: %s', log_server_socket)
os.environ['__cdist_log_server_socket_export'] = log_server_socket
with contextlib.suppress(FileNotFoundError):
os.remove(log_server_socket)
t = threading.Thread(target=run_log_server, args=(log_server_socket,))
# Deamonizing the thread means we don't have to care about stoping it.
# It will die together with the main process.
t.daemon = True
t.start()
setupDefaultLogging()

View file

@ -7,6 +7,7 @@ from docutils.io import FileOutput
from os import path
from sphinx.util.nodes import inline_all_toctrees
from sphinx import addnodes
from sphinx.util import logging
"""
Extension based on sphinx builtin manpage.
@ -15,6 +16,9 @@ from sphinx import addnodes
"""
logger = logging.getLogger(__name__)
class ManualPageTranslator(sphinx.writers.manpage.ManualPageTranslator):
def header(self):
@ -28,7 +32,7 @@ class ManualPageWriter(sphinx.writers.manpage.ManualPageWriter):
def __init__(self, builder):
super().__init__(builder)
self.translator_class = (
self.builder.translator_class or ManualPageTranslator)
self.builder.get_translator_class() or ManualPageTranslator)
class ManualPageBuilder(sphinx.builders.manpage.ManualPageBuilder):
@ -43,7 +47,7 @@ class ManualPageBuilder(sphinx.builders.manpage.ManualPageBuilder):
components=(docwriter,),
read_config_files=True).get_default_values()
self.info(bold('writing... '), nonl=True)
logger.info(bold('writing... '), nonl=True)
for info in self.config.man_pages:
docname, name, description, authors, section = info
@ -54,7 +58,7 @@ class ManualPageBuilder(sphinx.builders.manpage.ManualPageBuilder):
authors = []
targetname = '%s.%s' % (name, section)
self.info(darkgreen(targetname) + ' { ', nonl=True)
logger.info(darkgreen(targetname) + ' { ', nonl=True)
destination = FileOutput(
destination_path=path.join(self.outdir, targetname),
encoding='utf-8')
@ -63,7 +67,7 @@ class ManualPageBuilder(sphinx.builders.manpage.ManualPageBuilder):
docnames = set()
largetree = inline_all_toctrees(self, docnames, docname, tree,
darkgreen, [docname])
self.info('} ', nonl=True)
logger.info('} ', nonl=True)
self.env.resolve_references(largetree, docname, self)
# remove pending_xref nodes
for pendingnode in largetree.traverse(addnodes.pending_xref):
@ -76,7 +80,7 @@ class ManualPageBuilder(sphinx.builders.manpage.ManualPageBuilder):
largetree.settings.section = section
docwriter.write(largetree, destination)
self.info()
logger.info("")
def setup(app):

View file

@ -28,10 +28,12 @@ import argparse
from cdist import test
import cdist.argparse as cap
import logging
import sys
my_dir = op.abspath(op.dirname(__file__))
fixtures = op.join(my_dir, 'fixtures')
interpolation_config_file = op.join(fixtures, "interpolation-test.cfg")
colored_output_default = 'auto'
def newConfigParser():
@ -153,6 +155,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/bin/sh',
'inventory_dir': '',
'cache_path_pattern': '',
'colored_output': colored_output_default,
'conf_dir': '',
'init_manifest': '',
'out_path': '',
@ -184,6 +187,8 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/bin/sh',
'inventory_dir': None,
'cache_path_pattern': None,
'colored_output': cc.ColoredOutputOption.translate(
colored_output_default),
'conf_dir': None,
'init_manifest': None,
'out_path': None,
@ -390,6 +395,7 @@ class ConfigurationTestCase(test.CdistTestCase):
args = argparse.Namespace()
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'verbosity': 0,
},
}
@ -440,6 +446,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/bin/sh',
'inventory_dir': None,
'cache_path_pattern': None,
'colored_output': colored_output_default,
'conf_dir': None,
'init_manifest': None,
'out_path': None,
@ -515,6 +522,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/usr/bin/sh',
'inventory_dir': '/var/db/cdist/inventory',
'cache_path_pattern': None,
'colored_output': colored_output_default,
'conf_dir': ['/opt/cdist', ],
'init_manifest': None,
'out_path': None,
@ -556,6 +564,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/bin/sh',
'inventory_dir': '',
'cache_path_pattern': '',
'colored_output': colored_output_default,
'conf_dir': '',
'init_manifest': '',
'out_path': '',
@ -579,6 +588,8 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/usr/bin/sh',
'inventory_dir': None,
'cache_path_pattern': None,
'colored_output': cc.ColoredOutputOption.translate(
colored_output_default),
'conf_dir': [
'/opt/cdist/conf',
'/usr/local/share/cdist/conf',
@ -623,6 +634,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/bin/sh',
'inventory_dir': '',
'cache_path_pattern': '',
'colored_output': colored_output_default,
'conf_dir': '',
'init_manifest': '',
'out_path': '',
@ -645,6 +657,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'local_shell': '/usr/bin/sh',
'remote_shell': '/usr/bin/sh',
'inventory_dir': '/var/db/cdist/inventory',
'colored_output': colored_output_default,
'conf_dir': '/opt/cdist',
'remote_copy': 'myscp',
'remote_exec': 'myexec',
@ -663,6 +676,8 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/usr/bin/sh',
'inventory_dir': '/var/db/cdist/inventory',
'cache_path_pattern': None,
'colored_output': cc.ColoredOutputOption.translate(
colored_output_default),
'conf_dir': [
'/opt/cdist/conf',
'/usr/local/share/cdist/conf',
@ -694,6 +709,7 @@ class ConfigurationTestCase(test.CdistTestCase):
}
expected_config = {
'GLOBAL': {
'colored_output': colored_output_default,
'verbosity': 0,
},
}
@ -767,6 +783,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/usr/bin/sh',
'inventory_dir': '/opt/sysadmin/cdist/inventory',
'cache_path_pattern': None,
'colored_output': colored_output_default,
'conf_dir': [
'/opt/cdist/conf',
'/usr/local/share/cdist/conf',
@ -865,6 +882,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/usr/bin/sh',
'inventory_dir': '/var/db/cdist/inventory',
'cache_path_pattern': None,
'colored_output': colored_output_default,
'conf_dir': [
'/opt/conf/cdist',
],
@ -964,6 +982,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/usr/bin/sh',
'inventory_dir': '/var/db/cdist/inventory',
'cache_path_pattern': None,
'colored_output': colored_output_default,
'conf_dir': [
'/opt/conf/cdist',
],
@ -1063,6 +1082,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'remote_shell': '/usr/bin/sh',
'inventory_dir': '/var/db/cdist/inventory',
'cache_path_pattern': None,
'colored_output': colored_output_default,
'conf_dir': [
'/opt/conf/cdist',
],
@ -1095,6 +1115,7 @@ class ConfigurationTestCase(test.CdistTestCase):
'beta': True,
'inventory_dir': '/var/db/cdist/inventory',
'cache_path_pattern': None,
'colored_output': colored_output_default,
'conf_dir': [
'/opt/conf/cdist',
],
@ -1125,6 +1146,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'inventory_dir': None,
'colored_output': colored_output_default,
'conf_dir': None,
'verbosity': 0,
},
@ -1148,6 +1170,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'verbosity': cap.VERBOSE_DEBUG,
},
}
@ -1185,6 +1208,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'save_output_streams': True,
'verbosity': 0,
},
@ -1213,6 +1237,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'save_output_streams': False,
'verbosity': 0,
},
@ -1241,6 +1266,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'save_output_streams': False,
'verbosity': 0,
},
@ -1269,6 +1295,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'save_output_streams': False,
'verbosity': 0,
},
@ -1308,6 +1335,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'timestamp': True,
'verbosity': 0,
},
@ -1336,6 +1364,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'timestamp': True,
'verbosity': 0,
},
@ -1364,6 +1393,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'timestamp': False,
'verbosity': 0,
},
@ -1392,6 +1422,7 @@ class ConfigurationTestCase(test.CdistTestCase):
expected_config_dict = {
'GLOBAL': {
'colored_output': colored_output_default,
'timestamp': False,
'verbosity': 0,
},

View file

@ -13,17 +13,28 @@
# Specify cache path pattern.
# cache_path_pattern = %h
#
# colored_output
# Colorize cdist's output. If enabled, cdist will use different colors for
# different log levels.
# Recognized values are 'always', 'never', and 'auto'.
# If the value is 'auto', colors are enabled if stdout is a TTY unless
# the NO_COLOR (https://no-color.org/) environment variable is defined.
# colored_output = auto
#
# conf_dir
# List of configuration directories separated with the character conventionally
# used by the operating system to separate search path components (as in PATH),
# such as ':' for POSIX or ';' for Windows.
# If also specified at command line then values from command line are
# appended to this value.
# Notice that this works in a "last one wins" fashion, so if a type is redefined
# in multiple conf_dirs, the last one in which it is defined will be used.
# Consider using a unique prefix for your own roles if this can be an issue.
# conf_dir = <dir1>:<dir2>
#
# init_manifest
# Specify default initial manifest.
# init_mainfest = <path-to-init-manifst>
# init_manifest = <path-to-init-manifest>
#
# inventory_dir
# Specify inventory directory.

View file

@ -2,6 +2,33 @@ Changelog
---------
next:
* New type: __download (Ander Punnar)
6.6.0: 2020-06-17
* Type __ssh_authorized_keys: Add option for removing undefined keys (Ander Punnar)
* Core: Support colored log output (Evil Ham)
* Core: Tune colored log output and respect NO_COLOR env var (Dennis Camera)
* Documentation: Fix failing man pages build with newer sphinx versions (Darko Poljak)
* Documentation: Fix trivial grammatical mistakes (Jaak Ristioja)
* Explorer os: Fix for sles15 (Daniel Heule)
* Type __clean_path: Add --path parameter (Ander Punnar)
* Core: Increase minimal supported Python version to 3.5 (Darko Poljak)
* Core: Add log server for nested logging (Steven Armstrong)
6.5.6: 2020-05-25
* Type __pyvenv: Switch to python3 -m venv for Ubuntu (Nico Schottelius)
* Type __letsencrypt_cert: Whitelist Ubuntu (Nico Schottelius)
* Types __cron, __file, __link: Improve manpages (Matthias Stecher)
* Explorer machine_type: Add support for FreeBSD and OpenBSD, and simplify Linux code (Evil Ham)
* Type __ssh_authorized_key, __ssh_authorized_keys: Improve manpages (Evil Ham)
* Type __ssh_authorized_keys: Fix bug where --option was not multiple (Evil Ham)
* Type __motd: Debian/Ubuntu/Devuan use /etc/motd (Ander Punnar)
* Type __group: Fix --gid on FreeBSD (Ander Punnar)
* Configuration: Fix typos in cdist.cfg.skeleton (Jaak Ristioja)
* Type __user: Fix user deletion on FreeBSD (Ander Punnar)
* Core: Fix double log lines (Darko Poljak)
6.5.5: 2020-05-01
* Core: Fix XDG_CONFIG_HOME config file location (Joachim Desroches)
* Type __postgres_database: Add encoding, lc-collate, lc-ctype, template parameters (Timothée Floure)
* Type __motd: Improve documentation and support for FreeBSD (Evil Ham)
@ -10,6 +37,9 @@ next:
* New type: __pf_apply_anchor (Kamila Součková, Evil Ham)
* Type __pf_ruleset: Refactor (Kamila Součková, Evil Ham)
* Type __pf_apply: Deprecate type (Kamila Součková, Evil Ham)
* Configuration: Add notes to cdist.cfg.skeleton (Evil Ham)
* Explorers cpu_cores, memory: Improve *BSD support (Evil Ham)
* Core: Remove debug logging noise (Evil Ham)
6.5.4: 2020-04-11
* Explorer init: Do not grep on non-existent init (Steven Armstrong)

View file

@ -25,7 +25,7 @@ people, have a look at `cdist best practice <cdist-best-practice.html>`_.
Setup working directory and branch
----------------------------------
I assume you have a fresh copy of the cdist tree in ~/cdist, cloned from
one of the official urls (see `cdist quickstart <cdist-quickstart.html>`_ if you don't).
one of the official URLs (see `cdist quickstart <cdist-quickstart.html>`_ if you don't).
Entering the command "git branch" should show you "* master", which indicates
you are on the **master** branch.

View file

@ -59,7 +59,7 @@ typeorder
Object cache overview
~~~~~~~~~~~~~~~~~~~~~
Each object under :strong:`object` directory has its own structurue.
Each object under :strong:`object` directory has its own structure.
code-local
code generated from gencode-local, present only if something is

View file

@ -21,7 +21,7 @@ precedence. Configuration option value read from source with higher
precedence will overwrite option value, if exists, read from source with
lower precedence. That means that command-line option wins them all.
Users can decide on the local conifguration file location. It can be either
Users can decide on the local configuration file location. It can be either
~/.cdist.cfg or $XDG_CONFIG_HOME/cdist/cdist.cfg. Note that, if both exist,
then ~/.cdist.cfg is used.

View file

@ -9,15 +9,15 @@ Source Host
This is the machine from which you will configure target hosts.
* /bin/sh: A posix like shell (for instance bash, dash, zsh)
* Python >= 3.2
* /bin/sh: A POSIX like shell (for instance bash, dash, zsh)
* Python >= 3.5
* SSH client
* sphinx (for building html docs and/or the man pages)
Target Hosts
~~~~~~~~~~~~
* /bin/sh: A posix like shell (for instance bash, dash, zsh)
* /bin/sh: A POSIX like shell (for instance bash, dash, zsh)
* SSH server
Install cdist

View file

@ -198,7 +198,7 @@ We require package uWSGI present in order to create **/etc/uwsgi/apps-enabled/$u
Installation of uWSGI also creates configuration layout: **/etc/uwsgi/apps-enabled**.
If this directory does not exist then **__file** type would error.
We also use stdin as file content source. For details see `Input from stdin <cdist-type.html#input-from-stdin>`_.
For feading stdin we use here-document (**<<** operator). It allows redirection of subsequent
For feeding stdin we use here-document (**<<** operator). It allows redirection of subsequent
lines read by the shell to the input of a command until a line containing only the delimiter
and a newline, with no blank characters in between (EOF in our case).
@ -546,7 +546,7 @@ we have changed our **wsgi.py** file uWSGI reloads the application.
Our application selects and lists items from **items** table.
Openning application
Opening application
~~~~~~~~~~~~~~~~~~~~
Finally try the application::

View file

@ -222,65 +222,89 @@ __cdist_log_level, __cdist_log_level_name
| TRACE | 5 |
+----------------+-----------------+
Available for: initial manifest, explorer, type manifest, type explorer,
type gencode.
__cdist_colored_log
whether or not cdist's log has colors enabled.
Is set to the string ``true`` if cdist's output is using colors,
otherwise the variable contains the string ``false``.
Available for: initial manifest, explorer, type manifest, type explorer,
type gencode.
__cdist_dry_run
Is set only when doing dry run (``-n`` flag).
Available for: initial manifest, explorer, type manifest, type explorer,
type gencode.
__explorer
Directory that contains all global explorers.
Available for: initial manifest, explorer, type explorer, shell.
__files
Directory that contains content from the "files" subdirectories
from the configuration directories.
Available for: initial manifest, type manifest, type gencode, shell.
__manifest
Directory that contains the initial manifest.
Available for: initial manifest, type manifest, shell.
__global
Directory that contains generic output like explorer.
Available for: initial manifest, type manifest, type gencode, shell.
__messages_in
File to read messages from.
Available for: initial manifest, type manifest, type gencode.
__messages_out
File to write messages.
Available for: initial manifest, type manifest, type gencode.
__object
Directory that contains the current object.
Available for: type manifest, type explorer, type gencode and code scripts.
__object_id
The type unique object id.
Available for: type manifest, type explorer, type gencode and code scripts.
Note: The leading and the trailing "/" will always be stripped (caused by
| Note: The leading and the trailing "/" will always be stripped (caused by
the filesystem database and ensured by the core).
Note: Double slashes ("//") will not be fixed and result in an error.
| Note: Double slashes ("//") will not be fixed and result in an error.
__object_name
The full qualified name of the current object.
Available for: type manifest, type explorer, type gencode.
__target_host
The host we are deploying to. This is primary variable. It's content is
literally the one user passed in.
Available for: explorer, initial manifest, type explorer, type manifest, type gencode, shell.
__target_hostname
The hostname of host we are deploying to. This variable is derived from
**__target_host** (using **socket.getaddrinfo(__target_host)** and then
**socket.gethostbyaddr()**).
Available for: explorer, initial manifest, type explorer, type manifest, type gencode, shell.
__target_fqdn
The fully qualified domain name of the host we are deploying to.
This variable is derived from **__target_host**
(using **socket.getfqdn()**).
Available for: explorer, initial manifest, type explorer, type manifest, type gencode, shell.
__target_host_tags
Comma separated list of target host tags.
Available for: explorer, initial manifest, type explorer, type manifest, type gencode, shell.
__type
Path to the current type.
Available for: type manifest, type gencode.
__type_explorer
Directory that contains the type explorers.
Available for: type explorer.
Environment variables (for writing)
@ -344,6 +368,11 @@ CDIST_INVENTORY_DIR
CDIST_BETA
Enable beta functionalities.
CDIST_COLORED_OUTPUT
Colorize cdist's output. If enabled, cdist will use different colors for
different log levels.
Recognized values are 'always', 'never', and 'auto' (the default).
CDIST_CACHE_PATH_PATTERN
Custom cache path pattern.
eof

View file

@ -3,7 +3,7 @@ Support
Chat
~~~~
Chat with us: `ungleich chat <https://chat.ungleich.ch/ungleich/channels/cdist>`_.
Chat with us on `#cdist:ungleich.ch <https://ungleich.ch/u/projects/open-chat/>`_.
Mailing list
~~~~~~~~~~~~

View file

@ -14,7 +14,7 @@ To upgrade cdist in the current branch use
make man
export MANPATH=$MANPATH:$(pwd -P)/doc/man
If you stay on a version branche (i.e. 1.0, 1.1., ...), nothing should break.
If you stay on a version branch (i.e. 1.0, 1.1., ...), nothing should break.
The master branch on the other hand is the development branch and may not be
working, break your setup or eat the tree in your garden.
@ -61,7 +61,7 @@ After that, you can go back and continue the upgrade:
Update the python package
-------------------------
To upgrade to the lastet version do
To upgrade to the latest version do
.. code-block:: sh
@ -158,7 +158,7 @@ Updating from 1.5 to 1.6
* If you used **\_\_package_apt --preseed**, you need to use the new
type **\_\_debconf_set_selections** instead.
* The **\_\_package** types accepted either --state deinstalled or
--state uninstaaled. Starting with 1.6, it was made consistently
--state uninstalled. Starting with 1.6, it was made consistently
to --state removed.
Updating from 1.3 to 1.5

View file

@ -21,7 +21,7 @@ Not only is shell scripting widely known by system engineers,
but it is also a very powerful language. Here are some features
which make daily work easy:
* Configuration can react dynamicly on explored values
* Configuration can react dynamically on explored values
* High level string manipulation (using sed, awk, grep)
* Conditional support (**if, case**)
* Loop support (**for, while**)
@ -44,7 +44,7 @@ Cdist requires very little on a target system. Even better,
in almost all cases all dependencies are usually fulfilled.
Cdist does not require an agent or high level programming
languages on the target host: it will run on any host that
has a **ssh server running** and a posix compatible shell
has a **ssh server running** and a POSIX compatible shell
(**/bin/sh**). Compared to other configuration management systems,
it does not require to open up an additional port.

View file

@ -15,52 +15,51 @@ SYNOPSIS
cdist banner [-h] [-l LOGLEVEL] [-q] [-v]
cdist config [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-g CONFIG_FILE] [-4]
[-6] [-C CACHE_PATH_PATTERN] [-c CONF_DIR] [-i MANIFEST]
[-j [JOBS]] [-n] [-o OUT_PATH] [-P]
[-R [{tar,tgz,tbz2,txz}]] [-r REMOTE_OUT_PATH]
[--remote-copy REMOTE_COPY] [--remote-exec REMOTE_EXEC]
[-S] [-I INVENTORY_DIR] [-A] [-a] [-f HOSTFILE]
[-p [HOST_MAX]] [-s] [-t]
cdist config [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]
[-g CONFIG_FILE] [-4] [-6] [-C CACHE_PATH_PATTERN]
[-c CONF_DIR] [-i MANIFEST] [-j [JOBS]] [--log-server]
[-n] [-o OUT_PATH] [-P] [-R [{tar,tgz,tbz2,txz}]]
[-r REMOTE_OUT_PATH] [--remote-copy REMOTE_COPY]
[--remote-exec REMOTE_EXEC] [-S] [-I INVENTORY_DIR] [-A]
[-a] [-f HOSTFILE] [-p [HOST_MAX]] [-s] [-t]
[host [host ...]]
cdist install [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-g CONFIG_FILE] [-4]
[-6] [-C CACHE_PATH_PATTERN] [-c CONF_DIR] [-i MANIFEST]
[-j [JOBS]] [-n] [-o OUT_PATH] [-P]
[-R [{tar,tgz,tbz2,txz}]] [-r REMOTE_OUT_PATH]
[--remote-copy REMOTE_COPY] [--remote-exec REMOTE_EXEC]
[-S] [-I INVENTORY_DIR] [-A] [-a] [-f HOSTFILE]
[-p [HOST_MAX]] [-s] [-t]
cdist install [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]
[-g CONFIG_FILE] [-4] [-6] [-C CACHE_PATH_PATTERN]
[-c CONF_DIR] [-i MANIFEST] [-j [JOBS]] [--log-server]
[-n] [-o OUT_PATH] [-P] [-R [{tar,tgz,tbz2,txz}]]
[-r REMOTE_OUT_PATH] [--remote-copy REMOTE_COPY]
[--remote-exec REMOTE_EXEC] [-S] [-I INVENTORY_DIR] [-A]
[-a] [-f HOSTFILE] [-p [HOST_MAX]] [-s] [-t]
[host [host ...]]
cdist inventory [-h] {add-host,add-tag,del-host,del-tag,list} ...
cdist inventory add-host [-h] [-l LOGLEVEL] [-q] [-v] [-b]
[-g CONFIG_FILE] [-I INVENTORY_DIR]
[-f HOSTFILE]
cdist inventory add-host [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]
[-g CONFIG_FILE] [-I INVENTORY_DIR] [-f HOSTFILE]
[host [host ...]]
cdist inventory add-tag [-h] [-l LOGLEVEL] [-q] [-v] [-b]
[-g CONFIG_FILE] [-I INVENTORY_DIR]
[-f HOSTFILE] [-T TAGFILE] [-t TAGLIST]
cdist inventory add-tag [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]
[-g CONFIG_FILE] [-I INVENTORY_DIR] [-f HOSTFILE]
[-T TAGFILE] [-t TAGLIST]
[host [host ...]]
cdist inventory del-host [-h] [-l LOGLEVEL] [-q] [-v] [-b]
cdist inventory del-host [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]
[-g CONFIG_FILE] [-I INVENTORY_DIR] [-a]
[-f HOSTFILE]
[host [host ...]]
cdist inventory del-tag [-h] [-l LOGLEVEL] [-q] [-v] [-b]
cdist inventory del-tag [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]
[-g CONFIG_FILE] [-I INVENTORY_DIR] [-a]
[-f HOSTFILE] [-T TAGFILE] [-t TAGLIST]
[host [host ...]]
cdist inventory list [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-g CONFIG_FILE]
[-I INVENTORY_DIR] [-a] [-f HOSTFILE] [-H] [-t]
cdist inventory list [-h] [-l LOGLEVEL] [-q] [-v] [-b] [--colors WHEN]
[-g CONFIG_FILE] [-I INVENTORY_DIR] [-a] [-f HOSTFILE]
[-H] [-t]
[host [host ...]]
cdist preos [-h] [-l LOGLEVEL] [-q] [-v] [-c CONF_DIR] [-g CONFIG_FILE]
[-L]
cdist preos [-h] [-l LOGLEVEL] [-q] [-v] [-c CONF_DIR] [-g CONFIG_FILE] [-L]
[preos] ...
cdist preos [preos-options] debian [-h] [-l LOGLEVEL] [-q] [-v] [-b] [-a ARCH] [-B]
@ -84,9 +83,10 @@ SYNOPSIS
[-S SCRIPT] [-s SUITE] [-y REMOTE_COPY]
target_dir
cdist shell [-h] [-l LOGLEVEL] [-q] [-v] [-s SHELL]
cdist shell [-h] [-l LOGLEVEL] [-q] [-v] [--colors WHEN] [-s SHELL]
cdist info [-h] [-a] [-c CONF_DIR] [-e] [-F] [-f] [-g CONFIG_FILE] [-t] [pattern]
cdist info [-h] [-a] [-c CONF_DIR] [-e] [-F] [-f] [-g CONFIG_FILE] [-t]
[pattern]
DESCRIPTION
@ -95,7 +95,7 @@ cdist is the frontend executable to the cdist configuration management.
It supports different subcommands as explained below.
It is written in Python so it requires :strong:`python`\ (1) to be installed.
It requires a minimal Python version 3.2.
It requires a minimal Python version 3.5.
GENERAL
-------
@ -104,6 +104,14 @@ All commands accept the following options:
**-h, --help**
Show the help screen.
**--colors WHEN**
Colorize cdist's output. If enabled, cdist will use different colors for
different log levels.
WHEN recognizes the values 'always', 'never', and 'auto' (the default).
If the value is 'auto', colored output is enabled if stdout is a TTY
unless the NO_COLOR (https://no-color.org/) environment variable is defined.
**-l LOGLEVEL, --log-level LOGLEVEL**
Set the specified verbosity level. The levels, in
order from the lowest to the highest, are: ERROR (-1),
@ -194,6 +202,12 @@ Install command is currently in beta.
are supported. Without argument CPU count is used by
default.
**--log-server**
Start a log server for sub processes to use. This is
mainly useful when running cdist nested from a code-
local script. Log server is always implicitly started
for 'install' command.
**-n, --dry-run**
Do not execute code.
@ -673,6 +687,9 @@ The possible keywords and their meanings are as follows:
:strong:`cache_path_pattern`
Specify cache path pattern.
:strong:`colored_output`
Colorize cdist's output. cf. the :code:`--colors` option.
:strong:`conf_dir`
List of configuration directories separated with the character conventionally
used by the operating system to separate search path components (as in PATH),
@ -893,6 +910,9 @@ CDIST_BETA
CDIST_CACHE_PATH_PATTERN
Custom cache path pattern.
CDIST_COLORED_OUTPUT
Colorize cdist's output. cf. the :code:`--colors` option.
CDIST_CONFIG_FILE
Custom configuration file.
@ -955,5 +975,5 @@ such case and display a warning message. An example of such a case:
COPYING
-------
Copyright \(C) 2011-2019 Nico Schottelius. Free use of this software is
Copyright \(C) 2011-2020 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License v3 or later (GPLv3+).

View file

@ -60,10 +60,9 @@ def commandline():
if __name__ == "__main__":
cdistpythonversion = '3.2'
if sys.version < cdistpythonversion:
if sys.version < cdist.MIN_SUPPORTED_PYTHON_VERSION:
print('Python >= {} is required on the source host.'.format(
cdistpythonversion), file=sys.stderr)
cdist.MIN_SUPPORTED_PYTHON_VERSIO), file=sys.stderr)
sys.exit(1)
exit_code = 0