ungleich-public/cdist
ungleich-public/cdist
7
4
Fork 10
Code Issues 58 Pull requests 15 Projects Releases Wiki Activity

Merge branch 'master' into beta

Browse source
This commit is contained in:
Darko Poljak 2021-07-10 20:52:06 +02:00
commit 94a7975e32
66 changed files with 2019 additions and 516 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -24,8 +24,8 @@ For community-maintained types there is
## Participating
IRC: ``#cdist`` @ freenode
IRC: ``#cdist`` @ [libera](https://libera.chat)
Matrix: ``#cdist:ungleich.ch``
Mattermost: https://chat.ungleich.ch/ungleich/channels/cdist
Matrix and IRC are bridged.

24
cdist/argparse.py
View file

@ -486,19 +486,31 @@ def get_parsers():
parser['scan'].add_argument(
'-m', '--mode', help='Which modes should run',
action='append', default=[],
choices=['scan', 'trigger'])
choices=['scan', 'trigger', 'config'])
parser['scan'].add_argument(
'--list',
action='store_true',
help='List the known hosts and exit')
parser['scan'].add_argument(
'--config',
action='store_true',
help='Try to configure detected hosts')
parser['scan'].add_argument(
'-I', '--interfaces',
action='append', default=[],
'-I', '--interface',
action='append', default=[], required=True,
help='On which interfaces to scan/trigger')
parser['scan'].add_argument(
'-d', '--delay',
action='store', default=3600,
help='How long to wait before reconfiguring after last try')
'--name-mapper',
action='store', default=None,
help='Map addresses to names, required for config mode')
parser['scan'].add_argument(
'-d', '--config-delay',
action='store', default=3600, type=int,
help='How long (seconds) to wait before reconfiguring after last try')
parser['scan'].add_argument(
'-t', '--trigger-delay',
action='store', default=5, type=int,
help='How long (seconds) to wait between ICMPv6 echo requests')
parser['scan'].set_defaults(func=cdist.scan.commandline.commandline)
# Trigger

27
cdist/conf/type/__apt_key/explorer/state
View file

@ -27,18 +27,25 @@ else
keyid="$__object_id"
fi
# From apt-key(8):
# Use of apt-key is deprecated, except for the use of apt-key del in
# maintainer scripts to remove existing keys from the main keyring.
# If such usage of apt-key is desired the additional installation of
# the GNU Privacy Guard suite (packaged in gnupg) is required.
if [ -f "${__object}/parameter/use-deprecated-apt-key" ]; then
if apt-key export "$keyid" | head -n 1 | grep -Fqe "BEGIN PGP PUBLIC KEY BLOCK"
then echo present
else echo absent
fi
exit
fi
keydir="$(cat "$__object/parameter/keydir")"
keyfile="$keydir/$__object_id.gpg"
if [ -d "$keydir" ]
if [ -f "$keyfile" ]
then
if [ -f "$keyfile" ]
then echo present
else echo absent
fi
else
# fallback to deprecated apt-key
apt-key export "$keyid" | head -n 1 | grep -Fqe "BEGIN PGP PUBLIC KEY BLOCK" \
&& echo present \
|| echo absent
echo present
exit
fi
echo absent

53
cdist/conf/type/__apt_key/gencode-remote
View file

@ -25,11 +25,7 @@ else
fi
state_should="$(cat "$__object/parameter/state")"
state_is="$(cat "$__object/explorer/state")"
if [ "$state_should" = "$state_is" ]; then
# nothing to do
exit 0
fi
method="$(cat "$__object/key_method")"
keydir="$(cat "$__object/parameter/keydir")"
keyfile="$keydir/$__object_id.gpg"
@ -37,30 +33,18 @@ keyfile="$keydir/$__object_id.gpg"
case "$state_should" in
present)
keyserver="$(cat "$__object/parameter/keyserver")"
if [ -f "$__object/parameter/uri" ]; then
uri="$(cat "$__object/parameter/uri")"
if [ -d "$keydir" ]; then
cat << EOF
curl -s -L \\
-o "$keyfile" \\
"$uri"
key="\$( cat "$keyfile" )"
if echo "\$key" | grep -Fq 'BEGIN PGP PUBLIC KEY BLOCK'
then
echo "\$key" | gpg --dearmor > "$keyfile"
fi
EOF
else
# fallback to deprecated apt-key
echo "curl -s -L '$uri' | apt-key add -"
# Using __download or __file as key source
# Propagate messages if needed
if [ "${method}" = "uri" ] || [ "${method}" = "source" ]; then
if grep -Eq "^__(file|download)$keyfile" "$__messages_in"; then
echo "added '$keyid'" >> "$__messages_out"
fi
elif [ -d "$keydir" ]; then
exit 0
elif [ "${state_is}" = "present" ]; then
exit 0
fi
# Using key servers to fetch the key
if [ ! -f "$__object/parameter/use-deprecated-apt-key" ]; then
# we need to kill gpg after 30 seconds, because gpg
# can get stuck if keyserver is not responding.
# exporting env var and not exit 1,
@ -100,13 +84,16 @@ EOF
echo "added '$keyid'" >> "$__messages_out"
;;
absent)
if [ -f "$keyfile" ]; then
echo "rm '$keyfile'"
else
# Removal for keys added from a keyserver without this flag
# is done in the manifest
if [ "$state_is" != "absent" ] && \
[ -f "$__object/parameter/use-deprecated-apt-key" ]; then
# fallback to deprecated apt-key
echo "apt-key del \"$keyid\""
echo "removed '$keyid'" >> "$__messages_out"
# Propagate messages if needed
elif grep -Eq "^__file$keyfile" "$__messages_in"; then
echo "removed '$keyid'" >> "$__messages_out"
fi
echo "removed '$keyid'" >> "$__messages_out"
;;
esac

84
cdist/conf/type/__apt_key/man.rst
View file

@ -10,6 +10,14 @@ DESCRIPTION
-----------
Manages the list of keys used by apt to authenticate packages.
This is done by placing the requested key in a file named
``$__object_id.gpg`` in the ``keydir`` directory.
This is supported by modern releases of Debian-based distributions.
In order of preference, exactly one of: ``source``, ``uri`` or ``keyid``
must be specified.
REQUIRED PARAMETERS
-------------------
@ -18,21 +26,49 @@ None.
OPTIONAL PARAMETERS
-------------------
keydir
keyring directory, defaults to ``/etc/apt/trusted.pgp.d``, which is
enabled system-wide by default.
source
path to a file containing the GPG key of the repository.
Using this is recommended as it ensures that the manifest/type manintainer
has validated the key.
If ``-``, the GPG key is read from the type's stdin.
state
'present' or 'absent'. Defaults to 'present'
uri
the URI from which to download the key.
It is highly recommended that you only use protocols with TLS like HTTPS.
This uses ``__download`` but does not use checksums, if you want to ensure
that the key doesn't change, you are better off downloading it and using
``--source``.
DEPRECATED OPTIONAL PARAMETERS
------------------------------
keyid
the id of the key to add. Defaults to __object_id
the id of the key to download from the ``keyserver``.
This is to be used in absence of ``--source`` and ``--uri`` or together
with ``--use-deprecated-apt-key`` for key removal.
Defaults to ``$__object_id``.
keyserver
the keyserver from which to fetch the key. If omitted the default set
in ./parameter/default/keyserver is used.
the keyserver from which to fetch the key.
Defaults to ``pool.sks-keyservers.net``.
keydir
key save location, defaults to ``/etc/apt/trusted.pgp.d``
uri
the URI from which to download the key
DEPRECATED BOOLEAN PARAMETERS
-----------------------------
use-deprecated-apt-key
``apt-key(8)`` will last be available in Debian 11 and Ubuntu 22.04.
You can use this parameter to force usage of ``apt-key(8)``.
Please only use this parameter to *remove* keys from the keyring,
in order to prepare for removal of ``apt-key``.
Adding keys should be done without this parameter.
This parameter will be removed when Debian 11 stops being supported.
EXAMPLES
@ -40,33 +76,39 @@ EXAMPLES
.. code-block:: sh
# Add Ubuntu Archive Automatic Signing Key
__apt_key 437D05B5
# Same thing
__apt_key 437D05B5 --state present
# Get rid of it
__apt_key 437D05B5 --state absent
# add a key that has been verified by a type maintainer
__apt_key jitsi_meet_2021 \
--source cdist-contrib/type/__jitsi_meet/files/apt_2021.gpg
# same thing with human readable name and explicit keyid
__apt_key UbuntuArchiveKey --keyid 437D05B5
# remove an old, deprecated or expired key
__apt_key jitsi_meet_2016 --state absent
# same thing with other keyserver
__apt_key UbuntuArchiveKey --keyid 437D05B5 --keyserver keyserver.ubuntu.com
# Get rid of a key that might have been added to
# /etc/apt/trusted.gpg with apt-key
__apt_key 0x40976EAF437D05B5 --use-deprecated-apt-key --state absent
# download key from the internet
__apt_key rabbitmq \
--uri http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
# add a key that we define in-line
__apt_key jitsi_meet_2021 --source '-' <<EOF
-----BEGIN PGP PUBLIC KEY BLOCK-----
[...]
-----END PGP PUBLIC KEY BLOCK-----
EOF
# download or update key from the internet
__apt_key rabbitmq_2007 \
--uri https://www.rabbitmq.com/rabbitmq-signing-key-public.asc
AUTHORS
-------
Steven Armstrong <steven-cdist--@--armstrong.cc>
Ander Punnar <ander-at-kvlt-dot-ee>
Evilham <contact~~@~~evilham.com>
COPYING
-------
Copyright \(C) 2011-2019 Steven Armstrong and Ander Punnar. You can
Copyright \(C) 2011-2021 Steven Armstrong, Ander Punnar and Evilham. You can
redistribute it and/or modify it under the terms of the GNU General Public
License as published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

104
cdist/conf/type/__apt_key/manifest
View file

@ -2,7 +2,105 @@
__package gnupg
if [ -f "$__object/parameter/uri" ]
then __package curl
else __package dirmngr
state_should="$(cat "${__object}/parameter/state")"
incompatible_args()
{
cat >> /dev/stderr <<-EOF
This type does not support --${1} and --${method} simultaneously.
EOF
exit 1
}
if [ -f "${__object}/parameter/source" ]; then
method="source"
src="$(cat "${__object}/parameter/source")"
if [ "${src}" = "-" ]; then
src="${__object}/stdin"
fi
fi
if [ -f "${__object}/parameter/uri" ]; then
if [ -n "${method}" ]; then
incompatible_args uri
fi
method="uri"
src="$(cat "${__object}/parameter/uri")"
fi
if [ -f "${__object}/parameter/keyid" ]; then
if [ -n "${method}" ]; then
incompatible_args keyid
fi
method="keyid"
fi
# Keep old default
if [ -z "${method}" ]; then
method="keyid"
fi
# Save this for later in gencode-remote
echo "${method}" > "${__object}/key_method"
# Required remotely (most likely already installed)
__package dirmngr
# We need this in case a key has to be dearmor'd
__package gnupg
export require="__package/gnupg"
if [ -f "${__object}/parameter/use-deprecated-apt-key" ]; then
# This is required if apt-key(8) is to be used
if [ "${method}" = "source" ] || [ "${method}" = "uri" ]; then
incompatible_args use-deprecated-apt-key
fi
else
if [ "${state_should}" = "absent" ] && \
[ -f "${__object}/parameter/keyid" ]; then
cat >> /dev/stderr <<EOF
You can't reliably remove by keyid without --use-deprecated-apt-key.
This would very likely do something you do not intend.
EOF
exit 1
fi
fi
keydir="$(cat "${__object}/parameter/keydir")"
keyfile="${keydir}/${__object_id}.gpg"
keyfilecdist="${keyfile}.cdist"
if [ "${state_should}" != "absent" ]; then
# Ensure keydir exists
__directory "${keydir}" --state exists --mode 0755
fi
if [ "${state_should}" = "absent" ]; then
__file "${keyfile}" --state "absent"
__file "${keyfilecdist}" --state "absent"
elif [ "${method}" = "source" ] || [ "${method}" = "uri" ]; then
dearmor="$(cat <<-EOF
if [ '${state_should}' = 'present' ]; then
# Dearmor if necessary
if grep -Fq 'BEGIN PGP PUBLIC KEY BLOCK' '${keyfilecdist}'; then
gpg --dearmor < '${keyfilecdist}' > '${keyfile}'
else
cp '${keyfilecdist}' '${keyfile}'
fi
# Ensure permissions
chown root '${keyfile}'
chmod 0444 '${keyfile}'
fi
EOF
)"
if [ "${method}" = "uri" ]; then
__download "${keyfilecdist}" \
--url "${src}" \
--onchange "${dearmor}"
require="__download${keyfilecdist}" \
__file "${keyfile}" \
--owner root \
--mode 0444 \
--state pre-exists
else
__file "${keyfilecdist}" --state "${state_should}" \
--mode 0444 \
--source "${src}" \
--onchange "${dearmor}"
fi
fi

1
cdist/conf/type/__apt_key/parameter/boolean Normal file
View file

@ -0,0 +1 @@
use-deprecated-apt-key

3
cdist/conf/type/__apt_key/parameter/deprecated/use-deprecated-apt-key Normal file
View file

@ -0,0 +1,3 @@
apt-key(8) will last be available in Debian 11 and Ubuntu 22.04.
Use this flag *only* to migrate to placing a keyring directly in the
/etc/apt/trusted.gpg.d/ directory with a descriptive name.

5
cdist/conf/type/__apt_key/parameter/optional
View file

@ -1,5 +1,6 @@
state
keydir
keyid
keyserver
keydir
source
state
uri

1
cdist/conf/type/__apt_key_uri/deprecated Normal file
View file

@ -0,0 +1 @@
Please migrate to using __apt_key key_id --uri URI.

142
cdist/conf/type/__debconf_set_selections/explorer/state Normal file
View file

@ -0,0 +1,142 @@
#!/bin/sh -e
#
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
# Determine current debconf selections' state.
# Prints one of:
# present: all selections are already set as they should.
# different: one or more of the selections have a different value.
# absent: one or more of the selections are not (currently) defined.
#
test -x /usr/bin/perl || {
# cannot find perl (no perl ~ no debconf)
echo 'absent'
exit 0
}
linesfile="${__object:?}/parameter/line"
test -s "${linesfile}" || {
if test -s "${__object:?}/parameter/file"
then
echo absent
else
echo present
fi
exit 0
}
# assert __type_explorer is set (because it is used by the Perl script)
: "${__type_explorer:?}"
/usr/bin/perl -- - "${linesfile}" <<'EOF'
use strict;
use warnings "all";
use Fcntl qw(:DEFAULT :flock);
use Debconf::Db;
use Debconf::Question;
# Extract @known... arrays from debconf-set-selections
# These values are required to distinguish flags and values in the given lines.
# DC: I couldn't think of a more ugly solution to the problem…
my @knownflags;
my @knowntypes;
my $debconf_set_selections = '/usr/bin/debconf-set-selections';
if (-e $debconf_set_selections) {
my $sed_known = 's/^my \(@known\(flags\|types\) = qw([a-z ]*);\).*$/\1/p';
eval `sed -n '$sed_known' '$debconf_set_selections'`;
}
sub mungeline ($) {
my $line = shift;
chomp $line;
$line =~ s/\r$//;
return $line;
}
sub fatal { printf STDERR @_; exit 1; }
my $state = 'present';
sub state {
my $new = shift;
if ($state eq 'present'
or ($state eq 'different' and $new eq 'absent')) {
$state = $new;
}
}
# Load Debconf DB but manually lock on the state explorer script,
# because Debconf aborts immediately if executed concurrently.
# This is not really an ideal solution because the Debconf DB could be locked by
# another process (e.g. apt-get), but no way to achieve this could be found.
# If you know how to, please provide a patch.
my $lockfile = "%ENV{'__type_explorer'}/state";
if (open my $lock_fh, '+<', $lockfile) {
flock $lock_fh, LOCK_EX or die "Cannot lock $lockfile";
}
{
Debconf::Db->load(readonly => 'true');
}
while (<>) {
# Read and process lines (taken from debconf-set-selections)
$_ = mungeline($_);
while (/\\$/ && ! eof) {
s/\\$//;
$_ .= mungeline(<>);
}
next if /^\s*$/ || /^\s*\#/;
my ($owner, $label, $type, $content) = /^\s*(\S+)\s+(\S+)\s+(\S+)(?:\s(.*))?/
or fatal "invalid line: %s\n", $_;
$content = '' unless defined $content;
# Compare is and should state
my $q = Debconf::Question->get($label);
unless (defined $q) {
# probably a preseed
state 'absent';
next;
}
if (grep { $_ eq $q->type } @knownflags) {
# This line wants to set a flag, presumably.
if ($q->flag($q->type) ne $content) {
state 'different';
}
} else {
# Otherwise, it's probably a value…
if ($q->value ne $content) {
state 'different';
}
unless (grep { $_ eq $owner } (split /, /, $q->owners)) {
state 'different';
}
}
}
printf "%s\n", $state;
EOF

42
cdist/conf/type/__debconf_set_selections/gencode-remote
View file

@ -1,6 +1,7 @@
#!/bin/sh -e
#
# 2011-2014 Nico Schottelius (nico-cdist at schottelius.org)
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
@ -17,16 +18,37 @@
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Setup selections
#
filename="$(cat "$__object/parameter/file")"
if [ "$filename" = "-" ]; then
filename="$__object/stdin"
if test -f "${__object:?}/parameter/line"
then
filename="${__object:?}/parameter/line"
elif test -s "${__object:?}/parameter/file"
then
filename=$(cat "${__object:?}/parameter/file")
if test "${filename}" = '-'
then
filename="${__object:?}/stdin"
fi
else
printf 'Neither --line nor --file set.\n' >&2
exit 1
fi
echo "debconf-set-selections << __file-eof"
cat "$filename"
echo "__file-eof"
# setting no lines makes no sense
test -s "${filename}" || exit 0
state_is=$(cat "${__object:?}/explorer/state")
if test "${state_is}" != 'present'
then
cat <<-CODE
debconf-set-selections <<'EOF'
$(cat "${filename}")
EOF
CODE
awk '
{
printf "set %s %s %s %s\n", $1, $2, $3, $4
}' "${filename}" >>"${__messages_out:?}"
fi

51
cdist/conf/type/__debconf_set_selections/man.rst
View file

@ -8,15 +8,33 @@ cdist-type__debconf_set_selections - Setup debconf selections
DESCRIPTION
-----------
On Debian and alike systems debconf-set-selections(1) can be used
On Debian and alike systems :strong:`debconf-set-selections`\ (1) can be used
to setup configuration parameters.
REQUIRED PARAMETERS
-------------------
cf. ``--line``.
OPTIONAL PARAMETERS
-------------------
file
Use the given filename as input for debconf-set-selections(1)
If filename is "-", read from stdin.
Use the given filename as input for :strong:`debconf-set-selections`\ (1)
If filename is ``-``, read from stdin.
**This parameter is deprecated, because it doesn't work with state detection.**
line
A line in :strong:`debconf-set-selections`\ (1) compatible format.
This parameter can be used multiple times to set multiple options.
(This parameter is actually required, but marked optional because the
deprecated ``--file`` is still accepted.)
BOOLEAN PARAMETERS
------------------
None.
EXAMPLES
@ -24,30 +42,29 @@ EXAMPLES
.. code-block:: sh
# Setup configuration for nslcd
__debconf_set_selections nslcd --file /path/to/file
# Setup gitolite's gituser
__debconf_set_selections nslcd --line 'gitolite gitolite/gituser string git'
# Setup configuration for nslcd from another type
__debconf_set_selections nslcd --file "$__type/files/preseed/nslcd"
__debconf_set_selections nslcd --file - << eof
gitolite gitolite/gituser string git
eof
# Setup configuration for nslcd from a file.
# NB: Multiple lines can be passed to --line, although this can be considered a hack.
__debconf_set_selections nslcd --line "$(cat "${__files:?}/preseed/nslcd.debconf")"
SEE ALSO
--------
:strong:`debconf-set-selections`\ (1), :strong:`cdist-type__update_alternatives`\ (7)
- :strong:`cdist-type__update_alternatives`\ (7)
- :strong:`debconf-set-selections`\ (1)
AUTHORS
-------
Nico Schottelius <nico-cdist--@--schottelius.org>
| Nico Schottelius <nico-cdist--@--schottelius.org>
| Dennis Camera <dennis.camera--@--ssrq-sds-fds.ch>
COPYING
-------
Copyright \(C) 2011-2014 Nico Schottelius. You can redistribute it
and/or modify it under the terms of the GNU General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.
Copyright \(C) 2011-2014 Nico Schottelius, 2021 Dennis Camera.
You can redistribute it and/or modify it under the terms of the GNU General
Public License as published by the Free Software Foundation, either version 3 of
the License, or (at your option) any later version.

21
cdist/conf/type/__debconf_set_selections/manifest Executable file
View file

@ -0,0 +1,21 @@
#!/bin/sh -e
#
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
__package_apt debconf

0
cdist/conf/type/__debconf_set_selections/nonparallel Normal file
View file

0
cdist/conf/type/__debconf_set_selections/parameter/required → cdist/conf/type/__debconf_set_selections/parameter/deprecated
View file

1
cdist/conf/type/__debconf_set_selections/parameter/optional_multiple Normal file
View file

@ -0,0 +1 @@
line

19
cdist/conf/type/__download/explorer/remote_cmd
View file

@ -1,19 +0,0 @@
#!/bin/sh -e
if [ -f "$__object/parameter/cmd-get" ]
then
cmd="$( cat "$__object/parameter/cmd-get" )"
elif command -v curl > /dev/null
then
cmd="curl -L -o - '%s'"
elif command -v fetch > /dev/null
then
cmd="fetch -o - '%s'"
else
cmd="wget -O - '%s'"
fi
echo "$cmd"

16
cdist/conf/type/__download/explorer/remote_cmd_get Executable file
View file

@ -0,0 +1,16 @@
#!/bin/sh -e
if [ -f "$__object/parameter/cmd-get" ]
then
cat "$__object/parameter/cmd-get"
elif
command -v curl > /dev/null
then
echo "curl -sSL -o - '%s'"
elif
command -v fetch > /dev/null
then
echo "fetch -o - '%s'"
else
echo "wget -O - '%s'"
fi

82
cdist/conf/type/__download/explorer/remote_cmd_sum Executable file
View file

@ -0,0 +1,82 @@
#!/bin/sh -e
if [ ! -f "$__object/parameter/sum" ]
then
exit 0
fi
if [ -f "$__object/parameter/cmd-sum" ]
then
cat "$__object/parameter/cmd-sum"
exit 0
fi
sum_should="$( cat "$__object/parameter/sum" )"
if echo "$sum_should" | grep -Fq ':'
then
sum_hash="$( echo "$sum_should" | cut -d : -f 1 )"
else
if echo "$sum_should" | grep -Eq '^[0-9]+\s[0-9]+$'
then
sum_hash='cksum'
elif
echo "$sum_should" | grep -Eiq '^[a-f0-9]{32}$'
then
sum_hash='md5'
elif
echo "$sum_should" | grep -Eiq '^[a-f0-9]{40}$'
then
sum_hash='sha1'
elif
echo "$sum_should" | grep -Eiq '^[a-f0-9]{64}$'
then
sum_hash='sha256'
else
echo 'hash format detection failed' >&2
exit 1
fi
fi
os="$( "$__explorer/os" )"
case "$sum_hash" in
cksum)
echo "cksum %s | awk '{print \$1\" \"\$2}'"
;;
md5)
case "$os" in
freebsd)
echo "md5 -q %s"
;;
*)
echo "md5sum %s | awk '{print \$1}'"
;;
esac
;;
sha1)
case "$os" in
freebsd)
echo "sha1 -q %s"
;;
*)
echo "sha1sum %s | awk '{print \$1}'"
;;
esac
;;
sha256)
case "$os" in
freebsd)
echo "sha256 -q %s"
;;
*)
echo "sha256sum %s | awk '{print \$1}'"
;;
esac
;;
*)
# we arrive here only if --sum is given with unknown format prefix
echo "unknown hash format: $sum_hash" >&2
exit 1
;;
esac

61
cdist/conf/type/__download/explorer/state
View file

@ -1,6 +1,11 @@
#!/bin/sh -e
dst="/$__object_id"
if [ -f "$__object/parameter/destination" ]
then
dst="$( cat "$__object/parameter/destination" )"
else
dst="/$__object_id"
fi
if [ ! -f "$dst" ]
then
@ -16,57 +21,19 @@ fi
sum_should="$( cat "$__object/parameter/sum" )"
if [ -f "$__object/parameter/cmd-sum" ]
if echo "$sum_should" | grep -Fq ':'
then
# shellcheck disable=SC2059
sum_is="$( eval "$( printf \
"$( cat "$__object/parameter/cmd-sum" )" \
"$dst" )" )"
else
os="$( "$__explorer/os" )"
if echo "$sum_should" | grep -Eq '^[0-9]+\s[0-9]+$'
then
sum_is="$( cksum "$dst" | awk '{print $1" "$2}' )"
elif echo "$sum_should" | grep -Eiq '^md5:[a-f0-9]{32}$'
then
case "$os" in
freebsd)
sum_is="md5:$( md5 -q "$dst" )"
;;
*)
sum_is="md5:$( md5sum "$dst" | awk '{print $1}' )"
;;
esac
elif echo "$sum_should" | grep -Eiq '^sha1:[a-f0-9]{40}$'
then
case "$os" in
freebsd)
sum_is="sha1:$( sha1 -q "$dst" )"
;;
*)
sum_is="sha1:$( sha1sum "$dst" | awk '{print $1}' )"
;;
esac
elif echo "$sum_should" | grep -Eiq '^sha256:[a-f0-9]{64}$'
then
case "$os" in
freebsd)
sum_is="sha256:$( sha256 -q "$dst" )"
;;
*)
sum_is="sha256:$( sha256sum "$dst" | awk '{print $1}' )"
;;
esac
fi
sum_should="$( echo "$sum_should" | cut -d : -f 2 )"
fi
sum_cmd="$( "$__type_explorer/remote_cmd_sum" )"
# shellcheck disable=SC2059
sum_is="$( eval "$( printf "$sum_cmd" "'$dst'" )" )"
if [ -z "$sum_is" ]
then
echo 'no checksum from target' >&2
echo 'existing destination checksum failed' >&2
exit 1
fi

131
cdist/conf/type/__download/gencode-local
View file

@ -11,34 +11,133 @@ fi
url="$( cat "$__object/parameter/url" )"
tmp="$( mktemp )"
dst="/$__object_id"
if [ -f "$__object/parameter/destination" ]
then
dst="$( cat "$__object/parameter/destination" )"
else
dst="/$__object_id"
fi
if [ -f "$__object/parameter/cmd-get" ]
then
cmd="$( cat "$__object/parameter/cmd-get" )"
elif command -v wget > /dev/null
then
cmd="wget -O - '%s'"
elif command -v curl > /dev/null
then
cmd="curl -L -o - '%s'"
cmd="curl -sSL -o - '%s'"
elif command -v fetch > /dev/null
then
cmd="fetch -o - '%s'"
elif command -v wget > /dev/null
then
cmd="wget -O - '%s'"
else
echo 'no usable locally installed utility for downloading' >&2
echo 'local download failed, no usable utility' >&2
exit 1
fi
printf "$cmd > %s\n" \
"$url" \
"$tmp"
echo "download_tmp=\"\$( mktemp )\""
# shellcheck disable=SC2059
printf "$cmd > \"\$download_tmp\"\n" "$url"
if [ -f "$__object/parameter/sum" ]
then
sum_should="$( cat "$__object/parameter/sum" )"
if [ -f "$__object/parameter/cmd-sum" ]
then
local_cmd_sum="$( cat "$__object/parameter/cmd-sum" )"
else
if echo "$sum_should" | grep -Fq ':'
then
sum_hash="$( echo "$sum_should" | cut -d : -f 1 )"
sum_should="$( echo "$sum_should" | cut -d : -f 2 )"
else
if echo "$sum_should" | grep -Eq '^[0-9]+\s[0-9]+$'
then
sum_hash='cksum'
elif
echo "$sum_should" | grep -Eiq '^[a-f0-9]{32}$'
then
sum_hash='md5'
elif
echo "$sum_should" | grep -Eiq '^[a-f0-9]{40}$'
then
sum_hash='sha1'
elif
echo "$sum_should" | grep -Eiq '^[a-f0-9]{64}$'
then
sum_hash='sha256'
else
echo 'hash format detection failed' >&2
exit 1
fi
fi
case "$sum_hash" in
cksum)
local_cmd_sum="cksum %s | awk '{print \$1\" \"\$2}'"
;;
md5)
if command -v md5 > /dev/null
then
local_cmd_sum="md5 -q %s"
elif
command -v md5sum > /dev/null
then
local_cmd_sum="md5sum %s | awk '{print \$1}'"
fi
;;
sha1)
if command -v sha1 > /dev/null
then
local_cmd_sum="sha1 -q %s"
elif
command -v sha1sum > /dev/null
then
local_cmd_sum="sha1sum %s | awk '{print \$1}'"
fi
;;
sha256)
if command -v sha256 > /dev/null
then
local_cmd_sum="sha256 -q %s"
elif
command -v sha256sum > /dev/null
then
local_cmd_sum="sha256sum %s | awk '{print \$1}'"
fi
;;
*)
# we arrive here only if --sum is given with unknown format prefix
echo "unknown hash format: $sum_hash" >&2
exit 1
;;
esac
if [ -z "$local_cmd_sum" ]
then
echo 'local checksum verification failed, no usable utility' >&2
exit 1
fi
fi
# shellcheck disable=SC2059
echo "sum_is=\"\$( $( printf "$local_cmd_sum" "\"\$download_tmp\"" ) )\""
echo "if [ \"\$sum_is\" != '$sum_should' ]; then"
echo "echo 'local download checksum mismatch' >&2"
echo "rm -f \"\$download_tmp\""
echo 'exit 1; fi'
fi
if echo "$__target_host" | grep -Eq '^[0-9a-fA-F:]+$'
then
@ -47,12 +146,10 @@ else
target_host="$__target_host"
fi
printf '%s %s %s:%s\n' \
# shellcheck disable=SC2016
printf '%s "$download_tmp" %s:%s\n' \
"$__remote_copy" \
"$tmp" \
"$target_host" \
"$dst"
echo "rm -f '$tmp'"
echo 'downloaded' > "$__messages_out"
echo "rm -f \"\$download_tmp\""

46
cdist/conf/type/__download/gencode-remote
View file

@ -6,17 +6,51 @@ state_is="$( cat "$__object/explorer/state" )"
if [ "$download" = 'remote' ] && [ "$state_is" != 'present' ]
then
cmd="$( cat "$__object/explorer/remote_cmd" )"
cmd_get="$( cat "$__object/explorer/remote_cmd_get" )"
url="$( cat "$__object/parameter/url" )"
dst="/$__object_id"
if [ -f "$__object/parameter/destination" ]
then
dst="$( cat "$__object/parameter/destination" )"
else
dst="/$__object_id"
fi
printf "$cmd > %s\n" \
"$url" \
"$dst"
echo "download_tmp=\"\$( mktemp )\""
echo 'downloaded' > "$__messages_out"
# shellcheck disable=SC2059
printf "$cmd_get > \"\$download_tmp\"\n" "$url"
if [ -f "$__object/parameter/sum" ]
then
sum_should="$( cat "$__object/parameter/sum" )"
if [ -f "$__object/parameter/cmd-sum" ]
then
remote_cmd_sum="$( cat "$__object/parameter/cmd-sum" )"
else
remote_cmd_sum="$( cat "$__object/explorer/remote_cmd_sum" )"
if echo "$sum_should" | grep -Fq ':'
then
sum_should="$( echo "$sum_should" | cut -d : -f 2 )"
fi
fi
# shellcheck disable=SC2059
echo "sum_is=\"\$( $( printf "$remote_cmd_sum" "\"\$download_tmp\"" ) )\""
echo "if [ \"\$sum_is\" != '$sum_should' ]; then"
echo "echo 'remote download checksum mismatch' >&2"
echo "rm -f \"\$download_tmp\""
echo 'exit 1; fi'
fi
echo "mv \"\$download_tmp\" '$dst'"
fi
if [ -f "$__object/parameter/onchange" ] && [ "$state_is" != "present" ]

31
cdist/conf/type/__download/man.rst
View file

@ -8,7 +8,7 @@ cdist-type__download - Download a file
DESCRIPTION
-----------
By default type will try to use ``wget``, ``curl`` or ``fetch``.
By default type will try to use ``curl``, ``fetch`` or ``wget``.
If download happens in target (see ``--download``) then type will
fallback to (and install) ``wget``.
@ -16,6 +16,8 @@ If download happens in local machine, then environment variables like
``{http,https,ftp}_proxy`` etc can be used on cdist execution
(``http_proxy=foo cdist config ...``).
To change downloaded file's owner, group or permissions, use ``require='__download/path/to/file' __file ...``.
REQUIRED PARAMETERS
-------------------
@ -25,14 +27,29 @@ url
OPTIONAL PARAMETERS
-------------------
destination
Downloaded file's destination in target. If unset, ``$__object_id`` is used.
sum
Checksum is used to decide if existing destination file must be redownloaded.
By default output of ``cksum`` without filename is expected.
Other hash formats supported with prefixes: ``md5:``, ``sha1:`` and ``sha256:``.
Supported formats: ``cksum`` output without file name, MD5, SHA1 and SHA256.
Type tries to detect hash format with regexes, but prefixes
``cksum:``, ``md5:``, ``sha1:`` and ``sha256:`` are also supported.
Checksum have two purposes - state check and post-download verification.
In state check, if destination checksum mismatches, then content of URL
will be downloaded to temporary file. If downloaded temporary file's
checksum matches, then it will be moved to destination (overwritten).
For local downloads it is expected that usable utilities for checksum
calculation exist in the system.
download
If ``local`` (default), then download file to local storage and copy
it to target host. If ``remote``, then download happens in target.
If ``local`` (default), then file is downloaded to local storage and copied
to target host. If ``remote``, then download happens in target.
For local downloads it is expected that usable utilities for downloading
exist in the system. Type will try to use ``curl``, ``fetch`` or ``wget``.
cmd-get
Command used for downloading.
@ -62,7 +79,7 @@ EXAMPLES
require='__directory/opt/cpma' \
__download /opt/cpma/cnq3.zip \
--url https://cdn.playmorepromode.com/files/cnq3/cnq3-1.51.zip \
--sum md5:46da3021ca9eace277115ec9106c5b46
--sum 46da3021ca9eace277115ec9106c5b46
require='__download/opt/cpma/cnq3.zip' \
__unpack /opt/cpma/cnq3.zip \

2
cdist/conf/type/__download/manifest
View file

@ -1,6 +1,6 @@
#!/bin/sh -e
if grep -Eq '^wget' "$__object/explorer/remote_cmd"
if grep -Eq '^wget' "$__object/explorer/remote_cmd_get"
then
__package wget
fi

3
cdist/conf/type/__download/parameter/optional
View file

@ -1,5 +1,6 @@
sum
cmd-get
cmd-sum
destination
download
onchange
sum

7
cdist/conf/type/__git/explorer/group
View file

@ -14,6 +14,11 @@ then
then
printf '%u\n' "${group_gid}"
else
printf '%s\n' "$(id -u -n "${group_gid}")"
if command -v getent > /dev/null
then
getent group "${group_gid}" | cut -d : -f 1
else
awk -F: -v gid="${group_gid}" '$3 == gid { print $1 }' /etc/group
fi
fi
fi

3
cdist/conf/type/__letsencrypt_cert/explorer/certbot-path
View file

@ -1,3 +0,0 @@
#!/bin/sh -e
command -v certbot 2>/dev/null || true

78
cdist/conf/type/__letsencrypt_cert/explorer/certificate-data Executable file
View file

@ -0,0 +1,78 @@
#!/bin/sh -e
certbot_path="$(command -v certbot 2>/dev/null || true)"
# Defaults
certificate_exists="no"
certificate_is_test="no"
if [ -n "${certbot_path}" ]; then
# Find python executable that has access to certbot's module
python_path=$(sed -n '1s/^#! *//p' "${certbot_path}")
# Use a lock for cdist due to certbot not exiting with failure
# or having any flags for concurrent use.
_certbot() {
${python_path} - 2>/dev/null <<EOF
from certbot.main import main
import fcntl
lock_file = "/tmp/certbot.cdist.lock"
timeout=60
with open(lock_file, 'w') as fd:
for i in range(timeout):
try:
# Get exclusive lock
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
break
except:
# Wait if that fails
import time
time.sleep(1)
else:
# Timed out, exit with failure
import sys
sys.exit(1)
# Do list certificates
main(["certificates", "--cert-name", "${__object_id:?}"])
EOF
}
_certificate_exists() {
if grep -q " Certificate Name: ${__object_id:?}$"; then
echo yes
else
echo no
fi
}
_certificate_is_test() {
if grep -q 'INVALID: TEST_CERT'; then
echo yes
else
echo no
fi
}
_certificate_domains() {
grep ' Domains: ' | cut -d ' ' -f 6- | tr ' ' '\n'
}
# Get data about all available certificates
certificates="$(_certbot)"
# Check whether or not the certificate exists
certificate_exists="$(echo "${certificates}" | _certificate_exists)"
# Check whether or not the certificate is for testing
certificate_is_test="$(echo "${certificates}" | _certificate_is_test)"
# Get domains for certificate
certificate_domains="$(echo "${certificates}" | _certificate_domains)"
fi
# Return received data
cat <<EOF
certbot_path:${certbot_path}
certificate_exists:${certificate_exists}
certificate_is_test:${certificate_is_test}
${certificate_domains}
EOF

8
cdist/conf/type/__letsencrypt_cert/explorer/certificate-domains
View file

@ -1,8 +0,0 @@
#!/bin/sh -e
certbot_path=$("${__type_explorer}/certbot-path")
if [ -n "${certbot_path}" ]
then
certbot certificates --cert-name "${__object_id:?}" | grep ' Domains: ' | \
cut -d ' ' -f 6- | tr ' ' '\n'
fi

13
cdist/conf/type/__letsencrypt_cert/explorer/certificate-exists
View file

@ -1,13 +0,0 @@
#!/bin/sh -e
certbot_path=$("${__type_explorer}/certbot-path")
if [ -n "${certbot_path}" ]
then
if certbot certificates | grep -q " Certificate Name: ${__object_id:?}$"; then
echo yes
else
echo no
fi
else
echo no
fi

14
cdist/conf/type/__letsencrypt_cert/explorer/certificate-is-test
View file

@ -1,14 +0,0 @@
#!/bin/sh -e
certbot_path=$("${__type_explorer}/certbot-path")
if [ -n "${certbot_path}" ]
then
if certbot certificates --cert-name "${__object_id:?}" | \
grep -q 'INVALID: TEST_CERT'; then
echo yes
else
echo no
fi
else
echo no
fi

11
cdist/conf/type/__letsencrypt_cert/gencode-remote
View file

@ -1,6 +1,10 @@
#!/bin/sh -e
certificate_exists=$(cat "${__object:?}/explorer/certificate-exists")
_explorer_var() {
grep "^$1:" "${__object:?}/explorer/certificate-data" | cut -d ':' -f 2-
}
certificate_exists="$(_explorer_var certificate_exists)"
name="${__object_id:?}"
state=$(cat "${__object}/parameter/state")
@ -29,8 +33,9 @@ case "${state}" in
fi
if [ "${certificate_exists}" = "yes" ]; then
existing_domains="${__object}/explorer/certificate-domains"
certificate_is_test=$(cat "${__object}/explorer/certificate-is-test")
existing_domains=$(mktemp "${TMPDIR:-/tmp}/existing_domains.cdist.XXXXXXXXXX")
tail -n +4 "${__object:?}/explorer/certificate-data" | grep -v '^$' > "${existing_domains}"
certificate_is_test="$(_explorer_var certificate_is_test)"
sort -uo "${requested_domains}" "${requested_domains}"
sort -uo "${existing_domains}" "${existing_domains}"

2
cdist/conf/type/__letsencrypt_cert/manifest
View file

@ -1,6 +1,6 @@
#!/bin/sh
certbot_fullpath="$(cat "${__object:?}/explorer/certbot-path")"
certbot_fullpath="$(grep "^certbot_path:" "${__object:?}/explorer/certificate-data" | cut -d ':' -f 2-)"
state=$(cat "${__object}/parameter/state")
os="$(cat "${__global:?}/explorer/os")"

64
cdist/conf/type/__postgres_conf/explorer/postgres_user Normal file
View file

@ -0,0 +1,64 @@
#!/bin/sh -e
# -*- mode: sh; indent-tabs-mode: t -*-
#
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
os=$("${__explorer:?}/os")
case ${os}
in
(alpine)
echo 'postgres'
;;
(centos|rhel|scientific)
echo 'postgres'
;;
(debian|devuan|ubuntu)
echo 'postgres'
;;
(freebsd)
test -x /usr/local/etc/rc.d/postgresql || {
printf 'could not find postgresql rc script./n' >&2
exit 1
}
pg_status=$(/usr/local/etc/rc.d/postgresql onestatus) || {
printf 'postgresql daemon is not running.\n' >&2
exit 1
}
pg_pid=$(printf '%s\n' "${pg_status}" \
| sed -n 's/^pg_ctl:.*(PID: *\([0-9]*\))$/\1/p')
# PostgreSQL < 9.6: pgsql
# PostgreSQL >= 9.6: postgres
ps -o user -p "${pg_pid}" | sed -n '2p'
;;
(netbsd)
echo 'pgsql'
;;
(openbsd)
echo '_postgresql'
;;
(suse)
echo 'postgres'
;;
(*)
echo "Unsupported OS: ${os}" >&2
exit 1
;;
esac

223
cdist/conf/type/__postgres_conf/explorer/state Normal file
View file

@ -0,0 +1,223 @@
#!/bin/sh -e
# -*- mode: sh; indent-tabs-mode: t -*-
#
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
postgres_user=$("${__type_explorer:?}/postgres_user")
conf_name=${__object_id:?}
tolower() { printf '%s' "$*" | tr '[:upper:]' '[:lower:]'; }
tobytes() {
# NOTE: This function treats everything as base 2.
# It is not compatible with SI units.
awk 'BEGIN { FS = "\n" }
/TB$/ { $0 = ($0 * 1024) "GB" }
/GB$/ { $0 = ($0 * 1024) "MB" }
/MB$/ { $0 = ($0 * 1024) "kB" }
/kB$/ { $0 = ($0 * 1024) "B" }
/B?$/ { sub(/ *B?$/, "") }
($0*1) == $0 # is number
' <<-EOF
$1
EOF
}
tomillisecs() {
awk 'BEGIN { FS = "\n" }
/d$/ { $0 = ($0 * 24) "h" }
/h$/ { $0 = ($0 * 60) "min" }
/min$/ { $0 = ($0 * 60) "s" }
/[^m]s$/ { $0 = ($0 * 1000) "ms" }
/ms$/ { $0 *= 1 }
($0*1) == $0 # is number
' <<-EOF
$1
EOF
}
tobool() {
# prints either 'on' or 'off'
case $(tolower "$1")
in
(t|true|y|yes|on|1)
echo 'on' ;;
(f|false|n|no|off|0)
echo 'off' ;;
(*)
printf 'Inavlid bool value: %s\n' "$2" >&2
return 1
;;
esac
return 0
}
quote() { printf '%s\n' "$*" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/'/"; }
psql_exec() {
su - "${postgres_user}" -c "psql postgres -twAc $(quote "$*")"
}
psql_conf_source() {
# NOTE: SHOW/SET are case-insentitive, so this command should also be.
psql_exec "SELECT CASE WHEN source = 'default' OR setting = boot_val THEN 'default' ELSE source END FROM pg_settings WHERE lower(name) = lower('$1')"
}
psql_conf_cmp() (
IFS='|' read -r lower_name vartype setting unit <<-EOF
$(psql_exec "SELECT lower(name), vartype, setting, unit FROM pg_settings WHERE lower(name) = lower('$1')")
EOF
should_value=$2
is_value=${setting}
# The following case contains special cases for special settings.
case ${lower_name}
in
(archive_command)
if test "${setting}" = '(disabled)'
then
# DAFUQ PostgreSQL?!
# PostgreSQL returns (disabled) if the feature is inactive.
# We cannot compare the values unless it is enabled, first.
return 0
fi
;;
(archive_mode|backslash_quote|constraint_exclusion|force_parallel_mode|huge_pages|synchronous_commit)
# Although only 'on', 'off' are documented, PostgreSQL accepts all
# the "likely" variants of "on" and "off".
case $(tolower "${should_value}")
in
(on|off|true|false|yes|no|1|0)
should_value=$(tobool "${should_value}")
;;
esac
;;
esac
case ${vartype}
in
(bool)
test -z "${unit}" || {
# please fix the explorer if this error occurs.
printf 'units are not supported for vartype: %s\n' "${vartype}" >&2
exit 1
}
should_value=$(tobool "${should_value}")
test "${is_value}" = "${should_value}"
;;
(enum)
test -z "${unit}" || {
# please fix the explorer if this error occurs.
printf 'units are not supported with vartype: %s\n' "${vartype}" >&2
exit 1
}
# NOTE: All enums that are currently defined are lower case, but
# PostgreSQL also accepts upper case spelling.
should_value=$(tolower "$2")
test "${is_value}" = "${should_value}"
;;
(integer)
# split multiples from unit, first (e.g. 8kB -> 8, kB)
case ${unit}
in
([0-9]*)
multiple=${unit%%[!0-9]*}
unit=${unit##*[0-9 ]}
;;
(*) multiple=1 ;;
esac
is_value=$((setting * multiple))${unit}
if expr "${should_value}" : '-\{0,1\}[0-9]*$' >/dev/null
then
# default unit
should_value=$((should_value * multiple))${unit}
fi
# then, do conversion
# NOTE: these conversions work for integers only!
case ${unit}
in
(B|[kMGT]B)
# bytes
is_bytes=$(tobytes "${is_value}")
should_bytes=$(tobytes "${should_value}")
test $((is_bytes)) -eq $((should_bytes))
;;
(ms|s|min|h|d)
# seconds
is_ms=$(tomillisecs "${is_value}")
should_ms=$(tomillisecs "${should_value}")
test $((is_ms)) -eq $((should_ms))
;;
('')
# no unit
is_int=${is_value}
should_int=${should_value}
test $((is_int)) -eq $((should_int))
;;
esac
;;
(real|string)
# NOTE: reals could possibly have units, but currently there none.
test -z "${unit}" || {
# please fix the explorer if this error occurs.
printf 'units are not supported with vartype: %s\n' "${vartype}" >&2
exit 1
}
test "${is_value}" = "${should_value}"
;;
esac
)
psql_exec 'SELECT 1' >/dev/null || {
echo 'Connection to PostgreSQL server failed' >&2
exit 1
}
case $(psql_conf_source "${conf_name}")
in
('')
printf 'Invalid configuration parameter: %s\n' "${conf_name}" >&2
exit 1
;;
(default)
echo absent
;;
(*)
if ! test -f "${__object:?}/parameter/value"
then
echo present
elif psql_conf_cmp "${conf_name}" "$(cat "${__object:?}/parameter/value")"
then
echo present
else
echo different
fi
;;
esac

123
cdist/conf/type/__postgres_conf/gencode-remote Executable file
View file

@ -0,0 +1,123 @@
#!/bin/sh -e
# -*- mode: sh; indent-tabs-mode: t -*-
#
# 2019-2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
# 2020 Beni Ruef (bernhard.ruef at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
state_is=$(cat "${__object:?}/explorer/state")
state_should=$(cat "${__object:?}/parameter/state")
postgres_user=$(cat "${__object:?}/explorer/postgres_user")
conf_name=${__object_id:?}
if test "${state_is}" = "${state_should}"
then
exit 0
fi
quote() {
for _arg
do
shift
if test -n "$(printf '%s' "${_arg}" | tr -d -c '\t\n \042-\047\050-\052\073-\077\133\\`|~' | tr -c '' '.')"
then
# needs quoting
set -- "$@" "'$(printf '%s' "${_arg}" | sed -e "s/'/'\\\\''/g")'"
else
set -- "$@" "${_arg}"
fi
done
unset _arg
# NOTE: Use printf because POSIX echo interprets escape sequences
printf '%s' "$*"
}
psql_cmd() {
printf 'su - %s -c %s\n' "$(quote "${postgres_user}")" "$(quote "$(quote psql "$@")")"
}
case ${state_should}
in
(present)
test -n "${__object:?}/parameter/value" || {
echo 'Missing required parameter --value' >&2
exit 1
}
cat <<-EOF
exec 3< "\${__object:?}/parameter/value"
$(psql_cmd postgres -tAwq -o /dev/null -v ON_ERROR_STOP=on) <<'SQL'
\\set conf_value \`cat <&3\`
ALTER SYSTEM SET ${conf_name} = :'conf_value';
SELECT pg_reload_conf();
SQL
exec 3<&-
EOF
;;
(absent)
psql_cmd postgres -qwc "ALTER SYSTEM SET ${conf_name} TO DEFAULT"
;;
(*)
printf 'Invalid --state: %s\n' "${state_should}" >&2
printf 'Only "present" and "absent" are acceptable.\n' >&2
exit 1
;;
esac
# Restart PostgreSQL server if required to apply new configuration value
cat <<EOF
if test 't' = "\$($(psql_cmd postgres -twAc "SELECT pending_restart FROM pg_settings WHERE lower(name) = lower('${conf_name}')"))"
then
$(
init=$(cat "${__global:?}/explorer/init")
case ${init}
in
(systemd)
echo 'systemctl restart postgresql.service'
;;
(*openrc*)
echo 'rc-service postgresql restart'
;;
(sysvinit)
echo '/etc/init.d/postgresql restart'
;;
(init)
case $(cat "${__global:?}/explorer/kernel_name")
in
(FreeBSD)
echo '/usr/local/etc/rc.d/postgresql restart'
;;
(OpenBSD|NetBSD)
echo '/etc/rc.d/postgresql restart'
;;
(*)
echo "Unsupported operating system. Don't know how to restart services." >&2
exit 1
esac
;;
(*)
printf "Don't know how to restart services with your init (%s)\n" "${init}" >&2
exit 1
esac
)
fi
EOF

60
cdist/conf/type/__postgres_conf/man.rst Normal file
View file

@ -0,0 +1,60 @@
cdist-type__postgres_conf(7)
============================
NAME
----
cdist-type__postgres_conf - Alter PostgreSQL configuration
DESCRIPTION
-----------
Configure a running PostgreSQL server using ``ALTER SYSTEM``.
REQUIRED PARAMETERS
-------------------
value
The value to set (can be omitted if ``--state`` is set to ``absent``).
OPTIONAL PARAMETERS
-------------------
state
``present`` or ``absent``.
Defaults to ``present``.
BOOLEAN PARAMETERS
------------------
None.
EXAMPLES
--------
.. code-block:: sh
# set timezone
__postgres_conf timezone --value Europe/Zurich
# reset maximum number of concurrent connections to default (normally 100)
__postgres_conf max_connections --state absent
SEE ALSO
--------
None.
AUTHORS
-------
Beni Ruef (bernhard.ruef--@--ssrq-sds-fds.ch)
Dennis Camera (dennis.camera--@--ssrq-sds-fds.ch)
COPYING
-------
Copyright \(C) 2019-2021 SSRQ (www.ssrq-sds-fds.ch).
You can redistribute it and/or modify it under the terms of the GNU General
Public License as published by the Free Software Foundation, either version 3 of
the License, or (at your option) any later version.

1
cdist/conf/type/__postgres_conf/parameter/default/state Normal file
View file

@ -0,0 +1 @@
present

2
cdist/conf/type/__postgres_conf/parameter/optional Normal file
View file

@ -0,0 +1,2 @@
state
value

1
cdist/conf/type/__postgres_database/explorer/postgres_user Symbolic link
View file

@ -0,0 +1 @@
../../__postgres_conf/explorer/postgres_user

26
cdist/conf/type/__postgres_database/explorer/state
View file

@ -1,6 +1,7 @@
#!/bin/sh
#
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
@ -18,25 +19,18 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
case "$("${__explorer}/os")"
in
netbsd)
postgres_user='pgsql'
;;
openbsd)
postgres_user='_postgresql'
;;
*)
postgres_user='postgres'
;;
esac
postgres_user=$("${__type_explorer:?}/postgres_user")
dbname=${__object_id:?}
name="$__object_id"
quote() { printf '%s\n' "$*" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/'/"; }
psql_exec() {
su - "${postgres_user}" -c "psql $(quote "$1") -twAc $(quote "$2")"
}
if test -n "$(su - "$postgres_user" -c "psql postgres -twAc \"SELECT 1 FROM pg_database WHERE datname='$name'\"")"
if psql_exec postgres "SELECT datname FROM pg_database" | grep -qFx "${dbname}"
then
echo 'present'
echo 'present'
else
echo 'absent'
echo 'absent'
fi

108
cdist/conf/type/__postgres_database/gencode-remote
View file

@ -1,6 +1,7 @@
#!/bin/sh -e
#
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
@ -18,60 +19,63 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
case "$(cat "${__global}/explorer/os")"
in
netbsd)
postgres_user='pgsql'
;;
openbsd)
postgres_user='_postgresql'
;;
*)
postgres_user='postgres'
;;
esac
quote() {
for _arg
do
shift
if test -n "$(printf '%s' "${_arg}" | tr -d -c '\t\n \042-\047\050-\052\073-\077\133\\`|~' | tr -c '' '.')"
then
# needs quoting
set -- "$@" "'$(printf '%s' "${_arg}" | sed -e "s/'/'\\\\''/g")'"
else
set -- "$@" "${_arg}"
fi
done
unset _arg
# NOTE: Use printf because POSIX echo interprets escape sequences
printf '%s' "$*"
}
name="$__object_id"
state_should="$(cat "$__object/parameter/state")"
state_is="$(cat "$__object/explorer/state")"
postgres_user=$(cat "${__object:?}/explorer/postgres_user")
if [ "$state_should" != "$state_is" ]; then
case "$state_should" in
present)
owner=""
if [ -f "$__object/parameter/owner" ]; then
owner="-O \"$(cat "$__object/parameter/owner")\""
fi
dbname=${__object_id:?}
state_should=$(cat "${__object:?}/parameter/state")
state_is=$(cat "${__object:?}/explorer/state")
template=""
if [ -f "$__object/parameter/template" ]; then
template="--template \"$(cat "$__object/parameter/template")\""
fi
encoding=""
if [ -f "$__object/parameter/encoding" ]; then
encoding="--encoding \"$(cat "$__object/parameter/encoding")\""
fi
lc_collate=""
if [ -f "$__object/parameter/lc-collate" ]; then
lc_collate="--lc-collate \"$(cat "$__object/parameter/lc-collate")\""
fi
lc_ctype=""
if [ -f "$__object/parameter/lc-ctype" ]; then
lc_ctype="--lc-ctype \"$(cat "$__object/parameter/lc-ctype")\""
fi
cat << EOF
su - '$postgres_user' -c "createdb $owner \"$name\" $template $encoding $lc_collate $lc_ctype"
EOF
;;
absent)
cat << EOF
su - '$postgres_user' -c "dropdb \"$name\""
EOF
;;
esac
if test "${state_should}" = "$state_is"
then
exit 0
fi
case ${state_should}
in
(present)
set --
while read -r param_name opt
do
if test -f "${__object:?}/parameter/${param_name}"
then
set -- "$@" "${opt}" "$(cat "${__object:?}/parameter/${param_name}")"
fi
done <<-'EOF'
owner -O
template --template
encoding --encoding
lc_collate --lc-collate
lc_ctype --lc-ctype
EOF
set -- "$@" "${dbname}"
cat <<-EOF
su - $(quote "${postgres_user}") -c $(quote "$(quote createdb "$@")")
EOF
;;
(absent)
cat <<-EOF
su - $(quote "${postgres_user}") -c $(quote "$(quote dropdb "${dbname}")")
EOF
;;
esac

1
cdist/conf/type/__postgres_extension/explorer/postgres_user Symbolic link
View file

@ -0,0 +1 @@
../../__postgres_conf/explorer/postgres_user

41
cdist/conf/type/__postgres_extension/explorer/state Normal file
View file

@ -0,0 +1,41 @@
#!/bin/sh -e
# -*- mode: sh; indent-tabs-mode: t -*-
#
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
# Prints "present" if the extension is currently installed.
# "absent" otherwise.
quote() { printf '%s\n' "$*" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/'/"; }
postgres_user=$("${__type_explorer:?}/postgres_user")
IFS=: read -r dbname extname <<EOF
${__object_id:?}
EOF
psql_exec() {
su - "${postgres_user}" -c "psql $(quote "$1") -twAc $(quote "$2")"
}
if psql_exec "${dbname}" 'SELECT extname FROM pg_extension' | grep -qFx "${extname}"
then
echo present
else
echo absent
fi

63
cdist/conf/type/__postgres_extension/gencode-remote
View file

@ -2,9 +2,10 @@
#
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
# 2013 Tomas Pospisek (tpo_deb at sourcepole.ch)
# 2021 Dennis Camera (dennis.camera at ssrq-sds-fds.ch)
#
# This type was created by Tomas Pospisek based on the
#__postgres_role type by Steven Armstrong
# __postgres_role type by Steven Armstrong.
#
# This file is part of cdist.
#
@ -22,32 +23,38 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
case "$(cat "${__global}/explorer/os")"
postgres_user=$(cat "${__object:?}/explorer/postgres_user")
quote() { printf '%s\n' "$*" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/'/"; }
psql_cmd() {
printf 'su - %s -c %s\n' \
"$(quote "${postgres_user}")" \
"$(quote psql "$(quote "$1")" -c "$(quote "$2")")"
}
IFS=: read -r dbname extname <<EOF
${__object_id:?}
EOF
state_is=$(cat "${__object:?}/explorer/state")
state_should=$(cat "${__object:?}/parameter/state")
if test "${state_is}" = "${state_should}"
then
exit 0
fi
case ${state_should}
in
netbsd)
postgres_user='pgsql'
;;
openbsd)
postgres_user='_postgresql'
;;
*)
postgres_user='postgres'
;;
esac
dbname=$( echo "$__object_id" | cut -d":" -f1 )
extension=$( echo "$__object_id" | cut -d":" -f2 )
state_should=$( cat "$__object/parameter/state" )
case "$state_should" in
present)
cmd="CREATE EXTENSION IF NOT EXISTS $extension"
echo "su - '$postgres_user' -c 'psql -c \"$cmd\" \"$dbname\"'"
;;
absent)
cmd="DROP EXTENSION IF EXISTS $extension"
echo "su - '$postgres_user' -c 'psql -c \"$cmd\" \"$dbname\"'"
;;
(present)
psql_cmd "${dbname}" "CREATE EXTENSION ${extname}"
;;
(absent)
psql_cmd "${dbname}" "DROP EXTENSION ${extname}"
;;
(*)
printf 'Invalid --state: %s\n' "${state_should}" >&2
exit 1
;;
esac

55
cdist/conf/type/__postgres_extension/man.rst
View file

@ -3,32 +3,36 @@ cdist-type__postgres_extension(7)
NAME
----
cdist-type__postgres_extension - manage postgres extensions
cdist-type__postgres_extension - Manage PostgreSQL extensions
DESCRIPTION
-----------
This cdist type allows you to create or drop postgres extensions.
This cdist type allows you to manage PostgreSQL extensions.
The object you need to pass to __postgres_extension consists of
the database name and the extension name joined by a colon in the
following form:
.. code-block:: sh
dbname:extension
f.ex.
The ``__object_id`` to pass to ``__postgres_extension`` is of the form
``dbname:extension``, e.g.:
.. code-block:: sh
rails_test:unaccent
**CAUTION!** Be careful when installing extensions from (untrusted) third-party
sources:
| Installing an extension as superuser requires trusting that the extension's
author wrote the extension installation script in a secure fashion. It is
not terribly difficult for a malicious user to create trojan-horse objects
that will compromise later execution of a carelessly-written extension
script, allowing that user to acquire superuser privileges.
| – `<https://www.postgresql.org/docs/13/sql-createextension.html#id-1.9.3.64.7>`_
OPTIONAL PARAMETERS
-------------------
state
either "present" or "absent", defaults to "present"
either ``present`` or ``absent``, defaults to ``present``.
EXAMPLES
@ -36,24 +40,29 @@ EXAMPLES
.. code-block:: sh
__postgres_extension rails_test:unaccent
__postgres_extension --present rails_test:unaccent
__postgres_extension --absent rails_test:unaccent
# Install extension unaccent into database rails_test
__postgres_extension rails_test:unaccent
# Drop extension unaccent from database fails_test
__postgres_extension rails_test:unaccent --state absent
SEE ALSO
--------
:strong:`cdist-type__postgre_database`\ (7)
- :strong:`cdist-type__postgres_database`\ (7)
- PostgreSQL "CREATE EXTENSION" documentation at:
`<http://www.postgresql.org/docs/current/static/sql-createextension.html>`_.
Postgres "Create Extension" documentation at: <http://www.postgresql.org/docs/current/static/sql-createextension.html>.
AUTHOR
AUTHORS
-------
Tomas Pospisek <tpo_deb--@--sourcepole.ch>
| Tomas Pospisek <tpo_deb--@--sourcepole.ch>
| Dennis Camera <dennis.camera--@--ssrq-sds-fds.ch>
COPYING
-------
Copyright \(C) 2014 Tomas Pospisek. You can redistribute it
and/or modify it under the terms of the GNU General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.
Copyright \(C) 2014 Tomas Pospisek, 2021 Dennis Camera.
You can redistribute it and/or modify it under the terms of the GNU General
Public License as published by the Free Software Foundation, either version 3 of
the License, or (at your option) any later version.

1
cdist/conf/type/__postgres_role/explorer/postgres_user Symbolic link
View file

@ -0,0 +1 @@
../../__postgres_conf/explorer/postgres_user

25
cdist/conf/type/__postgres_role/explorer/state
View file

@ -19,19 +19,7 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
case $("${__explorer:?}/os")
in
(netbsd)
postgres_user='pgsql'
;;
(openbsd)
postgres_user='_postgresql'
;;
(*)
postgres_user='postgres'
;;
esac
postgres_user=$("${__type_explorer:?}/postgres_user")
rolename=${__object_id:?}
@ -55,8 +43,7 @@ role_properties=$(
BEGIN { RS = "\036"; FS = "\034" }
/^\([0-9]+ rows?\)/ { exit }
NR == 1 { for (i = 1; i <= NF; i++) cols[i] = $i; next }
NR == 2 { for (i = 1; i <= NF; i++) printf "%s=%s\n", cols[i], $i }
'
NR == 2 { for (i = 1; i <= NF; i++) printf "%s=%s\n", cols[i], $i }'
)
if test -n "${role_properties}"
@ -90,12 +77,10 @@ then
# Check password
passwd_stored=$(
psql_query "SELECT rolpassword FROM pg_authid WHERE rolname = '${rolename}'" \
| awk 'BEGIN { RS = "\036" } NR == 2'
printf .
)
passwd_stored=${passwd_stored%?.}
| awk 'BEGIN { RS = "\036" } NR == 2 { printf "%s.", $0 }')
passwd_stored=${passwd_stored%.}
if test -f "${__object:?}/parameter/password"
if test -s "${__object:?}/parameter/password"
then
passwd_should=$(cat "${__object:?}/parameter/password"; printf .)
fi

17
cdist/conf/type/__postgres_role/gencode-remote
View file

@ -28,20 +28,7 @@ quote() {
fi | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/'/"
}
case $(cat "${__global:?}/explorer/os")
in
(netbsd)
postgres_user='pgsql'
;;
(openbsd)
postgres_user='_postgresql'
;;
(*)
postgres_user='postgres'
;;
esac
postgres_user=$(cat "${__object:?}/explorer/postgres_user")
rolename=${__object_id:?}
state_is=$(cat "${__object:?}/explorer/state")
state_should=$(cat "${__object:?}/parameter/state")
@ -59,7 +46,7 @@ psql_query() {
psql_set_password() {
# NOTE: Always make sure that the password does not end up in psql_history!
# NOTE: Never set an empty string as the password, because they can be
# NOTE: Never set an empty string as the password, because it can be
# interpreted differently by different tooling.
if test -s "${__object:?}/parameter/password"
then

7
cdist/conf/type/__pyvenv/explorer/group
View file

@ -14,6 +14,11 @@ then
then
printf '%u\n' "${group_gid}"
else
printf '%s\n' "$(id -u -n "${group_gid}")"
if command -v getent >/dev/null 2>&1
then
getent group "${group_gid}" | cut -d : -f 1
else
awk -F: -v gid="${group_gid}" '$3 == gid { print $1 }' /etc/group
fi
fi
fi

2
cdist/conf/type/__rsync/gencode-local
View file

@ -34,6 +34,8 @@ if [ -f "$__object/parameter/rsync-opts" ]; then
done < "$__object/parameter/rsync-opts"
fi
# shellcheck disable=SC2086
echo rsync -a \
--no-owner --no-group \
-e \"${__remote_exec}\" \
-q "$@" "${source}/" "${remote_user}@${__target_host}:${destination}"

8
cdist/conf/type/__snakeoil_cert/explorer/ssl-cert-group Executable file
View file

@ -0,0 +1,8 @@
#!/bin/sh -e
if grep -Eq '^ssl-cert:' /etc/group
then
echo 'present'
else
echo 'absent'
fi

24
cdist/conf/type/__snakeoil_cert/explorer/state Executable file
View file

@ -0,0 +1,24 @@
#!/bin/sh -e
key_path="$( cat "$__object/parameter/key-path" )"
if echo "$key_path" | grep -Fq '%s'
then
# shellcheck disable=SC2059
key_path="$( printf "$key_path" "$__object_id" )"
fi
cert_path="$( cat "$__object/parameter/cert-path" )"
if echo "$cert_path" | grep -Fq '%s'
then
# shellcheck disable=SC2059
cert_path="$( printf "$cert_path" "$__object_id" )"
fi
if [ ! -f "$key_path" ] || [ ! -f "$cert_path" ]
then
echo 'absent'
else
echo 'present'
fi

73
cdist/conf/type/__snakeoil_cert/gencode-remote Executable file
View file

@ -0,0 +1,73 @@
#!/bin/sh -e
state="$( cat "$__object/explorer/state" )"
if [ "$state" = 'present' ]
then
exit 0
fi
if [ -f "$__object/parameter/common-name" ]
then
common_name="$( cat "$__object/parameter/common-name" )"
else
common_name="$__object_id"
fi
key_path="$( cat "$__object/parameter/key-path" )"
if echo "$key_path" | grep -Fq '%s'
then
# shellcheck disable=SC2059
key_path="$( printf "$key_path" "$__object_id" )"
fi
cert_path="$( cat "$__object/parameter/cert-path" )"
if echo "$cert_path" | grep -Fq '%s'
then
# shellcheck disable=SC2059
cert_path="$( printf "$cert_path" "$__object_id" )"
fi
key_type="$( cat "$__object/parameter/key-type" )"
key_type_arg="$( echo "$key_type" | cut -d : -f 2 )"
case "$key_type" in
rsa:*)
echo "openssl genrsa -out '$key_path' $key_type_arg"
;;
ec:*)
echo "openssl ecparam -name $key_type_arg -genkey -noout -out '$key_path'"
;;
esac
# shellcheck disable=SC2016
echo 'csr_path="$( mktemp )"'
echo "openssl req -new -subj '/CN=$common_name' -key '$key_path' -out \"\$csr_path\""
echo "openssl x509 -req -sha256 -days 3650 -in \"\$csr_path\" -signkey '$key_path' -out '$cert_path'"
# shellcheck disable=SC2016
echo 'rm -f "$csr_path"'
if [ "$( cat "$__object/explorer/ssl-cert-group" )" = 'present' ]
then
key_group='ssl-cert'
else
key_group='root'
fi
echo "chmod 640 '$key_path'"
echo "chown root '$key_path'"
echo "chgrp $key_group '$key_path'"
echo "chmod 644 '$cert_path'"
echo "chown root '$cert_path'"
echo "chgrp root '$cert_path'"

60
cdist/conf/type/__snakeoil_cert/man.rst Normal file
View file

@ -0,0 +1,60 @@
cdist-type__snakeoil_cert(7)
============================
NAME
----
cdist-type__snakeoil_cert - Generate self-signed certificate
DESCRIPTION
-----------
The purpose of this type is to generate **self-signed** certificate and private key
for **testing purposes**. Certificate will expire in 3650 days.
Certificate's and key's access bits will be ``644`` and ``640`` respectively.
If target system has ``ssl-cert`` group, then it will be used as key's group.
Use ``require='__snakeoil_cert/...' __file ...`` to override.
OPTIONAL PARAMETERS
-------------------
common-name
Defaults to ``$__object_id``.
key-path
``%s`` in path will be replaced with ``$__object_id``.
Defaults to ``/etc/ssl/private/%s.pem``.
key-type
Possible values are ``rsa:$bits`` and ``ec:$name``.
For possible EC names see ``openssl ecparam -list_curves``.
Defaults to ``rsa:2048``.
cert-path
``%s`` in path will be replaced with ``$__object_id``.
Defaults to ``/etc/ssl/certs/%s.pem``.
EXAMPLES
--------
.. code-block:: sh
__snakeoil_cert localhost-rsa \
--common-name localhost \
--key-type rsa:4096
__snakeoil_cert localhost-ec \
--common-name localhost \
--key-type ec:prime256v1
AUTHORS
-------
Ander Punnar <ander-at-kvlt-dot-ee>
COPYING
-------
Copyright \(C) 2021 Ander Punnar. You can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the Free
Software Foundation, either version 3 of the License, or (at your option)
any later version.

1
cdist/conf/type/__snakeoil_cert/parameter/default/cert-path Normal file
View file

@ -0,0 +1 @@
/etc/ssl/certs/%s.pem

1
cdist/conf/type/__snakeoil_cert/parameter/default/key-path Normal file
View file

@ -0,0 +1 @@
/etc/ssl/private/%s.pem

1
cdist/conf/type/__snakeoil_cert/parameter/default/key-type Normal file
View file

@ -0,0 +1 @@
rsa:2048

4
cdist/conf/type/__snakeoil_cert/parameter/optional Normal file
View file

@ -0,0 +1,4 @@
common-name
key-path
key-type
cert-path

92
cdist/scan/commandline.py
View file

@ -20,36 +20,98 @@
#
import logging
import sys
from datetime import datetime
log = logging.getLogger("scan")
# define this outside of the class to not handle scapy import errors by default
def commandline(args):
log.debug(args)
try:
import cdist.scan.scan as scan
except ModuleNotFoundError:
print('cdist scan requires scapy to be installed')
def run(scan, args):
# We run each component in a separate process since they
# must not block on each other.
processes = []
if not args.mode:
# By default scan and trigger, but do not call any action
args.mode = ['scan', 'trigger', ]
if 'trigger' in args.mode:
t = scan.Trigger(interfaces=args.interfaces)
t = scan.Trigger(interfaces=args.interface,
sleeptime=args.trigger_delay)
t.start()
processes.append(t)
log.debug("Trigger started")
if 'scan' in args.mode:
s = scan.Scanner(interfaces=args.interfaces, args=args)
s = scan.Scanner(
autoconfigure='config' in args.mode,
interfaces=args.interface,
name_mapper=args.name_mapper)
s.start()
processes.append(s)
log.debug("Scanner started")
for process in processes:
process.join()
def list(scan, args):
s = scan.Scanner(interfaces=args.interface, name_mapper=args.name_mapper)
hosts = s.list()
# A full IPv6 addresses id composed of 8 blocks of 4 hexa chars +
# 6 colons.
ipv6_max_size = 8 * 4 + 10
date_max_size = len(datetime.now().strftime(scan.datetime_format))
name_max_size = 25
print("{} | {} | {} | {}".format(
'name'.ljust(name_max_size),
'address'.ljust(ipv6_max_size),
'last seen'.ljust(date_max_size),
'last configured'.ljust(date_max_size)))
print('=' * (name_max_size + 3 + ipv6_max_size + 2 * (3 + date_max_size)))
for host in hosts:
last_seen = host.last_seen()
if last_seen:
last_seen = last_seen.strftime(scan.datetime_format)
else:
last_seen = '-'
last_configured = host.last_configured()
if last_configured is not None:
last_configured = last_configured.strftime(scan.datetime_format)
else:
last_configured = '-'
print("{} | {} | {} | {}".format(
host.name(default='-').ljust(name_max_size),
host.address().ljust(ipv6_max_size),
last_seen.ljust(date_max_size),
last_configured.ljust(date_max_size)))
# CLI processing is defined outside of the main scan class to handle
# non-available optional scapy dependency (instead of crashing mid-flight).
def commandline(args):
log.debug(args)
# Check if we have the optional scapy dependency available.
try:
import cdist.scan.scan as scan
except ModuleNotFoundError:
log.error('cdist scan requires scapy to be installed. Exiting.')
sys.exit(1)
# Set default operation mode.
if not args.mode:
# By default scan and trigger, but do not call any action.
args.mode = ['scan', 'trigger', ]
if 'config' in args.mode and args.name_mapper is None:
print('--name-mapper must be specified for scanner config mode.',
file=sys.stderr)
sys.exit(1)
# Print known hosts and exit is --list is specified - do not start
# the scanner.
if args.list:
list(scan, args)
else:
run(scan, args)

237
cdist/scan/scan.py
View file

@ -19,38 +19,6 @@
#
#
#
# Interface to be implemented:
# - cdist scan --mode {scan, trigger, install, config}, --mode can be repeated
# scan: scan / listen for icmp6 replies
# trigger: send trigger to multicast
# config: configure newly detected hosts
# install: install newly detected hosts
#
# Scanner logic
# - save results to configdir:
# basedir = ~/.cdist/scan/<ipv6-address>
# last_seen = ~/.cdist/scan/<ipv6-address>/last_seen -- record unix time
# or similar
# last_configured = ~/.cdist/scan/<ipv6-address>/last_configured -- record
# unix time or similar
# last_installed = ~/.cdist/scan/<ipv6-address>/last_configured -- record
# unix time or similar
#
#
#
#
# cdist scan --list
# Show all known hosts including last seen flag
#
# Logic for reconfiguration:
#
# - record when configured last time
# - introduce a parameter --reconfigure-after that takes time argument
# - reconfigure if a) host alive and b) reconfigure-after time passed
#
from multiprocessing import Process
import os
import logging
@ -61,7 +29,84 @@ import datetime
import cdist.config
logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger("scan")
datetime_format = '%Y-%m-%d %H:%M:%S'
class Host(object):
def __init__(self, addr, outdir, name_mapper=None):
self.addr = addr
self.workdir = os.path.join(outdir, addr)
self.name_mapper = name_mapper
os.makedirs(self.workdir, exist_ok=True)
def __get(self, key, default=None):
fname = os.path.join(self.workdir, key)
value = default
if os.path.isfile(fname):
with open(fname, "r") as fd:
value = fd.readline()
return value
def __set(self, key, value):
fname = os.path.join(self.workdir, key)
with open(fname, "w") as fd:
fd.write(f"{value}")
def name(self, default=None):
if self.name_mapper is None:
return default
fpath = os.path.join(os.getcwd(), self.name_mapper)
if os.path.isfile(fpath) and os.access(fpath, os.X_OK):
out = subprocess.run([fpath, self.addr], capture_output=True)
if out.returncode != 0:
return default
else:
value = out.stdout.decode()
return (default if len(value) == 0 else value)
else:
return default
def address(self):
return self.addr
def last_seen(self, default=None):
raw = self.__get('last_seen')
if raw:
return datetime.datetime.strptime(raw, datetime_format)
else:
return default
def last_configured(self, default=None):
raw = self.__get('last_configured')
if raw:
return datetime.datetime.strptime(raw, datetime_format)
else:
return default
def seen(self):
now = datetime.datetime.now().strftime(datetime_format)
self.__set('last_seen', now)
# XXX: There's no easy way to use the config module without feeding it with
# CLI args. Might as well call everything from scratch!
def configure(self):
target = self.name() or self.address()
cmd = ['cdist', 'config', '-v', target]
fname = os.path.join(self.workdir, 'last_configuration_log')
with open(fname, "w") as fd:
log.debug("Executing: %s", cmd)
completed_process = subprocess.run(cmd, stdout=fd, stderr=fd)
if completed_process.returncode != 0:
log.error("%s return with non-zero code %i - see %s for \
details.", cmd, completed_process.returncode, fname)
now = datetime.datetime.now().strftime(datetime_format)
self.__set('last_configured', now)
class Trigger(object):
@ -69,12 +114,14 @@ class Trigger(object):
Trigger an ICMPv6EchoReply from all hosts that are alive
"""
def __init__(self, interfaces=None, verbose=False):
def __init__(self, interfaces, sleeptime, verbose=False):
self.interfaces = interfaces
# Used by scapy / send in trigger/2.
self.verbose = verbose
# Wait 5 seconds before triggering again - FIXME: add parameter
self.sleeptime = 5
# Delay in seconds between sent ICMPv6EchoRequests.
self.sleeptime = sleeptime
def start(self):
self.processes = []
@ -93,9 +140,14 @@ class Trigger(object):
time.sleep(self.sleeptime)
def trigger(self, interface):
packet = IPv6(dst="ff02::1{}".format(interface)) / ICMPv6EchoRequest()
log.debug("Sending request on %s", interface)
send(packet, verbose=self.verbose)
try:
log.debug("Sending ICMPv6EchoRequest on %s", interface)
packet = IPv6(
dst="ff02::1%{}".format(interface)
) / ICMPv6EchoRequest()
send(packet, verbose=self.verbose)
except Exception as e:
log.error("Could not send ICMPv6EchoRequest: %s", e)
class Scanner(object):
@ -103,41 +155,62 @@ class Scanner(object):
Scan for replies of hosts, maintain the up-to-date database
"""
def __init__(self, interfaces=None, args=None, outdir=None):
def __init__(self, interfaces, autoconfigure=False, outdir=None,
name_mapper=None):
self.interfaces = interfaces
self.autoconfigure = autoconfigure
self.name_mapper = name_mapper
self.config_delay = datetime.timedelta(seconds=3600)
if outdir:
self.outdir = outdir
else:
self.outdir = os.path.join(os.environ['HOME'], '.cdist', 'scan')
os.makedirs(self.outdir, exist_ok=True)
self.running_configs = {}
def handle_pkg(self, pkg):
if ICMPv6EchoReply in pkg:
host = pkg['IPv6'].src
log.verbose("Host %s is alive", host)
host = Host(pkg['IPv6'].src, self.outdir, self.name_mapper)
if host.name():
log.verbose("Host %s (%s) is alive", host.name(),
host.address())
else:
log.verbose("Host %s is alive", host.address())
dir = os.path.join(self.outdir, host)
fname = os.path.join(dir, "last_seen")
host.seen()
now = datetime.datetime.now()
# Configure if needed.
if self.autoconfigure and \
host.last_configured(default=datetime.datetime.min) + \
self.config_delay < datetime.datetime.now():
self.config(host)
os.makedirs(dir, exist_ok=True)
def list(self):
hosts = []
for addr in os.listdir(self.outdir):
hosts.append(Host(addr, self.outdir, self.name_mapper))
# FIXME: maybe adjust the format so we can easily parse again
with open(fname, "w") as fd:
fd.write(f"{now}\n")
return hosts
def config(self):
"""
Configure a host
def config(self, host):
if host.name() is None:
log.debug("config - could not resolve name for %s, aborting.",
host.address())
return
- Assume we are only called if necessary
- However we need to ensure to not run in parallel
- Maybe keep dict storing per host processes
- Save the result
- Save the output -> probably aligned to config mode
previous_config_process = self.running_configs.get(host.name())
if previous_config_process is not None and \
previous_config_process.is_alive():
log.debug("config - is already running for %s, aborting.",
host.name())
"""
log.info("config - running against host %s (%s).", host.name(),
host.address())
p = Process(target=host.configure())
p.start()
self.running_configs[host.name()] = p
def start(self):
self.process = Process(target=self.scan)
@ -148,47 +221,9 @@ class Scanner(object):
def scan(self):
log.debug("Scanning - zzzzz")
sniff(iface=self.interfaces,
filter="icmp6",
prn=self.handle_pkg)
if __name__ == '__main__':
t = Trigger(interfaces=["wlan0"])
t.start()
# Scanner can listen on many interfaces at the same time
s = Scanner(interfaces=["wlan0"])
s.scan()
# Join back the trigger processes
t.join()
# Test in my lan shows:
# [18:48] bridge:cdist% ls -1d fe80::*
# fe80::142d:f0a5:725b:1103
# fe80::20d:b9ff:fe49:ac11
# fe80::20d:b9ff:fe4c:547d
# fe80::219:d2ff:feb2:2e12
# fe80::21b:fcff:feee:f446
# fe80::21b:fcff:feee:f45c
# fe80::21b:fcff:feee:f4b1
# fe80::21b:fcff:feee:f4ba
# fe80::21b:fcff:feee:f4bc
# fe80::21b:fcff:feee:f4c1
# fe80::21d:72ff:fe86:46b
# fe80::42b0:34ff:fe6f:f6f0
# fe80::42b0:34ff:fe6f:f863
# fe80::42b0:34ff:fe6f:f9b2
# fe80::4a5d:60ff:fea1:e55f
# fe80::77a3:5e3f:82cc:f2e5
# fe80::9e93:4eff:fe6c:c1f4
# fe80::ba69:f4ff:fec5:6041
# fe80::ba69:f4ff:fec5:8db7
# fe80::bad8:12ff:fe65:313d
# fe80::bad8:12ff:fe65:d9b1
# fe80::ce2d:e0ff:fed4:2611
# fe80::ce32:e5ff:fe79:7ea7
# fe80::d66d:6dff:fe33:e00
# fe80::e2ff:f7ff:fe00:20e6
# fe80::f29f:c2ff:fe7c:275e
try:
sniff(iface=self.interfaces,
filter="icmp6",
prn=self.handle_pkg)
except Exception as e:
log.error("Could not start listener: %s", e)

14
docs/changelog
View file

@ -5,6 +5,20 @@ next:
* Core: Add trigger functionality (Nico Schottelius, Darko Poljak)
* Core: Implement core support for python types (Darko Poljak)
6.9.7: 2021-07-10
* New type: __postgres_conf (Beni Ruef, Dennis Camera)
* Types __postgres_*: Improve OS support and do some cleanup (Dennis Camera)
* Type __apt_key_uri: Deprecate in favour of __apt_key --uri (Evilham)
* Type __apt_key: Documentation improvements, support in-type/in-manifest provision with --source, make fallback to apt-key(8) explicit with --use-deprecated-apt-key (Evilham)
* Type __letsencrypt_cert: Bugfix, performance; revamp explorers, add locking (Evilham)
* Type __git: Fix group explorer (Ander Punnar)
* Type __pyvenv: Fix group explorer (Dennis Camera)
* Type __download: Improve checksum verification, add optional --destination (Ander Punnar)
* Type __debconf_set_selections: Add state explorer (Dennis Camera)
* Core: Implement usable cdist scan (Timothée Floure)
* New type: __snakeoil_cert (Ander Punnar)
* Type __rsync: Honour $__remote_exec env var (Daniel Fancsali)
6.9.6: 2021-04-20
* Type __pyvenv: Fix user example in man page (Dennis Camera)
* Core: config: Make local state directory available to custom remotes (Steven Armstrong

82
docs/src/cdist-scan.rst Normal file
View file

@ -0,0 +1,82 @@
Scan
=====
Description
-----------
Runs cdist as a daemon that discover/watch on hosts and reconfigure them
periodically. It is especially useful in netboot-based environment where hosts
boot unconfigured, and to ensure your infrastructure stays in sync with your
configuration.
This feature is still consider to be in **beta** stage, and only operate on
IPv6 (including link-local).
Usage (Examples)
----------------
Discover hosts on local network and configure those whose name is resolved by
the name mapper script.
.. code-block:: sh
$ cdist scan --beta --interface eth0 \
--mode scan --name-mapper path/to/script \
--mode trigger --mode config
List known hosts and exit.
.. code-block:: sh
$ cdist scan --beta --list --name-mapper path/to/script
Please refer to `cdist(1)` for a detailed list of parameters.
Modes
-----
The scanner has 3 modes that can be independently toggled. If the `--mode`
parameter is not specified, only `tigger` and `scan` are enabled (= hosts are
not configured).
trigger
Send ICMPv6 requests to specific hosts or broadcast over IPv6 link-local to
trigger detection by the `scan` module.
scan
Watch for incoming ICMPv6 replies and optionally configure detected hosts.
config
Enable configuration of hosts detected by `scan`.
Name Mapper Script
------------------
The name mapper script takes an IPv6 address as first argument and writes the
resolved name to stdout - if any. The script must be executable.
Simplest script:
.. code-block:: sh
#!/bin/sh
case "$1" in
"fe80::20d:b9ff:fe57:3524")
printf "my-host-01"
;;
"fe80::7603:bdff:fe05:89bb")
printf "my-host-02"
;;
esac
Resolving name from `PTR` DNS record:
.. code-block:: sh
#!/bin/sh
for cmd in dig sed; do
if ! command -v $cmd > /dev/null; then
exit 1
fi
done
dig +short -x "$1" | sed -e 's/.$//'

1
docs/src/index.rst
View file

@ -35,6 +35,7 @@ It natively supports IPv6 since the first release.
cdist-inventory
cdist-trigger
cdist-preos
cdist-scan
cdist-integration
cdist-reference
cdist-best-practice

27
docs/src/man1/cdist.rst
View file

@ -97,6 +97,8 @@ SYNOPSIS
[-R [{tar,tgz,tbz2,txz}]] [-r REMOTE_OUT_PATH]
[--remote-copy REMOTE_COPY] [--remote-exec REMOTE_EXEC]
[-S] [-D DIRECTORY] [-H HTTP_PORT] [--ipv6] [-O SOURCE]
cdist scan -I INTERFACE [--m MODE] [--name-mapper PATH_TO_SCRIPT] [--list]
[-d CONFIG_DELAY] [-t TRIGGER_DELAY]
DESCRIPTION
@ -748,6 +750,31 @@ This command returns the following response codes to client requests:
**-S, --disable-saving-output-streams**
Disable saving output streams.
SCAN
----
Runs cdist as a daemon that discover/watch on hosts and reconfigure them
periodically.
**-I INTERFACE, --interfaces INTERFACE**
Interface to listen on. Can be specified multiple times.
**-m MODE, --mode MODE**
Scanner components to enable. Can be specified multiple time to enable more
than one component. Supported modes are: scan, trigger and config. Defaults
to tiggger and scan.
**--name-mapper PATH_TO_SCRIPT**
Path to script used to resolve a remote host name from an IPv6 address.
**--list**
List known hosts and exit.
**-d CONFIG_DELAY, --config-delay CONFIG_DELAY**
How long (seconds) to wait before reconfiguring after last try (config mode only).
**-t TRIGGER_DELAY, --tigger-delay TRIGGER_DELAY**
How long (seconds) to wait between ICMPv6 echo requests (trigger mode only).
CONFIGURATION
-------------