Compare commits

..

53 commits

Author SHA1 Message Date
Nico Schottelius
7dd2d1025a ++changelog 2023-02-03 22:54:18 +01:00
513a8ae177 Merge pull request 'Make sure flag is followed by end of line or space.' (#349) from mark/cdist:machine_type into master
Reviewed-on: ungleich-public/cdist#349
2023-02-03 21:53:12 +00:00
2a2f91959e Merge pull request 'Updated the python version of cerbot freebsd' (#359) from CamilionEU/cdist:cerbot-freebsd into master
Reviewed-on: ungleich-public/cdist#359
2023-02-03 21:43:02 +00:00
fcf76cdb2c Merge pull request 'Added support for Devuan Daedalus' (#358) from CamilionEU/cdist:explorer-devuan-update into master
Reviewed-on: ungleich-public/cdist#358
2023-02-03 21:40:33 +00:00
Michelle
1450861e26
Updated the python version of cerbot freebsd
The package referenced for cerbot to be install in the FreeBSD platform
used python 3.7 package, updated to python 3.9
2023-02-02 18:21:43 -05:00
Michelle
08a6b467fa
Added support for Devuan Daedalus
Added one line that allows cdist to support Devuan Daedelus version
2023-01-25 16:06:35 -05:00
Nico Schottelius
ed3da3c829 ++changes 2022-12-26 21:02:41 +01:00
Nico Schottelius
b974969f28 Remove double definition of scan parser
Fixes #353
2022-12-26 20:59:16 +01:00
Nico Schottelius
bdfd92dc37 ++changes 2022-12-21 09:41:33 +01:00
5c85b04309 Merge pull request 'Initialise options variable to avoid unset variable expansion' (#351) from mark/cdist:options-initialise into master
Reviewed-on: ungleich-public/cdist#351
2022-12-20 17:04:45 +00:00
Nico Schottelius
f36069754c ++changelog 2022-12-20 18:03:15 +01:00
d4dfe95a97 Merge pull request 'Make grep more specific' (#352) from marcoduif/cdist:master into master
Reviewed-on: ungleich-public/cdist#352
2022-12-20 17:02:16 +00:00
ffeaafe9b6 Make grep more specific
package name should be an exact match, not a substring
2022-10-07 07:22:31 +00:00
Mark Verboom
62db96bb37 Initialise options variable so expansion when running files/source.list.template
there will not be an error when the variable is not set.
2022-09-29 16:19:07 +02:00
Mark Verboom
c85184dcb4 Make sure flag is followed by end of line or space. 2022-09-18 08:49:37 +02:00
Nico Schottelius
90488d2e9e [doc] add release process documentation 2022-08-01 00:03:51 +02:00
Nico Schottelius
be6e7fcc08 Prepare release of cdist 7.0.0 2022-07-31 21:59:35 +02:00
Nico Schottelius
d4bf41ce3b ++changelog 2022-07-29 10:57:01 +02:00
7de931829a Merge pull request 'Add Check Point Gaia (FW1) management and firewall appliances to explorers' (#339) from stephan/cdist:master into master
Reviewed-on: ungleich-public/cdist#339
2022-07-29 08:56:09 +00:00
17466452f0 revert __line for clean PR history 2022-07-28 17:53:41 +02:00
7d8fc8a5c3 improve checkpoint sed, add __line changes 2022-07-28 17:18:41 +02:00
6243165645 add create and ifexists to line type 2022-07-28 16:27:12 +02:00
483f0c1614 add Check Point Gaia 2022-07-13 14:50:17 +02:00
ff6b2d0abf Merge pull request 'master' (#2) from ungleich-public/cdist:master into master
Reviewed-on: stephan/cdist#2
2022-07-13 11:58:31 +00:00
Nico Schottelius
339ca9347b ++changelog 2022-07-02 19:21:27 +02:00
5a7542db75 Merge pull request 'Handle signed-by option in __apt_source' (#335) from fancsali/cdist:apt-source-signed-by into master
Reviewed-on: ungleich-public/cdist#335
2022-07-02 17:20:29 +00:00
0ae37b3445 Handle signed-by option in __apt_source
Allow users to specify a GPG key fingerprint or keyring file to be
included as the 'signed-by' option.
2022-07-01 16:14:38 +01:00
5e6cde1398 Merge pull request 'master' (#1) from ungleich-public/cdist:master into master
Reviewed-on: stephan/cdist#1
2022-05-20 13:17:47 +00:00
Nico Schottelius
77d9a757ec ++changelog 2022-05-20 14:58:45 +02:00
e5adcf451b Merge pull request 'bug: apt-ppa-noninteractive' (#327) from romain-dartigues/cdist:apt-ppa-noninteractive into master
Reviewed-on: ungleich-public/cdist#327
2022-05-20 12:57:08 +00:00
Nico Schottelius
9839c2d8ec ++changelog
Signed-off-by: Nico Schottelius <nico@nico-notebook.schottelius.org>
2022-05-20 14:55:12 +02:00
1edc4d0a60 Merge pull request 'add optional file parameter to allow for use in a loop without object_id clashes' (#334) from stephan/cdist:master into master
Reviewed-on: ungleich-public/cdist#334
2022-05-20 12:53:13 +00:00
3d58c9b24f add optional file parameter to allow for use in a loop without object_id clashes 2022-05-20 13:48:07 +02:00
Steven Armstrong
6c8c692a22 __file: kiss and fix regression on Mac OSX
Signed-off-by: Steven Armstrong <steven@armstrong.cc>
2022-05-02 23:25:59 +02:00
Steven Armstrong
abbc7dfc37 since we already remove the destination, we have no need to use -T on move, fixes #333
Signed-off-by: Steven Armstrong <steven@armstrong.cc>
2022-04-16 19:05:31 +02:00
Steven Armstrong
8b915b15b5 __file: make the create-empty-file case work again
Signed-off-by: Steven Armstrong <steven@armstrong.cc>
2022-04-14 00:46:13 +02:00
Steven Armstrong
2df2578e36 __file: remove the questionable check for uploadfile existence
Signed-off-by: Steven Armstrong <steven@armstrong.cc>
2022-04-14 00:27:28 +02:00
Steven Armstrong
6f8c774cb0 workaround mktemp -u checking for write access
Signed-off-by: Steven Armstrong <steven@armstrong.cc>
2022-04-14 00:16:10 +02:00
54a5cb17b7 use add-apt-repository instead of add-apt-repository
Remove `remove-apt-repository` which is now no longer needed;
use `add-apt-repository` which allow removal through the `-r` flag.
2022-04-11 21:09:31 +02:00
cb0fa0f2e4 force add-apt-repository to act in non-interactive mode 2022-04-11 21:05:56 +02:00
Steven Armstrong
af54fe6feb changelog++
Signed-off-by: Steven Armstrong <steven@armstrong.cc>
2022-04-11 00:04:41 +02:00
Steven Armstrong
22039284f5 __file: make file uploading and attribute changes more atomic
Fixes ungleich-public/cdist#331

Signed-off-by: Steven Armstrong <steven@armstrong.cc>
2022-04-10 23:52:53 +02:00
bd44c023d3 Fix typos; add default priority; comments in generated files 2022-03-09 16:17:11 +01:00
Nico Schottelius
e0150e7796 ++changes 2022-03-09 16:16:49 +01:00
15e1ce6450 Merge pull request 'Added rm of tmpfile.' (#330) from mark/cdist:__ssh_authorized_keys-rm into master
Reviewed-on: ungleich-public/cdist#330
2022-03-09 15:12:21 +00:00
Mark Verboom
08ff41efde Added rm of tmpfile. 2022-03-08 12:04:58 +01:00
c2c5668b70 ++changelog 2021-12-23 20:08:49 +01:00
6e3ad11ea0 [__package_upgrade_all] Add new --apt-with-new-pkgs argument 2021-12-23 20:07:28 +01:00
fnux
fc6ddac718 Merge pull request 'Python 3.10: collections.X -> collections.abc.X' (#323) from py3.10 into master
Reviewed-on: ungleich-public/cdist#323
2021-12-16 13:04:51 +00:00
3a321469a8
Python 3.10: collections.X -> collections.abc.X 2021-12-02 12:02:36 +01:00
e2500248f2 ++changelog 2021-11-03 11:03:33 +01:00
0b710c6173 Merge branch 'haproxy-dualstack' into 'master'
[__haproxy_dualstack] New type with PROXY protocol support

See merge request ungleich-public/cdist!1027
2021-11-03 07:38:24 +01:00
c33d99ee12 [__haproxy_dualstack] New type with PROXY protocol support
This is backwards compatible with what is already used internally @ungleich, but
adds on top of that the ability to customise ports and, most importantly, it
adds PROXY protocol support.
2021-10-31 17:38:10 +01:00
44 changed files with 581 additions and 105 deletions

View file

@ -1,6 +1,6 @@
#!/bin/sh #!/bin/sh
# #
# 2011-2013 Nico Schottelius (nico-cdist at schottelius.org) # 2011-2022 Nico Schottelius (nico-cdist at schottelius.org)
# 2016-2019 Darko Poljak (darko.poljak at gmail.com) # 2016-2019 Darko Poljak (darko.poljak at gmail.com)
# #
# This file is part of cdist. # This file is part of cdist.

View file

@ -472,9 +472,6 @@ def get_parsers():
parser['info'].set_defaults(func=cdist.info.Info.commandline) parser['info'].set_defaults(func=cdist.info.Info.commandline)
# Scan = config + further # Scan = config + further
parser['scan'] = parser['sub'].add_parser('scan', add_help=False,
parents=[parser['config']])
parser['scan'] = parser['sub'].add_parser( parser['scan'] = parser['sub'].add_parser(
'scan', parents=[parser['loglevel'], 'scan', parents=[parser['loglevel'],
parser['beta'], parser['beta'],

View file

@ -21,6 +21,9 @@
set +e set +e
case "$("$__explorer/os")" in case "$("$__explorer/os")" in
checkpoint)
awk '{printf("%s\n", $(NF-1))}' /etc/cp-release
;;
openwrt) openwrt)
# shellcheck disable=SC1091 # shellcheck disable=SC1091
(. /etc/openwrt_release && echo "$DISTRIB_CODENAME") (. /etc/openwrt_release && echo "$DISTRIB_CODENAME")

View file

@ -21,6 +21,9 @@
set +e set +e
case "$("$__explorer/os")" in case "$("$__explorer/os")" in
checkpoint)
cat /etc/cp-release
;;
openwrt) openwrt)
# shellcheck disable=SC1091 # shellcheck disable=SC1091
(. /etc/openwrt_release && echo "$DISTRIB_DESCRIPTION") (. /etc/openwrt_release && echo "$DISTRIB_DESCRIPTION")

View file

@ -21,6 +21,9 @@
set +e set +e
case "$("$__explorer/os")" in case "$("$__explorer/os")" in
checkpoint)
echo "CheckPoint"
;;
openwrt) openwrt)
# shellcheck disable=SC1091 # shellcheck disable=SC1091
(. /etc/openwrt_release && echo "$DISTRIB_ID") (. /etc/openwrt_release && echo "$DISTRIB_ID")

View file

@ -21,6 +21,9 @@
set +e set +e
case "$("$__explorer/os")" in case "$("$__explorer/os")" in
checkpoint)
sed /etc/cp-release -e 's/.* R\([1-9][0-9]*\)\.[0-9]*$/\1/'
;;
openwrt) openwrt)
# shellcheck disable=SC1091 # shellcheck disable=SC1091
(. /etc/openwrt_release && echo "$DISTRIB_RELEASE") (. /etc/openwrt_release && echo "$DISTRIB_RELEASE")

View file

@ -515,7 +515,7 @@ check_vm_arch_specific() {
&& return 0 && return 0
fi fi
if has_cpuinfo \ if has_cpuinfo \
&& grep -q -i -e '^flags.*:.*\(hypervisor\|vmm\)' /proc/cpuinfo && grep -q -i -e '^flags.*:.*\(hypervisor\|vmm\)\( \|$\) /proc/cpuinfo
then then
return 0 return 0
fi fi

View file

@ -116,6 +116,13 @@ if [ -f /etc/slackware-version ]; then
exit 0 exit 0
fi fi
# Appliances
if grep -q '^Check Point Gaia' /etc/cp-release 2>/dev/null; then
echo checkpoint
exit 0
fi
uname_s="$(uname -s)" uname_s="$(uname -s)"
# Assume there is no tr on the client -> do lower case ourselves # Assume there is no tr on the client -> do lower case ourselves

View file

@ -34,5 +34,9 @@ elif test -f /var/run/os-release
then then
# FreeBSD (created by os-release service) # FreeBSD (created by os-release service)
cat /var/run/os-release cat /var/run/os-release
elif test -f /etc/cp-release
then
# Checkpoint firewall or management (actually linux based)
cat /etc/cp-release
fi fi

View file

@ -41,6 +41,9 @@ in
# empty, but well... # empty, but well...
cat /etc/arch-release cat /etc/arch-release
;; ;;
checkpoint)
awk '{version=$NF; printf("%s\n", substr(version, 2))}' /etc/cp-release
;;
debian) debian)
debian_version=$(cat /etc/debian_version) debian_version=$(cat /etc/debian_version)
case $debian_version case $debian_version
@ -79,6 +82,7 @@ in
# ceres versions don't have a number, so we decode by codename: # ceres versions don't have a number, so we decode by codename:
case ${devuan_version} case ${devuan_version}
in in
(daedalus/ceres) echo 4.99 ;;
(chimaera/ceres) echo 3.99 ;; (chimaera/ceres) echo 3.99 ;;
(beowulf/ceres) echo 2.99 ;; (beowulf/ceres) echo 2.99 ;;
(ascii/ceres) echo 1.99 ;; (ascii/ceres) echo 1.99 ;;

View file

@ -24,4 +24,4 @@ else
name="$__object_id" name="$__object_id"
fi fi
apt-mark showhold | grep -Fq "$name" && echo hold || echo unhold apt-mark showhold | grep -q "^${name}$" && echo hold || echo unhold

View file

@ -57,6 +57,11 @@ __file "/etc/apt/preferences.d/$name" \
--owner root --group root --mode 0644 \ --owner root --group root --mode 0644 \
--state "$state" \ --state "$state" \
--source - << EOF --source - << EOF
# Created by cdist ${__type##*/}
# Do not change. Changes will be overwritten.
#
# $name
Package: $package Package: $package
Pin: $pin Pin: $pin
Pin-Priority: $priority Pin-Priority: $priority

View file

@ -0,0 +1 @@
500

View file

@ -1,2 +1,3 @@
state state
package package
priority

View file

@ -1,2 +1 @@
distribution distribution
priority

View file

@ -1,55 +0,0 @@
#!/usr/bin/env python
#
# Remove the given apt repository.
#
# Exit with:
# 0: if it worked
# 1: if not
# 2: on other error
import os
import sys
from aptsources import distro, sourceslist
from softwareproperties import ppa
from softwareproperties.SoftwareProperties import SoftwareProperties
def remove_if_empty(file_name):
with open(file_name, 'r') as f:
if f.read().strip():
return
os.unlink(file_name)
def remove_repository(repository):
#print 'repository:', repository
codename = distro.get_distro().codename
#print 'codename:', codename
(line, file) = ppa.expand_ppa_line(repository.strip(), codename)
#print 'line:', line
#print 'file:', file
deb_source_entry = sourceslist.SourceEntry(line, file)
src_source_entry = sourceslist.SourceEntry('deb-src{}'.format(line[3:]), file)
try:
sp = SoftwareProperties()
sp.remove_source(deb_source_entry)
try:
# If there's a deb-src entry, remove that too
sp.remove_source(src_source_entry)
except:
pass
remove_if_empty(file)
return True
except ValueError:
print >> sys.stderr, "Error: '%s' doesn't exists in a sourcelist file" % line
return False
if __name__ == '__main__':
if (len(sys.argv) != 2):
print >> sys.stderr, 'Error: need a repository as argument'
sys.exit(2)
repository = sys.argv[1]
if remove_repository(repository):
sys.exit(0)
else:
sys.exit(1)

View file

@ -29,9 +29,9 @@ fi
case "$state_should" in case "$state_should" in
present) present)
echo "add-apt-repository '$name'" echo "add-apt-repository -y '$name'"
;; ;;
absent) absent)
echo "remove-apt-repository '$name'" echo "add-apt-repository -r -y '$name'"
;; ;;
esac esac

View file

@ -20,9 +20,4 @@
__package software-properties-common __package software-properties-common
require="__package/software-properties-common" \
__file /usr/local/bin/remove-apt-repository \
--source "$__type/files/remove-apt-repository" \
--mode 0755
require="$__object_name" __apt_update_index require="$__object_name" __apt_update_index

View file

@ -2,13 +2,14 @@
set -u set -u
entry="$uri $distribution $component" entry="$uri $distribution $component"
cat << DONE cat << DONE
# Created by cdist ${__type##*/} # Created by cdist ${__type##*/}
# Do not change. Changes will be overwritten. # Do not change. Changes will be overwritten.
# #
# $name # $name
deb ${forcedarch} $entry deb ${options} $entry
DONE DONE
if [ -f "$__object/parameter/include-src" ]; then if [ -f "$__object/parameter/include-src" ]; then
echo "deb-src $entry" echo "deb-src $entry"

View file

@ -23,6 +23,9 @@ OPTIONAL PARAMETERS
arch arch
set this if you need to force and specific arch (ubuntu specific) set this if you need to force and specific arch (ubuntu specific)
signed-by
provide a GPG key fingerprint or keyring path for signature checks
state state
'present' or 'absent', defaults to 'present' 'present' or 'absent', defaults to 'present'
@ -56,6 +59,11 @@ EXAMPLES
--uri http://archive.canonical.com/ \ --uri http://archive.canonical.com/ \
--component partner --state present --component partner --state present
__apt_source goaccess \
--uri http://deb.goaccess.io/ \
--component main \
--signed-by C03B48887D5E56B046715D3297BD1A0133449C3D
AUTHORS AUTHORS
------- -------

View file

@ -21,6 +21,7 @@
name="$__object_id" name="$__object_id"
state="$(cat "$__object/parameter/state")" state="$(cat "$__object/parameter/state")"
uri="$(cat "$__object/parameter/uri")" uri="$(cat "$__object/parameter/uri")"
options=""
if [ -f "$__object/parameter/distribution" ]; then if [ -f "$__object/parameter/distribution" ]; then
distribution="$(cat "$__object/parameter/distribution")" distribution="$(cat "$__object/parameter/distribution")"
@ -31,9 +32,15 @@ fi
component="$(cat "$__object/parameter/component")" component="$(cat "$__object/parameter/component")"
if [ -f "$__object/parameter/arch" ]; then if [ -f "$__object/parameter/arch" ]; then
forcedarch="[arch=$(cat "$__object/parameter/arch")]" options="arch=$(cat "$__object/parameter/arch")"
else fi
forcedarch=""
if [ -f "$__object/parameter/signed-by" ]; then
options="$options signed-by=$(cat "$__object/parameter/signed-by")"
fi
if [ "$options" ]; then
options="[$options]"
fi fi
# export variables for use in template # export variables for use in template
@ -41,7 +48,7 @@ export name
export uri export uri
export distribution export distribution
export component export component
export forcedarch export options
# generate file from template # generate file from template
mkdir "$__object/files" mkdir "$__object/files"

View file

@ -1,4 +1,5 @@
state state
distribution distribution
component component
arch arch
signed-by

View file

@ -37,6 +37,12 @@ state
source source
forwarded to :strong:`__file` type forwarded to :strong:`__file` type
file
forwarded to :strong:`__file` type
This can be used if multiple users need to have a dotfile updated,
which will result in duplicate object id errors. When using the
file parameter the object id can be some unique value.
MESSAGES MESSAGES
-------- --------
@ -61,6 +67,15 @@ EXAMPLES
# Install default xmonad config for user 'eve'. Parent directory is created automatically. # Install default xmonad config for user 'eve'. Parent directory is created automatically.
__dot_file .xmonad/xmonad.hs --user eve --state exists --source "$__files/xmonad.hs" __dot_file .xmonad/xmonad.hs --user eve --state exists --source "$__files/xmonad.hs"
# install .vimrc for root and some users
for user in root userx usery userz; do
__dot_file "${user}_dot_vimrc" \
--user $user \
--file .vimrc \
--state exists \
--source "$__files/$user/.vimrc"
done
SEE ALSO SEE ALSO
-------- --------

View file

@ -20,13 +20,19 @@ user="$(cat "${__object}/parameter/user")"
home="$(cat "${__object}/explorer/home")" home="$(cat "${__object}/explorer/home")"
primary_group="$(cat "${__object}/explorer/primary_group")" primary_group="$(cat "${__object}/explorer/primary_group")"
dirmode="$(cat "${__object}/parameter/dirmode")" dirmode="$(cat "${__object}/parameter/dirmode")"
if [ -f "${__object}/parameter/file" ]; then
file="$(cat "${__object}/parameter/file")"
else
file="${__object_id}"
fi
# Create parent directory. Type __directory has flag 'parents', but it # Create parent directory. Type __directory has flag 'parents', but it
# will leave us with root-owned directory in user home, which is not # will leave us with root-owned directory in user home, which is not
# acceptable. So we create parent directories one-by-one. XXX: maybe # acceptable. So we create parent directories one-by-one. XXX: maybe
# it should be fixed in '__directory'? # it should be fixed in '__directory'?
set -- set --
subpath=${__object_id} subpath=${file}
while subpath="$(dirname "${subpath}")" ; do while subpath="$(dirname "${subpath}")" ; do
[ "${subpath}" = . ] && break [ "${subpath}" = . ] && break
set -- "${subpath}" "$@" set -- "${subpath}" "$@"
@ -64,4 +70,4 @@ if [ "${source}" = "-" ] ; then
fi fi
unset source unset source
__file "${home}/${__object_id}" --owner "$user" --group "$primary_group" "$@" __file "${home}/${file}" --owner "$user" --group "$primary_group" "$@"

View file

@ -1,7 +1,7 @@
#!/bin/sh -e #!/bin/sh -e
# #
# 2011-2012 Nico Schottelius (nico-cdist at schottelius.org) # 2011-2012 Nico Schottelius (nico-cdist at schottelius.org)
# 2013 Steven Armstrong (steven-cdist armstrong.cc) # 2013-2022 Steven Armstrong (steven-cdist armstrong.cc)
# #
# This file is part of cdist. # This file is part of cdist.
# #
@ -72,6 +72,7 @@ if [ "$state_should" = "present" ] || [ "$state_should" = "exists" ]; then
if [ "$type" != "file" ]; then if [ "$type" != "file" ]; then
# destination is not a regular file, upload source to replace it # destination is not a regular file, upload source to replace it
upload_file=1 upload_file=1
echo upload >> "$__messages_out"
else else
local_cksum="$(cksum < "$source")" local_cksum="$(cksum < "$source")"
remote_cksum="$(cat "$__object/explorer/cksum")" remote_cksum="$(cat "$__object/explorer/cksum")"
@ -88,27 +89,39 @@ if [ "$state_should" = "present" ] || [ "$state_should" = "exists" ]; then
mkdir "$__object/files" mkdir "$__object/files"
touch "$__object/files/set-attributes" touch "$__object/files/set-attributes"
# upload file to temp location if [ "$create_file" ]; then
tempfile_template="${destination}.cdist.XXXXXXXXXX" # When creating an empty file we create it locally and then
cat << DONE # upload it so that permissions can be set before moving the file
destination_upload="\$($__remote_exec $__target_host "mktemp $tempfile_template")" # into place.
DONE source="$__object/files/empty"
if [ "$upload_file" ]; then touch "$source"
echo upload >> "$__messages_out"
# IPv6 fix
if echo "${__target_host}" | grep -q -E '^[0-9a-fA-F:]+$'
then
my_target_host="[${__target_host}]"
else
my_target_host="${__target_host}"
fi
cat << DONE
$__remote_copy "$source" "${my_target_host}:\$destination_upload"
DONE
fi fi
# move uploaded file into place
cat << DONE # upload file to temp location
$__remote_exec $__target_host "rm -rf \"$destination\"; mv \"\$destination_upload\" \"$destination\"" upload_destination="${destination}.cdist.${__cdist_object_marker}.$$"
# Yes, we are aware that this is a race condition.
# However:
# a) cdist usually writes to directories that are not user writable
# (probably > 99.9%)
# b) if they are user owned, the user / attacker always wins
# (probably < 0.1%)
# c) the only case which we could improve are tmp directories and we
# don't think managing tmp directories with cdist is a typical case
# ("the rest %)"
# Tell gencode-remote to where we uploaded the file so it can move
# it to its final destination.
echo "$upload_destination" > "$__object/files/upload-destination"
# IPv6 fix
if echo "${__target_host}" | grep -q -E '^[0-9a-fA-F:]+$'
then
my_target_host="[${__target_host}]"
else
my_target_host="${__target_host}"
fi
cat << DONE
$__remote_copy "$source" "${my_target_host}:${upload_destination}"
DONE DONE
fi fi
fi fi

View file

@ -1,7 +1,7 @@
#!/bin/sh -e #!/bin/sh -e
# #
# 2011-2013 Nico Schottelius (nico-cdist at schottelius.org) # 2011-2013 Nico Schottelius (nico-cdist at schottelius.org)
# 2013 Steven Armstrong (steven-cdist armstrong.cc) # 2013-2022 Steven Armstrong (steven-cdist armstrong.cc)
# #
# This file is part of cdist. # This file is part of cdist.
# #
@ -62,6 +62,13 @@ set_mode() {
case "$state_should" in case "$state_should" in
present|exists) present|exists)
if [ -f "$__object/files/upload-destination" ]; then
final_destination="$destination"
# We change the 'global' $destination variable here so we can
# change attributes of the new/uploaded file before moving it
# to it's final destination.
destination="$(cat "$__object/files/upload-destination")"
fi
# Note: Mode - needs to happen last as a chown/chgrp can alter mode by # Note: Mode - needs to happen last as a chown/chgrp can alter mode by
# clearing S_ISUID and S_ISGID bits (see chown(2)) # clearing S_ISUID and S_ISGID bits (see chown(2))
for attribute in group owner mode; do for attribute in group owner mode; do
@ -81,6 +88,11 @@ case "$state_should" in
fi fi
fi fi
done done
if [ -f "$__object/files/upload-destination" ]; then
# move uploaded file into place
printf 'rm -rf "%s"\n' "$final_destination"
printf 'mv "%s" "%s"\n' "$destination" "$final_destination"
fi
if [ -f "$__object/files/set-attributes" ]; then if [ -f "$__object/files/set-attributes" ]; then
# set-attributes is created if file is created or uploaded in gencode-local # set-attributes is created if file is created or uploaded in gencode-local
fire_onchange=1 fire_onchange=1

View file

@ -0,0 +1,8 @@
frontend http
bind BIND@:80
mode http
option httplog
default_backend http
backend http
mode http

View file

@ -0,0 +1,10 @@
frontend https
bind BIND@:443
mode tcp
option tcplog
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
default_backend https
backend https
mode tcp

View file

@ -0,0 +1,12 @@
frontend imaps
bind BIND@:143
bind BIND@:993
mode tcp
option tcplog
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
default_backend imaps
backend imaps
mode tcp

View file

@ -0,0 +1,12 @@
frontend smtps
bind BIND@:25
bind BIND@:465
mode tcp
option tcplog
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
default_backend smtps
backend smtps
mode tcp

View file

@ -0,0 +1,121 @@
cdist-type__haproxy_dualstack(7)
================================
NAME
----
cdist-type__haproxy_dualstack - Proxy services from a dual-stack server
DESCRIPTION
-----------
This (singleton) type installs and configures haproxy to act as a dual-stack
proxy for single-stack services.
This can be useful to add IPv4 support to IPv6-only services while only using
one IPv4 for many such services.
By default this type uses the plain TCP proxy mode, which means that there is no
need for TLS termination on this host when SNI is supported.
This also means that proxied services will not receive the client's IP address,
but will see the proxy's IP address instead (that of `$__target_host`).
This can be solved by using the PROXY protocol, but do take into account that,
e.g. nginx cannot serve both regular HTTP(S) and PROXY protocols on the same
port, so you will need to use other ports for that.
As a recommendation in this type: use TCP ports 8080 and 591 respectively to
serve HTTP and HTTPS using the PROXY protocol.
See the EXAMPLES for more details.
OPTIONAL PARAMETERS
-------------------
v4proxy
Proxy incoming IPv4 connections to the equivalent IPv6 endpoint.
In its simplest use, it must be a NAME with an `AAAA` DNS entry, which is
the IP address actually providing the proxied services.
The full format of this argument is:
`[proxy:]NAME[[:PROTOCOL_1=PORT_1]...[:PROTOCOL_N=PORT_N]]`
Where starting with `proxy:` determines that the PROXY protocol must be
used and each `:PROTOCOL=PORT` (e.g. `:http=8080` or `:https=591`) is a PORT
override for the given PROTOCOL (see `--protocol`), if not present the
PROTOCOL's default port will be used.
v6proxy
Proxy incoming IPv6 connections to the equivalent IPv4 endpoint.
In its simplest use, it must be a NAME with an `A` DNS entry, which is
the IP address actually providing the proxied services.
See `--v4proxy` for more options and details.
protocol
Can be passed multiple times or as a space-separated list of protocols.
Currently supported protocols are: `http`, `https`, `imaps`, `smtps`.
This defaults to: `http https imaps smtps`.
EXAMPLES
--------
.. code-block:: sh
# Proxy the IPv6-only services so IPv4-only clients can access them
# This uses HAProxy's TCP mode for http, https, imaps and smtps
__haproxy_dualstack \
--v4proxy ipv6.chat \
--v4proxy matrix.ungleich.ch
# Proxy the IPv6-only HTTP(S) services so IPv4-only clients can access them
# Note this means that the backend IPv6-only server will only see
# the IPv6 address of the haproxy host managed by cdist, which can be
# troublesome if this information is relevant for analytics/security/...
# See the PROXY example below
__haproxy_dualstack \
--protocol http --protocol https \
--v4proxy ipv6.chat \
--v4proxy matrix.ungleich.ch
# Use the PROXY protocol to proxy the IPv6-only HTTP(S) services enabling
# IPv4-only clients to access them while maintaining the client's IP address
__haproxy_dualstack \
--protocol http --protocol https \
--v4proxy proxy:ipv6.chat:http=8080:https=591 \
--v4proxy proxy:matrix.ungleich.ch:http=8080:https=591
# Note however that the PROXY protocol is not compatible with regular
# HTTP(S) protocols, so your nginx will have to listen on different ports
# with the PROXY settings.
# Note that you will need to restrict access to the 8080 port to prevent
# Client IP spoofing.
# This can be something like:
# server {
# # listen for regular HTTP connections
# listen [::]:80 default_server;
# listen 80 default_server;
# # listen for PROXY HTTP connections
# listen [::]:8080 proxy_protocol;
# # Accept the Client's IP from the PROXY protocol
# real_ip_header proxy_protocol;
# }
SEE ALSO
--------
- https://www.haproxy.com/blog/enhanced-ssl-load-balancing-with-server-name-indication-sni-tls-extension/
- https://www.haproxy.com/blog/haproxy/proxy-protocol/
- https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/
AUTHORS
-------
ungleich <foss--@--ungleich.ch>
Evilham <cvs--@--evilham.com>
COPYING
-------
Copyright \(C) 2021 ungleich glarus ag. You can redistribute it
and/or modify it under the terms of the GNU General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

View file

@ -0,0 +1,155 @@
#!/bin/sh -eu
__package haproxy
require="__package/haproxy" __start_on_boot haproxy
tmpdir="$__object/files"
mkdir "$tmpdir"
configtmp="$__object/files/haproxy.cfg"
os=$(cat "$__global/explorer/os")
case $os in
freebsd)
CONFIG_FILE="/usr/local/etc/haproxy.conf"
cat <<EOF > "$configtmp"
global
maxconn 4000
user nobody
group nogroup
daemon
EOF
;;
*)
CONFIG_FILE="/etc/haproxy/haproxy.cfg"
cat <<EOF > "$configtmp"
global
log [::1] local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
EOF
;;
esac
cat <<EOF >> "$configtmp"
defaults
retries 3
log global
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
EOF
dig_cmd="$(command -v dig || true)"
get_ip() {
# Usage: get_ip (ipv4|ipv6) NAME
# uses "dig" if available, else fallback to "host"
case $1 in
ipv4)
if [ -n "${dig_cmd}" ]; then
${dig_cmd} +short A "$2"
else
host -t A "$2" | cut -d ' ' -f 4 | grep -v 'found:'
fi
;;
ipv6)
if [ -n "${dig_cmd}" ]; then
${dig_cmd} +short AAAA "$2"
else
host -t AAAA "$2" | cut -d ' ' -f 5 | grep -v 'NXDOMAIN'
fi
;;
esac
}
PROTOCOLS="$(cat "$__object/parameter/protocol")"
for proxy in v4proxy v6proxy; do
param=$__object/parameter/$proxy
# no backend? skip generating code
if [ ! -f "$param" ]; then
continue
fi
# turn backend name into bind parameter: v4backend -> ipv4@
bind=$(echo $proxy | sed -e 's/^/ip/' -e 's/proxy//')
case $bind in
ipv4)
backendproto=ipv6
;;
ipv6)
backendproto=ipv4
;;
esac
for proto in ${PROTOCOLS}; do
# Add protocol "header"
printf "\n# %s %s \n" "${bind}" "${proto}" >> "$configtmp"
sed -e "s/BIND/$bind/" \
-e "s/\(frontend[[:space:]].*\)/\1$bind/" \
-e "s/\(backend[[:space:]].*\)/\\1$bind/" \
"$__type/files/$proto" >> "$configtmp"
while read -r hostdefinition; do
if echo "$hostdefinition" | grep -qE '^proxy:'; then
# Proxy protocol was requested
host="$(echo "$hostdefinition" | sed -E 's/^proxy:([^:]+).*$/\1/')"
send_proxy=" send-proxy"
else
# Just use tcp proxy mode
host="$hostdefinition"
send_proxy=""
fi
if echo "$hostdefinition" | grep -qE ":${proto}="; then
# Use custom port definition if requested
port="$(echo "$hostdefinition" | sed -E "s/^(.*:)?${proto}=([0-9]+).*$/:\2/")"
else
# Else use the default
port=""
fi
servername=$host
res=$(get_ip "$bind" "$servername")
if [ -z "$res" ]; then
echo "$servername does not resolve - aborting config" >&2
exit 1
fi
# Treat protocols without TLS+SNI specially
if [ "$proto" = http ]; then
echo " use-server $servername if { hdr(host) -i $host }" >> "$configtmp"
else
echo " use-server $servername if { req_ssl_sni -i $host }" >> "$configtmp"
fi
# Create the "server" itself.
# Note that port and send_proxy will be empty unless
# they were requested by the type user
echo " server $servername ${backendproto}@${host}${port}${send_proxy}" >> "$configtmp"
done < "$param"
done
done
# Create config file
require="__package/haproxy" __file ${CONFIG_FILE} --source "$configtmp" --mode 0644
require="__file${CONFIG_FILE}" __check_messages "haproxy_reload" \
--pattern "^__file${CONFIG_FILE}" \
--execute "service haproxy reload || service haproxy restart"

View file

@ -0,0 +1 @@
http https imaps smtps

View file

@ -0,0 +1,3 @@
protocol
v4proxy
v6proxy

View file

@ -85,7 +85,7 @@ if [ -z "${certbot_fullpath}" ]; then
esac esac
;; ;;
freebsd) freebsd)
__package py37-certbot __package py39-certbot
certbot_fullpath="/usr/local/bin/certbot" certbot_fullpath="/usr/local/bin/certbot"
;; ;;
ubuntu) ubuntu)

View file

@ -28,6 +28,10 @@ apt_clean="$__object/parameter/apt-clean"
apt_dist_upgrade="$__object/parameter/apt-dist-upgrade" apt_dist_upgrade="$__object/parameter/apt-dist-upgrade"
if [ -f "$__object/parameter/apt-with-new-pkgs" ]; then
apt_with_new_pkgs="--with-new-pkgs"
fi
if [ -f "$type" ]; then if [ -f "$type" ]; then
type="$(cat "$type")" type="$(cat "$type")"
else else
@ -54,7 +58,7 @@ case "$type" in
apt) apt)
if [ -f "$apt_dist_upgrade" ] if [ -f "$apt_dist_upgrade" ]
then echo "$aptget dist-upgrade" then echo "$aptget dist-upgrade"
else echo "$aptget upgrade" else echo "$aptget $apt_with_new_pkgs upgrade"
fi fi
if [ -f "$apt_clean" ] if [ -f "$apt_clean" ]

View file

@ -33,6 +33,14 @@ BOOLEAN PARAMETERS
apt-dist-upgrade apt-dist-upgrade
Do dist-upgrade instead of upgrade. Do dist-upgrade instead of upgrade.
apt-with-new-pkg
Allow installing new packages when used in conjunction with
upgrade. This is useful if the update of an installed package
requires new dependencies to be installed. Instead of holding the
package back upgrade will upgrade the package and install the new
dependencies. Note that upgrade with this option will never remove
packages, only allow adding new ones.
apt-clean apt-clean
Clean out the local repository of retrieved package files. Clean out the local repository of retrieved package files.

View file

@ -1,2 +1,3 @@
apt-clean apt-clean
apt-dist-upgrade apt-dist-upgrade
apt-with-new-pkgs

View file

@ -40,6 +40,7 @@ if [ -f "$file" ]; then
grep -v -F -x '$line' '$file' >\$tmpfile grep -v -F -x '$line' '$file' >\$tmpfile
fi fi
cat "\$tmpfile" >"$file" cat "\$tmpfile" >"$file"
rm -f "\$tmpfile"
DONE DONE
} }

View file

@ -84,7 +84,7 @@ def _process_hosts_simple(action, host, manifest, verbose,
""" """
if isinstance(host, str): if isinstance(host, str):
hosts = [host, ] hosts = [host, ]
elif isinstance(host, collections.Iterable): elif isinstance(host, collections.abc.Iterable):
hosts = host hosts = host
else: else:
raise cdist.Error('Invalid host argument: {}'.format(host)) raise cdist.Error('Invalid host argument: {}'.format(host))

View file

@ -33,7 +33,7 @@ class AbsolutePathRequiredError(cdist.Error):
return 'Absolute path required, got: {}'.format(self.path) return 'Absolute path required, got: {}'.format(self.path)
class FileList(collections.MutableSequence): class FileList(collections.abc.MutableSequence):
"""A list that stores it's state in a file. """A list that stores it's state in a file.
""" """
@ -102,7 +102,7 @@ class FileList(collections.MutableSequence):
self.__write(lines) self.__write(lines)
class DirectoryDict(collections.MutableMapping): class DirectoryDict(collections.abc.MutableMapping):
"""A dict that stores it's items as files in a directory. """A dict that stores it's items as files in a directory.
""" """

View file

@ -1,15 +1,32 @@
Changelog Changelog
--------- ---------
next: 7.0.1:
* Core: Remove double definition of scan parser (Nico Schottelius)
* Type __apt_mark: Narrow down grep for hold packages (marcoduif)
* Type __apt_source: Set required options variable (Mark Verboom)
* Type __letsencrypt_cert: Update python version (Michelle)
* Explorer os_version: Add support for Daedalus (Michelle)
* Explorer machine_type: Correct incorrect VMM matching (Mark Verboom)
7.0.0: 2022-07-31
* Explorer machine_type: Rewrite (Dennis Camera) * Explorer machine_type: Rewrite (Dennis Camera)
* New type: __sed (Ander Punnar) * New type: __sed (Ander Punnar)
* New type: __haproxy_dualstack (Evilham and ungleich)
* Type __apt_update_index: Fix complaint about suite change (Matthias Stecher) * Type __apt_update_index: Fix complaint about suite change (Matthias Stecher)
* Type __package_update_index: Fix complaint about suite change (Matthias Stecher) * Type __package_update_index: Fix complaint about suite change (Matthias Stecher)
* Type __package_upgrade_all: Add new --apt-with-new-pkgs argument (Evilham)
* Type __apt_source: Fix complaint about suite change (Matthias Stecher) * Type __apt_source: Fix complaint about suite change (Matthias Stecher)
* Type __package_apt: Fix complaint about suite change (Matthias Stecher) * Type __package_apt: Fix complaint about suite change (Matthias Stecher)
* Type __debconf_set_selections: Fix bug where --file was unsupported (Evilham) * Type __debconf_set_selections: Fix bug where --file was unsupported (Evilham)
* Types __letsencrypt_cert, __grafana_dashboard: Improve bullseye support (Evilham) * Types __letsencrypt_cert, __grafana_dashboard: Improve bullseye support (Evilham)
* Type __ssh_authorized_key: Also remove tmpfile if removing line (Mark Verboom)
* Type __apt_pin: Add default priority, add comment in generated files (Daniel Fancsali)
* Type __file: make file uploading and attribute changes more atomic (Steven Armstrong)
* Type __dot_file: Add support for using --file parameter (Stephan Leemburg)
* Type __apt_ppa: Replace custom "remove-apt-repository" with add-apt-repository -r (Romain Dartigues)
* Type __apt_source: Add signed-by parameter (Daniel Fancsali)
* Explorer: add support for checkpoint (Stephan Leemburg)
6.9.8: 2021-08-24 6.9.8: 2021-08-24
* Type __rsync: Rewrite (Ander Punnar) * Type __rsync: Rewrite (Ander Punnar)

View file

@ -0,0 +1,90 @@
* Install requirements (Alpine)
- apk add py3-pycodestyle shellcheck py3-sphinx py3-sphinx_rtd_theme \
py3-build twine
* Ensure your gpg setup works with the email used in the git commit!
- For me this is nico@nico-notebook.schottelius.org
- Signature / id is on nb2
* Create ~/.pypirc
[distutils]
index-servers =
pypi
cdist
[pypi]
username = __token__
password = ...
[cdist]
repository = https://upload.pypi.org/legacy/
username = __token__
password = ...
* Add date in docs/changelog
* Run ./bin/cdist-build-helper
* TODO Move to "build"
- python3 -m build
* DONE git tag: when?
CLOSED: [2022-07-31 Sun 23:58]
** Asked during release process: ok
* DONE Pypi error with distutils: do not use distutils anymore
CLOSED: [2022-07-31 Sun 23:58]
python3 setup.py sdist upload
...
Creating tar archive
removing 'cdist-7.0.0' (and everything under it)
running upload
Submitting dist/cdist-7.0.0.tar.gz to https://upload.pypi.org/legacy/
Upload failed (400): Invalid value for blake2_256_digest. Error: Use a valid, hex-encoded, BLAKE2 message digest.
error: Upload failed (400): Invalid value for blake2_256_digest. Error: Use a valid, hex-encoded, BLAKE2 message digest.
(venv2) [22:50] nb2:cdist%
* DONE Pypi error with twine: fixed in twine 4.0.1
CLOSED: [2022-07-31 Sun 23:58]
Seeing:
(venv2) [22:47] nb2:cdist% twine upload dist/cdist-7.0.0*
Uploading distributions to https://upload.pypi.org/legacy/
Traceback (most recent call last):
File "/usr/bin/twine", line 8, in <module>
sys.exit(main())
File "/usr/lib/python3.10/site-packages/twine/__main__.py", line 28, in main
result = cli.dispatch(sys.argv[1:])
File "/usr/lib/python3.10/site-packages/twine/cli.py", line 68, in dispatch
return main(args.args)
File "/usr/lib/python3.10/site-packages/twine/commands/upload.py", line 197, in main
return upload(upload_settings, parsed_args.dists)
File "/usr/lib/python3.10/site-packages/twine/commands/upload.py", line 141, in upload
resp = repository.upload(package)
File "/usr/lib/python3.10/site-packages/twine/repository.py", line 189, in upload
resp = self._upload(package)
File "/usr/lib/python3.10/site-packages/twine/repository.py", line 144, in _upload
data = package.metadata_dictionary()
File "/usr/lib/python3.10/site-packages/twine/package.py", line 181, in metadata_dictionary
"dynamic": meta.dynamic,
AttributeError: 'Wheel' object has no attribute 'dynamic'
Fix:
(venv2) [23:43] nb2:cdist% pipx run twine upload dist/*
⚠️ twine is already on your PATH and installed at /home/nico/venv2/bin/twine. Downloading and running anyway.
Uploading distributions to https://upload.pypi.org/legacy/
Uploading cdist-7.0.0-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 868.6/868.6 kB • 00:04 • 221.3 kB/s
Uploading cdist-7.0.0.tar.gz
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB • 00:08 • 169.3 kB/s
View at:
https://pypi.org/project/cdist/7.0.0/
* TODO cdist web
- on staticweb-2022
- Should be moved to sftp/k8s
Manual steps:
~/bin/permissions.public html/
rsync -a html/ staticweb.ungleich.ch:/home/services/www/nico/www.cdi.st/www/manual/7.0.0/
ssh staticweb.ungleich.ch "cd /home/services/www/nico/www.cdi.st/www/manual; ln -sf 7.0.0 latest"