Commit graph

7051 commits

Author SHA1 Message Date
81b426e4e2 [__letsencrypt_cert] Revamp explorers, add locking.
Closes #839

See merge request ungleich-public/cdist!976

This patch joins all explorers in one to avoid starting multiple remote python
processes and uses a cdist-specific lock in /tmp/certbot.cdist.lock with a
60 seconds timeout.
2021-05-10 12:10:01 +02:00
a696f3cf00 [__letsencrypt_cert] Revamp explorers, add locking.
This would fix #839

Certbot uses locking [1] even for read-only operations and does not properly
use exit codes, which means that sometimes it would print:
"Another instance of Certbot is already running" and exit with success.

However, the previous explorers would take that as the certificate being absent
and would trigger code generation.

The issue was made worse by having many explorers running certbot, so for N
certificates, we'd run certbot N*4 times, potentially "in parallel".

[1]: https://certbot.eff.org/docs/using.html#id5

This patch joins all explorers in one to avoid starting multiple remote python
processes and uses a cdist-specific lock in /tmp/certbot.cdist.lock with a
60 seconds timeout.

It has been tested with certbot 0.31.0 and 0.17 that the:

    from certbot.main import main

trick works. It is somewhat well documented so it can be somewhat relied upon.
2021-05-10 12:10:00 +02:00
0b05a8f5f7 [__apt_key*] Deprecate __apt_key_uri and improve __apt_key
See: https://code.ungleich.ch/ungleich-public/cdist/-/merge_requests/994

Previously this type was falling back to using the deprecated apt-key(8) by
checking for existence of files/directories on the controller host in
gencode-remote.

Adding `--use-deprecated-apt-key` as an explicit boolean serves two purposes:
1. It prevents fallbacks that might end up doing the wrong thing
   (as was the case)
2. It allows for a simple way to remove keys from the keyring that were
   previously added with apt-key(8) to /etc/apt/trusted.gpg

This parameter is added marked as deprecated as is only intended use is to
migrate to directory-based keyrings as recommended by Debian for a few releases.
It will be removed when Debian 11 stops being supported.

During the review process of this merge request, it was noted that the state of
PGP Key Servers is somewhat suboptimal, that the examples encouraged bad
practise (it is trivial to produce collisions for short key IDs), and that 
this use does not require the Web of Trust, but instead only the public key
that is signing the repository.

That is why this also adds `--source` as an argument allowing for in-type or
in-manifest provision of such public keys by the type/manifest maintainer and
the use of Key Servers is still supported, but discouraged.
2021-05-10 12:08:23 +02:00
c00c8c2012 [__apt_key*] Deprecate __apt_key_uri and improve __apt_key
Previously this type was falling back to using the deprecated apt-key(8) by
checking for existence of files/directories on the controller host in
gencode-remote.

Adding `--use-deprecated-apt-key` as an explicit boolean serves two purposes:
1. It prevents fallbacks that might end up doing the wrong thing
   (as was the case)
2. It allows for a simple way to remove keys from the keyring that were
   previously added with apt-key(8) to /etc/apt/trusted.gpg

This parameter is added marked as deprecated as is only intended use is to
migrate to directory-based keyrings as recommended by Debian for a few releases.
It will be removed when Debian 11 stops being supported.

During the review process of this merge request, it was noted that the state of
PGP Key Servers is somewhat suboptimal, that the examples encouraged bad
practise (it is trivial to produce collisions for short key IDs), and that
this use does not require the Web of Trust, but instead only the public key
that is signing the repository.

That is why this also adds `--source` as an argument allowing for in-type or
in-manifest provision of such public keys by the type/manifest maintainer and
the use of Key Servers is still supported, but discouraged.
2021-05-10 12:08:22 +02:00
Darko Poljak
3a25b80466 ++changelog 2021-04-26 21:27:15 +02:00
3e190c3481 Merge branch 'feature/type/__postgres/postgres_user-explorer' into 'master'
__postgres_*: Improve OS support and some cleanup

See merge request ungleich-public/cdist!990
2021-04-26 21:26:38 +02:00
Dennis Camera
0f05f38384 [type/__postgres_role] Treat --password '' like no --password 2021-04-25 20:01:36 +02:00
Dennis Camera
0d33407b18 [type/__postgres_database] Proper quoting in state explorer 2021-04-25 20:01:36 +02:00
Dennis Camera
8296051653 [type/__postgres_extension] Add state explorer 2021-04-25 20:01:36 +02:00
Dennis Camera
3cf93249c3 [type/__postgres_extension] Include postgres_user explorer from __postgres_conf 2021-04-25 20:01:36 +02:00
Dennis Camera
beb8da6d5f [type/__postgres_role] Include postgres_user explorer from __postgres_conf 2021-04-25 20:01:36 +02:00
Dennis Camera
58b279a8d0 [type/__postgres_database] Improve quoting 2021-04-25 20:01:36 +02:00
Dennis Camera
6ac8cbf98f [type/__postgres_database] Include postgres_user explorer from __postgres_conf 2021-04-25 20:01:36 +02:00
Darko Poljak
512e9b23c0 ++changelog 2021-04-25 15:53:40 +02:00
71d79ed6ee Merge branch 'feature/type/__postgres_conf' into 'master'
__postgres_conf: new type

See merge request ungleich-public/cdist!972
2021-04-25 15:49:19 +02:00
Darko Poljak
1bb696a410 Release 6.9.6 2021-04-20 07:33:07 +02:00
2f05467358 Merge branch 'fix/py-version-check' into 'master'
Fix Python version check

See merge request ungleich-public/cdist!991
2021-04-19 07:02:15 +02:00
Dennis Camera
1c047353a9 [bin/cdist] Fix Python version check 2021-04-17 09:57:10 +02:00
Dennis Camera
19bf37be1a [type/__postgres_conf] Update man.rst 2021-04-15 15:56:15 +02:00
Dennis Camera
686e4f0f2d [type/__postgres_conf] Reverse state logic (decide based on source first) 2021-04-15 15:50:03 +02:00
Dennis Camera
bef1433ba3 [type/__postgres_conf] Accept empty values 2021-04-15 15:50:03 +02:00
Dennis Camera
12c2995494 [type/__postgres_conf] Implement complex state compare logic 2021-04-15 15:50:02 +02:00
Dennis Camera
e0416403c4 [type/__postgres_conf] Add psql_conf_source function to state explorer 2021-04-15 15:50:02 +02:00
Dennis Camera
2ccc03fef1 [type/__postgres_conf] Add psql_conf_cmp function to state explorer 2021-04-15 15:50:02 +02:00
Dennis Camera
92b8942a8c [type/__postgres_conf] Add psql_exec function to state explorer 2021-04-15 15:50:02 +02:00
Darko Poljak
9ec01d9f97 ++changelog 2021-04-13 12:22:45 +02:00
e27e88512b Merge branch '__download_optional_sum' into 'master'
[__download] make --sum optional

See merge request ungleich-public/cdist!989
2021-04-13 12:21:49 +02:00
d2eec60668
[__download] make --sum optional 2021-04-11 23:16:00 +03:00
Darko Poljak
750c71fb5a Minor refactoring and remove code duplication 2021-04-07 10:25:26 +02:00
Darko Poljak
199effb7ef Improve unfinished object requirements bool check
When we need only boolean value for unfinished object requirements then
we don't need to determine the whole list of unfinished objects.
2021-04-06 19:35:14 +02:00
Darko Poljak
ab811ad282 ++changelog 2021-04-01 15:37:11 +02:00
ce79a2069c Merge branch 'fix/type/__pyvenv/numeric-owner' into 'master'
__pyvenv: Fix if --owner / --group is numeric

See merge request ungleich-public/cdist!988
2021-04-01 15:36:02 +02:00
c981f654f1 Merge branch 'fix/type/__git/numeric-owner' into 'master'
__git: Fix if --owner / --group is numeric

See merge request ungleich-public/cdist!987
2021-04-01 15:35:46 +02:00
87698395b8 Merge branch 'cleanup/string-formatting' into 'master'
Cleanup/string formatting

Closes #855

See merge request ungleich-public/cdist!985
2021-03-31 08:28:21 +02:00
Darko Poljak
4c2d273f07 Unify string formatting
Use one way of string formatting: replace old `%` style with new `str.format`.

Resolve #855.
2021-03-31 08:19:34 +02:00
Darko Poljak
f984a918b9 Fix log message string formatting
Use logging message format with args, instead of direct `%` or `str.format`.

Resolve #855.
2021-03-31 08:19:28 +02:00
Darko Poljak
1e765fcab7 ++changelog 2021-03-31 07:54:00 +02:00
28c13bd29b Merge branch 'feature/type-relationship-graph' into 'master'
Implement maintaining object relationship graph

See merge request ungleich-public/cdist!986
2021-03-31 07:52:42 +02:00
Dennis Camera
985252585c [type/__pyvenv] Fix if --owner / --group is numeric
Before, if --owner and/or --group was numeric, gencode-remote would generate
`chown` code every time.
2021-03-30 13:26:21 +02:00
Dennis Camera
167c2ad7ea [type/__git] Fix if --owner / --group is numeric
Before, if --owner and/or --group was numeric, gencode-remote would generate
`chown` code every time.
2021-03-30 13:24:56 +02:00
Darko Poljak
7a0b697f4c Implement maintaining object relationship graph
For each object maintain parent-child relationship graph, i.e. list of
parent objects ('parents' property) and list of children objects ('children'
property).

Objects without parent(s) are objects specified in init manifest.
Objects without children are object of types that do not reuse other types.
2021-03-30 12:09:59 +02:00
Darko Poljak
10ca1c12fd ++changelog 2021-03-12 08:21:03 +01:00
c55397766e Merge branch 'feature/type/__sshd_config/whitelist-openbmc' into 'master'
__sshd_config: Whitelist OpenBMC

See merge request ungleich-public/cdist!980
2021-03-12 08:20:35 +01:00
Dennis Camera
e47c4dd8a4 [type/__sshd_config] Whitelist OpenBMC in manifest 2021-03-11 14:17:44 +01:00
Darko Poljak
31cc592aa1 ++changelog 2021-03-10 19:25:04 +01:00
2f4a7e1a94 Merge branch 'fix/type/__ssh_authorized_key/grep-only-if-file-exists' into 'master'
__ssh_authorized_key: only grep if file exists

See merge request ungleich-public/cdist!979
2021-03-10 19:24:23 +01:00
Dennis Camera
fb19f34266 [type/__ssh_authorized_key] Only grep if file exists 2021-03-09 21:15:26 +01:00
Steven Armstrong
ecba284fc8 changelog++
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2021-03-05 16:13:02 +01:00
Steven Armstrong
ea0126dd81 Make local state dir available to custom remote scripts
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
2021-03-05 16:11:49 +01:00
Darko Poljak
e7d33891df ++changelog 2021-03-02 09:29:33 +01:00