nonparallel will not be honored for explorers #18

New Issue

ungleich-gitea · 2021-11-20T11:24:50Z

ungleich-gitea commented

2021-11-20 11:24:50 +00:00

By using the type __letsencypt_cert, I noticed that it generated code all times. I found this somehow suspicious because I have it different in memory. After a bit of research, I found that it generated code cause the explorer certificate-exists returned "no" all the time. While debugging, I found error messages that Certbot can't be executed if an other instance is still running.

After I checked this, I first ran it single-threaded and it doesn't generated code for the type. Then, I wanted to add the nonparallel file but saw it was already added. So executing it a second time with -j 4 led to the same problem again.

Testing:

No problem when running with only one thread
No problem when running with only one object instance of __letsencrypt_cert
Run with multiple instances of __letsencrypt_cert and multiple threads, which now generates code each execution

TL;DR: nonparallel doesn't apply to explorers from different instances of the same type because the explorer says that certificates don't exist cause of an "Another instance of Certbot is already running." error. This indicates that the explorers of different object instances are executed at the same time, which don't work for Certbot.

Btw. this might not be that good that the certificates will be refreshed at every execution. Maybe not in production, but by debugging the problem, I got into the rate-limiting of Let's Encrypt :-)

By using the type `__letsencypt_cert`, I noticed that it generated code all times. I found this somehow suspicious because I have it different in memory. After a bit of research, I found that it generated code cause the explorer `certificate-exists` returned "no" all the time. While debugging, I found error messages that Certbot can't be executed if an other instance is still running. After I checked this, I first ran it single-threaded and it doesn't generated code for the type. Then, I wanted to add the `nonparallel` file but saw it was already added. So executing it a second time with `-j 4` led to the same problem again. **Testing**: 1. No problem when running with only one thread 2. No problem when running with only one object instance of `__letsencrypt_cert` 3. Run with multiple instances of `__letsencrypt_cert` and multiple threads, which now generates code each execution **TL;DR**: `nonparallel` doesn't apply to explorers from different instances of the same type because the explorer says that certificates don't exist cause of an "Another instance of Certbot is already running." error. This indicates that the explorers of different object instances are executed at the same time, which don't work for Certbot. Btw. this might not be that good that the certificates will be refreshed at every execution. Maybe not in production, but by debugging the problem, I got into the rate-limiting of Let's Encrypt :-)

ungleich-gitea commented

2021-11-20 15:25:59 +00:00

mentioned in issue #839

ungleich-gitea commented

2021-11-20 15:26:00 +00:00

nonparallel only applies to code-*. Since explorers are read-only and manifests do not touch the target system directly, I think this makes sense (unless you have to configure broken software).
Ideally, each type should be able to specify which parts need which locking, but this is not currently possible.

I had the same problem in !896.
Since certbot is written in Python, the locking code could maybe be simplified by using the fcntl Python module.

`nonparallel` only applies to `code-*`. Since explorers are read-only and manifests do not touch the target system directly, I think this makes sense (unless you have to configure broken software). Ideally, each type should be able to specify which parts need which locking, but this is not currently possible. I had the same problem in !896. Since certbot is written in Python, the locking code could maybe be simplified by using the `fcntl` Python module.

Sign in to join this conversation.