From dd167f075d25ecec4f1aca461658d2b93ff1ae8f Mon Sep 17 00:00:00 2001 From: Matthias Stecher Date: Wed, 9 Sep 2020 19:08:46 +0200 Subject: [PATCH] __netbox*: fix service restart order Cause of corrupt databases if the services are restarted incorrectly, the order and dependencies are adjusted. Now, the `netbox-rq` service will be included in restarts of `netbox` and required for the WSGI servers that it must running. For these changes, the restart command of `__netbox` was adjusted. The other ones where edited too, to use the same command. All services now require redis and postgresql to be started before them to prevent any start order issues. If someone asked for what the RQ worker is required, see here: https://netbox.readthedocs.io/en/stable/additional-features/webhooks/#webhook-processing --- type/__netbox/files/netbox-rq.service | 7 ++++-- type/__netbox/files/netbox.service | 2 +- type/__netbox/gencode-remote | 7 +++--- type/__netbox/man.rst | 24 ++++++++++++++++++++- type/__netbox_gunicorn/files/netbox.service | 4 +++- type/__netbox_gunicorn/gencode-remote | 2 +- type/__netbox_uwsgi/files/netbox.service | 4 +++- type/__netbox_uwsgi/gencode-remote | 2 +- 8 files changed, 41 insertions(+), 11 deletions(-) diff --git a/type/__netbox/files/netbox-rq.service b/type/__netbox/files/netbox-rq.service index 77d7091..330e675 100644 --- a/type/__netbox/files/netbox-rq.service +++ b/type/__netbox/files/netbox-rq.service @@ -1,8 +1,11 @@ [Unit] Description=NetBox Request Queue Worker Documentation=https://netbox.readthedocs.io/en/stable/ -After=network-online.target -Wants=network-online.target +PartOf=netbox.service +Wants=network.target +After=netbox.service +After=network.target +After=redis-server.service postgresql.service [Service] Type=simple diff --git a/type/__netbox/files/netbox.service b/type/__netbox/files/netbox.service index 738e178..68010e9 100644 --- a/type/__netbox/files/netbox.service +++ b/type/__netbox/files/netbox.service @@ -1,8 +1,8 @@ [Unit] Description=NetBox Service Wrapper Documentation=https://netbox.readthedocs.io/en/stable/ -After=network.target Wants=network.target +After=network.target [Service] Type=oneshot diff --git a/type/__netbox/gencode-remote b/type/__netbox/gencode-remote index 9ae6b5b..0370949 100755 --- a/type/__netbox/gencode-remote +++ b/type/__netbox/gencode-remote @@ -105,9 +105,10 @@ fi # check for changes if [ "$changes" = "yes" ]; then + # to avoid database corruption at config changes, both services must be + # stopped at the same time (noted in the manual, too). cat << EOF -# Restart service. -service netbox restart -service netbox-rq restart +# Restart service. All required services are included with netbox.service. +systemctl restart netbox EOF fi diff --git a/type/__netbox/man.rst b/type/__netbox/man.rst index bc4bbc9..3330b1b 100644 --- a/type/__netbox/man.rst +++ b/type/__netbox/man.rst @@ -12,7 +12,11 @@ This (singleton) type installs and configures a NetBox instance, a web application to help manage and document computer networks. It installs it with the user ``netbox`` at ``/opt/netbox`` with `python-venv`. -It setup systemd unit files for the services `netbox` and `netbox-rq`. To +It setup systemd unit files for the services `netbox` and `netbox-rq`. The +`netbox` service only wrap all netbox related services, e.g. restarting and +so one will be delegated to all related services. + +The application is still not accessable because WSGI server is required. To access the application through WSGI, uWSGI or Gunicorn can be used. The setup can be done via there own types `__netbox_gunicorn` and `__netbox_uwsgi`. @@ -229,6 +233,24 @@ If you not setup ldap authentification, you may be interested into how to ` directly on the machine to be able to access and use NetBox. +If you change a configuration, the database may go corrupt if two instances of +the application are running with different configurations at the same time. +This most commonly happens when the WSGI server and RQ-worker restarts after a +configuration change. This occours in the following case for example: + +.. code-block:: sh + + systemctl restart gunicorn-netbox # WSGI-server already online with new + # configuration after this command. + systemctl restart netbox-rq # RQ-Worker still worked with the old + # configuration till here. + +This type handles the restart of both services correctly to avoid such database +corruptions. To safely manual restart the whole netbox instance manual, simply +restart all services in one ``systemctl restart netbox`` command, as it ensures +that first all services are shut down before starting one of them. The service +``netbox`` wraps all required services that are available. + SEE ALSO -------- diff --git a/type/__netbox_gunicorn/files/netbox.service b/type/__netbox_gunicorn/files/netbox.service index 55f3549..dcf1a66 100644 --- a/type/__netbox_gunicorn/files/netbox.service +++ b/type/__netbox_gunicorn/files/netbox.service @@ -2,9 +2,11 @@ Description=NetBox Gunicorn WSGI Service Documentation=https://netbox.readthedocs.io/en/stable/ PartOf=netbox.service +Requires=netbox-rq.service +Wants=network.target After=netbox.service After=network.target -Wants=network.target +After=redis-server.service postgresql.service [Service] Type=simple diff --git a/type/__netbox_gunicorn/gencode-remote b/type/__netbox_gunicorn/gencode-remote index 0da92f0..4fae788 100755 --- a/type/__netbox_gunicorn/gencode-remote +++ b/type/__netbox_gunicorn/gencode-remote @@ -34,7 +34,7 @@ case "$state" in if [ "$do_restart" ] && [ "$state" != "disabled" ]; then cat << EOF # Restart service -service gunicorn-netbox restart +systemctl restart gunicorn-netbox EOF fi ;; diff --git a/type/__netbox_uwsgi/files/netbox.service b/type/__netbox_uwsgi/files/netbox.service index 8d41cd0..cd06e82 100644 --- a/type/__netbox_uwsgi/files/netbox.service +++ b/type/__netbox_uwsgi/files/netbox.service @@ -2,9 +2,11 @@ Description=Netbox uWSGI WSGI Service Documentation=https://netbox.readthedocs.io/en/stable/ PartOf=netbox.service +Requires=netbox-rq.service +Wants=network.target After=netbox.service After=network.target -Wants=network.target +After=redis-server.service postgresql.service [Service] Type=simple diff --git a/type/__netbox_uwsgi/gencode-remote b/type/__netbox_uwsgi/gencode-remote index 9b6a603..c219643 100755 --- a/type/__netbox_uwsgi/gencode-remote +++ b/type/__netbox_uwsgi/gencode-remote @@ -30,7 +30,7 @@ case "$state" in if [ "$do_restart" ] && [ "$state" != "disabled" ]; then cat << EOF # Restart service -service uwsgi-netbox restart +systemctl restart uwsgi-netbox EOF fi ;;