scan files not included when installing using setup.py #19

New Issue

ungleich-gitea · 2021-11-20T11:24:50Z

ungleich-gitea commented

2021-11-20 11:24:50 +00:00

When I generate a package using a PKGBUILD for version 6.9.2 generated using PKGBUILD.in the files needed for the scan feature are not included and it seems like the argument parser needs them in order to function.

$ cdist                                                                                                                 (12-04 13:32)
Traceback (most recent call last):
  File "/usr/bin/cdist", line 39, in <module>
    import cdist.argparse   # noqa 402
  File "/usr/lib/python3.9/site-packages/cdist/argparse.py", line 11, in <module>
    import cdist.scan.commandline
ModuleNotFoundError: No module named 'cdist.scan'

It looks like the scan files are not included in the archive at https://pypi.python.org/packages/source/c/cdist/cdist-6.9.2.tar.gz either.

$ curl -s -L https://pypi.python.org/packages/source/c/cdist/cdist-6.9.2.tar.gz | bsdtar -tf - | grep scan/ || echo nope
nope

If I instead look at the archive uploaded here the files are present.

$ curl -s -L https://code.ungleich.ch/ungleich-public/cdist/uploads/a8f2f379b08deba0c4abf8038587a217/cdist-6.9.2.tar.gz | bsdtar -tf - | grep scan/ || echo nope
cdist-6.9.2/cdist/scan/
cdist-6.9.2/cdist/scan/commandline.py
cdist-6.9.2/cdist/scan/scan.py

If I replace the URL in the PKGBUILD and rebuild the files are still not there.

$ bsdtar -tf cdist-6.9.2-1-any.pkg.tar.zst | grep /scan || echo nope
nope

I then tried to add the scan directory.

$ sed -n 's/\(data_finder("conf")\)$/\1 + data_finder("scan")/p' setup.py
package_data = data_finder("conf") + data_finder("scan")

If I apply that change in the PKGBUILD the files are now included

$ bsdtar -tf cdist-6.9.2-1-any.pkg.tar.zst | grep /scan || echo nope
usr/lib/python3.9/site-packages/cdist/scan/
usr/lib/python3.9/site-packages/cdist/scan/__pycache__/
usr/lib/python3.9/site-packages/cdist/scan/__pycache__/commandline.cpython-39.pyc
usr/lib/python3.9/site-packages/cdist/scan/__pycache__/scan.cpython-39.pyc
usr/lib/python3.9/site-packages/cdist/scan/commandline.py
usr/lib/python3.9/site-packages/cdist/scan/scan.py

I don't know if this is the right way to do it but it got me up and running.

When I generate a package using a PKGBUILD for version 6.9.2 generated using PKGBUILD\.in the files needed for the scan feature are not included and it seems like the argument parser needs them in order to function. ```console $ cdist (12-04 13:32) Traceback (most recent call last): File "/usr/bin/cdist", line 39, in <module> import cdist.argparse # noqa 402 File "/usr/lib/python3.9/site-packages/cdist/argparse.py", line 11, in <module> import cdist.scan.commandline ModuleNotFoundError: No module named 'cdist.scan' ``` It looks like the scan files are not included in the archive at https://pypi.python.org/packages/source/c/cdist/cdist-6.9.2.tar.gz either. ```console $ curl -s -L https://pypi.python.org/packages/source/c/cdist/cdist-6.9.2.tar.gz | bsdtar -tf - | grep scan/ || echo nope nope ``` If I instead look at the archive uploaded here the files are present. ```console $ curl -s -L https://code.ungleich.ch/ungleich-public/cdist/uploads/a8f2f379b08deba0c4abf8038587a217/cdist-6.9.2.tar.gz | bsdtar -tf - | grep scan/ || echo nope cdist-6.9.2/cdist/scan/ cdist-6.9.2/cdist/scan/commandline.py cdist-6.9.2/cdist/scan/scan.py ``` If I replace the URL in the PKGBUILD and rebuild the files are still not there. ```console $ bsdtar -tf cdist-6.9.2-1-any.pkg.tar.zst | grep /scan || echo nope nope ``` I then tried to add the scan directory. ```console $ sed -n 's/$data_finder("conf")$$/\1 + data_finder("scan")/p' setup.py package_data = data_finder("conf") + data_finder("scan") ``` If I apply that change in the PKGBUILD the files are now included ```console $ bsdtar -tf cdist-6.9.2-1-any.pkg.tar.zst | grep /scan || echo nope usr/lib/python3.9/site-packages/cdist/scan/ usr/lib/python3.9/site-packages/cdist/scan/__pycache__/ usr/lib/python3.9/site-packages/cdist/scan/__pycache__/commandline.cpython-39.pyc usr/lib/python3.9/site-packages/cdist/scan/__pycache__/scan.cpython-39.pyc usr/lib/python3.9/site-packages/cdist/scan/commandline.py usr/lib/python3.9/site-packages/cdist/scan/scan.py ``` I don't know if this is the right way to do it but it got me up and running.

ungleich-gitea closed this issue

2021-11-20 11:24:51 +00:00

ungleich-gitea commented

2021-11-20 15:26:02 +00:00

@oliryde Not an Arch Linux user myself, so I can't really help you there (apart from adding an s to the URL 😉).
The PKGBUILD.in file looks abandoned to me. The last real change was made when freecode went down.

I've been told in chat that cdist is in AUR, so you might want to try that instead of the PKGBUILD.in.

@oliryde Not an Arch Linux user myself, so I can't really help you there (apart from adding an `s` to the URL :wink:). The `PKGBUILD.in` file looks abandoned to me. The last *real* change was made when freecode went down. I've been told in chat that cdist is in AUR, so you might want to try that instead of the `PKGBUILD.in`.

ungleich-gitea commented

2021-11-20 15:26:03 +00:00

@ssrq I can confirm the files are included as expected.

I see the merge request was created before I made this issue.
I did not consider looking at merge requests before creating the issue but I will make sure to do that in the future.

A different issue entirely is that pypi.python.org seemingly only accepts https requests and the PKGBUILD.in script is set to http resulting in the following output.

$ ./PKGBUILD.in 6.9.3
==> Retrieving sources...
  -> Downloading cdist-6.9.3.tar.gz...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 403 SSL is required
==> ERROR: Failure while downloading http://pypi.python.org/packages/source/c/cdist/cdist-6.9.3.tar.gz
    Aborting...

@ssrq I can confirm the files are included as expected. I see the merge request was created before I made this issue. I did not consider looking at merge requests before creating the issue but I will make sure to do that in the future. A different issue entirely is that pypi.python.org seemingly only accepts https requests and the PKGBUILD.in script is set to http resulting in the following output. ```console $ ./PKGBUILD.in 6.9.3 ==> Retrieving sources... -> Downloading cdist-6.9.3.tar.gz... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 403 SSL is required ==> ERROR: Failure while downloading http://pypi.python.org/packages/source/c/cdist/cdist-6.9.3.tar.gz Aborting... ```

ungleich-gitea commented

2021-11-20 15:26:05 +00:00

@oliryde Should be fixed in release 6.9.3.