hello folks, I am new to python packaging, hoping to get some ideas on resolving the build error I am seeing. I have a Dockerfile, where its running the following command and its failing with an exception.
python3 setup.py install
Exception says
/usr/local/lib/python3.8/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
Error parsing
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/pbr/core.py", line 96, in pbr
attrs = util.cfg_to_args(path, dist.script_args)
File "/usr/local/lib/python3.8/dist-packages/pbr/util.py", line 271, in cfg_to_args
pbr.hooks.setup_hook(config)
File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/__init__.py", line 25, in setup_hook
metadata_config.run()
File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/base.py", line 27, in run
self.hook()
File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/metadata.py", line 25, in hook
self.config['version'] = packaging.get_version(
File "/usr/local/lib/python3.8/dist-packages/pbr/packaging.py", line 870, in get_version
raise Exception("Versioning for this project requires either an sdist"
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name sb-aq-common was given, but was not able to be found.
error in setup command: Error parsing /home/aqua/sb-aq-common/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name sb-aq-common was given, but was not able to be found.
I have tried this link
and https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html#summary
and replaced it with pip install and pip install -e . but no luck.
I would appreciate if someone can guide me on how I can change this project sb-aq-common which contains setup.cfg and setup.py , it apprears that this way of creating this project is deprecated . may be I need to change this to contain pyproject.toml but I am missing the clarity on what/how to change this. appreciate your guidance
You should just be able to use pip install . instead of invoking
setup.py. I doubt anything about PBR is complicating this case, itās
simply a SetupTools plugin, so solve this the way you would any
SetupTools using project.
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Itās also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name sb-aq-common was given, but was not able to be found.
Apparently (Iāve never used pbr), pbr is trying to automatically determine the project version by querying Git tags, but whatever you copied into the Docker image doesnāt include the Git history. If copying the entire Git repo into the image isnāt an option, the only solution would be to build an sdist or wheel on your host machine outside Docker, copy that into your image, and install it there. This has nothing to do with invoking python setup.py, but thatās still discouraged anyway.
the only solution would be to build an sdist or wheel on your host machine outside Docker, copy that into your image, and install it there
which is a better practice anyway
(though to be fair, Iāve got.a LOT if python pacakges bulind inside docker images ā¦)
But, IIUC, when you pip install from source, what (modern) pip does is first build an sdist, then build a wheel, and then instlal the wheel ā so that would probably work in this case.
All this is why it is no longer recommended to run setup,py install directly.
When I run pip install . I see the following error. basically i am not sure how to modify the setup.py or setup.cfg files to get a successful build. I tried modifying as per the message, by adding find_packages() but didnāt work, most likely there is more to this which I didnāt do
root@8a55f6c97c0a:/home/aqua/myownprojectdir# pip install .
Processing /home/aqua/myownprojectdir
Preparing metadata (setup.py) ... done
Requirement already satisfied: cliff==4.1.0 in /usr/local/lib/python3.8/dist-packages (from myownprojectdir==1.0.0.dev63) (4.1.0)
...
...
...
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema<5,>=4->warlock>=1.2.0->python-glanceclient>=2.8.0->python-openstackclient==3.18.1->myownprojectdir==1.0.0.dev63) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema<5,>=4->warlock>=1.2.0->python-glanceclient>=2.8.0->python-openstackclient==3.18.1->myownprojectdir==1.0.0.dev63) (0.16.0)
Building wheels for collected packages: myownprojectdir
Building wheel for myownprojectdir (setup.py) ... error
error: subprocess-exited-with-error
Ć python setup.py bdist_wheel did not run successfully.
ā exit code: 1
ā°ā> [180 lines of output]
/usr/local/lib/python3.8/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
[pbr] Generating ChangeLog
/usr/local/lib/python3.8/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` and ``easy_install``.
Instead, use pypa/build, pypa/installer, pypa/build or
other standards-based tools.
See https://github.com/pypa/setuptools/issues/917 for details.
********************************************************************************
!!
easy_install.initialize_options(self)
/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer, pypa/build or
other standards-based tools.
See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************
!!
self.initialize_options()
running bdist_wheel
running build
running build_py
running egg_info
writing pbr to myownprojectdir.egg-info/pbr.json
writing myownprojectdir.egg-info/PKG-INFO
writing dependency_links to myownprojectdir.egg-info/dependency_links.txt
writing entry points to myownprojectdir.egg-info/entry_points.txt
writing requirements to myownprojectdir.egg-info/requires.txt
writing top-level names to myownprojectdir.egg-info/top_level.txt
[pbr] Processing SOURCES.txt
[pbr] In git context, generating filelist from git
warning: no previously-included files found matching '.gitreview'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
adding license file 'AUTHORS'
writing manifest file 'myownprojectdir.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/build_py.py:201: _Warning: Package 'myownprojectdir.etc' is absent from the `packages` configuration.
!!
********************************************************************************
############################
# Package would be ignored #
############################
Python recognizes 'myownprojectdir.etc' as an importable package[^1],
but it is absent from setuptools' `packages` configuration.
This leads to an ambiguous overall configuration. If you want to distribute this
package, please make sure that 'myownprojectdir.etc' is explicitly added
to the `packages` configuration field.
Alternatively, you can also rely on setuptools' discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
If you don't want 'myownprojectdir.etc' to be distributed and are
already explicitly excluding 'myownprojectdir.etc' via
`find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
you can try to use `exclude_package_data`, or `include-package-data=False` in
combination with a more fine grained `package-data` configuration.
You can read more about "package data files" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/datafiles.html
[^1]: For Python, any directory (with suitable naming) can be imported,
even if it does not contain any `.py` files.
On the other hand, currently there is no concept of package data
directory, all directories are treated like packages.
********************************************************************************
!!
check.warn(importable)
/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer, pypa/build or
other standards-based tools.
See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************
!!
self.initialize_options()
installing to build/bdist.linux-x86_64/wheel
running install
[pbr] Writing ChangeLog
[pbr] Generating ChangeLog
[pbr] ChangeLog complete (0.0s)
[pbr] Generating AUTHORS
[pbr] AUTHORS complete (0.0s)
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/myownprojectdir
copying build/lib/myownprojectdir/main.py -> build/bdist.linux-x86_64/wheel/myownprojectdir
creating build/bdist.linux-x86_64/wheel/myownprojectdir/tests
copying build/lib/myownprojectdir/tests/test_aqua_os_cliff_command.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests
creating build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
creating build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_2_copy_api_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_2_copy_resource_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_1_copy_resource_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_2_creds_2_nonadmin_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_1_copy_api_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_2_copy_gui_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_1_cred_2_nonadmin_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_1_copy_gui_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/data/accounts_file_3_creds_1_nonadmin_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
copying build/lib/myownprojectdir/tests/account/test_account_associate_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
copying build/lib/myownprojectdir/tests/account/test_account_build_file2.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
copying build/lib/myownprojectdir/tests/account/test_account_clean_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
copying build/lib/myownprojectdir/tests/account/test_account_associate.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
copying build/lib/myownprojectdir/tests/account/test_account_build_file_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
copying build/lib/myownprojectdir/tests/account/test_account_build_file2_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
copying build/lib/myownprojectdir/tests/account/test_account_build_file.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
copying build/lib/myownprojectdir/tests/account/test_account_clean.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
copying build/lib/myownprojectdir/tests/account/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
creating build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
copying build/lib/myownprojectdir/tests/common/test_common.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
copying build/lib/myownprojectdir/tests/common/test_tabular_data.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
copying build/lib/myownprojectdir/tests/common/test_accounts_data.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
copying build/lib/myownprojectdir/tests/common/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
copying build/lib/myownprojectdir/tests/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests
creating build/bdist.linux-x86_64/wheel/myownprojectdir/etc
copying build/lib/myownprojectdir/etc/aqua-os.cfg -> build/bdist.linux-x86_64/wheel/myownprojectdir/etc
creating build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/account_build_file.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/account_clean.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/account_associate.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/account_build_file2.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/account_build_file2_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/account_build_file_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/account_associate_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/account_clean_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
copying build/lib/myownprojectdir/account/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
creating build/bdist.linux-x86_64/wheel/myownprojectdir/common
copying build/lib/myownprojectdir/common/templates.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
copying build/lib/myownprojectdir/common/exceptions.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
copying build/lib/myownprojectdir/common/openstacksdk_client.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
copying build/lib/myownprojectdir/common/common.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
copying build/lib/myownprojectdir/common/accounts_data.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
copying build/lib/myownprojectdir/common/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
copying build/lib/myownprojectdir/common/tabular_data.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
copying build/lib/myownprojectdir/aqua_os_cliff_command.py -> build/bdist.linux-x86_64/wheel/myownprojectdir
copying build/lib/myownprojectdir/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir
running install_data
creating build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data
creating build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data/data
creating build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data/data/myownprojectdir
creating build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data/data/myownprojectdir/etc
copying myownprojectdir/etc/aqua-os.cfg -> build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data/data/myownprojectdir/etc
running install_egg_info
Copying myownprojectdir.egg-info to build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63-py3.8.egg-info
running install_scripts
error: invalid command 'bdist_wininst'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for myownprojectdir
Running setup.py clean for myownprojectdir
Failed to build myownprojectdir
ERROR: Could not build wheels for myownprojectdir, which is required to install pyproject.toml-based projects
I notice youāre installing relatively old versions of packages (for
example, python-openstackclient 3.18.1 is from 2019) with what
appears to be fairly new SetupTools. This is likely to lead to
problems. What version of PBR is being pulled in during the build?
Newer versions of PBR attempt to maintain backward compatibility
with older versions of SetupTools, but older versions of PBR canāt
predict what changes are needed for newer SetupTools versions due to
lack of available time machines (Iām hoping we find a solution to
the time machine shortage soon).
I changed this line in the setup.py, to be line ā setup_requires=[āpbrā],
it was setup_requires=[āpbr>=2.0.0ā], I can update to whatever works. I am assuming if this would pull the latest pbr although I didnāt verify
Using setup_requires is deprecated, I believe, as it installs build-time dependencies using easy_install, which is no longer considered good practice. That might explain the āRequirements should be satisfied by a PEP 517 installerā message in the output.
Just to circle back around to the beginning on this, PBR needs one
of three things to determine the version of the package youāre
installing:
A Git repository so it can inspect tag info to guess a version
Python package metadata (i.e. an already built sdist)
An environment variable setting an explicit version string
Any of those three options should suffice. For option #1 you need
the actual repository, not just a copy of the worktree (so a GitHub
style ārelease tarballā is insufficient). For option #2 you can use build Ā· PyPI to create the sdist, but if youāre
doing that you may as well just use it to make a wheel and then PBR
isnāt even needed during installation. For option #3 just export
PBR_VERSION with the version you want used, but note that if PBR is
being used multiple times in that process for building different
packages then youāll end up overriding all their versions, so this
is the least preferable solution.
As for doing modern PEP 517 style packaging with a pyproject.toml
file, PBR supports that. It still needs a stub of a setup.py file
present in the project to turn on the plugin:
With very recent versions of SetupTools you can almost entirely get
rid of setup.cfg and move the rest of the metadata into
pyproject.toml instead, but at the moment PBR still expects to find
metadata.name set in setup.cfg (Iām hoping to get that fixed soon).
One other thing worth checking I guess, make sure you have a new
enough pip. Old versions of pip lacking ABI3 support may be
retrieving sdists from PyPI because it doesnāt notice there are
suitable wheels for your interpreter version.
As for the second error, PBR hasnāt used bdist_wininst for several
years since calls for that were removed in 5.5.0, which is what
leads me to suspect old versions of PBR and other things.