Replace deprecated "python3 setup.py install" call with new fails

hello folks, I am new to python packaging, hoping to get some ideas on resolving the build error I am seeing. I have a Dockerfile, where its running the following command and its failing with an exception.

python3 setup.py install

Exception says :slight_smile:

/usr/local/lib/python3.8/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  dist.fetch_build_eggs(dist.setup_requires)
Error parsing
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pbr/core.py", line 96, in pbr
    attrs = util.cfg_to_args(path, dist.script_args)
  File "/usr/local/lib/python3.8/dist-packages/pbr/util.py", line 271, in cfg_to_args
    pbr.hooks.setup_hook(config)
  File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/__init__.py", line 25, in setup_hook
    metadata_config.run()
  File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/base.py", line 27, in run
    self.hook()
  File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/metadata.py", line 25, in hook
    self.config['version'] = packaging.get_version(
  File "/usr/local/lib/python3.8/dist-packages/pbr/packaging.py", line 870, in get_version
    raise Exception("Versioning for this project requires either an sdist"
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name sb-aq-common was given, but was not able to be found.
error in setup command: Error parsing /home/aqua/sb-aq-common/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name sb-aq-common was given, but was not able to be found.

I have tried this link

and https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html#summary
and replaced it with pip install and pip install -e . but no luck.
I would appreciate if someone can guide me on how I can change this project sb-aq-common which contains setup.cfg and setup.py , it apprears that this way of creating this project is deprecated . may be I need to change this to contain pyproject.toml but I am missing the clarity on what/how to change this. appreciate your guidance

Seems like the project is using PBR. Maybe @fungi can help?

You should just be able to use pip install . instead of invoking
setup.py. I doubt anything about PBR is complicating this case, itā€™s
simply a SetupTools plugin, so solve this the way you would any
SetupTools using project.

The second-to-last line of the traceback says:

Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Itā€™s also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name sb-aq-common was given, but was not able to be found.

Apparently (Iā€™ve never used pbr), pbr is trying to automatically determine the project version by querying Git tags, but whatever you copied into the Docker image doesnā€™t include the Git history. If copying the entire Git repo into the image isnā€™t an option, the only solution would be to build an sdist or wheel on your host machine outside Docker, copy that into your image, and install it there. This has nothing to do with invoking python setup.py, but thatā€™s still discouraged anyway.

1 Like

ā€¦ And what exactly happened when you tried doing those things?

the only solution would be to build an sdist or wheel on your host machine outside Docker, copy that into your image, and install it there

which is a better practice anyway :slight_smile:
(though to be fair, Iā€™ve got.a LOT if python pacakges bulind inside docker images ā€¦)

But, IIUC, when you pip install from source, what (modern) pip does is first build an sdist, then build a wheel, and then instlal the wheel ā€“ so that would probably work in this case.

All this is why it is no longer recommended to run setup,py install directly.

When I run pip install . I see the following error. basically i am not sure how to modify the setup.py or setup.cfg files to get a successful build. I tried modifying as per the message, by adding find_packages() but didnā€™t work, most likely there is more to this which I didnā€™t do

root@8a55f6c97c0a:/home/aqua/myownprojectdir# pip install .
Processing /home/aqua/myownprojectdir
  Preparing metadata (setup.py) ... done
Requirement already satisfied: cliff==4.1.0 in /usr/local/lib/python3.8/dist-packages (from myownprojectdir==1.0.0.dev63) (4.1.0)
...
...
...
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema<5,>=4->warlock>=1.2.0->python-glanceclient>=2.8.0->python-openstackclient==3.18.1->myownprojectdir==1.0.0.dev63) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema<5,>=4->warlock>=1.2.0->python-glanceclient>=2.8.0->python-openstackclient==3.18.1->myownprojectdir==1.0.0.dev63) (0.16.0)
Building wheels for collected packages: myownprojectdir
  Building wheel for myownprojectdir (setup.py) ... error
  error: subprocess-exited-with-error

  Ɨ python setup.py bdist_wheel did not run successfully.
  ā”‚ exit code: 1
  ā•°ā”€> [180 lines of output]
      /usr/local/lib/python3.8/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
      !!

              ********************************************************************************
              Requirements should be satisfied by a PEP 517 installer.
              If you are using pip, you can try `pip install --use-pep517`.
              ********************************************************************************

      !!
        dist.fetch_build_eggs(dist.setup_requires)
      [pbr] Generating ChangeLog
      /usr/local/lib/python3.8/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
      !!

              ********************************************************************************
              Please avoid running ``setup.py`` and ``easy_install``.
              Instead, use pypa/build, pypa/installer, pypa/build or
              other standards-based tools.

              See https://github.com/pypa/setuptools/issues/917 for details.
              ********************************************************************************

      !!
        easy_install.initialize_options(self)
      /usr/local/lib/python3.8/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
      !!

              ********************************************************************************
              Please avoid running ``setup.py`` directly.
              Instead, use pypa/build, pypa/installer, pypa/build or
              other standards-based tools.

              See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
              ********************************************************************************

      !!
        self.initialize_options()
      running bdist_wheel
      running build
      running build_py
      running egg_info
      writing pbr to myownprojectdir.egg-info/pbr.json
      writing myownprojectdir.egg-info/PKG-INFO
      writing dependency_links to myownprojectdir.egg-info/dependency_links.txt
      writing entry points to myownprojectdir.egg-info/entry_points.txt
      writing requirements to myownprojectdir.egg-info/requires.txt
      writing top-level names to myownprojectdir.egg-info/top_level.txt
      [pbr] Processing SOURCES.txt
      [pbr] In git context, generating filelist from git
      warning: no previously-included files found matching '.gitreview'
      warning: no previously-included files matching '*.pyc' found anywhere in distribution
      adding license file 'AUTHORS'
      writing manifest file 'myownprojectdir.egg-info/SOURCES.txt'
      /usr/local/lib/python3.8/dist-packages/setuptools/command/build_py.py:201: _Warning: Package 'myownprojectdir.etc' is absent from the `packages` configuration.
      !!

              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'myownprojectdir.etc' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.

              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'myownprojectdir.etc' is explicitly added
              to the `packages` configuration field.

              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).

              You can read more about "package discovery" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

              If you don't want 'myownprojectdir.etc' to be distributed and are
              already explicitly excluding 'myownprojectdir.etc' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.

              You can read more about "package data files" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************

      !!
        check.warn(importable)
      /usr/local/lib/python3.8/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
      !!

              ********************************************************************************
              Please avoid running ``setup.py`` directly.
              Instead, use pypa/build, pypa/installer, pypa/build or
              other standards-based tools.

              See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
              ********************************************************************************

      !!
        self.initialize_options()
      installing to build/bdist.linux-x86_64/wheel
      running install
      [pbr] Writing ChangeLog
      [pbr] Generating ChangeLog
      [pbr] ChangeLog complete (0.0s)
      [pbr] Generating AUTHORS
      [pbr] AUTHORS complete (0.0s)
      running install_lib
      creating build/bdist.linux-x86_64
      creating build/bdist.linux-x86_64/wheel
      creating build/bdist.linux-x86_64/wheel/myownprojectdir
      copying build/lib/myownprojectdir/main.py -> build/bdist.linux-x86_64/wheel/myownprojectdir
      creating build/bdist.linux-x86_64/wheel/myownprojectdir/tests
      copying build/lib/myownprojectdir/tests/test_aqua_os_cliff_command.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests
      creating build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      creating build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_2_copy_api_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_2_copy_resource_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_1_copy_resource_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_2_creds_2_nonadmin_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_1_copy_api_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_2_copy_gui_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_1_cred_2_nonadmin_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_1_copy_gui_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/data/accounts_file_3_creds_1_nonadmin_acc.yaml -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account/data
      copying build/lib/myownprojectdir/tests/account/test_account_associate_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      copying build/lib/myownprojectdir/tests/account/test_account_build_file2.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      copying build/lib/myownprojectdir/tests/account/test_account_clean_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      copying build/lib/myownprojectdir/tests/account/test_account_associate.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      copying build/lib/myownprojectdir/tests/account/test_account_build_file_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      copying build/lib/myownprojectdir/tests/account/test_account_build_file2_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      copying build/lib/myownprojectdir/tests/account/test_account_build_file.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      copying build/lib/myownprojectdir/tests/account/test_account_clean.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      copying build/lib/myownprojectdir/tests/account/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/account
      creating build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
      copying build/lib/myownprojectdir/tests/common/test_common.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
      copying build/lib/myownprojectdir/tests/common/test_tabular_data.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
      copying build/lib/myownprojectdir/tests/common/test_accounts_data.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
      copying build/lib/myownprojectdir/tests/common/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests/common
      copying build/lib/myownprojectdir/tests/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/tests
      creating build/bdist.linux-x86_64/wheel/myownprojectdir/etc
      copying build/lib/myownprojectdir/etc/aqua-os.cfg -> build/bdist.linux-x86_64/wheel/myownprojectdir/etc
      creating build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/account_build_file.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/account_clean.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/account_associate.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/account_build_file2.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/account_build_file2_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/account_build_file_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/account_associate_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/account_clean_cli.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      copying build/lib/myownprojectdir/account/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/account
      creating build/bdist.linux-x86_64/wheel/myownprojectdir/common
      copying build/lib/myownprojectdir/common/templates.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
      copying build/lib/myownprojectdir/common/exceptions.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
      copying build/lib/myownprojectdir/common/openstacksdk_client.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
      copying build/lib/myownprojectdir/common/common.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
      copying build/lib/myownprojectdir/common/accounts_data.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
      copying build/lib/myownprojectdir/common/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
      copying build/lib/myownprojectdir/common/tabular_data.py -> build/bdist.linux-x86_64/wheel/myownprojectdir/common
      copying build/lib/myownprojectdir/aqua_os_cliff_command.py -> build/bdist.linux-x86_64/wheel/myownprojectdir
      copying build/lib/myownprojectdir/__init__.py -> build/bdist.linux-x86_64/wheel/myownprojectdir
      running install_data
      creating build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data
      creating build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data/data
      creating build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data/data/myownprojectdir
      creating build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data/data/myownprojectdir/etc
      copying myownprojectdir/etc/aqua-os.cfg -> build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63.data/data/myownprojectdir/etc
      running install_egg_info
      Copying myownprojectdir.egg-info to build/bdist.linux-x86_64/wheel/myownprojectdir-1.0.0.dev63-py3.8.egg-info
      running install_scripts
      error: invalid command 'bdist_wininst'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for myownprojectdir
  Running setup.py clean for myownprojectdir
Failed to build myownprojectdir
ERROR: Could not build wheels for myownprojectdir, which is required to install pyproject.toml-based projects

I notice youā€™re installing relatively old versions of packages (for
example, python-openstackclient 3.18.1 is from 2019) with what
appears to be fairly new SetupTools. This is likely to lead to
problems. What version of PBR is being pulled in during the build?

Newer versions of PBR attempt to maintain backward compatibility
with older versions of SetupTools, but older versions of PBR canā€™t
predict what changes are needed for newer SetupTools versions due to
lack of available time machines (Iā€™m hoping we find a solution to
the time machine shortage soon).

I changed this line in the setup.py, to be line ā†’ setup_requires=[ā€˜pbrā€™],
it was setup_requires=[ā€˜pbr>=2.0.0ā€™], I can update to whatever works. I am assuming if this would pull the latest pbr although I didnā€™t verify

Using setup_requires is deprecated, I believe, as it installs build-time dependencies using easy_install, which is no longer considered good practice. That might explain the ā€œRequirements should be satisfied by a PEP 517 installerā€ message in the output.

Just to circle back around to the beginning on this, PBR needs one
of three things to determine the version of the package youā€™re
installing:

  1. A Git repository so it can inspect tag info to guess a version

  2. Python package metadata (i.e. an already built sdist)

  3. An environment variable setting an explicit version string

Any of those three options should suffice. For option #1 you need
the actual repository, not just a copy of the worktree (so a GitHub
style ā€œrelease tarballā€ is insufficient). For option #2 you can use
build Ā· PyPI to create the sdist, but if youā€™re
doing that you may as well just use it to make a wheel and then PBR
isnā€™t even needed during installation. For option #3 just export
PBR_VERSION with the version you want used, but note that if PBR is
being used multiple times in that process for building different
packages then youā€™ll end up overriding all their versions, so this
is the least preferable solution.

As for doing modern PEP 517 style packaging with a pyproject.toml
file, PBR supports that. It still needs a stub of a setup.py file
present in the project to turn on the plugin:

import setuptools
setuptools.setup(pbr=True)

But everything else can be specified in pyproject.toml and setup.cfg
instead, as described in the documentation:
https://docs.openstack.org/pbr/latest/user/using.html#pyproject-toml

With very recent versions of SetupTools you can almost entirely get
rid of setup.cfg and move the rest of the metadata into
pyproject.toml instead, but at the moment PBR still expects to find
metadata.name set in setup.cfg (Iā€™m hoping to get that fixed soon).

One other thing worth checking I guess, make sure you have a new
enough pip. Old versions of pip lacking ABI3 support may be
retrieving sdists from PyPI because it doesnā€™t notice there are
suitable wheels for your interpreter version.

As for the second error, PBR hasnā€™t used bdist_wininst for several
years since calls for that were removed in 5.5.0, which is what
leads me to suspect old versions of PBR and other things.