This thread is intended to be used as a canonical location to point users to when asked about building a RPM package without a setup.py.
Background
Traditionally, Python packages are built as a RPM package by running setup.py. The Python packaging community, however, is steadily moving away from relying on setuptools as a first-order tool, promoting instead to use wheels as the primary exchange artifact format, and PEP 517 as a way to build them. This enables Python projects to use alternative packaging tools while maintaining interoperability of their artifacts, and there is a visible trend for Python projects to not include a setup.py as a result. (See: Which mainstream projects removed setup.py files?) And even for a setuptools-based project, the recommendation is still to build a wheel using setuptoolsās PEP 517 hooks, instead of invoking setup.py directly.
Building for RPM
Meanwhile, however, RPM packaging tools are still catching up to this new trend. Fedoraās official guidelines, for example, still instructs using setup.py directly. Developer from Fedora are working on a provisional implementation, pyproject-rpm-macros, to provide tooling to build a PEP 517 project using the pyproject.toml build tool definition, as specified in PEP 518. This provides a solution to build a RPM package from a Python project that does not contain a setup.py, and also a migration path to move off invoking setup.py directly if a Python project has declared itself as using PEP 517, to better ensure the resulting RPM package has the same behaviour as the Python package installed as a wheel.
Note that those are Fedora guidelines, not general RPM guidelines. E.g. openSUSE has their own ā possibly also using setup.py behind the macros.
While it is true that Red Hat is my employer, Iād rather say āDevelopers from Fedoraā. We donāt name Python developers based on the company that pays them. We also have contributions from people not employed by Red Hat. Thanks.
Note that pyproject-rpm-macros are late alpha and may still change.
If you want to use them, please subscribe to Fedoraās python-devel list; thatās where we discuss and announce changes.
Any progress on this? Iām trying to move my package to use hatchling, but what is stopping me is that I havenāt found a way to build an rpm (to replace python3 setup.py bdist_rpm)
I had seen that, but Iām still missing some pieces.
The Packaging Guidelines assume a starting point of a .spec file. Python setup.py bdist_rpm would build the spec file, so that nothing additional to setup.py was needed.
Is there a means to generate a .spec file from a pyproject.toml file than can then be used with normal rpm tools, like was possible with bdist_rpm. Are we expected
to build a spec file as a starting point, and maintain it in parallel to the pyproject.toml?
for what itās worth, I have pip packaging fine, and use the debian integration (which is excellent) for debian and ubuntu and friendsā¦ the purpose of the rpm packaging is for linux distributions that use that natively. So I think itās the right tool for that job.
OK, so put together a fedora 39, and built a spec file working to the guidelines above.
Now I need to install on Redhat Enterprise 8 and nothing worksā¦ everything is too oldā¦
hatchling doesnāt even install, none of the pep macros are there, itās python 3.6ā¦
Since hatchling doesnāt install, the build is brokenā¦
have to support multiple OS versions at the same timeā¦ any possibility
of backport support to python 3.6ā¦ or should I be using a different build tool?
Thatās not surprising. RHEL 8 is based on a Fedora that was released 5 years ago.
However, you can choose to use Python 3.11 on it ā but not all the other Python packages exist for it there.
Anyway, the approach described in the Fedora packaging guidelines works on Fedora, not ancient RHELs. It works on RHEL 9 today, but it is quite possible that when you read this post in 2028, this is no longer true.
I have no idea what are you trying to do. If you are trying to target Debian, Ubuntu, Fedora, RHELā¦ maybe you should use wheels?
I do produce wheels, itās on pypi.org, it installs the ānormalā way. This thing I package is more of a system utility, than an application. I work in a systems support group, so clients expect the package to be part of the operating system, rather than something they configure. so the client expectation is for it to ābe there.ā in /usr/binā¦ On ubuntu, for many years, this was literally automatic. I have a PPA on launchpad, and the debian ecosystem deals seamlessly with it. But then we migrated some clusters to RHEL.
These are IBM supercomputer clusters with vendor support. The ācurrentā OS there is RHEL8. RHEL8 is still getting maintenance releases, with 8.10 scheduled for next year, and another 5 years of support after that. Similarly ubuntu 18.04, is also python 3.6 and is expected to remain supported until 2028.
These 10 year life spans are sort of normal in the industrial/scientific realm.
Running setup.py might be deprecated in newer Python/setuptools versions but if you want to target older versions then you might need to use the older ways of doing things. PEP 517 based tools sort of start from Python 3.7 so if you need to straddle Python 3.6 through to 3.12 then the simplest solution is just to use a setuptools-based setup.py as you presumably already are/were. You can still add a pyproject.toml to be used in newer versions but you would still need to run setup.py directly on Python 3.6 when making wheel/sdist.
That doesnāt really change the situation: Fedora will nearly always be
ātoo newā (glibc versions, etc) for a binary built there to have any
promises on systems that have a many year older baseline. The model for
decades has been to pick an unofficial ālowest common denominatorā for
your build - which in your case would be EL 8, or to bundle the
necessary support bits, which is what happens when you build a wheel
against manylinux.
For what itās worth, 3.6 would support twine 3.8.0, build 0.9.0 and Setuptools 59.6.0. This should be enough to build wheels following the PEP 517/518/621 standards (IIRC the minimum Setuptools version specified for Pip to use as the ādefaultā backend is 40.8).
Many downstreams are explicitly against the package projects making their own spec files, but if you want to test them upstream, you kinda need those.
Thereās a project called Packit that is specifically targeting upstreams, providing them with a CI/CD infra for making and testing RPMs, with some automation for sending them upstream later on.
Generating a spec file is sometimes done with Jinja2, when people want to make physically different files for different targets; I do this in another way ā one spec file, with conditionals/params.
I have a rather complicated example to show, maybe itāll be helpful for you:
Note, that using UBI and centos stream often requires adding external repos like EPEL or linking RPMs for some deps. Ideally, those RPM deps would need to be added to the upstream repos. In case of UBI, thereās a limited number of things in the default repo without a subscription.
P.S. @petersilva oh and I wanted to point out that there was no support for the PEP 517 in-tree build backend that our project implemented at the time (pip was too old, no build), so I just included the corresponding tarballs from PyPI in the SRPM and made use of them during the build. So vendoring such artifacts is sometimes an acceptable workaround for lacking build deps in the distro repos. Only use this approach for things that are absent from the distro.
I just wanted to thank you for the example. It sounds reasonable, but a bit hard to follow.
It will take me a while to digest and try out, but it looks helpful.
Have the same problem the python package inquirer had a setup.py at v2.6.3 and we used to build that to a RHEL rpm with python3 setup.py bdist_rpm
Latest v3.1.4 no longer has a setup.py just a pyproject.toml so weāre hitting the same issue
hi folks, as an update. Iāve done a few versions since producing a hatchling pyproject.toml, and this is what Iāve settled on. I need to support ubuntu 18, and redhat 8 which both have python3.6, and ubuntu20 does not quite work with hatchling either. so I keep a separate branch for older OSās where I remove pyproject.toml, and edit debian/control to remove the modern debian packaging dependencies.
So on older platforms I build using setup.py, and using setup.py I use bdist_rpm which builds an RPM that doesnāt do dependencies properly, and I advise users about how to separately install the dependencies. I just gave up on proper dependencies.
On ubuntu 22.04, and fc39, the hatchling stuff works well, and I have a spec file to build my own RPMS just fine (with good dependencies.)
Iāll keep using parallel branches until OS/platforms that require setup.py go away.