I would like to add some datafiles to my PyPi package. This documentation for setup.cfg is aimed at someone more expert than me. It states the following without any explanation of what the elements are:
I’ve no idea what site.d… or host.d… mean. I imagine that the first line starting ‘data’ means ‘create a directory called “data” and copy the following files into it’. I tried this
[options.data_files]
data = reference.dat, help.html
But PIP doesn’t like that.
Is there any setup.cfg for a newbie? Or just an answer to this problem?
Are you sure you want data files (files installed outside of site-packages, which are currently not well-supported) and not package data (files installed inside your package)?
That’s a smart question! I was using “data files” as a generic term, just meaning “files containing data which my package needs”. It looks like I need “package data”.
So far I have found that MANIFEST.in is only good for source packages and I want a binary package (Wheel) as well. I read that setup.py is deprecated and I should be using setup.cfg.
Your link
Contents
[Including Package Data in Projects]
[Including Package Data via MANIFEST.in
[Including Package Data via setup.py]
[Excluding Package Data via setup.py]
seems to have everything that I don’t want except perhaps
[Including Package Data via Setuptools Plugins]
but that is adding another level of complexity - is that the only way? I only want to add two plain-text files.
Typing python setup.py bdist_wheel in your terminal is deprecated. You can still write it, you just need to let pip or build run it, not run it directly yourself.
There are two types of metadata: static and dynamic.
Static metadata ( setup.cfg ): guaranteed to be the same every time. This is simpler, easier to read, and avoids many common errors, like encoding errors.
Dynamic metadata ( setup.py ): possibly non-deterministic. Any items that are dynamic or determined at install-time, as well as extension modules or extensions to setuptools, need to go into setup.py .
Static metadata ( setup.cfg ) should be preferred. Dynamic metadata ( setup.py ) should be used only as an escape hatch when absolutely necessary. setup.py used to be required, but can be omitted with newer versions of setuptools and pip.
I’m beginning to think that the advice in the tutorial above is a load of old ****** as there doesn’t seem to be any simple way to do it otherwise. I included some data files in a Debian package like this:
#package_data={'': ['*.png']}, #ONLY TEXT OR RST OR MSG*
That’s now a Flatpak and the files were managed by Meson which IMHO is much more straightforward.
So is setup.py really deprecated or not ? Is there a sensible way of using setup.cfg? Would it be possible to create this package using Meson instead (in my dreams)? I’d give up and create a Flatpak but it doesn’t seem appropriate for a simple command-line program.
Yeah, setuptools makes a lot of the process really straightforward, but when you hit an edge case, it gets very messy. If you’re doing pure Python stuff, you may prefer flit, and there are a few other build backends available now, though none better for cross-platform native extension modules yet.
Well it’s messy and it’s against the rules, but it works (pending some testing) - that’s what counts. I’m not going to waste any more time trying to follow the recommendations in the tutorial.
This PyPI packaging system seems messy to me, a bit like a Debian package with all its files, some doing very similar things to each other.
from setuptools import setup, find_packages
import os.path
As far as I know, anything you can pass to the setuptools.setup function (in setup.py), you can pass to setup.cfg (unless you need to run code at build time, which is not common). If this is the case, and the tutorials don’t make this obvious, then perhaps we should update the docs.
For your example, try this:
[metadata]
name = myapp
version = 0.0.4
[options]
packages = find:
[options.package_data]
* =
myapp.dat
myapp.html
The documentation appears to have been written by people who already know how it works, for people who also already know how it works. StackExchange has many queries on this subject with conflicting answers. So…after much tedious trial-and-error:
setup.cfg [options.package_data]
will put the reference files in the tar.gz file.
won’t put the reference files in the whl file.
setup.py
will put the reference files into both, so despite it being deprecated, that’s the one to use.
package_data doesn’t work
data_files=[(’’, [‘myapp.dat’, ‘myapp.html’])], does work .
I can see the data files in the tar.gz and whl files but not where I think the package has been installed: /home/chris/.local/lib/python3.10/site-packages/myapps
Whether files are placed into sdists or wheels is not a matter of whether you’re using setup.cfg or setup.py; you just need to supply the right options, which can be given in either file. Using just [options.package_data] should place the files in both build formats; how are you checking whether they’re ending up in there? For a wheel, it’s just a zipfile, so you can use zipinfo or the like to inspect the contents. If [options.package_data] really isn’t working for you, could you supply a link to your repository?
Also, on the matter of the section titles in the article I linked to, they only refer to setup.py by contrast with MANIFEST.in; the actual instructions cover both setup.py and setup.cfg.
Also, as I indicated above, if you’re using data files (as opposed to package data), then the files, by definition, will not be installed inside your package in the site-packages directory. Where they do get installed is complicated and depends on whether you’re doing a system, user, or virtualenv install.
Package data sounds like what I want as I’ve no idea why there is a distinction, I just want to find the files. Laurie’s statements:
[options.package_data]
* =
myapp.dat
myapp.html
need a comma between the files, which I added, but the files don’t appear in the package. What does the ‘*’ indicate? The root directory of the package?
The files are local so far as I’m messing about with them so much.
I wish I had started with ‘flit’ as that looks like a more straightforward and up-to-date way of doing things. However I don’t really want to have to start all over again by learning that now (:. Maybe if all else fails.
The asterisk matches any package in your source tree that happens to contain a myapp.dat or myapp.html. Where are these files located in your repository? Package data only works if the files are actually located inside your Python package (the folder with the actual *.py files) to begin with.
It’s all in the same directory. It looks as though the problem with setup.cfg was the references to src. Now I copy the reference files to src in my build directory, the reference files are included in both the wheel and tar.gz files.
Still no sign of them in /home/chris/.local/lib/python3.10/site-packages/myapps
It was installed with python3 -m pip install ~/Packages/myapp/build/dist/myapp-0.0.1-py3-none-any.whl --force-reinstall
Your project structure seems incorrect - you should have files like pyproject.toml and setup.py at the top level, most definitely not in a subdirectory called pip.
except perhaps