Is there a specification for how data files are included in sdists and bdists (wheels)?

I am aware of Including files in source distributions with MANIFEST.in — Python Packaging User Guide which describes how files are included in source distributions, but I wonder if there is a PEP where this procedure is described (or if that Packaging guide is the most authoritative source I can look at on the topic). setuptools has package_data and include_package_data, whereas flit includes all data files, and I understand that poetry does a similar thing.

On the other hand, I see no description of how data files are included in wheels, or if is there any difference at all. I usually go to Less known packaging features and tricks to recollect how this is done right, because I usually get it wrong.

You just put them inside, like you do to python files.

1 Like

So for the title question, the answer is, well, the question is invalid. Neither sdist nor wheel distinguish between source and data files; a file is included if it’s in the archive and (in wheel’s case) listed in RECORD. But since OP mentioned MANIFEST.in, package_data, and include_package_data, I’m guessing the actual question is something like Is there a standard way I can tell a build tool what files to include when it builds my sdist and wheel. The answer for which is no, the rule of file inclusion is entirely a contract between you and the build tool (setuptools, poetry, flit, etc.), and there is not a standard rule tools must follow.

The reason is two-fold. First of all, the standard is not technically necessary, since it is never expected a developer would freely switch between tools anyway; if a project uses flit, its maintainers all use flit, and all the flit configs only need to make sense for flit. However, the same can be said for all project configurations, yet we still have PEP 621. The reason this was not included in PEP 621 is because PEP 621, as it stands, is already very long, and the discussion around it too diverging, we need to draw a line somewhere to limit the scope so the discussion and specification is managable, and file inclusion/exclusion logic is one of the things that are 1. not tightly related to anything else, 2. tools currently do things sufficiently different that a discussion around it can derail other topics, and 3. few people strongly want it standardised. So we decided to leave it out of PEP 621.

This does not mean there cannot be a standard for this, however. You can write a PEP for it yourself, and as long as setuptools, flit, poetry, etc. people can gather around and agree on a common logic, it will be standardised. That’s how every packaging PEP works.

4 Likes

Thanks a lot @uranusjr - yes, Is there a standard way I can tell a build tool what files to include when it builds my sdist and wheel is what I was asking. It makes sense that it was left outside of PEP 621 for the reasons you outlined. I have no interest in writing a PEP myself, just wanted to make sure I was not missing anything obvious :slight_smile:

1 Like