Should we exclude CI/CD files from sdist archives or not?

Yesterday I started a social survey related to inclusion or not of CI/CD configuration inside sdist archives. You can find it at https://twitter.com/sbarnea/status/1465357109269348364 and also vote (it will be accept votes for one week).

Now I realised that a more appropriate place to ask about this subject would be the python packaging topic as that is likely the place where people with most packaging experience would be lurking.

Keep in mind that my question is not about exclusion of test files, is about CI/CD specific files likes .github folder, stuff that very unlikely to be of any use for someone else.

I kinda assumed that anyone would know to strip big files that do not ship, like “assets” from their sdist, but I failed to find any official documentation related to those dotfiles related to CI/CD config.

I would assume that sort of thing would be omitted from a sdist. It’s omitted by default, isn’t it, so people would have to explicitly include it? Actually, maybe flit works differently here, as it decides what’s “part of the project” based on VCS data.

Sigh, yes it looks like flit includes everything in the sdist, even things like .github :slightly_frowning_face: That seems like a bad default, IMO.

They’re not particularly big, and they could be useful if someone wants to recreate the project (test) setup from scratch.

For example, you can see what is usually tested and how from the CI config. Not every project has a tox you can just run, so I sometimes need to check in there to see what I need to do setup and run tests locally.

There can also be contributing docs and other community files in the .github directory, they could be useful later too.

(And setuptools_scm includes them by default.)

2 Likes

PBR similarly computes the package manifest based on Git’s file
list, but can be adjusted to filter specific files by default if
there’s some consensus. The rationale there is that at least some
dotfiles are relevant to include, and it’s not easy to
programmatically guess which ones should be omitted, so it’s safer
to include them all rather than expect the maintainer to realize and
explicitly list the ones which are needed. In a vast majority of
cases I’ve seen, these additional configs don’t bloat the source
distribution significantly.

AFAIK flit only does that if you run flit build. The PEP 517 hook has different inclusion rules.

setuptools_scm which is quite popular with people who work on packaging in particular also works the same way.

Most of the project I manage are using setuptools-scm which includes all git tracked files by default.

2 Likes