Sdists for pure-Python projects

[mikeshardmind (Michael H)]

@EpicWink

@sirosen

seems to me that it is equally hard to carefully inspect a repo vs the sdist.

This isn’t true. For an sdist, I need a Tar extractor (not available natively on Windows), then some way to through each file and read the source.

tar has been available by default in windows since 2018 , but even if it hadn’t been, I don’t view this as a particularly compelling argument. you could just as easily point to needing git which hasn’t been (And still isn’t) installed by default on more than just windows if the project is hosted on a git server without a browser interface. There’s also an implementation of tar in python

I think if we’re talking about people who are reviewing code from an auditing perspective, we should be able to assume that basic common dev tools are things they will have at their disposal.

[sinoroc]

@mikeshardmind

@EpicWink

Tar extractor (not available natively on Windows)

tar has been available by default in windows since 2018

In case someone does not know it yet (or forgot about it), if you have Python you can handle tar archives even the compressed ones, no coding required:

python -m tarfile --extract some-sdist-1.2.3.tar.gz

tarfile — Read and write tar archive files

[brettcannon]

@kknechtel

Anyway, the main thing I want to say here is that all this time later, it still strikes me as strange that we have two totally different formats that are both fundamentally an archive of code with metadata, that record functionally the same metadata, in different formats.

The metadata is the same format, just with different requirements about what’s allowed to be dynamic.

@kknechtel

And that even when the code is pure Python, people are asked to upload both the sdist and the wheel, to serve different audiences, except that the sdist will be used as a fallback for the wheel (and not the other way around even though it should be perfectly technically possible), except that there are lots of disadvantages to using that fallback.

Pip lets you turn off that fallback.

@kknechtel

As far as I’m aware, a wheel’s metadata records everything that an sdist’s metadata would need to, and the only things that fundamentally prevent someone from manually downloading and unpacking a wheel and then treating it like an sdist are a) the assumption, built into the wheel concept, that there’s no more “building” to do and b) the absence of setup.py and pyproject.toml.

Basically, but that “no more ‘building’” is the key concept here.

[kknechtel (Karl Knechtel)]

@brettcannon

The metadata is the same format, just with different requirements about what’s allowed to be dynamic… Basically, but that “no more ‘building’” is the key concept here.

Well, yes, an sdist’s PKG-INFO matches a wheel’s METADATA. But I was thinking more about the fact that a wheel also needs WHEEL and RECORD files, and has a different folder structure, and uses a different archive format. It’s not as if Pip unpacks a .whl and then discovers that there’s still a build step. The difference is signaled up front, and then Pip needs to take a totally different sequence of steps to unpack and validate an archive, before even considering building.

And I’m thinking, I see the value in the up-front signal; but not in the other consequences of having a differently-compressed archive that’s missing two metadata files and gives the third one a different name and puts it in a different place and doesn’t have entry_points.txt pre-generated even though that generation is completely independent of a build step and probably other things I’m forgetting.