Test.pypi.org is marginally unusable (if my explanation is correct)

Packaging
1

When I do pip install -i https://test.pypi.org/... package , it fails on dacite 1.8.0 . The same package exists in pypi.
I think that is because it uses only dependent packages from test.pypi when you install from test.pypi.
If that is really the case, I don’t see how it provides an educate simulation of the real scenario.
Am I right? Is there other explanation?

2

I think it depends on what you expect test.pypi to be. As far I’ve seen, it’s mostly used either as just a little sandbox, or a place to stage packages for a human to verify everything looks ok (the description rendered correctly, the right wheels are there, etc…)

I’ve used it to demo to people how to build+publish packages, knowing I’m not dirtying the real PyPI, since no one should be installing these things.

I suspect if you allowed pip to also pull from vanilla PyPI your installation would succeed.

3

Another thing the test PyPI has been useful for is validating
uploads, since twine check still doesn’t verify lots of the things
that could cause PyPI to reject an upload, e.g. invalid Trove
classifiers: Twine check fails to warn on invalid classifier · Issue #976 · pypa/twine · GitHub

2 Likes
4

You can also try:

pip download --no-deps --index-url http://test.pypi.org 'dacite == 1.8.0'

to test if the package was actually uploaded. Any other tests aren’t much different from installing from a package on your local filesystem

1 Like
5

Ideally build backends would catch most errors that would invalidate upon upload. For example, Hatchling does error for invalid classifiers and I think others do as well.

1 Like
6

poetry check also validates metadata, such as Trove classifiers.

1 Like
7

I thought the use case of it is to make sure a package works prior to publishing a package in pypi. Arguably you could install the wheel yourself, but it is better to use it. I use different computers to test it (VM) , so it is handy.

8

I used it a lot to exercise package upload while making my personal
workflow for that.

I don’t think anyone should expect it to have a complete suite of
packages, or a consistent set of them.

On 04Oct2023 23:41, Eyalk11 via Discussions on Python.org

9

I strongly recommend you never install anything from test.pypi.org. If pip refused to install from it, I’d be happy.

There’s no guarantee that any package (including all of its [transitive] dependencies) are owned by the same people who control the package on actual PyPI, and as far as I’m aware nobody is actively scanning it for malware. Your chances of being typosquatted are rapidly approaching 100%.

Use it to do a test publish and review the package page. That’s it. If you need to test installing a package or share a package, you need some kind of private feed.

7 Likes
10

I kinda like the idea of not being able to install things uploaded to test.pypi.org. The one thing I’d prefer we do differently would be to prevent the installation abilities from PyPI’s end, refusing to serve a /simple/ page under TestPyPI.

This is a change that’s easier to backtrack, if we get feedback that persuades us accordingly and it’s also addressing the problem at the “source” in some senses.

5 Likes
11

How does that look now that we have trusted publishers? Would you set up a trusted publisher on test.pypi.org as well? Not worry about it and still use twine just for PyPI’s upload validation checks?

12

The Packaging docs do a reasonable job of warning about test pypi:

Note: This example uses --index-url flag to specify TestPyPI instead of live PyPI. Additionally, it specifies --no-deps. Since TestPyPI doesn’t have the same packages as the live PyPI, it’s possible that attempting to install dependencies may fail or install something unexpected. While our example package doesn’t have any dependencies, it’s a good practice to avoid installing dependencies when using TestPyPI.

My suggestion would be for test pypi to prevent installing any package that a user did not upload themselves. That way it would really be for testing only, and avoid the pitfalls of installing packages from an index that was never intended to be used for dependencies. As far as I know, test pypi is only there to trial uploading wheels and checking that the page rendering is as expected.

2 Likes
13

Yes, it’s useful to have a trusted publisher set up for Test PyPI, for example, to deploy on merges to the default branch; so you know the release machinery is running smoothly and prevent surprises on release day.

3 Likes
14

What if the dependences of your package in test.pypi.org would be taken from pypi.org unless otherwise specified? Wouldn’t it be more useable that way?

15

I think allowing this allows for further dependency confusion.

For example, given someone is using test.pypi.org as their primary index, and allows for this kind of pull-through behavior, and they depend on package foobar from pypi.org , when someone else uploads a foobar to test.pypi.org , then the end user will no longer receive the package they asked for.

16

IIUC, ~all use cases for test.pypi.org go away if we can have Draft release feature on main archive to allow testing a release before it goes live · Issue #726 · pypi/warehouse · GitHub on the main pypi.org website; so it might make sense to actually push any energy we might have toward solving the test.pypi.org papercuts toward implementing that suggestion.

5 Likes
17

While I generally agree that it would be better, this issue is from 2015, and still no PR. Do you actually think it will be implemented?

18

To allow for backwards compatibility , it could be specifically mentioned in the dependencies, or as a pyproject.toml option:
Let say testpypy_deps_from_pypy=true on an appropriate section.

Additionally, why on earth someone other a maintainer of the package would use it instead of pypi?

And as an author myself, I prefer packages from pypi, just like in the real case. This discrepancy between test env and “production” env is not good IMO.

19

Not sure! People surprise me all the time.

There’s an open issue that entertains removing TestPyPI: Stop having a "staging" environment · Issue #918 · pypi/warehouse · GitHub

1 Like
20

Yes? I will point out that my first bit of major involvement in Python packaging was an issue from 2011 that I contributed a fix for in 2016/17. These things are reliant on volunteer resources and the timelines for those look very different from timelines of full-time funded roles – especially when it’s a “thing we want to do”.

1 Like