Please make `package.__version__` go away!

[bwoodsend (Brénainn Woodsend)]

It sounds to me like we’re drifting towards wanting an end to end tutorial for each setuptools/hatch/… tool? In which case, it would probably make more sense for them to be maintained by them on their own websites? packaging.python.org would then just become a list of URLs to those tutorials and possibly, if it doesn’t overflow the opinionated-iness threshold, with some high level descriptions about what makes each one unique.

[oscarbenjamin]

I don’t think that the provision or use of package.__version__ is specific to any particular tools, backends etc. The backends can provide ways to help a package author provide the attribute and can document how to do that in their own docs. The question in this thread is whether the attribute should be expected or provided at all or whether importlib.metadata should always be preferred and should be the one way to do it in future.

This isn’t purely a packaging question for the project providing the attribute but also for downstream dependents: should they use __version__ rather than importlib.metadata?

It doesn’t really make sense to answer this question in the docs for the build backend of the upstream project because it is really a question of community/ecosystem wide expectations or standards.

[ncoghlan (Alyssa Coghlan)]

@oscarbenjamin

This isn’t purely a packaging question for the project providing the attribute but also for downstream dependents: should they use __version__ rather than importlib.metadata?

This consumption question does have a genuine answer:

• All installed distribution packages will have a distribution version. If that’s the info you want, then you want the importlib metadata APIs.
• Some specific libraries also expose runtime version information. If you are working with one of those libraries, then access the runtime version info as that library specifies (this is often a top level __version__ attribute using the same string format as distribution package versions, but may be an API call returning structured version information or some other interface)

[pitrou (Antoine Pitrou)]

I’ll point out that package.__version__ is much easier to remember and type at the prompt than the equivalent importlib incantation (which, tellingly, I do not remember at the moment). This kind of developer ergonomics is part of what made Python successful: you normally don’t have to invoke sophisticated reflection APIs to inspect object metadata, you just look up well-known “dunder” attributes (and auto-completion suggestions make them even more discoverable).

[pf_moore (Paul Moore)]

Agreed. And I think that in line with @ncoghlan’s comment on the consumption question, the production question also has a relatively straightforward answer:

• Always publish a distribution version via your project metadata.
• If you version your import packages independently, publish their versions via a __version__ attribute.
• If for any other reason, you want to let your users access your version via a __version__ attribute, do so.

If you want to populate your project version and/or your __version__ attribute from a single canonical source, then there are various ways to do so. Your build backend probably has at least one. But how you do it depends on what tools you are using, and you’ll need to find a solution that suits your workflow.

IMO, the only really controversial question here is how to populate multiple locations where the version is stored from a single source. But that’s precisely the one where individual tools are the right place to find a solution - and tool selection is always a personal choice.

In some ways, this whole controversy is pointless, fuelled by an over-zealous insistence on the “don’t repeat yourself” principle.

[ncoghlan (Alyssa Coghlan)]

Indeed, one of the potential answers to single sourcing the package version is “Don’t do that, just add a test case that fails if they are different”.

[python-intermediate]

@pitrou

I’ll point out that package.__version__ is much easier to remember and type at the prompt than the equivalent importlib incantation (which, tellingly, I do not remember at the moment). This kind of developer ergonomics is part of what made Python successful: you normally don’t have to invoke sophisticated reflection APIs to inspect object metadata, you just look up well-known “dunder” attributes (and auto-completion suggestions make them even more discoverable).

@pf_moore

If you want to populate your project version and/or your __version__ attribute from a single canonical source, then there are various ways to do so. Your build backend probably has at least one. But how you do it depends on what tools you are using, and you’ll need to find a solution that suits your workflow.

IMO, the only really controversial question here is how to populate multiple locations where the version is stored from a single source. But that’s precisely the one where individual tools are the right place to find a solution - and tool selection is always a personal choice.

@ncoghlan

Indeed, one of the potential answers to single sourcing the package version is “Don’t do that, just add a test case that fails if they are different”.

Gives the consensus around these ideas (as evidenced by the amount of “likes”), how do people here feel if we update the discussion page to include a paraphrase of these these comments, add links to different backend build systems and remove specific instruction for a single build system by closing #1276?

I’m happy to create a PR that includes and summarizes the core developers’ comments and cements our consensus around these guidelines.

[pf_moore (Paul Moore)]

At a minimum, I’d want a couple of things before this was done:

1. The “in progress PEP” mentioned earlier in this thread was formally abandoned, and the author(s) explicitly give their approval for a new PR in the form you suggest.
2. One or more maintainers of the packaging user guide explicitly agree that this is the direction they want the documentation to go in.

Without both of those, I still maintain my position that we should do nothing until we get community consensus (i.e., I consider those two groups to be essential to form consensus).

Note that even then, that’s a minimum. If anyone speaks up in opposition to this idea, then I’m going to say we don’t have consensus, and I’d rather my name wasn’t associated with what would then simply be yet another iteration of this endless argument

In case it’s not obvious, I’m pretty sick of all this. My absolute preferred option would be to remove all discussion of this topic from the packaging user guide, and then for everyone to just move on to talk about something else.

[ncoghlan (Alyssa Coghlan)]

From my PoV (wearing my PyPUG maintainer hat), I need to re-read the open PR against the discussion page, and once I muster up the motivation to go that far, I’ll likely also have sufficient motivation to make a new PR of my own to address the points raised in this thread.

Having a second pending PR to reconcile wouldn’t help on that front (although the offer is appreciated).

Edit: to clarify, even if I do make my own PR, it would be to post the link to this thread for further feedback, I wouldn’t just go ahead and merge it without further discussion.

[ncoghlan (Alyssa Coghlan)]

PR addressing the comments in this thread is here: Add notes on runtime version access by ncoghlan · Pull Request #1607 · pypa/packaging.python.org · GitHub

It incorporates the previously pending change to outright remove the old setuptools-specific guide, but replaces it with a HTTP meta-refresh to the discussion page that was added in August rather than having the link die completely.

The single-source-version discussion page is updated to emphasise that it specifically applies when the distribution package version and the import package API version are directly coupled and should always have the same value. It also now mentions the approach of adding a test case for consistency to prevent divergence, rather than actually single sourcing the version information.

The overall version identifier discussion gains a new section (mutually cross-referenced with the single-source-version discussion) about accessing version information at runtime, pointing out that:

• all distribution packages will have an accessible distribution package version (via importlib.metadata.version)
• import packages may expose a top-level API version via a __version__ attribute
• import packages may expose additional runtime version information when it makes sense to do so (using the ssl module publishing OpenSSL information as an example)

The rejection of PEP 396 is also mentioned, when noting that __version__ should only be queried on import packages that are known to provide it.

[pf_moore (Paul Moore)]

I haven’t approved that PR because I still want to see evidence that if we make this^[1] change, the community will finally drop this whole debate and move on. In particular, I want the “in progress PEP” to be explicitly abandoned. But having said that, the PR seems reasonable to me.

---

1. or any ↩︎

[ncoghlan (Alyssa Coghlan)]

Reading version_pep_draft/pep-NNN.rst at main · tacaswell/version_pep_draft · GitHub, I think my PR describes a slightly weaker version of what that PEP suggests (cc @tacaswell ).

Both proposals are aimed at countering a specific misreading of the PEP 396 rejection (as seen in the initial post in this thread): that because PEP 396 was rejected, import package APIs should never publish a __version__ attribute.

That isn’t why that PEP was rejected. It was rejected because adding __version__ only makes sense for some packages, and the specific way that PEP was originally written didn’t even make sense as an opt-in guide any more due to the packaging ecosystem changes between its initial publication in March 2011 and its eventual formal rejection a decade later in April 2021.

The draft PEP opposes that misreading by saying that package publishers should version their import APIs (^[1]).

The open packaging user guide PR doesn’t go that far, and instead simply says that many import packages do provide an import API version as __version__, but you should check if the package of interest does so before relying on it being there (^[2]).

Edit: assuming the packaging guide PR does get merged, I’d be inclined to add a note to the top of PEP 396, referencing the new runtime-version-access section (to help pre-empt any future misreadings of the PEP 396 rejection)

---

1. (to which I would personally say: if you don’t have a specific use case for doing so, why bother? The distribution package version is always available when it comes to describing environments, and if runtime access to the version information turns out to be a recurring need for a particular library’s users, then __version__ can be added later). ↩︎

2. (while it’s a common convention, it doesn’t rise to the level of an ecosystem interoperability standard, but it also doesn’t need that level of adoption to be useful in the cases where it is relevant). ↩︎

[ChrisBarker-NOAA (Chris Barker)]

@ncoghlan

It was rejected because adding __version__ only makes sense for some packages, and the specific way that PEP was originally written didn’t even make sense as an opt-in guide any more due to the packaging ecosystem changes between its initial publication in March 2011 and its eventual formal rejection a decade later in April 2021.

We’d have to ask Barry about the details, but it’s my impression that the PEP was not so much rejected as withdrawn.

Also note that the PEP included some bits about adding __version__ to standard library modules, and I don’t think anyone still thinks that’s a good idea.

I do want to apologize – an oversight of mine resulted in this all being more painful at the end – when I submitted the PR for the “new page” in discussions, I intended the old one to be removed, but I got lost in the shuffle of the reorg of the document, so didn’t put that in as part of the same PR.

The good news is that my mistake resulted in this conversation, which, in the end spurned @ncoghlan to make it better – so painful as it was – we got to a better place: Thanks Alyssa!

[ncoghlan (Alyssa Coghlan)]

@webknjaz and I have merged the PyPUG update at Add notes on runtime version access by ncoghlan · Pull Request #1612 · pypa/packaging.python.org · GitHub

https://packaging.python.org/en/latest/guides/single-sourcing-package-version/ is now just a HTTP redirect to the discussion at Single-sourcing the Project Version - Python Packaging User Guide which also references specific notes on “Accessing version information at runtime” as a new subsection in Versioning - Python Packaging User Guide

The version querying documentation uses cryptography as an example of a package that versions both its distribution package and its import API with the same version number:

Version information for all distribution packages that are locally available in the current environment can be obtained at runtime using the standard library’s importlib.metadata.version() function:

>>> importlib.metadata.version("cryptography")
'41.0.7'

Many projects also choose to version their top level import packages by providing a package level __version__ attribute:

>>> import cryptography
>>> cryptography.__version__
'41.0.7'

The new runtime version querying subsection also references back to the single-sourcing discussion.

Edit: I also closed the 4(!) open PyPUG issues relating to this topic (links on the PR page)