Hello,
PEP-689, and the current devguide, specify that if a function “graduates” from private/internal API to unstable, its old name (with an underscore) should be kept as a deprecated alias in the headers.
In effect, if people resort to using private API, and later we decide it’s good enough to make public, we “pardon” the users rather than “punish” them by requiring them to change the code in time for the next release.
But, we don’t currently do that when a function “graduates” from private/internal API directly to public.
Should we?
If not, then keeping the old names for private→unstable transition would be weirdly inconsistent, and we should probably get rid of that too.
FWIW, I don’t think keeping aliases around is a big maintenance burden.
We can change the underscored names: rename them, add new parameters, change the returning type, change the behavior in corner cases. We did this in the past. It is okay as far as it serves our needs and leavs users with the ability to achieve the same result at the same cost. Of course, we don’t intentionally break them unnecessarily, but users who use them do so at their own risk.
But removing function without offered something in return is not good. And the replacement must be of equal value. For example, to convert long integers between Python and a third-party implementation, you can convert them to a hex string on one side and then parse it on the other. It works, I did it in Tkinter. But it is more efficient to use packed intermediate representation, and it is even more efficient to have access to internal PyLong representation.
I think that we should not keep underscored names after providing an equivalent public API, but we should not remove them until the equivalent public API is available.
I suggest we don’t do that for aliases to things that became public API (without any modification). Especially for functions that are used in the wild.
For those migrating to the new name is easy to do in a way that doesn’t cause problems when supporting older python releases in an extension. Either by adding #define-s to the extension or using @vstinner’s pythoncapi_compat project.
There’s IMHO a better use case for keeping underscored names around of a while when the replacement has a different interface.
I beg to differ. “Naming is hard” is an oft-heard excuse, and there are several naming-related issues already on the table (e.g. the endless discussion about whether “steal” is politically incorrect :-).
Well, each forced change to working code is a chance to re-evaluate whether to stick with Python.
But, well, if that’s the way, I’ll accept it; PEP-689 needs to change.
But this I don’t understand at all. Why use a different project, nominally maintained by the same set of people as CPython? Why not put the alias in Python.h itself?
I think pythoncapi_compat is great for new API, when it can be implemented in terms of the old. We can’t change the old versions of Python But we can change new ones.
I meant picking a new name for new private API.
If that API has intentions of becoming public, then yes, naming is hard – but in that case you need to pick a good new name regardless of whether the old one is kept around.
We’re talking about underscored APIs here, using those is a bit icky in the first place. I’m probably showing that I’m getting old and am still not used to the popularity of Python, but I consider having to adapt to changes in private APIs something that’s part of the cost of using those APIs.
That said, I don’t mind adding backward compatibility aliases for private APIs that are used by popular projects. But if we do we should also consider adding backward compatibility stubs for private APIs where the public replacement has a different shape.
But this I don’t understand at all. Why use a different project, nominally maintained by the same set of people as CPython? Why not put the alias in Python.h itself?
I think pythoncapi_compat is great for new API, when it can be implemented in terms of the old. We can’t change the old versions of Python But we can change new ones.
[/quote]
Right, using pythoncapi_compat allows one to adopt the new public API without having to worry about backward compatibility. There’s still churn in the extension code, but at least the extension author doesn’t have to add preprocessor code to support older python versions.
It’s not just about churn, but also the timing of the churn. How long is the window where the extension author expected to do the update?
Finding a free evening (or weekend) during several years is easier than during one beta or RC period. A grace period of 2 years is much friendlier than a few months.
It might seem that there’s not much difference between 2 years and 7 years (the procrastinators will do the change at the last possible moment). But there is: if you can drop support for Python versions that don’t provide the new API, switching is easier – you don’t need #ifdefs.
FWIW, adding a pythoncapi_compat dependency to a project is not free. C doesn’t really have a package manager, so you’re on your own there. And vendoring it isn’t straightforward. (When should you update? Will you/your users need to worry about things like SBOM? DId you include the licence?)
Not to mention pythoncapi_compat only works for C/C++.
It’s often easier to add an #define directly. (But by doing that you’re defining a name in the _Py* namespace: the next version of CPython is free to reuse the name for something completely different. pythoncapi_compat helps here, as a supported “graveyard” of names that shouldn’t be reused.)
Agreed. That’s always been the meaning of the private API. “Don’t come crying when it breaks on you.” Adding aliases only kicks the can down the road, and it’s only useful for cases where a private API is “promoted” to being public. If does nothing for the case where a private API is removed.
But: Where do we document that this is the meaning of the underscore?
The docs are being tightened, but even if we’re now very clear that this API is off-limits, removing it isn’t fair to people who already use it. Some of it was even publicly documented!