Formalize the concept of "soft deprecation" (don't schedule removal) in PEP 387 "Backwards Compatibility Policy"

Hmm, I guess the issue with “obsolescent” though is it still implies that the feature was once useful, but no longer is, and specifically per that definition that it might be considered for removal in future versions…which sounds a lot like what a PendingDeprecationWarning was supposed to be. And on the other hand, it puts the emphasis on it being old, no longer useful and considered for removal, as opposed to being discouraged for existing and especially new use, which seems to be more the message that we want to send here in most of these cases.

As such, to more imply the latter, what about calling this state “Discouraged” instead?

3 Likes

I don’t think this should apply to “soft deprecations”, where it’s OK to use the old code – it has some surprising behaviour, or is missing support for some new features, or doesn’t follow the latest naming conventions – generally it’s just no longer the best way to do something.
When changing the code that uses “soft-deprecated” API, you’re likely to hit its limitations, and you should consider moving away from it. But otherwise, there’s no reason to do that. Your time is much better spent on something else.

3 Likes

FWIW, I think obsolescent is too obscure of a word, I just mentioned it because it’s used by another large/old language in this context, but I personally think “obsolete” would be fine (and obviously we don’t have to take over the wording from the C standard, but can come up with something appropriate for us).

Its definition doesn’t even imply removal; obsolete:

  1. out of use or practice; not current
  2. out of date; unfashionable or outmoded

Personally, I think that it’s a more accurate word than discouraged, but that would also work.

2 Likes

Obscurity of a word can sometimes be an advantage though. If people don’t know what a word means, they’re more open to looking up how it’s being used, instead of thinking they already understand it. A lot of the issues here stem from preconceptions of terms like “deprecated” and what they imply, and no matter how much you document something, people won’t always interpret the words the same way you do. (Though sometimes you can completely dodge the issue by assigning completely arbitrary meaning to words, like talking about “blue” functions and then using a colour highlight on them, but then you lose ALL intuition, so that’s less effective in another way.)

1 Like

IMO, familiar or not, “obsolete” has still has the same fundamental problem I raised with “obsolescent” above:

And on the other hand, it puts the emphasis on it being old, no longer useful and considered for removal, as opposed to being discouraged for existing and especially new use, which seems to be more the message that we want to send here in most of these cases.

At least in my thinking here, I’m most concerned about sending a message for cases, like PyObject_SetAttrString(obj, attr_name, NULL), or, say `tempfile.mktemp, where something is actively not recommended, as opposed to simply being an old alias or not merely the new hotness. Though, arguably, those cases might be most deserving of an actual (possibly off by default) warning of some kind, rather than a docs-only “soft” deprecation.

Right, very true, and we could always embed a abbr to provide a pop-up definition. On the other hand, at least for me “obsolescent” does in fact carry plenty of loaded meaning, if there’s already another word fairly well understood that has the meaning we want to send, such as “discouraged”, then maybe best to use that?

About projects CI failing because of new warnings, catching “soft deprecation” in linters, and not showing such warning by default: an option for that would by to only emit such warning when the Python Development Mode is explicitly turned on: Python Development Mode — Python 3.13.0a0 documentation I’m not talking about warnings filters, but emitting the warning or not.

Not sure this would totally solve the issue, though, since when testing in CI, unless the performance tradeoff is not acceptable, I (and various other projects) already run with -X dev anyway as part of the standard python -bb -X dev -W error invocation (maybe with -I thrown in).

So, users who used that but didn’t want it would need to add a warning filter anyway, and those who don’t but do want it won’t get it without the other performance, etc. tradeoffs of dev mode. And perhaps the most important population of users who linters and doc warnings would be least likely to reach but for the warning would perhaps be the most important, would mostly not get it either as they are typically not aware dev mode exists, and if they do not using it all the time.

So not sure how much of a net benefit it would be over a linter warning.

Soft deprecation / obsolescence can be useful for colorsys, part of the PEP 594 to-keep list. The rationale to keep is good enough (it really brings no maintenance overhead), but the module’s relevance is quickly reducing because:

  • CSS syntax is no longer limited to HSL & HSV. They’ve added LAB and OKLab, because the old two are just bad at what they do.
  • The current see also box is quite hard to follow for colorsys users because they are actually sound color advice, and sound color advice just don’t do YIQ, HSV, or HSL. But we can’t quite say “don’t use this in new code but it’s totally not deprecated” in the doc, because that’s even more confusing.

You’d really want a way to tell users “we will not work on it any more” here. Like, “don’t use it not because we are going to remove it, but because we care about everyone’s sanity.”

Examples of other projects using the expression “soft deprecation”:

  • Swift: deprecated: 10000 means “soft-deprecated”, i.e. it’s not deprecated now and will continue to function, but it’s not recommended.
  • R Project: gentler form of deprecation designed to prevent new uses of a function and encourage package developers to move away from it. Soft deprecated allows a package to change its interface to encourage package developers to update their code before their users are forced to change.
  • Elixir: “Elixir deprecations happen in 3 steps: (1) The feature is soft-deprecated. It means both CHANGELOG and documentation must list the feature as deprecated but no warning is effectively emitted by running the code. There is no requirement to soft-deprecate a feature.”
  • MediaWiki. This one sounds more aggressive: “Developers or teams deprecating code SHOULD remove usages in Wikimedia maintained code as soon as possible.”
  • Mention in PHP: Soft-deprecation notices for ext/mysql
  • PHP Elementor

I created PEP 387: Add Soft Deprecation section by vstinner · Pull Request #3182 · python/peps · GitHub to define Soft Deprecation in PEP 387.

We already made the choice to hide deprecation warnings by default. It would be very weird to make the opposite choice for soft deprecation warnings.

In my PEP change (PR), I propose to:

  1. Add a new SoftDeprecationWarning warning class
  2. Require soft deprecated functions to emit SoftDeprecationWarning
  3. But only emit SoftDeprecationWarning in the Python Development Mode and on Python built in debug mode

In short, SoftDeprecationWarning is hidden by default (even with -Werror or -Walways). Moreover, if a test suite is run with PYTHONDEVMODE=1 (or with a debug build of Python), the new SoftDeprecationWarning warning can be ignored explicitly.

I prefer to add a new SoftDeprecationWarning class, so developers can choose to take care of relevant DeprecationWarning, but ignore SoftDeprecationWarning when they are well aware of a soft deprecation and consider that it’s ok to leave code with soft deprecation unchanged.

2 Likes

Isn’t “soft deprecation” just “deprecation”?

It seems to me that confusing “deprecation” with “pending removal” is the problem.
Renaming “deprecation” to “soft deprecation” when no removal is planned seems unnecessary.

What we should do is make pending removals clearer.
If a removal is pending, then make that clear. Obviously, it should be deprecated as well.
Pending removal implies deprecation, not the other way around.

Maybe we just need to calm down with the removals. It is fine for something to stay for a few years.
java.lang.Thread.stop was deprecated for over 20 years without being removed.

If something is deprecated, that suggests that it shouldn’t be used. It doesn’t say that it has to be removed.

Deprecation Definition & Meaning - Merriam-Webster

5 Likes

See my first message which explains the rationale. In Python, “deprecation” is often associated with “scheduled removal”. I prefer a separated term for “removal is not planned soon or later: it’s not planned at all”. There is also the question of the DeprecationWarning warning: some deprecated functions emit DeprecationWarning, some others don’t. What does it mean? Is the feature really deprecated or not? IMO we need to name properly the two different cases, since it was misleading and caused eagger removal in the past.

That’s my second goal: mark clearly what “must no longer be used” (or just “is no longer recommended”) but make sure that no one is tempted to remove it soon or later. That’s why I want to better specify the change from “soft deprecation” (no removal scheduled) to “hard deprecated with a scheduled removal”.

Currently, the only difference is deprecated vs deprecated-removal in the documentation. For example, both use the same warning category. PendingDeprecationWarning is not helpful here.

2 Likes

Sorry, I should have been clearer. My point is having no way for code to tell me it’s “soft deprecated” is worrisome, not that you need e.g. -W error to turn on warnings.

1 Like

I propose to ignore soft deprecation warnings by default, even with -Wdefault and -Werror. Enable the Python Development Mode (python3 -X dev or set PYTHONDEVMODE=1 env var) to display soft deprecation warnings. Moreover, a new SoftDeprecationWarning warning category is added to Python 3.13 to allow ignoring these warnings with filters (ex: -Wignore::SoftDeprecationWarning cmdline option).

1 Like

Should SoftDeprecationWarning warnings be displayed by default in the __main__ module, as done for the DeprecationWarning warnings?

This is adding a lot of special-casing and cognitive overhead for developers. We would now have an entire array of different warning activation mechanisms to remember about.

3 Likes

Perhaps we should reboot this discussion, since the terminology is confusing to say the least.

Sketching a possibly better solution:

First, we should clearly define “deprecation” as what it is: we believe there are better alternatives to this API/module and don’t intend to put a lot of effort into its future maintenance; please consider upgrading to XYZ. Such a “deprecation” has no explicit or implied connection to removals or changes.

Since there’s no need to have a pending deprecation state (there’s nothing immediately actionable, since this is only a suggestion by the core devs), we can drop PendingDeprecationWarning and only use DeprecationWarning.

Next, we should clearly separate “removals” from “deprecations”. If there is an intent to remove an API/module, this needs to be called a “removal”, since that’s what’s ultimately going to happen.

Accordingly, we would need to use PendingRemovalWarning and RemovalWarning before the API/module is finally removed in a subsequent release.

Some of the current warnings would have to be changed to removal warnings.

10 Likes

One reservation I have with this is that (as I’ve said before) it would be a significant issue for pip if optparse were to start issuing warnings - we have users who test with -Werror and we will not in the near future be moving off optparse. How would optparse (and similar cases like getopt) be treated in your proposal?

Having said that, I agree that we’d be better removing the association between “deprecated” and “(scheduled for) removal”, rather than adding more flavours of deprecation.