Formalize the concept of "soft deprecation" (don't schedule removal) in PEP 387 "Backwards Compatibility Policy"

Is it something that linters should/could react to? So that the warnings appear to the maintainers instead of the end users of the applications or scripts. I must say I do not know what the current state is. Do linters maintain their own lists of deprecated things?

Speaking only for pip (but I imagine we’re similar to anyone else using a deprecated-but-won’t-go-away module), we don’t want anyone telling us optparse is deprecared. We know. There simply isn’t anything we need or want to do about it.

We can of course suppress the relevant linter warning, just like we can suppress a runtime warning. But just like a runtime warning, why should we have to?

Should someone decide to rip it out, then we’d want to know. And I’d hope that any such removal would be done on the same timescales as an actively maintained module like argparse would have been - if “soft deprecated” actually means “we can decide to rip this with no warning because you shouldn’t be using it”, that’s a very different promise than the current one, and I’d expect it to be communicated as such.

There’s a few things being brought up in this thread.


(A) The documentation issue: Yes I think it is worthwhile making it more clear what type of “deprecation” something is. Right now the term is overloaded with a lot of meanings, so adding clarity to differentiate between “no longer recommended” (APIs we don’t recommend people to use for many possible reasons, but don’t believe it is worth planning to remove) and “stable, but we’ll never remove it and are unlikely to fix bugs or enhance it in the future” (getopt, optparse) and “actual” deprecations where we’ve got a scheduled version in which we intend to make the thing go away (ala PEP-594).


(B) The (f)utility of PendingDeprecationWarning - this topic has come up before: PendingDeprecationWarning is really useful? - our current “wisdom” is basically “don’t bother using that warning, pretend it doesn’t exist”. (hat tip to the lets soft-deprecate PendingDeprecationWarning comment above, we effectively have).

A warning being emitted should always have a justifyable reason for the action it suggests taking to avoid future breakage becoming someones priority to fix in their code.

Having code in place to emit warnings at runtime about things we already know we’ll never break does a disservice to everyone and just cries :deprecated-wolf: when there is no reason for code to be changed. That erodes trust in warnings being meaningful.

So lets not promote the addition of warnings on soft deprecations. Just guide that any form of warning should only be emitted if acting on it clearly improves the future health and longevity of the code triggering it.

For example: We’ll never warn about getopt and optparse. Those are effectively long term stable/stale modules. It isn’t even clear that they should even be called “soft-deprecated”, more that “hey, there’s a fancier thing over here in argparse - but use whichever floats your boat”.

We obviously warn about specific planned date API removals (good riddance asynchat). And we may choose to warn on APIs that “work” but we consider harmful because they’re too easy to use wrong, especially where we have an alternate API with less sharp edges. We may never remove them, thus soft-deprecation, but peoples code is healthier by not using them.

7 Likes

Hello Victor,

I like the concept and your proposal, with the exception of the name. What is a “soft” deprecation? A deprecation that does not issue a warning? A drepecation that will be enforced after several releases, not a couple? A deprecation that doesn’t harm you if it falls on your foot? :stuck_out_tongue:

Let’s find a better name. What about “discouraged”?

Regards,

My concern with skipping any sort of warning is people who discover via the REPL or copy some code from the internet which is “soft” deprecated and then come wanting a fix or change and get upset we didn’t actively notify them that the thing they were using was abandoned code.

3 Likes

Thanks so much, Victor, for bringing this up and for the comprehensive summary! IMO, formalizing the concept of a “soft” deprecation and a consistent, machine-readable way to explicitly document that would be a major improvement to the status quo. It would also address not only my own specific concerns around the getopt deprecation, but also the more general ones expressed on python/cpython#92564 by allowing us to provide additional clarity and specificity with deprecations and their messaging to reduce user and tool uncertainty, but without requiring committing to a specific removal version for cases where it isn’t appropriate.

As a result of your previous PR, I’d actually been considering proposing an extension to the current directives to explicitly mark a deprecation as a “soft” deprecation, and thinking about it some more in light of your comments above, ISTM that the simplest way to implement that for both user and programmatic consumption would be extending our existing custom deprecated-removed directive to support one or more special string values for the second argument in place of the removal version.

For example, we could use .. deprecated-removed:: 3.14 notplanned <message> to signal a “soft” deprecation where the API is deprecated but there are no plans to remove it for the immediate future. This would automatically modify the message to read Deprecated since version X.Y, removal not planned: <message>, with not planned being a dotted-underline tooltip (:abbr:) providing additional clarification, e.g. There are no immediate plans to remove this, but it is no longer actively developed and should be avoided in new code.

We could also add a few other special values if needed, e.g. notdecided for things that may be removed but the removal timeline is not yet decided upon, with a similar clarifying tooltip.

For distinctions as to the reason something was deprecated, e.g. because it is superseded/no longer developed (optparse) as opposed to actively bad to use (PyObject_SetAttrString(obj, attr_name, NULL)), because it is a deprecated name alias, etc., I agree this would be valuable to be explicit about in the semantic markup (which could be both read programmatically and generate an appropriate, consistent message for humans).

While we could shove in more special values to the above, after some thought, it would be a lot cleaner, consistent with current usage and more generally useful to add a :reason: option, so it would be usable with both planned and unplanned removals. It would be set to one of a few option values (e.g., notdeveloped, unsafe, alias, obsolete, superseded, etc.), and so:

.. deprecated-removed:: 3.2 notplanned MESSAGE
   :reason: alias

could render as, for instance, “Deprecated since version 3.2, removal not planned (legacy alias): MESSAGE”, with legacy alias having an automatic mouseover text describing what that means in more detail.

It wouldn’t be hard to add a :replace: option too, so you could do

.. deprecated-removed:: 3.2 notplanned
   :reason: alias
   :replace: Logger.warning

And get automatically “Deprecated since version 3.2, removal not planned (legacy alias of Logger.warning)”. We could add others, too, which would be very helpful for automation, but that’s getting a bit off topic here.

If we’re going to emit a warning where we aren’t already for these types of things, IMO it shouldn’t be the existing, established PendingDeprecationWarning (which implies it will be removed at some point in the forseeable future) but rather something new that clearly communicates the intent, and that devs can easily individually enable/disable if they want it

As for whether there should be some kind of warning in the code, I don’t have particularly strong opinion either way. It would be nice to give developers some form of runtime indication, but it could of course be noisy enough that it is more of a net nuisance than a help. Perhaps it should depend on the type of deprecation like those @gpshead mentioned — the relative cost-benefit of warning seems a lot better for “not recommended APIs” vs. just “superseded and no longer actively developed” ones, with things like name aliases falling in between—they are easier for users to find and fix than either, but are also less directly harmful.

With appropriate docs markup, though, linters could handle the warnings, which are more dev-focused, “softer” and easier to silence than warnings. I’m (slowly) working on some tooling that would allow parsing deprecation and removals out of the docs programmatically (as well as other things like the APIs were added, and other structured metadata), and in turn could then be used to detect (or in some cases even autofix) these issues. It’s not perfect, but it might be a workable middle ground between a runtime warning and a docs message (which unfortunately most devs will probably never read directly), and the proposed docs markup will substantially aid that.

From my point of view deprecations in general should be caught by linters, rather than by the Python interpreter at run-time. That is for the general case, and the other hand I am sure there are plenty of cases where we want the Python interpreter itself to react to deprecations.

It feels to me like it is important to get in touch with authors of linting tools to see what would make sense. I guess one important feature here is that deprecations should be easy to filter out (or in) individually or in sub-groups, over the whole project level or on a case-by-case basis per line of code.

For catching deprecations in linters, let me just link to my PEP 702 – Marking deprecations using the type system | peps.python.org, which is currently pending before the Steering Council.

2 Likes

For developers that rely more on unit testing than on static analysis [1], some kind of run-time warning of deprecations is important (even if it’s disabled by default, like DeprecationWarning is).


  1. In my modest experience, that’s a whole lot of people. ↩︎

3 Likes

Apparently, the “soft deprecation” expression is bad since it’s too close to “regular hard deprecation”, and usually “deprecation” is associated with a short-term or long-term plan to remove a feature. Is there a better term to say “you should not use this function but there is no plan to remove it”?

It seems like this concept should be dissociated from deprecations, so reusing the existing PendingDeprecationWaring is a bad idea since PendingDeprecationWaring means indirectly “pending removal” which is not the case here.

1 Like

Abandoned?

A possibility would be: “This function is obsolete” or perhaps “obsolescent”. The latter is much less common, and means “Being in the process of passing out of use or usefulness; becoming obsolete.”; it’s used by the C standard for example, which says:

3 Likes

Hmm, I guess the issue with “obsolescent” though is it still implies that the feature was once useful, but no longer is, and specifically per that definition that it might be considered for removal in future versions…which sounds a lot like what a PendingDeprecationWarning was supposed to be. And on the other hand, it puts the emphasis on it being old, no longer useful and considered for removal, as opposed to being discouraged for existing and especially new use, which seems to be more the message that we want to send here in most of these cases.

As such, to more imply the latter, what about calling this state “Discouraged” instead?

3 Likes

I don’t think this should apply to “soft deprecations”, where it’s OK to use the old code – it has some surprising behaviour, or is missing support for some new features, or doesn’t follow the latest naming conventions – generally it’s just no longer the best way to do something.
When changing the code that uses “soft-deprecated” API, you’re likely to hit its limitations, and you should consider moving away from it. But otherwise, there’s no reason to do that. Your time is much better spent on something else.

3 Likes

FWIW, I think obsolescent is too obscure of a word, I just mentioned it because it’s used by another large/old language in this context, but I personally think “obsolete” would be fine (and obviously we don’t have to take over the wording from the C standard, but can come up with something appropriate for us).

Its definition doesn’t even imply removal; obsolete:

  1. out of use or practice; not current
  2. out of date; unfashionable or outmoded

Personally, I think that it’s a more accurate word than discouraged, but that would also work.

2 Likes

Obscurity of a word can sometimes be an advantage though. If people don’t know what a word means, they’re more open to looking up how it’s being used, instead of thinking they already understand it. A lot of the issues here stem from preconceptions of terms like “deprecated” and what they imply, and no matter how much you document something, people won’t always interpret the words the same way you do. (Though sometimes you can completely dodge the issue by assigning completely arbitrary meaning to words, like talking about “blue” functions and then using a colour highlight on them, but then you lose ALL intuition, so that’s less effective in another way.)

1 Like

IMO, familiar or not, “obsolete” has still has the same fundamental problem I raised with “obsolescent” above:

And on the other hand, it puts the emphasis on it being old, no longer useful and considered for removal, as opposed to being discouraged for existing and especially new use, which seems to be more the message that we want to send here in most of these cases.

At least in my thinking here, I’m most concerned about sending a message for cases, like PyObject_SetAttrString(obj, attr_name, NULL), or, say `tempfile.mktemp, where something is actively not recommended, as opposed to simply being an old alias or not merely the new hotness. Though, arguably, those cases might be most deserving of an actual (possibly off by default) warning of some kind, rather than a docs-only “soft” deprecation.

Right, very true, and we could always embed a abbr to provide a pop-up definition. On the other hand, at least for me “obsolescent” does in fact carry plenty of loaded meaning, if there’s already another word fairly well understood that has the meaning we want to send, such as “discouraged”, then maybe best to use that?

About projects CI failing because of new warnings, catching “soft deprecation” in linters, and not showing such warning by default: an option for that would by to only emit such warning when the Python Development Mode is explicitly turned on: Python Development Mode — Python 3.13.0a0 documentation I’m not talking about warnings filters, but emitting the warning or not.

Not sure this would totally solve the issue, though, since when testing in CI, unless the performance tradeoff is not acceptable, I (and various other projects) already run with -X dev anyway as part of the standard python -bb -X dev -W error invocation (maybe with -I thrown in).

So, users who used that but didn’t want it would need to add a warning filter anyway, and those who don’t but do want it won’t get it without the other performance, etc. tradeoffs of dev mode. And perhaps the most important population of users who linters and doc warnings would be least likely to reach but for the warning would perhaps be the most important, would mostly not get it either as they are typically not aware dev mode exists, and if they do not using it all the time.

So not sure how much of a net benefit it would be over a linter warning.

Soft deprecation / obsolescence can be useful for colorsys, part of the PEP 594 to-keep list. The rationale to keep is good enough (it really brings no maintenance overhead), but the module’s relevance is quickly reducing because:

  • CSS syntax is no longer limited to HSL & HSV. They’ve added LAB and OKLab, because the old two are just bad at what they do.
  • The current see also box is quite hard to follow for colorsys users because they are actually sound color advice, and sound color advice just don’t do YIQ, HSV, or HSL. But we can’t quite say “don’t use this in new code but it’s totally not deprecated” in the doc, because that’s even more confusing.

You’d really want a way to tell users “we will not work on it any more” here. Like, “don’t use it not because we are going to remove it, but because we care about everyone’s sanity.”