PEP 702: Marking deprecations using the type system

I present PEP 702, which proposes to add an @typing.deprecated decorator that provides a way to communicate to static type checkers about deprecated functionality.

Let me know if you have any thoughts on this proposal.

There is currently one open issue: Should the decorator raise a runtime warning? I propose that it should not, but there are some strong arguments for doing this. I would be interested to hear more opinions. Previous discussion is in typing-sig.

6 Likes

The mailman archive seems incomplete. It doesn’t contain @sobolevn’s reply (and my subsequent, but a bit inconsequential reply). Maybe there are some issues with non-ASCII characters?

Notably, it does not issue a runtime DeprecationWarning.

Can it be managed by an interpreter cmdline option instead? I believe piggybacking -d or adding -W deprecated can be the fitting one.

It would benefit CPython itself replacing older warnings.warn() and newer warnings._deprecated().

Edit: whether we should hide stdlib deprecations by default like it’s proposed for the third party libraries, or do the opposite and always issue the warning for both, or leave these parts as is (non-uniform) is a subject of a separate discussion.

Warnings filter settings (which can also be altered with -W flag and PYTHONWARNINGS env var) already determine whether particular warnings are shown. If typing.deprecated raised warnings, why should there be a separate option for determining whether those specific warnings are shown? Note that this can be achieved simply by raising a subclass of DeprecationWarning as then you can filter based on that but I’m unsure what would make warnings from typing.deprecated so special.

I followed the discussion on typing-sig and I can see some merit to having an opt-in way to raise a warning without having to manually call warnings.warn but I’m not convinced that it warrants the increase in complexity. It would require adding 2 or 3 more arguments - an argument for determining whether a warning should be raised (I’m more for opt-in than for opt-out), stacklevel (I have no clue what the default should be here, Python documentation suggests using stacklevel=2 for wrapper functions but that makes the warning point to the code of the function that is deprecated rather than the code that called that function; OT but IMO the current documentation, in addition to bad default, suggests an anti-pattern that makes it hard to pinpoint what uses the deprecated thing), and maybe category (to specify subclass of DeprecationWarning).

Additionally, it still leaves the inconsistencies that we’re bound to have when it comes to overload signatures where we can’t raise the warning since it would require verifying types at runtime.

I personally believe that @deprecated should have the option to issue a deprecation warning and should be placed in the warnings module. I suspect that even if it won’t have such an option, IDEs, linters, etc. - and not only type checkers - will start to rely on the decorator. But users will want to issue warnings at runtime. This means either using multiple decorators or custom decorators. The latter won’t necessarily be understood by tools, which could lead to monkey-patching of typing.deprecated. I believe just supporting warnings from the beginning would prevent all those potential issues. (Try to) do it right now instead of having to change it later.

11 Likes

The no-warning behavior in the PEP makes me uneasy.

The PEP’s boilerplate code looks reasonable to me and points out that it could be encapsulated in a library, but I don’t find the arguments against doing this in typing very compelling:

  • The point about generally avoiding runtime work is understandable, but I don’t think it aligns with the actual behavior of the library, e.g. assert_never() which rhymes with the proposed decorator in terms of use cases and has a much more drastic runtime effect than a warning would. On the other hand, I am much less familiar with the machinery than Jelle is, maybe I am missing some nuance here.

  • The points about edge cases don’t seem to be represented in the example boilerplate, so I think the ‘real’ idiom is more complicated than the PEP is letting on, and I think that’s a point in favor of making it the standard library’s problem.

    • Maybe a wider survey of existing ‘in the wild’ implementations of this decorator is in order? For example, the Deprecated library referenced in the Rationale section has a more complex implementation, although not all of that is functionality that would be obligatory for an arbitrary end-user.

Overall, I very much like the idea of an official way to declare something as deprecated, but not being able to opt into warnings (I’d prefer having to opt out of a warning but I figure that’s a bridge too far here :sweat_smile:) seems to me like missing a trick and sticking the end user with the bill for the resulting complexity of there being two not-quite-equivalent Ways to Do It.

As someone who works on libraries without static type checking, I am uncomfortable with the absence of runtime effects of this PEP. It creates a situation where I can miss a deprecation because it is only communicated via typing.deprecated.

By way of example, not proposal, similar to __all__, there could be

__deprecated__ = ["Class", "function", "CONSTANT"]

making use of the deprecated items warn at import time. There are problems with this under this form (like the question of how to make it work for from … import *) but you see the general idea. I think there are ways to communicate deprecations in a way that botj type checkers and the runtime will understand, and I would like to invite exploring them.

Based on the feedback here I’m going to change the decorator to provide a warning, with an opt-out mechanism. I’ll be back soon with details.

2 Likes

There’s a lot that’s missing from this that would make me unlikely to use it in Flask.

I try to be as specific and helpful as possible in my deprecation messages, because otherwise (and regardless) I have to deal with more user reports. I use some common message patterns, like:

  • '{name}' is deprecated and will be removed in Flask {version}. Use '{other}' instead.
  • The '{name}' parameter is deprecated and will be removed in Werkzeug {version}. It's always enabled now.
  • '{name}' has been renamed to '{other}'. The old name is deprecated and will be removed in Jinja {version}.

I write mine manually for my specific situations, but SQLAlchemy has a whole set of decorators that can handle all different types of deprecations, check call arguments, and show specific messages at runtime and in documentation. sqlalchemy/deprecations.py at 586df197615d91af56aefc0d5ff94ceac13154eb · sqlalchemy/sqlalchemy · GitHub

Despite the stats you have, I do remove or rename arguments and attributes/properties, and move things between modules, often enough. I don’t plan to do it a lot, because it’s disruptive, but it still happens. Perhaps your stats are only considering the latest versions, whereas if you checked a few versions ago you’d see a lot more of them (they’ve been fully removed at this point).

The stack level can’t always be assumed to be 2. I’d need to go back and look, but I remember using 3 for base classes where you want to show the warning from __new__ or __init__ when a subclass inherit them.

We also have to consider imports, not only calls. I’ve run into cases where a downstream library re-exported a name we deprecated. They never saw the warning during tests because they never called it themselves, so users broke when we updated even though nothing changed for them. (Yes, they should pin their dependency tree.) Now that module-level __getattr__ exists, we can make sure that warnings happen at import. This also helps for moves, allowing the old name to still work with a warning pointing at the new name. I’d hope that something in the standard library would be able to warn on both import and use at runtime.

Finally, projects like Flask and SQLAlchemy already have a really hard time using typing correctly and keeping up with changes. Adding another decorator we have to use on top of all the work we already do for deprecations, and that users will complain about if we don’t use, isn’t really helping. What would help is basically including SQLAlchemy’s solution, or another solution, for full control over messaging about changing APIs.

Thanks for your feedback!

Adding a complex mechanism, like the SQLAlchemy code you link, to the standard library is risky because our backward compatibility constraints are such that we basically have to get it right the first time. In addition, I don’t have an appetite for the amount of consensus-building that would be required to come up with a more complex API. If someone else is willing to do it, I won’t stop them.

My stats are based only on the standard library, which may not be representative. I do acknowledge that deprecations for the things you mention occur and it would be useful to mark them, but I would like to defer that because it introduces significant additional complexity.

I plan to provide a stacklevel= parameter to override the default. I will look out for edge cases like you mention when deciding on the default; sounds like it may have to be different for classes and functions.

When I presented a first version of this proposal at the typing-sig meeting, I suggested a “deprecated_transform” mechanism (similar to PEP 681’s dataclass_transform) that would mark a third-party decorator as working like typing.deprecated. I took it out because the feedback was that it would introduce too much complexity, but it may be worth reconsidering. Would that alleviate your concern?

1 Like

Also note that gh-39615: Add warnings.warn() skip_file_prefixes support by gpshead · Pull Request #100840 · python/cpython · GitHub just landed which lets you skip frames based on file prefix.

Very nice. I’d consider accepting only skip_file_prefixes for @deprecated, except that then I’d have to backport the feature to typing-extensions. I’ll see if that is feasible.

Hi, I’m currently working on a similar library.

The goal of codecrumbs is not only to provide deprecation messages and extend the docstring, but also to offer a way to fix the code which is using the deprecated apis.

It supports currently renaming of arguments and attributes, but the deprecation and replacing of functions is also planned (but more complicated). The library is also in an very early stage and is not advertised very much. But I would be happy about any feedback or beta tester :slightly_smiling_face:

It currently works by inspecting the code during runtime, but using static typing is maybe also a way this could be implemented.
The reason I did not chose this approach is that there are different static type checkers and it would require that the code which is checked uses type annotations. The benefit would be that the user does not have to run the whole test suite to fix the deprecated code.
I also have no idea how it would be possible to hook into the type checker to provide the code fixes for the user. If anyone has an idea please let me know.

I see currently no way to combine my library with this PEP, but maybe this can serve as an inspiration for new ideas.