Using `@deprecated` for all overloads in stubs

Typing
,
1

The upcoming SciPy 1.18.0 release will deprecate the scipy.spatial.minkowski_distance function (and two similar other functions as well). In scipy-stubs, it’s defined using two overloads, and the deprecation applies to the function as a whole, i.e. all overloads are deprecated.

According to the typing-spec:

With overloaded functions, the decorator may be applied to individual overloads, indicating that the particular overload is deprecated. The decorator may also be applied to the overload implementation function, indicating that the entire function is deprecated.

These are stubs we’re talking about, so there’s no overload implementation function, leaving me no other option than to deprecate each overload:

@overload
@deprecated("This function is deprecated in favor of `scipy.spatial.distance.minkowski` and will be removed in SciPy 1.20.0.")
def minkowski_distance(x: onp.ToFloatND, y: onp.ToFloatND, p: float = 2.0) -> onp.ArrayND[np.float64]: ...
@overload
@deprecated("This function is deprecated in favor of `scipy.spatial.distance.minkowski` and will be removed in SciPy 1.20.0.")
def minkowski_distance(x: onp.ToComplexND, y: onp.ToComplexND, p: float = 2.0) -> onp.ArrayND[np.float64 | np.complex128]: ...

(source)

So could we amend the @deprecated typing spec to say something like "if @deprecated is placed before the first @overload of a function in a type-check-only context (i.e. in .pyi stubs, Protocol, or if TYPE_CHECKING:), it applies to all overloads of that function"?

That way, we could simplify the example:

@deprecated("This function is deprecated in favor of `scipy.spatial.distance.minkowski` and will be removed in SciPy 1.20.0.")
@overload
def minkowski_distance(x: onp.ToFloatND, y: onp.ToFloatND, p: float = 2.0) -> onp.ArrayND[np.float64]: ...
@overload
def minkowski_distance(x: onp.ToComplexND, y: onp.ToComplexND, p: float = 2.0) -> onp.ArrayND[np.float64 | np.complex128]: ...
2 Likes
2

Here’s another recent example with 5 overloads:

@overload
@deprecated("`tsearch` is deprecated in favor of `Delaunay.find_simplex` and will be removed in SciPy 1.22.0.")
def tsearch(tri: Delaunay, xi: _ToArrayStrictND) -> onp.ArrayND[np.int32]: ...
@overload
@deprecated("`tsearch` is deprecated in favor of `Delaunay.find_simplex` and will be removed in SciPy 1.22.0.")
def tsearch(tri: Delaunay, xi: onp.ToFloatStrict1D) -> onp.Array0D[np.int32]: ...
@overload
@deprecated("`tsearch` is deprecated in favor of `Delaunay.find_simplex` and will be removed in SciPy 1.22.0.")
def tsearch(tri: Delaunay, xi: onp.ToFloatStrict2D) -> onp.Array1D[np.int32]: ...
@overload
@deprecated("`tsearch` is deprecated in favor of `Delaunay.find_simplex` and will be removed in SciPy 1.22.0.")
def tsearch(tri: Delaunay, xi: onp.ToFloatStrict3D) -> onp.Array2D[np.int32]: ...
@overload
@deprecated("`tsearch` is deprecated in favor of `Delaunay.find_simplex` and will be removed in SciPy 1.22.0.")
def tsearch(tri: Delaunay, xi: onp.ToFloatND) -> onp.ArrayND[np.int32]: ...

source

3

To me this is clearly a good idea from the motivating examples, but I’d hope that there’s a clearer way to spell this?

I appreciate that stubs are often (mostly?) read by type checkers, but I tend to read a lot of stubs myself (e.g. to understand the surface of a module) and would find this rather confusing as to what the @deprecated applies to.

I’ve never read code and thought that the order of decorators was significant, as there is often only one permutation of decorators that works at runtime, so it’s merely a practical detail. I’m sure someone will have an example of where the order is already used to change the meaning, it’s just not something I see often.

4

Instead of using subtle positioning, maybe we could add some additional rules to make the repeats less cumbersome. For instance allow using a Final variable for the message, or specify that an empty string can be used for subsequent decorators to mean the first should be used for each value. Perhaps an ellipsis would also make sense, but that’d need runtime changes.

5

Perhaps something generic but not valid by itself could be added to represent either the overload implementation function or equivalently all overloads.

@deprecated("....")
def minkowski_distance
@overload
def minkowski_distance(...)...
...
6

Another idea would be to use a contextmanager, which has the additional benefit of being able to deprecate multiple things at the same time, and would also make it possible to deprecate constants, type aliases, and other such things:

with deprecated("Will be removed in Python 4.0"):
    from collections.abc import Container as Container  # deprecated re-export

    AnyStr = TypeVar("AnyStr", str, bytes)

    type Text = str

    def no_type_check_decorator(decorator): ...

That’d be require a bit more work than just a typing-spec change though.

2 Likes
7

What if you add a dummy implementation function with @deprecated to the overload in the stub? Haven’t checked if type checkers support that, but it would feel natural.

2 Likes
8

Perhaps a special argument like all=True would be all that’s needed?

If the decorator was to be applied to the first overload only, and that would mean that all following overloads are to be deprecated as well, that would leave ambiguity when trying to only deprecate the first overload. Overloads are order dependant, so there wouldn’t really be a way to move the overload if the first overload only should be deprecated. And if such a change was made in the spec, all following overloads that weren’t deprecated would be now, and ambiguity would arise for a non-deprecated first overload, where e.g. the second is deprecated. Should the third, fourth, … be deprecated too now?

That’s why either the idea of Jelle works here, or one could just add a all Boolean with default being False. Then we could do

@deprecated("...", all=True)
@overload
def fn(...) -> ...: ... # Deprecated 
@overload
def fn(...) -> ...: ... # Deprecated

# OR

@deprecated("...", all=False) # Default: False
@overload
def fn2(...) -> ...: ... # Deprecated
@overload
def fn2(...) -> ...: ... # NOT Deprecated

Same would obviously happen for some @deprecated("...", all=True) for e.g. the second overload, where all following overloads are deprecated too, but the first is not.

3 Likes
9

That hack seems to work, but it doesn’t solve the underlying problem of unnecessary code duplication:

@overload
def minkowski_distance(x: onp.ToFloatND, y: onp.ToFloatND, p: float = 2.0) -> onp.ArrayND[np.float64]: ...
@overload
def minkowski_distance(x: onp.ToComplexND, y: onp.ToComplexND, p: float = 2.0) -> onp.ArrayND[np.float64 | np.complex128]: ...
@deprecated("This function is deprecated in favor of `scipy.spatial.distance.minkowski` and will be removed in SciPy 1.20.0.")
def minkowski_distance(x: onp.ToComplexND, y: onp.ToComplexND, p: float = 2.0) -> onp.ArrayND[np.float64 | np.complex128]: ...

And as someone who spends most of hist time in stubs, I actually think it feels pretty unnatural (but maybe that’s just me).

10

You don’t need to repeat the signature besides the function’s name, do you?

@deprecated("...")
def minkowski_distance() -> ...: ...
11

The type-checker army protests when I do this though

12

Hmm, pyright is not giving me any errors when I do that inside a .pyi file. It does complain within a .py file though:

# ruff: noqa
# pyright: reportInconsistentOverload=true

from typing import overload, NewType

from warnings import deprecated

ToFloatND = NewType("ToFloatND", float)
ToComplexND = NewType("ToComplexND", complex)

class onp:
    type ToFloatND = ToFloatND
    type ToComplexND = ToComplexND
    type ArrayND[T] = T

class np:
    type float64 = object
    type complex128 = object


@overload
def minkowski_distance(
    x: onp.ToFloatND, y: onp.ToFloatND, p: float = 2.0
) -> onp.ArrayND[np.float64]: ...
@overload
def minkowski_distance(
    x: onp.ToComplexND, y: onp.ToComplexND, p: float = 2.0
) -> onp.ArrayND[np.float64 | np.complex128]: ...
@deprecated(
    "This function is deprecated in favor of `scipy.spatial.distance.minkowski` and will be removed in SciPy 1.20.0."
)
def minkowski_distance() -> ...: ...

image

image
image1031×319 47 KB

image
image1105×134 17.9 KB

13

I’m guessing that’s a config thing then:

Screenshot2026-05-3012-04-09
Screenshot2026-05-3012-04-091138×379 66.6 KB

14

Stubtest also doesn’t seem to allow this:

error: not checking stubs due to mypy build errors:
.venv/lib/python3.14/site-packages/scipy-stubs/spatial/_kdtree.pyi:120: error: An implementation for an overloaded function is not allowed in a stub file  [misc]
15

You didn’t provide a return annotation, and that seems to be the only complaint?
-> ... works in pyright, and you can always do -> object

16

scipy-stubs currently uses mypy, pyright, basedpyright, pyrefly, ty, ruff, and stubdefaulter for validating the stubs. Zuban will hopefully also soon be added to that list.

Adding the suggested fake overload implementation leads to errors being reported by mypy, pyright, basedpyright, and pyrefly.

So I don’t consider this to be a viable option.

1 Like
17

Taking a step backwards for a moment, it seems to me that the real problem here is that it’s unnecessarily complicated to deprecate an overloaded function declaration.

Even if there is a way of getting the current crop of type checkers to do the right thing, do we really want it to be this hard for users to do something that feels pretty natural to me? In the spirit of Guido’s keynote at the typing summit, shouldn’t we be thinking about making the user experience better, not trying to find workarounds?

4 Likes
18

Sure, we may need to change the spec here. But before we can do that, we need to identify the best way to express this behavior. What do you think that would be?

3 Likes
19

@Jelle I still think that my initial proposal would be the easiest and cleanest solution here. There’s a lot going on the example in my original post, so I’ll try again to illustrate what I mean using a simpler example:

@deprecated("f is deprecated")  # deprecates all overloads
@overload
def f(x: int) -> int: ...
@overload
def f(x: str) -> str: ...

I propose that the above example (which the typing spec does not allow at the moment) will be treated as equivalent to the following code:

@overload
@deprecated("f is deprecated")  # deprecates only f(x: int)
def f(x: int) -> int: ...
@overload
@deprecated("f is deprecated")  # deprecates only f(x: str)
def f(x: str) -> str: ...

There is no ambiguity: it is still possible to deprecate individual overloads:

@overload
@deprecated("f is deprecated for integer input")  # deprecates only f(x: int)
def f(x: int) -> int: ...
@overload
def f(x: str) -> str: ...

So the idea is that @overload is the marker that says “this def is one of the ways this function definition can be used”. So:

  • a decorator placed outside that marker (above the first @overload) scopes to the whole function definition
  • a decorator placed inside it (below a specific @overload) scopes to that single overload

This isn’t a new idea either: @final and @override already work this way (spec):

If a @final or @override decorator is supplied for a function with overloads, the decorator should be applied only to the overload implementation if it is present. If an overload implementation isn’t present (for example, in a stub file), the @final or @override decorator should be applied only to the first overload.

So I propose that we also allow using @deprecated in this way if the overload implementation isn’t present, without changing anything about the current per-overload @deprecated usage rules.

2 Likes
20

My focus is on matching user expectations, and I can only really go off what my expectations are. For me, the key ideas are:

  • This is a fairly simple concept, so the way it’s expressed should also be simple.
  • Stubs are basically just a way of storing type information separately from the implementation, so they should look as similar to the implementation as possible.
  • The way this is expressed in normal Python code (attach the deprecated operator to the thing being deprecated - either one of the overloads, or the implementation itself) seems logical and easy to understand.

With that in mind, I think we should be allowing a dummy implementation definition in the stub file, and deprecating that:

@overload
def f(x: int) -> int: ...
@overload
def f(x: str) -> str: ...
@deprecated("f is deprecated") 
def f(): ...
3 Likes