Could we add multiple dispatch as a core concept without breaking backwards compatibility?

Ideas
1

I wrote this proposal with a friend (Clement Gehring) a few years ago, and I’ve been waiting for a good time to post ever since. With the recent discussion on multiple dispatch slowing down, I thought now might be a good time to finally post it.

We wrote it up in a collab here.

1 Like
2

The motivation here specifically mentions type-checking. I think type-checking tools should just get smarter here, however, given that there are non-typing motivations that have come up in other discussions, including the one you linked, that could really benefit from the addition of multiple dispatch to the language, I’m not opposed on those grounds.

I have a few notes, but, not much to add past that. I’m generally supportive of this as an option.

  1. I do think that the existing dunders that are shown in the examples here need to also remain supporting their current form where switching on type compatibility is done in function.

  2. This does not resolve the case for LSP violations on object.__eq__ alone, and that object.__eq__'s type may still require adjusting in the typeshed.

  3. Having two decorators here may be more helpful. naming not considered, but in the same vein as final, positional only parameters, etc. being able to explicitly decorate in two ways, one that says “do not allow extending to other types in subclasses”, and one that does, without being as strict as final is. The one that allows extending to other types should be considered by type checkers as never constituting an LSP violation so long as dispatch is unambiguous.

  4. This blurs the lines on typing having runtime behavior in the standard library. I’m not strictly opposed as the first point leaves a non typing-based option available, but I’m hesitant on it as a result. The optional nature of typing currently provides a bit of a buffer on the fact that the type system cannot express every type. Currently, not even dataclasses actually uses the type information, this would.

with respect to your question on performance, copied here for direct reference:

Someone more familiar with CPython’s internals would be able to comment more intelligently about performance.

With multiple dispatch, the dispatch resolution can be cached the first time a new type-method combination is called. The cache would have to be cleared if any new dispatch functions are declared for the given method. Subsequent calls do a cache lookup to resolve the method.

In current Python, any complex switching on types in Python code has to be done on every call due to Python’s dynamic nature. This might be slower than the multiple dispatch with caching. When there’s no switching or type-checking in a binary operator, then the current Python system might be faster.

The work on improving interpreter performance could utilize this, but doing this would heavily skew typing as required, and I’m less favorable to this personally.

On the ergonomic side, the linked julia talk makes the case better than I can, and I enjoy this feature from julia personally.

3

The examples as shown all use type hints but they are based on the multipledispatch library which itself does not use type hints. Actual use of multipledispatch as it stands looks like:

@dispatch(int, str)
def f(x, s):
    # do stuff

It is nicer to use annotations e.g.

@dispatch
def f(x: int, s: str):
    # do stuff

but this is not essential.

1 Like