An idea to allow implicit return of NamedTuples

In the current versions of Python, there is a convenient way to implicitly return a tuple using commas or a parenthesis-based syntax. For example:

def fx():
   return 1, 2

and

def fx():
    return (1, 2)

Both of these examples return a tuple containing multiple values. However, it would be even more convenient if we could return named tuples to allow for easier access using dot syntax, similar to what data classes provide.

For instance, it would be great if we could do something like this:

def fx():
    return name1=1, name2=2

or

def fx():
    return (
            name1=1,
            name2=2
       )

These improvements would allow us to return named tuples directly from functions, eliminating the need to explicitly create named tuples separately.

I am suggesting this improvement on the Python discourse platform as I believe it could enhance code readability and provide a more elegant way to access values. By sharing this idea and discussing it with the Python community, we can contribute to the ongoing development and improvement of the Python language.

3 Likes

In the current versions of Python, there is a convenient way to implicitly return a tuple using commas or a parenthesis-based syntax.

Can you please clarify what you mean by “implicitly return”? To me it looks like the tuples in the examples you gave are explicitly returned, so I think I’m misunderstanding what you’re saying.


Thanks

2 Likes

My understanding of this is that they are implicit namedtuple classes. This would probably be a better fit for something like types.SimpleNamespace, though, since each one doesn’t need a dedicated class.

But really, this isn’t about function returns - it’s about having a literal (or, more technically, display) syntax for a namedtuple. I’m not sure we need one, but it’s an interesting idea.

1 Like

I personally would be against it because namedtuples give you a dual API of indexes and attributes when typically people really only wanted one of the two and found it more convenient to use a namedtuple than to choose one over the other (i.e. the “I want a frozen dict” argument).

3 Likes

Can you please clarify what you mean by “implicitly return”? To me it looks like the tuples in the examples you gave are explicitly returned, so I think I’m misunderstanding what you’re saying.

Thank you for your feedback! When I mentioned “implicitly return”, I was referring to the syntax’s simplicity rather than the underlying mechanism. In Python, we can return multiple values using a comma-separated list without explicitly mentioning the Tuple class.

e.g.
You don’t need to return like the following

def fx():
    return Tuple(1, 2)

and can use the simpler syntax

I’m not very familiar with the types.SimpleNamespace type. However, after reading @brettcannon’s comment, I agree that I would also like an index-based access if possible.

I find this syntax proposal unintuitive for any return type that isn’t a built-in data type.

Function inputs use *args and **kwargs, where args are a tuple, and kwargs are a dictionary. In kwargs, you separate values with commas and use ‘=’ between keys and values, to me this similiarty of syntax would imply a dictionary return type, especially since it naturally unpacks with **, similar to how tuples naturally unpack with *.

Although personally, I don’t find myself having a strong need for this regardless of data type, it’s easy to write return dict(a=value1, b=value2).

As a newbie Python programmer I always thought multiple return values were very cool, with a bit more experience I consider now the real magic to be in unpacking of values.

4 Likes

Yep; this is nothing to do with returning from a function - what you’re seeing there is the tuple display syntax. You can do the same sort of thing in any other context. (Some people call this a “tuple literal”, although that’s not quite true.) You’re asking for a namedtuple display syntax here.

3 Likes

You’re asking for a namedtuple display syntax here.

Oh, alright. Thank you! :smiley:

Should open another issue with an appropriate title? Or should I edit the idea here only?

This one works; you can edit the title if you’d like, but it’s not strictly necessary. Just clarifying what the idea really is here.

1 Like

I was thinking about this in the past as well, often a namedtuple is much nicer than returning a plain tuple. In fact, I wrote a decorator that can automatically do that. However, the main issue is that currently, it doesn’t really play well with type-checking / autocompletion, so I am not actively using it…

It checks the AST of a function if all return node are tuples of equal length using equal variable names, and if so constructs a namedtuple with these names.


import ast
import textwrap
from collections.abc import Callable, Sequence
from functools import partial, wraps
from inspect import getsource
from types import GenericAlias
from typing import NamedTuple, Optional, ParamSpec

P = ParamSpec("P")


def get_exit_point_names(func: Callable) -> list[tuple[str, ...]]:
    """Return the variable names used in exit nodes."""
    source = textwrap.dedent(getsource(func))
    tree = ast.parse(source)
    exit_points = [node for node in ast.walk(tree) if isinstance(node, ast.Return)]

    var_names = []
    for exit_point in exit_points:
        assert isinstance(exit_point.value, ast.Tuple)

        e: tuple[str, ...] = ()
        for obj in exit_point.value.elts:
            assert isinstance(obj, ast.Name)
            e += (obj.id,)
        var_names.append(e)
    return var_names


def decorator(deco):
    """Decorator Factory."""

    @wraps(deco)
    def __decorator(__func__=None, **kwargs):
        if __func__ is None:
            return partial(deco, **kwargs)
        return deco(__func__, **kwargs)

    return __decorator


@decorator
def return_namedtuple(
    func: Callable[P, tuple],
    /,
    *,
    name: Optional[str] = None,
    field_names: Optional[Sequence[str]] = None,
) -> Callable[P, tuple]:
    """Convert a function's return type to a namedtuple."""
    name = f"{func.__name__}_tuple" if name is None else name

    # noinspection PyUnresolvedReferences
    return_type: GenericAlias = func.__annotations__.get("return", NotImplemented)
    if return_type is NotImplemented:
        raise ValueError("No return type hint found.")
    if not issubclass(return_type.__origin__, tuple):
        raise TypeError("Return type hint is not a tuple.")

    type_hints = return_type.__args__
    potential_return_names = set(get_exit_point_names(func))

    if len(type_hints) == 0:
        raise ValueError("Return type hint is an empty tuple.")
    if Ellipsis in type_hints:
        raise ValueError("Return type hint is a variable length tuple.")
    if field_names is None:
        if len(potential_return_names) != 1:
            raise ValueError("Automatic detection of names failed.")
        field_names = potential_return_names.pop()
    elif any(len(r) != len(type_hints) for r in potential_return_names):
        raise ValueError("Number of names does not match number of return values.")

    # create namedtuple
    tuple_type: type[tuple] = NamedTuple(name, zip(field_names, type_hints))  # type: ignore[misc]

    @wraps(func)
    def _wrapper(*func_args: P.args, **func_kwargs: P.kwargs) -> tuple:
        # noinspection PyCallingNonCallable
        return tuple_type(*func(*func_args, **func_kwargs))

    return _wrapper


def test_namedtuple_decorator():
    @return_namedtuple
    def foo(x: int, y: int) -> tuple[int, int]:
        q, r = divmod(x, y)
        return q, r

    assert str(foo(5, 3)) == "foo_tuple(q=1, r=2)"

    @return_namedtuple(name="divmod")
    def bar(x: int, y: int) -> tuple[int, int]:
        q, r = divmod(x, y)
        return q, r

    assert str(bar(5, 3)) == "divmod(q=1, r=2)"
1 Like

Honestly, sometimes you kind of just want both. You want indexing for the ability of tuple unpacking. At the same time, having the names can be incredibly helpful, as it is a form of self-documentation. When I run numpy.linalg.svd in an ipython session, I immediately see what the order of the results is and that the last matrix is transposed.

Thus, it can guide one to use correct tuple unpacking, without having to switch back and forth to documentation. I can just run numpy.linalg.svd(A), see that the result is a namedtuple of (U, S, Vh) and then modify the line to U, S, Vh = numpy.linalg.svd(A).

Imho, the best solution compared to what OP proposed, and the decorator I showed, would be if it were part of the typing system.

def svd(A: Array) -> tuple[U: Array, S: Array, Vh: Array]: ...

Where tuple[U: Array, S: Array, Vh: Array] could be a type-hinting format for “anonymous” namedtuples.

Don’t you think this would read better if you did:

s = np.linalg.svd(A)
# use s.U, s.S, s.Vh

I feel like the reason that svd returns a named tuple is for backwards compatibility with an old tuple interface. If we were designing the interface in modern Python (3.9+), I would probably prefer using a dataclass:

  • Iterability is more a source of potential bugs,
  • It’s easy to add members to the dataclass (which you can’t do with the named tuple since consumers are already decomposing it),
  • the dataclass is an ordinary class, so you can add member functions too, and
  • you can also control various things like adding or removing comparability, hashing, making it non-frozen, etc.

Cool decorator by the way!

I feel like these longer variable names would make complex computational formulas just more unreadable. I prefer when numerical code looks close to the pseudocode in the book/paper.

One could of course unpack by key instead, if iterability is a concern.

svd = np.linalg.svd(A)
U, S, Vh = svd.U, svd.S, svd.Vh

Unfortunately, there is no good one-liner for this.

1 Like

Somehow I had a false memory that there was an easy way to unpack a dataclass here, and now I’m wondering why there isn’t one.

It’s not pretty, but there is a one-liner:

preamble
from collections import namedtuple

import numpy as np

svd = namedtuple('svd', ['U', 'S', 'Vh'])

def mySVD(A):
    return svd(*np.linalg.svd(A))

A = np.eye(4)
from operator import attrgetter

U, S, Vh = attrgetter('U', 'S', 'Vh')(mySVD(A))
1 Like

Well you could just unpack the namedtuple, but this would work for a dataclass too. But yeah not so pretty and the necessary import stretches “one liner” a bit.

Oh, for dataclases, you do have:

U, S, Vh = dc.astuple(mySVD(A))
1 Like

I was working on a project when I thought of the idea. Your code snippet might work for me as a quick solution. Thank you!

I would still love this as a built-in behavior though.

Yes, but that wouldn’t be unpacking by key anymore, and is no safer than using a tuple to begin with.

1 Like