Snargs : triple-star packing/unpacking of self-named arguments (and alternative to PEP 736)

Ideas
1

I had this idea while reading the topics about PEP 736.
(Thus related : Syntactic sugar to encourage use of named arguments, PEP 736: Shorthand syntax for keyword arguments at invocation, Shorthand notation of dict literal and function call)
I realized these propositions are simply summarized as “self-named arguments packing”.

The creation of a dictionary with self named arguments could be done simply with a triple-star packing

x, y, z = 1, 2, 3
snargs = {***(x, y, z)}  # self-named arguments packing
# equivalent to >>> snargs = dict(x=x, y=y, z=z)
snargs  # {'x': 1, 'y': 2, 'z':3}

Then unpacking could be also defined in a convenient way, implicitly checking names consistency :

a, b, c = ***snargs  # KeyError: <dict> has no key 'a'

x, y, z = ***snargs  # self-named arguments unpacking
x, y, z  # (1, 2, 3)

This yields a convenient way of using functions with snargs:

def func(***snargs):
    x, y, z = ***snargs  # self-named arguments unpacking
    x, y, z = x + 1, x + z, 3*y + x
    return {***(x, y, z)}

func(***(x, y, z))  # {'x': 2, 'y':4, 'z': 7}

But actually, for further convenience, we might have an even more concise way of using it :

def func(***(x, y, z)):
    x, y, z = x + 1, x + z, 3*y + x
    return {***(x, y, z)}

new_x, new_y, new_z = ***func(***(x, y, z))  # KeyError : no key 'new_x'

x, y, z = ***func(***(x, y, z))
# Any unordered args will be reordered, any modified variable name will raise KeyError

Thus these two signatures would be equivalent

def func_with_default_y(x=x, y=2, z=z) : ...
def func_with_default_y(y=2, ***(x, z)) : ...

Allowing for clarification of functions with long-self-named arguments, for example :

def complicated_function(argument0,
                         argument_with_default_value = default_value,
                         **kwargs,
                         ***(my_long_argument_name,
                             my_other_long_argument_name,
                             another_one_again,
                             and_a_last_one_for_fun)):
    ...

CONS

  • Difficulty to implement in python ???
  • Complicated new syntax
  • Multiple ways for doing the same thing :
    x, y, z = ***func(***(x, y, z)) is equivalent to simply x, y, z = func(x=x, y=y, z=z), except for the pros given below.

PROS

  • Automatic check for modified names :
    Whenever someone would reimplement/refactor a code, if the way a variable is managed is modified, the user could just rename the variables and update some necessary functions, any function not modified subsequently, if using the snargs convention, would raise an error, thus informing the user the modification has not been applied within this function.
  • Reducing name redundancies (as was the initial purpose of PEP 736), also applicable for simple dict creation. Only one name has to be updated during refactoring, thus fool-proofing functions and dictionary instantiation update.

At the end, this proposition is not really improving “final codes”, but maintainability and refactorability of “temporary codes” might be easier by the fool-proofing properties of the snargs.
Possibly, nobody will want these triple-stars spawning everywhere in python code, as underlined by CON#3.

Thus I am not really convinced this idea will be accepted but I would like to open this discussion anyway… Opinions?

1 Like
2

Sorry, the comment I am about to make is off-topic, but your idea made me think of something else that is a mild annoyance in cases where I encounter it.

I’m not much bothered with this category of problems (i.e. reducing code duplication for self-named arguments). But if *** were to be introduced, I’d like it to be able to unpack mix of iterable and mapping, in that order. e.g.

# unpacking
>>> def func(a, b, c, /, *, d, e): print(locals())
>>> params = (1, 2, {"c": 3}, {"d": 4, "e": 5})
>>> func(***params)
{'a': 1, 'b': 2, 'c': {'c': 3}, 'd': 4, 'e': 5}

# packing
>>> def func(***params): print(params)
>>> func(1, 2, {"c": 3}, d=4, e=5)
(1, 2, {"c": 3}, {"d": 4, "e": 5})

The rule would be, if the last item in iterable is a mapping, it gets unpacked with **.

>>> *args, kwargs = params
>>> func(*args, **kwargs) if isinstance(kwargs, Mapping) else func(*args, kwargs)

I didn’t include the check for empty iterable unpacking, but that would have to be supported as well.

2 Likes
3

Another con is that there are alternatives how to use ***.

One of those that I think could be a good idea is:

def foo(***all_args):
    return all_args

all_args = foo(1, 2, a=3)
print(all_args)    # AllArgs((1, 2), {'a': 3}))
foo(***all_args)

This might not seem like a lot of value superficially, but it might have potential to simplify things and improve performance in various places (potentially including C-side of things).

I would be against anything else taking this syntax before the above is explored and is deemed worse than proposed alternative. And unfortunately, to do that, there is a fair bit of work and things to consider.

1 Like
4

Interesting approach.

While this looks appealing to me:

x = 0
y = 1

my_func(***(x, y)) # equivalent to my_func(x=x, y=y)

…the following does not:

def func(***snargs): ...
def func(***(x, y, z)): ...
def func_with_default_y(y=2, ***(x, z)): ...
# ...and the other more complex signatures

IIUC the goal of PEP 736 is to incite users to always pass arguments as keyword arguments, by making it easier. But this proposal seems to focus on another aspect: declaring functions with argument defaults that have the same name as the arguments :thinking:? I do not think this is a good idea. It makes signatures less informative (no type annotations?), and this use-case probably doesn’t appear often (contrary to passing kwargs which happens everywhere, all the time).

2 Likes
5

I actually do not think it a good idea either, and is not the focus I intend to (even if my examples possibly show that, I admit).

The real value I intend is more about “name checking for refactoring safety”.
Let say you have several functions of the same kind within a module

def func1(***(x, y, z)) :
     x = pre_process_x(x)
     return sum(x, y, z)

Now you refactor the module, you introduce an alternative pre_process_x_alt function that manages x, and you rename x to x_alt, because it is managed differently. Thus you update the function to

def func1(***(x_alt, y, z)) :
     x = pre_process_x_alt(x_alt)
     return sum(x, y, z)

and while the initial call was

result = func1(***(x, y, z))

you have to update it to

result = func1(***(x_alt, y, z))

The improvement here is that if you don’t consistently update the function/call/variable names, you get a KeyError, thus pointing you where you forgot to apply a necessary update during refactoring, instead of silently propagating the unproperly managed data.
The more numerous nested/parallel functions with common “snargs” you have, the more this “fool-proof safety” becomes convenient.

6

(You’re probably going to get a lot of, uhh, snargy comments about the name, but anyhow…)

I think this is a neat concept, but done like this, it makes “self-named” and “named” into completely different things in the same way that positional and named arguments are different things. (A parameter can be “positional or named”, but an argument can only ever be one or the other.) What exactly does your example function accept?

def func(***snargs):

What can I pass it? Any keyword arguments, but not positional? I think that’s what’s going on, but I’m very unsure. And what function signatures does your call match?

func(***(x, y, z))

Would it be valid to call def func(x, y, z) in that way? What about def func(*, x, y, z) ?

What exactly is a “self-named argument” in a function definition? Using another of your examples:

def complicated_function(argument0,
                         argument_with_default_value = default_value,
                         **kwargs,
                         ***(my_long_argument_name,
                             my_other_long_argument_name,
                             another_one_again,
                             and_a_last_one_for_fun)):
    ...

What exactly can you pass to this? It accepts any number of kwargs, and then also some… self-named? What are those? Are snargs compatible with kwargs or do we now have three entirely separate ways to pass arguments?

1 Like
7

Also, this seems like a lot of change with fairly little value, while the main issue can be addressed by introducing something similar to json object constructor:

const a = 'foo';
const b = 42;
const c = {};
const object3 = { a, b, c };
console.log(object3);
// Object { a: "foo", b: 42, c: Object {  } }

Of course, this is set’s definition so different syntax is needed, e.g.:

a, b, d = 1, 2, 4
d = {{a, b, 'c': 3, d}}
print(d)    # {'a': 1, 'b': 2, 'c': 3, 'd': 4}
# With or without possibility to mix in usual syntax

I believe this has been discussed in PEP 736: Shorthand syntax for keyword arguments at invocation and in other places, which I can not find now (there definitely were discussions about this specifically).

If this was done at dict level (as opposed to function call syntax), it would have wider benefits, while also providing shorthand for function calls. e.g.:

def foo(a, b, c):
    pass

a, b, c = 1, 2, 3
foo(**{{a, b, c}})

Maybe not as convenient as foo(a=,b=,c=), but not that much longer, while this way it has much more value with much less effort.

  1. Less effort:

    • Adding additional dict constructor syntax I would guess is a bit easier than function call modifications. At least from what I have seen, function argument logic is already quite involved.
    • PEP736 rejection reason1: “… complicating the grammar around function arguments …”

  2. More value:

    • Addresses general dict construction and not only function keywords.
    • PEP736 rejection reason2: “… new syntax does not have a high enough payoff to justify the added complexities …”

So yes, it wouldn’t be able to do this:

But:

  1. How often is this needed?
  2. This is quite a lot of dev work for replacing 2 lines with 1:
a, b, c = ***snargs
# vs
assert set(snargs) == {'a', 'b', 'c'}
a, b, c = snargs.values()

I think adapting JSs dict constructor variant is pretty much the best bet to address this, which from what I have seen is the main motivator for this direction in the first place.

4 Likes
8

I admit there are some undefined conventions in my proposal, that leaves degrees of freedom to get the most advantageous proposal.
I thought about it for some time, and I ended up with the idea that snargs and kwargs should be totally compatible / interchangeable. Any other way would become overly complicated.

The thing is the packing would implicitly assign the data to keys corresponding to the name of the data in the namespace where the packing is done. Reciprocally, the unpacking would assign the keys of the data to the same names in the namespace the unpacking is done. (At some point I thought it could use some namespace instead of dicts (thus unlike kwargs) but it felt complicated)

9

(Minor clarity nitpick: The dict constructor is dict(a=1, b=2, c=3) which is the function call syntax. What you’re talking about is called dict display and is the one that’s spelled {"a": 1, "b": 2, "c": 3} .)

The double brace syntax is reasonably elegant, but if you permit both syntaxes in it, the question then becomes, what’s the difference? It seems oddly magical that you have two nearly identical syntaxes {...} and {{...}} with nearly identical functionality, but one of them is a superset of the other.

Currently, {{a, b, c}} is valid syntax. At run time, it will bomb with TypeError, but it is valid syntax. Are you planning for {{ and }} to be single tokens (in the same way that <= is a single token, not < followed by =), or separate brace tokens? If they’re a new token, that means that {{a, b, c}} has one meaning and { {a, b, c} } has a quite different meaning; and { {a, b, c}} is probably a syntax error. OTOH, having them as separate tokens will mean that there’s very confusing parsing for something like { { a, b }, { c, d } } .

TBH I don’t think there’s room for Python to cram much more into display delimiters. It’s not easy to fit anything else into it. Even though function call syntax is already complicated, it’s likely easier to further augment that than to try to make more weird cases around braces.

3 Likes
10

That would mean that there’s really no such thing as snargs in a function definition (that is, there are no “self-named parameters”), which simplifies a lot of the proposal. When snargs are used in a function call, they arrive as kwargs.

IMO that is an improvement to the proposal; keep it focused :slight_smile:

3 Likes
11

I think you are right, and the following line does not actually provide any value :

def func(***snargs) : ...

But this has some value

snargs = {***(x, y, z)}  # self-named arguments packing

(yet snargs is only an arbitrary and funny name)
It gives a “self named” version of dict packing, and provides clarity to grasp what happens here :

x, y, z = ***func(***(x, y, z))

Furthermore, this should be almost equivalent

def func_a(***(x, y, z)) : ...
def func_b(x=x, y=y, z=z)) : ...

except that func_a will not look into outer scope to get x.
If one default value should be given to x, let say the global X, then the signature will be

def func_a(x=X, ***(y, z)) : ...

→ yes… things are simplified here.

Also one other thing is undefined and might bring another advantage, if you have:

snargs = ***(x, y, z)

then you unpack a subset :

x, z = ***snargs

→ Should this be a partial unpacking (thus novel functionality) or raise an error that states the incorrect number of keys were unpacked (thus fool-proofing even more) ?

1 Like
12

Not sure what you mean by “look into outer scope” here?

13

This is meant to address this problem

… or if some variable x is defined in global scope, def func_a(***(x, y, z)) : ... should probably not be allowed to pick it up as default value.

14

I don’t necessarily propose {{, this is just one option (which I have seen somewhere) - I had to use something.

If this is desirable, naturally, best possible syntax would need to be determined to see if it is acceptable or not.

I think this is concerning syntax implementation. If so, then I guess that depends on whether it is {{ or something else.

But I was not referring to syntax specifically, but about the logic of arguments and various nuances that arise at various levels of having to deal with it. Mental effort to figure things out there is already much higher compared to construction of dict.

If change is only syntactic, then this doesn’t apply, but if not, then further complicating things like inspect.signature routines and similar points of logic at C level is highly undesirable IMO.

Learning effort still applies in either case though.

1 Like
15

Ah. The use of parameters whose defaults are the same as the parameter names is used for getting a local reference to a global (for performance), so I’m not sure why there would need to be an exclusion of globals. For example, let’s suppose you’re going to need to find the lengths of a bunch of things; you might do:

def something(x, y, z, len=len):

which will do one expensive global lookup when the function is defined, and then fast local lookups when it’s running. IMO there’s no reason to have a special case to prevent that; although, personally I would also be inclined to keep function parameters out of the proposal altogether, and focus on the call site.

16

By the way, this will be available via t-strings:

snargs = tdict(t'{x}{y}{z}')
17

Yup, I think there is better way to do this.

Somehow I have a hunch that you are already aware about it. Has this already been proposed before?

a = 1
d = {a:}         # {'a': 1}
d = {a:, b: 2}   # {'a': 1, 'b': 2}
2 Likes
18

I think it has; if not, others very similar to it have. And I would be in favour of doing this. Much better than trying to change the delimiters.

19

Something similar was linked at the top:

1 Like
20

Thanks!

Both of these look ok:

{:a, 'b': 2}
{a:, 'b': 2}

I would probably favour the first one as:

  1. It’s logic is more in-line with current situation. It can be summarised as “if key is missing, it is substituted as string expression of value input”. The 2nd one is a bit more complex.
  2. I suspect implementation is a bit more straight forward.