Allow identifiers as keyword arguments at function call site (extension of PEP 3102?)

This has been brought up at least once that I can remember (this being the elimination of a = a, and the general argument against it that still holds is that it’s much too magical. It would do your case well to look up those earlier discussions, find the arguments for and against and argue what has changed to again make this an option to consider.

2 Likes

Hi Jacob - thanks for replying on my post. It doesn’t surprise me that this idea has been raised before. I would be very interested to read past discussions on this topic

I have searched for this topic here and on google and so far come up empty handed. It’s hard to think of search terms that will catch this. If you are able to find a link to the discussion you’ve seen before I would be very grateful

Finding old discussions is annoying because they are spread over multiple possible forums, here, github issues, the python-ideas mailing list and the python-dev mailing list. I’m on my phone now but I’ll post the links after work unless somebody else does so first.

Regarding the idea itself, I’m not a huge fan. Apart from it being a bit too implicit, I can also see potential confusing situations like this:


def foo(x, y): ...

x = 1
y = 2

# Will this be called with args or kwargs semantics?
foo(y, x)
1 Like

This part, unfortunately, is way too magical. A call MUST be well-defined without reference to the function being called, otherwise all kinds of things become near-impossible.

But if done with a syntactic marker, that would be a bit more reasonable. Your second example:

seems far more plausible to me. This eliminates the “is this positional or keyword semantics?” problem and is purely and simply syntactic sugar.

1 Like

Major issues:

  1. When code is generated at the call site, it doesn’t know anything about the function being called (because it will be looked up at runtime). If I write func(params, d_p_list, momentum_buffer_list, weight_decay) then Python cannot determine that weight_decay is a keyword-only argument of func. It can’t even determine that func is callable (maybe there will be a TypeError at runtime).

  2. What would happen if the function also accepts var_positional arguments, or positional arguments with defaults? How do we know when the implicitly-keyword arguments start?

def func(*x, y=None):
    pass

func(y) # is this equivalent to `func(y, y=None)`, or to `func(y=y)`?
  1. What about unpacking arguments with *? Could those results end up accidentally being keyword arguments?

I think having a syntactic marker at the call site - such as a bare * argument (mirroring how it works in parameters) - solves all these problems, but I’m not sure:

func(params, d_p_list, momentum_buffer_list, *, weight_decay) # etc.

It would also be needed explicitly in all cases, even the ones where * is used to unpack positional arguments - because currently, func(x, *y, z) can mean that z is intended as a positional argument as well.

Another option would be to say that ** unpacking can unpack from a set as well as a mapping, and have it implicitly look up “values” for those “keys’” from the local namespace. Then we could write func(a, b, **{'c', 'd', 'e'}) as equivalent to func(a, b, c=c, d=d, e=e). This could even become a more general unpacking rule - e.g.

# with this hypothetical feature implemented
normal_meal = {'bacon': 1, 'eggs': 2}
spam = 3
baked_beans = 4
viking_meal = {**normal_meal, **{'spam', 'baked_beans'}}
assert viking_meal == {'bacon': 1, 'eggs': 2, 'spam': 3, 'baked_beans': 4}
2 Likes

Previous discussions: Shorthand notation of dict literal and function call

1 Like

many thanks!

This case would be unambiguous because (under the proposal) kwargs semantics would apply only when positional arguments are illegal. However, I agree that one could get tripped up on this by not being sure about the keyword-only argument status of the function (moreover subsequent changes to the keyword-only argument status of the function would have unexpected impact on the caller here)

This is a very good point. I suppose this is necessary in order to be able to compile the calling code to bytecode without locking down the function definition code we are calling into

It seems then that the * marker is really necessary both for the compiler and for clarity of intention for human readers of code. And as per comments from Karl, it is important that this is a bare * and not *args since the latter doesn’t preclude more positional args

1 Like

Thanks Chris

I guess in answer to Jacob’s question “what has changed to again make this an option to consider” all that’s different is an alternative syntax proposal to address the same issue

I guess that’s a matter of taste but maybe folks find this:

dict(*, a, b, c)

more palatable than this:

dict(a=, b=, c=)

or:

dict(:a, :b, :c)

as previously proposed

Apologies for having raised this issue as a new topic. Hopefully at least the discussion of call-site unambiguity is helpful for others reading this as it was for me

4 Likes

Just to adapt my list of advantages from the previous proposal (var=) to the * proposal:

  1. You get dict(*, key1, key2) for free.
  2. An API changing an argument with a default to keyword-only can be dealt with (if parameters and variables match) by changing f(a, b, c) to f(a, b, *, c).
  3. It encourages using variables with names that match parameters. f(*, long_variable_name) is cleaner than f(long_variable_name=longvar), which some might prefer to f(long_variable_name=long_variable_name).

You obviously already covered 1.

I do prefer this approach to the previous suggestions, as it closely mirrors the function signature, which is a well-established concept. I agree with others in this thread that this should require an explicit *,, and never be implicit. Regardless of whether the technical ambiguities could be resolved, I would still want a clear syntactic marker for visual clarity.

I still find the idea of a shorthand appealing, for whatever that’s worth. The main argument I see against it is that positional-only and keyword-only arguments are advanced concepts that beginners can go quite a while before needing to understand. You can read code that calls functions with keyword-only arguments with no new concepts, and it’s only when you need to write that code that it matters. This would shift the need to understand these to the first time you encounter something like dict(*, a, b, c), which has the potential to become a common idiom.

2 Likes

It’s worth noting that p-only and kw-only are actually irrelevant here; the arguments passed in this way will always be passed as keywords, regardless of what’s defined at the other end.

1 Like

I like the idea of the func(*, a, b, c)-like syntax as an equivalent to func(a=a, b=b, c=c), yet I am not sure the * character is the most appropriate choice of the marker – because in the case of the definition syntax, the * and *somevar constructs are consistent with each other in that they have the same consequences for any following parameters (that they will be keyword-only ones), e.g.:

def func1(*args, kwonly, kwonly2): ...
def func2(*, kwonly, kwonly2): ...

…but, when it comes to the new imaginary call syntax, there would be no analogous consistency:

func(*args, a, b, c)  # existing syntax, we cannot (and don't want to) change its meaning
func(*, a, b, c)  # new syntax, supposed to be equivalent to `func(a=a, b=b, c=c)`

…so it might be confusing.

I believe that a lone = character would be a much better choice.

func(=, a, b, c)  # new syntax, supposed to be equivalent to `func(a=a, b=b, c=c)`

Not only the aforementioned confusion would be avoided, but also the = character it this role seems quite suggestive. I believe, it would be easy to learn and remember (or maybe even guess?) what this shortcut is supposed to mean.

3 Likes

The hash character already has meaning, this won’t work. Sorry.

@Rosuav looks like you fired before reading. His proposal is to use =, not #.

(Not that I think Python needs this feature.)

1 Like

This approach would essentially prevent code refactoring. If you were to rename a parameter in a function definition, it would cause issues in the parts of the code where this function is employed.

1 Like

Dangit, I think I was messed up by a dirty screen there. My bad. I swear it looked like a hash sign.

TBH though, I think the asterisk is still sufficiently parallel in meaning that it’s a better choice, and the equals doesn’t have much to brag of. If this feature were to be implemented (far from certain), the equals sign would be appropriate alongside a specific argument, but the asterisk would be a viable separator. So these two notional syntaxes make at least some sense:

func(a=, b=, c=)
func(*, a, b, c)

But I’m not convinced that they’re offering enough here.

2 Likes

Me neither. But for the sake of argument (or to show off how devious my mind works :slight_smile: ), I would prefer the former (a=) because that draws the reader’s attention to the unusual syntax for each keyword. The * having an “effect at a distance” on all following arguments in the call is more easily missed when skimming code.

Also, we should at least consider how this would combine with keyword arguments that do specify both the keyword and value – in case you have a bunch of keywords following a similar pattern but one exception. Again, I think here a= wins – we can easily understand what f(a=, b=1, c=) would mean, given a minimal explanation of a=.

Also, someone previously worried about what happens if a refactoring of the callee causes a keyword to change. Again, I think the a= notation, augmented with the idea of mix-and-match, would win out here, since you could just change the call site from f(a=, b=, c=) to f(a=, new_b=b, c=).

Still, this is all an intellectual puzzle that we can try to solve, but it leaves the larger question unanswered – does Python really become a better language with this feature? I don’t think it does – this isn’t like something I’ve seen in other languages (maybe Perl? :slight_smile: ) and if you don’t already know what it means it’s hard to look up (you’d probably end up finding the trailing = in f-strings, which has totally different grammar and meaning). It also feels a bit too easy to overlook. This can all be litigated endlessly (“but what if we …”), but in the end it’s not me you have to convince but the Steering Council, and they’ve rejected much less niche proposals.

11 Likes

Thanks for the feedback, Guido!

I agree with the benefits you point out

The improvement is of convenience and readability. It’s not a huge deal of course. But I feel like there are already dozens (hundreds?) of areas of Python where convenience and readability wins compared to other languages have contributed to the popularity of Python. This would just add to that list

Is the meaning totally different? To me there is something similar here:

f"{long_identifier=}" is shorthand for f"long_indentifier={long_identifier}"

func(long_identifier=) is shorthand for func(long_identifier=long_identifier)

One relevant example I know of from another language is implicit in Scala. A good explanation is here. implicit in Scala is way more magical than the feature we are discussing - it sources parameters not just by matching to the call site identifier but actually searching for any suitably typed variable in the calling scope to plug in :slight_smile:

I can believe this. I would argue though that this proposal is, an a sense, not that niche. It is arguably a minor convenience add. But it is widely applicable - a large percentage of significant codebases presumably include func(x=x, y=y, z=z)-like calls. My guess is many Python users would take up the opportunity to use the proposed concise syntax going forward if this feature were available in the language

2 Likes

Hmm, can we replace that “presumably” with an actual list of examples?

Systemverilog has a similar feature. It is called “implicit port connection” there. Its syntax is dot. f(.a, .b, c=z).

I think this feature can make Python code a lot less verbose when applicable. I would then be more inclined to use keyword arguments instead of positional arguments. This can make Python code safer.

I believe this contradicts the very essence of a function definition. A function is meant to be a self-contained unit, a distinct part of a larger whole. Allowing variable names as keywords would interlink function definitions, making them dependent on each other and essentially forcing them to behave as a single entity. What term would accurately describe this concept? Co-joined functions?