Conditional elements/arguments

Hi, I would like to know what you think about an idea I got recently.

Problem

It is sometime required to pass an argument to an function if a condition is true

if sys.version_info >= (3.8):
    ast.dump(tree, indent=4)
else:
    ast.dump(tree)

This can get difficult to maintain if the arguments getting more.

It is possible to write this in one line.

cnd=sys.version_info >= (3.8)
ast.dump(tree, **({"indent":4} if cnd else {}) )

this can be used for multiple arguments if necessary and scales but looks even worse.

Idea

The new syntax I propose looks like follow:

ast.dump(tree, indent = 4 if cnd )

This would allow to express the thoughts declarative without any control flow.

Another example:

def style_print(msg, style="default"):
    ...

def log(msg, error):
    style_print(msg, "red" if error)

the benefit here is that the calling code does not have to know the default value and can use the default behavior.

Other usecases

The condition could also be used for data structures.

l = [1,2,3 if cnd]

Instead of

l = [1,2,*([3]*bool(cnd))]

which is compact but takes a while to wrap your head around (I’m not a fan of this pattern).

Or

l = [1,2]

if cnd:
    l += [3]

Which is ok but a bit longer.

In general, this new syntax could be used where list or dict expansions are valid.

  • Dict
  • Function arguments
  • List
  • Tuple
  • Set

The implementation would just be syntactic sugar for list/dict expansion and does not introduce something total new to the language.

  • [x if c] becomes [*([x] if c else [])]
  • [*x if c] becomes [*(x if c else [])]
  • func(a=x if c) becomes func(**({"a":x} if c else {}))
  • {"a":x if c} becomes {**({"a":x} if c else {})}
  • {**d if c} becomes {**(d if c else {})}

In general I would think that it enables a more declarative approach to write code.

I’m not aware of any other programming language with a similar feature, please let me know if you have seen something similar.

The important questions I think are:

  • How difficult is it to learn or teach?
  • Is it possible to understand this code if you don’t now this syntax?
  • Are there enough use cases which could benefit from this syntax?
  • Would you like to use it?

I look forward to your feedback.

7 Likes
  1. It’s not particularly difficult to teach the basics, but there are a lot of edge cases that would be very tricky to explain. And as soon as you start trying to think about general rules rather than specifics, things get confusing fast (I spent a while trying to understand the precendence of f(1,2 if cond) - could it mean “pass the single arg (1,2) or no args, depending on cond”, before I came to the conclusion that the grammar would need parens for this interpretation). So overall, OK for the basics, but it’s painful once you go beyond that.
  2. It’s not really got any other possible interpretation, but it’s unfamiliar (and there’s no equivalent in any other programming language that I know of) and it could easily be a typo for a conditional expression with the else part missing - making what’s currently an immediate error into a silent wrong answer. So beyond the basic idea that “you can learn to understand anything given time”, IMO it leads to code that is tricky to understand, and hard to interpret.
  3. Not in my experience. The only times I’ve needed something like this, I’ve felt no need to compress it into a single line. Quite the opposite - the multi-line version is clearer to me.
  4. I’d hate to use it, and even more so I’d hate to have to maintain code that used it. I’d likely ask for it to be rewritten as a multi-line version if I encountered this in code review.

Maybe, but it feels more like it encourages trying to cram too much meaning into single expressions. Python has both statements and expressions, and forcing everything into fewer expressions abandons the structuring benefits you get from using statements and expressions appropriately and leveraging the strengths of each.

No, and that’s a mark against this proposal. Prior art, and compelling real-world use cases, are crucial to a successful proposal, and you haven’t provided either (your examples are theoretical, and show marginal benefits so aren’t what I’d describe as “compelling”).

Sorry, but it’s a very strong -1 from me.

2 Likes

Thank you for your feedback. I just got the idea and saw more and more use cases over time. It is sometimes difficult to view your own ideas from the outside.

I’ve seen this in another language, maybe F#?

2 Likes

while the second version is definitely more condensed, I would personally write this pattern this way:

kwargs = {}
if sys.version_info >= (3.8):
    kwargs["indent"] = 4
ast.dump(tree, **kwargs)

I find this pattern fairly readable, and it scales very well, including things like followup conditionals that pop a kwarg or modify its value.

at first glance of the proposed syntax:

ast.dump(tree, indent = 4 if cnd )

I initially liked it, as it seems like a clean and clever way to achieve this behavior.
but reading your discussion prompts, I think my initial response was biased by the set up.
I think that if I came across this in the wild without any set up or context, I would have been surprised that the indent keyword was completely omitted. I’d first think there’s a missing else .. part, and then I’d maybe think this is a new shortcut I wasn’t familiar with for something like 4 if cond else False.

2 Likes

Although I’ve employed this exact approach hundreds of times over the last few years, I very much don’t like it. It moves the argument specification away from the functional call, and loses parameter type checking (without a TypedDict for each function which is explicitly used, or a much more complex type checker).

I like the implementation of the proposal, however I think having the parameter name first can cause readers to think that the argument is definitely passed, so I would prefer a syntax which declares the conditional first, for example:

foo(1, if cond1: 2, bar=3, if cond1: baz=4)

I’ve encountered a need for this feature quite often, not only so I don’t have to keep track of default values, but because some libraries change behaviour on the existence of an argument (boto3).

PS: there’s been one or two other threads with the same or similar proposal, could you please link them in the original post?

2 Likes

F# has a conditional expression where the else can be missing.

The return type of the then branch in this case must be unit (something like void for my understanding, but I don’t know enough about F#).

But I didn`t find where this can be used to pass conditional parameters to a function.

2 Likes

For a function call, this wouldn’t be an extension of the conditional expression, but an extension of the function call syntax itself. cnd = False; foo(x = 4 if cnd) is supposed to be equivalent to foo(), not foo(whatever_default_x_has).

1 Like

I will try to fill some gaps based on your feedback.

lets talk about lists:

list comprehensions in python do already what you say, they allow to replace imperative code with a more deklarative alternative.

l=[]
for x in range(5):
    if x%2==0:
        l.append(x)

l=[x for x in range(5) if x%2==0]

I gave a short introduction to python for some colleges and I remember that some thought the same way about this syntax. It is unfamiliar and new for every one learning python. But every language has its unique features and strength which are worth learning because they are usually the reason we use this languages.

The funny thing I discovered about my proposal recently is that the implementation is similar to list comprehensions.

l=[x if x%2==0]

l=[]
if x%2==0:
    l.append(x)

It is not a abstraction for creating lists but for combining. I think that complements the existing language in a useful way.

I was currious how some use cases might look like and how many there are.
So I did some analysis and transformed some code into the new syntax. This gives an good overview how the pattern might look in different real world use cases.

An interesting example is the following:

python3.11/tkinter/init.py:3875
current syntax:

args = [self._w, 'search']
if forwards: args.append('-forwards')
if backwards: args.append('-backwards')
if exact: args.append('-exact')
if regexp: args.append('-regexp')
if nocase: args.append('-nocase')
if elide: args.append('-elide')
if count: args.append('-count'); args.append(count)
if pattern and pattern[0] == '-': args.append('--')
args.append(pattern)
args.append(index)
if stopindex: args.append(stopindex)

new syntax:

args=[
    self._w,
    'search',
    '-forwards' if forwards,
    '-backwards' if backwards,
    '-exact' if exact,
    '-regexp' if regexp,
    '-nocase' if nocase,
    '-elide' if elide,
    *['-count', count] if count,
    '--' if pattern and pattern[0] == '-',
    pattern,
    index,
    stopindex if stopindex,
]

The wish to write this code in a compact form exist (this is not the only case in the results with this formatting style), but the current possibilities are limited and I would not say that this leads to more readable code.

I agree that the if with the missing else looks unfamiliar if you read it the first time. But the question is: are there enough use cases where people will like to use it so it becomes not something rare which surprises you every time you read it.
The semantic is very simple and the same in every use case (argument, list-entry, set-entry, …)

Regarding the typo problem: I don’t remember that I got a syntax error because of a missing else but I remember a lot syntax errors where I missed a :. I don’t think that the missing else will be a big problem, but I might be wrong here. @EpicWink provided a alternative syntax.

l=[1, 2, if cond: c]

This would solve the missing else problem but it introduces a : inside the expression. The usage in dict literals would look like this.

d={"a": 1, "b": 2, if cond: "c": c}

Putting the if behind something is already a know concept in python.

[x for x in values if x>0]
[a if b else c]
[a if b] # new syntax

I think it would fit in the existing style of the language but this is a personal taste.

I every thing has been sometime invented without prior art. Real world use cases are important. I analysed some source code and showed one example here. I might not have found every case where the new syntax can be applied. There might be more cases in the python codebase.

I don’t want this proposal to be accepted soon. I want people to spend some time thinking about it, remembering it, and perhaps finding use cases in their daily work where they would like to use it.

3 Likes

Everything does have to be invented at some point, but there is usually some related or semi-related concept already in the world. If there isn’t, why not? There are three possibilities that come to mind, and I’ll let you decide which is the most likely:

  1. You are more brilliant than literally everyone who has come before you, and thus you have thought of something nobody else has
  2. Prior art DOES exist, but we just need to cast a wider net (maybe we need people with experience of more languages) so we can better evaluate the proposal
  3. This is actually a really bad idea, which other language designers have considered and rejected.

One of them is extremely hard to prove, and the other two basically have the same strategy for proof: research more languages and language designers. Hence the call for prior art :slight_smile:

Sorry :see_no_evil: , I really didn’t mean to sound pretentious here and would like to offer me a fourth option.

  1. I am the one lucky monkey with the typewriter.

You are right. Study of prior art is important and I will search for it.

Ah yes, luck or genius, either way :slight_smile: Anyhow, that’s why people want to see prior art, even though there IS every possibility that it doesn’t exist. And you didn’t sound pretentious (not intentionally at least), that was just me taking the presumption to its illogical extreme :smiley:

Just yesterday I learned that there is such feature in Dart (or there will be in future Dart 3.0?). Collections | Dart

In addition to collection-if, they have collection-for, which looks like a generalization of comprehensions syntax. Adapting it in Python would allow to write [3, i**2 for i in range(5), 7] and get [3, 0, 1, 4, 9, 16, 7]. No doubts that if collection-if be added in Python, we will immediately get requests for adding collection-for especially since it already exists in a partial form of comprehensions, so it is better to think about it in advance. However would not these features conflict if collection-for contains also the if clause? [i**2 if i%2 for i in range(5)] could look more natural than current [i**2 for i in range(5) if i%2]. The latter could be re-interpreted as [*((i**2 for i in range(5)) if i%2 else ())] which is very different from the current semantic.

Thank you for this resource.
I tried it in https://dartpad.dev and it worked there.

My current Idea is that the condition should only be valid at the end of an argument/collection element.
much like *list is only valid at specific places.
I think using the same rules for both makes it a bit easier to understand.

So the question is if

[*sub_list for sub_list in list_of_lists] 

should be valid for flattening lists?

in this case

[x if condition(x) for x in some_list] 

would mean the same as:

[x for x in some_list if condition(x)] 

The reinterpretation in this case would not be a problem, because:

[*sub_list for sub_list in list_of_lists] 

is not interpreted as:

[*[sub_list for sub_list in list_of_lists]] 

in other words the string “sub_list for sub_list in list_of_lists” is not a expression which could be expanded with [* ...] so it can also not be conditional with [... if condition].

I fully agree on this point.

1 Like

Hi Frank,

Regarding F# - What Guido is referring to is the use of conditionals in a number of places when generating objects and collections. This is relevant but not precisely aligned, but perhaps could be. Guido encountered this in this enjoyable session we did together: Don Teaches Guido F# | #dotNETConf: Focus on F# - YouTube

In F#, there is a powerful and general form of “list expression” that goes beyond list comprehensions, and is really intuitive, in particular you can have conditionals, loops and pattern matching cases within a list expression, e.g. some examples below

[ 1; 2; 3 ]

[ 1+1
  2+2
  3+3 ]

[ 1+1 // always emitted
  if tomorrow then 
    2+2 // conditional emit
  if yesterday then
    3+3 // conditional emit
  4+4 // always emitted
] 

[ for i in 0..4 do
    (i, i+4) // emit a tuple
]

In the conditional emit example, the expression evaluates to either [ 2, 8 ], [ 2, 4, 8 ] or [ 2, 6, 8 ] depending on the day.

The point here is that the emit of an element can be conditional. Basically you can take any F# expression, put [ ... ] around it and the control flow gets interpreted in the same way, but the expression results get emitted to build a list. The same can be done with mutable arrays, sequences and all these are instances of a more general thing called “computation expressions”. The smooth transition from imperative code to conditional emit of elements in list expressions is actually incredibly useful and is, in my judgement, perhaps the simplest and most intuitive thing missing from most expression-oriented languages. Some of the relationship to “do” syntax in Haskell is covered in About F# | History.

Now in F# this doesn’t apply to records, tuples, argument lists or objects, because it’s fairly difficult to track the types of things then (though computational type systems as used in TS and typed Python may be able to do it). But in theory the same thinking could absolutely be applied to conditionalize or arbitrarily compute and emit the properties of records, tuples, argument lists and objects. In JS you could imagine this being something like this

// unconditional
{ 
  A: 1,
  B: 2 
}

// conditional
{ 
  A: 1,
  if (today) {
     B: 2
  }
  if (tomorrow) {
     C: 2
  }
}

I hope this helps. I know you’re also after a super succinct syntax for this kind of conditionalization, and I don’t know if that’s possible.

2 Likes

For Python, my specific recommendation would be that Python somehow find a way to allow things like this

[ 6+8 // unconditional emit

  if foo:
    1+1  // conditional emit

  if bar: 
    2+2 // conditional emit

  // emit governed by a loop
  for x in fruits:
    3+x.count // loop with emit

  // emit governed by a match
  match status:
    case 400:
      400+2 
    case 404:
      404+10  
]

I’m sure that exact thing won’t be possible for whatever million reasons, but hopefully you get the idea.

Hmm. That is ALMOST possible with this…

@list
def items():
    yield 6+8 # unconditional emit

    if foo:
        yield 1+1 # conditional emit

    if bar: 
        yield 2+2 # conditional emit

     # emit governed by a loop
     for x in fruits:
         yield 3+x.count # loop with emit

     # emit governed by a match
     match status:
         case 400:
             yield 400+2 
         case 404:
             yield 404+10

Unfortunately, “almost” isn’t good enough, so you need a dedicated decorator:

def as_list(gen):
     return list(gen())

Still, pretty close.

Well, we can do @lambda _: list(_()) now. Still, wrapping it all in a generator is exactly as nice as building the item by item.

Hmm. That is ALMOST possible with this…

Yup. In F# generators are expression forms seq { .... generating code ... } (for on-demand iterators) or [ ... generating code ... ] (for lists) rather than tied to function or method definitions.

Yeah, but the elegance of writing @list would have been very tempting!

Yup. What often happens with these proposals is that the complete power IS already available, but requires a longhand notation. The hard part is finding which subset of that power is useful enough and/or common enough to warrant its own dedicated shorthand syntax.

2 Likes