PEP 802: Display Syntax for the Empty Set

I’m unfamiliar with the internals and can imagine a few edge cases but is it feasible to make set() as performant as the theoretical {/}?

2 Likes

I only found 84 results for = set() in CPython. repo:python/cpython language:python "= set()" -path:test

There are over 2.6 million results for Python code. language:python "= set()"

I’m not sure what effects the widespread use of set() might have on the adoption of the new syntax. However, the strong inertia created by its current usage could hinder acceptance. I don’t believe there will be much enthusiasm for replacing it with a less readable form.

“In theory” there’s no real limit on what sufficiently ambitious optimizers can do. In this case, if set() is in a loop, a number of more-than-less standard JIT tricks can all but eliminate the overhead of looking up the name; and if it’s not in a loop, who cares :wink:?

But, according to me, it won’t sway anyone in either direction. I expect that if set() were made 10x faster than today, or 10x slower, essentially no real program would notice.

Because sets are relatively heavy objects, While it’s very common for programs to create new ints, floats, small strings, small tuples, and sometimes even small lists, at a ferocious rate, sets (& dicts) tend to stick around. You mutate, test membership, etc, over time, and the creation time gets lost in the noise.

There’s also that any way of spelling set() generally needs to create a brand new object each time it’s encountered. In the absence of whole-program analysis, there’s no way to know it won’t be mutated So it’s no good to return the same empty set each time (although, sure, a more complicated copy-on-write implementation could overcome that).

In contrast, e.g., tuple() (and ()) happens to return the very same empty tuple object each time it’s encountered. Looking up the name is most of the cost. Not so for set():

$ py -m timeit "set"
20000000 loops, best of 5: 29.8 nsec per loop

$ py -m timeit "set()"
5000000 loops, best of 5: 86.6 nsec per loop

So building a new empty set cost a lot more than just looking up the name.

Overall, I’m -0.6 on this. It’s too late to do more than just pile on new minor inconsistencies.

19 Likes

I agree that the use of a / here feels weird when it has no logical connection to other parts of Python, but what if it does?

Namely, if we generalize / as a sentinel object to denote an absence of value such that an item of / in a sequence, or a key with with a value of / in a mapping, is omitted, then {/} as an empty set becomes just a natural byproduct..

Possible use cases:

  1. Conditionally building command-line arguments:
# equivalent to subprocess.run(['cmd']) when DEBUG is false
subprocess.run(['cmd', '--debug' if DEBUG else /])
  1. Conditionally specifying keyword arguments inline:
# ast.dump(tree) when Python version < 3.8
ast.dump(tree, indent=4 if sys.version_info >= (3, 8) else /)
  1. A mapping function that can conditionally skip output:
def func(num):
    if num > 5:
        return /
    return num * 2

list(map(func, range(1, 10))) # [2, 4, 6, 8, 10]

/ can be remembered as part of a prohibition sign :prohibited:, though in a different orientation.

6 Likes

What would this do?

def func(num):
    if num > 5:
        return /
    return num * 2
print(func(6))

This would output the sentinel object /'s repr directly, which can be '/' or '<Absence object>' or 'Absence', etc., to be decided by consensus, though I would lean towards Absence since print(...) outputs Ellipsis.

EDIT: Oops my bad. print(func(6)) should really output nothing but a newline, since argument list should be considered a sequence context so print(func(6)) should be evaluated as print().

1 Like

Your idea is neat, but having / be an object, rather than a piece of special syntax, worries me, for a few reasons.

The parser is going to be at least a little bit sad at having to decipher things like x /= / // / / / // /. I can’t think of any totally breaking parse problems, but their existence wouldn’t surprise me.

It creates a lot of weird edge cases in, e.g.

def append(xs, x):
    xs.append(x)
    assert x in xs
    # or, at the very least,
    assert any(y is x for y in xs)
 
def map_(xs, f):
    ys = [f(x) for x in xs]
    assert len(xs) == len(ys)
    return ys

class NonEmptyList:
    def __init__(self, x):
        self.inner = [x]
        assert len(self.inner) > 0

def id_(x):
    return id(x)

/and None remind me of JavaScript’s undefined and null. I find having the two of these tends to feel like a distinction without a difference: it’s often unclear which you ought to be using or checking for.

My concern with making incremental additions to the language like this is that at some point all syntax errors become valid code. We don’t want the error my_dict = {:} to pass silently like my_dict = {a:=1, b:=2} does since the introduction of the Walrus operator (this typo can easily happen when converting a sequence of assignment statements into a dictionary).

Obviously, the more logical thing in hindsight would have been to make {} the literal for an empty set and choose something else for dictionaries ({:} maybe?). But I don’t think one sub-optimal decision in the past justifies introducing another.

3 Likes

-1 on the proposal as it stands (unsurprisingly, but I align entirely with Guido and Tim on this).

The one slightly compelling case for doing anything at all is the occasional confusion that removing all the elements from a literal {a, b, c} changes the type of the container.

But the best idea I have there is for dict to gain enough interface to be compatible with set, probably by treating sets like dicts where every value is None. In my mind, it’s a bit like how ints auto-promote to float by assuming a trailing .0.

I’m sure there’s a reason that it’s entirely infeasible, and obviously there will always be performance benefits to using set over dict when you want a set (just as there are perf benefits to choosing a specific type in most cases where multiple possibilities exist), but it would smooth over most of the practical impacts of the ambiguity, even if the theoretical situation is less pure.

11 Likes

I don’t think ambiguity is going to be a realistic issue because the / object will not support any operation and will not allow override, so any combination of slashes will result in a TypeError and will not survive as working code anyway. It’s certainly no worse than deciphering what '''''' should be parsed into.

Yes, support for the / object from existing code will have to be assumed to be none unless explcitly opted in through default values, typing and/or documentation.

One should explicitly test for an / object when supporting it in a non-container context:

def append(xs, x=/): # we can alias / as Absent if desired
    if x is not /:
        xs.append(x)
        assert x in xs
        assert any(y is x for y in xs)

If {/} represents an empty set, what syntax would represent an empty frozenset?

If there should be no syntax for an empty frozenset, what arguments would one use against a future PEP that suggested such syntax?

4 Likes

I don’t see why you think an empty set literal {/} would prevent the possibility of an empty frozenset literal such as {{/}} in the future. Can you elaborate?

I was fairly on the fence about this PEP (and leaning towards against), until I saw your post, and it sold me.

As Chris Angelico alluded to above, there just are not enough single character delimiters to capture all of the collections cleanly. The proposed {/} token attempts to address this problem but does so in a very clunky and unintuitive way, it is completely alien from everything else syntactically.

Pairing it with {:} for dicts elegantly resolves this inconsistency, while also improving the clarity of Python’s syntax. The legacy empty dictionary definition syntax ({}) would hang around for backwards compatibility reasons, but in my opinion your proposed alternate syntax should eventually become the recommended syntax after sufficiently many releases, if it were included in a future language revision. I think the reason that this syntax seems so intuitive is that it ties directly into the existing <key> : <value> pair syntax, while making it clear that there is neither a key nor a value: its an empty dict and it speaks for itself.

If your alternative syntax were to come into the language, the {/} would fit fairly cleanly along side it, being significantly more intuitive than the existing {*()} notation (or other iterable-unpacking-based-notations for that matter). The {<set-item>} and set() syntactic discrepancy threw me for a loop when I was first learning Python and while I memorized this idiosyncratic mapping a long time ago, having something more consistent would make the new-comer’s experience a little easier.

With that being said, I agree with others in this discussion that the relation between {/} and is tenuous at best, but I really cannot think of a better notation given the cards that have been dealt, so to speak. Given that {} is already reserved for dicts, and has been for long enough that it is effectively intransigently set in stone, I don’t think there can be a better syntactic combination for the null set. At least, not one which can be easily typed on a standard keyboard.

4 Likes

The question would then become: Why should lists and tuples be written simply as [] and (), but dictionaries have to be {:}? It doesn’t remove the inconsistency, it only shifts it.

4 Likes

I don’t think that. I’m asking what would be the proposed syntax for an empty frozenset. Or would empty frozensets purposefully not have their own syntax?

Also, {{/}} couldn’t be used because that would mean a set containing the empty set.

1 Like

It has been previously argued that {{...}} may be used as a frozenset literal because it currently produces a TypeError so it does not exist in any real code to cause compatibility issues.

I’d suggest {*} as empty set and {**} as empty dict, as mnemonic/shorthand for {*()} and {**{}} respectively. This would be concise and hint at the unpacking feature, which would be educational for a new user to learn about.

I’m not a fan of {/} as it seems like a special case to remember without any link to existing language features. Personally I’m fine with set(), and I agree with the point that Paul Moore makes that introducing a new syntax will create social pressure to change existing code.

16 Likes

We already have context dependent overloading for the { and } in the collections space, in a way that is distinct from lists and tuples. Adding the {/} and {:} notations would resolve the inconsistency in empty case declarations for both of these collections.

The dual uses of the curly brace to delimit both the set and the dict fundamentally impose conflicting requirements. Either you have external consistency (between the empty list, empty tuple, and empty dict, to the exclusion of the empty set) or you have internal consistency (between the dict and the set, to the exclusion of the empty tuple and list).

It could be argued, that the internally consistent approach to this syntactic sugar is more externally consistent than the current externally consistent approach for denoting the empty case of these basic collections:

list  -> []
tuple -> ()
dict  -> {}
set   -> set()

versus

list  -> []
tuple -> ()
dict  -> {:}
set   -> {/}

So yes, some inconsistency is shifted, but in the average case the entire group slightly more consistent with one and other, meaning that the net inconsistency has been reduced.

While I like the proposed {:} for its relation to the dict’s structure I do admit that the usage of : might also increase the inconsistency, given : usage in list slicing operations. I don’t really like the {/}, the / seems vaguely out of left field, as it doesn’t really tie into the rest of the language.

This seems like an even better suggestion, it achieves the same ends, while hinting at collection expansion but in a more digestible way than shoving a collection expansion in there.

5 Likes

This would be fair, if all four types were used equally frequently. They’re not. The “average case”, assuming all these display syntaxes get equal use, is very different from the “realistic case” where you see the different displays as often as they are actually seen.

I did a quick search for Python courses and tutorials. Here’s what I found, and the order that data types get taught:

If you search for courses, you’ll likely find different ones, but it’s fairly consistent that sets either aren’t taught at all, or are taught much later than others. Lists are by far the most approachable collection type, so I would expect that a typical novice Python programmer will come across those. Tuples aren’t always taught, and dicts are omitted sometimes too, but sets are omitted far more often.

Or look at a large corpus of Python code and see how many dictionaries are used, how many lists, etc. They’re definitely not equally important.

If you approach language design as though it were a puzzle or game, sure, making dict and set {:} and {/} makes some sense. But that’s not what makes a good language.

6 Likes

I am +0,5 for this as I would like empty set syntax for consistency, +1 for {*} and {**} as they have an existing analogue in the language, and -1 for special-casing set() as it effectively makes it a keyword.

Also, isn’t {,} the empty dict?

1 Like