Add list.discard() and deque.discard()

Sometimes you want to remove something collection if it’s in there.
set → discard()
dict → pop()
list → requires checking first on try / except.
deque → requires checking first on try / except.
It’s a weird inconsistency and I’m not aware of any reason beyond tradtion.

discard would be consistent with set and list and deque already have pop methods.

12 Likes

Personal im +1 on this. Perhaps adding a dict.discard (same as dict.pop) would help it be the same name everywhere.

But generally, great idea, I’ve found myself often implementing helpers doing this kind of stuff.

Or an elephant.

Elephant in Cairo - Wikipedia

def discard(xs, x):
    xs.append(x)
    xs.remove(x)
    if xs and xs[-1] is x:
        xs.pop()

xs = [1, 2, 3]
discard(xs, 3)
print(xs)
discard(xs, 3)
print(xs)

Attempt This Online!

4 Likes

This works, but it’s both less readable and slower (assuming .remove, .pop, .append and .__contains__each take some time C).

It’s complexed than it needs to be. Another thing this lacks are standards. .append, .remove are not guaranteed for, say a Sequence[...]). Just adding a simple .discard to all sequences would be easier.

1 Like

I think it’s actually stranger that the indexed sequence types support .remove() at all :wink:.

The dict and set types are “content-addressable”. Insertion and deletion work by object value, are O(1), and can’t be indexed directly by ordinals.

The indexed sequence types share none of those properties.

It became “a tradition” because nobody cared enough about sequence.discard(unwanted) to press for it in the early years. Lack of compelling use cases. Things like dict.pop(unwanted) were driven by frequently expressed need. dict.pop() in fact was added in Python 2.0, along with dict.popitem(). They didn’t exist from the start.

I’m neutral on whether sequence.discard() is added now. Not an itch I have, but if enough others do, fine by me.

7 Likes

Only searches once, so can be twice as fast as “checking first”. And can be hundreds of ns faster than “try / except” when the value isn’t there.

4 Likes

sequence.remove can be more efficient than set.remove/dict.pop when the size of a typical sequence for the specific use case is expected to be small, since sets and dicts require hashing first.

Use cases for sequence.discard are not uncommon at all, including:

So while yes it is easily worked around with a try-except block, it would still be a meaningful method to have.

17 Likes

I’ve learnt something new today, (despite it being dated 1989).

Thank you :slight_smile:

3 Likes

For those curious, list.discard() can be implemented by removing a single line (i.e. raise ValueError) from the list.remove() implementation. Here is a patch.

1 Like

I suspect that some of the ‘remove’ examples could now use a set rather than a list, but predate that addition, or wrap OS functions that return (short) lists.

Rather than a new, near duplicate function, I would prefer a proposal to add a parameter to the existing list.remove to raise or not upon value absence (defaulting to True for back compatibility).

Rather than a new, near duplicate function, I would prefer a proposal to add a parameter to the existing list.remove to raise or not upon value absence (defaulting to True for back compatibility).

I think this would be inconsistent with the other API’s as mentioned in the original post. As for the implementation, that could be moved to a helper to deduplicate.

Anyhow, if anyone wants to propose this as a PEP, (I myself would find it useful, though not particularly important) I am happy to help write it/co-author.

3 Likes

I have opened an issue to add list.discard.

(I have lurked for years and this is topic that is making me sign up after all this time. Apologies ahead of time for being a grouch!)

I would like to see a PEP properly justifying adding this before it is added because I am not sure it is a good idea anymore after reading this while trying to fall asleep and waking up. I fear its presence will have a negative impact on code quality and understandability of the best way to do certain things in Python.

My thoughts aren’t very clearly phrased, but I want to share them before it is ‘too late’ and this ends up in Python and it takes too much effort to roll back only for people to later wonder ‘why didn’t we think about it a bit more?’ which is a feeling I’ve had a couple of times seeing how changes crystallized since Python 3 initially came out.

My first concern is that with a dict and set, you are dabbling in a bag of unique items. Trying to discard something that is already there just means you are ensuring it isn’t in the set going forward. For a list/deque, you either remove multiple items which is not expected for some users since the naming implies removing a single item, or remove only the first sometimes, leaving a scenario where that item is still in the list/deque, creating a subtle bug where people think they had made sure that item would not be in the container. The remove name very clearly signals a difference in behavior compared to discard, especially when mixing sets and lists in the same code. The operation may be similar, but the usable outcomes differ strongly. Explicitly handling the ‘failure to remove’ is preferable to me because it makes clear one object is not like the other.

My second concern is more about algorithmic complexity of actions becoming more hidden. I don’t like hiding that complexity. Using remove has in my mind become a source of code smell, because it implies a linear search. If I see it used in code, I immediately feel the ‘position’ is likely to matter, which brings with it the penalties of dealing with such ordering. Note that if the position didn’t matter but I wanted to track duplicates, I would use a dict with a key:count mapping instead!

With those arguments in mind, I would vastly prefer list.remove_all as a non-throwing alternative rather than re-using thediscard name for lists, and vehemently opposite any form of list.discard especially where it concerns only a single item removal.. It preserves the hint towards algorithmic complexity for that family of positional data structures, and it avoids the bug-ridden pitfall of having a single-item discard-lookalike that leaves the list in an unknown state people are unlikely to consider.

Finally, I also feel this difference will be beneficial to people who use Python to start with coding. The discussion of ‘why does list.discard not exist, but list.remove_all does despite their similarity’ is something one can learn from. With the same name, I would argue it would even inhibit critical thinking to some degree, which is already ballooning as a big problem in this age of ease-of-use touchscreen devices and AI.

3 Likes

Thanks for your thoughts. :slight_smile:

This concern is understandable, but in my opinion it is a documentation problem rather than a design problem. The docs make the outcome explicit rather than silent. Furthermore, anyone who reads discard() as “ensure not present” on a list has the same misreading available for remove() today, and from what I have heard, that has not been a notable source of bugs.

I do not think discard() is more hidden than remove(). However, in the PR I propose to document them both on the time complexity page.

remove_all() is a different operation with its own use cases. It does not replace the common “remove it if it is there” pattern that discard() is for, so I don’t see them as alternatives to each other. If there is demand for removing all occurrences, a count parameter on remove()/discard(), like str.replace() has, could be considered in the future, and would fit naturally on top of this.

1 Like

I accept your viewpoint, but I unfortunately disagree. Code that self-documents its correctness is the best code and I feel we already have the explicit outcome for discard that this is proposing to take away. If I have some variables and I do

# example code; I have plenty of similar snippets in codebases though

# Remove x from current state
self.options.discard(x)
self.actions.discard(x)
self.layers.discard(x)  # silent footgun: x could still be in this list!

# as opposed to

self.options.discard(x)
self.actions.discard(x)
if x in self.layers:
    self.layers.remove(x)  # explicitly removing only the first entry

then I lose the visual separation that I am dealing with different containers with different behaviors, and that the discard action has a different result compared to my expectation of them all being sets. (Maybe there is a linting hint that can identify it, but I personally view lint as a bandaid for mistakes that result from bad language design, so to me that is not a valid argument in this discussion as it puts the cart before the horse.)

For over twenty years, I have mixed sets and lists in ways resembling this depending on my needs to track state. I discard things to make sure they aren’t present in my set. It is the only discard that Python’s structures have ever had, so it has taken that meaning by sheer usage.

That is the underlying goal of why I use it, and the set implemented this implication and separation as early as Python 2.3. You can’t discard from a list because discarding is a set operation. Sure, it means exactly ‘remove one if it is there’ on a programmatic level, but for all these decades, the end result of this method has been “after running s.discard(x) the invariant x not in s is achieved”. It may not be in the docs like that, but it is what is expected.

Between set and list, there is a clear distinction between add and append/insertbecause they have different meanings and results due to the positional nature of the list that does not apply to a set. Adding is ambiguous compared to appending for lists, and appending has no meaning for a set.

So why can we not have a different name for a list.discardto sidestep this entire issue? I’m all up for a bunch of bikeshedding as to what would be proper, but even if the implementation is nearly the same, I do not believe that the people using it will have the same expectations of the things it achieves.

To be clear: I completely understand the benefits of this functionality in terms of speed and avoiding the penalty of atry or if statement to accomplish the same thing. I just don’t believe that the argument that ‘it is documented’ is sufficient reason to have a footgun facing you in your code that we can easily avoid. The fact my editor and Python yell at me when I try to discard a list has saved me numerous times from making the mistake that you now argue can be avoided by reading the documentation. I know what the functions do and always have, but easily seeing and confronting my mistakes as opposed to silently letting them happen is a huge gotcha.

I feel the only thing accomplished now is that every discard in a codebase that does not check a return value is going to look like a potential code smell, because lists will need those checks most of the time (did we remove something, yes or no? Did such an entry remain afterwards, yes or no?) while for sets they are pointless.

From where I am standing, list.discardas proposed is looking like it will have tricky edge cases resulting akin to many of the C string library functions (length off by ones) that people will tend to not handle properly (except with leftover entries because they thought the list was a set). The only thing that can make those functions remotely safe is if you have to treat them everywhere equally, but the way to deal with the aftermath ofdiscard will differ depending on the container.

To me, it feels like we are 20 years too late to implement discard on other containers that have wildly different implications on what they factually accomplish. The only container that has ever had it is set, so I think the practical way set.discard has been used and expectations of it are very much the definition of what discard is for.

I’m sorry if I’m too passionate or wordy, by the way! Succinctness is not something I’m good at. :slight_smile:

TL;DR: container.discard has always been safe and predictable, container.remove has always needed attention of some kind… and now we propose to make both of them take pointless brainpower away from the real problems to guard against a new kind of footgun where we mentally apply setexpectations on a list container. Please keep Python simple to reason about.

To make sure I understand your concerns, one is that set.discard() is O(1) while list.discard() would be O(N). The other concern is that up until now discard() has been isolated to set and has the (not explicitly documented) invariant that s.discard(x); assert x not in s, and alist.discard() will not have this invariant?

Would it be fair to summarize your position that adding dict.discard() and list.discard() will not actually provide a unified/common operation because the semantics will be fundamentally different?

Regarding algorithmic complexity, that is roughly it. I consider it an optimized operation that perfectly matches set semantics, whereas removeidentifies a more generic removal operation that can be implemented as an expensive search with a variety of extra complexity depending on the data structures characteristics.

If I see list.remove used, I usually check whether I might already have the index so I can delit instead. It is for the same reason that I try to structure my code to avoid list.insert so that I can use list.append instead. There’s nothing wrong with the capability since you need it sometimes, but it is not the desired way to interact with the data structure. If you have an index handy already, you’d usually want del to be most efficient about your deletion! In the same vein, offering an alternative to the technically already less-ideal list.remove that performs similarly except sounds like a more efficient set seems like it not only muddies the water of the specialized operation of set.discard, but also encourages wrong usage of the list data structure.

Given how ubiquitous lists are in every part of Python compared to sets, I’m not sure that is desired. I always liked the fact that the operations subtly steer me into thinking properly about the right way to use them due to the operations they support. Actively supporting alternative modes of less-optimal access goes counter to that in the name of what is really nothing more than a convenience function that will have its fair share of surprises that list.remove already forces us to actively consider.

Exactly. Depending on the kind of implementation of list.discard, there is either…

Remove the first found item (like remove), resulting in

  • Item not in list, nothing changes, maybe returning a value to say nothing was removed, item not in list, OR…
  • Item in list one time, item is removed, returning a value to say something was removed, item not in list, OR…
  • Item in list multiple times, first item is removed, returning a value to say something was removed, item remains in list.

If someone wants to remove all the items from the list, you might end up with intuitive yet badly performing code like

while apples.discard(rotten): pass  # a new search for every iteration

Remove all occurrences of item (to approximate set.discard), resulting in

  • Iterate over entire list, items removed one by one, likely optimized to minimize unnecessary shuffling, no more item in list.

If for every occurrence in my code of variable.discard(x)I need to start wondering whether it is a list and what result the three possible outcomes will have on my code, I end up with both extra cognitive load, and with me frequently putting in the sort of boilerplate we are already expecting from list.removeexcept differently. The benefit of list.remove is that not thinking about the code paths and meaning is never an option; it is very much part of the contract. But the popular first option for list.discardis created primarily because people don’t want to deal with that complexity to begin with. Its intended existence is innately self-contradictory as a result, and we will exchange the burden of extra if/try checks for the uncertainty and confusion of whether we want and/or need to apply any kind of special handling at all in our logic.

Exactly. We have survived 20+ years without list.discard, and adding this confusion to the mix for the sake of saving one line is a very daring and silly approach IMHO.

If we want a non-throwing list.remove, I champion giving it a different name like sequence.ditchor some other reasonable synonym (toss?). While the three different varieties of possible outcomes would still be the same (and in my opinion still not desired), we would at least avoid all the confusion over set semantics, making that much a non-issue at least.

3 Likes

This proposal has now been written up as PEP 851, let’s continue this discussion in the PEP thread at PEP 851: Add list.discard().