To be clear I’m still -1 on actually changing anything but to entertain the thought.
If this was to be supported the logical place in my mind would be to treat ranges as sets.
They already support effective __contains__ check as in 10**10 in range(1, 10**11) which makes them set-like.
Any skip mechanism would still have to deal with things like the order of skips and if they overlap or not as well as the step argument if they are to take arbitrary range-objects as arguments.
Given that the more principled implementation would then be to implement the set-theoretic operators, i.e. union(|), intersection(&), and in this case symmetric difference(-) to be able to write it as for i in (range(100) - range(30, 41) - range(55, 66) - range(80, 91)).
It would probably have to keep some internal chaining / tree of range references if we want to avoid materializing the full sequence and I very much feel that complexity is not worth it but well within what’s technically feasible (and what current suggestion would need to support anyway if ranges with skip should be allowed as arguments to skip itself.)
You can literally get wheels on PyPI. People are not forced to reinvent stuff just because it isn’t in the standard library. You also didn’t respond to my question about PyPI. Do you maybe not know about it?
Great point! That’s a very creative approach, but doesn’t the set-theoretic analogy (-) limp a bit because of ordering? (As you already said.)
Ranges are ordered sequences, whereas sets are inherently unordered. A loop depends entirely on a predictable, sequential order. If we treat ranges as sets, we conceptually lose that sequential progression and the deterministic behavior of the ‘step’ argument.
This is exactly why I initially proposed a .skip() method instead of operators. With range(1, 101).skip(...), the main range remains the master of the sequence—it defines the start, the end, the step, and the explicit order. The .skip() part acts merely as a directional filter during iteration, rather than converting the structure into a complex, unordered set-like tree.
Wouldn’t a sequence-maintaining method like .skip() be much closer to the actual nature of a range loop than a set operation?
Using sets would also mean that you need to have all the elements in memory instead of generating it on the fly (like what range does). If you also have large ranges, you are paying for iterating over them multiple times.
Aren’t __contains__ checks on range objects pretty cheap?
You could just filter the range:
def skip(n: int):
return n not in range(50,60)
for i in filter(range(0, 100), skip):
...
You’re still iterating over the whole range, but now you can modify that skip function to filter out anything you don’t want without worrying about storing a set in memory.
The calculations can be performed beforehand. You simply compute the subranges:
from itertools import chain
def subranges(Left, Right, excluded):
# sort + merge
excluded.sort()
merged = []
for a, b in excluded:
if not merged or merged[-1][1] < a:
merged.append([a, b])
else:
merged[-1][1] = max(merged[-1][1], b)
# build boundaries
boundaries = []
current = Left
for a, b in merged:
if current < a:
boundaries.append((current, a))
current = max(current, b)
if current < Right:
boundaries.append((current, Right))
# build ranges
subranges = [range(*subrange) for subrange in boundaries]
return subranges
Left, Right = 0, 100
excluded = [(50, 61), (75, 81), (10, 20), (18, 25)]
for i in chain(*subranges(Left, Right, excluded)):
print(i)
Not fully tested, but the main point is to offload complex calculations to code rather than perform them mentally.
I didn’t refer to the iteration order of ranges. Keeping direction of leftmost operand would be the practical choice.
This has the same issue in that we arbitrarily have to decide that the first set decides the order. My point wasn’t that it was a problem to decide but rather that your skip proposal has to decide and solve all the same corner cases and their semantic complexity as the full operator case but is much less general.
If you are ok with an approach that iterates the full range and doesn’t effectively “jump” to the next member but instead checks it against each exclusion you can skip the tree part yes. If it doesn’t materialise the skipped ranges it will have problems with nesting though:
# Should this work and what should it print?
print(list(range(1, 100).skip(range(20, 40).skip(25, 35))))
Those are not unsolvable either but for something shipped in the standard library people will expect it to handle even the complex cases sensibly (even when the code for most eyes wouldn’t be ).
That’s actually often one of the the main reasons Ideas here get push-back with preference to short patterns. With patterns the tradeoffs and corner cases are directly visible and can be confirmed but if it’s a case where there are a lot of arbitrary decisions that can be made it forces users to look them up in documentation (or worse code).
I don’t have the source but remember reading that Bjarne Strousroup left out a power operator partially due to the question if it should be left or right associative. Developers generally had a clear confident expectation of a right answer, but in both ways (mathematically it’s assumed to be right associative which makes sense but all other binary operators are left associative).
Programming languages introduce a bit of a quirk that mathematicians don’t have to worry about, and that’s evaluation order. In the expression 2**3**4 a mathematician knows that this is the same as 2**(3**4) and not (2**3)**4, but only a computer programmer will ask which of those integers should be evaluated first. Python gives us one plausible evaluation order, but I can also see the validity of doing the opposite:
>>> class X:
... def __init__(self, val):
... print("Creating", val)
... self.val = val
... def __pow__(self, other):
... print("Raising", self.val, "to", other.val)
... return self
...
>>> X(2)**X(3)**X(5)**X(7)**X(11)
Creating 2
Creating 3
Creating 5
Creating 7
Creating 11
Raising 7 to 11
Raising 5 to 7
Raising 3 to 5
Raising 2 to 3
<__main__.X object at 0x7fd3830d6050>
So all of the operands get evaluated left to right, and then they get paired up from right to left. This is in contrast to addition, where they only get evaluated as needed, since they get paired up from left to right. (It’d create 2 and 3, then add them, then create 5, etc.)
In this particular case, though, I really think that it would be fine with using simple subtraction. Implementing an object that tracks a set of subranges and supports addition and subtraction with range objects would be relatively straight-forward and would not run into weird evaluation order problems. But if I were doing it myself, I would figure out what works for my project, which might result in bizarre APIs that definitely don’t belong in the stdlib, such as R(1, 101) - [40, 50] or even allowing the subtraction of a single integer from a range, depending on what’s needed. Fortunately, Python gives us the tools to make these things
I, and many other people that have commented before, have never had the need to use this “common” looping pattern - so, perhaps it is not as common as you think.
When something is added at the built-in language level as you propose:
Someone has to write the code, and maintain it.
Someone has to write various tests for it. As what you propose is something which may suffer from off-by-1 problem, multiple tests should be written, sot that it is tested exhaustively.
Someone has to write the documentation for it.
The Python documentation is translated in many languages, with new languages added all the time. Any change to the documentation requires additional work from the translators.
All of the work mentioned above is done by volunteers. Unless something is truly common pattern as demonstrated by a wide range of active participants on this forum, it is hard to justify trying to impose this additional work on volunteers for little benefit to the greater Python community.
Using a subsequence of an interated stream is sufficiently common that Python provides two general-purpose methods: the filter transformer filter(keep_function, iterable), which was omitted from the original post, and comprehensions f(x) for x in iterable if <keep_expression>, which were included. Range has a step arg to efficiently keep only 1 of every ‘step’ items. Range does not need a new special syntax for filtering that would be even less used.
Here is an additive approach equivalent to chaining.
>>> for block in ((0, 5, 2), (9, 11)):
... for i in range(*block):
... print(i)
...
0
2
4
9
10
The implementation detail in question is that they iterate in hash order, and that integers hash to themselves. So the suggestion does technically work (at least in some python implementations), but I wouldn’t want to rely on it haha
Small integers do. Larger ones will hash down into a more manageable range, which might not be consecutive (they may have some runs of consecutive numbers but it’ll be disjoint in places). Of course, none of that is a language guarantee.
existing sets would have to materialize the ranges (i.e. eagerly create one instance/reference of each number in the range).
Having a multirange class that would implement the “set” protocol, would be cool.
Still, I believe (1) this is not a frequent needed pattern and (2) the current syntax and features of the language are more than enough to cover the eventual need. - so, for this proposal as a whole “I am -1” .
A small pypi package with an “addable and subtractable” MultiRange class, like the idea by Chris Angelico, though, would be nice!
).