Unless I’m mistaken, next is the only built-in function that changes the object it is called on, rather than return a new object. To me, this behavior seems surprising.
All next does is call the __next__ method of the iterator it is called on. What is the rationale for making __next__ a dunder? Wouldn’t it make more sense to just make iterators have a next method instead of __next__, and not have it be a built-in function?
To be clear, this is not an idea or a proposal to change anything, I would just like to understand why it works like this.
Python does a lot of these sorts of things with protocols. Want to know the length of something? Call len(x). Creating an object type and want to define how that object’s length is calculated? Define a __len__() method. Notably, the Python interpreter itself will call magic methods when needed (eg as a for loop iterates over something), giving an expectation that these method names belong to the language, and you should be cautious about creating your own.
Another reason is that the next() function has more power than the __next__() method needs to provide. When writing the method, all you have to do is return a value or raise StopIteration; but next(iterator, default) will catch that StopIteration and return the default instead. If, in the future, new functionality is added to the next function, EVERY iterator will automatically have that power, even if it means adding another parameter to next(); and adding a parameter to a widely-used method name is a nightmare for compatibility.
Incidentally, Python 2 called the magic method next(). The rationale for renaming this in Python 3.0 is in PEP 3114 and will give more details on the pros and cons of protocols (and contocols, if you have any of those).
Yes, but usually these functions don’t change the object they are called on. Sure, you could create a class with a __len__ method that does any number of changes to the instance, but I think most people would find that behavior surprising. next is the odd one out here, in that that this behavior is typical.
That’s fair, although I feel like that argument could be used to justify a lot of things being built-ins that aren’t.
Interesting, thanks. I started using python when 2.7 was already ancient history, so I don’t have a very good idea of the changes that were made for python3.
Not sure that’s really a pattern; the __i*__ dunder methods (in-place operations eg iadd, isub, etc) can mutate their arguments. (They have the option to not mutate, but the in-place methods exist for the cases where they can take advantage, eg lists.)
Out of interest, what? Maybe there’s something that could be improved.
I meant that in the specific case of magic methods which have a corresponding built-in function (abs/__abs__, len/__len__, hash/__hash__, etc), the method usually doesn’t change its instance. That’s certainly not generally true for all magic methods.
Again, not really a suggestion for anything to be added to the language, but since you asked
An example that comes to mind is a generalization of str.split to any type of iterator. I semi-regularly find myself wanting to divide iterators into smaller chunks according to some criterion. The criterion can be as simple as “n evenly sized parts” or something more complex like a pattern that reoccurs irregularly. Having a split built-in with a corresponding __split__ magic method that could be called like:
for chunk in split(myiter, condition):
...
where condition is a callable that returns True where the iterator should be split (or something, I haven’t really thought through the implementation very carefully). That would certainly be useful to me in many situations.
Ehh, sure, but those are such a small fraction of all dunder methods that the distinction doesn’t really make a lot of difference. It’s always possible to pick out one thing from a group and prove that it is the exception. Consider the US presidents and their uniquenesses.
That’s fair. Though strings are often special cases, due to their distinctly different needs; and by your description, a string wouldn’t actually split correctly on a multi-character condition. But there might well be value in something like this. IMO it’s not important enough to be a builtin, but it could go into itertools; and, depending how you write it, could actually be a relatively simple construct on top of itertools.groupby().
Iterators originally had a next() method and you were expected to call it directly. It was changed to a dunder as part of the transition to Python 3.0, with the builtin next() function backported to Python 2.6 to make it easier to migrate.
I don’t recall there being any grand reason for this, probably just to make it more consisent with protocols using a builtin function as the public interface and a dunder for the implementation.
E.g. we have len calling the __len__ dunder, not obj.len().
As for the question of next() being the only builtin that modifies its argument, delattr and setattr do that as well
I don’t think that’s an important distinction. The iterator design pattern is possibly the only major, builtin protocol that depends on having an object with internal mutable state. All(?) of the others are happy to operate on immutable objects as well as mutable ones, because they don’t need to modify the argument’s internal state. (Although they may.)
