Optionally include pair of last and first element of iterator in `itertools.pairwise`

This is a link to wikipedia article garbage in, garbage out.

Please don’t use URL shorteners – it makes the link opaque and raises the suspicion of spams (for readers who don’t recognize your name in this forum)

2 Likes

Ah, thanks for editing that.

So what @Stefan2 is saying is that, yes, you ARE dismissing open file objects as broken, and pretending that they don’t exist. My point is, broken iterators exist and are quite common :slight_smile:

No, I’m not saying they don’t exist. Just that one shouldn’t expect things to work well when you give them something broken.

Yes, that was the point.

Why?

Also: Why are posts required to have more characters in them? This post would have been better as a succinct one-word response.

Um, can some explain how my comment linking to a Wikipedia article was “offensive, abusive, to be hateful conduct or a violation of our community guidelines”?

It WASN’T a comment linking to a Wikipedia article. It was linking to a URL shortener.

It was linking to Wikipedia. Via a shortener. And how does that make it "offensive, abusive, to be hateful conduct or a violation of our community guidelines”?

Lemme explain how things work. You post a link. I hover the link. At that moment, I have to decide whether to click the link or not.

Is it a link to Wikipedia? You didn’t say so. The ONLY way for me to find out where it links is to click it. Do I trust you enough to click the link? Every one of us has to make that decision, with zero information.

It’s different if you’re posting in a context where long URLs are actually a fundamental problem. I’ve used MUDs, where any text over about 60 characters long is going to get broken across multiple lines, and that’s a pain. URL shorteners are great there, but even so, not everyone will click on links that use shortener services, since they could point literally anywhere.

So, why did you not link directly to Wikipedia? I’m pretty sure the original link would have been fine, even with the comma. Let’s try it, just pasting the URL in and letting Discourse autolinkify it: Garbage in, garbage out - Wikipedia Looks fine to me. Of course, if you want to change the link text, it’s even easier: That’s a known concept, or whatever your link text was. There isn’t a lot of reason, on Discourse, to use a URL shortener; all I can think of is that a URL with mismatched parentheses in it might cause issues, but you can always escape them as %28 and %29. (Matched parentheses aren’t a problem, so Wikipedia on the Python language is fine.)

If there was a technological reason for needing the shortener, then it was simply a mistake, and hey, in the future you can go to less effort and get better results, so all’s great, right? But if making the link opaque was the point… then were you intentionally deceiving us as to the destination? Is that what you’re saying?

3 Likes

Jokes don’t work well if you can see the punchline coming, that’s all.

Besides, when it got hidden it had already been changed to a direct Wikipedia link ~2 hours before. So something else about it must be terrible and I don’t know what, so I don’t know how I’m supposed to edit it now to make it not terrible. I do believe it was a valid point, after all that’s why there’s a serious Wikipedia article about it.

An example of an existing itertool that doesn’t work well with a broken iterator:

from itertools import batched

class BrokenIterator:
  def __iter__(self):
    self.i = 0
    return self
  def __next__(self):
    self.i += 1
    if self.i == 3 or self.i >= 9:
      raise StopIteration
    return self.i

for batch in batched(BrokenIterator(), 5):
    print(batch)

Output:

(1, 2)
(4, 5, 6, 7, 8)

But I wouldn’t blame batched for that.

Maybe. But here’s a question. Why write something that looks worse and doesn’t work with broken iterators, when you could write something more expressive that DOES work with broken iterators? You’re fighting for something that doesn’t seem worth fighting for.

It doesn’t look worse to me (maybe because I’m used to that) and it’s less code and faster.

That’s not at all what it actually reads like. It reads like “for the first element in the iterator, do the following: yield that first element, yield the rest from the iterator, and yield the first element again”. (I might somewhat agree with your reading if I had written for element in iterator: instead of for first in iterator:.)

That… isn’t how most people interpret a for statement. It’s usually “for EVERY element in the iterator”. I’m not sure where you get this “for the first element” idea from.

That’s how people interpret the word “first”. And I get the “for the first element” from the for first in iterator:. Do you not see the resemblance? It’s almost exactly the same words. Just filled in the implicit words “the” and “element” to make it a proper English sentence.

You’re working in a programming language. Syntactic elements like for statements carry a lot of weight. Variable names carry some weight, but not enough to contradict the well-known behaviour of the actual statement you’re using.

I’m done discussing this. Do whatever you like, if you REALLY think that this is more readable than a simple try-except then do whatever, I don’t care. Just don’t try telling me that I should do it that way.

Leaning hard into idiomatic English at the expense of writing idiomatic Python doesn’t seem like a recipe for clear code. It’s also worth thinking about how the code reads for those who do not speak English as their first language.


Back on topic…

  1. Is this really a common need? The OP says it is, but it isn’t in my experience. To justify the addition to the stdlib, show some examples where this is needed or already implemented in existing projects.

  2. Why not add it to more-itertools first? The stdlib will always move more slowly than pypi libraries. There’s good reason to celebrate that – once something goes into the stdlib, it’s very hard to remove, so it’s usually good to prove out ideas on pypi first.

  3. The proposal would define a “circular iterator” (in the calling context) which is finite. I tend to read “circular” as meaning infinite, and I doubt I’m alone. I don’t have a better name to offer, unfortunately. But it’s worth thinking about alternatives.

2 Likes

I’d never do that (except maybe if I had a project of my own and you contributed to it).

I would rather keep this API simple and orthogonal. If needed, prepending and appending values is easily done with chain() prior to calling pairwise().

Note this has existed in more-itertools for a good while and no one there noted a need for extended functionality. That package also has the sliding_window() recipe for which pairwise is just the special case with n=2. That has been in common use and no need has arisen for a prefix or suffix.