Nwise itertools

Ideas
43

This seems pretty important though. If the proposed algorithm is of the wrong complexity for the cited popular use-cases then it sounds like it would be misleading at best to include it in the standard library?

2 Likes
44

But that first example is about a different sliding window concept, one whose entire point is optimization. Where using the itertool proposed here would be detrimental or not even work at all.

1 Like
45

I agree with opinion that if this was to be added, it would be more beneficial to have full more_itertools.windowed functionality.

For my personal use, I found that even that one is not “complete” in functionality for my needs, so I ended up adding one more argument:

def windowed(seq, n, fillvalue=None, step=1, n_partials=Auto):
    """
    Args:
        seq : list, tuple
        n_partials : int
            Auto: behaviour as per mitl.windowed
              -1: will do maximum partial elements
    """
    ...

In [14]: btl.windowed([1, 2, 3], 4)
Out[14]: [[1, 2, 3, None]]

In [15]: btl.windowed([1, 2, 3], 4, n_partials=0)
Out[15]: []

In [16]: btl.windowed([1, 2, 3], 4, n_partials=1)
Out[16]: [[1, 2, 3, None]]

In [17]: btl.windowed([1, 2, 3], 4, n_partials=2)
Out[17]: [[1, 2, 3, None], [2, 3, None, None]]

In [18]: btl.windowed([1, 2, 3], 4, n_partials=3)
Out[18]: [[1, 2, 3, None], [2, 3, None, None], [3, None, None, None]]

In [19]: btl.windowed([1, 2, 3], 4, n_partials=4)
Out[19]: [[1, 2, 3, None], [2, 3, None, None], [3, None, None, None]]
46

For what it’s worth, I (the more-itertools maintainer) wrote the original more_itertools.windowed function. In retrospect, I regret adding the fillvalue and step parameters - I should have just put in examples where I composed a solution out of the other itertools.

I won’t take them away - lots of projects use windowed, and I won’t break them even with a major version change. But I wish I hadn’t complicated the code originally!

47

That makes sense. Didn’t think there were good (performant and concise) solutions for it. Would you be kind and share solution emulating the full behaviour composed of other tools?

48

For the fill values, you can pad the input with chain and repeat. step is more complex, but the code here uses islice, map, and chain.

1 Like
50

Sorry I have been busy with school. I want to write up a bit of a better proposal, and see what we can do to fit everyone who has commented’s opinions in the spec. I will try to see if we can have a performance testing plan as well, so we can be more sure of the implementation.

51

Thanks for this. This is much nicer recipe than the current one IMO. Unfortunately, wasn’t able to extend it for n_partials parameter. At least not yet.

Apologies for mixing in another thread, but I think Itertools.ilen(iterable) is part of the same book. I am curious what is your opinion on ilen addition to itertools (from the perspective of more_itertools maintainer)?

52

I have played around with your recipe and it works very well. I even managed to add parameter that I needed.

However, it still seems that it is difficult to wrap windowed function without arguments to provide them from outside without complicating matters more than they currently are.

53

Re: ilen, I’m happy to have it in more-itertools. I probably won’t add any extra arguments or features to it.