Trick: Faster functions (by using generators)

An extra nice use case: Packed arguments.

Let’s say you have this:

xs = [2, 3, 4, 6, 8, 9]

And you want to group into runs of consecutive numbers:

[[2, 3, 4], [6], [8, 9]]

A common solution is to group by difference to index:

groupby(enumerate(xs), lambda ix: ix[1] - ix[0])

That was so much nicer in Python 2, where we could do this:

groupby(enumerate(xs), lambda (i, x): x - i)

And with the generator way, we can bring that back:

def send2():
    i, x = yield
    while True:
        i, x = yield x - i

And it saves me some more nanoseconds per call :smiley:

116.5 ± 0.4 ns  send2
122.8 ± 0.6 ns  send1
146.7 ± 0.6 ns  normal

Python: 3.13.0 (main, Nov  9 2024, 10:04:25) [GCC 14.2.1 20240910]

The code:

normal = lambda ix: ix[1] - ix[0]

def send1():
    ix = yield
    while True:
        ix = yield ix[1] - ix[0]
send1 = send1()
next(send1)
send1 = send1.send

def send2():
    i, x = yield
    while True:
        i, x = yield x - i
send2 = send2()
next(send2)
send2 = send2.send

funcs = [normal, send1, send2]

from timeit import timeit
from statistics import mean, stdev
from itertools import repeat, groupby
from collections import deque
import sys
import random

for f in funcs:
    print([[x for _, x in g] for k, g in groupby(enumerate([2, 3, 4, 6, 8, 9]), f)])

n = 10**5
xs = range(n)
times = {f: [] for f in funcs}
def stats(f):
    ts = [t * 1e9 for t in sorted(times[f])[:5]]
    return f'{mean(ts):5.1f} ± {stdev(ts):3.1f} ns '
for _ in range(100):
    random.shuffle(funcs)
    for f in funcs:
        t = timeit(lambda: deque(groupby(enumerate(xs), f), 0), number=1) / n
        times[f].append(t)
for f in sorted(funcs, key=stats):
    print(stats(f), {normal: 'normal', send1: 'send1', send2: 'send2'}[f])

print('\nPython:', sys.version)

Attempt This Online!