If I have this kind of generator:
def special_gen(d): for n in count(1): if is_special(d, n): yield n
The call to is_special() takes time and I need to generate the values in increasing order.
I thought that a part of the solution would involve creating multiple generators and combining their results. Something starting like this (not in parallel):
PROCESSES = 4 def special_gen_mod(d, mod=1, offset=0): for n in count(1 + offset, mod): if is_special(d, n): yield n def special_gen(d): sub_generators = [special_gen_mod(d, PROCESSES, off) for off in range(PROCESSES)] yield from heapq.merge(*sub_generators)
But I am stuck on how to add the parallelism. I’ve read and reread the docs and googled but couldn’t come up with a neat way of creating a “parallelised generator”.
My question is also on Stack Overflow, but I posted here in the hope that someone already had ideas on
Thanks in advance