Asyncio `as_completed`

Python Help
1

I’m trying to understand what’s happening with the as_completed function as I want to use it to update a progress bar after a file is scanned but it appears to “stall” after 8 coroutines are run.

completed = 0
coros = [scan_file(file_to_scan) for file_to_scan in files]
print("count: ", len(coros))
for coro in asyncio.as_completed(coros):
    info = await coro
    completed += 1
    print(completed)

Prints

count: 200
1
2
3
4
5
6
7
8

pauses for some time and the prints all the other numbers in a rush at the end. Is this as expected, because of the print, or for some other reason? Should I use a different function such as wait called until pending is empty?

2

Most likely there’s a bug in scan_file(). Does it do anything that uses the number 8? (Or 7, or 9?)

3

There’s no 8s in the scan_file function (I think even I may have spotted that), I’ve probably been hit by the asyncio is not a magic bullet bullet.

I’ve switched to the concurrent.futures.ProcessPoolExecutor which does what I want and gives a nice speedup. Code is here