Asyncio IocpProactor does not close

Python Help
1

Hello everyone!

I hit a wall with debugging, and I hope someone might point me in a right direction.

I use qasync to mix asyncio and Qt event loops. Basically, my app does not exit when all Qt windows are closed (it should do so).
Messages are being logged:

2023-10-01 22:35:37,406 DEBUG    (asyncio): <_IocpProactor overlapped#=1 result#=0> is running after closing for 1.0 seconds
2023-10-01 22:35:38,407 DEBUG    (asyncio): <_IocpProactor overlapped#=1 result#=0> is running after closing for 2.0 seconds
2023-10-01 22:35:39,422 DEBUG    (asyncio): <_IocpProactor overlapped#=1 result#=0> is running after closing for 3.0 seconds
2023-10-01 22:35:40,438 DEBUG    (asyncio): <_IocpProactor overlapped#=1 result#=0> is running after closing for 4.0 seconds

Environment:

  • Python 3.10.11 on Windows 10
  • PySide6 6.5.2
  • qasync 0.24.0

I don’t hit that problem on Linux.

My app entry point is as follows:

async def start_app():
    def close_app():
        close_future.set_result(True)

    close_future = asyncio.get_event_loop().create_future()
    app = QApplication.instance()
    app.aboutToQuit.connect(close_app, Qt.ConnectionType.SingleShotConnection)

    from launcher.launcher import LauncherWindow

    p = LauncherWindow()
    p.show()

    await close_future
    breakpoint()  # this breakpoint is reached ok


if __name__ == "__main__":
    qasync.run(start_app())

I’ve set a breakpoint at lib/asyncio/windows_events.py:884, and got the following:

(Pdb) c
DEBUG    (asyncio): <_IocpProactor overlapped#=4 result#=0> is running after closing for 872.9 seconds
> c:\users\user\appdata\local\programs\python\python310\lib\asyncio\windows_events.py(884)close()
-> next_msg = time.monotonic() + msg_update
(Pdb) pp self._cache
{1583221963024: (<_OverlappedFuture finished result=2840 created at C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py:502>,
                 <_overlapped.Overlapped object at 0x000001709F61F900>,
                 <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>,
                 <function IocpProactor.recv_into.<locals>.finish_recv at 0x000001709F61F9A0>),
 1583221963312: (<_OverlappedFuture finished result=0 created at C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py:502>,

                 <_overlapped.Overlapped object at 0x000001709F61FA20>,
                 <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>,
                 <function IocpProactor.recv_into.<locals>.finish_recv at 0x000001709F61FAC0>),
 1583221964608: (<_OverlappedFuture finished result=1149 created at C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py:502>,
                 <_overlapped.Overlapped object at 0x000001709F61FF30>,
                 <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>,
                 <function IocpProactor.recv_into.<locals>.finish_recv at 0x000001709F848B80>),
 1583224233056: (<_OverlappedFuture finished result=0 created at C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py:502>,

                 <_overlapped.Overlapped object at 0x000001709F849C50>,
                 <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>,
                 <function IocpProactor.recv_into.<locals>.finish_recv at 0x000001709F849CF0>)}



(Pdb) !self._cache = {}
(Pdb) pp self._cache
{}
(Pdb) c
DEBUG    (asyncio): Close <QIOCPEventLoop running=False closed=False debug=True>

Those objects in self._cache stay there seemingly infinitely and prevent loop from ending.
At this point I lack fundamental knowledge of inner workings of asyncio and IocpProactor.

My only thoughts are:

  • Maybe those hanging Future’s are there because some coroutines were not awaited upon?
  • But I run asyncio in debug mode, non-awaited coroutines should cause warnings? (no such warnings present)
  • Maybe those coroutines are not GC-ed, so warnings are not logged?
  • If so, then there are hanging references to coroutines somewhere.
  • What are those sockets? Does it indicate that root cause is some async net IO? Or is it some internal stuff to IocpProactor?

Thanks for reading this much. I think overall my question is: what to do next? What debug approaches to try?

2

I thought this was PyQt5…

3

Just in case anyone would find it useful: new version of qasync library (>=0.25.0) should be able to solve problems like described in the first post.

Also, this theory from the original post is likely incorrect:

And the link to the relevant issue in qasync repo: