Support async logging module

Ideas
1

Currently, the python logging module works synchronously.
Even if we write logic asynchronously with the asyncio module, the asynchronous functions in the
logger will be block to thread lock.

This is not a critical performance issue, so it would be low priority, but I think we should have a discussion about supporting asynchronous logging module function interfaces.

If someone know of a link where this is being discussed, please let me know.

2

The answer is to use a non-blocking handler and a separate thread for the actual file accesses. Does that not solve your problems?

3

Thanks for the answer, I found anothersolution thanks to you.
But the reason why I open this issue is curiosity
Why, by design, the Python logging module should be enhanced to support asynchronous, but there hasn’t been any discussion about it.
“from asyncio import logging as alogging”

I thought there would be an asynchronous log implementation like as above
so I looked it up, but I didn’t found so I was curious about that

like a Java log4j2 AsyncAppender , golang AsyncLogger

is it just lowest priority in python? or it not necessary or something like that?

4
  • The stdlib logging module is far older than the async stuff.
  • Noone has seen it as necessary enough to be added
  • Generally, before something like this is added to the stdlib a third party library should be created. I don’t think there currently is one? There is aiologger, but it seems to be unmaintained.
  • There are perfectly functional solutions (e.g. using QueueHandler)

If you want this, the best way to get it is to create it as a third party package, show that it is being used and then ask for it to be added to the stdlib.

3 Likes
5

I’ll add a few to the list

  • asynchronous cleanup has much weaker guarantees.
  • there isn’t support for asynchronous file apis in a cross-platform manner currently.
  • writing to sys.stdout (and many other streams) can meaningfully block the eventloop too.

The QueueHandler is the correct way to do this. Any way that “looked” async would just be defering to a thread executor for the io currently anyway.

1 Like