Logging QueueHandler prepare() method mutates record.msg. Why?

Python Help
1

The logging module has a class called a QueueHandler which accepts log record events and puts them onto a queue. The records are then typically read from the queue by a QueueListener object. I’m trying to generate functionality for logging dictionaries instead of strings so I might make calls like logger.INFO({'dict_message': 'Hello World!', metadata: 42}). In this case I want the message which is a dictionary to get passed through to the endpoint handlers without being modified. Unfortunately, however, the QueueListener puts the record through a prepare() method which essentially does record.msg = self.format(record.msg) using the default formatter for the QueueHandler object. The default formatter ends up turning my dict into a string.

The documentation for the method indicates that the purpose of the prepare() method is to ensure the queued record is pickleable. The documentation also specifically indicates that you may want to override this method if you are converting the record into a dict (which is what I’m doing) so I’ve found an approach that works for my use case.

But my question is essentially: Why does the QueueHandler need any prepare method at all? Why not just queue whatever record is passed in directly? Why should the QueueHandler have anything so say about whether a record is pickleable or not? That seems out of scope for its job. Perhaps I’m missing something about queueing in python and not all types of objects can be put in a queue? So the prepare method is doing some safety to make sure the queueing will go ok? If this is not the reason then what is the reason for the prepare() method on QueueHandler?

2

Another justification given in the docs is that the prepare method modifies the record to prevent further formatting. For my use case it was somehow natural to think that the QueueHandler would do no formatting and the downstream handlers would be responsible for their own formatting. But I guess I could see an argument that the QueueHandler does formatting and then downstream handlers should do no formatting… The possibility of chained handlers makes it non-obvious which one in the chain should be responsible for formatting I guess.

3

Why not implement your own queue of records and handlethe formatting when you pull records from the queue?

Why does logger need to do this for you?

4

This is pretty much what the QueueHandler and QueueListener does and it does it to my liking. The only thing I don’t like is that the QueueHandler mutates the LogRecord before passing it on. I can avoid this mutation by overwriting the prepare() method of my child QueueHandler but I don’t understand why the parent behavior coded into QueueHandler is desirable.

So the question isn’t: “How can I do something”, I already know the answer to that. The question is “Why is something this way?”