Diagnostic logging: reinventing the wheel?


First of all, thanks a lot to everyone in this community for the helpful and insightful discussions!

Now, one thing that I have always missed in the otherwise very useful standard logging library is the ability to not invoke complex functions that are supposed to format objects and data structures as strings for diagnostic purposes only. That is, if I want to add a log.debug(...) statement that does some special formatting (str.join, only some members of some data structures, conversion functions, etc), I would very much like that special formatting to not be done if the program is run without verbose output, so that the debug-level message will not actually be logged.

A couple of years ago, in the true NIH spirit, I wrote a small library to help me with that - cfg-diag - pass a lambda to a diag() method. However, as time went by, I decided that it was not so good to be so different after all, so gradually I came back to the standard logging library…

…and a couple of days ago I wrote a kind of an extension to it, logging-std - add a new ‘I’ conversion modifier (see also the rendered documentation). It mainly consists of two parts:

Of course, any comments on the library itself are welcome. However, the main reason I am writing this post is to ask am I reinventing the wheel here?

  • is there a somewhat-standard logging library that provides a way to not invoke formatting functions unless the messages will actually be logged?
  • if not, is there a somewhat-standard implementation of a parser for the Python printf-like format strings that I might use instead of implementing my own, possibly poorly?

Thanks in advance for any ideas and pointers, and keep up the great work!


And yeah, I forgot to mention: I have not uploaded this logging-std library to PyPI yet mainly because it may turn out that I don’t have to, if somebody bonks me over the head with a “see, this is how you do it” reply :slight_smile:

Hi - I haven’t looked closely at your code, so please excuse any (lazy) misunderstandings.

My first reaction to your post is: Is this needed at all? The std lib logging library already applies lazy evaluation in log messages, so there doesn’t seem a need to do anything special for variable arguments?
(In other words, this solution seems a work-around for a problem that is not related to logging at all?)

Second question: Is all the code in your main formatter classes needed (just to add one new formatting indicator)? Isn’t there a much simpler way to implement this?

Third reaction: Adding a special format letter to do what you want to do here, doesn’t seem the right approach to me. If something special is needed, wouldn’t it then be better to define a special logging or formatter class that is a plug-in replacement of the std lib logging class but that also handles any callbacks or functions? (This way people using the new library don’t need to do anything special - apart from using it instead of std lib logging. It could easily be switched on and off, or replaced by other loggers, without having to worry about that extra format letter that makes it incompatible with other loggers.)(It’s not trivial but also not impossible to do sth like this. The new format letter may have the advantage of being easy to implement, but the disadvantage of tieing the user to this library and introducing this incompatibility.)

The following old Stack Overflow post also has some ideas that might be relevant:

Plus a reference to a stringlike library that supports lazy string evaluation in general.

As I read it, the main question was exactly whether there was such a thing in the standard library, so that indeed answers it :slight_smile: That said, OP should be aware that this uses the old %-style formatting, not .format-style formatting. (Of course, f-strings cannot be lazy, but .format can.)