The logging library provides methods with various severity levels from info to critical, and lets you set a log level so that messages with less than that severity won’t be printed. But sometimes (particularly when the thing you want to log about is the logging configuration itself) you’d like to unconditionally log a message, no matter what the current log level.
import logging
l = logging.getLogger()
mylevel = logging.ERROR
l.setLevel(mylevel)
# At this point I would like to unconditionally log:
l.unconditional(f'Set log severity level to {mylevel}')
# This message will always be printed, no matter what level has been configured.
Now, I could call critical to make sure the message gets printed, but that seems wrong somehow, as it’s not “a serious error” as the documentation of level CRITICAL states. It’s simply a message that I want to always output, because it describes the kind of messages that will be logged from now on.
Another example might be if you want to definitely print something just after opening a new log source, just to make sure it works before continuing with the application. (If there might be socket errors or file permission errors on writing, better to hit those immediately.) Again, it’s not a critical message, but you want to ensure it will be output every time.
While you could save the existing severity, downgrade to INFO, call the info method and then put the severity back, I think it is simpler and more explicit for logging to provide a method that will always log. It should be used sparingly, of course.
The existence of an unconditional log method does mean that you can’t be certain of turning off all log messages by setting a high enough log level. But that is true anyway: any code could change the level and print its message. The principle of consenting adults applies. (If you did want to disable all logging, there are other ways.)
When you are trying to debug the logging system and wondering why your log messages aren’t reaching their destination, a print statement can help a little bit, as far as showing that the code did reach that point, but it doesn’t help you diagnose the reason why the logs are going missing if the code is indeed running. Sometimes the process might be running in an environment where you don’t see its stdout or stderr.
An unconditional method would help you diagnose missing log messages by ruling out, at least, that the message has been suppressed because you made a mistake in the severity levels somewhere.
And if you want to log meta-information like “new chapter of log messages starts here”, again print isn’t quite it, because it will go to stdout whereas your logging might be going to a network socket or whatever.
You can control where print outputs to with its file= parameter. But I can see how it would be cumbersome to keep that in sync with the logging output.
First, let’s ignore the usual methods, as they are boilerplate for constructing calls to the logging method, log.
log does little more than check the current logging level before calling a nonpublic, unconditional logging method _log:
def log(self, level, msg, *args, **kwargs):
"""
[...]
"""
if not isinstance(level, int):
if raiseExceptions:
raise TypeError("level must be an integer")
else:
return
if self.isEnabledFor(level):
self._log(level, msg, args, **kwargs)
I would just skip the middle man and use _log directly.
This doesn’t address handlers using their own level configuration to simply ignore an incoming message, however. It might be sufficient to just pass a high-enough logging level to _log, or you might need to define your own handler that ignores logging levels itself. (Clearly, I haven’t looked too deeply into how the handlers operate.)
It’s still important to set the level if you use _log. Handlers can set different log levels – that’s how you can send debug to a logfile and info to stdout using the same logger.
There are multiple easy solutions, no need for any stdlib change.
For most folks who want to test their config, you can also just logger.log(100, "foo") and have done with it. The level name will be "Level 100". (100 is bigger than logging.CRITICAL, which is 50.)
For many, many people, print() is actually better, precisely because it doesn’t go through the logging machinery. If you want to get unconditional output without worrying about something like a log filter suppressing it, print() does that. And you can send stdout or stderr to the place of your choosing or use print() to write directly to a file.
print is often better, but not when the particular thing you want to investigate is the logging system itself.
I agree there are ways to log the message, by choosing a custom level of 100, or maxsize, or calling _log directly, or just using critical. The feature request is not to add a way to do it, but to add a documented and self-evident way to do it. The others seem a bit obfuscated.
The problem is that it’s difficult to be clear what “unconditionally log” means. If, as a result of a bug, a filter discards any messages containing the letter “e”, then you can’t debug this with any sort of unconditional_log("line XXX").
A custom log level higher than all of the other levels you use in your program is the obvious way to ensure that a message always gets sent through the logging subsystem. Maybe an explicit level.UNCONDITIONAL would be worth adding, but it’s a relatively minor convenience for a fairly uncommon use case. You could raise a PR if you think it’s worth it. Personally, I don’t.
But you’re still going to have to do some work if the problem is in how you’ve configured the logging system itself. Bypassing the logging system completely (i.e., print) is really the only answer in that case.
Maybe you have a “credential filter” which tries to detect and reject logs with long, high-entropy strings. And you want to test it by sending a high level log and confirming that the message doesn’t make it through.
The logging system is pretty powerful. If the solutions offered in this thread aren’t satisfactory, then I’m not sure what the feature request even is.
I suspect this issue could be resolved by using existing more sophisticated configuration of the logging module. For example, you could configure your test environment as desired wrt your source code and your expected application logging configuration. Then you could add an additional logger, e.g. logging_diagnostic_logger or something and configure logging to expose that logger at an appropriate level (either INFO or DEBUG based on what you’re describing).
Have you tried setting up a dedicated logger and specifically configuring it to log at the level you want?
The logging module configuration is pretty confusing in my opinion. But, it is powerful and it does follow some good principles (once you understand them). I also experienced a desire for “unconditional logging” while I was passing through the logging learning curve. But now that I’m over-the-curve I see that an unconditional logger would really undermine some of the principles of the logging module.
Sorry if this post is patronizing. I’m writing as if I were writing to myself a couple years ago when I was first learning about the logging module. Perhaps you, at this time, understand it much better than I did back then.
Thanks. This is all good advice. My situation is having inherited a large, unfamiliar codebase and trying to diagnose a bug where log messages go missing when running in a particular environment. While investigating, I want to change the code as little as possible, to not introduce other variables. I agree that ‘unconditional’ is a crutch and not good practice in the final codebase, but like many other things it can be an effective tool for debugging and narrowing down a problem. As others have remarked you can log with a custom severity, or call _log directly, so it’s not strictly necessary to have a named and documented method for this escape hatch.