Logging - track maximum level seen?

I’m using 3.11 on Debian. Experienced Java dev with some Python but still learning.

Does the logging module keep track of the maximum level seen? Ideally I’d be able to query the logger instance for the highest error level encountered so far.

I’ve found only one post on the internet, from 2019, which suggests using a custom handler to do this. The problem with that method is that the max level is stored in the handler. It doesn’t appear to be easy to get a reference to the handler from just the logger. I see a module function logging.getHandlerByName() method, but no way to give a handler a name (no constructor argument or setName() method).

Also, a handler does not expose a reference to the logger(s?) to which it’s attached, otherwise the handler could record the info in its owning logger.

I could subclass logging.logger and use logging.setLoggerClass(). In that case would it be sufficient to override just the logger.log() method, or would I have to override all the logging methods? Is that the only way?

This all seems really hard for something that should be (IMHO) part of the basic functionality…

There is a set_name method which AFAICT from the source code puts the handler in a dict where it can be found by logging.getHandlerByName. Inconsistent naming in the logging module is something to watch out for.

Reading a few lines down, that’s also part of the implementation of a name property.

On the other hand, this doesn’t seem to be documented. Probably a lot is missing from the documentation.

Hmm. It doesn’t read to me like the issue was properly understood. It combined a feature request to add a constructor parameter and a bug against documentation to mention the setter, and it was closed because… the property is “exposed”? Which doesn’t even seem to mean it’s documented, at least not on the website. name, get_name and set_name all appear in help(logging.Handler), but without any description.

2 Likes

Hmm… methinks I asked too many questions at once; sorry, I should know better :slight_smile:

Regardless, below is my solution, which seems to work. I stepped through the calls to the logging module and all looks right.

My question: Is this correctly implemented, or have I misunderstood something about Python classes?

import logging
import sys

# Simple subclass of logging.Logger to track the maximum severity level seen

class CFLog2(logging.Logger):
    def __init__(self, name, level=logging.NOTSET):
        logging.Logger.__init__(self, name, level)
        self.maxLevel = 0
        
    def _log(self, level, msg, *args, **kwargs):
        super()._log(level, msg, *args, **kwargs)
        self.maxLevel = max(level, self.maxLevel)
        
# Test code
if __name__ == "__main__":
    logging.setLoggerClass(CFLog2)
    
    log = logging.getLogger("CF")
    log.setLevel(logging.DEBUG)
    
    handler = logging.StreamHandler(sys.stdout)
    handler.setLevel(logging.DEBUG)
    formatter = logging.Formatter('%(asctime)s %(name)s %(levelname)s %(message)s')
    handler.setFormatter(formatter)
    log.addHandler(handler)
    
    log.info("Test Info")
    print(log.maxLevel)
    
    log.error("Logged error")
    print(log.maxLevel)
    
    log.warning("Logged warning")
    print(log.maxLevel)
    
    log.critical("Logged critical")
    print(log.maxLevel)

Output:

2024-05-04 15:35:14,893 CF INFO Test Info
20
2024-05-04 15:35:14,893 CF ERROR Logged error
40
2024-05-04 15:35:14,893 CF WARNING Logged warning
40
2024-05-04 15:35:14,893 CF CRITICAL Logged critical
50

Not at all; I’d say the rest of us got sidetracked.

The only suspicious thing here is overriding _log which presumably isn’t meant to be part of the interface. But then, the documentation doesn’t seem to say anything about how users are intended to subclass Logger, even though logging.setLoggerClass is provided specifically to support doing so.

You could, I suppose, have your custom Logger instantiate a custom Handler and hold onto it in a hard-coded way, then add a property to expose the handler’s tracked “max level” value. Something like:

class CFLog2(logging.Logger):
    def __init__(self, name, level=logging.NOTSET):
        logging.Logger.__init__(self, name, level)
        self._level_tracker = MaxLevelHandler() # left as an exercise
        self.addHandler(self._level_tracker)

    @property
    def max_level(self):
        # perhaps using a property provided by the handler, in turn.
        return self._level_tracker.max_level

But with everything else going on with logging, it hardly seems worth the effort to be so “neat” and OO-dogmatic about it.

But with everything else going on with logging, it hardly seems worth the effort to be so “neat” and OO-dogmatic about it.

Agreed. There seem to be several ways to attack this, and I think mine is simplest, subclassing only Logger instead of Logger and Handler. But as I’m still learning how to impedance-match between my Java brain and Python patterns, if anybody else wants to comment I’m all ears.

I think ideally the low-level _log() method should be exposed for just this type of subclassing (and renamed to something like raw_log or whatever would be Pythonic). This smells like a possible enhancement request.

Well, part of the problem is that a lot of the older parts of the Python standard library follow Java idioms too much :wink: IIRC, logging is actually based on Log4j in some sense. (I don’t use it and hardly do anything that could be called “logging” in my own projects, so I can’t really explain more.)