How to implement logging properly?

Python Help
1

Hi,

I have several modules, each with a class. I want to be able to turn off the messages for each module/class independently, so that my screen does not get flooded with messages or that I can see the messages that matter.

My idea is:

  1. Create a logger instance for each class and make it a class attribute (not instance attribute)

  2. Create a logger manager class to store all those attributes. This class would be like a logger factory and would also enable me to set logging levels as I wish from my main function.

Of course the class would have to be something like a singleton, and shared across all the code. Is this a good design? Is there a better way to do this?

Thanks.

2

That’s logically what I have done for my non-trivia apps.

I’m guessing that you are thinking to process a command line arg like --debug=topic1,topic2 with your scheme to only have the debug messages you want and indeed a --version=topic3,topic4 etc.

I have found that some of my classes benefit from multiple log streams that I can control. I have a mapping of names to loggers that the UI can use with the --debug and --verbose options.

1 Like
3

The logging module already acts like the singleton factory you want. logging.getLogger takes a string argument, and always returns the same instance of Logger whenever the same argument is used. This name can be used by logging.fileConfig and logging.dictConfig for logger configuration, and the names live in a hierarchical namespace, e.g., configuration applied to getLogger("a") applies to getLogger("a.b"), getLogger("a.c"), etc.

1 Like
4

Dear Clint,

Thanks for your reply. I am currently using logzero and I imagine that it is also a factory of loggers like logging. However, wouldn’t it be better to have a kind of interface to the logger, such that for instance, we can switch loggers more easily in the future or have a central place where our loggers are configured for the whole project? I was thinking of something like:

import logzero

from logzero import logger as log
#------------------------------------------------------------
class log_store:
    d_logger = {}
    d_levels = {}
    @staticmethod
    def add_logger(name=None):
        if name is None:
            log.error(f'Logger name missing')
            raise
        elif name in log_store.d_logger:
            log.error(f'Logger name {name} already found')
            raise

        level          = logzero.INFO if name not in log_store.d_levels else log_store.d_levels[name] 
        logger         = logzero.setup_logger(name=name, level=level)
        log_store.d_logger[name] = logger

        return logger

    @staticmethod
    def set_level(name, value):
        log_store.d_levels[name] = value

    @staticmethod
    def show_loggers():
        log.info(f'{"Name":<20}{"Level":<20}')
        for name, logger in log_store.d_logger.items():
            log.info(f'{name:<20}{logger.level:<20}')
#------------------------------------------------------------
5

logzero already implements such a store behind the scenes, just like logging. (In fact, logzero appears to be an interface to the logging module, and probably lets logging manage the store.)

When you call logzero.setup_logger("foo"), it returns a logger bound to the name "foo", creating it first if necessary.

2 Likes
6

Are you sure that is a good idea? Does it not entangle the logging implementation too much with those classes and lead to a lot of reduplicated boilerplate code? Perhaps not – perhaps in your project it really makes sense to do this, but in the projects I worked on I have always preferred having just one logstream with a uniform format.

Logging itself is a prime example of a cross-cutting concern. So, imo, it’s good to actually also treat it that way in the implementation – which implies not tieing it (too closely) with particular business logic. Just reading a bit about aspect-oriented programming might give you more ideas about how to proceed.

1 Like
7

Dear Hans,

Thank you for your reply and the links. My idea was that maybe it’s a good idea to have some sort of interface to the logger, such that if we want to move to a different library (e.g. because the old one is not maintained anymore) we could just change things in the interface instead of everywhere in the code. However I am not an expert and I will go through your links and consider this more carefully.

8

That smacks of premature abstraction. Worry about that when you change logging library/frameworks; the amount of work needed may not even be that much.

1 Like