I have an application with several modules. Each uses the logger as follows:
logger = logging.getLogger(__name__)
logger.setLevel(vsw.vsw_globals.log_level)
# https://docs.python.org/3/library/logging.html#logrecord-attributes for formatting options
formatter = logging.Formatter('%(asctime)s.%(msecs)03d: %(levelname)s: \
%(threadName)s: %(name)s.%(funcName)s().%(lineno)d: %(message)s', datefmt="%Y-%m-%d %H:%M:%S")
stream_handler = logging.StreamHandler()
stream_handler.setFormatter(formatter)
file_handler = logging.handlers.RotatingFileHandler('/var/log/h3l/studywatch.log', maxBytes = 5000000, backupCount = 4)
file_handler.setFormatter(formatter)
logging.Formatter.converter = time.gmtime
logger.addHandler(file_handler)
As you see, the maxBytes is fairly large, so it took a while to run into this issue. Then the log got big enough four backups were quickly made. Now whenever the logger writes something, the modules each write to one of the backup files. The main log file (studywatch.log) is only written to by the module containing the ‘main’ program.
Anyone understand this behavior? What am I doing wrong. I am using Python 3.8 and some of the modules are threads.
I looked at the issue 'RotatingFileHandler seems to write logs to the .1 file that is already rotated. It didn’t help though it suggested that threading has something to do with it. I cannot avoid threads. If that is the problem, how do I get around it?