Change file name to correct syntax when rotating logs

Python Help
1

hi all,

so i have log rotate working but it save them as

password.log.{date} instead of
{date}.password.log

heres pic of what i mean

password

can i change this please

thanks,
rob

2

Hi,

can you in the code reverse the order in which they are displayed? Does the package provide this option?

If not, you may potentially perform the changes manually as in:

orig_dsply = "password.log.2024-07-18_12_25-33"  # Simulates your original string
split_orig_dsply = orig_dsply.split('.')

print(split_orig_dsply [2] + '.' + split_orig_dsply [0] + '.' + split_orig_dsply [1])
3

sorry i should had put my code in

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
handler = TimedRotatingFileHandler(filename='logs\\password.log', when='S', interval=20, backupCount=5, delay=True)
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s', datefmt='%Y/%m/%d %H:%M:%S')
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
logger.addHandler(handler)

this rotates the logs but as said instead of saving in this syntax
{date}.password.log
it saves like this
password.log.{date}

4

I believe you can do this by subclassing the handler and writing your own namer to generate the filenames you want. I haven’t done this before, though.

5

Following the tutorial from here:

I ran this simple little test script, and I am able to print according to the desired order:

import logging

# create logger
logger = logging.getLogger('password')
logger.setLevel(logging.DEBUG)

# create console handler and set level to debug
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)

# create formatter
formatter = logging.Formatter('%(asctime)s.%(name)s.%(message)s', datefmt='%Y-%m-%d_%H-%M-%S')

# add formatter to ch
ch.setFormatter(formatter)

# add ch to logger
logger.addHandler(ch)

# 'application' code
logger.debug('log')

Would you be able to edit it and incorporating the line(?):

handler = TimedRotatingFileHandler(filename='logs\\password.log', when='S', interval=20, backupCount=5, delay=True)
6

You have misunderstood what they are requesting. It is not about formatting the messages, it’s about the name of the file.

7

From the OPs original post:

That to me appears as if the ordering is out of the desired order.

8

Yes, they want to change the format of the file name. How does your example code address this?

9

From the example, I am showing that the desired order is being displayed:
date → password → log
vs.
password → log → date.

10

The point is not the format of the log messages. It’s the name of the log file.

12

I could not find a way from the handler TimedRotatingFileHandler documentation where you can append the date in front of the base file name versus the standard of appending it at the tail end.

Here is a potential work around.

import os

def update_logfile_names():

    # Update it to the directory where your log files are located
    update_log_names = r'/Users/your_comp_name/Desktop/logs'

    for file_name in os.listdir(update_log_names):

        period_count = 0

        for char in file_name:
            if char == ".":
                period_count += 1

        if period_count > 1:

            split_file = file_name.split('.')

            if split_file[0] == 'password':
                new_name = split_file[2] + '.' + split_file[0] + '.' + split_file[1]

                os.rename(file_name, new_name)

You can periodically or at any time in your program call this function to update the file names to your preferred naming convention.

new_file_name

There is another way where you can monitor the directory directly and update the filenames on the fly but it is a bit more complex. I kept it simple here.

Here is a link where it explains it if you want to perform this action automatically:

Create a watchdog in Python to look for filesystem changes - GeeksforGeeks

13

The correct way to do this is the method I suggested earlier: providing a custom namer for the TimedRotatingFileHandler

Before I suggested subclassing, which feels cleaner to me, but simply assigning to the namer attribute works just as well:

import logging
import logging.handlers
import os
import time


def flip_name(log_path):
    """flips the file name of a log file to put the date in front"""
    
    # normally it's the filename.timestamp, see
    # https://github.com/python/cpython/blob/7b36b67b1e585c541b418eaa190c56e73f9a2d87/Lib/logging/handlers.py#L440
    # so we'll split on the last '.' and flip it around
    
    log_dir, log_filename = os.path.split(log_path)
    file_name, timestamp = log_filename.rsplit(".", 1)
    return os.path.join(log_dir, f"{timestamp}.{file_name}")


if __name__ == "__main__":
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.DEBUG)
    handler = logging.handlers.TimedRotatingFileHandler(
        filename='example.log', when='S', interval=20, backupCount=3, delay=True
    )

    # here's where we use our custom namer
    handler.namer = flip_name

    logger.addHandler(handler)
    
    for i in range(10):
        logger.debug(f"it's time for log {i}")
        time.sleep(10)

Give me the set of log files:

(base) ➜  Desktop head *log
==> 2024-07-18_18-00-06.example.log <==
it's time for log 0
it's time for log 1

==> 2024-07-18_18-00-26.example.log <==
it's time for log 2
it's time for log 3

==> 2024-07-18_18-00-46.example.log <==
it's time for log 4
it's time for log 5

==> 2024-07-18_18-01-06.example.log <==
it's time for log 6
it's time for log 7

==> example.log <==
it's time for log 8
it's time for log 9
14

Ok, yes, this makes sense.

However, as great as documentation is, unless it is accompanied by examples, it makes for a much drier read. Once I briefly skimmed over it, I was off to other pastures. Thus, my alternate solution.

15

hi @jamestwebber if i use “namer” can i still leave the date/time attribute in the filename and just change the word “password.log” at the end of the filename?

16

I’m not sure what you mean. The name password.log is what you input in your code–so of course you can change that, by passing in something different.

If you want to customize the name further: The namer can be anything you want. It will receive a string that looks like "{filename that you passed}.{timestamp}" and your function can do what it wants, with this caveat from the docs:

17

ok done using loguru

logger.remove(0)
logger.add("logs\\{time:YYYY-MM-DD-HH-mm-ss}-password.log", format="{time:YYYY/MM/DD HH:mm:ss} | {level} | {message}", rotation="1 day", retention="2 days", delay=True)