No debug messages going into log file

Python Help
1

hi all,

the below code works

logger = logging.getLogger("Rotating Log")
logger.setLevel(logging.DEBUG)
handler = TimedRotatingFileHandler(filename='password.log', when='S', interval=30, backupCount=5, delay=True)
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s', datefmt='%Y/%m/%d %H:%M:%S')
handler.setFormatter(formatter)
logger.addHandler(handler)

but none of the debug messages are going in the log file

thanks,
rob

2

The logging level must be set on handlers too.

1 Like
3

you mean like this

handler.setLevel(logging.DEBUG)
4

Yep.

If that still doesn’t worked, I got it working without much hassle, using an ordinary FileHandler, not that rotating one.

5

still not working, still not seeing debug messages ie messages that i see on console, i want to log the messages that i see in console aswell

logger = logging.getLogger("Rotating Log")
logger.setLevel(logging.DEBUG)
handler = TimedRotatingFileHandler(filename='password.log', when='D', interval=1, backupCount=5, delay=True)
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s', datefmt='%Y/%m/%d %H:%M:%S')
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
logger.addHandler(handler)
6

I don’t know how to use that TimedRotating thing but this:

import logging

logger = logging.getLogger("Rotating Log")
logger.setLevel(logging.DEBUG)
handler = logging.FileHandler(filename='password.log')
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s', datefmt='%Y/%m/%d %H:%M:%S')
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
logger.addHandler(handler)

logger.debug('Test debug message')

produces:

>py logging_test.py

>more password.log
2024/07/11 17:14:07:DEBUG:Test debug message
7

thanks but i want to get console messages aswell ie messages like this

python password.py
 * Serving Flask app 'password'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on https://127.0.0.1:5000
 * Running on https://10.100.1.10:5000
Press CTRL+C to quit
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 118-343-405
127.0.0.1 - - [11/Jul/2024 18:08:05] "POST /password HTTP/1.1" 200 -
127.0.0.1 - - [11/Jul/2024 18:08:07] "POST /password HTTP/1.1" 200 -
127.0.0.1 - - [11/Jul/2024 18:08:08] "POST /password HTTP/1.1" 200 -
127.0.0.1 - - [11/Jul/2024 18:08:09] "POST /password HTTP/1.1" 200 -

ive created a couple of logger.info and logger.warning and they work fine, i just want the console messages to be redirected to the log file aswell

8

That’s just not what a logger does–it doesn’t redirect all of stdout or stderr to a file. It writes logs to a file.

If you want to record all the console output, I’d do that as part of invocation, e.g. python password.py > all_the_output.log 2>&1 will direct stdout to the file with > and redirects stderr to stdout (and thus to the file) with 2>&1.

This won’t rotate the log files the way the file handler does, however.

9

Connect one of these:

to a subclass of one of these:

on which the write method calls logger.info

10

this is what i mean, when i use this config in my code

logger = logging.getLogger(__name__)
logging.basicConfig(filename='password.log', format='%(asctime)s:%(levelname)s:%(message)s', datefmt='%d/%m/%Y %H:%M:%S', encoding='utf-8', level=logging.DEBUG)

even when i refresh the web page i start to get logs

12/07/2024 07:25:03:INFO:e[31me[1mWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.e[0m
 * Running on all addresses (0.0.0.0)
 * Running on https://127.0.0.1:5000
 * Running on https://10.100.1.10:5000
12/07/2024 07:25:03:INFO:e[33mPress CTRL+C to quite[0m
12/07/2024 07:25:03:INFO: * Restarting with stat
12/07/2024 07:25:03:WARNING: * Debugger is active!
12/07/2024 07:25:03:INFO: * Debugger PIN: 118-343-405

i dont understand why “RotatingFileHandler” doesnt do it but “basicConfig” does

its like its paying no attention to my debug lines

11

You do know storing PINs or any credentials in log files is a security vulnerability - it’s just as bad as storing passwords in there?

But anyway. It sounds like Flask is writing log messages, not just printing to stdout. But Flask doesn’t know about the logger you created - its logging messages will possibly first go to its own logger, then be passed upwards to the root logger. Logs do not go down to your child logger.

As it says in the manual:

Does basic configuration for the logging system by creating a [StreamHandler]… adding it to the root logger.

A reference to the root logger is returned by getLogger if called with no args. Add your handlers to that instead.

12

sorry for the late reply, still doesnt work

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
handler = TimedRotatingFileHandler(filename='password.log', when='D', interval=1, backupCount=5, delay=True)
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s', datefmt='%Y/%m/%d %H:%M:%S')
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
logger.addHandler(handler)
13

logger = logging.getLogger(__name__) creates a Module or script level logger, not a reference to the Root logger, which is required, as Flask only knows about that logger (and perhaps its own internal loggers, but that’s not the common pattern).

Please reread the last sentence of my previous post.

Python 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> logger = logging.getLogger()
>>> logger
<RootLogger root (WARNING)>
14

you mean like this

logger = logging.getLogger()
15

Yes - exactly.

16

nice thank you @JamesParrott it works !!!

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
handler = TimedRotatingFileHandler(filename='password.log', when='D', interval=1, backupCount=5, delay=True)
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s', datefmt='%Y/%m/%d %H:%M:%S')
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
logger.addHandler(handler)
1 Like
17

ok doing some tests, changed the D to S and interval 30 so it would rotate logs quicker and without

logger = logging.getLogger(__name__)

i get this

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\python\\venv\\password.log' -> 'C:\\python\\venv\\password.log.2024-07-18_12-04-16'

so it looks like i need the “(name)” otherwise i get that error

i just think this will never work with the “TimedRotatingFileHandler”

but my issue is in production if my script comes into problems i wont know whats wrong with it to fix it

18

Yep logging’s complicated. I’ve only used Python’s logging to optionally help users capture debug output, in an app. On my own prod servers I just containerise each service and let Docker and the host OS’s systemd / journald handle logging.

I don’t use Arch by the way, but this feature was useful: https://wiki.archlinux.org/title/Systemd/Journal#Journal_size_limit