Some easy and pythonic way to bind warnings to loggers?

Ideas
1

I’m developing a new application where I want to use the facilities from logging and warning.

Reading the docs, it seems as simple as calling logging.captureWarnings(True). But it is not that simple.

To begin with, my app is using its own logger. It leverages coloredlogs to have nice output by default, but that’s just a commodity that helps seeing the issue:

from logging import getLogger, captureWarnings
import warnings
import coloredlogs

logger = getLogger("myapp")
coloredlogs.install(logger=logger)

# Combine with warnings
warnings_logger = getLogger("py.warnings")
coloredlogs.install(logger=warnings_logger)
captureWarnings(True)

class MyAppWarning(Warning):
    pass

logger.warning("Logging a warning message")
warnings.warn("A warning object", MyAppWarning)

Now, run myapp.py and see:

imagen
imagen959×85 22.7 KB

Nice, now you can see some problems:

  1. The warnings stop coming from the myapp logger, and they start being logged in the py.warnings logger instead.
  2. py.warnings is a global logger, so when combining this library in other app, it will alter that other app warnings style.
  3. The warning message when using warnings.warn() shows a lot more information that (for my use case) is useless.

So, the easiest way to workaround this problem would be to just use logger.warning instead, which fixes all those problems. But then I cannot have all the extra nice tools that I get while using the warnings module (for instance, being able to convert those to exceptions while testing, and assert them).

This is very confusing. Some questions arise, like “why am I forced to use the py.warnings logger?”

My proposal is that Python adds a supported simple way to wire a logger and a warning class:

from logging import getLogger
logger = getLogger("myapp")
logger.captureWarnings(MyAppWarning, ...)

As simple as that, then when calling warnings.warn("Message", MyAppWarning), the warnings module knows that warning has to be logged in the myapp logger, not in the py.warnings logger. Also, it will log it using the default output format configured in the logger.

Warnings from any other class that doesn’t inherit from MyAppWarning would behave as normal.

The result would be that the logging output would be exactly the same as when using logger.warning(), but I’d be able to leverage all other tools from the warnings module. Also, a program can declare its own warning classes and configure how and where they are logged.

2

You could implement your own showwarning:

import os
import sys
import logging
import warnings

def showwarning(
    message, category, filename, lineno, file=None, line=None
):
    if file:
        raise ValueError(file)  # exercise for the reader
    message = warnings.formatwarning(
        message, category, filename, lineno, line
    )
    for module_name, module in sys.modules.items():
        module_path = getattr(module, "__file__", None)
        if module_path and os.path.samefile(module_path, filename):
            break
    else:
        module_name = os.path.splitext(os.path.split(filename)[1])[0]
    logger = logging.getLogger(module_name)
    # TODO: handle when not logger.hasHandlers()
    logger.warning(message)

warnings.showwarning = showwarning

You may also want to check out warnings.catch_warnings for easy resetting.

3

Ok, but that is not pythonic. It goes against https://www.python.org/dev/peps/pep-0020/#the-zen-of-python in several ways. Also you’re monkey patching a function from the stdlib. It might work, but it doesn’t solve the problem IMHO.

4

If you had read the documentation, warning.showwarnings explicitly says that it’s there to be patched. Also, my example is just a small modification of logging.captureWarnings's implementation. I’m only using public API of Python’s internals and library.

5

Thanks for your help. Yes, I had read the docs and I already understood that point you mention before opening this thread.

I guess that will help somebody else that reaches the forum looking for a solution that works with current python versions.

OTOH, you can easily see that I didn’t come asking for that. I never asked “how to do it?” Actually, this post is in the “ideas” category, not in “looking for help” or similar.

So, my point is that this specific part of Python needs to improve, why is it needed, and one idea on how that could be done.

You proved this can be done, but that doesn’t mean it is good as it is, or pythonic, or that it can’t improve.

6

Right, fair enough. Criticising your idea then, I don’t think that this feature should be added to the standard-library as I think it’s a very niche use-case with a moderately straight-forward implementation. Users who need the functionality can write it themselves or use an implementation on PyPI.

Specifically with your suggested signature, I don’t like how the idea of capturing global warnings is attached to local loggers: rather, I would like to see either warnings go to their module’s logger (as in my implementation), or a module-level function (context-manager?) which sends warnings to a user-specified logger (eg logging.captureWarnings(enabled, logger="py.warnings")).

7

Yes, after double-thinking about it, maybe it would be easier to implement a logger that has the features I miss from the warnings module.

logger = nicelogger.getLogger(__name__)

def a():
    logger.warning('Do a warning here')

def test_a():
    with pytest.assertRaises(), logger.raise_level(logging.WARNING):
        a()

It seems to me this would be easier to implement, more pythonic and would avoid ugly and fragile hacks that rely on monkeypatching the stdlib (which is itself a weird recommendation).

Indeed this can be done in a separate library.