Logging config documentation is too restrictive wrt ext:// and cfg:// references in custom objects

Documentation
,
1

Problem description

The docs warn that for the following formatter config…
formatters:
custom:
(): my.package.customFormatterFactory
bar: baz
spam: 99.9
answer: 42
… following restrictions apply:

The values for keys such as bar, spam and answer in the above example should not be configuration dictionaries or references such as cfg://foo or ext://bar, because they will not be processed by the configuration machinery, but passed to the callable as-is.

  • However in reality:
    ext:// references are resolved correctly, since resolution happens via __import__.

  • cfg:// reference resolution is undefined (I suppose it could resolve to original configuration or to an object constructed from it, depending on when the configured object is built). seems to work correctly since cfg is meant to resolve to raw config data (cfg_convert source).

Code example

This script configures a custom formatter and passes one ext and one cfg reference:

import logging
import logging.config
from collections.abc import Mapping
from typing import Any, Literal


class CustomFormatter(logging.Formatter):
    def __init__(
        self,
        fmt: str | None = None,
        datefmt: str | None = None,
        style: Literal["%", "{", "$"] = "%",
        validate: bool = True,
        *,
        defaults: Mapping[str, Any] | None = None,
        **kwargs: Any,
    ) -> None:
        print("handler_stream is", kwargs.pop("handler_stream", None))
        print("used_handler is", kwargs.pop("used_handler", None))

        super().__init__(
            fmt=fmt,
            datefmt=datefmt,
            style=style,
            validate=validate,
            **kwargs,
        )


logging.config.dictConfig({
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {
        "stream": {
            "class": "logging.StreamHandler",
            "formatter": "structured",
            "level": "INFO",
            "stream": "ext://sys.stderr",
        },
    },
    "formatters": {
        "structured": {
            "()": "__main__.CustomFormatter",
            "handler_stream": "ext://sys.stderr",
            "used_handler": "cfg://handlers.stream",
        },
    },
    "root": {
        "level": "INFO",
        "handlers": ["stream"],
    },
})

log = logging.getLogger(__name__)
log.info("Message")

Output:

handler_stream is <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
used_handler is {'class': 'logging.StreamHandler', 'formatter': 'structured', 'level': 'INFO', 'stream': 'ext://sys.stderr'}
Message

As we can see, the ext reference for the handler_stream key got resolved to the actual stderr stream, while the cfg reference for used_handler got resolved to the actual config value.

Change proposal

Let’s change “User-defined objects” documentation to allow shallow (not nested) ext references in custom objects but warn against using cfg references in formatter config and correspondingly, allow shallow cfg references.

2

I suppose the warning could be made correct by stating that ext and cfg references are resolved if they are placed at the same level as "()" but are not resolved when nested in collections underneath.