I would argue these changes would make python based app deployment easier and reduce development effort for supporting specific deployments.
Add native support for json logging
Natively support iso time format and utc time
Enable to overwrite logging configuration via ENV variable(s)
At the moment one needs to use cutom logger to get json format or write log parsers to get structured logs.
At the moment developer needs to foresee the need of logging to be configurable, while the need to configure logging is usually at deployment, thus it would make sense to have standard way, without modification to source code, to override logging configuration for any python app/script using the standard logger.
I’d be interested in a stdlib way to get JSON-formatted logs, and I think containers deployed in cloud services are a good motivating use case – several services will stream stdout to some log destination like an ELK stack.
That said, I think it’s better to focus on one idea, rather than multiple. Some people will agree with one part but not others. For example, I think a JSON formatter would be interesting, but I don’t agree that this is a problem:
Yep! Being a developer often requires thought and foresight! Sometimes, you even make mistakes and need to change things!
JSON formatted output makes the most sense to me as a formatter. I’m not sure what pitfalls there might be in that approach, but it seems to me that it would be a little bit fiddly to configure the fields to emit, the encoder to use (can I plug and play orjson?), and handling of extra. If it can do all of that and pipe through a stream handler, it would be nifty.
It is mostly from Deployment side of things, same code will be deployed in different env, organizations etc. I think it would be nice, if there was one straight forward way how to configure python app logging. At the moment it is either hardcoded, some part made configurable by env variables or by using the config json (the best option in my view).
Thus I think it would be nice to have somthing like PYTHON_LOGGING_CONFIG = path to config json
That would owerride/set the logging module configurations.
I don’t think this env var idea is likely to be worth implementing. Sure, you can add a system to do this, but it will only work if the application doesn’t configure logging and thus blow away those settings.
If you build those sorts of capabilities into your application (which is easy to do and does not require a stdlib change), then you’ll have much finer grained control.
My opinion is that you already need your application and infrastructure to be harmonious, and adding another way to put configuration data into the system doesn’t change that.
If you want to pursue this idea, I think you need an explicit proposal for a new method in logging.config which loads this configuration. That way, the time at which the config is loaded is well defined, and it’s possible to execute code before and after that step. Without an explicit hook which loads the config, I don’t think the contract is sufficiently clear.
Maybe env is not a good solution, the idea is that there is good and stable configuration API for python logging. I think current json file is good, just additional way to enforce it is needed (instead what is in code). This way one could take existing python app and deploy it for example with structural json logging without modifications to source code.