Dear friends, I am proposing to introduce a new syntax named Section Statement or Section Label Expression, which aims to bring python language the power of expressiveness on the domain of configuration, just like toml or ini or env does.
The syntax Section Statement is quite simple:
it is a simple statement, which is written in a single line;
the whole line is enclosed by square brackets, like list expression, but
there is only an undefined identifier within the square brackets.
The undefined identifier here is named as Section label.
This is how the syntax expected to work:
split a .py file to several sections separated by such statements;
the variables (as well as functions, classes) defined in a section, are interpreted as the items of a dict named after the section label.
Let me bring some codes to demonstrate this idea. For example, the python file demo_config.py are written with following codes:
With this syntax, we can absorb almost the whole features of toml config language to native python. The builtin configparser module can also be integrated.
It has also some further usage:
the section can be regarding as a lite implementation of submodule, with which you need not to create a module folder and __init__.py .
Tell the code editor or IDE to fold and unfold specified parts of a file.
I thought out this idea when I was working on cloud native programming. I really think it will be helpful when python has a subset for config markup.
# this helper function could be imported from another file
def section(c): return dict(c.__dict__)
@section
class section1:
var1 = "value1"
var2 = "value2"
class cls1:
pass
@section
class section2:
var21 = "value21"
var22 = "value22"
def method23():
pass
If you want cleaner dictionaries, you could define the section decorator to exclude anything with a leading underscore: def section(cls): return {k, v for k, v in cls.__dict__ if not k.startswith("_")} or whatever other rule makes sense for you.
Thank you very mush Chris! I am just waiting for your reply after I post this thread
The function of this new syntax is simple & clear, so it can also be achieved by existed syntax like decorator or others without much hard works.
What I expect is such a goal: the python file with this new syntax is valid code for program developers, and also it appears absolutely a valid, human-read-friendly configure file (like toml or ini) for configuration ops and maintenance staff. It is a dual benefit, which makes me like bilingual programmer.
Dear Elis: the approach by class statements can achieve the same goal, but it is not the style of a config file look likes. For a config file, the main principle is for human reading friendly and across language individual syntax boundaries. That is why many config markup languages are created and keep evolving.
I don’t find anything friendly about these config formats; it’s just a trend, perhaps a retrospective. If I can’t automate it, it’s not friendly to me.
Hmmmmmmmm. I don’t have an actual implementation to offer you, but in theory, it ought to be possible to make an import hook that makes INI files directly importable? It’s not something I would personally do, but if you want it, the option should be there I think.
Yes, just make the implementation to the interpreter
Regarding this: any json file is almost a valid python dict expression (except null for None, true for True), and json is a widely used, popular config file format; but for so long years, it has not been able to be imported directly yet.
This doesn’t seem like a good thing? Config files are typically treated as user input, so you’d normally distrust then and prefer not to send them to a full interpreter.
I don’t want configuration files that can execute arbitrary code, or arbitrary code that masquerades as a data structure. I want a separate configuration language that is explicitly limited in what it can compute. Dhall is a good example.
Config file content does not come from end-user with uncertain trustworthiness; it is usually from System Administrator | DevOps Engineer | Operation Staff who collaborate with us the developers. The content provided should be trusted in most cases.
Think about the config file jupyter_notebook_config.py of a jupyter server. In this config file you can even disable password check and allow all remote network access, which is hard to say untrusted. In some cases, the config file editor has even a higher security level compared to program developer.
The Section Statement is actually an assign expression for a section object which as I design is expected to be of a type like Dict or ModuleType. The code you put above is about re-assign section object.
In my opinion, well organized codes should avoid re-assign object with differenet types. For a same variable, it should be consist in typing no matter how many times re-assigned with values. But I agree this rule is slightly strict. Maybe it can be moved into a linter rule instead of syntax rule ?
The proposal does not intend to bring arbitrary code to config file; it aims to bring a strict subset dedicated to configuration. For config file editors, they should be aware of this.
Integrating a human reading friendly syntax for configuration can be highly beneficial. Take a commonly found example: JSON, short for JavaScript Object Notation, is a strict subset of javascript. The native syntax support for javascript to operating with json data brings huge advantages on the domain of network data processing.
But it’s clear from the def statement in your example configuration that you’re bringing arbitrary code to a config file. Or at least it is unclear where you intend to draw the line of this strict subset dedicated to configuration.
The example is token for demostrate that the configuration style is introduced as Non-intrusive feature with regular python expressions. It is a natural way that a rule for scalar values also appliable for class & functions since they are all PyObject. This is exactly the princple of language strict subset.
As for introducing a new syntax instead of just writing toml, there are several reasons:
Directly import config instead of open, read, load;
you have not to learn another specialized language;
You underestimate the specialization of configuration style. It should not be consider as a kind of ordinary customized text.
After years of evolving, the pattern of square brackets enclosed sections becomes dominant in configuration languages, which undoubtedly reveals that the style has great advantage in providing human-reading-friendly structured data. After all, loading toml-like config is far more common in development than other ordinary file format. It is worthwhile to introduce a new syntax for this scenario.
A bracketed name is a valid Python list of a name, so all you need is a loader that parses the file of a given path into an AST, find each bare expression of a list of a name at the module level, repackage the sibling nodes that follows until the next bracketed name into its own module node and then execute it in a module namespace.
That’s something a linter should warn about, not a restriction to build into the language. Not worrying about this also makes the implementation simpler.
