Checking against Functional Programming HOWTO — Python 3.14.3 documentation, the existing documentation does reflect the sentiment of this thread, but not necessarily all of the reasoning and depth. Would a revision of this page including some of what has been covered and what could be described at perennial issues be welcome and desirable?
Why? It seems to me much more simple to propose adding a typing.Pure and hope that in a future mypyc will optimize it.
From my current understanding, function purity requires enforcement mechanisms for all function arguments to be immutable, the return to be immutable, and no accesses outside of function local and argument data exists. Some of what this means is that there would have to be a new keyword like async to cause the interpreter to check for all these conditions. In more literal terms, it means that all called functions must be passed as arguments, those functions must also all be pure, and a variant of object created which is immutable. While these are strictly speaking implementable, they are onerous and are anti-pythonic. So it goes beyond ‘what’ a type is, and alters ‘how’ functions work. There are some type checkers that can be used with some foreknowledge to get more functional-like implementation patterns, but these are not enforced to be actually pure functions.
It’s not easy to define an immutable argument. Examples modifying tuple, frozenset and frozendict which are usually seen as immutable:
>>> t=(1, 2, [])
>>> t[-1].append(3)
>>> t
(1, 2, [3])
>>> class Object:
... def __hash__(self): return 5
...
>>> obj=Object(); s=frozenset({obj})
>>> obj.attr = 123
>>> s
frozenset({<__main__.Object object at 0x7f4613353cb0>})
>>> fd=frozendict(x=[])
>>> fd['x'].append(123)
>>> fd
frozendict({'x': [123]})
5 Likes
So there is a missing notion of recursively immutable?
I’d be more inclined to say that immutability is an informal concept in Python, only enforced by convention (i.e., documentation), not by the language itself.
This is at the very core of the “consenting adults” principle, often quoted in Python design discussions. We don’t typically expect to enforce constraints, we assume that any callers and callees act in good faith, and don’t violate “the usual conventions” without understanding that they are responsible for any unexpected consequences. This principle is very different from “strict” languages which have language-enforced constraints. Trying to transplant concepts from such languages into Python is very often a frustrating and largely pointless exercise.
4 Likes
Surely we can say that ints, strings, and floats at least are immutable, and enforced by the language.
3 Likes
Yes. Sorry, I was thinking about “container immutability”, because that’s the context of the post I was replying to. Atomic types can be immutable (assuming they are implemented that way) and the three you mention are examples of that.
Until you use ctypes to change the value of 3, yes.
Let’s say they’re immutable in normal code.
1 Like
ctypes never counts.
2 Likes
Should I create a new thread for typing properties? Things like <type>.mutable: bool, <type>.ordinal: bool, <type>.lossless: bool? If I were to want to recursively inspect a value and all of its contents, these could be helpful to determine things like how something could or could not be changed. I’m not immediately finding a similar thread, but I really don’t have a good enough history over what has been talked about before to actually know if this has been hashed out.
That’s why I suggested to Josh Yet Another Typing annotation. But I suppose that the chances that a type hint that currently do nothing and maybe in a future can do something is unrealistic.
Maybe someone can create a custom type hint and a mypy plugin:
You can check if an object has a hash, but hash != immutable. For example, custom classes by default have a hash.
I don’t know if you can propose your idea as a Faster Python issue.
I’m just trying to catch up here to not be so far out of my depth. The first hint wasn’t quite blunt enough, sorry. I’ll look into these now.
Probably not (ed: probably not unless you are doing it to discuss the the larger path to being able to enable this, rather than opening it to propose adding it to the existing type system)
In terms of being able to type an ordinal, there’s prior work that’s largely seen as a failure (see various threads about why the numeric tower doesn’t work for typing).
In terms of mutability, lossless mathematical operations, etc, it’s not something that fits into python’s type system compatibly with those chosen foundation right now.
You can view this thread about why collections.abc.Hashable is “Broken” by the current foundation of the type system, the same applies to any type that indicates “this type doesn’t do something” (such as allow mutating) because subtypes are not prohibited from adding behavior. This has a path to being fixable, but the general idea of encoding certain properties of types that aren’t statically determinable is going to be a hard sell prior to fixing those, and there’s been very little appetite shown by type checker authors on topics like this.
It’s not impossible to model these things in a type system, but it’s not reasonably possible to model these in python’s static type system without changes to the foundation to have it be correct.
4 Likes
What else should I read beyond PEP 484 to understand the current type system?
The typing specification; or the mypy docs, which may be an easier read.
When starting my current project, which contains a lot of pure math, I’ve longed and looked for a similar capability like this with the purpose of setting guardrails for our developers.
Just trusting the devs to make the right decisions was not a sufficient solution in this case, but I’m sharing the sentiment of others in this thread that ultimately, Python is too flexible of a language to make a feature like this possible to implement. You need to start with a functional language where functions are pure by default and there are no escape hatches for this purity unless explicitly declared in the function’s type.
The solution I’ve landed on is to strictly enforce typing annotations on all code in the project and that all input to the mathematical “core” of the application is an object of a class that inherits from a metaclass which checks the recursive immutability of the types of all its attributes. This is nearly impossible to do correctly generally, but because I control the libraries and the application, I know exactly which types to expect and whether they are immutable or mutable. The recently approved PEP for frozendict makes this a little easier as well.
In my case I used pydantic’s BaseModel as a starting point, but you could probably do this with attrs or dataclasses as well. You might also be interested in inspect_annotations, though I implemented my own solution to this.
For your proposed solution of analyzing bytecode, you can implement an import hook, check the bytecode after running compile and see how far you get with that. Maybe it will be sufficient for your usecase. I don’t believe you’ll manage to make it work for the general case, though.