This is my second and last thread (see first thread) about a library about mutability I was working on.
After receiving feedback on this forum (thank you, @Daverball) I solved most of the problems and I was about to publish the second version, but today Pyright in VSCode was updated from v. 1.1.363 to v. 1.1.364, breaking my code.
I won’t open an issue on this because it’s clear Eric Traut is not a fan of my work.
I’ll just document what happened, technically, and move on.
My library introduces read and write qualifiers so that one can control mutability. For instance:
def f1(xs: list_r[int]):
# the list CANNOT be modified here...
xs.sort() # type error
xs[0] = 4 # type error
def f2(xs: list_w[int]):
# the list can be modified here...
xs.sort()
list_r[T] is covariant in T, of course. With a little bit of work, one can support other types such as set and dict (Indeed, the library already supports them).
I think write permissions should be explicitly granted:
def f1(xs: list_r[int]) -> int: ...
def f2(xs: list_w[int]): ...
def f3(xs: list_w[int]):
res = f1(xs)
f2(w(xs))
...
Since xs is of type list_w[int], we can pass it to f2, but f2(xs) won’t work: we have to be explicit and write f2(w(xs)), where w is a little function that gives write permission. Of course, w(xs) will raise a type error when xs is read-only.
This is not a simple thing to do, though. This was my first attempt:
class _LK1: ...
def f[L: _LK1](xs: list_w[int, L]):
...
class _LK2: ...
def f[L: _LK2](xs: list_r[int], ys: list_w[int, L]) -> list_w[int, None]:
f(ys) # type error
f(w(ys)) # OK
How does it work?
Here’s the idea:
list_whas an additional type parameter for the “lock”.- when
list_wis first created, the lock isNone wchanges the lock fromNonetoAny.f(ys)doesn’t work becauseNoneis incompatible with_LK1.f(w(ys))works becauseAnyis compatible with_LK1.
Note that the return type should have a lock equal to None or the caller might pass the returned value back to the same function without having to use w.
This is not ideal, but it works.
Is there another way?
If only we could manipulate function signatures!
Well, it turns out we can…or, better, we COULD 
First, the idea:
@lock
def f[L](_: L,
xs: list_r[int],
ys: list_w[int, L],
zs: list_w[int, L]) -> list_w[int, L]:
...
Up to version 1.1.363, lock transforms that function into the following:
def f(xs: list_r[int],
ys: list_w[int, _Unlocked],
zs: list_w[int, _Unlocked]) -> list_w[int, None]:
...
Try the (stripped-down) code for yourself on Pyright Playground.
Up to v. 1.1.363 we get
Type of
fis(x: int, y: X_[_Unlocked], z: str) -> X_[None]
From 1.1.364 onwards we just get
Type of
fis(x: int, y: X_[LK@f], z: str) -> X_[LK@f]
The main idea is this:
- we add a “temporary”
_: LKto get a hold of the type paramaterLK - we force Pyright to replace
LKwith_UnlockedandNone. - we remove
_: LKfrom the final signature
If some of you are thinking “Why don’t you do the transformation by hand?”, keep in mind that the original function still sees the original signature. Only the callers see the modified signature. That’s the whole point! We can do this transformation by hand not by modifying the signature, but by adding an “external” signature through @overload. Basically, @lock automates that.
I was thrilled when I discovered this trick.
I had already solved the lifting problem. I was able to replace all the overloads (slow) with a type-level binary search (100x faster), and then even moved to small local lifts (much lighter and even faster).
I’ve learned a lot in these few weeks and I believe I can do pretty interesting things with the Python type system, but it seems my energy would be better spent elsewhere, so I’m going back to (ethical) hacking where my “abuses” are rewarded 