Namespace blocks

Ideas
1

I started playing with the idea of namespaces in Python due to the recent discussions about public/private.

It’s often useful to group related items under a name, but creating a submodule isn’t always desirable.

Python currently has two constructs that introduce a new namespace (and scope), class and def.
class is fine for grouping constants or even classes, but due to Python’s name resolution rules, not for functions.
On the other hand, functions are obviously not evaluated immediately. This can be solved with a simple decorator, but retrieving the new namespace requires either an explicit return locals() or some fragile reflection. Additionally, classes and methods defined in a function get .<locals>. in their __qualname__.

I’d like to propose a new general-purpose namespace block:

namespace consts:
    X = 1
    Y = 2

# Name can be omitted to create a simple lexical scope,
# useful to prevent littering the outer scope.
namespace:
    global x
    y = ...
    z = ...
    x = y + z
# y and z are undefined here

Name resolution inside a namespace is similar to functions:

namespace a:
    x = 1
    f: Callable[[], int]
    namespace b:
        a.x += 1
        y = 2

        nonlocal f
        def f() -> int:
            return a.x + y

Private namespaces:

# mod.py
namespace __internal__:
    from ._private_utils import my_util

    def private_func(): ...

    global public_func
    def public_func():
        return my_util(...)

# script.py
import mod
assert mod.public_func.__qualname__ == 'mod.public_func'
# Internal names are still accessible for monkey-patching and experimentation
assert mod.__internal__.private_func.__qualname__ == 'mod.__internal__.private_func'

When used as a scope, namespace can be useful for cleaning up intermediate variables that would otherwise live forever (or until the end of a function):

namespace:
    # No need to `del big_object1, big_object2`, they're refcounts will be zero
    # after exiting this anonymous namespace.
    big_object1 = json.loads(big_json.read_string())
    big_object2 = json.loads(big_json.read_string())
    global x, y
    x = big_object1['x']
    y = big_object2['y']
5 Likes
2

Could you expand on this? Is it not sufficient to decorate a function with @staticmethod?

4 Likes
3

I have the same question and am probably missing something, but why even bother with staticmethod?

class namespace:
    def double(value):
        return 2*value

namespace.double("abc")  # "abcabc"
namespace.double.__qualname__ # "namespace.double"
1 Like
4

I think this is a great idea and I have wished for this before.

Regarding whether you can just use class: maybe you technically can, but it looks strange to have procedural code in a class and it just makes the intent clearer to use a namespace. Also makes it clear you’re not supposed to instantiate this namespace as an object (the way you can do with a class).

I think ideally the namespace would just behave as a submodule. So

# a.py

a = 1

namespace b:
  b = 1

is equivalent to

# a/__init__.py

a = 1

# a/b.py

b = 1
2 Likes
5

Because that version is missing the self parameter (and would fail with the additional input you give it).

6

This is a pretty different proposal though, in that there’s no scope effects.

7

not exactly. You don’t need a self parameter. If you access the function from the class then self is not automatically given to the function as it would be when calling it from an instance of a class.

1 Like
8

Somewhat related, there’s a pre-draft PEP for a make statement that could be used for this:

The actual PEP is a bit more general, but one of the examples shows how it could be used to build simple namespaces:

namespace = ...  # <setup omitted>

make namespace Constants:
    AIRSPEED_AFRICAN = 5
    AIRSPEED_EUROPEAN = 8
4 Likes
9

huh TIL. I was thinking of an instance of the class

10

Name resolution for methods skips the containing class(es).

Take this example:

class MyNamespace:
    x = 1
    def f():
        return x  # NameError: name 'x' is not defined

You would need to use MyNamespace.x, which is cumbersome, especially with more than one nesting level.

With namespace, f would close over x.

3 Likes
11

That’s roughly the idea, but I chose to not phrase it that way because modules are heavily linked to the import system. My proposal wouldn’t allow import mod.namespace, nor would it appear in sys.modules.

1 Like
12

Thanks for the link! It seems that @Jelle’s proposal would have the same name resolution issue as class though.

13

I really like this idea, I hope we get something like this! I’ve been fighting python for the past day and a half trying to figure out something similar.

If they do go ahead with this I hope that we can parameterize the namespace to make it rebindable. Maybe that’s not the right terminology. but I’d like for it to take arguments. So that you can create shared functionality without having to deal with attribute lookup and self and wrapping 50 functions with staticmethod.

namespace math_funcs(a):
    def add(x):
        return a + x
    def div(x):
        return a / x

math1 = math_funcs(1)
assert math1.add(1) == 2

math2 = math_funcs(2)
assert  math2.div(4) == 0.5

classes are unnecessarily heavy and verbose for this. If you aren’t using actual OOP concepts like polymorphism and inheritance then I’d even go so far as to say that a class is an anti-pattern. It’s like buying a giant pickup truck when your only need for transportation is going to get groceries.

You could use a function but then you end up having to re-specify all the types and add everything to the return statement and that’s a big ol repetitive PITA. It would be like only having a bicycle for transportation when you live 25 miles from the grocery store.

So trying to use a class you need to strip unnecessary parts from it. and trying to use a function you have to try and reintroduce a subset of those parts on your own every time.

14

Well, one of the best point of computer science is that you can reuse the same vehicle for multiple different uses, rather than having a huge garage with one vehicule per travel distance. What you call an anti-pattern for class is just regular programming in my eyes.

Regarding the namespace proposal, it’s true that it seems nice, but at the same time I don’t think I would use it anywhere in my code, there are already so many tools to do a sufficiently equivalent result.

15

python added the matrix multiply operator @ for scientific computing. I don’t use it in my code. That doesn’t mean it isn’t a valuable addition to the language.

This proposal would allow for solving problems in a more efficient manner. Looking up a closed over variable is more efficient than looking up an attribute. Creation of a namespace will almost certainly be more efficient than creating a class because it is a simpler tool.

Personally I think this would be a big win for the language, it would lead to objectively more efficient code and subjectively I think it would lead to cleaner more expressive and succinct syntax. I don’t think that the current tooling is equivalent or sufficient, but these are opinions, we may disagree that’s ok :slight_smile:

16

This is seems like the perfect use case for a class. What don’t you like about this?

@dataclass(frozen=True, slots=True)
class math_funcs:
    a: float
    def add(self, x):
        return self.a + x
    def div(self, x):
        return self.a / x

math1 = math_funcs(1)
assert math1.add(1) == 2

math2 = math_funcs(2)
assert  math2.div(4) == 0.5
6 Likes
17

It’s hard to take this claim at face value without a reference implementation demonstrating performance gains.

18

I appreciate the frozen and slots with your example. But basically there is no need for self. So why include something that there is no need for?

Yeah I could use static methods but it just feels wrong and takes up a lot of extra space if you’re using it a lot. and then I can’t have access to a. which we do have in the namespace example. So you either get rid of self and have no shared state or you have to add a pointless parameter to your def.

19

The first part is true, that much is known. The second part yes I agree we need a reference implementation. But the thinking is that a namespace is a simpler thing. A class does nearly everything a namespace does but has the added complexity of creating instances.

initializing an instance of a class would more than likely be cheaper. but using the functions from a namespace would be cheaper than calling an instances method. or at least that’s what I’m thinking, definitely TBD.

20

I think namespaces would be super useful with or without parameters. If it doesn’t have parameters it could just be wrapped in a function to accomplish the same thing. slightly bulkier but still much better than what we can do currently.