Using exec to define a member function that uses super()

Python Help
1

Suppose I am defining a class. This works:

class Foo:
    exec('def f(self): pass')

However, things go wrong if the function defined that way uses super():

class Base:
    pass
class Derived(Base):
    exec('def f(self): super()')

>>> Derived().f()
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    Derived().f()
    ~~~~~~~~~~~^^
  File "<string>", line 1, in f
RuntimeError: super(): __class__ cell not found

It’s not wrong: Derived.f.__code__.co_freevars is () and Derived.f.__closure__ is None. The effect is equivalent to:

class Base:
    pass
def f(self): super()
class Derived(Base):
    f = f

Is there any way to tease exec() (or something else) into doing what I hoped for?

2

Why would you want to do this?

3

To mechanically generate complex member functions.

Yes, you could eval() the entire class, or generate an entire .py file, but this is a case where most of the code is straightforward and I only want to generate one or two functions.

An ugly workaround is to Base.method(self, ...) instead of super().method(...), but I confess at this point I’m at least as interested in filling a hole in my knowledge concerning how classes are built and whether that functionality is user-accessible.

Having prodded things a little more, I infer that:

  • The special behaviour is implemented in compiling the AST to bytecode
  • It seems to be triggered by the presence of a Name('super') within the syntax of a function definition within a class.

Honestly, that feels a tiny bit hacky…

4

I don’t think exec was designed for this type of thing, and I would instead look into metaclasses. But could you show examples of the real sorts of methods you’re trying to auto-generate?

5

While I’m not familiar with the actual internals involved, I do know that dataclasses does this. Perhaps that might be a good reference for how best to implement such a thing?

6

Are you able to use the two-arg version instead? super(Derived, self).whatever() ? The zero-arg version is a very nice convenience, but it’s ultimately syntactic sugar for that. (The class name is whatever one you’re currently defining, no need to pick a base class or anything.)

7

…good point. Feeling stupid for not thinking of that. :flushed_face:

8

It’s (almost) always something simple - the work of debugging is working out which if the tens of thousands of simple things is the one that’s the problem here. No need to feel stupid for not thinking of the exact one that it turned out to be!

9

There are a few ways you can make an exec-ed function work with a no arg super, although it’s a little sneaky.

Essentially you want to get your function from inside a function that defines __class__ inside the string you’re exec-ing with a way to reassign __class__. Providing you don’t need to actually perform your exec call or assignment inside the class body you can do something like this.

s = """
def maker(__class__):
    def f(self):
        super().f()
        print(f"and {type(self).__name__}")
    return f
"""
globs = {}
exec(s, globs)
maker = globs["maker"]

class Base:
    def f(self):
        print(f"From Base")

class Derived(Base):
    pass

Derived.f = maker(Derived)
Derived().f()

From Base
and Derived

It’s also possible to patch __class__ inside the closure cell from a method defined on another class (you can see this kind of work in dataclasses where methods on a dataclass are patched to work on a new slotted copy of the class).

10

Ooh, sneaky. I like it!