Implement opposite of __init__?

Python Help
1

What is the best approach to implement the opposite of __init__ so that users can type del obj and expect that the deleted object is cleaned up properly?

Example:

import weakref
registry = weakref.WeakValueDictionary()

class myClass(object):
    def __init__(self) -> None:  # CREATOR
        registry[id(self)] = self
    def __destroy__(self):  # DESTROYER 
        del registry[id(self)]

To make the case very explicit: My users expect that del obj results in a clean database that sits on disk. I merely use registry as a mock example hereof.

Test case:

a = myClass()
assert a in registry
del a  # must trigger call to __destroy__
assert a not in registry

test case 2:

a = MyClass()
b = a
del a
assert b in registry
del b
assert len(registry) == 0

This problem is very well documented in [1, 2, 3, 4], but there does not seem to be any solution:

  • __del__ does not guarantee any action until garbage collection, and that wont happen immediately.

  • atexit.register is too late, as I want the program to react to del obj

  • I lack creativity to make a context managers __enter__ & __exit__ work as the objects may have overlapping lifecycles (test case 2).

  • Monking patching del is not possible as del is a reserved keyword. If an atexit-type register existed I would be able to attach __terminate__ to the del keywork:

import keywords
keywords.register(del, myClass.__terminate__)

such that the behaviour would be:

def del(*args):
    for arg in args:
        if isinstance(arg, myClass):
            arg.terminate()
        keywords.del arg  # let gc deal with the clean up
  • The problem is not associated with cyclic references [4], but rather that asserts are executed straight after del and that is too early for garbage collection.

[1] Python Gotchas 1: __del__ is not the opposite of __init__

[2] destructor - How do I correctly clean up a Python object? - Stack Overflow

[3] python - Opposite of __init__ in thread class? - Stack Overflow

[4] Safely using destructors in Python - Eli Bendersky's website

2

To be explicit, use a context manager instead:

a = myClass()
with a:
    assert a in registry
assert a not in registry

Wouldn’t this look better than calling del manually?

1 Like
3

You can’t guarantee the result you want.

There is a __del__ method which may run when the last reference to an object is freed, but that is not 100% reliable.

Typing del obj does not guarantee to delete the object. All it does is clear the reference between the name “obj” and the actual object itself, which may have any number of other references to it.

Only when the last reference to that object goes is the object actually deleted, and even then, there are circumstances where the __del__ method may not be called, or may be called in a useful state.

For example, if the object isn’t deleted until the interpreter is shutting down, the builtins may not even exist any more, and calling even builtin functions may fail.

Let’s take your example:

registry = weakref.WeakValueDictionary()

class myClass(object):
    def __init__(self) -> None:  # CREATOR
        registry[id(self)] = self

    # actually use __del__
    def __destroy__(self):  # DESTROYER 
        del registry[id(self)]

That’s unsafe, as it is quite possible that the global name “registry” and maybe even the builtin name “id” will be deleted before your destructor method runs.

Unless you can guarantee every one of your MyClass instances are fully deleted before interpreter shutdown, using a destructor method __del__ is perilous.

There may be other unsafe situations as well.

The recommendation is to use an explicit close() or save() method, possibly even a context manager which will automatically close the resources, rather than rely on the destructor method.

You need to change their expectation.

Under the common circumstances that your object has only a single reference, and no cycles, then you can reasonably expect the __del__ method to be called reasonably promptly. And so long as the happens before interpreter shutdown, that’s fine.

So for your use-case:

a = myClass()
assert a in registry
del a  # must trigger call to __destroy__
assert a not in registry

that will be absolutely fine and will do what you want. But as soon as you have more complex examples, like inserting b = a in there, things can start going wrong.

This is simply a fact of life. The Python execution model just does not support what you want.

So you might teach your users that

del a

is reliable for the straight-forward cases, but for complex cases they should use an explicit a.close() or a.save(), whatever method makes more sense for you.

You said:

But that’s the problem – you don’t have objects plural, you have one object with two names. This is fundamental to Python’s execution model.

Without knowing what your destructor method is supposed to do, I would suggest that you revise your API design so that the preferred usage is:

with myClass() as a:
    do_stuff_with(a)

and you make it clear that while people are perfectly entitled to write
code like this:

a = myClass()
b = a
do_stuff_with(a)

they will need to manually call a close() or save() method to guarantee that the database is updated promptly.

1 Like
4

This reminds me of a recent discussion about moving __Init__() to be a method called reset(). I think it was @vbrozik’s suggestion. I will post a link when I get back to my computer (am on my phone right now).

You could also create a reset() class method that is a subset of the __Init__() code (and maybe even call it from __Init__() to avoid redundancy, but this seems unlikely; it would need to be tested and confirmed).

1 Like
5

Here is @vbrozik’s post: opposite of __init__?

He shows that __init__() is calling reset(), so I assume that you can call reset() from any line in the __init__ section. (I’m 99.9% sure, but assumptions are not safe with computers.)

6

I think this is the closest I can get to an implementation that works is using a function delete

import gc
from weakref import WeakValueDictionary

registry = WeakValueDictionary()


class MyClass(object):
    def __init__(self):
        registry[id(self)] = self
        print(id(self), 'registered')

    def __del__(self):
        if id(self) in registry:
            del registry[id(self)]
        print(id(self), 'deregistered')
        
def delete(*args):
    snapshot = globals().items() 
    for k,v in snapshot:
        if v in args:
            print('clearing', k, 'for link to', id(v))
            globals()[k] = None
    gc.collect()

The test

b = MyClass()
c = b
delete(b)  # cascading __del__
assert c is None

The output

1233896998176 registered
clearing b for link to 1233896998176
clearing c for link to 1233896998176
1233896998176 deregistered

Thank you @mlgtechuser @frostming & @steven.daprano for your insights.
I really appreciate the second pair of eyes.