What are the cases where objects will never be collected?

Python Help
1

My class defines __del__ but it is not called when I delete the object.:

You can run gc.collect() to force a collection, but there are pathological cases where objects will never be collected.

Does it mean the case stated in Why isn’t all memory freed when CPython exits?? (If not, what pathological cases is it talking about?):

Objects referenced from the global namespaces of Python modules are not always deallocated when Python exits.
This may happen if there are circular references.

But Why isn’t all memory freed when CPython exits? also says:

Python is, however, aggressive about cleaning up memory on exit and does try to destroy every single object.

If Python is really aggressive, then after cleaning up all single objects, it certainly knows that all the rest objects are those involved in circular references.
They can also all be cleaned up without having to be decided by the cycle detection algorithm. Am I missing something? :face_with_spiral_eyes:

(I’m treating “clean up” “destroy” “call __del__” “collect” “deallocate” as very similar things here; not sure whether it’s the right way to interpret them.)

2

I would strongly recommend using a context manager and avoiding __del__ like the plague.

3 Likes
3

Good advice, thx. I know that __del__ is not recommended; I asked this question out of curiosity.

4

Objects referenced from the global namespaces of Python modules are not always deallocated when Python exits.

import gc


class Node:
    def __init__(self, value):
        self.value = value
        self.next = None


# Create a function to create a complex circular reference structure
def create_pathological_case():
    nodes = [Node(i) for i in range(10000)]  # Create a large number of nodes
    
    # Create circular references between nodes
    for i in range(len(nodes)):
        nodes[i].next = nodes[(i + 1) % len(nodes)]
    
    # Create a reference to the first node from the global namespace
    global node_ref
    node_ref = nodes[0]


# Function to check if the reference to the first node still exists after garbage collection
def check_reference():
    if 'node_ref' in globals():
        print("Reference to the first node still exists.")
    else:
        print("Reference to the first node is garbage collected.")


# Create the pathological case and check the reference
def _pathological_case():
    create_pathological_case()
    print("Reference to the first node:", node_ref)
    
    # Explicitly trigger garbage collection
    gc.collect()
    
    check_reference()
    
    # Check if any objects are identified as garbage
    if gc.garbage:
        print("Objects identified as garbage:", gc.garbage)
    else:
        print("No objects identified as garbage.")


# Test the pathological case
_pathological_case()
1 Like
5

That doesn’t look pathological at all but like a very normal case, you even still have access to the nodes. And note what you quoted, saying “not always deallocated when Python exits”. Your test doesn’t even look for that. Try adding

def __del__(self):
        print('del')

to your class, don’t you see lots of dels printed then, because Python does deallocate them when it exits?

It would also be good to show your output.

1 Like
6

With a bit more logging I even see that all 10000 dels happened:

dels = 0
class Node:
    def __init__(self, value):
        self.value = value
        self.next = None
    def __del__(self):
        global dels
        dels += 1
        if dels > 9995:
            print('del', dels)

Output:

Reference to the first node: <__main__.Node object at 0x7946f124e8d0>
Reference to the first node still exists.
No objects identified as garbage.
del 9996
del 9997
del 9998
del 9999
del 10000
1 Like
7

Thank you for your review, but it’s obvious that this is a template.

Here’s an example of a pathological case (a problem or situation occurring only outside normal operating parameters):

import gc


class Node(object):
    def __init__(self, value):
        self.value = value
        self.next = None
    
    def __del__(self):
        print(self.value)


# Create a function to create a complex circular reference structure
def create_pathological_case():
    nodes = [Node(i) for i in range(10000)]  # Create a large number of nodes
    
    # Create circular references between nodes
    for i in range(len(nodes)):
        nodes[i].next = nodes[(i + 1) % len(nodes)]
    
    # Create a reference to the first node from the global namespace
    global node_ref
    node_ref = nodes[0]
    
    # Create a circular reference between a global object and one of the nodes
    global circular_ref
    circular_ref = {'node': nodes[0]}
    nodes[-1].next = circular_ref


# Function to check if the reference to the first node still exists after
# garbage collection
def check_reference():
    if 'node_ref' in globals():
        print("Reference to the first node still exists.")
    else:
        print("Reference to the first node is garbage collected.")


# This function creates a daemon thread
def daemon():
    import threading
    import time
    
    def target():
        time.sleep(50)
    
    thread = threading.Thread(target=target, daemon=True)
    thread.start()
    # thread.join()


# Create the pathological case and check the reference
def _pathological_case():
    create_pathological_case()
    print("Reference to the first node:", node_ref)
    
    # Explicitly trigger garbage collection
    gc.collect()
    
    check_reference()
    daemon()
    
    # Check if any objects are identified as garbage
    if gc.garbage:
        print("Objects identified as garbage:", gc.garbage)
    else:
        print("No objects identified as garbage.")


# Test the pathological case
_pathological_case()

Output:

Reference to the first node: <__main__.Node object at 0xrandomnumber>
Reference to the first node still exists.
No objects identified as garbage.

And this is my statement:
Calling the “daemon” function prevents objects from being deallocated. Note that we are not waiting for the thread to finish, which is inherently pathological.