On The Topic of Slots - Novel or Practical?

Python Help
1

Hello,

I am currently on the topic of slots. I am following this video tutorial:

I am using Python v3.12. The test code is:

import sys
from getsize import get_size
# from pympler import asizeof as get_size

def why_use_slots_example():
    
    print('WHY USE SLOTS EXAMPLE')
    
    class A:
        
        def __init__(self):
            
            self.x = 42
            self.y = 42
            self.z = 42
            self.t = 42
            self.u = 42
            self.v = 42
            self.w = 42
            
    class B:
        
        __slots__ = 'x', 'y', 'z', 't', 'u', 'v', 'w'
        
        def __init__(self):
            
            self.x = 42
            self.y = 42
            self.z = 42
            self.t = 42
            self.u = 42
            self.v = 42
            self.w = 42

    print('Size of A (not slotted) instance:', sys.getsizeof(A()))    
    print('Size of B (slotted) instance:    ', sys.getsizeof(B())) 
    
    # Use a recursive getsize function that counts the size of objects and their
    # sub objects
    print('\nRecursive size of A (not slotted) instance: ', get_size(A()))            
    print('Recursive size of B (slotted) instance:      ', get_size(B()))         
    print('Memory savings factor = size A / size B =    {:1.1f}'.format(get_size(A()) / get_size(B())))

I notice that in the video, the value for the recursive slotted B (get_size(B)) result is greater than the non-recursive value. However, when I test it, the value is unchanged. The recursive function that I used is:

'''
Recursive function obtained from here:
https://gist.github.com/durden/0b93cfe4027761e17e69c48f9d5c4118
'''
import sys

def get_size(obj, seen = None):
    
    """Recursively finds size of objects"""

    size = sys.getsizeof(obj)
    
    if seen is None:
        seen = set()

    obj_id = id(obj) # Get memory address of object
    
    if obj_id in seen:
        return 0

    # Important mark as seen *before* entering recursion to gracefully handle
    # self-referential objects
    seen.add(obj_id)

    if isinstance(obj, dict):
        
        size += sum([get_size(v, seen) for v in obj.values()])
        size += sum([get_size(k, seen) for k in obj.keys()])
        
    elif hasattr(obj, '__dict__'):
        
        size += get_size(obj.__dict__, seen)
        
    elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)):
        
        size += sum([get_size(i, seen) for i in obj])

    return size

My test results are:

Size of A (not slotted) instance: 48
Size of B (slotted) instance:     88

Recursive size of A (not slotted) instance:  658
Recursive size of B (slotted) instance:       88
Memory savings factor = size A / size B =    7.4

This part of the tutorial begins at the 3:46 minute mark. Is there something that I am doing wrong here? If so, can you please point it out.

Thank you.

2

These are the things I would say you’re doing wrong:

  1. Trying to learn from the sort of video tutorial where a presenter is basically just reading off the screen - this is better edited (and voiced) than most I’ve seen, but it fundamentally adds no value vs a static web page presentation.

  2. Expecting the getsize code you found to do the same thing as pympler’s code, without attempting to verify that.

  3. Expecting these kinds of implementation details to be stable across Python versions, platforms etc. (Even though most of them are in practice)

  4. Most importantly: focusing on these kinds of specifics, rather than making sure to understand the underlying concept.

For what it’s worth, the recursive code you’re using doesn’t count the size of the sub-objects - it only knows about objects that don’t use __slots__ (and therefore have a __dict__). It should add a case like:

elif hasattr(obj, '__slots__'):
    size += sum([get_size(getattr(obj, i), seen)  for i in obj.__slots__])

Notice that the __slots__ attribute can be looked up from the instance, but belongs to the class - and that it contains attribute names rather than values. Similarly, the non-recursive size increases by 8 bytes for each attribute listed in the __slots__, because the instance’s memory layout needs to include an additional pointer to the sub-object stored there.

That said: the difference you’re seeing in the video is 28 bytes - the size of the integer object 42 which is shared among each slot of the B instance in the example. That has been pretty stable, but the non-recursive size of “empty” user-defined class instances - both with and without __slots__ - has varied recently.

1 Like
3

A random side note, when starting out, the optimization from slots is probably something not really thought of.

I really only see them in places where space efficiency is utmost importance. A lot of the time Python code doesn’t really have space efficiency as a first point.

Interesting: sure. Practical for early on usage: probably not.

1 Like
4

Thank you @kknechtel for your response. As of this posting, I only had read a small intro on this subject as the major discussion on the topic was slated until the next chapter. Thus, I took a small detour to view this video which provided a small introduction. I just wanted to get a heads up on the subject before going all in.

However, yes, I understand that the use of slots’ main objective is to decrease the memory footprint when using a lot of small ‘things’. Another apparent benefit, is that it keeps you from inadvertently assigning values to attributes that do not exist (creating new attributes via a spelling error, for example, when you don’t want to create new attributes) since when using slots, creating new attributes is not allowed. When you don’t use slots, you can of course, create new attributes on the fly.

In cany case, thank you for your feedback.

Much appreciated.

5

Hi,

thank you for your time in responding to my post.

Yes, that is what I have gathered so far - slots not used ubiquitously due to modern systems having enormous memory capacity (memory budgeting is really not a big concern).

However, since I am following the book as my guide as an introduction to Python fundamentals, I at least wanted to be informed and know what they are.

Thank you gain.

Much obliged. :blush:

1 Like
6

Hi everyone,

I have now had some time to study in a bit more detail the subject of slots. Based on the books cautionary message from the year 2013 (over a decade now), when the book was published, I would like to know if this still holds true:

As we’ll find, though, slots should be used only in applications that clearly warrant the added complexity. They will complicate your code, may complicate or break code you may use, and require universal deployment to be effective.

Is further understanding on this topic worth pursuing? Are slots fairly common in practice?
I would appreciate additional feedback on this subject (thoughts, insights if practical, etc.). Or this generally just novel?