Immutability in Python is *really* hard

__slots__ is primarily present for the purpose of improving attribute lookup performance and memory usage, not limiting functionality. @guido mentioned this in a blog post:

(“Slots” section of The History of Python: The Inside Story on New-Style Classes)

1 Like

On Tue, Nov 26, 2019 at 12:19:29AM +0000, Marco Sulla via Discussions on
Python.org asked:

“So why __slots__?”

It’s a memory optimization, although I suspect that in recent versions
of Python the memory saving is much smaller than it used to be.
Nevertheless, you still save some memory.

By using __slots__, you avoid needing an instance __dict__, which
saves a pointer to the dict, plus the space of the dict which starts at
136 bytes and goes up from there according to my quick and dirty tests.

See also:

The use of __slots__ to prevent adding new attributes is officially an
abuse of the feature, although it’s one that probably most people have
done :slight_smile:

1 Like

Indeed. And IMHO,__slots__ is the poor cousin, for only class members and difficult to inherit, of the Java final statement.

Maybe I was not clear at the start of my thread, but I’m not searching immutability for teaching people how to live. I’m searching it for thread-safety, and to discourage monkey-patching. And adding performance and memory saving is a good side effect :smiley:

__slots__ isn’t for class members (class attributes), it is for
instance attributes. And it is nothing like Java’s final:

  • final classes can not be subclassed;
  • final methods can not be overloaded or overridden;
  • final variables (attributes) can only be assigned to once.

None of those apply to __slots__ in Python.

Marco, you say that you don’t want to teach people how to live
(program) but then say that you want to discourage monkey-patching.
That is a contradiction.

Why are you so against monkey-patching? It is a useful technique and not
abused by the Python community. I would say that monkey-patching is
probably used less often than global variables, at least in my
experience: I’ve seen plenty of global variables in production code, but
very little use of monkey-patching outside of test suites.

3 Likes

I have to disagree with this analogy. As Steven mentioned, Java’s final and Python’s __slots__ are not comparable: they have entirely different behaviors and intended purposes.

If you’re looking for implementing thread safety in multi-threaded code, __slots__ is not the way to go about doing so. A combination of good design practices and synchronization primitives (for interacting with shared objects) such as locks, events, conditions, and semaphores will accomplish that. In an asynchronous context, if you need to schedule a callback in a thread outside of the main thread, we have an asyncio.loop.call_soon_threadsafe() method.

While an object is that fully immutable is inherently thread safe, immutability is far from the only way to accomplish thread safety.

Also, I’m not convinced that monkey patching is something we should discourage in any official capacity. In the the contexts of unit tests or extending upon existing functionality, it can be quite useful when it’s done correctly.

Immutability by itself does not guarantee any significant memory saving or performance improvements. Especially in the case of simply limiting access/visibility of instance attributes, which seems to be a focus in this topic.

In the specific case of __slots__, the improvement is largely from:

  1. Not having to initialize or maintain __dict__ (internal instance dictionary)
  2. Not having to initialize or maintain __weakref__ (attribute for maintaining weak references to the current object)
  3. Storing the declared attributes within a static array and using descriptors to interact with them through their index

As demonstrated in the stackoverflow question linked above, the improvement is only substantial when a very large number of objects are created. For most situations, there simply isn’t a need to use __slots__ in the first place.

If anything, this topic seems to indicate that perhaps we aren’t as clear as we could be about the purpose of __slots__. The documentation mentions the memory saving and attribute lookup speed, but it doesn’t mention that it’s not meant to be used for restricting access to instance attributes or that it’s only a significant memory reduction when a very large number of objects are initialized from the class that defines it.

I suspect that some readers may be misled by “The space saved over using __dict__ can be significant. Attribute lookup speed can be significantly improved as well”, assuming that __slots__ provides a significant performance gain in most situations, when in reality it has a more specific purpose.

Marco said:

“My wish is that __slots__ will be more simple,”

Can you be more specific? What part of slots is not simple enough? It
seems pretty simple to me:

class MyClass:
    __slots__ = ('spam', 'eggs')

and then write the rest of the class as you normally would do. What
would you like to change?

Marco: “and to discourage bad coding practices, like monkey patching.”

The Python community is not like the Ruby community, we do not have a
problem with monkey-patching.

(Note that this blog post was more than a decade ago, I have no idea if
the Ruby community still has a problem in this regard.)

I’ve never come across anything like that in the Python community, and I
don’t know anyone who has. So I don’t think we need to do more to combat
monkey-patching than what the Python culture already does: we mildly
discourage our fellow developers from using monkey-patching, and the
language prevents the worst excesses by disallowing the modification of
builtin classes.

(But if you really want to, you can shadow the original.)

We allow monkey-patching in the language because it is sometimes useful,
especially for experimentation, testing, and in the REPL.

What more would you like to do, and why?

1 Like