Hasattr() performance on objects with __slots__

Python Help
1

For a particular application I was wondering whether to represent a state by the absence of an attribute, or by the attribute being None. Curious, I decided to benchmark it, and the result surprised me a lot:

hasattr() is very slow on specifically a slot that has no value.

Really, very very slow. I created two classes:

class C1:
    __slots__ = ('x',)

class C2:
    pass

…and benchmarked hasattr(obj, 'x') for both of them in the case where x was and was not set:

  • C1, hasattr→False: 242 ns
  • C1, hasattr→True: 30 ns
  • C2, hasattr→False: 33 ns
  • C2, hasattr→True: 31 ns

So that’s a slowdown by a factor of 7-8. Does anybody know why this is the case? Is it a bug?

Any insight would be gratefully appreciated!

Supplementary observations:

Testing was on a local build of 3.14.0 under x86_64 Ubuntu, but I replicated the phenomenon under Ubuntu’s builds of 3.13.0 and 3.12.7 .

getattr(obj, 'x', sentinel) is not sentinel is a consistent 20ns slower than hasattr(obj, 'x') across the board: the same spike is observed for a slot that has no value.

Conversely, try / getattr(obj, 'x') / catch AttributeError is slow any time the exception path is taken, for obvious reasons, but there is not an additional slowdown for a slot.

It makes very little difference what other slots the object has, whether or not it has a __dict__ slot, what other attributes there are in slots and/or __dict__, etc.

Down in the C code, the key difference seems to be whether PyObject_GetOptionalAttr() or PyObject_GetAttr() gets called. I observed that the implementation of the former looks very much more complicated than the latter, but then realised I’d gone way too far down a rabbit hole and backed out.

2

When an object has __slots__, attributes are populated ahead of time with a special descriptor object. For example:

>>> import inspect
>>> class Test:
...     __slots__ = ('x',)
...     
>>> inspect.getattr_static(Test(), 'x')
<member 'x' of 'Test' objects>

When actually looking up the attribute, Python then has to invoke __get__, which in this case means formatting and constructing an AttributeError object, which the other paths don’t have to do (because they have fast paths that know when they’re being used under hasattr), thus causing the apparent slowdown.

I suppose we can optimize this by adding a special case in _PyObject_GenericGetAttrWithDict that avoids invoking tp_descr_get when looking up an exact PyMemberDescr_Type instance. Please file an issue for this on our issue tracker.

3

Thanks; done.

I guess you’re right and the special case is the easy win. It feels a little inelegant, but anything cleaner would involve an API change…

4

Actually, are C extensions allowed to call tp_descr_get directly? If not, maybe its spec could be augmented to permit returning NULL without setting an exception, to indicate the attribute was not found. That could give a clean separation of concerns. :thinking: