Allow `obj[key]` to fallback to attribute-based `__getitem__`

Let me clarify my proposal a bit, because my earlier explanation may not have been clear enough.

What I noticed is that, in CPython, special methods do not behave like normal user-defined methods: they bypass __getattribute__, and they also don’t fall back to __getattr__. Ideally, one might imagine a model where special methods participate in the same lookup chain as regular methods — but introducing that globally, for all special methods, would be a far-reaching change, and even if it were technically possible, it would absolutely require a PEP.

That’s why my patch does not change the general special-method model.
Instead, it adds a fallback after the existing slot-based lookup fails, and only for __getitem__. In this approach:

  • the normal behavior stays untouched,
  • the fallback runs only when the special method is not defined,
  • and the patch does not affect any existing code paths.

As you mentioned, this idea could theoretically be extended to other special methods. I haven’t done that because I simply don’t have the bandwidth to evaluate every method thoroughly — so I only implemented and tested __getitem__ for now. But the same pattern could apply to methods like __contains__ or __len__, and even __eq__ wouldn’t be impacted, because Python automatically provides an implementation when one is not defined, meaning the fallback would never trigger for it anyway.

So this proposal isn’t about making __getitem__ behave differently from other special methods — it’s about adding a narrow, post-failure fallback that preserves all existing semantics, while making delegation-based patterns slightly less repetitive.

Here is a concrete example:

1. Original version (without any fallback patch)

When using _Config as a read-only, delegation-based wrapper, a load_config proxy must manually forward every operation it wants to expose:

class _Config(dict):
    def __getitem__(self, key):
        value = super().__getitem__(key)
        if isinstance(value, dict):
            return _Config(value)
        elif isinstance(value, list):
            return [_Config(v) if isinstance(v, dict) else v for v in value]
        return value

    def __getattr__(self, key):
        if key.startswith("__") and key.endswith("__"):
            cls = type(self)
            if key in cls.__dict__:
                func = cls.__dict__[key]
                return func.__get__(self, cls)
            return super().__getattribute__(key)
        return self.__getitem__(key)

    # Explicitly block all mutation to preserve immutability
    def __setitem__(self, key, value): raise NotImplementedError
    def __setattr__(self, key, value): raise NotImplementedError
    def __delitem__(self, key): raise NotImplementedError
    def __delattr__(self, key): raise NotImplementedError

    def items(self):
        for key, value in super().items():
            if isinstance(value, dict):
                yield key, _Config(value)
            elif isinstance(value, list):
                yield key, [_Config(v) if isinstance(v, dict) else v for v in value]
            else:
                yield key, value


class load_config:
    def __init__(self, filepath):
        self.filepath = filepath

    def _load(self):
        row = yaml.safe_load(open(self.filepath, "r", encoding="utf-8"))
        return _Config(row)

    # Without a fallback, every operation must be forwarded manually
    def __getitem__(self, key): return self._load()[key]
    def __getattr__(self, name): return getattr(self._load(), name)
    def __setitem__(self, key, value): return self._load().__setitem__(key, value)
    def __setattr__(self, key, value):
        if key == "filepath":
            return super().__setattr__(key, value)
        return setattr(self._load(), key, value)
    def __delitem__(self, key): return self._load().__delitem__(key)
    def __delattr__(self, key):
        if key == "filepath":
            return super().__delattr__(key)
        return delattr(self._load(), key)

This is a large amount of boilerplate for something that is conceptually just:

“load the file and then behave exactly like _Config.”


2. Version with the proposed fallback patch

With a fallback that applies only when the class has no corresponding slot or method, the proxy can be dramatically simplified:

class load_config:
    def __init__(self, filepath: str):
        self.filepath = filepath

    def __getattribute__(self, key):
        row = yaml.safe_load(open(
            object.__getattribute__(self, "filepath"),
            "r",
            encoding="utf-8"
        ).read())
        return _Config(row).__getattr__(key)

That’s all that is needed.

Because _Config already implements (and blocks) all relevant special methods, the proxy no longer has to forward: __getitem__, __setitem__, __delitem__, __setattr__, __delattr__ and so on.

The fallback means the proxy “inherits” all these behaviors automatically — only in the case where it has not defined them itself — which is exactly the desired delegation pattern.

And since the fallback is triggered only after normal slot lookup fails, there is no conflict when key = filepath.

The example above reloads the file at the root level on every access, which would obviously be inefficient in real use. It’s only meant as a demonstration: there are real scenarios where a wrapper needs to delegate almost all operations to the underlying object.

One currently workable alternative is to use decorators like @wrapper on each method to forward behavior. But that means placing a decorator in front of every function you want to proxy. It’s not elegant, not visually clean, and it’s easy to miss one during maintenance.

If this improvement is considered acceptable, I can try to extend the patch to all applicable special methods and provide thorough testing when I have time.

I don’t know if the “you” there is me, but I have not talked about extending this idea to other special methods. I have pointed out that you are doing something only for __getitem__ while claiming that the goal is to make it behave like other special methods.

You can’t have both of these sentences be true. If you are changing something “only for __getitem__”, then it will behave differently than other special methods. Granted, much of the lookup mechanism is the same, but you are providing a fallback only for one special method. This is a bad idea.

You have a two-line fix to your proxy object that works today and makes clear that you want to forward on both attribute access and item access. It’s a good solution. There’s no reason to change anything about Python to avoid those two lines.

3 Likes

@mmdsl you are replying at a rate and volume, and with a tone, that leaves me concerned that you are relying on an LLM to produce your text, without disclosing that to your readers.
If you are doing so, please stop so that we can actually talk to one another.

More generally, please just slow down and avoid overwhelming the discussion. I have use cases for proxy objects which might want a way to opt in to forwarding dunders, but the current thread doesn’t feel like it leaves room for conversation.

12 Likes

What I would ideally like is for special methods to behave more like regular methods, but that isn’t realistic at the moment. So instead, I’m proposing a simple fallback mechanism as a lightweight way to approximate that behavior. It requires very little modification, has no measurable performance impact, and slightly reduces the amount of boilerplate code.

I’m starting by testing this with a single special method and gathering feedback. If the idea proves viable, I can explore applying and testing the same approach for other special methods as well.

That what I’am tring to do!

I am sorry about this: I am not English native speaker, so LLM is used to translate my word into English. I apologize for this, and now I am tring to type words by myself.

2 Likes

I am newbie here, if there is anything I’am dong wrong, please tell me. Thanks!

2 Likes

You should take a minute to look at the participation guidelines for this forum:
https://discuss.python.org/faq

I’m particular, there’s strong advice there to avoid trying to reply to every post or posting too much in a thread. It makes it hard for those of us who check in infrequently to participate when threads move too fast, and it tends to stifle, rather than encourage, good communication.

I personally very strongly prefer to read your original text (translated nearly verbatim from your primary language if you feel that is necessary) vs LLM output. I know several others in these forums share that preference. But you don’t have to accommodate me on that. I think it’s much more important that you tell people that you are using such a tool – it significantly changes how we read the text.


Back to the original topic, I find it’s a rare but real need to wrap some object, proxy all methods including dunders, and intercept one or two specific methods.

I don’t think making getattr do that is appropriate. But I’d be interested in a dedicated proxy type or something similar.

3 Likes

Can we not have ChatGPT running our discussions, please?

1 Like

Apart for the fact that you were using y some kind of AI / LLM (as stated before, it is pretty obvious too), this wouldn’t really make sense to implement that way. If I really need a proxy that works for any kind of attribute access, even when it is done via syntax, just make a big class defining all these attributes, and have a default implementation for each. I have done such things before, so I will happily share this code.

Code

Note that some things may be missing / not yet implemented 100% correctly, but this should help you get started.

class suppress:

    def __enter__(self) -> Self:
        return self

    def __exit__(self, *excinfo: Any) -> bool:
        return True

    async def __aenter__(self) -> Self:
        return self

    async def __aexit__(self, *excinfo: Any) -> bool:
        return True


class arithmetics:

    def __neg__(self) -> Any:
        return -1

    def __pos__(self) -> Any:
        return 1

    def __add__(self, other: Any) -> Any:
        # Usually x + y is the same as y + x;
        # Users could define it differently.
        # E.g. {1, 2} + [3, 4] is not the same
        #      as [3, 4] + {1, 2}
        # Instead of trying to compute x + y,
        # or even raising an error, we simply
        # return self.
        return self

    def __radd__(self, other: Any) -> Any:
        return self

    def __sub__(self, other: Any) -> Any:
        return self

    def __rsub__(self, other: Any) -> Any:
        return self

    def __mul__(self, other: Any) -> Any:
        return self

    def __rmul__(self, other: Any) -> Any:
        return self

    def __truediv__(self, other: Any) -> Any:
        return self

    def __rtruediv__(self, other: Any) -> Any:
        return self

    def __mod__(self, other: Any) -> Any:
        return self

    def __rmod__(self, other: Any) -> Any:
        return self

    def __floordiv__(self, other: Any) -> Any:
        return self

    def __rfloordiv__(self, other: Any) -> Any:
        return self

    def __pow__(self, other: Any) -> Any:
        return self

    def __rpow__(self, other: Any) -> Any:
        return self

    def __mathmul__(self, other: Any) -> Any:
        return self

    def __rmathmul__(self, other: Any) -> Any:
        return self

    def __divmod__(self, other: Any) -> Any:
        raise NotImplementedError("divmod operation is not defined for nul objects")

    def __rdivmod__(self, other: Any) -> Any:
        raise NotImplementedError("divmod operation is not defined for nul objects")


class assignments:
    def __iadd__(self, other: Any) -> Self:
        return self

    def __isub__(self, other: Any) -> Self:
        return self

    def __imul__(self, other: Any) -> Self:
        return self

    def __itruediv__(self, other: Any) -> Self:
        return self

    def __imod__(self, other: Any) -> Self:
        return self

    def __ifloordiv__(self, other: Any) -> Self:
        return self

    def __ipow__(self, other: Any) -> Self:
        return self

    def __imatmul__(self, other: Any) -> Self:
        return self

    def __iand__(self, other: Any) -> Self:
        return self

    def __ior__(self, other: Any) -> Self:
        return self

    def __ixor__(self, other: Any) -> Self:
        return self

    def __irshift__(self, other: Any) -> Self:
        return self

    def __ilshift__(self, other: Any) -> Self:
        return self


class booleans:

    def __invert__(self) -> Any:
        return not bool(self)

    def __and__(self, other: Any) -> Any:
        # bool(self) == False; False & x => False
        return False

    def __rand__(self, other: Any) -> Any:
        # bool(self) == False; False & x => False
        return False

    def __or__(self, other: Any) -> Any:
        # bool(self) == False; False | x => x
        return bool(other)

    def __ror__(self, other: Any) -> Any:
        # bool(self) == False; False | x => x
        return bool(other)

    def __xor__(self, other: Any) -> Any:
        return False ^ other

    def __rxor__(self, other: Any) -> Any:
        return False ^ other

    def __rshift__(self, other: Any) -> Any:
        return self

    def __rrshift__(self, other: Any) -> Any:
        return self

    def __lshift__(self, other: Any) -> Any:
        return self

    def __rlshift__(self, other: Any) -> Any:
        return self


class collections:

    def __len__(self) -> int:
        return 0

    def __iter__(self) -> Iterator[Any]:
        yield None

    def __getitem__(self, item: Any) -> Any:
        return self

    def __setitem__(self, item: Any, value: Any) -> None:
        pass

    def __delitem__(self, item: Any) -> None:
        pass

    def __contains__(self, item: Any) -> bool:
        return False

    def __reversed__(self) -> Iterator[Any]:
        yield None

    def __next__(self) -> Iterator[Any]:
        yield None

    # TODO
    def __missing__(self, item: Any) -> None:
        pass

    def __length_hint__(self) -> int:
        return len(self)


class comparisons:

    def __eq__(self, other: Any) -> bool:
        return self is other

    def __ne__(self, other: Any) -> bool:
        return self is not other

    def __lt__(self, other: Any) -> bool:
        with suppress():
            return not other <= self
        return False

    def __gt__(self, other: Any) -> bool:
        with suppress():
            return not other >= self
        return False

    def __le__(self, other: Any) -> bool:
        with suppress():
            return not other < self
        return False

    def __ge__(self, other: Any) -> bool:
        with suppress():
            return not other > self
        return False


class convert:

    def __repr__(self) -> str:
        return f"<nul object at {id(self)}>"

    def __str__(self) -> str:
        return f"<nul object at {id(self)}>"

    def __bool__(self) -> bool:
        return False

    def __int__(self) -> int:
        return 0

    def __float__(self) -> float:
        return 0.0

    def __bytes__(self) -> bytes:
        return b""

    def __complex__(self) -> complex:
        return complex()

    def __format__(self, string: str) -> str:
        return string


class mathematics:

    def __abs__(self) -> float:
        return float(self)

    def __index__(self) -> int:
        return int(self)

    def __round__(self) -> int:
        return int(self)

    def __trunc__(self) -> int:
        return int(self)

    def __floor__(self) -> int:
        return int(self)

    def __ceil__(self) -> int:
        return int(self)


class nul(
    arithmetics,
    assignments,
    booleans,
    collections,
    comparisons,
    convert,
    mathematics,
):

    def __del__(self) -> None:
        pass

    def __hash__(self) -> int:
        return id(self)

    def __call__(self, *args: Any, **kwargs: Any) -> None:
        pass

    # TODO
    def __getattribute__(self, attr: str) -> Any:
        return self.__dict__.get(attr, None)

    # TODO
    def __getattr__(self, attr: str) -> Any:
        return self.__dict__.get(attr, None)

    def __setattr__(self, attr: str, value: Any) -> None:
        pass

    def __delattr__(self, attr: str) -> None:
        try:
            # Don't delete following:
            # - __getattribute__
            # - __getattr__
            # - __setattr__
            # - __delattr__
            if not "attr" in attr:
                del self.__dict__[attr]
        except KeyError:
            pass

    def __dir__(self) -> Iterable[str]:
        return self.__dict__.keys()

    # TODO
    def __set_name__(self, cls: type, name: str) -> None:
        pass

    # TODO
    def __get__(self, instance: Any, owner: type) -> Any:
        return None

    # TODO
    def __set__(self, instance: Any, value: Any) -> None:
        pass

    # TODO
    def __delete__(self, instance: Any) -> None:
        pass

    # TODO
    def __init_subclass__(cls) -> None:
        super().__init_subclass__()

    # TODO
    def __mro_entries__(self, mro: tuple[Any, ...]) -> tuple[Any, ...]:
        return ()

    def __class_getitem__(cls, item: Any) -> type:
        return cls

    # __prepare__
    # __instancecheck__
    # __subclasscheck__

    # TODO
    def __await__(self) -> Iterator[Any]:
        yield None

    # __buffer__
    # __release_buffer__

    # __subclasses__
1 Like

This is a good idea to use inheritance, but in my case( _Config), I hope “some opration”(in this case, reload config file) can be done before each call, not only when init. If you have any idea, please tell me!

Something to consider is that special methods proxied in this way won’t
quite behave the same way as non-proxied ones, because the lookup on the
target done by the proxy’s getattr will be a normal one and not a
class-only one. So making this behaviour built-in still wouldn’t allow
creating completely transparent proxies.

To make that possible, there would need to be a new special method such
as getspecial used as the fallback for special method lookups
instead of getattr.

@nedbat First, I apologize for using LLM and made a lot of confusions. Now I did more test and re-edit the topic to more clearly show my idea. Would you mind watching my topic again?

1 Like

You haven’t responded to my points. Your proposal would treat __getitem__ differently than other special methods, and you haven’t explained why that is a good idea. The benefit doesn’t outweigh the extra twists in the behavior.

You have a solution that involves adding two lines to your proxy class to be explicit about what you want it to do. That is a good solution, and requires making no changes to Python. Why add a special case for one special method when you can do what you need already?

For your personal use case, you wanted __getitem__ to work without extra code, but the next person might want __len__, or __str__, or some other subset of the special methods to work without extra code. We can’t guess what methods people will want to work this way, and we can’t change the behavior for all of them.

It is surprising at first that __getattr__ didn’t do what you wanted. That is something that people need to learn about Python. We shouldn’t add a special case to one special method to avoid that surprise for some users.

Write the two lines in your proxy class. Done.

2 Likes

How about this?

MAGIC_NAMES = (
    'abs', 'add', 'aenter', 'aexit', 'and', 'bool', 'bytes', 'ceil', 'complex',
    'contains', 'delitem', 'divmod', 'enter', 'eq', 'exit', 'float', 'floor',
    'floordiv', 'format', 'ge', 'getitem', 'gt', 'iadd', 'iand', 'ifloordiv',
    'ilshift', 'imatmul', 'imod', 'imul', 'index', 'int', 'invert', 'ior',
    'ipow', 'irshift', 'isub', 'iter', 'itruediv', 'ixor', 'le', 'len',
    'length_hint', 'lshift', 'lt', 'mathmul', 'missing', 'mod', 'mul', 'ne',
    'neg', 'next', 'or', 'pos', 'pow', 'radd', 'rand', 'rdivmod', 'repr',
    'reversed', 'rfloordiv', 'rlshift', 'rmathmul', 'rmod', 'rmul', 'ror',
    'round', 'rpow', 'rrshift', 'rshift', 'rsub', 'rtruediv', 'rxor',
    'setitem', 'str', 'sub', 'truediv', 'trunc', 'xor'
)
def make_forward(name):
    def forward(self, *args, **kwargs):
        return self.__getattr__(name)(*args, **kwargs)

    return forward

class ProxyMeta(type):
    def __new__(mcls, name, bases, namespace, **kwargs):
        for magic_name in MAGIC_NAMES:
            dunder_name = f"__{magic_name}__"
            namespace.setdefault(dunder_name, make_forward(dunder_name))

        return super().__new__(mcls, name, bases, namespace, **kwargs)

class Proxy(metaclass=ProxyMeta):
    def __init__(self, target):
        self.target = target

    def __getattr__(self, name):
        return getattr(self.target, name)

print(Proxy({"a": 1})["a"]) # 1
1 Like

@nedbat I do not suggest to treat __getitem__ differently, __getitem__ is just a special example here. From __getitem__ example, I found python have different behavior on special methods.
Actually bind(or inject) special method on class level always work! What comfuse me is: this do not always work on instance level:
Example, run code:

def f(*args,**kwargs):
    return 1

class A:
    pass

A.__getitem__ = f
print(A()[0])

class B:
    pass

b=B()
b.__getitem__ = f
print(b[0])

outputs:

>>> def f(*args,**kwargs):
...     return 1
...
>>> class A:
...     pass
...
>>> A.__getitem__ = f
>>> print(A()[0])
1
>>>
>>> class B:
...     pass
...
>>> b=B()
>>> b.__getitem__ = f
>>> print(b[0])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'B' object is not subscriptable

The key is: Why do not make them have same behavior?

You need to stop saying this because your proposal is exactly about treating __getitem__ differently.

Read this: 3. Data model — Python 3.14.0 documentation

The rationale behind this behaviour lies with a number of special methods such as __hash__() and __repr__() that are implemented by all objects, including type objects. If the implicit lookup of these methods used the conventional lookup process, they would fail when invoked on the type object itself

This behavior is surprising at first, but is clearly documented, and will not change. Please find a different way to solve your problem. I’ve shown you a way.

2 Likes

Iirc that doesn’t work in all cases, as dynamic attribute setting (even in namespace for __new__) does not get resolved for syntactic structures, e.g. __getitem__ for x[y].

If it does work, that would obviously be shorter, although stuff like type-hints, default behaviors and argument positions are lost.

From my testing that does appears to work. But obviously you need to add all magic names to the list.

1 Like