I’m struggling to use contextmanager when an instance of a class has __enter__ and __exit__ attributes, but the class does not. The code below shows it best.
class ManagedResource:
def __enter__(self):
return self
def __exit__(self, *args):
pass
class ResourceWrapper:
def __init__(self, resource):
self.resource = resource
def __getattr__(self, name: str):
# Logging, other wrapper stuff
return getattr(self.resource, name)
wrapped_resource = ResourceWrapper(ManagedResource())
print(hasattr(wrapped_resource, "__enter__")) # True
print(hasattr(wrapped_resource, "__exit__")) # True
with wrapped_resource:
print("Doesn't work :(")
I can see why this happens looking at
Would it be acceptable to check for the existence of the functions using hasattr on the instance to cover use cases such as mine? Is my use case stupid?
By Gracecr via Discussions on Python.org at 09Sep2022 17:00:
I’m struggling to use contextmanager when an instance of a class has __enter__ and __exit__ attributes, but the class does not.
You’re writing a proxy!
In your case, the proxied object’s class (self.resource.__class__) has
the methods.
Your difficulty comes from the fact that dunder methods such as __enter__ are looked up on the class and your wrapper/proxy class does
not have these methods. By contrast, hasattr() and getattr() look on
the instance and get helped by your __getattr__.
You could just provide stubs for each dunder method you want to support,
like:
I think the general way to do this uses a metaclass:
Making proxies is actually listed as one of the many things you can do
with metaclasses, but my metaclass fu is weak. Have a read of the above
docs and make yourself a small metaclass and see what you can do with
it. I suspect you want to give your ManagedResource class a metaclass
to provide the omitted dunder methods on demand, but I’m not sure how to
do it.
In my case, I need to accept potentially unknown types. I’ve wound up with something like the following:
from typing import TypeVar
T = TypeVar("T")
class ManagedResource:
def __enter__(self):
return self
def __exit__(self, *args):
pass
def create_proxy(resource_type: T) -> T:
class ResourceProxy(resource_type):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def __getattribute__(self, name: str):
# Logging, other wrapper stuff
return super().__getattribute__(name)
return ResourceProxy
wrapped_resource = create_proxy(ManagedResource)()
print(hasattr(wrapped_resource, "__enter__")) # True
print(hasattr(wrapped_resource, "__exit__")) # True
with wrapped_resource:
print("Works! :)")
As a bonus, isinstance and issubclass also work. This proxy is about as transparent as it gets. The type hinting feels a bit weird, as the output of create_proxy is not really the same class as the input, but it seems to work.
I do not believe this implementation needs __getattribute__ at all. Or
the __init__. It would just want whatever shims are needed for the
“manage” part - i.e. intercept (“override”) the methods which want
extra behaviour.
As a bonus, isinstance and issubclass also work. This proxy is about as transparent as it gets.
This is a factory for a subclass, not a proxy. That’s why isinstance
and issubclass work.
A proxy is a distinct object which provides access to another (proxied)
object. Usually because the other object already exists. Example:
with open("somefile") as f:
proxy = LoggingFileProxy(f)
print("something", file=proxy)
where LoggingFileProxy is a proxy class to handle the various file
methods, and hand them off to the underlying file. Maybe such a proxy
might log read/write or serialise some things or do synchronous writes
or… whatever. A proxy class will have an __init__ which does some
variation on this:
class ProxyClass:
def __init__(self, proxied):
self.__proxied = proxied
and then have all the methods call the corresponding method on self.__proxied much as in your original example.
A subclass is not the same thing at all: it is for when you get to
create the original thing, rather than wrapping a preexisting thing.
Now, maybe a factory for a subclass is actually suitable for what you’re
doing. It’s a valid thing to write; I wrote one just yesterday:
The type hinting feels a bit weird, as the output of create_proxy is
not really the same class as the input, but it seems to work.
I think the type hinting is irrelevant. Try throwing it away entirely
and retesting.
This type variable is later used to annotate create_proxy() to tell that it returns the same type as the type of its argument:
def create_proxy(resource_type: T) -> T:
...
It is normal that type annotation does not have a run-time effect. It is normally intended for type-checkers, documentation and as an aid for IDEs.
I do not think the type-hinting is wrong. create_proxy() accepts a class and returns a class. Variables bound to a class are of the same type. Type hinting is relevant when we want to use it.
By Václav Brožík via Discussions on Python.org at 12Sep2022 22:58:
This type variable is later used to annotate create_proxy() to tell that it returns the same type as the type of its argument:
def create_proxy(resource_type: T) -> T:
...
It is normal that type annotation does not have a run-time effect. It is normally intended for type-checkers, documentation and as an aid for IDEs.
Yes. I should have been clear that it has no runtime effect on making
the OP’s thing function.
When debugging, it is good to pare things back as far as possible so
that you know what changes helped fix things, and what changes were not
relevant. Otherwise people, myself included, sometimes leave things in
because “after I did that (and other things) the thing worked”. And
that’s just witchcraft.
I do not think the type-hinting is wrong. create_proxy() accepts a class and returns a class. Variables bound to a class are of the same type. Type hinting is relevant when we want to use it.
Sure, but the OP’s returning a subclass, not the original class. So the
type annotation is at least slightly incorrect, which will
confuse/mislead type checkers if used. At the least the return type of create_proxy should probably be TypeVar(f'{resource_type.__name__}_proxy',bound=T) or something like
that. Since I gather that’s how you express a “subclass of T” in typing.
Anyway, my point was that I do not think the type annotations had any
relationship to having the code function, and could be removed if only
to test whether that was the case.
The type hinting was a bit of a tangent, probably shouldn’t have included that.
I’ve asked for help with a symptom of a bad solution to a different problem. You’ve offered great advice for my attempt at a minimal reproducible example, but my example lacks context. Sorry about that. Nonetheless, your advice pointed me to a solution!
Let me try to explain my use case with more context.
I am building a utility for test bench configuration. The user requests a type of instrument from my utility, and my utility returns a driver for that instrument. For example, the user requests a power supply, the utility returns an object that controls the power supply.
I want every call to the power supply driver to be logged.
The drivers are described in a configuration file that the user is able edit. Because I don’t know what methods or properties the drivers have, I use __getattribute__ to cover them all.
Previously I was using a proxy with __getattribute__ and returning the driver proxy to the user. This worked well, but I ran into the posted issue: drivers that supported contextlib didn’t work.
I could have added __enter__ and __exit__ to my proxy as you suggested, but not all drivers support these methods. I could have raised NotImplementedError in those cases, and that would have worked.
The subclass factory method also makes isinstance and issubclass work in a more sensible way for my users, so that’s the way I think I’ll go.
I don’t control class creation, so I don’t think metaclasses would work for me.
I could also give up on the logging. Seems more trouble than it’s worth!
I’ll be sure to change my function and class names so that they don’t refer to things as proxies when they aren’t really proxies.
Thanks again for your help. The repo you linked is quite impressive! I spent an hour browsing around and I feel I’ve only scratched the surface!
By Gracecr via Discussions on Python.org at 13Sep2022 19:39:
I am building a utility for test bench configuration. The user requests
a type of instrument from my utility, and my utility returns a driver
for that instrument. For example, the user requests a power supply, the
utility returns an object that controls the power supply.
Nice.
I want every call to the power supply driver to be logged.
The drivers are described in a configuration file that the user is able edit. Because I don’t know what methods or properties the drivers have, I use __getattribute__ to cover them all.
Previously I was using a proxy with __getattribute__ and returning the driver proxy to the user. This worked well, but I ran into the posted issue: drivers that supported contextlib didn’t work.
I believe you’d have run into the same issue with other things backed by
dunder methods. Eg something as simple as len(your_driver).
I could have added __enter__ and __exit__ to my proxy as you suggested, but not all drivers support these methods. I could have raised NotImplementedError in those cases, and that would have worked.
Except that you’d need to provide stubs for everything which might be
done via a dunder. A defined (but open ended in the future) set. Tedious
and hard to do completely.
The subclass factory method also makes isinstance and issubclass
work in a more sensible way for my users, so that’s the way I think
I’ll go.
It is also more complete. Since you get a type rather than an instance,
it is indeed the way to go.
I don’t control class creation, so I don’t think metaclasses would work for me.
I imagine you could add a metaclass to the subclass, but that doesn’t
actually bring any benefit.
I could also give up on the logging. Seems more trouble than it’s worth!
Logging is good. Even if you’re not logging, I can imagine wanting other
instrumentation (which might itself do some logging): timing of calls,
logging of calls which raise exceptions, coverage checks to see if a
driver does some minimal level of testing or that the tests cover enough
methods of the type. Dunno. I can even imagine wanting to be sure that a
user defined type provide certain minimum methods, which you could do by
making an abstract class requiring those methods, and making your
subclass inherit from that abstract class: missing methods → failure to
instantiaite a subclass instance.
I’d be interested to hear how this stuff pans out for you.