__decorators__ / functools.idempotent

[chrisrodrigue]

Issues

Python has no standard way to check if a decorator already been applied to a callable, and some of the methods to check are a bit cumbersome. Additionally, decorator placement matters when descriptors come into play.

# Works: trace gets a plain function
@classmethod
@trace
def bar(cls): ...

# Breaks: trace gets a classmethod object
@trace
@classmethod
def bar(cls): ...

Ideas

__decorators__: a registry of decorator function references for easier tracking.
functools.idempotent: a decorator helper (or helper decorator?) that makes wrappers idempotent and descriptor-aware.

def idempotent(fn):
    @wraps(fn)
    def inner(*args, **kwargs):
        *prefix, obj = args
        if isinstance(obj, (staticmethod, classmethod)):
            func, desc = obj.__func__, type(obj)
        else:
            func, desc = obj, None
        decorated = getattr(func, '__decorators__', frozenset())
        if inner in decorated:
            return obj
        wrapped = fn(*prefix, func, **kwargs)
        wrapped.__decorators__ = decorated | {inner}
        return desc(wrapped) if desc else wrapped
    return inner

The intended target for @idempotent is other wrapper decorators (basically anything that you would use with functools.wraps):

@idempotent
def trace(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print(f"calling {func.__name__}")
        return func(*args, **kwargs)
    return wrapper

@trace
@trace                # no-op
def connect(): ...

@trace                # should work now (previously a footgun)
@classmethod
def from_config(cls, config): ...

Edits:

• changed fn to inner and made @idempotent more concise
• removed LLM formatting and additions and distilled it down to my original ideas (sorry for triggering some folks)

[NeilGirdhar]

I didn’t have time to really go through this proposal, but a couple thoughts:

It’s fairly common for function decoration to involve returning a function object with some additional attributes.

In my opinion, setting attributes on objects outside of __init__ is a code smell. One problem is that it nearly always fails type checking when you go to look for that attribute.

Therefore, I think the idea of having a dictionary on all functions to be beneficial. It would provide an obvious place to set and look for various things.

I’m not sure I like the idea of making this a dunder. Why not just make a catch-all for function attributes like function_attrs? Or something like that, and make it a dictionary dict[str, Any]. When coding a function decorator, you can invoke some other decorator to lift the function_attrs dict from the old function and augment it in the way you want.

[Rosuav (Chris Angelico)]

@chrisrodrigue

id() returns a memory address that the allocator can reuse after garbage collection. A stale id matching a new object would produce a false positive. Storing the reference itself is correct. Decorators are module-level objects with module lifetime, so the reference prevents no meaningful garbage collection.

So you’re saying both that stale IDs would be a problem and that stale IDs aren’t possible? I’m confused. What’s your point?

Did you get an LLM to generate this text?

@NeilGirdhar

I didn’t have time to really go through this proposal, but a couple thoughts:

Yeah, that’s the problem with AI-generated proposals. They’re incredibly long, and we have to spend more effort reading them than the OP put in creating them.

[xitop]

If I understand correctly the proposal to attach a frozenset after each decoration, it assumes that each decoration wraps the callable. But this is not always the case. Many decorators just register the callable and returns it unmodified like e.g. atexit.

[chrisrodrigue]

@Rosuav

So you’re saying both that stale IDs would be a problem and that stale IDs aren’t possible? I’m confused. What’s your point?

The point is that storing references is safer than storing IDs in principle. If one module is reloaded and others aren’t, what happens to the objects that were in the reloaded module? Could IDs be reused if those object were garbage collected?

[Rosuav (Chris Angelico)]

@chrisrodrigue

The point is that storing references is safer than storing IDs in principle.

Sure. You didn’t respond to the other part of the question though, so I’m not sure how much effort I should put into this discussion.

[chrisrodrigue]

@xitop

If I understand correctly the proposal to attach a frozenset after each decoration, it assumes that each decoration wraps the callable. But this is not always the case. Many decorators just register the callable and returns it unmodified like e.g. atexit.

I had wrapper decorators specifically in mind for this (think wherever you’d use functools.wraps).

[chrisrodrigue]

@Rosuav

Sure. You didn’t respond to the other part of the question though, so I’m not sure how much effort I should put into this discussion.

Understandable. You might want to conserve your effort then. Constantly derailing ideas with the same tangential accusations looks exhausting.

Adopt proven anyio/Trio patterns natively into asyncio (multi-release roadmap)

Yep, that checks out, given the way this proposal looks. I don’t think the thread needs to be locked. The normal reaction to LLM-generated threads is to ignore them, and that’s fine here.

Memory-Efficient Dictionary via Elastic Hashing (Achieving 40%+ RAM savings at high load factors)

And were your statistics LLM-generated too?

Optional Syntax for Typed Lambda Expressions

sigh So once again, I’ve been wasting time giving someone the benefit of the doubt as to whether the post was “polished” by an LLM. I can’t find the forum rules where it’s stated, but wasn’t it a violation of those rules to use an LLM and not say so? But, better, just don’t. Don’t use LLMs to “fix” or “improve” your posts. They invariably end up making the post worse. Sure, it might be nicely formatted, but it’s a nicely formatted piece of AI garbage. How am I supposed to judge your point abo…

Add str.tokenize() for basic alphanumeric word tokenization

Yet another proposal where I have no idea how much of the proposal came from an LLM rather than actually from the OP. Not going to engage. Stop using unacknowledged AI to generate your posts.

[pf_moore (Paul Moore)]

@chrisrodrigue

You might want to conserve your effort then.

I would also like to know whether the original proposal was LLM-generated. You’re clearly interacting now, but the original post is very like LLM-generated content (and in particular is hard to read and understand), so at a minimum, I’d like you to rewrite the proposal (in your own words, if it was LLM-generated) to be more concise and to the point.

Even if you feel that questioning whether a proposal was LLM-generated is a “tangential accusation”, it is a legitimate question, and it’s extremely easy for you to answer with a minimal “yes”, or “no”. Why have you not done so? Are you concerned that people will attack you for using an LLM? Personally, I don’t care what tools you use to help you produce a proposal, but I do care that the version you post is easy to understand and discuss, and content that LLMs produce nearly always isn’t, without significant human editing. I’d find it a lot easier to know how to advise you on improving your proposal if I knew whether what I was seeing was your writing style, or an LLM’s. (At least, if I were to offer anything more concrete than "don’t write your proposal in a way that looks like an LLM ).

[pf_moore (Paul Moore)]

@pf_moore

At least, if I were to offer anything more concrete than "don’t write your proposal in a way that looks like an LLM

… A very simple piece of advice - start by posting your idea as if you were talking casually to someone. When you’re initially exploring an idea, a formal proposal with headings and bullet points is generally offputting, rather than helpful. Formalising the proposal comes later, when you have something worth writing up as a PEP. Until then, keep things informal.

[MegaIng (Cornelius Krupp)]

I also want to point out that the proposal contains critical flaws in it’s idea that a human would have caught - making me believe that it’s mostly AI generated with little oversight. Most notably, this statement is just false:

Same convergence for staticmethod, and for multiple idempotent-wrapped decorators stacked in any order.

It just doesn’t and can’t do the latter. (it doesn’t even try)

If you want to make a proposal, at the very least make sure that it actually does what you say it does.

[Rosuav (Chris Angelico)]

@chrisrodrigue

Constantly derailing ideas with the same tangential accusations looks exhausting.

Cool. If those are “tangential accusations”, then I take it you’re admitting that it WAS generated by an LLM, and your proposal has no merit. Good to know.

[chrisrodrigue]

@MegaIng

Same convergence for staticmethod, and for multiple idempotent-wrapped decorators stacked in any order.

It just doesn’t and can’t do the latter. (it doesn’t even try)

Mind explaining? This branch should explicitly guard against reapplying the same decorator.

if decorated is not None and inner in decorated:
    return obj

[chrisrodrigue]

Some tests, and no I didn’t write these.

Online Python - IDE, Editor, Compiler, Interpreter

Edit: changed fn to inner.

[MegaIng (Cornelius Krupp)]

@chrisrodrigue

Mind explaining?

So you actually don’t understand the text you posted.

The section I quoted is under “Order Independence”, that is the property being discussed. No, the code does not guaranteed order independence. Yes, it does guarantee only-once execution if all decorates in the chain are cooperative.

[chrisrodrigue]

@MegaIng

The section I quoted is under “Order Independence”, that is the property being discussed. No, the code does not guaranteed order independence. Yes, it does guarantee only-once execution if all decorates in the chain are cooperative.

By order independence I was referring to the set of recorded decorators, not the execution order of the wrappers. That’s probably an important distinction to make.

[chrisrodrigue]

@pf_moore

so at a minimum, I’d like you to rewrite the proposal (in your own words, if it was LLM-generated) to be more concise and to the point.

Done, boiled down to the basics. It was pretty wordy.

[pf_moore (Paul Moore)]

Thanks (although you might have been better re-posting the proposal - editing the original post is generally considered bad etiquette, because people who read this site via the email interface won’t see the changes and will be confused because comments won’t refer to the text they think was posted).

Anyway, the one glaring problem here is that you’ve offered no examples at all of why anyone would care if a decorator has been applied. There’s no point adding this mechanism to Python unless it’s useful and you’ve given no indication that it would be. Just because there isn’t a way to check, doesn’t mean there should be - you have to demonstrate the need.

Also, how would __decorators__ be maintained? Just by your idempotent function? Because if so, you can write that now and don’t need a language change. On the other hand, if the language runtime is supposed to maintain it, how do you address the fact that by design decorator syntax is defined to just be application of the decorator to the function object? So, for example:

@deco
def fn(x):
    ...

is by definition equivalent to

def fn(x):
    ...

fn = deco(fn)

You can’t have an arbitrary assignment suddenly try to look at __decorators__, so anything that made the decorator syntax do so would be a breaking change, removing the existing equivalence.

[chrisrodrigue]

Maybe I’m more interested in why descriptors were implemented as decorators, since they don’t play nicely with programmatic application of other decorators. I’ll have to go dig up the PEP to figure out the rationale.

What if you want to decorate every public callable on a class, but one of those methods already has that decorator applied? Should you have to manually remove the decorator from that method? My argument is no, since applying the same decorator to an object that already has it is almost always an error.

I also don’t think that one should have to unwrap descriptors to apply decorators to methods with those descriptors. Nor should one have to worry about the order of decorators and descriptors. The fact that things break when decorators are applied over descriptors is certainly surprising.

[kkirsche (Kevin Kirsche)]

To be fair you have also mistaken people with English as a second language as just one example. This is a frequent point of yours not a once off, and it is perfectly reasonable to push back with the frequency you deploy it.