Proposal: Deprecate @types.coroutine decorator and types._GeneratorWrapper class

Core Development
1

It was briefly discussed at github issue to deprecate @types.coroutine.

I’ve made some analysis and want to make a proposal:
mark @types.coroutine decorator and types._GeneratorWrapper class as deprecated in Python 3.15.

The main reasons are that asyncio doesn’t support generator based coroutines anymore and that those features are used very little in practice.

In fact, there is only about 25 packages from top 10,000 PyPI projects that mention them. I’ll list them below.

Packages that mention types._GeneratorWrapper:

  • cython - in tests only;
  • nuitka - it has some replacement for this class which is created based on original types._GeneratorWrapper.

Packages that mention types.coroutine I divide into some parts:

  1. Mentions of types.coroutine but w/o real usages:
  • amazon-ion - introduce its own @coroutine decorator.
    This package was mentioned by @AA-Turner in the issue linked above,
    but I can’t find real usage of @types.coroutine in it.
  • aiohttp - 1 usage in payload_streamer, which is already deprecated;
  • backports_abs - 2 usages in tests, commented out;
  • jedi - 1 usage, mentioned in typeshed stubs.
  1. Usages of types.coroutine in tests only and on regular generator functions only:
  • aspidites - 1 usage;
  • ijson - 1 usage;
  • lazy_object_proxy - 9 usages;
  • nuitka - 2 usages;
  • pycapnp - 1 usage.
  1. Usages of types.coroutine on regular generator functions only (and not in tests only):
  • aiodocker - 3 usages;
  • async-generator - 5 usages;
  • curio - 1 usage;
  • esphome - 1 usage;
  • langgraph - 1 usage;
  • memory_profiler - 2 usages;
  • pony - 1 usage;
  • pytest_homeassistant_custom_component - 1 usage;
  • taskgroup - 1 usage;
  • telnetlib3 - 1 usage;
  • trio - 2 usages.
  1. Other packages:
  • cython - some usages (about 15), mostly in tests and mention it in comments in C code;
  • mypy - some usages, for example, in semantical analysis and stub generation;
  • pytype - 2 usages, some kind of overlay and mentioned in typeshed stubs.

For all packages that have usages for regular generator functions only (i.e. for all these packages, except section 4), very simple piece of code can be added:

import inspect

def types_coroutine(func):
    """Convert regular generator function to a coroutine."""

    if not callable(func):
        raise TypeError('types_coroutine() expects a callable')

    if not inspect.iscoroutinefunction(func) and inspect.isgeneratorfunction(func):
        co_flags = func.__code__.co_flags
        if not co_flags & inspect.CO_ITERABLE_COROUTINE:
            co_flags |= inspect.CO_ITERABLE_COROUTINE
            func.__code__ = func.__code__.replace(co_flags=co_flags)

    return func

Moreover, any Python code that uses @types.coroutine decorator or types._GeneratorWrapper class can copy those parts from Lib/types.py and use it with very little changes since no specific imports or C code are needed.

And in CPython codebase on main branch we have some usages of @types.coroutine:

  • 1 usage in asyncio/tasks.py
  • some mentions in docs (about 5 times)
  • some mentions in tests (about 50 times).

I’d like to thank @AA-Turner for the idea, @sobolevn for constructive instructions about discuss thread, and @hugovk and @vstinner for creating tool for analysis PyPI projects.

6 Likes
2

The alternative, slightly smaller, proposal is just to deprecate types._GeneratorWrapper and the ‘wrapping’ mechanism of @types.coroutine, as there seems to be some utility in the co_flags manipulation logic. However, I’d also be fine with deprecating the entire function, should that be the consensus.

A false positive, sorry.

cc @njs who had views from Trio previously.

A

1 Like
3

Onetime Trio developer here. I think deprecating @types.coroutine now would remove an important feature for low-level async implementors, for which no appropriate replacement currently exists.

Without @types.coroutine, how do you write an async function that can be awaited but directly yields messages to the event loop? async def functions can await other functions, but they can’t yield messages saying when they should be awoken.

In asyncio, this isn’t an issue because all asyncio event loop traps are instances of Future, which implements __await__. But we can’t assume that applies in general to all event loop traps. Curio yielded tuples; those don’t have an __await__.

@types.coroutine is also helpful in writing shims that wrap another coroutine in order to adapt it in some way (to a different async environment, for example). Examples:

2 Likes