Using get_type_hints() with cyclic imports

Typing
1

Hello,

I think I’m asking an obvious question, but maybe I’m missing something. In the face of type checking cyclic import dependencies the recommendation here is to use typing.TYPE_CHECKING so that checkers are still able to import the “other” module to access its types. This is all good and well for a static type checker.

Switching to runtime, however, if I want to reflect on types using get_type_hints() then I get a NameError (expectedly so) because — drawing from the example in the docs above — the name 'bar' (used by type annotation 'bar.BarClass') wasn’t found:

import typing
import foo
import bar

print(typing.get_type_hints(bar.BarClass.listifyme))  # {'return': list[bar.BarClass]}
print(typing.get_type_hints(foo.listify))  # NameError: name 'bar' is not defined

However, I know that bar was imported here and so I can pass it down:

print(typing.get_type_hints(foo.listify, globalns=globals()))  # {'arg': <class 'bar.BarClass'>, 'return': list[bar.BarClass]}

Problem is that this solution doesn’t scale well when nesting function calls. For example, suppose a module utils.py which contains a helper function:

import typing

def get_type_hints(thing: typing.Any) -> dict:
    return typing.get_type_hints(thing, globalns=globals())

then that module’s global namespace doesn’t contain bar:

import utils

print(utils.get_type_hints(foo.listify))  # NameError: name 'bar' is not defined

It seems to me that I’d have to pass around a namespace (or a copy thereof) which contains necessary names, if I’d like to make get_type_hints() work across multiple function calls, and that doesn’t feel right to me. It get’s more complicated because not always do I know the names required by this or that thing.

But… do I have another choice? Refactor, and consolidate the multiple modules into one? Am I perhaps missing something that might help?

Would it make sense for get_type_hints() to return a “type unknown” sentinel instead of raising an error? Is incomplete type information — in some scenarios — better than none at all?

Thanks!
Jens

2

This probably isn’t the answer you were hoping for, but…

I recommend against using TYPE_CHECKING conditionals to work around cyclical imports even for static typing. It may seem like a convenient and expedient solution, but it’s masking a deeper structural problem in your code. If you have cyclical imports, that means your code is not well layered, and your code should be refactored.

If you’re using pyright as a type checker, you can enable the reportImportCycles check to report any import cycles within your code. I always enable this rule for my team’s code bases because it enforces good architectural layering. It also eliminates issues with get_type_hints.

1 Like
Lazy imports and PEP 649
3

The opposite to Eric’s answer: maybe you can pass a custom mapping to globalns that gives you the “type sentinel” behaviour you want when it encounters a missing key. IIRC PEP 649 might use this trick.

4

Thanks @erictraut, and I have to agree with you.

The code I’m looking at is inherited from another team and they had disabled pylint’s cyclic-import warning as well. So I’m now faced with another bandaid (hence this thread) or taking the bulldozer-approach and refactoring the whole thing.

Thank you @hauntsaninja, I’ll read through PEP 649. The problem with passing a custom mapping around is that

  • I don’t always know what needs to go into the mapping; or
  • The names that need to go into the mapping aren’t in the calling module, so more imports required; or
  • There are multiple call-levels between the relevant calling module which contains the names and the function which calls get_type_hints() — all of which would have to be extended for that new argument.

But let’s see what other opinions folks here would like to share :nerd_face:

5

Doing fair bit of runtime type introspection sometimes I have code in module to register a type to a global namespace of types to deal with knowing what namespace to evaluate. But I mostly use this for cases like recursive type aliases where forward refs are inherent. Otherwise I’ve mostly avoided TYPE_CHECKING for import cycles. I do use TYPE_CHECKING but mostly for imports from places like _typeshed or if a type is generic in stubs but not generic at runtime.

So I’d lean closer to Eric’s answer of TYPE_CHECKING for import cycles does not work out well if you also care about runtime behavior. If refactor is hard registering types/objects to a global scope can help, but then you enter in fun place of import side effects. Having imports run code is both useful and easy to start seeing strange behaviors.

There’s another library sphinx-autodoc-typehints that also deals with this issue. There answer boils down to don’t use TYPE_CHECKING for import cycles.

6

While I agree with not using if TYPE_CHECKING in this manner, if you have existing code and need something that will give you a placeholder until you can refactor:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    import Foo
else:
    Foo = object()

If you want something that shows what it is and plays nicely with generics:

from typing import TYPE_CHECKING, GenericAlias


class PlaceholderGenericAlias(GenericAlias):
    def __repr__(self):
        return f"Circular import placeholder for {super().__repr__()}"

class PlaceholderMeta(type):
    def __getitem__(self, item):
        return PlaceholderGenericAlias(self, item)
    def __repr__(self):
        return f"Circular import placeholder for {super().__repr__()}"


if TYPE_CHECKING:
    import Foo
else:
    class Foo(metaclass=PlaceholderMeta):  pass

which is much more verbose, but may assist in your refactoring efforts.