isinstance() doesn’t work with Annotated[] either.
It’s true that isinstance() does work with some simple TypeAlias aliases, but not with type statement aliases.
I have observed too that sometimes I cannot use type X = ... and have to use X = ....
For example, in my current project, I have two occurrences of something like this:
# NOTE: Don't use `type`, pydantic doesn't like it.
Value = str | float | bool | Quantity
It seems to me like both of the most recent issues here would be solved by pydantic understanding type aliases (and accessing some_alias.__value__ as appropriate.)
1 Like
I haven’t used pydantic before but maybe this issue is the problem?
It seems that type checkers cannot understand this:
from typing import TypeAlias, reveal_type
from pydantic import TypeAdapter
my_type_alias: TypeAlias = list[dict[str, list[int]]]
x: my_type_alias = [{"a": [1, 2, 3]}]
res_x = TypeAdapter(type=my_type_alias).validate_python(x)
reveal_type(res_x) # type of res is "list[dict[str, list[int]]]"
type my_type = list[dict[str, list[int]]]
y: my_type = [{"a": [1, 2, 3]}]
res_y = TypeAdapter(type=my_type).validate_python(y)
reveal_type(res_y) # Any/Unknown
It does work correctly at runtime as in if you pass an invalid object for y then it will be rejected at runtime. The type checkers infer Any because this is how pydantic spells the type annotations for TypeAdapter:
class TypeAdapter(Generic[T]):
@overload
def __init__(self, type: type[T]): ...
@overload
def __init__(self, type: Any): ...
def validate_python(...) -> T: ...
A TypeAliasType is not a type so the the type checkers conclude that you have a TypeAdapter[Any] meaning that the parameters and return type for validate_python are all Any.
I’m not sure how it should be typed though and looking at it now should TypeAliasType not be a generic type like TypeAliasType[T] so that you can reference the underlying type T in annotations?
It seems that TypeAliasType.__value__ is marked in stubs as Any but I would have envisaged something like:
class TypeAliasType(Generic[T]):
@property
def __value__(self) -> T: ...
Any is the correct annotation for .__value__ currently. This is a place that typecheckers have to special-case to support in one way or another, as there’s a syntactic construction that’s not a standard object.
Much as typecheckers synthesize method signatures for typeddicts, they probably should synthesize knowledge of the type of .__value__ here
I don’t think that has anything to do with the __value__ attribute. The issue seems to be that TypeAdapter is annotated as taking type[T], and mypy allows list[...] to be passed for type[T] (incorrectly, in my view), but not an instance of TypeAliasType.
TypeForm (PEP 747) should fix this.
2 Likes
Pydantic has proper support for TypeAliasType (and we improved it in 2.11). However, such aliases aren’t treated the same way implicit/old style aliases are. The reason for this difference is documented in the warning here.
I think it might be worth keeping TypeAlias not deprecated, and documenting it as an alternative to the type statement for runtime purposes?
4 Likes
I would support this. Maybe generic TypeAliass should still be deprecated though. I don’t think you can really use them at runtime anyway and they’re weird because of the implicitly defined order of the type parameters.
I believe pydantic to be doing users a disservice by treating type statements differently like this. In the typing specification and in the proposal, the intent was clear that typecheckers should treat it equivalently to if the equivalent statement was expanded inline. For a runtime use by libraries doing runtime typechecking (and validation) to intentionally diverge in behavior here is against design, and likely to clash with static typing users reasonable expectations.
1 Like
I get your point, but runtime type checking can’t be a 100% aligned with static type checking and the current spec (other examples: imports in if TYPE_CHECKING: blocks, resolving forward annotations inside inner scopes, and more).
For the reason mentioned in the documentation I linked, we can’t eagerly unpack the TypeAliasType.__value__ property. In fact, we could, but it would have to be configurable to satisfy both uses (and still, how to make it configurable if users want to selectively unpack such aliases?), and so far we didn’t have any request to do so.
It is true that the situation is confusing, we had a number of reports asking for this to work:
type MyAnn = Annotated[int, Field(default=1)]
class Model(BaseModel):
a: MyAnn # Currently the default value is ignored
But (without eagerly unpacking the alias) we can’t take into account the default (which ironically isn’t understood by static type checkers as well).
(I’ll note that anything mentioned in Run-time behaviour of `TypeAliasType - #60 by tibbe is directly related to Pydantic – apart from the TypeAdapter type checking issue requiring PEP 747)/
Right, but I think it’s reasonable here for pydantic to just evaluate it (by accesing .__value__). it would be unreasonable for someone to expect this alias not to be evaluated by pydantic because pydantic’s purpose is to use this information at runtime.
To be clear, we do evaluate type aliases at some point (code is available here), just in a way to allow us to generate a core schema reference (a Pydantic-specific concept, that can later be translated to a JSON Schema reference). This is possible thanks to type aliases being proper TypeAliasType instances, with a __name__.
Due to how Pydantic processes a model class definition, it isn’t possible to “extract” the Field() spec inside the type alias while still allowing it to be referenceable (the code link I referenced is code happening at a later point in the class processing, where model fields – and as such the default values for each field – already have been processed).
95% of the time, using old-style/new-style type aliases won’t be different for end users (validation works the same). The only place where it would be different is when attaching field-specific metadata (such as defaults) to a type alias, which is debatable whether it should be supported – presumably you don’t want to add field-specific metadata to a type alias that isn’t “tied” to any field in particular.
Yes, I think that adding non-type metadata such as Field descriptors is a misuse of type aliases regardless of whether you’re using the new or old type alias syntax. Type qualifiers and non-type metadata shouldn’t be allowed in type alias definitions. With the old syntax, pydantic had no way of preventing this misuse. With the new syntax, it can.
1 Like
I don’t think it should be seen as a misuse, the specification says
anything that is acceptable as a type hint is acceptable in a type alias
It’s also a natural way for someone to write a reusable type with runtime constraints that are shared across usages.
type Int64 = Annotated[int, Meta(bits=64)]
class SomeFFIType:
x: Int64
2 Likes
In your example, the metadata applies to the type int, so I consider this “type metadata”, and it has meaning anywhere this type alias is used.
Metadata that describes a field and its default value is “non-type metadata” because the metadata is not describing attributes of or runtime constraints of the type. A field descriptor default has no legitimate meaning in most places where a type alias can be used.
No, it should always apply to places where this alias is then used. It’s just an alias (with better scoping rules, as per your description of it in the pep). Both the pep and specification seem aligned with that.
Then an error should be emitted if the alias is used somewhere where a Field has no meaning, not disallowing the ergonomic use entirely in contradiction to what the specification says is allowed.
Looks like the thing they say they don’t support is using Annotated in the type alias. This makes sense to me.
But my examples don’t use Annotated. And yet when asking for the schema of a set of classes (all marked as pydantic dataclasses) I get the error ‘VerbTense’ cannot be used as a type annotation. Here’s the relevant part of the class:
@dataclass
class Action:
verbs: Annotated[list[str], Doc("Each verb is typically a word.")]
verb_tense: VerbTense
...
So I still believe it’s BS that Pydantic claims that PEP 595 type aliases are “supported”.
OT: On my IPhone, in Chrome, that Pydantic doc link has a bug (not your fault): every few seconds the page jumps a little up and down.
Specifically, it’s a type alias. It describes a type, and it should have a well-defined meaning anywhere a type expression can be used. There’s good reason we don’t allow type qualifiers (like ReadOnly, Final or Required) within a type alias definition. For the same reason, non-type metadata shouldn’t be allowed, IMO. Unfortunately, there’s currently no way for a static type checker to distinguish between type metadata and non-type metadata. The Annotated mechanism provides no such distinction today. I think we will eventually want to provide such a mechanism, and I’ve even explored some ideas in this direction.
Pydantic does not raise any error with [typ] cannot be used as a type annotation, I assume this is coming from TypeChat (which you’re working on) according to this search.
Edit: had a quick look, it seems like your is_python_or_alias() function is broken as it applies the isinstance() call against typing_extensions.TypeAliasType, which is different from typing.TypeAliasType. See this comment, and how we handle it currently in Pydantic (see also the actual implementation, working around a CPython bug).
There’s good reason we don’t allow type qualifiers (like
ReadOnly,FinalorRequired) within a type alias definition. For the same reason, non-type metadata shouldn’t be allowed, IMO
I don’t actually agree with this or think it to be a good reason. I think keeping aliases simply as they were originally (and still) described is much better for users expressing “reusable things”, including things like Field specifiers.
Annotated has a meaning in a type expression. You’re inserting your own view of it here without any specification, but Annotated is a way for users to attach non-type system data to a type expression. The only way to know if it is valid or not is to know how and where that type expression (ie. where the alias is later used, something expecting that data determines it) is used.
1 Like