Incompatible types in assignment when casting function argument

Python Help
1 
from typing import *

JsonValue = None | bool | int | float | str
JsonObject = dict[str, 'JsonData']
JsonArray = list['JsonData']
JsonData = JsonValue | JsonObject | JsonArray


def parse(data: JsonData):
    ...  # JSONschema validation here
    data = cast(list[str] | None, data)  # After validation we know that data is list[str] or None
    # ^^^ Incompatible types in assignment (expression has type "list[str] | None", variable has type "JsonData")

    ...  # Further data processing

I have a function that takes JSON data as an argument, validates it using JSONschema and parsing the given configuration after validation. I’m using typing.cast to tell type checker (mypy) that data is now list[str] or None. Mypy reports an error: Incompatible types in assignment (expression has type "list[str] | None", variable has type "JsonData"). How to cast the data variable here correctly?

2

Found a way to get rid of the typing error by casting to a new variable, but can it be done without new variable creation (cast to the same variable)?

4

Unlikely, because the issue is that you declared the type of data in your function signature: data: JsonData says that the data object is expected to have that type.

typing.cast is meant to be used in the other direction, e.g. you have a variable declared to take JsonData and you want to assign something that isn’t a JsonData into it, because you know that’s safe to do, then you could write:

data = cast(JsonData, my_var_that_mypy_knows_holds_a_string_list)

…and mypy will let that through without complaint, which it wouldn’t if you just wrote:

data = my_var_that_mypy_knows_holds_a_string_list

…when data is annotated as holding JsonData.

But changing a variable’s type mid-stream breaks the static analysis, not sure if there’s a mypy-approved way of doing that.

1 Like
5

Thanks for your answer.

Will it be mypy-approved to assign casted value to new name, e.g. like the code below?

data_validated = cast(list[str] | None, data)
6

Oh, sure, because then you’re effectively declaring the type of data_validated with the cast(). It’s the same as writing,

data_validated: list[str] | None
data_validated = cast(list[str], data)

(Assuming data actually does hold a list[str].)

…Of course, you have to be careful, because you’re overriding mypy’s type-checking, and if you lie to it it’ll believe you. This will also pass mypy without complaint, despite being completely false:

int_list = [1, 2, 3, 4, 5]
data_validated: list[str] | None
data_validated = cast(list[str], int_list)

…and then you’ll get a runtime error if you try something like,

if data_validated is not None:
    print(" ".join(data_validated))
1 Like
7

This is one of the most common question about Python’s type system: Incompatible types in assignment (expression has type “List[str]”, variable has type “List[Union[str, int]]”) (I can’t find a more highly upvoted post).

The reason is that list is invariant, and so list[str] is not assignable to list[JsonData], which expands to list[None | str | bool | int | ...].

2 Likes
8

It’s hard to tell from the small example you gave, but my best guess is that there’s some complex validation and if/else statements within your method.

If that’s the case, here’s what I would do:

def is_valid_list_of_strings(data: JsonData) -> bool:
    return True # add logic, of course

def parse_list_of_strings(data: list[str]):
    pass # do the parsing, of course

def parse(data: JsonData):
    if is_valid_list_of_strings(data):
        return parse_list_of_strings(cast(list[str], data))
    # other conditions similarly

I.e create a separate method that deals with the specific type you’ve validated. This should satisfy mypy because you’re not re-assigning the value to the existing variable.

10

Validation uses JSON schema, if the validation fails, the exception is raised, if it’s valid, the code continues to the parse section.

11

If you’re only expecting to use a specific type of data (list[str]), why would you allow a broader type in the function signature?

If your function only works with list[str] as an argument, then it should be typed as such.
Otherwise it’s like telling the waiter to choose any meal for you and then refusing to pay when they bring you a beef burger because you expected a vegan meal.
The function signature should be as broad as possible, but as narrow as needed. If the method can’t handle anything other than list[str] then that’s what’s needed.

12

The method is defined to take JsonData as the first argument in the ABC, so it will break Liskov Substitution Principle.

Also user may pass invalid configuration to the argument (list element strings format is limited by regex), so an exception can be raised anyway (even if the argument was annotated as list[str])

13

There was nothing in the initial post (or up to now) that would indicate you were dealing with a bound method or inheritence. You just provided an example function.

As I’m sure you’re aware, type hints are generally just decorative. Exceptions may be raised at runtime anyway since no types are actually guaranteed.

You brought up Liskov Substitution Principle. I think the whole problem here is that your derived class is (effectively) breaking it already. You just want to fool static checkers (such as mypy) into thinking it’s not.

Now, the trivial solution would be to just use cast as you did in the OP and set it to a different variable and document the raise exception in the doc string for the base class (in a general manner, of course). As pointed out above, you can’t just overwrite the type of the existing one.

But if you want to have all your types line up nicely, you may be able to use generics to have each implementation explicitly state what it can and cannot parse.

14

I made it to simplify the example. The question was about casting existing variable, not about changing annotations of the function (method) argument.

Yes, because type checker does not take JSON schema validation into account. Also I need this function to take any variable of JsonData type without type errors, validating it inside function.

15

You’re absolutely right! This was a question about typing. And I lost (missed?) the scope!

So it sounds like you’d like is a sort of isinstance(data, (list[str], None)) method. Because generally an instanceof check will narrow down the type of the same variable neatly and in an acceptable manner.

So best I can think of would be something like this:

def parse(data: JsonData):
    if not (data is None or (isinstance(data, list) and
                             (not data or isinstance(data[0], str)))):
        raise ValueError
    parts: list[str] | None = data
    return parts # deal with it, separately, of course

But this doesn’t quite work because mypy doesn’t fully understand the contents of the list.

So best I can tell, you’ll still need to do a cast and assign to another variable. It’s probably fine. I’d probbaly leave a # NOTE next to it as to why that is necessary.

16

If you use a good JSON schema validator, like pydantic or msgspec, then the type checker can take JSON schema validation into account.