The NewType feature of Python’s static typing is extremely useful for ensuring that different types of otherwise similar data aren’t mingled (for example mixing UserIds with PostIds on a social media platform). One of my projects is effectively taking advantage of them to ensure strong type-safety for its data.
The problem
However, one common issue I have run into is that these IDs, which should be ints are often given as strs due to limitations of the way that HTTP requests work. As such, in order to convert to the correct type, I need to manually perform an additional conversion, which isn’t as clean as I would prefer.
UserId = typing.NewType('UserId', int)
# We need to manually convert to an int to get this to work correctly
user_id = UserId(int(request.args['user_id']))
From what I can tell, this is currently extremely difficult or impossible to do. So far, all suggestions for workarounds have not been effective, and I’ve been unable to create my own class that has the desired effect whilst remaining typesafe.
My proposed solution
In order to address this, it could be useful to allow developers to specify a convert=True flag when creating their NewType definitions.
UserId = typing.NewType('UserId', int, convert=True)
# When we specify the flag, the conversion from a str to an int is performed automatically
user_id = UserId(request.args['user_id'])
Obviously, this behaviour isn’t what all developers would want, but having the option would be extremely useful, in some situations. I believe that this kind of attention to detail is what makes Python such a pleasant language to work with, and I’d love to help by suggesting this feature to make it even better!
Write a function def make_user_id(user_id: int | str) -> UserId: return UserId(int(str)) and use that.
The if TYPE_CHECKING approach that someone suggested in a StackOverflow comment should also work.
Your convert=True proposal implies that the conversion function used is simply the type that the NewType is created over, such as int. Two issues with that:
This means any type accepted by the int() constructor will be accepted, such as float. Do you really want UserId(3.14159) to be valid?
It is possible to create a NewType over a type that doesn’t have a valid single-argument constructor, such as int | None. What should convert=True do in that case?
The function to do the conversion is nice, but a bit repetitive given that our app requires lots of different IDs - I suppose we could do something using meta-functions to help our with that, but I’m not sure how easy it would be to make that type-safe.
Sadly the if TYPE_CHECKING thing does not work - Mypy freaks out with a ton of errors, such as Variable "backend.types.identifiers.UserId" is not valid as a type.
I think your points regarding unintended other type conversions are valid though - I’ll need to reconsider the design for it a bit. I wasn’t aware that it was possible to create a NewType that couldn’t be constructed, since is supposed to make the type appear as a subclass of whatever you give it - having a subclass of int | None doesn’t make much sense to me. If that is possible as you say, then you’re right in saying that it wouldn’t be possible.
That being said, I realised relatively soon after posting this that I completely overlooked the simplest solution, which is to just manually create the subclasses.
The if TYPE_CHECKING version should definitely work. If you can’t get it to work, you can open a topic at Discussions · python/typing · GitHub with your code and we should be able to help you figure it out.
I was wrong about NewTypes over unions—that’s not actually legal (PEP 484 – Type Hints | peps.python.org). However, it remains true that classes may not have a one-argument constructor.
I’m pretty sure that it makes sense that the if TYPE_CHECKING thing doesn’t work. Mypy would consider the type of the result to be the NewType which requires an int, so even though string conversion would work at runtime, Mypy wouldn’t realise this and I would get errors about giving a str to a callable that expects an int. I’m not sure why I’m getting the "backend.types.identifiers.UserId" is not valid as a type error instead of errors about giving str instead of int, but my best guess is that it sees the conditional initialisation and freaks out.
One thing I did slightly differently to the original was using a ternary expression to perform the initialisation, ie UserId = NewType('UserId', int) if TYPE_CHECKING else int rather than a full if statement. I did this to save space, since it’ll get pretty cluttered pretty fast if I need to do a full if statement for every single type of identifier. As far as I know, ternary operations should be equivalent to their standard if/else coutnerparts, but please correct me if I’m wrong.