Introduction to a new runtime Python typing library - `python-newtype`!

Typing
1

:rocket: Hey everyone! I’m beyond excited to share python-newtype, a new Python C-extension library I’ve been working on that makes Domain-Driven Development (DDD) not only easier but also super fast! :snake::zap:

Here’s what makes python-newtype stand out: It’s built with a C-extension under the hood, which means it’s optimized for performance. While you get all the flexibility of Python’s dynamic typing and extensibility, the C-extension ensures that method calls and type wrapping are lightning-fast. If you’ve ever felt that Python’s type-heavy libraries slow things down, you’re going to love this!

One of the coolest features is how python-newtype handles methods of the supertype. If a method from the supertype returns an instance of the supertype, python-newtype automatically converts it to an instance of the subtype. Thanks to the C-extension, this process is super-efficient, with almost no overhead compared to normal Python method calls.

Here’s an example of how it works:


from newtype import NewType



class EnhancedStr(NewType(str)):

  def reverse(self):

    return self[::-1]

text = EnhancedStr("Hello World")



# Use a standard string method that returns an instance of the supertype

upper_text = text.upper() # This normally returns a `str`



# But with `python-newtype`, it’s automatically converted to `EnhancedStr`

print(isinstance(upper_text, EnhancedStr)) # True

print(upper_text.reverse()) # "DLROW OLLEH"

Even though .upper() is a method from str, its return value is automatically converted to EnhancedStr—and the C-extension makes this transformation incredibly fast. You get type invariance without sacrificing performance. :tada:

Why the C-extension? Python’s dynamic type system is powerful, but when you start wrapping and intercepting methods, performance can take a hit. By implementing the core logic in C, python-newtype minimizes the overhead involved in type conversions and method interceptions, making it ideal for production environments where speed matters. :rocket:

This library is perfect for building domain-specific models where you want to extend existing types with additional logic while preserving their behavior. It’s also memory-efficient, thanks to the use of weak references for caching.

You can try it out now with:


pip install python-newtype

I’d love for you to check it out and let me know what you think! Whether you’re into clean code, type safety, or just love Python, this library has something for everyone.

Full documentation is available here :point_right: [https://py-nt.asyncmove.com/]

Check out the PYPI and GitHub in the Comments section!

Thanks for reading, and happy coding! :computer::sparkles:

5 Likes
2

The library claims “comprehensive type hints and IDE support”, but this doesn’t seem to be the case:

Stub file not found for "newtype"
Stub file not found for "newtype"865×292 24.2 KB

Type of "reverse" is unknown
Type of "reverse" is unknown874×187 24.4 KB

You are relying on str’s definition in typeshed, but its .upper() is hardcoded to return str rather than Self. This eventually leads to upper_text.reverse() being Unknown.

class C(NewType(...)) is also a pattern unsupported by Mypy:

$ uvx mypy --strict hello.py
hello.py:1: error: Cannot find implementation or library stub for module named "newtype"  [import-not-found]
hello.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
hello.py:5: error: Unsupported dynamic base class "NewType"  [misc]
hello.py:7: error: Function is missing a return type annotation  [no-untyped-def]
Found 3 errors in 1 file (checked 1 source file)

I haven’t tried the library in earnest, so I can’t comment on its general usefulness, but these are major red flags for static type checking users.

2 Likes
3

That is weird, I have a .pyi files in the /extensions subdirectory which should help typechecker to properly get the stubfiles and I have checked that the packaged tar file does have these files included.

Given your experience, what do you think might be a plausible cause?

4

You need a py.typed marker file, as per PEP 561[1].

Although that would still not really help with the issues that have been pointed out. Dynamic base classes are generally unsupported in mypy. You would either need to switch to a class decorator or metaclass for the type checker to be able to do anything, but even then you would not really get the desired semantics in static type checking, where methods on the superclass that don’t return Self but rather a fixed class like str, will not have the correct signature.

markupsafe.Markup is a good example of a str subclass with support for static typing. Even if you used your library to get rid of the boilerplate to wrap each method return in Markup[2], you would still have to write boilerplate so the type checker knows about the new correct signature.

There’s just no way to express your desired semantics without writing a mypy plugin, and then only mypy users could use your library. The rest would probably have to manually add a pyi file that copies all the signatures for all the methods that are supposed to be there.

  1. And make sure it’s actually included in the distribution’s package data ↩︎

  2. this would only work for the methods with no str arguments, since the arguments have to get escaped ↩︎

5

Thank you for your prompt reply.

I do have a py.typed marker file within the same directory as with the *.pyi files (/extension directory).

Not sure why it doesn’t make it work still.

6

You need to move the file to the top-level package, currently your marker file only covers newtype.extensions, so any imports from newtype rather than newtype.extensions will still be considered untyped by type checkers.

Perhaps you thought you only need a py.typed marker for pyi files? That is not the case, you always need it. Stubs-only packages work a bit differently, but if you ship either inline stubs or separate stub files in the same distribution, you need the py.typed marker file.

1 Like