Pre-PEP: Rust for CPython

Introduction

We (@emmatyping, @eclips4) propose introducing the Rust programming language to CPython. Rust will initially only be allowed for writing optional extension modules, but eventually will become a required dependency of CPython and allowed to be used throughout the CPython code base.

Motivation

Rust has established itself as a popular, memory-safe systems programming language in use by a large number of projects. Memory safety is a property of a programming language which disallows out-of-bounds reads and writes to memory, as well as use-after-free errors. Rust’s safety properties are enforced by an ownership mode for code, which ensures that memory accesses are valid. Rust’s memory safety guarantees have been formally proven by the RustBelt project for code that does not use “unsafe” . By adopting Rust, entire classes of bugs, crashes, and security vulnerabilities can be eliminated from code.

Due to Rust’s ownership model, the language also enforces thread safety: data races are prevented at compile time. With free-threaded Python becoming officially supported and more popular, ensuring the standard library is thread safe becomes critical. Rust’s strong thread safety guarantees would ease reasoning around multi-threaded code in the CPython code base.

Large C/C++ based projects such as the Linux kernel, Android, Firefox, and many others have begun adopting Rust to improve memory safety, and some are already reporting positive results from this approach. Furthermore, Rust has become a popular language for writing 3rd-party extension modules for Python. At the 2025 Language Summit, it was mentioned that 25-33% of 3rd-party Python extension modules use Rust, especially new extension modules. By adopting Rust in CPython itself, we expect to encourage new contributors to extension modules.

CPython has historically encountered numerous bugs and crashes due to invalid memory accesses.We believe that introducing Rust into CPython would reduce the number of such issues by disallowing invalid memory accesses by construction. While there will necessarily be some unsafe operations interacting with CPython’s C API to begin with, implementing the core module logic in safe Rust would greatly reduce the amount of code which could potentially be unsafe.

Rust also provides “zero-cost”, well designed implementations of common data structures such as Vectors, HashMaps, Mutexes, and more. Zero cost in this context means that these data structures allow implementations to use higher-level constructs with the performance of hand-rolled implementations in other languages. The documentation for the Rust standard library covers these data structures very thoroughly. By having these zero-cost, high-level abstractions we expect Rust will make it easier to implement performance-sensitive portions of CPython.

Rust additionally enables principled meta-programming through its macro system. Rust has two types of macros: declarative and procedural. Declarative macros in Rust are hygienic, meaning that they cannot unintentionally capture variables, unlike in C. This means it is much easier to reason about macros in Rust compared to C. Procedural macros are a way to write Rust code which does token transformations on structure definitions, functions, and more. Procedural macros are very powerful, and are used by PyO3 to ergonomically handle things like argument parsing, class definitions, and module definitions.

Finally, Rust has an excellent build system. Rust uses the Cargo package manager, which handles acquiring dependencies and invoking rustc (the Rust compiler driver). Cargo supports vendoring dependencies out of the box, so that any Rust dependencies do not need to be downloaded (see open questions about dependencies). Furthermore, Rust has easy to set up cross-compilation which only requires installing the desired target and ensuring the proper linker is available.

In summary, Rust provides many extremely useful benefits that would improve CPython development. Increasing memory safety would be a significant improvement in of itself, but it is far from the only benefit Rust provides.

Implementation

The integration of Rust would begin by adding Rust-based extension modules to the “Modules/” directory, which contains the C extensions for the Python standard library. The Rust modules would be optional at build time, dependent on if the build environment has Rust available.

Integrating Rust with the CPython C API requires foreign function interface (FFI) definitions in Rust to describe the APIs available. A new crate (a library in Rust terminology) cpython-sys will be introduced to handle these FFI definitions. To automate the process of generating Rust FFI bindings, we use bindgen. Bindgen is not only an official Rust project, but also used ubiquitously throughout the Rust ecosystem to bind to C APIs, including in the Linux and Android projects. Bindgen uses libclang at build time to read C headers and automatically generate Rust FFI bindings for the current target. Unfortunately, due to the use of C macros to define some constants and methods, the cpython-sys crate will also need to replicate a few important APIs like PYOBJECT_HEAD_INIT manually. However these should all be straightforward to replicate and few in number.

With the C API bindings available in Rust, contributors can call the FFI definitions to interact with CPython. This will necessarily introduce some unsafe Rust code. However extension modules should be written such that there is a minimal amount of unsafe at the edges of FFI boundaries.

Eventually safe abstractions to simplify and standardize code like module definitions, function argument parsing, and class definitions could be adopted to reduce the amount of raw FFI code written.

A reference implementation which includes a proof-of-concept _base64 module which uses Rust to provide a speedup to base64 is available.

Distribution

Rust supports all platforms which CPython supports and many more as well. Rust’s tiers are slightly different, and include information on whether host tools (such as rustc and cargo) are provided. Here are all of the PEP 11 platforms and their corresponding tiers for Rust:

Platform Python Tier Rust Tier
aarch64-apple-darwin 1 1 with Host Tools
aarch64-unknown-linux-gnu (gcc, glibc) 1 1 with Host Tools
i686-pc-windows-msvc 1 1 with Host Tools
x86_64-pc-windows-msvc 1 1 with Host Tools
x86_64-unknown-linux-gnu (gcc, glibc) 1 1 with Host Tools
aarch64-unknown-linux-gnu (clang, glibc) 2 1 with Host Tools
wasm32-unknown-wasip1 2 2 without Host Tools
x86_64-apple-darwin 2 2 with Host Tools
x86_64-unknown-linux-gnu (clang, glibc) 2 1 with Host Tools
aarch64-linux-android 3 2 without Host Tools
aarch64-pc-windows-msvc 3 1 with Host Tools
arm64-apple-ios 3 2 without Host Tools
arm64-apple-ios-simulator 3 2 without Host Tools
armv7l-unknown-linux-gnueabihf (Raspberry Pi, gcc) 3 2 with Host Tools
aarch64-unknown-linux-gnu (Raspberry Pi, gcc) 3 1 with Host Tools
powerpc64le-unknown-linux-gnu 3 2 with Host Tools
s390x-unknown-linux-gnu 3 2 with Host Tools
wasm32-unknown-emscripten 3 2 without Host Tools
x86_64-linux-android 3 2 without Host Tools
x86_64-unknown-freebsd 3 2 with Host Tools

In summary, every platform Python supports is supported Rust at tier 2 or higher, and host tools are provided for every platform other than those where Python is already cross-compiled (e.g. WASI and mobile platforms).

Rejected Ideas

Use PyO3 in CPython

While CPython could depend on PyO3 for safe abstractions over CPython C APIs, this may not provide the flexibility desired. If a new API is added to the C API, it would need to be added to PyO3, then the version of PyO3 would need to be updated in CPython. This is a lot of overhead and would slow down development. Using bindgen, new APIs are automatically exposed to Rust.

Keep Rust Always-Optional

Rust could provide many benefits to the development of CPython such as increased memory safety, increased thread safety, and zero-cost data structures. It would be a shame if these benefits were unavailable to the core interpreter implementation permanently.

Open Questions

How should we manage dependencies?

By default cargo will download dependencies which aren’t already cached locally when cargo build is invoked, but perhaps we should vendor these? Cargo has built-in support for vendoring code. We could also cargo fetch to download dependencies at any other part of the build process (such as when running configure).

How to handle binding Rust code to CPython’s C API?

The MVP currently uses bindgen, which requires libclang at build time and a number of other dependencies. We could pre-generate the bindings for all supported platforms, which would remove the build-time requirement on vendoring bindgen and all of its dependencies (including libclang) for those platforms.

When should Rust be allowed in non-optional parts of CPython?

Given the numerous advantages Rust provides, it would be advantageous to eventually introduce Rust into the core part of CPython, such as the Python string hasher, SipHash. However, requiring Rust is a significant new platform requirement. Therefore, we propose a timeline of:

  1. In Python 3.15, ./configure will start emitting warnings if Rust is not available in the environment. Optional extension modules may start using Rust
  2. In Python 3.16, ./configure will fail if Rust is not available in the environment unless --with-rust=no is passed. This will ensure users are aware of the requirement of Rust on their platform in the next release
  3. In Python 3.17, Python may require Rust to build

We choose this timeline as it gives users several years to ensure that their platform has Rust available (most should) or otherwise plan for the migration. It also ensures that users are aware of the upcoming requirement. We hope to balance allowing time to migrate to Rust with ensuring that Rust can be adopted swiftly for its many benefits.

How to handle bootstrapping Rust and CPython

Making Rust a dependency of CPython would introduce a bootstrapping problem: Rust depends on Python to bootstrap its compiler. There are several workarounds to this:

  1. Rust could always ensure their bootstrap script is compatible with older versions of Python. Then the process is to build an older version of Python, build Rust, then build a newer version of CPython. The bootstrap script is currently compatible with Python 2, so this seems likely to continue to be the case
  2. Rust could use PyPy to bootstrap
  3. Rust could drop their usage of Python in the bootstrap process

I (@emmatyping) plan to reach out to the Rust core team and ask what their thoughts are on this issue.

What about platforms that don’t support Rust?

Rust supports all platforms enumerated in PEP 11, but people run Python on other operating systems and architectures. Reviewing all of the issues labeled OS-unsupported in the CPython issue tracker, we found only a few cases where Rust would not be available:

  1. HPPA/PA-RISC: This is an old architecture from HP with the last released hardware coming out in 2005 and a community of users maintaining a Linux fork. There is no LLVM support for this architecture.
  2. RISC OS: This is a community maintained version of an operating system created by Acorn Computers. There’s no support in Rust for this OS.
  3. PowerPPC OS X: This older OS/architecture combination has a community of users running on PowerBooks and similar. There is no support in Rust for this OS/architecture combination, but Rust has PowerPC support for Linux.
  4. CentOS 6: Rust requires glibc 2.17+, which is only available in Centos 7. However, it is unlikely users on a no-longer-supported Linux distribution will want the latest version of CPython. Even if they do, CPython would have a hard time supporting these platforms anyway.

How should current CPython contributors learn/engage with Rust portions of the code base?

Current contributors may need to interact with the Rust bindings if they modify any C APIs, including internal APIs. This process can be well covered in the devguide, and there are many great resources to learn Rust itself. The Rust book provides a thorough introduction to the Rust programming language. There are many other resources in addition, such as Rust for C++ programmers and the official learning resources Learn Rust - Rust Programming Language.

To ease this process, we can introduce a Rust experts team on GitHub who can be pinged on issues to answer questions about interacting with the API bindings. Furthermore, we can add a Rust tutorial focused on Rust’s usage in CPython to the devguide.

Obviously any extension modules written in Rust will require knowledge of Rust to contribute to.

What about Argument Clinic?

Argument Clinic is a great tool that simplifies the work of anyone writing C functions that require argument processing. We see two possible approaches for implementing it in Rust:

  1. Adapt the existing Argument Clinic to parse Rust comments using the same DSL as in C extensions, and generate Rust function signatures.
  2. Create a Rust procedural macro capable of parsing a similar DSL. This approach might allow it to be used by any third-party package, whereas the C-based Argument Clinic does not guarantee compatibility with third-party extensions.

Using a proc macro would allow for better IDE integration and could become useful to 3rd party extension authors.

Should the CPython Rust crates be supported for 3rd-party use? Should there be a Rust API?

Having canonical Rust crates for bindings to CPython’s C API would be advantageous, but the project is ill-prepared to support usage by 3rd-parties at this time. Thus we propose deferring making Rust code supported for 3rd-party use until a later date.

25 Likes

Isn’t the experience in the linux kernel with adding rust support as a core part more a cautionary tale? At least it looks that way from the outside. There is constant friction between the two different “types of programmers”, causing a lot of quite public disagreement. If rust becomes a requirement for core devs, might that scare away some current maintainers who don’t want to deal with it?

1 Like

Hello, and thanks for your question!
We will try to make this transition as smooth as possible. We’re planning to write a “migration guide” for CPython contributors to help make their adoption of Rust as easy as possible.
Can this scare current maintainers? Yes.
Will we do everything we can to make sure it doesn’t? Absolutely.

4 Likes

I’m not a core dev nor expert in the internals of CPython, but I wanted to chime in to resonate with the question from @MegaIng , pointing out though that it looks to me (as a Python user) that the community in general is more “approachable” in comparison to what happened during the integration of some Rust in the Linux kernel.

Out of curiosity, what would this mean for both PyO3 and RustPython? Have you reached out to the mantainers of the latter for feedback on how to approach this possibility? If so, what’s their opinion? It looks to me like this possible PEP might benefit greatly from their experience (I don’t know if you guys are part of their team, I’m just assuming you’re not but I apologize if that’s incorrect).

Also, I’m probably jumping the gun with this question, but since also RustPython seems to be implementing the GIL in a similar fashion, would you expect some challenges in applying PEP 703 efficiently for rust-based extensions? I imagine (or rather, hope) that by the time 3.17 is out that CPython will be completely GIL-less by then. Do you expect that this additional feature will be provided smoothly in Rust extensions as well?

Same question applies for the new experimental JIT which should be more stable in future releases.

Finally, if I recall rust applications tend to be a bit “bloated” in binary size, although there are some tricks that can be done at compile time to reduce this - what do these tricks imply on performance I have no idea.

As a general direction, I’d rather see “optional extension modules”[1] living outside the main repo and brought in later in the build/release process. Presumably such modules have no tight core dependency, or they wouldn’t be able to be optional, and so they should be able to build on pre-release runtimes and work fine with released runtimes (as we expect of 3rd party developers).

The bootstrapping position is discussed in the proposed text and determined to be viable. I think this position applies equally well to bootstrapping the modules we theoretically expect people to write that rely on non-core dependencies.

Merging in additional modules as part of our release process is fine. Bundling additional sources and optional build steps into the source tarball is fine (if we think that Linux distros will just ignore us when we say “you should include these modules in your Python runtime”).

For what it’s worth, I believe that the parts of CPython that directly depend on OpenSSL and Tcl/Tk are also fit for this. So don’t see my position as being about Rust itself[2] - it’s about the direction we should be taking with adding new modules to the core runtime.

In short, adding new ways to add new, non-essential modules to the core is counter to the approach we’ve been taking over the last few years. So I’m -1 on adding one.


  1. Also optional regular modules. ↩︎

  2. If you want my position about Rust itself, well… it’s not going to help your PEP at all :wink: ↩︎

1 Like

As I think many folks know, I’ve been quite involved in these sorts of efforts (starting the effort that became Rust for Linux at the PyCon sprints, migrating pyca/cryptography to Rust, and helping to maintain pyo3). As a result, it will come to no one’s surprise that I’m very supportive of this pre-PEP :slight_smile:

I’m happy to answer any questions folks have about those experiences and lessons learned.

7 Likes

As a Rust fan I think this is super cool! I do wonder if this is too much to figure out in one PEP, when parts of it seems pretty easily separable. I can see how the overall vision fits together, but it’s a lot.

Would it make sense to restrict the initial proposal as just “create the cpython-sys crate and allow optional Rust extensions in the stdlib”? That would allow iteration on making a good generic interface[1] without requiring the SC to make a decision on the long-term plan.


  1. there is plenty of prior art in PyO3, and I think the HPy project has some relevance there too ↩︎

I would feel more comfortable with this if it was dependent on custom json targets stabilizing in Rust. Right now, targetting platforms that Rust itself doesn’t support still requires nightly, and python is used to bootstrap a lot of things in a lot of places.

I don’t think the impact of this is going to be reasonably predictable, and I’d want there to be a stable upstream escape hatch for those in unusual situations.

2 Likes