# Encode \`str\` (utf-8 uncodeobject) without a copy

**URL:** <https://discuss.python.org/t/encode-str-utf-8-uncodeobject-without-a-copy/105572>\
**Category:** Ideas\
**Created:** [January 7, 2026, 7:03am UTC](https://discuss.python.org/t/encode-str-utf-8-uncodeobject-without-a-copy/105572 "2026-01-07T07:03:28Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![cmaloney](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cmaloney/32/15851_2.png) [@cmaloney](https://discuss.python.org/u/cmaloney)\
**Post date:** [January 9, 2026, 2:50am UTC](https://discuss.python.org/t/encode-str-utf-8-uncodeobject-without-a-copy/105572/4 "2026-01-09T02:50:23Z")

</div>

> [@vstinner](#):
>
> It’s not easy to judge proposed options without a concrete implementation. Can you show that using a `Py_buffer` in C or `memoryview` in Python is faster than creating a `bytes` object at each call? The benchmark likely depends on the string length. For example, is it worth it for strings shorter than 100 characters?

Planning to gather a CPU profile + runtime traces / benchmarks soon; will update this thread when I have.

This is the only common copy in the Text I/O stack I don’t currently have a way to resolve (caveats apply / there are some non-copy fast paths today). With [Reworking "Buffered I/O" in CPython](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019) the copy of the `.write` argument into the shared buffer in BufferedReader will go away unless it’s required (the buffer is mutable/non-frozen and need to return from `write` while not flushing immediately). Inside `TextIOWrapper` there’s the encoding copy (this) and a second copy to merge the encoded strings into one final buffer which is then passed to `BufferedIO.write`. I’m hoping with the rework I can eliminate most that / just defer to `BufferedIO` which internally I plan to have keep a list of parts like TextIOWrapper does internally for efficiency.

All of that _hopefully_ will also mean eliminating locking in many cases; although that depends in part on if I have Rust for making it an experimental I/O stack (`_rustio` 🦀 ). My hope is to have a full working `_pyio` prototype which demonstrates the ideas + efficiency before end of February. Starting this discussion so hopefully can make it a zero-copy I/O stack in the 3.15 ship window :).

> [@vstinner](#):
>
> [Better API for encoding unicode objects with UTF-8](https://discuss.python.org/t/better-api-for-encoding-unicode-objects-with-utf-8/2909) proposed:

> [@vstinner](#):
>
> You may have a look at withdrawn [PEP 756 – Add PyUnicode\_Export() and PyUnicode\_Import() C functions](https://peps.python.org/pep-0756/) which proposed adding a `PyUnicode_Export()` function. The PEP discuss tricky issues about surrogate characters and embedded null characters.

> [@methane](#):
>
> The best solution in the UTF-8 era is for Unicode objects to use UTF-8 as their internal representation and only cache PEP 393 data when requested for compatibility.

Really appreciate these references. Working on reading through them / ingesting what they say. I wasn’t aware that if we start with `utf-8` (ex. parsed Python source code or `.decode('utf-8')`) that CPython always currently copies that out of UTF-8 form and needs to then copy back into it. Resolving that + this should make CPython file → text → file round trip dramatically lower overhead.

That parsing utf-8 doesn’t keep the original bytes if we know they are 1. immutable / owned by python internals already and 2. correctly normalized/encoded is something I’d like to incorporate into shipping this change set potentially. I Think it would get Python a really nice place for UTF-8 processing.

---

_[View the full topic](https://discuss.python.org/t/encode-str-utf-8-uncodeobject-without-a-copy/105572)._