# PEP 674 "Disallow using macros as l-values" and Python 3.11 **URL:** \ **Category:** Core Development\ **Created:** [August 18, 2022, 2:31pm UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297 "2022-08-18T14:31:59Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 18, 2022, 2:32pm UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/1 "2022-08-18T14:32:00Z")
Hi, Recently, [CAM-Gerlach asked me](https://github.com/python/peps/pull/2756) the status of my [PEP 674](https://peps.python.org/pep-0674/). Oh. I forgot about that one… As currently written in PEP 674, the PEP is basically already implemented in Python 3.11. Problem: last February, the [SC requested some changes](https://mail.python.org/archives/list/python-dev@python.org/thread/CV6KWDRHV5WP6TIDK3Z46PW7HNSHYOWG/) to have a smoother transition towards the new API: - Document that the old way to set Py\_SIZE() with `Py_SIZE(obj) = size` is deprecated, - Wait until all supported Python versions provide Python Py\_SET\_SIZE() (Python 3.9: so wait until Python 3.8 is no longer supported), - And only then disallow `Py_SIZE(obj) = size` (fail with a compiler error). PEP 674 is mostly about changing two macros, Py\_TYPE() and Py\_SIZE(). The Py\_TYPE() change was [approved by the previous SC](https://github.com/python/steering-council/issues/79) (last year), and the new SC doesn’t want to change that. The question is about the Py\_SIZE() change. I disagreed with the SC statement and [explained that in practice](https://mail.python.org/archives/list/python-dev@python.org/message/YQOSKYGDGRKBAFYW2AJQIIMYMQHALUOQ/), projects affected by the Py\_TYPE() change are already affected by the Py\_SIZE() change, so changing both at once is easy. Moreover, all the work (changing Python and updating PyPI top 5000 projects) was already done. My plan for PEP 674 was to write a new PEP explaining my overall long term vision for the C API: something like [PEP 620](https://peps.python.org/pep-0620/) “Hide implementation details from the C API” and the [HPy project](https://hpyproject.org/). Such PEP should justify why waiting for 5 years just for changing is a single macro would be a big issue shortly, since Python must change (see Python 3.11 C API changes pulled by the optimization work). But I got swallowed by Python 3.11 regressions at work. I helped investigating a bunch of Python 3.11 issues, fix Python and help projects to be adapted to Python 3.11. I helped adding many C API functions filling the hole of features removed by Python 3.11 C API changes (structures were made opaque like PyFrameObject, or PyCodeObject changes) **So here we are:** PEP 674 is basically implemented since Python 3.11 alpha 1, and the Python 3.11 release candidate 1 got released recently with the changes. What should be done with PEP 674? Revert the Py\_SIZE() change in Python 3.11? Update the PEP? Honestly, I’m now confused, I’m not sure what’s the best. When PEP 674 issues were discussed, it was proposed to keep the incompatible changes in beta versions and then revert it before the final release. Someone said that a project does something like that (numpy? I failed to find the email). By the way, I don’t recall any recent bug reports about Py\_TYPE() or Py\_SIZE() changes, it seems like people just managed to updated their C extensions. Moreover, the [pythoncapi-compat project](https://github.com/python/pythoncapi-compat/) was moved under the GitHub Python organization (python/pythoncapi-compat) and contains a tool to automate the migration (without losing support for old Python versions). ---
**Author:** ![nad](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/nad/32/51_2.png) [@nad](https://discuss.python.org/u/nad)\ **Post date:** [August 18, 2022, 3:01pm UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/2 "2022-08-18T15:01:04Z")
Sounds like the first thing to do is to ensure that there is a **release-blocker** issue open about this until it is resolved one way or another. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [August 18, 2022, 3:13pm UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/3 "2022-08-18T15:13:54Z")
… and to be honest, I think the change should be reverted - having something go in, when the SC explicitly said that it should not, simply because someone implmented it into main _before getting approval_ and then forgot to remove it, is pretty bad 🙁 Of course, removing it at this late stage would be a huge pain for the RMs, but I think we need to acknowledge that conversely, not doing so undermines the authority of the SC. @vstinner - why did you merge this without SC approval in the first place? ---
**Author:** ![ronaldoussoren](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ronaldoussoren/32/61_2.png) [@ronaldoussoren](https://discuss.python.org/u/ronaldoussoren)\ **Post date:** [August 18, 2022, 4:17pm UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/4 "2022-08-18T16:17:49Z")
That’s something for the RM to decide 😉 But as someone that was affected by this, I don’t think it is absolutely necessary to roll back this change. Adjusting C extensions for this change is pretty easy and also easy to do cleanly in a backward compatible way. Rolling back this change also has its risks, which includes breaking code that has already adjusted to these changes (depending on how that was done). ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [August 18, 2022, 6:49pm UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/5 "2022-08-18T18:49:40Z")
I’ve added this to the SC’s agenda to figure out what to do about this mistake. There’s at minimum a lesson to be learned here about keeping track of things and using issues as reminder is probably labels to prevent slip-ups from occurring in the future, regardless of whether anything gets reverted. ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 19, 2022, 9:49am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/6 "2022-08-19T09:49:48Z")
> [@pf\_moore](#): > > @vstinner - why did you merge this without SC approval in the first place? For 5 years, I made many changes in the C API. At the end, I started to do change which might affect C extensions (backward compatibility). I changed Py\_TYPE() and Py\_SIZE() and adapted third-party C extensions to these changes. But some people started to complain about these changes so I wrote a PEP to explain the rationale and communicate on the change. See [PEP 674 – Disallow using macros as l-values | peps.python.org](https://peps.python.org/pep-0674/#implementation) I made these changes in May 2020, but revert them because it broke too many 3rd party extensions. When enough extensions were ready, I made the change again. And then I wrote a PEP. I know that it’s common that a PEP is already implemented before being written down 🙂 ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 19, 2022, 9:54am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/7 "2022-08-19T09:54:53Z")
Adding abstraction (Py\_SET\_REFCNT, Py\_SET\_SIZE, Py\_SET\_TYPE) layers to no longer access directly PyObject members allows nice enhancements like the nogil project and PEP 683 Immortal objects. For the long term, it should give the ability to move away completely from PyObject and switch to another more efficient structure (or even not have objects at all but tagged pointers, for example). ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 19, 2022, 10:09am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/8 "2022-08-19T10:09:50Z")
If we decide to revert the change, here is a change for the main branch which should be then be backported to 3.11: [gh-89639: Revert Py\_SIZE() static inline function by vstinner · Pull Request #96119 · python/cpython · GitHub](https://github.com/python/cpython/pull/96119) ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 19, 2022, 10:15am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/9 "2022-08-19T10:15:55Z")
Ah, I noticed that I forgot something about PEP 674 context. When I wrote the first draft of [PEP 670: Convert macros to functions in the Python C API](https://peps.python.org/pep-0670/), I put Py\_TYPE() and Py\_SIZE() in this PEP. But the problem was how to explain the rationale for Py\_TYPE() and Py\_SIZE() changes in the same PEP. It was already tricky to explain PEP 670 rationale (convert macros to functions) and justify that PEP 670 has no significant impact on performance. [PEP 674: Disallow using macros as l-values](https://peps.python.org/pep-0674/) is built on top of PEP 670. PEP 670 was approved in April 2022, while PEP 670 was being implemented and PEP 674 was still being discussed. Then as I wrote, I got swallowed in other C API issues related to the optimization work. ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 19, 2022, 10:46am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/10 "2022-08-19T10:46:25Z")
> [@ronaldoussoren](#): > > But as someone that was affected by this, I don’t think it is absolutely necessary to roll back this change. Adjusting C extensions for this change is pretty easy and also easy to do cleanly in a backward compatible way. Rolling back this change also has its risks, which includes breaking code that has already adjusted to these changes (depending on how that was done). I suggested using the pythoncapi-compat project to projects which wanted to use Py\_SET\_SIZE() on old Python versions, but many projects went with their own solution. For example, I vaguely recall a project which redefines Py\_SIZE macro to get back the Python 3.10 Py\_SIZE macro which can be used as a l-value. I didn’t keep track of the diversity of solutions chosen by all projects, but PEP 674 links to many projects (and their fix). It’s not easy to guess if reverting the change now can break a project. ---
**Author:** ![ronaldoussoren](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ronaldoussoren/32/61_2.png) [@ronaldoussoren](https://discuss.python.org/u/ronaldoussoren)\ **Post date:** [August 19, 2022, 11:00am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/11 "2022-08-19T11:00:13Z")
> [@vstinner](#): > > I suggested using the pythoncapi-compat project to projects which wanted to use Py\_SET\_SIZE() on old Python versions, but many projects went with their own solution. I don’t use pythoncapi-compat in my projects, in part because these project are much older that your compatibility headers but also because I don’t want to track another project to see if something changed. > [@vstinner](#): > > For example, I vaguely recall a project which redefines Py\_SIZE macro to get back the Python 3.10 Py\_SIZE macro which can be used as a l-value. IMHO that would be the wrong solution to deal with changes like this, but at least its compatible with a revert. I’m more worried abut projects that switch to Py\_SET\_SIZE and define that macro themselves if Py\_VERSION\_HEX is before 3.11. That would break if the revert entails not just changing the definition of Py\_SIZE but also reverts adding Py\_SET\_SIZE. As a datapoint: I’m keeping Py\_SET\_SIZE in my projects, I’ve never really liked how using Py\_SIZE as an lvalue looks and like the new macro better. ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 19, 2022, 11:28am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/12 "2022-08-19T11:28:40Z")
> [@ronaldoussoren](#): > > I’m more worried abut projects that switch to Py\_SET\_SIZE and define that macro themselves if Py\_VERSION\_HEX is before 3.11. That would break if the revert entails not just changing the definition of Py\_SIZE but also reverts adding Py\_SET\_SIZE. [My PR](https://github.com/python/cpython/pull/96119) only changes Py\_SIZE() (convert it back to a macro), it leaves Py\_SET\_SIZE() unchanged. ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [August 30, 2022, 2:31pm UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/13 "2022-08-30T14:31:58Z")
> [@ronaldoussoren](#): > > I don’t use pythoncapi-compat in my projects, in part because these project are much older that your compatibility headers but also because I don’t want to track another project to see if something changed. About pythoncapi-compat, you **don’t have** to update it frequently. You only have to update it if it becomes incompatible with a newer version of CPython or PyPy, or if you want to get a new compatibility function (use a new Python version on old Python versions). In short, this is no need to update it unless there is a problem. ---
**Author:** ![pablogsal](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pablogsal/32/32246_2.png) [@pablogsal](https://discuss.python.org/u/pablogsal)\ **Post date:** [September 15, 2022, 10:55am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/14 "2022-09-15T10:55:33Z")
Hi, **I am writing this message on behalf of the Steering Council**. The Steering Council has discussed this in one of the recent meetings. Although we have to say that we are very unhappy that this happened, after discussion with the release management team, the Steering Council have decided not to revert these changes in 3.11 to not affect the stability of the release as at the time this was discussed, the 3.11 branch was already in release candidate 1. This doesn’t mean that the SC thinks this outcome is ok. We want to emphasise that this outcome is not acceptable, and neither is the way this was done. Changes that require a PEP **should never be included in a stable branch before the PEP is done** and if they ever land in one of the supported branches without the backing PEP, they should be reverted immediately. The Steering Council would like to remind everyone that respecting the process is fundamental to a good working environment in the core development team and failing to do so not only makes Python less stable but can also affect the perception of users on the project and its will also make life more difficult to other core devs and colleagues. Thanks for your understanding. In the name of the Steering Council, Pablo Galindo Salgado ---
**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\ **Post date:** [September 16, 2022, 9:09am UTC](https://discuss.python.org/t/pep-674-disallow-using-macros-as-l-values-and-python-3-11/18297/15 "2022-09-16T09:09:15Z")
> [@pablogsal](#): > > This doesn’t mean that the SC thinks this outcome is ok. We want to emphasise that this outcome is not acceptable, and neither is the way this was done. Changes that require a PEP **should never be included in a stable branch before the PEP is done** and if they ever land in one of the supported branches without the backing PEP, they should be reverted immediately. Sorry about that ☹ I didn’t come back to PEP 674 until recently for personal reasons. And it seems like nobody else noticed the issue in Python 3.11 before I did. The situation was special. I wrote the PEP _after_ the changes were made, since I knew that these were incompatible changes, some people had concerns and I didn’t want having the discussion on a bug tracker. I wanted a broader discussion with a PEP which is an efficient tool to take a decision. The history of the changes is documented in the [PEP 674: Implementation](https://peps.python.org/pep-0674/#implementation). The SC didn’t reject the PEP. Right now it still has the “Draft” status by the way. The SC asked to introduce incompatible changes differently: document well that the old API is deprecated and wait until the new API is available in all supported Python versions. Next time, I will revert first, before an agreement can be found (when the SC will give his decision). Also, I suggest the SC to request more explicitly an immediate revert next time. Sadly, it’s too easy to lose track of an issue and forget about it. In the meanwhile, nobody reported any issue about the Py\_SIZE() change, so I forgot about it. Well, I fixed most projects affected by PEP 674 before writing the PEP, so I’m not surprised 😉