Segfault in calling `locals` within list comprehension

Python Help
1

Reference: A YouTube video from one of the core developers: Gao Tian

Link: https://youtu.be/ltCNkcl9HGI

Code snippet that segfaults:

def f():
    lambda: k
    k = 1
    print([locals() for k in [0]])

f()

I watched the above video with amazement, and I am attempting to explain in English (because the video is in Chinese) and hope to get some feedback on whether my explanation is accurate or not.

First,

lambda: k
k = 1

this creates a closure, and puts k as a PyCell, storing the value of 1.

Second,

[locals() for k in [0]]

because of inline list comprehension, the interpreter recognizes that k is a variable that has the same name as the captured variable k, it stores k, of type PyCell somewhere and deletes it from the current scope.

Third,

when locals() is invoked within the inlined list comprehension:

a. locals() searches within the function scope of f() to determine the local variables.
b. it uses the PyCodeObject, importantly, f()'s PyCodeObject, to determine what variables is defined in its local scope.
c. Why is it f()'s PyCodeObject? It is because inlined list comprehension does not create a new PyCodeObject.
d. f() thinks that the variable k is a captured variable stored in aPyCell because of lambda: k; k = 1;.
e. However, because [locals() for k in [0]] stores k somewhere else, k is not a PyCell.
f. Therefore, attempting to read the value of whatever that is stored in that k as a PyCell segfaults.

Do let me know if there are inaccuracies and please do provide elaboration on each of them.

Thank you.

1 Like
2

I can confirm it crashes on Fedora 41 with python 3.13.

3

Raise a bug ticket segv with locals() using in list comprehension · Issue #130451 · python/cpython · GitHub

4

No, thank you for raising an issue but I think this has been resolved. I am purely asking this question for pedagogical reasons.

I need to understand why this happens.

5

You mean there is git commit fixing the crash?
Without that its not fixed.

6

On Windows it crashes the REPL in Python 3.14.0a3 & Python 3.13.1. I assume it crashes when run as a script too - nothing is printed - but the OS doesn’t complain (I don’t know what to look for, for a segfault in Windows).

In Python 3.12.1 I get this:

>>> def f():
...   lambda: k
...   k = 1
...   print([locals() for k in [0]])
...
>>> f()
[{'k': 0}]

Python 3.9.13 prints [{'.0': <tuple_iterator object at 0x000001E2CB49FCD0>, 'k': 0}] and curiously, Python 2.7 prints the same as 3.12.1

Thanks for the great short reproduction.

7

I made the video after I fixed the problem, so yes it’s already fixed. The fix is too recent to be released now. They will be in the next release for 3.14a and 3.13. You can confirm the fix with the main branch.

3 Likes
8

Would have been nice to know in the OP’s first post that its fixed and where the bug ticket was.

2 Likes
9

Guys…

It’s cool you guys resolved on whether the problem is fixed or not, but…

Can someone affirm my explanation in the original post? :sweat_smile:

10

Hi @gaogaotiantian

Thank you for your reply.

I would like to check with you if the explanation in my original post is accurate to explain why the segmentation fault happens.

Thank you.

11

I don’t e. is accurate. k is stored at the same place as the frame level k - the original one is pushed to stack. I think my video is pretty clear and I tried my best to explain it in the video. If you are still super interested in what happened, feel free to read the source code :slight_smile:

1 Like