Frame state always `running` in `sys.setprofile` callback

Core Development C API
1

Hi,

During adding support for 3.14in Yappi profiler, we realized incorrect wall-time calculations in tests. Digging the issue it turns out that it basically fails to return if a frame is suspended or not in a profiler callback.

I have written some details for this on my other thread here, but I think this question very different than the other one, so here I ask it again.

The best way to show this problem is an example:

import sys
import asyncio
import inspect

def profile_callback(frame, event, arg):
    if event == 'return':
        func_name = frame.f_code.co_name
        
        if func_name in ['foo']:
            state = inspect.getcoroutinestate(frame.f_generator)
            print(f"Coroutine state: {state}")


async def foo(n):
    await asyncio.sleep(n)

def main():
    sys.setprofile(profile_callback)
    asyncio.run(foo(0.1))

if __name__ == "__main__":
    main()

When I ran above on 3.14 I got two subsequent running states:

p » py g.py
Coroutine state: CORO_RUNNING
Coroutine state: CORO_RUNNING

profile_callback is called(RETURN) first because of an await and next because the function finished. Now, on 3.13 and before I was accessing the PyFrame_GetGenerator(frame)→gi_frame_stateand it was always returning FRAME_SUSPENDED , now this behaviour changed.

I am wondering:

  1. is this behaviour ok? I feel that, in the first callback the coroutine state should be CR_SUSPENDED?
  2. if this is somehow ok, what can I do? how can I detect if a coroutine is suspended vs exit in a profiler callback?

Any idea is appreciated,

2 Likes
2

FWIW: I started using following approach: (get current instruction from the frame obj. and detect if it is a YIELD or not)

#if PY_MAJOR_VERSION >= 3 && PY_MINOR_VERSION == 14
    unsigned char curr_op_code = (*frame->f_frame->instr_ptr).op.code;
    return curr_op_code == YIELD_VALUE || curr_op_code == INSTRUMENTED_YIELD_VALUE;
#elif PY_MAJOR_VERSION >= 3 && PY_MINOR_VERSION == 13

Related PR: Fix #193: Invalid coroutine state in `RETURN` event by sumerc · Pull Request #194 · sumerc/yappi · GitHub

Tests are green but this is a hackish way to do this kind of thing. I am hoping maybe we can somehow improve the situation for sys.setprofile hook?