PyUnicode_AsUTF8(obj) crashes program in mebedding Python in C++

Python Help
1

I’m trying to embed Python to JUCE for an audio plugin and my host program crashes as soon as I choose the plugin. Running the host program with GDB gave me this backtrace:

#0  PyUnicode_AsUTF8AndSize (unicode=0x0, psize=0x0) at Objects/unicodeobject.c:4110
#1  0x00007fffcb4f6e1e in pyo_get_input_buffer_address (interp=0x1a2f0f0) at ../../../../../Applications/pyo/pyo-1.0.6/embedded/juceplugin/m_pyo.h:153
#2  Pyo::setup (this=this@entry=0x1794938, _inChannels=<optimized out>, _outChannels=2, _bufferSize=<optimized out>, _sampleRate=<optimized out>) at ../../../../../Applications/pyo/pyo-1.0.6/embedded/juceplugin/PyoClass.cpp:40
#3  0x00007fffcaf1fa63 in PyoJuceAudioProcessor::prepareToPlay (this=0x17946f0, sampleRate=<optimized out>, samplesPerBlock=<optimized out>) at ../../Source/PluginProcessor.cpp:98
#4  0x00007fffcaf11882 in juce::JuceVST3Component::preparePlugin (this=this@entry=0x1eaacf0, sampleRate=48000, bufferSize=1024, callPrepareToPlay=callPrepareToPlay@entry=juce::JuceVST3Component::CallPrepareToPlay::yes)
    at /home/alex/Applications/JUCE/modules/juce_audio_plugin_client/juce_audio_plugin_client_VST3.cpp:3899
#5  0x00007fffcaf12198 in juce::JuceVST3Component::setActive (this=0x1eaacf0, state=<optimized out>) at /home/alex/Applications/JUCE/modules/juce_audio_plugin_client/juce_audio_plugin_client_VST3.cpp:2828
#6  0x0000000000a621f9 in ??? ()
#7  0x0000000000a38908 in ??? ()
#8  0x0000000000a4273b in ??? ()
#9  0x00000000008609e7 in ??? ()
#10 0x00000000008227f1 in ??? ()
#11 0x00000000007fcf33 in ??? ()
#12 0x00000000007fd5bc in ??? ()
#13 0x00000000007fdac4 in ??? ()
#14 0x0000000000815127 in ??? ()
#15 0x00007ffff762b836 in ??? () at /home/alex/Applications/reaper_linux_x86_64/REAPER/libSwell.so
#16 0x00007ffff76253a7 in ??? () at /home/alex/Applications/reaper_linux_x86_64/REAPER/libSwell.so
#17 0x00007ffff762d87b in ??? () at /home/alex/Applications/reaper_linux_x86_64/REAPER/libSwell.so
#18 0x00007ffff76257fd in ??? () at /home/alex/Applications/reaper_linux_x86_64/REAPER/libSwell.so
#19 0x00007ffff764a0d7 in ??? () at /home/alex/Applications/reaper_linux_x86_64/REAPER/libSwell.so
#20 0x00007ffff764d0fd in ??? () at /home/alex/Applications/reaper_linux_x86_64/REAPER/libSwell.so
#21 0x00007ffff705b5c9 in ??? () at /lib/x86_64-linux-gnu/libgdk-3.so.0
#22 0x00007ffff70b2226 in ??? () at /lib/x86_64-linux-gnu/libgdk-3.so.0
#23 0x00007ffff6e8e385 in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#24 0x00007ffff6e905b7 in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#25 0x00007ffff6e90d20 in g_main_context_iteration () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#26 0x00007ffff764fdaa in ??? () at /home/alex/Applications/reaper_linux_x86_64/REAPER/libSwell.so
#27 0x00007ffff762bbf2 in ??? () at /home/alex/Applications/reaper_linux_x86_64/REAPER/libSwell.so
#28 0x000000000040af66 in ??? ()
#29 0x00007ffff7917ca8 in __libc_start_call_main (main=main@entry=0x40a4b0, argc=argc@entry=1, argv=argv@entry=0x7fffffffde08) at ../sysdeps/nptl/libc_start_call_main.h:58
#30 0x00007ffff7917d65 in __libc_start_main_impl (main=0x40a4b0, argc=1, argv=0x7fffffffde08, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffddf8) at ../csu/libc-start.c:360
#31 0x000000000041926a in _start ()

This comes from the following function in a header file I’m using:

INLINE unsigned long pyo_get_input_buffer_address(PyThreadState *interp) {
    PyObject *module, *obj;
    const char *address;
    unsigned long uadd = 0;
    PyEval_AcquireThread(interp);
    module = PyImport_AddModule("__main__");
    obj = PyObject_GetAttrString(module, "_in_address_");
    address = PyUnicode_AsUTF8(obj);
    uadd = strtoul(address, NULL, 0);
    PyEval_ReleaseThread(interp);
    return uadd;
}

and specifically, this line: address = PyUnicode_AsUTF8(obj);. I have used this header file in other projects in C/C++ (namely, Pure Data and openFrameworks) and it works fine. In this case though, I guess address = PyUnicode_AsUTF8(obj);returns NULL, right? How can I check this? Any ideas on why this is happening?

2

Python C API functions usually return NULL to indicate that an exception has been raised. Specifically in this case:

So you’ll want to check for that before using the resultant bytes object:

The simplest, for debugging purposes, would be if (!address) PyErr_Print() although I don’t know what else you would do after that.

3

I tried both if (!address) PyErr_Print() and if (!obj) PyErr_Print(), just above address = PyUnicode_AsUTF8(obj);, but nothing happened.

4

Check every return value if necessary. Do you have a module? Do you have an object in that module? Do you have an address? If any step fails, the next steps won’t be able to work.

5

I changed the function to the following:

INLINE unsigned long pyo_get_input_buffer_address(PyThreadState *interp) {
PyObject *module, *obj;
const char *address;
unsigned long uadd = 0;
PyEval_AcquireThread(interp);
module = PyImport_AddModule(“main”);
if (!module) printf(“module error\n”);
else printf(“module initialized\n”);
obj = PyObject_GetAttrString(module, " _in_address_");
if (!obj) printf(“obj error\n”);
else printf(“obj initialized\n”);
address = PyUnicode_AsUTF8(obj);
if (!address) printf(“address error\n”);
else printf(“address initialzed\n”);
uadd = strtoul(address, NULL, 0);
PyEval_ReleaseThread(interp);
return uadd;
}

and it seems that the obj variable fails. I tried PyErr_Print() but didn’t get anything, so I switched to simple prints to the console.
Reading the docs of PyObject_GetAttrString()I can’t see what is wrong? Is " _in_address_“causing the issue?

6

Yes. Why do you think that __main__._in_address_ is defined?

7

I have no idea. My knowledge in embedding Python to C/C++ is next to nothing. Sorry about that.

Do you suggest a specific change in this line?

8

You are importing the __main__ module and query the member _in_address_. The query will give you a NULL pointer and sets the exception flag if that thing does not exist. Because of a lack of context, I have not the slightest idea why you expect that __main__._in_address_ is defined.

Btw, your code shows " _in_address_" with a space in the string. That seams to be incorrect in any case.

1 Like
9

Hmm. That would be something to look into. I’m not sure what’s going on with the embedding, but I would recommend trying some of the other ways of displaying errors, for example calling PyErr_GetRaisedException() and then manually printing out something (starting with the name of the exception’s type).

1 Like
10

Excuse my ignorance, I tried to find information online on how to use this and found this. So I wrote this at the top of the file:

INLINE void handle_exception() {
    PyObject *exception = PyErr_GetRaisedException();
    if (exception != NULL) {
        // Do something with the exception
		printf("got exception\n");
        PyErr_Print();  // Print the exception to standard error
        Py_DECREF(exception);  // Don't forget to release the reference
    }
}

but nothing happens, not even got exception is printed. How should I go about using PyErr_GetRaisedException()? BTW, I removed the space in "_in_address_”.

11

I have used the same header file in a C project (building an object for Pure Data) and for embedding Python in C++ (with openFrameworks) and it works fine, that’s why I was expecting this header file to work with Juce too.

12

Digging a bit more, I found that _in_address_ is indeed specified with the following:

    PyRun_SimpleString("_in_address_ = _s_.getInputAddr()");

Here, _s_ is an object of the Pyo module, and reading its docs, I read that getInputAddr() returns the address of the input buffer (Pyo is a module for DSP). But digging a bit further, I figured out that the audio server (which is supposed to return the input buffer address) does not exist. So, a line earlier in this header file:

    sprintf(msg, "_s_ = Server(sr=%f, nchnls=%d, buffersize=%d, duplex=1, ichnls=%d)", sr, ochnls, bufsize, ichnls);
    PyRun_SimpleString(msg);

fails in the first place. I guess that Stefan was right in the first place. I have to figure out what is wrong with the audio server, which might not be C API related.

Thanks for your answers so far, I’ll get back here if I figure it out.

1 Like
13

Okay, cool! I’m still concerned about two other things, though:

  1. Why is GetAttrString returning NULL but then GetRaisedException isn’t giving you an exception? That seems wrong.
  2. What is the best way to print out exceptions? You will undoubtedly run into them again in the future.

They’re meta-issues, and if you don’t have time, you can ignore them, but I always want to have a way to report errors when they occur.

1 Like
14

One problem might occur if the output buffer has not been flushed before the program crashes.

15

Digging further, this is a possible cause of the crash:

ImportError: /usr/local/lib/python3.13/lib-dynload/math.cpython-313-x86_64-linux-gnu.so: undefined symbol: PyFloat_Type

After this, the Pyo audio Server can’t be created, because it’s not defined, so does the _s_ object, which results in _in_address_ not being defined.

I’m trying to find out what causes this error. In the mean time, if anyone knows more about this, I would really appreciate the help (I already appreciate your help thus far!).

I’m on Debian 13 with Python3.13.2.

16

I finally solved it! I added -rdynamic to the linker flags and dlopen("libpython3.13.so.1.0", RTLD_LAZY | RTLD_GLOBAL) before the call to PyInitialize() and it finally worked.

Thank you both for your help!