Silent fail during initialization of embedded Python in a PYD

I participate in a Python project, which utilizes industry cameras, such as Basler’s or Allied Vision’s, to inspect quality of products’ packaging. I am using Basler’s Python Pylon API to communicate with cameras. I have stumbled upon a problem that is fixable only from the Pylon C++ API. Since it is impossible for Python and C++ Basler APIs to talk to each other (like, the camera object created in Python cannot be converted anyhow to a C++ object), I had to move camera object’s creation and management into the C++ code. There is a separate implementation for every camera manufacturer (vimba.py for Allied Vision, basler.py for Basler, etc.), but all of them must satisfy the inner common interface used by the camera manager class in our project so that they remain compatible (Init, Open, etc.), so I have decided to not touch the signatures, yet simply swap their bodies with the calls to the C functions (exported in the PYD file, which, in turn, is imported by basler.py), which now contain the actual Basler implementation code. Basically speaking, basler.py imports the PYD file with the functions and basler.py’s methods call them. I have read that you must Py_Initialize before calling any Python API functions so that embedded Python is initialized correctly. And… I think it doesn’t work for me, because, when there’s the need to use any integer from Python’s small int range ([-5, 256]), then there is a segmentation fault. I should note that I run everything in a venv.

What I have tried:

  • prioritizing venv Python in the PATH,
  • modifying PYTHONPATH and PYTHONHOME anyhow,
  • adding paths to sys.path,
  • checking all paths in pyvenv.cfg,
  • trying the actual system interpreter instead of the venv one to run the application (and prioritizing it in the PATH),
  • checking PyErr_Occurred and Py_IsInitialized,
  • checking literally every Python C API call (and I stick with this now),
  • running the app as an administrator,
  • reinstalling Python and even completely resetting Windows to the clean and new state.

I also should note there are no system codec or missing packages errors, like many other users experience. I can easily check if e.g. small integers have been initialized by calling PyLong_FromLong(1L);, which causes a crash already. Is there anything I could have missed?

Best regards

Is your C++ code in a Python extension module? That is, do you use “import” to load the C++ code and access its interface from Python?

An extension module should not call Py_Initialize, that’s already done before the import system tries to load the extension module.

Py_Initialize is only used if you start the Python interpreter from a C++ program and want to run Python code in that program (such as in a webserver that can run Python scriptlets to handle request).

I use import_module like this:
self.basler_impl = import_module(".BaslerImpl", package="Inspect2.acquisition.subsystems")
The result is always the same, no matter if I run the Python init function or not. Also it tells me it is initialized from the beginning (Py_IsInitialized), so I see it myself, yet it does not want to work… :frowning:

@spectral Since I contribute to Basler’s pypylon from time to time, I would be interested to know what the problem is.

@RoccoMatano there is the fact that I called this a problem from my perspective, but, AFAIK, after speaking with one of Pylon C++ engineers, it’s just that Pypylon doesn’t support some features yet, such as UseExtendedId, because they have been implemented very recently, you need the newest (7.2.1 as of 18.04.2023) version of Pylon in order to run this. I do the following stuff in C++ (and it’s proven it DOES change UseExtendedId):

        c.Open();
        if (!c.GevGVSPExtendedIDMode.TrySetValue(Basler_UniversalCameraParams::GevGVSPExtendedIDMode_On)) {
            PyErr_WarnEx(PyExc_RuntimeWarning, (std::string{ "Unfortunately, it is impossible to set the GevGVSPExtendedIDMode parameter to \"On\" for the camera id \"" } + std::string{ id } + "\"!\nThe camera will most likely stop after grabbing 32767 images!").c_str(), 1);
        }
        if (!c.GetStreamGrabberParams().UseExtendedIdIfAvailable.TrySetValue(true)) {
            PyErr_WarnEx(ditto);
        }
        if (!c.UseExtendedIdIfAvailable.TrySetValue(true)) {
            PyErr_WarnEx(ditto);
        }

Anyway, I can actually close the question. I learnt yesterday that the camera may save the UseExtendedId value between openings. I wrote a short C++ code to open the camera, set the options (without these Python warnings) and close, only return whether the whole operation was successful or not. It turned out that I opened Pylon Viewer and saw this option being turned on after running the “pre-initialization phase”. I call this special, single C function from Python in BaslerCamera’s constructor and leave all remaining code exactly the way it had been before.

@spectral FYI: pypylon 2.0.0 is going to be released soon. It builds on pylon 7.2 and has support for UseExtendedIdIfAvailable. With pypylon 2.0.0 you can do something like this:

from pypylon import pylon, genicam
print(pylon.__version__)

TLF = pylon.TlFactory.GetInstance()
for di in TLF.EnumerateDevices():
    if di.GetDeviceClass() != 'BaslerGigE':
        continue

    cam = pylon.InstantCamera(TLF.CreateDevice(di))
    cam.Open()
    ueiia = cam.GetStreamGrabberNodeMap().GetNode("UseExtendedIdIfAvailable")
    print(f"previous value of 'UseExtendedIdIfAvailable': {ueiia.GetValue()}")
    ueiia.SetValue(True)
    print(f"current value of 'UseExtendedIdIfAvailable': {ueiia.GetValue()}")

which will print something like this:

2.0.0rc1
previous value of 'UseExtendedIdIfAvailable': False
current value of 'UseExtendedIdIfAvailable': True