Let me start by pointing out that beyond very simple examples, I’m jumping off the deep end with C. I have some C# & Swift experience prior (also consider myself ~master but shy of Doctorate in pure Python) which has all helped me pick up on this pretty quickly, and I’ve already ported several functions (surprised myself with how fast I picked it up even). However, I’m stuck on this one issue pretty hard and not quite sure even how to google it (obviously have tried).
My module/extension needs some global dictionaries avaliable both in the module, and in Python.
To be very specific, from Bitbucket, I’m trying to express lines 36, 40, 44, and 47 via the CPython API. However, just putting e.g.:
PyObject* worldBlocks = PyDict_New();
in the global scope of my C does not work. The compiler throws:
world_test.c:17:25: error: initializer element is not constant
If I put the same code in a function that returns void, I get the same error. If I put it in a function that returns PyObject, then it works. This behavior baffles me, probably due to my awkward language background and overall lack of knowledge in C. Actually, I more or less understand why the above doesn’t work in C global scope, but not quite sure about why it doesn’t in a generic void function (discovered that on accident)… Anyway, still not sure how to get the dicts I need into the this mystical global land that certainly has to exist…
I’m lost on this one, which feels weird with how far I’ve come. I ported my terrain generator (Bitbucket) with relative ease!
Sorry, I guess you’re right - Popping those lines into a function that returns void works perfectly fine. Got my wires crossed somehow there, or perhaps made another error last night while I was trying things out.
My bad! I’ve tried a few more things with that in mind, but not getting anywhere. Tried adding a get function for each global dictionary, but it segfaults after a couple of calls. I was burned out so haven’t looked into that any further yet.
Still feels like I’m overlooking something. Much more googling lead me here, which may have the solution but it’s late and I’m tired so will read up on that tomorrow.
Thanks for the replies. I need access to the dictionaries in BOTH Python and C.
I did try Cython initially but I spent a week just to break even on performance. It became clear I’d need to not only learn many C concepts but also Cython syntax… Picking up C has been quite a bit easier and the returns on my time way better.
I think I"m on the right track, defining my dicts as C globals, initializing them in my PyMODINIT_FUNC function, then writing “get” functions (that also has to increment the reference counter) get them into Python.
The weird thing now, though, is that my test.py segfaults, but if I run the interpreter via gdb, I can’t reproduce the issue and everything seems to work perfectly.
Yes, which is why I pointed you to a real-world example. The os.environ object is exactly such a dictionary, it is created in a C-level module and accessible from Python too.
Sorry, I was on my phone when I replied, and overlooked that. I figured it out anyway and just came back to update that I was able to implement it. Edit: Thought it solved my segfault, but a bit more testing brought it back.
Thank you… I had a sneaking suspicion it might have something to do with the reference counters, but since I couldn’t reproduce the issue with gdb, was feeling a bit helpless to solve it (I had actually already solved one such issue using gdb so thought it would show me where I’m making rookie mistakes like that).
I will go over everything again with that info in mind.
Update: Success! Martijn Pieters’ tip helped me get the global dicts setup properly and Jeremy Kloth’s tip solved my segfault. This is probably not the best way to do this, but I basically just printed before and after reference counts throughout the code, to look for unexpected results. Turns out, a PyListObject I was adding to a dictionary was losing reference, and need an Py_INCREF. For now, seems like the ref counts stay correct (actually, maybe some areas to Py_DECREF)!
Cheers and many thanks to all who replied…Everyone taught me something valuable!