Embedding Python 3.12 error

cpuchakayala_pfptbb · January 8, 2024, 5:54pm

I am getting the following error when using Python C API version 3.12
embed python error "this header file must not be included directly", I am embedding in a C application, how do I solve this?

barry-scott · January 8, 2024, 6:10pm

Please provide context for that error.

cpuchakayala_pfptbb · January 8, 2024, 6:49pm

I am embedding Python in a GO application. My app is calling python function via C API, here is the sample app,

main.go

package main

// #cgo pkg-config: python3-embed
// #define PY_SSIZE_T_CLEAN
// #include "Python.h"
import "C"

func main() {
	PythonInit()
	PythonFinalize()
}

func Py_Initialize() {
	C.Py_Initialize()
}

func Py_Finalize() {
	C.Py_Finalize()
}

func PythonInit() {
	Py_Initialize()
}

func PythonFinalize() {
	Py_Finalize()
}

here is the build process

 (.venv) ➜  gopy2 export PKG_CONFIG_PATH=/Users/cpuchakayala/.pyenv/versions/3.12.1/lib/pkgconfig
(.venv) ➜  gopy2 go build main.go                                                                  (kind-knative/default)
# command-line-arguments
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:41:
../../../../../.pyenv/versions/3.12.1/include/python3.12/pymem.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:44:
../../../../../.pyenv/versions/3.12.1/include/python3.12/object.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:45:
../../../../../.pyenv/versions/3.12.1/include/python3.12/objimpl.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:49:
../../../../../.pyenv/versions/3.12.1/include/python3.12/bytearrayobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:50:
../../../../../.pyenv/versions/3.12.1/include/python3.12/bytesobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:51:
../../../../../.pyenv/versions/3.12.1/include/python3.12/unicodeobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:52:
../../../../../.pyenv/versions/3.12.1/include/python3.12/longobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:55:
../../../../../.pyenv/versions/3.12.1/include/python3.12/floatobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:56:
../../../../../.pyenv/versions/3.12.1/include/python3.12/complexobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:58:
../../../../../.pyenv/versions/3.12.1/include/python3.12/memoryobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:59:
../../../../../.pyenv/versions/3.12.1/include/python3.12/tupleobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:60:
../../../../../.pyenv/versions/3.12.1/include/python3.12/listobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:61:
../../../../../.pyenv/versions/3.12.1/include/python3.12/dictobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:64:
../../../../../.pyenv/versions/3.12.1/include/python3.12/setobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:65:
../../../../../.pyenv/versions/3.12.1/include/python3.12/methodobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:69:
../../../../../.pyenv/versions/3.12.1/include/python3.12/fileobject.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:72:
../../../../../.pyenv/versions/3.12.1/include/python3.12/pyframe.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:73:
../../../../../.pyenv/versions/3.12.1/include/python3.12/traceback.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
In file included from ./main.go:5:
In file included from ../../../../../.pyenv/versions/3.12.1/include/python3.12/Python.h:78:
../../../../../.pyenv/versions/3.12.1/include/python3.12/pystate.h:2:4: error: "this header file must not be included directly"
#  error "this header file must not be included directly"
   ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generate

cpuchakayala_pfptbb · January 8, 2024, 6:52pm

The same code works for 3.7.x and 3.10.x, there have been changes with header files from 3.11.x

barry-scott · January 8, 2024, 7:02pm

Odd i do not see those error messages in the sources on github for the 3.12 branch.

Have you read the header files to see why that error is reported?

cpuchakayala_pfptbb · January 8, 2024, 7:25pm

Yes, here is an

example from object.h file

#ifndef Py_CPYTHON_OBJECT_H
#  error "this header file must not be included directly"
#endif

ex: dictobject.h

#ifndef Py_CPYTHON_DICTOBJECT_H
#  error "this header file must not be included directly"
#endif

So all the header files, have similar macros

So Python.h file has this

 // Entry point of the Python C API.
// C extensions should only #include <Python.h>, and not include directly
// the other Python header files included by <Python.h>.

which is what I am doing

So how do I embed python in another application?

barry-scott · January 8, 2024, 9:42pm

I compile against Python.h for my pysvn project and do not see this error.

I think you are using the wrong include directory based on what I see on macOS install of python 3.12.

What I see is this;

% find . -name object.h
./python3.12/cpython/object.h
./python3.12/object.h

The #error pragmas are in python3.12/cpython/object.h but that is not what you must include which is python3.12/object.h.

My guess is that you need to fix the path for the -I to the compiler.

cpuchakayala_pfptbb · January 8, 2024, 10:36pm

the headers was from python3.12 directory not the python3.12/cpython. all the headers have the error pragma, i installed via pyenv.

here is the pkg-config

# See: man pkg-config
prefix=/Users/cpuchakayala/.pyenv/versions/3.12.1
exec_prefix=${prefix}
libdir=/Users/cpuchakayala/.pyenv/versions/3.12.1/lib
includedir=${prefix}/include

Name: Python
Description: Embed Python into an application
Requires:
Version: 3.12
Libs.private: -lintl -ldl -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -framework CoreFoundation
Libs: -L${libdir} -lpython3.12
Cflags: -I${includedir}/python3.12

barry-scott · January 9, 2024, 9:14am

Then it seems pyenv is not installing all the headers correctly.

Topic		Replies	Views
How to include modules in my embedded python static library? Python Help c-api	0	893	January 3, 2023
Python Embedded application failing to import http.server Python Help help	1	292	October 29, 2021
Http.server not loading in embedded python Python Help	0	517	October 28, 2021
Embedding Python in a C++ self-contained executable Python Help help	24	9238	November 14, 2023
Where is io.h file used as an #include in pyconfig.h (Python 3.12) Python Help	3	2821	January 10, 2024

Embedding Python 3.12 error

Related Topics