As a developer not so familiar with Python, I found it very hard and disappointing to read the official documentation for writing C extension, which lacks a minimal working example and leads to many errors when following those guides.
To begin with, I would like to create a Python wrapper for my library, and as a common practice for C projects, I organized the files as
.
├── inlcude
│ └── foo.h
└── lib
└── libfoo.so
To create a Python wrapper, I prefer the official documentation, which I think, unlike some outdated blogs, it would be up-to-date and correct. So I began to read the Extending Python with C or C++.
The code part for the documentation is very detailed, and I quickly write the necessary C glue codes including PyMethodDef, PyModuleDef, and the PyMODINIT_FUNC initialization function.
Then how can I build this project? From the Compilation and linkage subsection, it recommends to go to the building section, which further guided me to the setuptools documentation, where I found the related docs are in Building Extension Modules part.
In that documentation, it tells me to create a pyproject.toml alongside my C wrapper. So now, the project layout is
.
├── foo_python_wrapper.c
├── inlcude
│ └── foo.h
├── lib
│ └── libfoo.so
└── pyproject.toml
And the content of pyproject.toml is
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "mylib-foo"
version = "0.1"
[tool.setuptools]
ext-modules = [
{name = "mylib.foo", sources = ["foo_python_wrapper.c"]}
]
This would obviously not work, since I have not written the header path and library path. And unfortunately, the documentation does not contain the solutions, and searching google did not help 
By looking at the “Extension API Reference” at the bottom of the documentation, I found the include_dirs and library_dirs may be what I want, but how can I use them in a TOML file? After some trails, I managed to write the pyproject.toml as
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "mylib-foo"
version = "0.1"
[tool.setuptools]
ext-modules = [
{name = "mylib.foo", sources = ["foo_python_wrapper.c"], include-dirs=["./include"], library-dirs=["./lib"]}
]
Now can we proceed? After running python3 -m build, another error occurs:
error: Multiple top-level packages discovered in a flat-layout: ['lib', 'inlcude'].
To avoid accidental inclusion of unwanted files or directories,
setuptools will not proceed with this build.
If you are trying to create a single distribution with multiple packages
on purpose, you should not rely on automatic discovery.
Instead, consider the following options:
1. set up custom discovery (`find` directive with `include` or `exclude`)
2. use a `src-layout`
3. explicitly set `py_modules` or `packages` with a list of names
To find more information, look for "package discovery" on setuptools docs.
To be honest, I still don’t know what to do by looking at those three options. So I followed its last line, and went to the Package Discovery and Namespace Packages part of official documentation.
In that documentation, it explains the two common project layouts used for Python packages. However, the examples are all pure-python packages, and I still don’t know what to do with my library wrapper.
Then there is no clue at official documentations, leaving me confusing and lost in strange blogs found in google…
