Shared source tree for monorepo?

Python Help
1

I’m trying to create a set of Python packages that each contain cleanly delineated subsets of a single, massive upstream C library.

This seemed to be a natural use-case for the “Monorepo” pattern:

Source Tree Skeleton

(Full WIP source tree can be found at the pypqc#rewrite/2024-08-23 branch if you prefer to see a real directory rather than squinting at the output of tree)

.
│   CHANGELOG.md
│   CONTRIBUTING.rst
│   COPYING.rst
│   README.md
│
├───scripts/ 
└───src
    ├───lib
    │   └───PQClean <Git submodule>
    │       ├───common/
    │       ├───crypto_kem/
    │       └───crypto_sign/
    │
    └───projects
        ├───pypqc-cffi-bindings-falcon
        │   │   LICENSE-Falcon.txt
        │   │   pyproject.toml
        │   │   setup.py
        │   │   
        │   ├───lib <symlink to ../../lib>
        │   │
        │   └───src
        │       └───pqc
        │           └───_lib
        │               └───kem_falcon
        │                       __init__.py
        │
        ├───pypqc-cffi-bindings-hqc
        │   │   LICENSE-HQC.txt
        │   │   pyproject.toml
        │   │   setup.py
        │   │
        │   ├───lib <symlink to ../../lib>
        │   │
        │   └───src
        │       └───pqc
        │           └───_lib
        │               └───kem_hqc
        │                       __init__.py
        │
        ├───pypqc-cffi-bindings-kyber
        │   │   LICENSE-Kyber.txt
        │   │   pyproject.toml
        │   │   setup.py
        │   │
        │   ├───lib <symlink to ../../lib>
        │   │
        │   └───src
        │       └───pqc
        │           └───_lib
        │               └───kem_kyber
        │                       __init__.py
        │
        └───pypqc-cffi-bindings-libre
            │   pyproject.toml
            │   setup.py
            │   
            ├───lib <symlink to ../../lib>
            │
            └───src
                └───pqc
                    └───_lib
                        ├───kem_mceliece
                        │       __init__.py
                        │       
                        ├───sign_dilithium
                        │       __init__.py
                        │       
                        └───sign_sphincs
                                __init__.py

however, I’m running up against the issue that Setuptools (from v68.1 a year ago) will refuse to add referenced C sources to the sdist if they’re “outside of the project root”, even if they’ve been specially marked for inclusion with depends=, and have been symlinked in with relative, existent, ..-free paths that wouldn’t cause any heartburn if it just would try to include them.

Is there a workaround? A simple fix? Am I just barking up the wrong tree entirely with the design here?

2

For now, it looks like Setuptools still respects MANIFEST.in when it references symlinked-in files, so this works:

recursive-include cffi_modules *.py
# the below works even when ./lib is a symlink!
recursive-include lib/PQClean/crypto_kem/mceliece* *
recursive-include lib/PQClean/crypto_sign/dilithium* *
recursive-include lib/PQClean/crypto_sign/sphincs* *
include lib/PQClean/common/crypto_declassify*
include lib/PQClean/common/compat*
include lib/PQClean/common/fips202*
include lib/PQClean/common/randombytes*
include lib/PQClean/common/sha2*