(I have this question yesterday on Stack Overflow, How to clone and compile C++ code into binaries within the package during `pip install`? - Stack Overflow, and an answer suggested me to ask it here instead)
I need to make a Python package openface, that will act as an API of some existing C++ research code OpenFace, that I don’t own.
I have forked the existing code base into openface-backend, and rewrote the installation script install.sh to not require sudo, such that after cloning the repository and installing the system requirements, it compiles the code and download assets build/bin.
At that point, one can run ./bin/foo --bar /path/to/abc --baz /path/to/xyz and it all works.
Regarding the Python package, for a starter it will be very basic.
For the executable ./bin/foo, I will have a corresponding function foo():
import subprocess
import shlex
BINARY_PATH = "/path/to/bin/foo"
def foo(bar: str, baz: str):
args = f"{BINARY_PATH} --bar {bar} --baz {baz}"
try:
process = subprocess.run(
args=shlex.split(args),
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
)
print(process.stdout)
except subprocess.CalledProcessError as error:
print(error.stdout)
I would like that when installing the package with pip, such as pip install 'git+https://github.com/GuillaumeRochette/openface.git', the openface-backend repository would be downloaded and compiled, and that the binaries are to be moved somewhere in the package.
To try and make it work, I have followed the instructions from another answer.
This is my setup.py:
from setuptools import setup
from setuptools.command.develop import develop
from setuptools.command.install import install
import subprocess
import shlex
def compile():
args = (
'bash -c "'
"git clone https://github.com/GuillaumeRochette/openface-backend.git"
" && "
"cd openface-backend"
" && "
"bash install.sh"
" && "
"mv build/bin .."
" && "
"cd .."
" && "
"rm -rf openface-backend"
'"'
)
try:
process = subprocess.run(
args=shlex.split(args),
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
)
print(process.stdout)
except subprocess.CalledProcessError as e:
print(e.stdout)
class CustomDevelop(develop):
def run(self):
develop.run(self)
compile()
class CustomInstall(install):
def run(self):
install.run(self)
compile()
if __name__ == "__main__":
setup(
cmdclass={
"develop": CustomDevelop,
"install": CustomInstall,
},
)
While the code is getting downloaded and compiled when doing python setup.py install within that repository, this is not happening when doing pip install 'git+https://github.com/GuillaumeRochette/openface.git'.
I don’t know if it is the pyproject.toml which is missing something, or if my setup.py needs some changes or if this is something totally different.