# Packaging a compiled executable **URL:** \ **Category:** Packaging\ **Tags:** help\ **Created:** [August 1, 2021, 10:16pm UTC](https://discuss.python.org/t/packaging-a-compiled-executable/9954 "2021-08-01T22:16:55Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![LewisGaul](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@LewisGaul](https://discuss.python.org/u/LewisGaul)\ **Post date:** [August 1, 2021, 10:16pm UTC](https://discuss.python.org/t/packaging-a-compiled-executable/9954/1 "2021-08-01T22:16:56Z")
I’m trying to write a `setup.py` (or `pyproject.toml`/`setup.cfg` equivalent) for a project that contains an executable compiled for the target archicture. I’m hoping to support: - Creation of target-dependent wheels - pip installing the project to trigger the executable to be compiled when wheels are unavailable More details on the setup I’m trying to achieve: - `setup.py` runs a compiler to create `my-executable` - My project depends on this executable, running it via `subprocess.run(os.path.join(some_path, "my-executable")`, where I’m unsure what `some_path` should be (determined by `pkg_resources` perhaps?) I believe this is effectively what’s being asked in [Compiling & installing C executable using python's setuptools/setup.py? - Stack Overflow](https://stackoverflow.com/questions/33168482/compiling-installing-c-executable-using-pythons-setuptools-setup-py), but I’m not convinced by the solution there (and installing the executable to a venv’s bin directory isn’t particularly the goal for me - I’m expecting it to be internal to the project’s package). The executable is written in Zig in my case, and I plan on using [ziglang · PyPI](https://pypi.org/project/ziglang/) as a build-time dependency to fetch the Zig compiler, although I believe my question would be effectively the same if the executable were written in C/Rust/… . I was wondering if the expectation for things like this is to declare them as ‘extension modules’, but I couldn’t see an obvious way to do that in this case. If anyone could provide any pointers for how to go about this it would be much appreciated 🙂 ---
**Author:** ![FFY00](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ffy00/32/4007_2.png) [@FFY00](https://discuss.python.org/u/FFY00)\ **Post date:** [August 2, 2021, 12:10am UTC](https://discuss.python.org/t/packaging-a-compiled-executable/9954/2 "2021-08-02T00:10:26Z")
> [@LewisGaul](#): > > I believe this is effectively what’s being asked in [Compiling & installing C executable using python’s setuptools/setup.py? - Stack Overflow](https://stackoverflow.com/questions/33168482/compiling-installing-c-executable-using-pythons-setuptools-setup-py), but I’m not convinced by the solution there (and installing the executable to a venv’s bin directory isn’t particularly the goal for me - I’m expecting it to be internal to the project’s package). Yeah, you don’t want to customize the `install` command, that generates eggs, which are deprecated, and is not compatible with PEP 517. I would customize the `build` command instead. You can generate your executable and place it inside your package (in the build directory) so that it is shipped as a package resource. Eg. ```auto my_package/ __init__.py my_executable ``` You can then access it via the `importlib.resources` API: ```python import importlib.resources my_executable = importlib.resources.files('my_package') / 'my_executable' ``` See [importlib — The implementation of import — Python 3.12.1 documentation](https://docs.python.org/3/library/importlib.html#importlib.resources.files) For Python versions older than 3.9, there is also a backport: [GitHub - python/importlib\_resources: Backport of the importlib.resources module](https://github.com/python/importlib_resources) ---
**Author:** ![LewisGaul](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@LewisGaul](https://discuss.python.org/u/LewisGaul)\ **Post date:** [August 2, 2021, 12:31am UTC](https://discuss.python.org/t/packaging-a-compiled-executable/9954/3 "2021-08-02T00:31:21Z")
Thanks, that makes sense. > I would customize the `build` command instead. You can generate your executable and place it inside your package (in the build directory) so that it is shipped as a package resource. Are there any official docs on customising the `build` command? I couldn’t find any on a quick look - the closest I found is [python - Hook to add commands to distutils build? - Stack Overflow](https://stackoverflow.com/questions/11331175/hook-to-add-commands-to-distutils-build/11332078), although this doesn’t make it clear how to control where the executable is placed. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [August 2, 2021, 7:06am UTC](https://discuss.python.org/t/packaging-a-compiled-executable/9954/4 "2021-08-02T07:06:47Z")
> [@LewisGaul](#): > > Are there any official docs on customising the `build` command? Basically no, customising distutils has always been a matter of reading the sources and trying to work out what to do from that - and I assume the same remains true of setuptools. (Offtopic, but this is one of the reasons distutils was so hard to maintain - without a documented API, changing anything could be a compatibility problem…) ---
**Author:** ![FFY00](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ffy00/32/4007_2.png) [@FFY00](https://discuss.python.org/u/FFY00)\ **Post date:** [August 2, 2021, 1:40pm UTC](https://discuss.python.org/t/packaging-a-compiled-executable/9954/5 "2021-08-02T13:40:15Z")
It should be the same as the `install` command – subclass the default `build` command and override `run`. ---
**Author:** ![LewisGaul](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@LewisGaul](https://discuss.python.org/u/LewisGaul)\ **Post date:** [August 2, 2021, 5:00pm UTC](https://discuss.python.org/t/packaging-a-compiled-executable/9954/6 "2021-08-02T17:00:22Z")
I think I’ve got this working by subclassing `distutils.command.build.build`, but I thought `distutils` was [in the process of being] deprecated? I couldn’t see an equivalent in `setuptools`. This also doesn’t result in a custom platform tag being applied to wheels - I find myself manually changing e.g. from `zig_minesolver-0.1.0-py3-none-any.whl` to `zig_minesolver-0.1.0-py3-none-win_amd64.whl` before uploading. I’ve decided I’m probably better off just making the project installable only with wheels, partly because the `ziglang` build dependency seems to be kind of slow to install (no fault of Python), but also this dependency is only available in wheels itself. I can then just write a script to iterate over target platforms performing the zig build (using cross-compilation) independently from `setup.py`, package up the compiled file as ‘package data’, and then rename the wheels. A couple of related links I came across: > > > > I've seen cases where users wish to add custom steps to the build process (\[exam…ple\](https://stackoverflow.com/questions/14441955/how-to-perform-custom-build-steps-in-setup-py), \[example\](https://stackoverflow.com/questions/52994857/how-do-i-generate-python-grpc-code-from-within-a-setuptools-installer-setup-py/54236739#54236739)). Historically, users have managed to work around the issue by overriding the \`build\` or \`install\` commands (using \`cmdclass\`), but this approach requires replacing the default command just to inject some behavior. Preferable would be to avoid having to override the default behavior and for Setuptools to present a protocol for extending build steps. > > And indeed, distutils \_already\_ provides a mechanism to extend build steps through \[sub\_commands\](https://github.com/pypa/distutils/blob/35e1c9d4d788094c254485dfdefa0c24cd5820ee/distutils/command/build.py#L153-L157). > > My proposal is that the \`build\` command present an interface to append or prepend commands to the build subcommands such that a third-party provider could provide an independent command, expose it through \`distutils.commands\`, and either implicitly as part of the import or explicitly by the end user, append or prepend that command to the subcommands. > > I think it should be possible to demonstrate this behavior today just by mutating the sub\_commands list directly, though I'd probably recommend a proper API for users to use. ---
**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\ **Post date:** [August 2, 2021, 6:06pm UTC](https://discuss.python.org/t/packaging-a-compiled-executable/9954/7 "2021-08-02T18:06:47Z")
> [@LewisGaul](#): > > I think I’ve got this working by subclassing `distutils.command.build.build` , but I thought `distutils` was [in the process of being] deprecated? I couldn’t see an equivalent in `setuptools` . The latest setuptools versions have their own copy of distutils. It’s not fully integrated yet, because they want to be able to support code that keeps importing from distutils, but eventually their copy will be The copy. The distutils in CPython (as of 3.10) will report a warning whenever it is imported. The fix for that warning is to make sure you have an up to date setuptools so that you get theirs instead.