The difference between ``python setup.py build_ext -i'' and ``pip install -e ''

Packaging
1

Hi,

For setting up a development environment with specific package, specifically the ones cloned from git repo. the suggested method of installation maybe like the following:

$ python setup.py build_ext -i
or
$ pip install -e <path/to/local/repo>

What’s the differences between these two install methods?

Regards

1 Like
2

Is the first command meant to be python setup.py develop instead? I have never seen anyone recommend installing with build_ext, and I don’t believe it does install.

3

python setup.py build_ext -i will build extension modules (ie modules written in C or C++, and Cython) in-place. This allows Python to import the package with those modules, as long as the package is already installed (via pip install -e DIR or python setup.py develop) or is available in Python’s path (sys.path or PYTHONPATH). This technically means that build_ext doesn’t actually install the package, but is one part of the build process

pip install -e DIR will use PIP to install the package as editable, which will build any extension modules (setup.py build_ext), perform any other build requirements, then install the package symbolically (setup.py develop) including its dependencies

1 Like
4

Thanks @EpicWink

But I’ve also noted that the python setup.py develop also will invoke the python setup.py build_ext instead of python setup.py build_ext -i. So, it seems that pip install -e DIR and python setup.py develop are equivalent in the final results for setting up the develop environment workflow – they both will create easy-install.pth files in the current python interpreter’s site-package directory which pointing to the DIR of the python package we want to installed here.

Regards

5

I will note that while they are basically equivalent, all direct invocations of python setup.py will be deprecated and/or removed as we add replacements for them, so you should definitely move to using pip install -e . and in general find replacements for any python setup.py invocations you find yourself making.

3 Likes
6

As someone who currently has a development cycle of:

VAR=1 python3 setup.py build
VAR=1 python3 setup.py install

in a venv, what should I currently be doing?

7

pip install . most likely. Details might depend, but basically that.

8

I think that is not the same as setup.py build for c-extension modules since the build directory is then part of the, by default, isolated build. Since the isolated build is done in a temporary directory it makes debugging more difficult than setup.py build install. Is there a way to use pyproject.yaml to avoid the default behaviour?

9

I tried with the following command to inspect the internal workflow of pip:

$ pip install -e . -vvv --isolated

And find that the pip install -e will invoke many steps of the setup.py, say the following:

running egg_info
running develop
running egg_info
running build_ext

But it seems that the pip command cannot set the -j option for using the
python parallel build capability, which can be done with the python setup.py as following:

$ python setup.py build -i -j24
$ python setup.py build_ext -i -j24

Regards

10

It is possible to pass command options to the setup function call (e.g. setup(name="...", options={"build_ext": {"inplace": True, "j": 24}})), or have the options in a setup.cfg file (section [build_ext] with lines inplace = true and j = 24). The problem is that these options are desirable for local dev, but they would also apply for anyone building from the sdist, which could be unwanted depending on the case.

11

What is the replacement for the command in the OP (build_ext -i)?

12

Depends on your backend. Right now, there isn’t a frontend tool that can do it, so you’ll only get it by invoking the backend directly.