The difference between ``python setup.py build_ext -i'' and ``pip install -e ''

Hi,

For setting up a development environment with specific package, specifically the ones cloned from git repo. the suggested method of installation maybe like the following:

$ python setup.py build_ext -i
or
$ pip install -e <path/to/local/repo>

What’s the differences between these two install methods?

Regards

Is the first command meant to be python setup.py develop instead? I have never seen anyone recommend installing with build_ext, and I don’t believe it does install.

python setup.py build_ext -i will build extension modules (ie modules written in C or C++, and Cython) in-place. This allows Python to import the package with those modules, as long as the package is already installed (via pip install -e DIR or python setup.py develop) or is available in Python’s path (sys.path or PYTHONPATH). This technically means that build_ext doesn’t actually install the package, but is one part of the build process

pip install -e DIR will use PIP to install the package as editable, which will build any extension modules (setup.py build_ext), perform any other build requirements, then install the package symbolically (setup.py develop) including its dependencies

1 Like

Thanks @EpicWink

But I’ve also noted that the python setup.py develop also will invoke the python setup.py build_ext instead of python setup.py build_ext -i. So, it seems that pip install -e DIR and python setup.py develop are equivalent in the final results for setting up the develop environment workflow – they both will create easy-install.pth files in the current python interpreter’s site-package directory which pointing to the DIR of the python package we want to installed here.

Regards

I will note that while they are basically equivalent, all direct invocations of python setup.py will be deprecated and/or removed as we add replacements for them, so you should definitely move to using pip install -e . and in general find replacements for any python setup.py invocations you find yourself making.

1 Like

As someone who currently has a development cycle of:

VAR=1 python3 setup.py build
VAR=1 python3 setup.py install

in a venv, what should I currently be doing?

pip install . most likely. Details might depend, but basically that.

I think that is not the same as setup.py build for c-extension modules since the build directory is then part of the, by default, isolated build. Since the isolated build is done in a temporary directory it makes debugging more difficult than setup.py build install. Is there a way to use pyproject.yaml to avoid the default behaviour?

I tried with the following command to inspect the internal workflow of pip:

$ pip install -e . -vvv --isolated 

And find that the pip install -e will invoke many steps of the setup.py, say the following:

running egg_info
running develop
running egg_info
running build_ext

But it seems that the pip command cannot set the -j option for using the
python parallel build capability, which can be done with the python setup.py as following:

$ python setup.py build -i -j24
$ python setup.py build_ext -i -j24

Regards

It is possible to pass command options to the setup function call (e.g. setup(name="...", options={"build_ext": {"inplace": True, "j": 24}})), or have the options in a setup.cfg file (section [build_ext] with lines inplace = true and j = 24). The problem is that these options are desirable for local dev, but they would also apply for anyone building from the sdist, which could be unwanted depending on the case.