Trying to learn how to manage dependencies, stuck at conda vs pip

[CAM-Gerlach]

@Benji

But before I dive into the pip/venv/virtualenv/pipenv rabbit hole, does anyone have more comments on how conda users can use packages not in the defaults or conda-forge repositories, such as PyPI packages? If it’s not actually a limitation in practice I might stick with conda instead.

There’s no need to use venv or virtualenv (and pipenv is a whole different ball game, as its more akin to a tool like Poetry); you can still use pip just fine with conda environments. For that, just install Python itself with conda in a fresh environment, e.g. conda create -n pip-env python=3.9, then install everything else with pip, which will work as normal.

In fact, if you know what you’re doing, you can actually get away with mixing pip and conda packages just fine, particularly if they are pure-Python and near the top of the dependency chain, so long as you’re careful and follow a few rules:

• Never install anything into your base environment; just use it for conda itself. I suggest using Miniconda (or better yet, Miniforge) instead of full-fat conda.
• Either use Miniforge, or run conda config --add channels conda-forge to use the conda-forge channel, which has a much wider selection of packages than the default AD channels.
• Use pip installs only for packages that need it; even if you have a package that can only be installed by pip, typically most or all of its dependencies will be available through conda-forge, so you can manually install those first with conda (you might need to look at the package’s pyproject.toml, setup.cfg or setup.py to find them), then install the package itself with pip.