Environment marker for presence of a virtualenv

It seems that there is no support for using environment markers to enable installation of a package only inside a virtual environment, avoiding installing it when not using a virtualenv.

In case someone wonders why this would be needed, I will give a clear example: selinux. Major linux distributions which include python also provide the selinux-python-bindings library as a (separate/optional) package. This package is needed by big number of python libraries.

Still, the default behavior of virtualenv is to create isolated virtual enviroments, which means that the selinux library would not be available to the environment. Alternative is to either create the environments with --site-packages or to manually copy the few binary files from the system to the virtualenv (very hacky, likely to break on python or selinux upgrades,… but common approach).

As the selinux python library is quite simple, I come up with another approach: creating a shim pure-python selinux library, with the same name which loads the real library from its system location.

To my surprise it worked on most linux distros I tested with and also when installed at user or system level (outside virtualenv) – as the binaries happen not to be installed in a folder.

While I can still install the shim version it would really be nice to ablet make its installtion conditional and install it only when inside virtualenvs.

I don’t think it’s a good idea to provide a marker for virtual environments, because virtual environments are supposed to be an emulation layer for real environments. To the extent that something only works in a virtual environment or outside of one, that seems like a bug either in the project (because it is making some implementation-specific assumptions about its installation environment) or a bug in virtualenv (because it’s not perfectly emulating the system).

Virtualenvs are widely used for testing, and the idea that you may get something different when installing into your test environment from what you’d get in your target environment is very disturbing to me.

In this particular case, if I understand it correctly, I think a better solution to this problem would be giving first-class support for Provides-Dist. If both the system and shim packages declare Provides-Dist: selinux, pip or some other package manager should be able to automatically detect that selinux is present as a system package, and if not satisfy that with the shim package.

3 Likes

The usual solution to this kind of problem would be to install a copy of the real selinux bindings library inside the virtualenv. Can you say more about why that’s not possible?

Would another option be standardizing where wheels are cached so that if someone wants to install the wheel for selinux then it can pull from the system’s wheel cache?

AFAIK original selinux bindings are never builded as wheels and they require compilation, needing kernel headers and python sources too, the binaries being dependent on both. Shortly wheel cache unlikely to help. Even manylinux would not help in this case. Probably a nicer solution would be to transform the selinux shim library to make it load the selinux library in a dynamic way.

Historically I seen people copying the two binaries needed from system into virtualenv.

An environment marker would be handy and probably easy to implement. Also a feature to symlink selective system pckages to venv would be handy too but clearly more problematic to implement.