Tool to handle virtual environments

Python Help
1

I want to activate and deactivate a virtual environment. I want to have this environment in a separate shell and deactivate it simply via typing “exit”.

I did some research on existing tools. All of them have their shortcomings. Maybe I use them the wrong way or you may have another tool to suggest?

1. Vanilla venv

You know it. There are to many commands and explicit steps.

$ python3 -m venv venv --prompt="env-name"
$ source venv/bin/activate
$ doyourstuff
$ deactivate

2. pipenv

This is very near to what I am looking for.

$ pipenv shell
$ doyourstuff
$ exit

But the problem is that it always creates a Pipfile file in the project root folder. I don’t need and and don’t want it. It seems that pipenv is not invented for my purpose.

So pipenv would be my favorite if it would not create this trash file.

3. virtualenv

Faster than venv but also has this extra source step.

$ virtuelenv venv
$ source venv/bin/activate
$ doyourstuff
$ deactivate

4. virtualwrapper

Uncomfortable in its usage. I wasn’t able to use it.

5. poetry, conda & Co

Far to much, heavy and powerful for what I try to achieve.

2

Could you describe what your preferred workflow would look like?

1 Like
3

While I don’t have a convenient answer to your question (there are
certainly solutions I’m aware of, for example creating separate user
accounts and sourcing the venv in their ~/.profile or ~/.bashrc),
but what a lot of folks often fail to realize is that it’s almost
never necessary to “activate” a venv. I have dozens of tools
installed in separate venvs in my homedir and just run executable
entrypoints and sometimes python3 itself directly from them or via a
symlink to somewhere in my default $PATH.

# make a lib dir in case it doesn't exist yet
mkdir -p ~/lib

# create a venv for the tool you want to install, e.g. flamel
python3 -m venv ~/lib/flamel

# use your system pip in case there isn't one in your venv
python3 -m pip --python ~/lib/flamel/bin/python3 install flamel

# run the utility directly, no need to "activate" it
echo '{foo: bar, xyzzy: plugh;}'|~/lib/flamel/bin/flamel -

# you can also symlink it to somewhere in your path...
mkdir -p ~/bin
ln -sf ../lib/flamel/bin/flamel ~/bin/

# for demonstration, but would usually be handled in ~/.profile
export PATH="$HOME/bin:$PATH"

# run the utility from your executable search path
echo '{"foo": "bar", "xyzzy": "plugh"}'|flamel -

# you can also use the interpreter in the venv without activating
~/lib/flamel/bin/python3 -c 'import sys;print(sys.path)'

# even interactively use its REPL
~/lib/flamel/bin/python3
import flamel
print(flamel.yaml_dump(dict(foo="bar", xyzzy="plugh")))

I use venvs pretty much everywhere I need to install something from
PyPI or local source, and basically never “activate” them.

3 Likes
4

If you’re on Posix, check out:

1 Like
5 
$ create_and_or_activate_the_environment
$ (env_name) do_my_stuff
$ (env_name) exit
$

It is the same with pipenv.

6

You should be able to do that with a short shell script or alias.
For example on my mac this zsh -c ". zzz/bin/activate; exec zsh" does the hard work.
I would use bash on a linux system.
But as I force the PS1 the (zzz) is missing - should be simple to fix.

% which python3
/usr/local/bin/python3
% python3 -m venv zzz
% zsh -c ". zzz/bin/activate; exec zsh"
% which python3
/Users/barry/tmpdir/zzz/bin/python3
% exit
% which python3
/usr/local/bin/python3
1 Like
7

As an alternative, I always directly use the path to the Python executable in my virtual environments and never activate the environments.
For example I wouldn’t call source venv/bin/activate; python -c "...", but rather venv/bin/python -c "..."

I find several advantages to not activating environments:

  • Much easier to switch between several environments (e.g. ones using different Python versions)
  • It’s always clear which environment is being used, even if several environments have the same name
  • Scripts can be run as root very simply (inside an activated environment, you would need to use a workaround like sudo -E PATH=$PATH python)
  • You can simply define aliases in ~/.bashrc (or equivalent) for the environments that you use much, so that you can use project_python as an alias to project/venv/bin/python for example
2 Likes
8

You can skip the activation and deactivation steps if you use the binaries from the bin directory directly:

$ python3 -m venv venv --prompt="env-name"
$ venv/bin/pip install some-library
$ venv/bin/python some_script.py
9 
$ hatch shell

Works in any directory (no need for a pyproject.toml file like poetry shell), does not write out any extra files (unlike pipenv shell).

1 Like
10

All you really need is a simple script that sources the file, then starts a new interactive shell.

#!/bin/bash -i

. "$1"
bash

Call it pyshell, make it executable (chmod +x pyshell), then store it somewhere in your path.

$ pyshell foo/bin/activate
(foo) $ which python
[...]/foo/bin/python
(foo) $ exit
$

(Oops, basically Barry’s answer from a few hours ago; I slid right over that while skimming the thread.)