kknechtel
(Karl Knechtel)
April 16, 2023, 8:12am
1
This thread is motivated by a new Stack Overflow question:
Putting aside the usefulness of having both of these packages in the same project, there seems to be a problem here: two different distribution packages want to use the same import package name. In a test venv I was able to confirm that Pip happily installs both packages… with the contents for both going to the same lib/python3.x/site-packages/magic
folder inside the venv folder.
This doesn’t seem ideal. I assume it’s by design, and required by how the plugin/optional component, namespace package etc. systems work. But in this case, the two packages are not meant to combine, and in particular the first one’s __init__.py
will get overwritten with the other. It’s not hard to imagine worse conflicts. I don’t know what exactly, but I feel like there should be some form of mitigation for this.
uranusjr
(Tzu-ping Chung)
April 16, 2023, 8:26am
2
See thread on pip’s tracker for previous requests, discussions, and some of the difficulties changing this behaviour.
opened 03:18PM - 28 Jun 20 UTC
S: needs triage
(Forgive me if this is a duplicate... It seems like a problem that should have b… een reported in the past [I've heard that the transition from docker-py to docker resulted in similar problems]. I couldn't find any search terms that found an existing bug report so I'm opening a new one)
**Environment**
* pip version: 20.1.1
* Python version: 3.7
* OS: Fedora Linux
**Description**
The ansible-2.9.x package is one large, monolithic package. ansible-2.10.x splits this package into two:
* an ansible-base package which contains the executables (not entrypoints, if that matters) and the site-packages/ansible library with most of the files which were in ansible-2.9.x.
* an ansible package which contains all of the addons to the ansible-base package. These are installed into site-packages/ansible_collections.
The ansible-2.10.x package has a dependency on the ansible-base packages. This way people who only need the minimal functionality can `pip install ansible-base` (>=2.10.0b1). People who just want to get the same experience as ansible-2.9.x can `pip install ansible` (>=2.10.0a1) .
A clean install works fine:
```
pip uninstall ansible
pip install ansible==2.10.0a1
# Results in ansible-base-2.10.0b1 and ansible-2.10.0a1 installing correctly.
# Looking at the filesystem will find that all of the correct files are present.
# ansible --version will work
```
The problem is upgrades:
```
pip install ansible==2.9.10
pip install ansible==2.10.0a1
# Results in ansible-base-2.10.0b1 and ansible-2.10.0a1 being installed but with missing files
# The files which were in ansible-2.9.10 are erased from the filesystem,
# leaving ansible-base-2.10.0b1 in a broken state
# The reason is that pip first installs ansible-base-2.10.0a1 and then, when it uninstalls ansible-2.9.10,
# it doesn't realize that the ansible-base files have overwritten the same named files in ansible-2.9.10
# and that they should not be removed.
```
**Expected behavior**
I expect that upgrading the package will lead to a system where both new packages are installed with all of the files that the new packages include.
**How to Reproduce**
1. python3.7 -m pip install --user ansible==2.9.10
2. python3.7 -m pip install --user ansible==2.10.0a1
3. ls -al ~/.local/bin/ansible
* ls: cannot access '/home/badger/.local/bin/ansible': No such file or directory
4. ls -al ~/.local/lib/python3.7/site-packages/ansible/__init__.py
* ls: cannot access '/home/badger/.local/lib/python3.7/site-packages/__init__.py': No such file or directory
**Output**
``` bash
[pts/145@peru /var/tmp/testing]$ python3.7 -m pip install --user ansible==2.9.10 (08:10:37)
Processing /home/badger/.cache/pip/wheels/bb/e6/9a/05f0b546d96bc1da05865504c3481a4fd3a1b3fd48a38d53a1/ansible-2.9.10-py3-none-any.whl
Requirement already satisfied: PyYAML in /home/badger/.local/lib/python3.7/site-packages (from ansible==2.9.10) (5.3.1)
Requirement already satisfied: jinja2 in /home/badger/.local/lib/python3.7/site-packages (from ansible==2.9.10) (2.11.2)
Requirement already satisfied: cryptography in /home/badger/.local/lib/python3.7/site-packages (from ansible==2.9.10) (2.3.1)
Requirement already satisfied: MarkupSafe>=0.23 in /home/badger/.local/lib/python3.7/site-packages (from jinja2->ansible==2.9.10) (1.1.1)
Requirement already satisfied: cffi!=1.11.3,>=1.7 in /home/badger/.local/lib/python3.7/site-packages (from cryptography->ansible==2.9.10) (1.11.5)
Requirement already satisfied: idna>=2.1 in /home/badger/.local/lib/python3.7/site-packages (from cryptography->ansible==2.9.10) (2.8)
Requirement already satisfied: asn1crypto>=0.21.0 in /home/badger/.local/lib/python3.7/site-packages (from cryptography->ansible==2.9.10) (0.24.0)
Requirement already satisfied: six>=1.4.1 in /home/badger/.local/lib/python3.7/site-packages (from cryptography->ansible==2.9.10) (1.15.0)
Requirement already satisfied: pycparser in /home/badger/.local/lib/python3.7/site-packages (from cffi!=1.11.3,>=1.7->cryptography->ansible==2.9.10) (2.19)
Installing collected packages: ansible
Successfully installed ansible-2.9.10
[pts/145@peru /var/tmp/testing]$ python3.7 -m pip install --user ansible==2.10.0a1 (08:11:10)
Processing /home/badger/.cache/pip/wheels/1a/52/25/c9e776b2df588061cd9dc831740416edb7e276f5ffe7a8d8d3/ansible-2.10.0a1-py3-none-any.whl
Processing /home/badger/.cache/pip/wheels/dd/db/f0/0890f4f13dd6446092ba5d76f55d62528b52ca6ab74ea871d4/ansible_base-2.10.0b1-py3-none-any.whl
Requirement already satisfied: jinja2 in /home/badger/.local/lib/python3.7/site-packages (from ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (2.11.2)
Requirement already satisfied: cryptography in /home/badger/.local/lib/python3.7/site-packages (from ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (2.3.1)
Requirement already satisfied: packaging in /home/badger/.local/lib/python3.7/site-packages (from ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (20.4)
Requirement already satisfied: PyYAML in /home/badger/.local/lib/python3.7/site-packages (from ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (5.3.1)
Requirement already satisfied: MarkupSafe>=0.23 in /home/badger/.local/lib/python3.7/site-packages (from jinja2->ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (1.1.1)
Requirement already satisfied: asn1crypto>=0.21.0 in /home/badger/.local/lib/python3.7/site-packages (from cryptography->ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (0.24.0)
Requirement already satisfied: idna>=2.1 in /home/badger/.local/lib/python3.7/site-packages (from cryptography->ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (2.8)
Requirement already satisfied: six>=1.4.1 in /home/badger/.local/lib/python3.7/site-packages (from cryptography->ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (1.15.0)
Requirement already satisfied: cffi!=1.11.3,>=1.7 in /home/badger/.local/lib/python3.7/site-packages (from cryptography->ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (1.11.5)
Requirement already satisfied: pyparsing>=2.0.2 in /home/badger/.local/lib/python3.7/site-packages (from packaging->ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (2.4.7)
Requirement already satisfied: pycparser in /home/badger/.local/lib/python3.7/site-packages (from cffi!=1.11.3,>=1.7->cryptography->ansible-base<2.11,>=2.10.0.dev1->ansible==2.10.0a1) (2.19)
Installing collected packages: ansible-base, ansible
Attempting uninstall: ansible
Found existing installation: ansible 2.9.10
Uninstalling ansible-2.9.10:
Successfully uninstalled ansible-2.9.10
Successfully installed ansible-2.10.0a1 ansible-base-2.10.0b1
[pts/145@peru /var/tmp/testing]$ ls -al ~/.local/bin/ansible (08:12:13)
ls: cannot access '/home/badger/.local/bin/ansible': No such file or directory
[pts/145@peru /var/tmp/testing]$ ls -al ~/.local/lib/python3.7/site-packages/ansible/__init__.py (08:14:19)
ls: cannot access '/home/badger/.local/lib/python3.7/site-packages/ansible/__init__.py': No such file or directory
[pts/145@peru /var/tmp/testing]$ ls -al ~/.local/lib/python3.7/site-packages/ansible/modules/ping.py (08:14:29)
-rw-r--r--. 1 badger badger 2090 Jun 28 08:11 /home/badger/.local/lib/python3.7/site-packages/ansible/modules/ping.py
```
1 Like