Hello, I was thinking of developing a new package manager for python inspired by Java Maven. The idea would be to have a central local repository for all the dependencies (divided by version, platform, etc) and somehow update the PYTHONPATH accordingly instead of having completely isolated virtual environments. My motivation is that I create tons of new python projects that usually have very heavy dependencies (e.g. torch, tensorflow etc.) and each venv ends up being some gb in size. The new tool would instead download the same package only once and share it with all the projects. To guarantee interoperability it would rely on pip to download the packages and the resolution of dependencies but then it will restructure the destination folder in some way (I still need to figure it out). Do you guys see any issues with this idea that would make it unfeasible? Thank you in advance for your feedback!
Note that pip already downloads packages only once, unless you
explicitly disable its file caching feature or clear its cache.
I know at least PDM supports linking dependencies instead of copying them to each env https://pdm.fming.dev/latest/usage/config/#central-installation-caches
Yes, it is cached and not downloaded again but every time you create a new venv the packages are copied there.
Thanks, I didn’t know it. I’ll give it a look!