Add functionality to automatically install packages

Packaging
1

To make the execution of python code easier I suggest a way how the necessary packages could be installed automatically.

Instead of the current way

import numpy as np
from pandas import DataFrame

*realize you haven't installed the packages*
python -m pip install pandas numpy

I suggest the enhancement

import numpy package numpy==1.25.1 as np
from pandas package pandas==2.0.3 import DataFrame

This mixes import statements with the syntax of the requirements file. It should still be possible to use the current syntax. In that case no automatic installation should be performed.

What do you think of that idea? Do you think it would pass as as PEP?

2

What happens when two packages declare two different versions of numpy?

3

Read the ongoing discussions in the Packaging category for PEP 722 and PEP 723.

Overall, installing requirements dynamically doesn’t work well for a couple reasons, most importantly that it doesn’t make a lot of sense to install the packages separately. Consider

import numpy package numpy as np # running this installs the latest version of numpy and imports it


def foo():
    # running this will try to install somelib==1.2
    import somelib package somelib==1.2
    # oops, the resolver notices that somelib==1.2 requires an older version of numpy
    # which has already been imported in a newer version

If you were thinking of parsing all the requirements upfront and installing them all at once, then please read this section of PEP 722: PEP 722 – Dependency specification for single-file scripts | peps.python.org

2 Likes
4

Another issue is that package names don’t have to be unique, and even if they were, the mapping from package name to project name (the thing you need to pip install) isn’t easily available.

(see this discussion for more on that, and potential enhancements)

edit: eh this is wrong, you’re proposing to name the project directly, so that part would be okay at least

5

@effigies and @jeanas thank you for your suggestions. These are valid points.

As mentioned in the PEPs you provided this is targeted more towards little scripts or small scale projects. I would advise against using this in bigger projects. There a pyproject.toml is way cleaner.

What happens when two packages declare two different versions of numpy?
In that case I would perform the same behavious as npm does with its javascript packages. Show which package requires which version and throw an exception. I would also add the same options as there such as --force or --legacy-peer-deps that just overrides the old packages with the new ones. Additoinally printing a warning that this can cause unstable behaviour and that you should tidy up your dependencies.

Dynamic imports
To limit the errors you mentioned to compile time i would suggest to allow import … package statements only on the top of the file. It would be the intended use. So whenever you have written the first normal import statement or any other construct no import … package statements are allowed anymore.

6

In that case you should be aware that that is already the situation PEP-722 and PEP-723 aim to address. If you truly wanted to pursue this though, you would need to understand those PEPs very deeply, and explain why those approaches should be rejected in favor of this one. But be aware that it took weeks of conversation with hundreds of comments to bridge those two similar but competing PEP approaches together. I can’t imagine this very different approach could get traction without at least as much discussion, and just speaking frankly I doubt there is a collective appetite for that.

Besides the issues alluded to already, some other problematic areas come to mind:

  • what about custom or private pip package repositories?
  • what about packages that are only available via conda?
  • are you proposing making package a keyword? That would certainly break lots of existing code
7

Then how do you express, say, a Windows-only dependency? (See also the linked section of PEP 722.)

8

I am thinking, maybe fades is the closest thing that already exists. It supports something like this:

import bs4   # fades beautifulsoup4 == 4.2

– What is fades? — fades 4 documentation

But yes, I guess you should keep an eye on PEP 722 and PEP 723.

2 Likes
9

@sinoroc fades is pretty much exactly what I need! Also as pointed out by others PEP 722 and 723 already take care of it. Until they get implemented I’ll use fades. Thank you for your comments and improvement suggestions. Consider this thread closed.

1 Like