PEP 582 - Python local packages directory

[FFY00 (Filipe Laíns)]

@pf_moore

I wouldn’t want to use it in pip, especially if it’s not considered “not common practice”.

I mean, we shouldn’t make adding new API a common practice. We should try to fit new use-cases into existing API and very carefully consider when and when not it makes sense.

@pf_moore

As I’ve said a few times, we’re trying to move pip to work consistently with (normal) sysconfig schemes. I don’t see a problem for pip if normal sysconfig schemes omit a path (I’d just say that if the user tries to install to such a scheme, any files in the wheel that would go in that path would be ignored with a warning). But if you feel that sysconfig schemes must have all of the listed paths, I’m fine with that.

This is essentially a normal config scheme for pip, and you will need special handling for the local packages scheme even if you use the existing API anyway.

@pf_moore

I honestly don’t see why __pypackages__ can’t just have a normal scheme. There’s no reason I can understand why it has to be partial in any case. I guess that’s something @kushaldas needs to answer.

There are two things to consider here.

1. It will leak

   This is something I’d really like to avoid, but I suppose that is already the case with virtual environments, so it wouldn’t really be a blocker.

2. The current API isn’t ergonomic for this use-case

   The API isn’t designed to require adding variables, you need to copy the variables’ dictionary, update it, and pass it to sysconfig.get_path/sysconfig.get_paths.

   sysconfig.get_paths('local_packages', vars=sysconfig.get_config_vars() | {'local_packages_base': ...})
   

   Similarly to the missing scheme path issue, this will result in some API breakage because we do not document some schemes might require extra keys and raise an error if they are missing. I don’t think it’s as bad as the missing scheme paths, but I do think it will cause some breakage.

I guess this mostly depends on how much weight you give to 1). Neither 1) or 2) are blockers, and I think 2)'s weight will be similar for most people.

Personally, I think the separate API is worth it, but it is not required. I’ve opened Deprecating the `headers` wheel data key to try to mitigate my worries with 1).

@kushaldas sorry for putting more weight on you, but I think you’ll have to make a call for what to put in the PEP. Or is it possible to give both options to the SC and have them choose?

Resuming, the two options are:

• Normal sysconfig scheme
  • Can cause some API breakage
  • It will leak, just like virtual environments (Deprecating the `headers` wheel data key mitigates it for the most, but the leakage is still inherent in the design)
• New API (sysconfig.get_local_packages_paths(base_directory))
  • The main downside is it being new API IMO
  • Easier/better to backport (no patching required)