Installing headers from wheel: should the directory be normalized or not?

I’d like to discuss the correct behavior for installing headers (as in .h files) from wheels. What prompted me to do this is that the recent change in setuptools where wheel filenames started being normalized, domino-style broke installing PyCXX package, as headers started being installed into lowercase cxx directory rather than uppercase CXX as the reverse dependencies expect.


Currently, the headers included in wheels are installed by tools to a directory roughly resembling:

{ sysconfig.get_path("include") }/{ distribution_name }

The way distribution_name is defined depends on the tool. From a quick testing:

  1. pip seem to take the name directly from wheel filename.
  2. installer does the same. However, it has a TODO to “figure out original vs normalised distribution names”.
  3. uv seems to use the normalized distribution name.

From a quick reading, the binary distribution format specification doesn’t specify anything about the subdirectory at all. It says:

Each subdirectory of distribution-1.0.data/ is a key into a dict of destination directories, such as distribution-1.0.data/(purelib|platlib|headers|scripts|data). These subdirectories are installation paths defined by sysconfig.

This seems to indicate that the headers would be installed top-level to include directory, which is not what packages expect and not what tools do.


Now, my question is: is this a bug or a feature? The breakage is really a corner cases — I don’t think I’ve seen any other package relying on uppercase header directory. But I think it would make sense to normalize on a single, and preferably predictable, behavior here.

I’m opening a discussion here rather than filing individual bugs to avoid spreading it across three bug trackers.

CC @pradyunsg @charliermarsh @zanie @konstin

4 Likes

Imho it makes sense to use the normalized distribution name consistently for the header directory, too, given that other locations are also slowly converging towards using the normalized distribution name.

3 Likes

FYI, here’s some relevant discussion about the distutils migration aspect of it: Allow site packages to install header files · Issue #88611 · python/cpython · GitHub

Essentially, this is a edge case not covered by the current standards. My recommendation on the subdirectory name formatting is to replicate exactly what setuptools did.

As in what setup.py install used to do?

I disagree with this recommendation. I think the proper way forward is to have setuptools normalize, or in the worst case write a very small PEP and then have it follow the PEP. There is precedent for introducing technically breaking changes for correctness purposes such as PEP 685 which enforced normalization of extras.

Yes, which I am pretty sure it’s using the non-normalized version of the name.

Yeah, of course. Standardizing it is naturally the best option. My recommendation is just for how to proceed while that doesn’t happen.

1 Like