Wheels for musl (Alpine)

Yes, definitely. I just want to make the issue with Rust + musl 1.2 on 32-bit systems very clear, if anyone is planning on using Rust there. Some of the comments seemed dismissive of the issue.

Hmm OK, so I completely mischaracterised the issue. So, if I understand correctly, the real issue is not really about interoperability of libraries using musl in general, but specifically if the software (for example, Rust) is implemented incorrectly (in some sense)? Maybe I should remove that part from the PEP draft then.

I don’t think people are generally dismissive of the issue, but are trying to focus on the most immediate topic by pointing out the issue does not really need to affect the wheel tag. Many comments here seem to be aimed more specifically to solve the “cryptography on Alpine containers” situation, which needs a lot of things to happen. A wheel tag is merely the first step to the solution, and has nothing to do with Rust. The time64 issue is still not insignificant, but it’s actually counter-productive to include it in the wheel tag discussion, because it raises unnecessary questions we don’t need to get into (why do we choose to only target a very new musl version? why do we exclude 32-bit?) It is actually easier to specify a generic tag that contains unworkable combinations, because, well, nobody is going to use it anyway.

3 Likes

Exactly :slight_smile:

Specifically, the whole thing works fine for languages that use C headers, since the situation is completely solved there. The ugliness only appears when you combine things using C headers and things that hardcoded their own understanding of the ABI (which was correct in isolation).

That’s fair. I’m very much a new comer to this forum, so thank you for pointing this out plainly. In that case, I will try to document these concerns when we have a place to put documentation for the musllinux tag.

No problem at all! Your input is valuable to the conversation and definite helped me coming up with the PEP draft. I’m looking forward to hearing about your opinions more in the steps after the musllinux tag, when we actually try to make the build toolchain available for projects :slightly_smiling_face:

2 Likes

Is anybody working on a PEP and implementation? It’s been over a month since last reply.

I’m still looking for a sponsor for my draft

I missed the fact that you already wrote a PEP. Excellent!

The PEP looks good to me. I just miss two points

Nathaniel had a pretty extensive response on that topic to a comment I made.

TL;DR: it’s not exactly like glibc, but yes, for all (non-buggy) intents and purposes. that promise exists.

Nathaniel’s comment is great (as usual). It clearly shows that he works in academics. :slight_smile:

I like to see this information included in the document. A PEP should mention relevant and useful pieces of information. ABI stability is an important assumption and relevant for implementers of pip and warehouse (PyPI).

I’ll try to find some time and write these into the draft. Although I’d really like a better reference than a short email on the backwards compatibility issue.

1 Like

I can sponsor.

2 Likes

musl libc - Compatibility doesn’t mention a musl policy directly (and I didn’t find much searching the docs/wiki), but it does link to this comparison, which merely says “Backwards Compatibility: yes”.

I’ve updated the document and submitted it to New PEP: Platform Tag for Linux Distributions Using Musl by uranusjr · Pull Request #1876 · python/peps · GitHub

PEP 656 -- Platform Tag for Linux Distributions Using Musl | Python.org is live!

2 Likes

In the PEP there is some code to pull out the musl version. It doesn’t work for me. Is there a better way?

os_name, sep, arch = sysconfig.get_platform().partition("-")
assert os_name == "linux" and sep, "Not a Linux"
arch.replace("-", "_")
return f"musllinux_{musl_version[0]}_{musl_version[1]}_{arch}"

When I run that code in an Alpine container, I do not get a valid arch value:

docker run -it --rm python:3.7.2-alpine3.9 /bin/sh
...
/ # python
Python 3.7.2 (default, Mar  7 2019, 22:40:59) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_platform()
'linux-x86_64'

Not that glibc is any better, here is the dance needed to find it (either via os.confstr('"CS_GNU_LIBC_VERSION") or using ctypes to call gnu_get_libc_version())

Matti

That code’s actually expected to be supplied with the MUSL version. I don’t see any code in the PEP that explains how, for a given installation, you determine what the MUSL version that you’re running on is. I think the PEP needs to include that.

It looks like PEP 600 (manylinux) doesn’t cover the equivalent question for glibc directly - it just refers to an undefined get_system_glibc_version function, but the explanation of how to write that is in PEP 513

As there are no previous MUSL peps, we probably need to make sure that we pick up any other dependencies like this that might have come from basing PEP 656 on PEP 600…

I just opened PEP 656 musllinux support by uranusjr · Pull Request #411 · pypa/packaging · GitHub which includes an approach to fetch musl presence and the version in use by invoking ldd. Maybe I can update the PEP to include this as a reference implementation?

I think including the text of _get_musl_version from that PR into the PEP would be acceptable. I can’t comment on whether it’s the “right” approach, but I assume the MUSL experts in this discussion can let you know about that :wink:

PS As with PEP 600, I’m OK with being PEP-Delegate by default for this PEP on the basis that I’ll be looking at whether the discussion here looked sufficiently complete, and reached an acceptable compromise (I have no specialist MUSL knowledge). But if anyone wants to suggest someone who does know MUSL who would be better as PEP-delegate, feel free to speak up!

I added a code block to show how to retrieve the musl version. I chose to only briefly describe how to get the musl libc’s path from a Python executable, since the implementation is a bit too long to include in a document, but shouldn’t be too difficult to devise from reading the ELF spec (ref provided in the PEP).

Also note that there’s now a thread for the PEP, so it might be a good idea to move further discussions there so we can keep things in one place :slightly_smiling_face:

Please post further comments at Pre-PEP: Platform Tag for Linux Distributions Using Musl.