Packaging DLLs on Windows

Packaging
2

The thing Steve is talking about isn’t really helpful for shipping DLLs inside wheels. The challenge is that you want to isolate each wheel, so that if two wheels happen to ship different versions of (say) openssl, they don’t accidentally interfere with each other. To do this properly, you can’t put your openssl.dll on the search path, because that’s shared by all packages, and you can’t even call it openssl.dll, because the Windows DLL loader assumes that if it’s ever seen a file called openssl.dll, then that’s what it should use for all future files called openssl.dll, even if they’re being loaded from a different path.

AFAIK right now the only reliable ways to ship DLLs in Windows wheels are:

  • Manually give all your DLLs unique names. This probably requires manually hacking your build system to use the new name, maybe using black magic to generate some new .lib files, or else using a hacky tool built by some random person on github to patch your built binaries in-place. Then, hack your package’s __init__.py to either manually pre-load all these DLLs by absolute path, or else mutate the process’s PATH envvar to add a new directory you control, where you’ve placed all your DLLs. Or something involving AddDllDirectory (but that’s Win8+ only, so you can’t rely on it). The details are extremely complicated.

  • Don’t ship DLLs; use static linking instead.

NumPy actually does the first option – if you look at numpy-1.16.2-cp27-cp27m-win32.whl on pypi, you’ll see it contains a file called numpy/.libs/libopenblas.JKAMQ5EVHIVCPXP2XZJB2RQPIN47S32M.gfortran-win32.dll. I honestly have no idea how this is accomplished – I just spent 20 minutes searching through our wheel build infrastructure and can’t find it. Possibly Matthew Brett is the only person who knows :slight_smile:

I think sensible people generally use static linking.

IMO, comparing the three major platforms, they each have their own weird quirks, but in the end they’re all pretty similar in terms of how well they can support portable wheels. But currently our tooling for macOS and Linux is better than our tooling for Windows, so supporting Windows is probably harder in practice. Windows really needs an auditwheel equivalent to catch up. (And it would be technically straightforward to do this, just no-one has done it.)