Delocate/auditwheel, but for Windows?`

Has someone already written a tool that does the same job auditwheel does on Linux, and delocate does on MacOS, but for Windows? That is, for each compiled-code extension in a wheel, find all of the DLLs on the system that the extension needs in order to work correctly, filter out the ones that are guaranteed to be provided by Windows itself, and then copy the rest of them into the wheel? I think on Windows there isn’t any tricky object code rewriting to be done, so it shouldn’t be too complicated and basically I’m hoping someone else has already coded it up.

You do need tricky object code rewriting (the windows loader has the exact same funny glitch that the glibc loader does, that necessitates rewriting library names to avoid isolation between wheels), but that code exists. Just no one has packaged it up into a convenient tool. I think adding support to auditwheel itself would be nice, to avoid making the packaging zoo even more confusing.

See:

Is this the kind of thing the PSF and/or some external party could fund? I know debugging DLL issues in Windows wheels is kind of a chore for us, and any help through better tooling would be more than welcome :wink:

cc @brettcannon

Are you debugging using an actual debugger on Windows? Because WinDBG, Visual Studio or even Process Monitor are very good tools for this, and at a simpler level, dumpbin (to list load time dependencies) is helpful too.

By debugging I meant finding out which DLL fails loading when we get an ImportError. The only way I’ve found to do that is to use Process Monitor and search manually through trace entries to find the offending one. It is extremely tedious and, more annoying still, we’d need a CLI tool that works in CI (because situations like “the wheel fails loading on this particular AppVeyor setup but we can’t reproduce locally” have occurred several times already).

(as a last resort, we could use AppVeyor remote desktop and run Process Monitor over a high-latency connection. Let’s just say it’s not the most convenient solution)

If you’ve got a magic dumpbin incantation that finds out DLL dependencies recursively (and which ones are missing) then I’d love to know about it :slight_smile:

2 Likes

I assumed I got looped in for this sentence. For package-related funding the package WG has typically handled that and it requires a proposal to the PSF for a grant.

I’m pretty sure the CLI version would be based on xperf, but I don’t know the incantation to collect the events you want. Running under a native debugger should break at the load, but that’s not trivial to analyze either.

Dumpbin doesn’t attempt to resolve anything, so you really need to debug a running process to take search paths etc into account.

You might want to try:
https://lucasg.github.io/2018/04/29/Dependencies-command-line/

(rewrite of the invaluable Dependency Walker. I have used the Dependencies GUI version a few years ago, but have not tried the CLI)

1 Like